Winsock Programmer’s FAQ
Articles: The Straight Dope on Packet Sniffers
by Warren Young
Originally, Sniffer was the name of a hardware network analyzer, a device that you could plug into a LAN to capture packets, decode them, and display them. There were multiple versions of this device, some looking like a luggable PC from the late 1980s, others looking more like a high-end oscilloscope. It was basically a special-purpose computer, designed to capture, decode and display packets on the LAN you plugged it into. They cost over $10,000, and so were popular only among large network administrators. (That is to say, administrators of large networks. Some network administrators are large, but we’re not going to go into that here.)
Those of us not fortunate enough to have budgets that could absorb a 5-figure analyzer ran software on PCs that made them do much the same thing. The proper term for such software is a packet, network or protocol analyzer, but over time such programs became colloquially known as “sniffers.”
It is common for general-purpose PCs to be quite unable to cope with a full network pipe. (Ever try to get a solid gigabit per second out of a Windows 98 box?) This is not usually a problem, because most of the time a single PC has no actual need to fill a network pipe, or accept data at the full network data rate. Most of the traffic on the LAN is typically destined for other nodes, and most LANs aren’t overdesigned to the extent that they will actually allow all nodes to run at full capacity all the time. In practice, we usually don’t even want that. Small LANs, such as those run by the unfortunates mentioned above, often have lulls where there is little or no traffic at all.
This brings us to the raison d’être for hardware network analyzers: sometimes those assumptions are not valid. Sometimes you do have single computers studly enough to fill a network pipe to its full rated capacity. Sometimes the LAN is so heavily loaded that there are no lulls. Sometimes you need to sniff traffic between more than just two computers at a time. In the 1990s, if you had big data problems, you wanted a dedicated hardware packet analyzer; a PC running a freeware program downloaded from a BBS wouldn’t cut it.
There are workarounds for those with big-data problems but no money for a hardware analyzer. The better sort of software analyzers use special network stack facilities to push some filtering as far back toward the hardware level as possible. Failing that, a software sniffer can just drop packets when overloaded. A more elegant solution, often combined with kernel-level packet filtering, is to truncate captured packets, passing only as much as absolutely required down to the sniffer program in user space. If you have a computer with tcpdump on it, you’re likely to find that it was built with the default configuration, in which it defaults to capturing only the first 68 bytes of each packet. That reduces the load on the PC’s network stack and I/O subsystems, and is enough for many network analysis tasks.
Fast PCs with solid but inexpensive network hardware running Wireshark have evaporated the dedicated hardware network analyzer market. The Sniffer name has changed hands a few times in the turbulence caused by this market shift, and is now owned by NetScout as I write this. Today’s Sniffer is software for Windows PCs.
At bottom, a sniffer captures the raw data in each network packet, then parses that packet to make some sense of it. A sniffer may run packets through some selection process, write the selected packets’ raw data to disk, display some representation of each captured packet on the screen, or accumulate statistics from the captured packets for some aggregate display. Some sniffers do only one of these things, but most do some combination.
Virtually all sniffers have some sort of screen display. Such displays vary greatly in their degree of readability and usefulness for a particular purpose. Some are so primitive that they require you to have a book of network protocol specifications in one hand and a programmer’s calculator in the other if you are to make sense of them. Others layer so much GUI glitz on top that any techie with a pulse can be expected to get something useful out of the display; these necessarily hide a lot of details. Sniffers suitable for programmers fall somewhere between these extremes. Programmers usually don’t have to understand the protocols they use down to the bit level, but they frequently also need more info than you can get from cool gauges and graphs.
A sniffer may be able to see all packets on the LAN, even those not destined for the computer you are running it on. This is because most LAN technologies echo each packet from one LAN node to every other LAN node. They do this because it’s cheap.
This is true of the dumb hubs used for most 10BaseT and some 100BaseT Ethernet LANs, with the bus topology of 10Base2, and with the token passing structure of Token Ring. It is also true of all wireless LAN technologies like WiFi, due to the nature of radio communication. It is not true of all LAN technologies, a detail we’ll ignore for now.
A sniffer can take advantage of such LAN technologies to monitor network traffic between two other nodes on the LAN.
In the very earliest days of LAN technology, a LAN adapter merely transformed the electrical impulses on the LAN medium into data that could be processed in software. Virtually every network adapter produced in the past three decades performs at least one slightly more advanced function: it is able to look at the hardware address for each packet and based on that, decide whether to ignore the packet. A network adapter normally drops packets not addressed to it.
If you only wish to sniff packets to or from the computer you are running the sniffer on, this hardware address-based filtering is not a problem.
It is sometimes possible to defeat this filtering, so that your computer can eavesdrop on packets destined for another LAN node. Network adapters for LAN technologies that allow this can often be put into “promiscuous mode,” telling it that the computer is interested in all packets. The sniffer is then able to make its own decisions about which packets are interesting, then decode and display them for the benefit of the sniffer’s operator.
Not all network adapters support promiscuous mode, even when the underlying LAN technology allows it. You sometimes see this with really cheap Ethernet chips, as well as with many wireless LAN adapters.
Your PC may not be able to keep up with the increased load caused by enabling promiscuous mode. The solutions are:
Not all LAN technologies echo all packets to all nodes on the LAN.
Over the years, as networks have become faster and processing power cheaper, switched Ethernet has become more common. Packet switching makes for faster networks because it pushes many decisions about which LAN ports need to see which packets into the switch.
Consider a small switched Ethernet LAN with three nodes. If a PC plugged into port 1 on the Ethernet switch establishes a TCP connection to a PC plugged into port 2, a switch will route the packets making up this connection between just those two ports. The PC plugged into port 3 won’t be able to sniff those packets. The brains that make this possible are the reason switches cost more than comparably specified hubs.
Theoretically, an 8-port gigabit network switch is able to carry 8 total gigabits of data per second. For instance, you could have ports 1 and 2 sending a gigabit per second bidirectionally to each other, and the same for 3 and 4, 5 and 6, and 7 and 8. In actual fact, only the highest end switches can achieve this ideal, but even a cheap switch offers real speed advantages over a dumb hub. These speed adavantages are significant enough that for gigabit Ethernet and a faster, switches are actually required. They are also required when setting up mixed-speed LANs: a dumb 100BaseT hub requires that all nodes support 100 Mbit/s connections, or else all fall down to 10 Mbit/s. A switch is able to support a mix of 10 and 100 Mbit/s nodes, each operating at their fastest native speed.
You do still find cheap 100BaseT hubs, even as of this writing, in 2010. The cost difference between a 100BaseT hub and switch is small, but for applications where they’re cutting costs by still using 100BaseT in the first place, the cost savings can be important.
Packet switching was expensive tech back in the days of 10BaseT, something they did only at the core of large networks to help partition the traffic without resorting to even more expensive routers. Chances are excellent that if you run across a 10BaseT network, it uses dumb hubs for most, if not all, connections.
Above, we implied that with switched Ethernet, only the two nodes involved in a given conversation can see the packets going between them. That was a simplistic view. The real situation is a bit more nuanced.
First, Ethernet switches often have a “port mirroring” feature that makes it copy traffic destined for one port to another. Only the cheapest sort of switch lacks this feature today.
Second, broadcast traffic still goes out to all nodes on a switched Ethernet LAN, unless it’s purposely partitioned into two or more VLANs. Broadcast traffic is usually a minority player on any LAN because it is inefficient, but there is still a fair bit of it even on a well-managed LAN. ARP, Bonjour, UPnP, Windows file and print sharing, and other protocols require broadcast packets. Some of these protocols are entirely broadcast.
Third, the protocol you’re looking to monitor might use multicast. Monitoring multicast traffic isn’t effortless as with broadcast, but it’s pretty simple, even in the face of switched Ethernet. Say you have a small LAN of 8 nodes, and the PCs plugged into LAN ports 1-6 are watching a multicast video stream coming from a networked video camera plugged into port 7. A smart Ethernet switch won’t copy those packets to port 8. It is a trivial matter, however, to write a program to join that multicast group. This causes the stack to send out an IGMP packet which the switch will see, telling it to start forwarding multicast packets destined for that group to port 8 as well.
As computers have become more critical to daily life and business, the security systems on them have gotten stricter.
Ethernet switches with port mirroring abilities often protect access to this feature with a password. Packet switching offers a small measure of improved security on a network. It is the sort of security that adds depth to a strong security regime. Even though it is not enough in itself to secure a network, a network admin that doesn’t trust you still won’t turn on port mirroring for you.
Local computer security is another impediment to an intrepid packet sniffer. Modern OSes block access to the low levels of the stack needed to do packet capturing. There are several facets to this. On Windows, you need local admin access to install the driver you need to perform the packet capturing. Depending on how that driver is itself secured, you might then need admin access to talk to the driver. (Some systems are configured to let certain users or groups of users do packet capturing once support for it is enabled at the kernel level.) Finally, if you need to do promiscuous packet captures, you typically need admin access to enable this feature of the network adapter, quite separate from the actual packet capturing process.
The traffic you are interested in monitoring might be going over a WAN, such as the Internet. Because WAN links are typically much slower than even the cheapest common LAN technology of the day, they have to be guarded from overuse. This is what routers do.
A router is a smart device that is able to recognize which packets on the fast LAN really must be forwarded through the slow WAN link, and which do not.
You may have a cheap Internet router at home with several LAN ports on it, and one Internet port. You can use it to transfer files between two PCs on the LAN at much faster speeds than the Internet link can provide, while a third computer pulls down a different file at much lower speed from the Internet. Because there are routers on both ends of the Internet connection, PCs on either end cannot use sniffer software to see traffic that exists only one one end of the link. Your ISP cannot see the file transfer between the LAN nodes, and a PC on that LAN cannot sniff data going between two other customers of your ISP.
The boundary between LAN and WAN can get a bit fuzzy. Cable Internet systems often put many homes within a certain geographical area onto a single shared LAN. If neither you nor your neighbor have a home router, you might be able to sniff his Internet traffic, because you’re on the same LAN. That said, you probably can’t sniff traffic between two computers across town that also use the same cable Internet service, because there is probably a router somewhere between.
This space intentionally left blank. :)
<< Debugging TCP/IP
||BSD Sockets Compatibility >>|
This article is copyright © 2016 by Warren Young, all rights reserved.
|Updated Sun Jan 18 2015 04:24 MST||Go to my home page|