Winsock Programmer’s FAQ
Section 2: Information for New Winsockers
From the programmer’s standpoint, there are two major
versions of Winsock, 2.0 and 1.1. These are the version numbers
you pass to
WSAStartup(), and your choice affects whether
you #include <winsock2.h> and link against
ws2_32.dll or you #include <winsock.h>
and link against wsock32.dll. You could choose to declare
your program as using some other minor version, but unless you know
for a fact that you need to use something different, I recommend
using only 1.1 or 2.0, for best compatibility.
The Winsock design is a series of additions to older technologies, not something wholly new. Winsock 2.0 is mostly a strict superset of Winsock 1.1, and Winsock 1.1 is mostly a strict superset of BSD Sockets. (In both cases, there are some things left behind or which simply change. I cover these things elsewhere in the FAQ.)
The older APIs are still quite useful. If your program doesn’t need anything added in later versions of the API, it’s perfectly fine to limit your program to only Winsock 1.1 or to portable BSD Sockets.
The Winsock 1.1 specific DLLs are still provided on the newest versions of Windows. Winsock 1.1 has some limitations compared to Winsock 2, but there are few types of programs that absolutely require Winsock 2 features.
Likewise, BSD Sockets, though a very old API now, still contains a large amount of functionality. Most network programs running today on Linux, Unix, and OS X still use nothing but the plain old BSD Sockets API. Viewed that way, even Winsock 1.1 looks modern, new, and powerful. If your program doesn’t need any of the Windows-specific extensions, or simply has to be portable to these other operating systems, feel free to stick to the BSD Sockets subset of the API. There are some details that prevent you from simply recompiling a BSD Sockets program on Windows, but these are generally easy to work around.
I’m not aware of any penalty for choosing an older API on a newer operating system. Nor for that matter am I aware of penalties for using an older API subset while building your program against Winsock 2. There is the decreased portability, of course, but since there are Winsock 2 options for OSes going back 14 years now, it’s not generally a practical concern.
The FAQ’s example programs are all set up to build as Winsock 2 code just to be modern. (Again, 14 years!) That said, you could easily make most of them build against Winsock 1.1 simply by changing the things mentioned at the top of this FAQ item.
WSAStartup() fills this structure in for you.
You’re most likely not linking with the proper Winsock import library. This is ws2_32.lib if you’re targetting Winsock 2, or wsock32.lib for Winsock 1.1.
Absolutely! This common question is the result of confusing protocols with the APIs. Communicating programs need not have been created with the same APIs, as long as they are using the same transport and network protocols.
Before dealing with cross-platform networking, please read the FAQ article How to Use TCP Effectively. It covers several issues that bite cross-platform programs, like structure padding and data representation.
Maybe, but Winsock is usually only used directly in low-level languages like C and its close relatives. There are several reasons for this.
Reason 1: Some languages simply lack the language features to call the Winsock API directly.
A programming language needs the following features to access Winsock:
Pointers. (The ability to access a specific piece of memory by its address.)
Bitwise operators. (The ability to change specific bits in a byte.)
Structures or records. (The ability to define a block of memory that is an aggregate of simple data elements, such as two characters followed by a 16-bit integer. This feature must also allow some measure of control as to how the data is laid out in memory.)
Most high-level languages lack one or more of these features. Your language might have some kind of extension mechanism that lets you write code in C or a similar language to provide a bridge, but that is usually not necessary.
Reason 2: Most modern languages commonly used on Windows provide network support as part of the language or its runtime. This is typical of cross-platform languages like Java, Perl, Python, Tcl, Ruby, Lua... Even more specialized languages, like R, Mathematica, and Erlang usually include network support. Unless you are using a language without network support built in or need to dig into some Winsock-specific mechanism not exposed by the language’s network support, you won’t have cause to use Winsock directly. Using the language’s own network API usually results in smaller, more portable code.
Reason 3: Older languages often didn’t have networking as part of the language, but had some extension mechanism whereby networking was later added. A good example is old versions of Visual Basic, and the now-moribund market for ActiveX networking controls. Other than this external component difference, the argument follows that for Reason 2.
For these reasons and others, this FAQ is biased towards C and C++. It probably also applies just as well to other close C family members like Objective C and D, but I don’t use those, so I haven’t tried to ensure code compatibility.
If your language allows direct access to the Winsock API, you may be able to translate the C++ code in the FAQ into equivalent code in your chosen language. However, you’ll probably do better to try and find sample code in your chosen language elsewhere on the net. You may be able to find something that does exactly what you need, and if not, at least you will have a basis in your language’s network APIs which will make it easier to translate the code here into your language.
The two main categories of debugging tools are sniffers and shims.
I’ve moved all discussion of what sniffers are, their limitations, and workarounds for those limitations into a separate article, “The Straight Dope on Packet Sniffers.”
The main advantage of a sniffer is that it literally sees every bit of data in a network conversation, way down below the Winsock level to the network hardware’s protocol layer. (e.g. Ethernet) Another advantage is that a good sniffer is quite powerful and configurable. For example, some allow you to write “protocol plugins” that will decode any protocol, such as a custom protocol that you’ve developed. You can also build complex expressions to filter out just the small bit of traffic you need to see to debug a problem, so you don’t have to wade through the torrent of variegated data on a modern LAN.
By far, the most popular packet sniffer today is the open source and cross-platform Wireshark suite. (Previously called Ethereal.) In the same way that Firefox slowly, unevenly, yet inevitably steamrolled its commercial competition, Wireshark has pushed out all but the very best commercial offerings. It is not perfect, but it is more than good enough for day-to-day packet sniffing; I use it almost exclusively.
The main commercial sniffers left are WildPackets’ OmniPeek, Network Instruments’ Observer, and NetScout’s Sniffer. (An ancestor of the latter is the source of the now-kleenexed term “sniffer.”) These all cost north of US$ 1,000 and run into deep the thousands for enhanced versions and optional add-ons. I haven’t used any of them in many years, so I cannot advise about the conditions where they are worth the cost. (They must be, else they would not have survived this long in the face of a good-enough free alternative.)
The only other sniffer I use is tcpdump. Its main virtue is
that it’s installed by default on almost every Linux, Unix
and OS X box out there. You can usually find such a box somewhere
nearby when you need to debug something. I almost never use it to
analyze captured packets; its display format is difficult to read
and often doesn’t tell you enough besides. Instead, I use its
-w option to write the captured packets to a file, which
I then compress, transfer back to my PC, then load up in Wireshark
for analysis. I can do this even when not on-site with the problem
by dialing or
sshing into a *ix box on site. Beware
that you might need to give the
-s option to get it to
capture more than just the packet headers.
You can use Wireshark’s
companion for this instead, if available. Its output format is more
readable and it captures whole packets by default.
Another major category of debugging tools are shims. (Also called spies.) A shim uses any of several different methods to monitor a single application’s use of the Winsock APIs. Shims present a higher-level view of your program’s network interactions than sniffers.
The difference in perspective can make it easier to map what it displays onto what you know your program is trying to do. Captured packets are sometimes too far removed from the problem source to be immediately helpful.
(The reverse can be true, too: knowing the sequence of API calls doesn’t tell you every detail about the data that is actually going out on the wire. Sniffers tell you that.)
Shims can sometimes work where sniffers will not. Some types of point-to-point links (e.g. PPP over a modem) are sniff-resistant due to their nature, but if they’re exposed to Winsock, a shim can see them. Some types of shims work in such a way that they don’t require administrator privileges on the local machine, unlike most sniffers.
There are various other network debugging tools out there useful to programmers.
Every network programmer needs a port scanner, with which you can probe a machine to learn what ports are open, what ports are closed or blocked by a firewall, even what OS a remote machine is running based on how it responds to the scanning packets. One nice GUI tool of that sort is Atelier Web’s Security Port Scanner, a.k.a. AWSPS. (US $34 to 120, depending on options) A more powerful but less pretty open source alternative is Nmap.
Another useful class of tool is something that tells you
which programs on a given machine are listening on which
ports. Merely knowing that something is listening on
TCP port 8472 isn’t as useful as knowing what
is listening on that port. AWSPS does this, as well. One
free but closed-source tool to do this is SysInternals’ TCPView.
Another is Vision,
from security firm Foundstone. In Windows XP and up, you can
get a primitive form of this information at the command line
netstat -ba. The
option is Windows-specific; to get the same information on Linux,
BSD, and OS X systems, look into the awesome
command. (The example question posed above would be answered with
No one tool will do everything you need. You might use a port scanner to check for an open port, start a sniffer capturing packets on that port, point your program at it, capture the data, then go back and run it through a shim to find out why the API calls you’re making are giving you a some weird network packets the sniffer captured.
I suggest that you start with Wireshark, Nmap and TCPView, then explore the commercial alternatives’ demos to see if they add any features you just gotta have. I find less use for shims myself, but that may be a result of long experience; I rarely find myself at a loss to explain why a particular packet capture was produced by a given program.
You may also find the FAQ article Debugging TCP/IP useful for some less-automated methods of debugging a TCP program.
Methods That Don’t Work
There are a couple of debugging tools that are supposed to work
that don’t, or are too flaky to deal with. The first is the
SO_DEBUG socket option. It simply doesn’t work
on Microsoft stacks. The other is the Winsock DLL debugging plugin
dt_dll.dll; this method is flaky. Bob Quinn wrote an article
about this, but unfortunately the site that held it was bought by
another company that hasn’t yet made that article available
The problem with this question is that it assumes that there is
a “good” canned error message for every situation. The
reality is that many times, you need to know the program’s
context before you can turn an error value into a meaningful error
message. For example,
WSAEFAULT can mean any of the
following, depending on context:
Since the Winsock docs tell you the most likely error values that each function will return, you should use this information to construct intelligent error handlers.
Still, sometimes an API call returns something unexpected,
so a cryptic error message is better than none at all. For such
“default” case handling the
on all modern versions of Windows
(except NT 4) will return canned messages when given Winsock error
I must stress that you’re better off spending your time constructing meaningful error messages based on program context than chasing something that doesn’t work very well even in the best case. Use canned messages only when your program has no better idea what to do about an error.
WSAEWOULDBLOCK. What’s wrong with my program?
Not a thing.
WSAEWOULDBLOCK is a perfectly
normal occurrence in programs using non-blocking and asynchronous
sockets. It’s Winsock’s way of telling your program
“I can’t do that right now, because I would have to
block to do so.”
The next question is, how do you know when it’s safe to try
again? In the case of asynchronous sockets, Winsock will send you
FD_WRITE message after a failed
when it is safe to write; it will send you an
message after a
recv() call when more data arrives on that
socket. Similarly, in a non-blocking sockets program that uses
writefds will be set when it’s
okay to write, and the
readfds will be set if there is
data to read.
Don’t count on Winsock to handle blocking the way you think
it ought to. Whenever you use some form of nonblocking sockets,
you have to be prepared for
WSAEWOULDBLOCK at any
time. It’s simply a matter of defensive programming, just like
checking for null pointers.
There is a special address called the loopback or localhost address, 127.0.0.1. This lets two programs running on a single machine talk to each other. The server usually listens for connections on all available interfaces, and the client connects to the localhost address. (See the Example Programs section for basic client and server program code.)
If you have an Internet or LAN connection on your development machine, you’re already set up for this.
For machines with no network at all, you have to set up a “dummy” network. On modern versions of Windows, you can go into the Network control panel, tell it to add an interface, and select the Microsoft Loopback Device. You don’t need any other networking support for this to work.
Be warned: behavior through the loopback interface will differ from that on a “real” network, if only because conditions are much simpler within a single machine than over a LAN or WAN. You should try to test your application on a real network, even if you do primary development on a single machine.
TCP is a bidirectional protocol. Think about it like a conversation between two people. It’s rude for one person to simply stop talking and walk away. Proper social protocol requires that the person who is done talking wait and listen to what the other person has to say before walking away. One side starts winding down the conversation — “Well, it’s been nice, but I have to go.” — then politely waits for the other person to finish and agree to finish. (“Yes, okay, I’ll see you later.”)
TCP connections with Winsock are designed to work the same way:
One side finishes sending data. It tells this to the stack
shutdown() with the how parameter set to
Both sides continue to call
recv(). The side that
closed its sending half of the connection must because the remote
peer probably isn’t done sending yet. The peer that is still
sending must continue calling
recv() because it will get 0 when
the remote peer calls
If you are using asynchronous sockets or event objects, you will
FD_CLOSE when this happens, too. In programs
select(), you simply get a signaled socket in
readfds, which tells you to call
whereupon you get the 0 return value. Beware that both sides still
have to be prepared to receive a -1 return from
indicating an error.
Both sides call
closesocket() only when they are
convinced the conversation is over, either because they have stopped
sending and they got 0 from
recv(), or they got a connection
Not following this graceful shutdown protocol can cause data loss.
Nonblocking or asynchronous sockets complicate this. You can either build “finish sending/receiving” logic into your normal I/O loop, or you can temporarily put the socket in blocking mode and do the last bits of I/O that way. The proper choice depends on your program’s architecture and requirements.
Sure, but it’s an evil thing to do. :)
The simplest way is to call
shutdown() with the
how parameter set to
directions”), optionally followed by a
call. Another method is to set the
flag to 0 with the
setsockopt() call before you call
closesocket(). This causes a TCP connection reset. (That is,
a TCP packet goes out with the
RST flag set, forcing
the remote stack to drop the connection immediately.)
“Slamming the connection shut” is only justifiable in a very small number of cases. You must have fairly deep knowledge of the way TCP works before you can properly decide to use this technique. Generally, the perceived need to slam the connection shut comes from a broken program, either yours or the remote peer. I recommend that you try to fix the broken program so you don’t have to resort to such a questionable technique.
All of the I/O strategies discussed in the I/O strategies article have some way of indicating that the connection is closed.
First, keep in mind that TCP is a full-duplex network protocol. That means that you can close the connection half-way and still send data on the other half. An example of this is the old HTTP 1.0 web protocol. The browser sends a short request to the web server, then closes its half of the connection. The web server then sends back the requested data on the other half of the connection, and closes its sending side, which terminates the TCP session. (HTTP 1.1 is more complex than this, but at the end, the same basic thing happens.)
Normal TCP programs only close the sending half, which the remote peer perceives as the receiving half. So, what you normally want to detect is whether the remote peer closed its sending half, meaning you won’t be receiving data from them any more.
With asynchronous sockets, Winsock sends you an
FD_CLOSE message when the connection drops. Event
objects are similar: the system signals the event object with an
With blocking and non-blocking sockets, you probably have a
loop that calls
recv() on that socket.
0 when the remote peer closes the connection. As you would expect,
if you are using
select(), the SOCKET descriptor in the
read_fds parameter gets set when the connection drops. As
normal, you’ll call
recv() and see the 0 return value.
As you might have guessed from the discussion above, it is also possible to close the receiving half of the connection. If the remote peer then tries to send you data, the stack will drop that data on the floor and send a TCP RST to the remote peer.
See below for information on handling abnormal disconnects.
The previous question deals with detecting when a protocol connection is dropped normally, but what if you want to detect other problems, like unplugged network cables or crashed workstations? In these cases, the failure prevents notifying the remote peer that something is wrong. My feeling is that this is usually a feature, because the broken component might get fixed before anyone notices, so why demand that the connection be reestablished?
If you have a situation where you must be able to detect all network failures, you have two options:
The first option is to give the protocol a command/response structure: one host sends a command and expects a prompt response from the other host when the command is received or acted upon. If the response does not arrive, the connection is assumed to be dead, or at least faulty.
The second option is to add an “echo” function to your protocol, where one host (usually the client) is expected to periodically send out an “are you still there?” packet to the other host, which it must promptly acknowledge. If the echo-sending host doesn’t receive its response or the receiving host fails to see an echo request for a certain period of time, the program can assume that the connection is bad or the remote host has gone down.
If you choose the “echo” alternative, avoid the temptation to use the ICMP “ping” facility for this. If you did it this way, you would have to send pings from both sides, because Microsoft stacks won’t let you see the other side’s echo requests, only responses to your own echo requests. Another problem with ping is that it’s outside your protocol, so it won’t detect a failed TCP connection if the hardware connection remains viable. A final problem with the ping technique is that ICMP is an unreliable protocol: does it make a whole lot of sense to use an unreliable protocol to add an assurance of reliability to another protocol?
If you can restrict your program to Windows 2000 and its direct descendants, one option you might consider is TCP keepalives. This is a way to tell the stack to send a packet out over the connection at specific intervals whether there’s real data to send or not. If the remote host is up, it will send back a similar reply packet. If the TCP connection is no longer valid (e.g. the remote host has rebooted since the last keepalive), the remote host will send back a reset packet, killing the local host’s connection. If the remote host is down, the local host’s TCP stack will time out waiting for the reply and kill the connection. The main problem with keepalives is that these packets are entirely wasteful: they carry no useful data. At least with the command/response option, you have the possibility that every packet carries meaning.
Note that different types of networks handle physical disconnection differently. Ethernet, for example, establishes no link-level connection, so if you unplug the network cable, a remote host can’t tell that its peer is physically unable to communicate. By contrast, a dropped PPP link causes a detectable failure at the link layer, which propagates up to the Winsock layer for your program to detect.
Some of the blocking Winsock functions (e.g.
a timeout embedded into them. The theory behind this is that only
the stack has all the information necessary to set a proper timeout.
Yet, some people find that the value the stack uses is too long for
their application; it can be a minute or longer.
You can adjust the
setsockopt() options. .
For other Winsock functions, the best solution is to avoid blocking sockets altogether. All of the non-blocking socket methods provide ways for you to build custom timeouts:
Non-blocking sockets with
select() – The
fifth parameter to the
select() function is a timeout value.
Asynchronous sockets – Use the Windows API
Event objects –
has a timeout parameter.
Waitable Timers – Call
CreateWaitableTimers() to make a waitable timer, which you
can then pass to a function like
WSAEventSelect() along with
your sockets: if none of the sockets is signalled before the timer
goes off, the blocking function will return anyway.
Note that with asynchronous and non-blocking sockets, you may be able to avoid handling timeouts altogether. Your program continues working even while Winsock is busy. So, you can leave it up to the user to cancel an operation that’s taking too long, or just let Winsock’s natural timeout expire rather than taking over this functionality in your code.
Peeking is looking ahead in the TCP data stream: when you use
the MSG_PEEK flag with
recv(), it returns bytes from the
stack’s buffer without removing these from the buffer. (You
can also do a form of peeking with the
Peeking is essentially never necessary: you can always read data into your own buffers and process it there. This is good, because peeking often causes problems. Indeed, it’s so problematic it’s earned a place on the Lame List and in Microsoft’s Knowledge Base: see article KB192599 for specific info on the problems peeking causes with their Winsock stack.
Out-of-band (OOB) data is like a second data channel. The intent is
to use the regular TCP data stream for most data and the OOB stream
for “emergency” messages. The telnet protocol uses this
for “interrupt” keystrokes like Ctrl-C, so that they
don’t have to wait on the remote peer to handle regular TCP data
before the interrupt occurs. You can send OOB data by passing the
MSG_OOB flag to
send() and receive it by passing
recv(). You can also get OOB data by
SO_OOBINLINE flag with
OOB data is a useful concept, but unfortunately there are two conflicing interpretations of how OOB data should be handled at the stack level: the original description of OOB in the TCP protocol specification (RFC 793) was superceded by the “host requirements” spec (RFC 1122), but there are still many machines with RFC 793 OOB implementations. Section 3.5.2 in the Winsock 2.2.2 spec provides the gory details on why RFC 793 vs. RFC 1122 is a problem.
OOB also isn’t a fully functional second data channel: it’s rather limited, even between two machines that agree on what OOB means.
So, never use OOB except when implementing legacy protocols like telnet which demand it. You can get reliable OOB-like behavior by simply using two data connections: one for normal data, and the second for emergency data.
recv()’s flags parameter?
It’s perfectly valid to pass 0 for
recv()’s flags parameter.
<< General Winsock Information
||Intermediate Winsock Issues >>|
|Updated Sun Oct 25 2009 09:40 MDT||Go to my home page|