I've written a number of C++/MFC programs using the overlapped I/O approach, and it has always been a real pain. Even MSDN doesn't really seem to like it:



The problem with doing asynchronous I/O with OVERLAPPED function calls like ReadFileEx is that you have to be extremely careful with how you design your program to continue executing with a callback function or by waiting on an EVENT handle to get signaled. The callback function requires dealing with alertable states, which is really error-prone.  No, seriously, don't do it because dealing with the re-entrancy can drive you nuts. Alertable wait states are terribly hard to reason about because it is the non-GUI analog to pumping messages (c.f. Raymond Chen).

So I recently started using IOCP for the same purposes, and I'm happy to report that it works much better! I guess that shouldn't be a surprise because it's been recommended by some excellent programmers. Anyway, here's the rough idea of how I got it to work for me:

1. Create a background thread with an IOCP handle.


2. Open a file (or socket, serial port, etc.) with OpenFile and associate it with the IOCP.


3. Initiate I/O using ReadFile (not ReadFileEx).


4. Handle both synchronous and asynchronous completion of the I/O.

That last point was the hardest for me to realize that I absolutely have to deal with synchronous, immediate completion of the I/O (not just asynchronous completion from the IOCP). So you really do have to check the return value from ReadFile and also GetLastError() == ERROR_IO_PENDING.

IOCP is so good that it is the basis for C#'s multi-threading I/O. The designers of C# must not have even cared about the old ways to do I/O (e.g., waiting on events, alertable callbacks with APCs, OVERLAPPED callback functions, etc.).