What They Don't Tell You

Neither the .NET Framework SDK documentation for BeginWrite, nor the Windows API documentation for WriteFile (the API function that BeginWrite uses) make much mention of the possibility that an asynchronous write might block. The documentation for BeginWrite does say that "the underlying operating system resources might allow access in only one of these modes." Still, if the OS didn't support asynchronous writes, you'd expect BeginWrite to return an IAsyncResult that has the CompletedSynchronously flag set. Right?

It took some digging, but I finally came across a Microsoft Knowledge Base article titled Asynchronous Disk I/O Appears as Synchronous on Windows NT, Windows 2000, and Windows XP. This article is chock full of good information. For example, under the heading "Set Up Asynchronous I/O," the article states:

Be careful when coding for asynchronous I/O because the system reserves the right to make an operation synchronous if it needs to. Therefore, it is best if you write the program to correctly handle an I/O operation that may be completed either synchronously or asynchronously.

Having programmed computers for 25 years, I understood and even expected that. It's interesting, though, that the SDK documentation doesn't mention it, nor do any of the many articles I've found about asynchronous I/O under .NET. That still didn't answer my question as to why asynchronous writes were blocking.

The article covers many reasons why your asynchronous file operations might appear synchronous. I found my answer to the problem under the heading "Extending a file." As the article states:

Another reason that I/O operations are completed synchronously is the operations themselves. On Windows NT, any write operation to a file that extends its length will be synchronous.

The article mentions Windows NT specifically, but I've seen the same behavior on Windows 2000 and Windows Server 2003. The rest of the section goes on to explain how to get around this limitation, and the security reasons for not attempting the workaround.

The problem is that the FileStream constructor I call is creating a new file (FileMode.Create), which means that any existing file is truncated to zero bytes so that it's just like a newly-created file. Writing any data to that file extends it, thereby causing the operation to proceed synchronously. I proved that this is the cause of the error by running my program once and then changing the open mode to FileMode.Open. When I ran the program again, it opened the existing file and wrote data to it from the beginning, overwriting the existing data. That operation proceeded asynchronously, as you would expect.

I don't know enough about Windows internals and the NTFS file system to understand why extending a file has to be a synchronous operation. It'd be interesting, although not terribly useful, to learn the reason, and I encourage any member of the Windows team to contact me with an explanation. Understanding why doesn't solve my problem, though. I want my asynchronous writes!

Alternate Feline De-furring Method

The problem here is that the operating system blocks the thread during a write that extends a file, and I don't want my thread blocked. That was the whole point of attempting an asynchronous write operation in the first place. Sometimes if you want something done write (er, right), you just have to do it yourself. Since BeginWrite won't guarantee asynchronous operation, I decided to do the I/O by creating and calling an asynchronous delegate. Doing so presents two primary advantages:

• I/O is guaranteed to be asynchronous because it is executing on a background thread.
• The code executed in the background can involve multiple reads and writes, and arbitrarily complex calculations interspersed between I/O operations.

Creating an asynchronous delegate isn't much more complicated than creating a file I/O completion callback. All you need to do is define the method that will execute on the background thread, define a delegate for it, instantiate a delegate, and execute. The code below replaces the asynchronous write code from the last example.

// Define the method type that will be called
private delegate void WriteDelegate(byte[] dataBuffer);

static void asyncWriteWithDelegate()
{
    byte[] data = new byte[BUFFER_SIZE];

    // create a new WriteDelegate
    WriteDelegate dlgt = new WriteDelegate(WriteTheData);

    // Invoke the delegate asynchronously.
    IAsyncResult ar = dlgt.BeginInvoke(data, null, null);

    // WriteTheData is now executing asynchronously
    // Continue with foreground processing here.
    while (!ar.IsCompleted)
    {
           Console.Write('.');
           System.Threading.Thread.Sleep(10);
    }
    Console.WriteLine();

    // harvest the result
    dlgt.EndInvoke(ar);
}

static void WriteTheData(byte[] dataBuffer)
{
    using (FileStream fs = new FileStream("e:\\writetest.dat", FileMode.Create,
                            FileAccess.Write, FileShare.None))
    {
            Console.WriteLine("Begin write...");
            fs.Write(dataBuffer, 0, dataBuffer.Length);
            Console.WriteLine("Write complete");
    }
}

Notice the similarity between this code and the asynchronous write code that uses FileStream.BeginWrite. Granted, you have to define and create a delegate, but other than that, the code is almost identical. Calling the Invoke method is similar to BeginWrite, and you harvest the result by calling EndInvoke in the same way that you call EndWrite. You even check for completion in the same way: by testing IAsyncResult.IsCompleted, or by waiting on the AsyncWaitHandle.

You can even pass a completion callback routine and state object to BeginInvoke in the same way that you can for BeginRead and BeginWrite. The completion callback is executed on the background thread after the delegate is done executing, and the state object is passed in the AsyncState property of the IAsyncResult object.

Note also that WriteTheData calls Write rather than BeginWrite to initiate the I/O. Since WriteTheData is executing on a background thread, it makes little sense to do the I/O asynchronously. This would hold true for reads, too, although I guess there are rare situations in which you'd want a background thread to do asynchronous I/O.

The second advantage of using asynchronous delegates rather than BeginRead or BeginWrite sometimes is overlooked. More often than not, you want your program's initialization code to do more than just read a file into a buffer — something that the built-in asynchronous functionality does quite well. Very often, you want to read the file in blocks (or records), and process those blocks to create internal data structures, or do some post-processing as you're writing data to a file. Perhaps you want to read or write an encrypted file. In those and many other common situations, the standard asynchronous I/O support supplied by FileStream isn't enough. Considering how easy it is to create and invoke an asynchronous delegate, I've found that it's easier to do all of my asynchronous I/O—even simple file reads into a buffer—through asynchronous delegates. Doing so guarantees that the operation will execute in the background, and also makes it easier for me to make the inevitable changes when the client requests different functionality.

Asynchronous I/O to Other Devices

All classes that inherit from System.IO.Stream implement the BeginRead and BeginWrite methods. However, the default implementations of these methods in System.IO.Stream simply call Read and Write, making the operations synchronous. Only those classes that override BeginRead and BeginWrite actually support asynchronous I/O. Of the .NET Framework classes that inherit from System.IO.Stream, only System.Net.Sockets.NetworkStream includes full support for asynchronous operation. All others depend on the default implementation of BeginRead and BeginWrite, which meansthat all I/O operations on those streams will block the calling thread until the operation is completed.