In my previous article I detailed the many challenges I encountered while creating a .NET interface to the cabinet file API in CABINET.DLL. Obviously, I didn't develop that interface in a vacuum. Rather, I developed test programs in parallel and maintained a continual testing cycle: modify the interface, change the sample program, compile, test, debug, lather, rinse, repeat.

Fortunately, the Cabinet SDK available from MSDN includes two C sample programs, testfdi and testfdi, that implement all of the required callback functions and illustrate how to use the cabinet functions. I duplicated those two programs in C# so that I could verify that my cabinet interface works; the idea was that if my C# program created a file identical to the C version, then I'd know that the interface is correct.

On the surface, translating the C sample programs to C# appears to have been an almost mechanical exercise. It was, in some places, mostly because C and C# are such similar languages. In the testfdi program, for example, the initialization code and implementation of the notification callback function are very similar. The other callbacks, though — particularly the memory allocation and file I/O callbacks — are significantly different, because the environments' differences in semantics of memory allocation and file manipulation.

In addition to the callbacks semantics mentioned above, the primary issues I encountered when implementing the sample programs in C# had to do with error handling, and with passing managed objects through the unmanaged code. Error handling posed a problem because the ".NET way" of doing things involves using exceptions, but the Cabinet API is designed to use error return codes. I came up with an interesting way to meld the two so that exceptions could be thrown in the callbacks, passed through the unmanaged interface, and then re-thrown in the main program.

Memory Allocation

Cabinet API functions make many calls to allocate memory. Rather than rely on the system's heap or the C runtime library, the Cabinet API team decided to make the client program responsible for managing memory. This makes it possible to port the Cabinet API to other platforms very quickly, and also allows the client program full control over the way that memory is used. It can be allocated on the heap, reserved from the stack, allocated from a contiguous buffer, etc. It's up to the client program to determine how memory is allocated, but regardless of how it's done, the program must return an unmanaged pointer to the allocated block. The client also must be able to deallocate memory that's referenced by the pointer.

The memory allocation and deallocation delegates for the file decompression interface (FDI) are defined in CabDotNet.cs, like this:

public delegate IntPtr FdiMemAllocDelegate(int numBytes);
public delegate void FdiMemFreeDelegate(IntPtr mem);

The memory allocation delegates for the file compression interface (FCI) are identical to those for FDI.

The memory allocation delegate must allocate a block of memory of the size specified by the numBytes parameter, and return a pointer that the Cabinet API functions can use to access the memory. If the function can't allocate memory, it's supposed to return null—the universal "out of memory" error for C programs. When I originally wrote this, I created managed prototypes to the Windows API HeapAlloc interface. Although that worked, I wanted as much as possible an "all .NET" implementation. I finally ran across the Marshal.AllocHGlobal method, which allocates memory from the Windows global heap and returns a pointer to that memory as an IntPtr value. That's exactly what I was looking for, and the first version of the callback function looked like this:

public static IntPtr MemAlloc(int cb)
{
    return Marshal.AllocHGlobal(cb);
}

public static void MemFree(IntPtr mem)
{
    Marshal.FreeHGlobal(mem);
}

There also exists a method, Marshal.AllocCoTaskMem, that would serve the same purpose. I don't know if that would be preferable to allocating from the global heap.

The above works, except that the program crashs if the allocation fails. Marshal.AllocHGlobal throws an OutOfMemoryException if there is insufficient memory to satisfy the request. If the callback doesn't handle that exception, it's going to bubble back to the main program, bypassing the Cabinet API and leaving any previous memory allocations hanging and also leaving files open. It's ugly. The callbacks need to handle exceptions gracefully. The memory allocation function must return null (IntPtr.Zero) if it can't allocate memory. If the memory deallocation function fails for some reason, it should handle the exception silently and return. Here are the more robust memory allocation and deallocation callbacks:

public static IntPtr MemAlloc(int cb)
{
    try
    {
        return Marshal.AllocHGlobal(cb);
    }
    catch (Exception)
    {
        // Return null if unable to allocate memory
        return IntPtr.Zero;
    }
}

public static void MemFree(IntPtr mem)
{
    try
    {
        Marshal.FreeHGlobal(mem);
    }
    catch (Exception)
    {
        // just swallow the exception
    }
}