Most developers begin testing an application by writing a test suite, performing extensive studies, and then checking every possible facet of application performance (at least within reason). Obviously, this kind of testing is important. Without it, the developer doesn't know whether the application is as bug free as possible or that it will fulfill user expectations.

However, many developers stop at this exhaustive level of testing. It's an easy mistake to make, but the reality is that IT administrators, those tasked with installing the application, are unlikely to run the same exhaustive tests. In fact, it's very likely that an administrator will break out a copy of the Application Compatibility Toolkit (ACT) and test your application using it.

Why Test with ACT?

At this point, you are probably thinking that your application will pass any ACT test with ease, since you've already performed extensive testing and debugging using far fancier tools. It would seem like that would be the case, but the reality is that many good application fail the ACT test because it's looking for something different from good hard core coding.

What ACT checks, more than anything else, is compatibility with Windows. In fact, it's searching for compatibility with a specific version of Windows, the one running on the administrator's computer. Factors that you can't easily test using the usual developer diagnostic software come into play, such as the newest patch that Microsoft issued or the unsigned device driver the machine is using. Suddenly, your well-designed and -tested application comes under a kind of stress that you might not have anticipated.

Microsoft is applying additional pressure for administrators to use ACT at the request of the administrators themselves. Most companies that are preparing to deploy Vista, at some point, will rely on the Business Desktop Deployment (BDD) suite of tools, one of which is ACT. There's more than a little interest in BDD. Many administrators are already employing it for Windows XP, and BDD has become so popular that some people have created tutorials for using it (such as this one). The bottom line is that administrators want deployments that are fast, easy, reliable, and respond well to changing conditions. Administrators don't have time to worry about whatever complex tests you performed; all they know is that your application didn't pass ACT.

Don't imagine that you need the whole BDD suite. The package contains more than the average developer needs, such as tools for creating DVDs with complete system setups. While it's important to test your application with ACT, you really don't need to burden yourself with the other BDD features.

Getting into the ACT

Microsoft currently has two versions of ACT in place. You can get the released 4.1 version of ACT. This version requires that you have the .NET Framework 1.1 installed on your system. You also need a version of SQL Server installed. I tried ACT with Microsoft Data Engine (MSDE) and SQL Server Express; it works fine with both. This version of ACT is suitable for all versions of Windows except Windows Vista.

If you really want to test your application for compatibility with Vista, then you need the new 5.0 version of ACT. This new version works very much the same as the 4.1 version, but it provides more extensive testing and looks for problems that you'll encounter when moving your application to Windows Vista. Since Vista promises to make even well-written applications fail due to interesting new features such as User Account Control (UAC) and the inability to access most of the local hard drive, I recommend getting the 5.0 version.

Testing an Application with ACT

Figure 1: Make sure you evaluate your application thoroughly to ensure it passes any administrator test.

Depending on your system setup, you'll want to install the .NET Framework, followed by any supported SQL Server version, and finally ACT. When you start ACT, you'll find that it breaks the task of testing down into four phases: Introduction, Evaluate, Mitigate, and Deploy.

• The Introduction phase is actually a tutorial that tells you how to use ACT. Even though you're a developer and know how to use products instantly, it's a good idea to go through the introduction once, so that you know what Microsoft is telling administrators.
• The Evaluate phase is where testing takes place, and it's the focus of this article. Evaluation includes everything from configuring the collection tools, to prioritizing the test list, to performing the test, and creating a must fix list.
• The Mitigate phase defines what to do when compatibility problems exist. The purpose of ACT is to test an application in the user's environment, not yours. Consequently, it's best to assume some compatibility issues will remain despite your best efforts.
• The Deploy phase explores ways of getting the application onto a machine after compatibility testing. For example, it helps the administrator configure your application to install from Microsoft's System Management Server (SMS). After you get your application to complete the evaluation successfully and explore solutions to problems that will exist on the user's machine, it's a good idea to look at deployment as well, to ensure that the user won't run into problems once your application does work as intended.

For most developers, evaluating the application is the most important part of using ACT. You can see from Figure 1 that evaluation requires a number of steps. All of these steps rely on wizards and other helpful aids, but you should step through all of them to ensure you get the results that truly reflect what an administrator will see.