HomeTechniques A Gentle Introduction to Software Testing
A Gentle Introduction to Software Testing ByDee-Ann LeBlanc 2004-07-31
Article Rating: / 2
Rate This Article:
Add This Article To:
Software testing consists of more than random banging on an application, the way a toddler whacks on a piano to make lots of noise. Learn what's really necessary to test your software correctly.
Software Testers, or Test Engineers—depending on where you work, where you are on the payscale, what your duties are, and more—exist in a somewhat repetitive world. At its heart, the job is to take a hammer to someone's program, and see if you can make it break.
However, testing is more than sitting down at a keyboard and randomly trying things. Whether you're looking to integrate testing practices into your small company without adding a full-time staff member, or you're considering becoming a software tester as your own career path, many tools and techniques are available to test things at a far greater depth—and speed—than by haphazard typing and clicking.
ADVERTISEMENT
The initial techniques of testing falls under either white box testing, or black box testing. In white box testing, the tests that hammer on the program are designed by people familiar with the actual code. In these situations, testers (or, at least, those responsible for building the tests) need a strong familiarity with programming, or must be given the time to heavily interview the developers themselves. The white box approach lets you make sure the code was written properly.
Black box testing, however, is used by people who never look at the code. The emphasis and purpose of black-box testing is to judge whether all of the requirements and specifications have been met. A combination of both techniques is essential for a fully-rounded testing program.
Another distinction, along with white box and black box, is another pair of issues: automated tests and manual tests. Manual tests are typically those that work best with human eyes, brains, and intuition behind the keyboard.
As you might guess, automated tests are those that can be automated; they are typically repetitive in nature. Automated tests also can be run on a frequent, regular basis, giving constant feedback to the developers on how things are working. This makes your project more flexible, as it's much easier to change things if you know something you added in the last twenty-four hours slowed down performance than it is if you know something you added in the
last two months slowed down performance.
Once you go below the hood with both white and black box testing, you run into lots of different ways of building tests, and many different types of things to test. There are unit tests, Web tests, performance tests, functional tests, security tests, accessibility tests, network tests,
acceptance tests, regression tests, smoke tests, usability tests, and lots more.
Most tests actually end up falling into the automation camp, such as unit tests. Mathematically-oriented folks might suspect that unit tests refer to unit conversions (cm to ft), and business-oriented people might think of inventory. Instead, this vital component of methodologies, such as eXtreme Programming and Test-Driven Development (where unit tests are now officially referred to as programmer tests), involves building small, individual tests for each little piece of functionality. For example, a unit test might be as simple as checking to see that, when the configuration is saved to a file, it really was saved to a file of the proper format that can be properly opened by the program. (In fact, this scenario is complex enough that some might actually break this into multiple unit tests.)
Acceptance testing is a different beast altogether. Acceptance tests may be written by the end user or customer. Even when they're not programmed by the customer, they are actually designed by the customer, typically by having a test engineer sit down with the customer and work them through the process of how to test to ensure that the program does everything it is required to do. After drawing out of the customer an incredibly detailed walk-through, the test engineers can then turn around and design automated tests around these functions, and make sure that everything the customer will try to do is possible in the software, and works.
One relative of acceptance tests are functional tests, which are used to test functionality though may not necessarily have been designed by the customer. Another relative is the smoke test, which is less detailed.
Then there's regression testing. If you follow discussions of eXtreme Programming and Test-Driven Development (TDD), you're familiar with this concept, even though you may not recognize the term. Typically, regression tests are initiated when a developer feels she has fixed
a bug—in order to determine whether that bug has really been fixed—and after code is added that could possibly break another feature. These tests are cumulative; they watch for code farther down the road that may bring a bug back to life or introduce new ones.
TDD is an excellent example of regression testing in action, as its practitioners are proactive about creating the necessary tests. Automation comes into the picture because, ideally, regression tests are run at least nightly after all code is checked in, "just
in case," without needing a programmer to initiate them.
Performance testing, on the other hand, is rarely focused on overt coding bugs. Instead, its entire function is to see how an application handles particular levels of load. One result of performance testing is discovering sloppy code that creates infinite loops or other performance-slowing problems. Under the banner of performance testing are issues including load tests (just described) and scalability (increasingly bigger and bigger load) testing. Since performance testing tends to involve sending lots of traffic, it's ideal for automation, rather than hiring a thousand people to sit there and surf your site or access your software for an hour.
The network testing umbrella includes both network functionality and performance. In this testing field, you typically need an in-depth understanding of the network protocols and hardware you're dealing with, such as TCP/IP over Ethernet. Such a background helps in designing and selecting proper benchmarks for network performance and then tuning as needed, especially since it is so easy to overtune yourself into even worse performance. Hunting down
network bottlenecks is a popular focus in network testing.
Where you have networks, you have security issues. Security testing addresses all areas, from the login process to encryption to making sure that sensitive data is not displayed where it shouldn't be. There's also testing for common vulnerabilities, such as how your application responds to an SQL injection attack (trying to send unexpected SQL commands to a database to see if you can interact with it out of bounds of the application) and buffer overruns (sending so much information that an error is produced that can allow you to send instructions to a machine)... and lots, lots more. Proper security testing can make the difference between your product having as many vulnerabilities as swiss cheese has holes, or being rock solid and dependable.
For accessibility testing, you'll probably pay attention to both automated and manual approaches available. A combination probably does the job best. In some cases, government guidelines must be followed, so that anyone can use a program or service. The software testers must make sure that the software or site works well with screen readers for the blind, along with plenty of other accessibility requirements. These types of tests can be automated or can involve a person with a checklist who makes sure all legal requirements are addressed. Even better is having someone from the target audience try the system to see how it can be improved for their needs.
Some people confuse usability testing with accessibility testing, but these are very different arenas. Usability is focused on how easy a program is to use. The look and feel, help system design and content, terminology and jargon on the main screens, menu layouts, error messages, and even how the application handles keyboard and mouse input all wrap around the usability maypole. Here is where you start to fall out of the automated testing arena and into more manual testing; typically, people run through the program with checklists, determining whether the program is as usable as it should be.
Finally, Web tests in general refer to the overall genre of writing tests for specific Web sites, both their content and their performance. Web tests encompass all tests discussed elsewhere in this article. Not all Web tests can be automated, but most can be. For example, it's easy to automate a test that makes sure all of your links point to valid pages, though automation is often not simple when it comes to usability testing.
Now it should be easy to see why it is difficult to properly test software when you're a small shop with a limited budget. There are many ways to optimize your testing when you can't afford a fleet of professional testers. Primarily, keep these types of testing in mind as you design and implement the software. Doing so can help your team develop tests and checklists along the way.
On the other hand, if reading this article got you excited with lots of ideas of how you might test your code or code in general, then maybe you have a future as a software tester. This can be quite a creative field, especially when it comes to creating large amounts of test data to feed into systems, not to mention the fun you can have trying to think of ways that the average user might break an application, or the more than average cracker might use to try to take advantage of a system.