Test Case Generation

I like JUnit. I like it, a lot and I recommend it to everyone who will listen to me. JUnit is an invaluable tool for testing code, and is simply a requirement when writing J2EE code. However, I truly dislike writing test cases. No fun, no challenge, just grunt work.

Either Parasoft heard me muttering my dark thoughts (doubtful) or a large number of people share my views when it comes to writing test cases. Because, to my great relief, Jtest can generate test cases for you — automatically. In addition, Jtest can monitor a running application and generate test cases based on your input.

These features were quite useful. Jtest generated test cases that I would never have bothered with. While I would not have written them because I am quite sure I will never change those pieces of code, Jtest does not have my omniscience and correctly decided to test that code anyway. In the next iteration of my application, I will be able to test the existing functionality and confirm that my omniscience did not fail me — or more likely, to correct my inconsistencies.

While Jtest's feature does not replace building complex test cases yourself, it does lighten the load considerably. Being able to get out from under the tons of grunt test cases needed in an enterprise level system allows you to put your attention on writing the truly interesting test cases and confirm higher level functionality.

Test Case Parameterization

Parasoft Documentation: Jtest can parameterize any JUnit test case, then run it with a large number of runtime-generated inputs to emulate unexpected user actions and test the code's response.

In addition to generating test cases, Jtest lets you set up a spreadsheet full of data and then have Jtest run through the test data — doing multiple iterations of the same test using different data.

This feature is quite handy. Being able to set up some worst case data sets and then chew through them is simply sublime. In the case of PDAbs, I already had a large amount of data samples from users and was able to plug in that data and test. As more users discover fun ways to input data, I am sure my lists will grow. Since all of the data is stored in a spreadsheet, it does not require a coding session to update tests as the new data comes in.

Code Review

We all know that code reviews are necessary and should be considered a fundamental part of any development process. Code reviews are also a waste of time; they can turn into either a social hour or bickering session almost instantly. As a result, I have worked for many companies over the years, big and small, and code reviews are few and far between. Either deadlines do not permit them, the political situation is far too tense, or (at the worst) teams do not feel the need for them.

Jtest approached the issue of code reviews in a clean asynchronous way. Reviewers can review the code at any time and flag the issues that they find. Reviewees can then read the notes, make changes, and check the code back into version control. Once the changes are made, the Reviewers are notified, and they can review them again.

In addition to this manual review, Jtest can also be configured to automatically review the code base for coding standards and make notes itself on items that need to be corrected. The combination of these two features makes code reviews useful and painless. It almost makes me look forward to the next one.

Open Source Test

As a final test of this tool, I pointed it towards one of the smaller open source Java development efforts available on sourceforge.net. While I will not mention the name of the project as that is irrelevant to the test (and I don't want to embarrass anybody), it did come up with some interesting information.

First, at least in this project, coding standards were very lax. While Jtest did not identify many "big" issues, there were quite a few little items, such as variables not being used, methods not being used, etc. Also, it seemed that many developers on this project did not agree on some of the basics of keeping code consistent. Variable names, code indentation, and code layout were all over the place. While such things do not make or break a project, they can make maintainability a nightmare.

After testing it against someone else's code, I could definitely see the use of this tool from a team lead's point of view. Being able to look over a large code base and catch the obvious issues automatically is a great boon to any development team. Since Jtest will also track when an issue is changed, it also makes following up on coding issues much easier than having to take notes and then manually follow up with that developer.

Jtest was clearly meant for large teams to work together. While I was able to test some very interesting features, I did not give it the full run that it deserves. Specifically, I was not able to test the manual code review portions nor the automated server testing.

However, what I did work with was impressive. To have Jtest automatically test your entire code base, every night, for coding standards, defects, etc. is priceless. I recommend this product to any team that really wants to do unit testing and code reviews but either does not have the resources or processes in place to handle it. The $3,495 single user price tag is a bit daunting, and for a single developer or small team this might exhaust the budget, but on a large development team it is certainly worth the price.