In order to make software more secure, the industry must get on the bandwagon for code scanning tools, incorporating them into the daily development cycle.

But are the tools currently sturdy enough to stand up to immense code loads?

Scanning tools are in fact getting better.

They're scaling better, for one thing.

They're able to run on parallel machines, which means they can handle much bigger code loads and get results to developers in a reasonable amount of time—in other words, before the code in question has been revised two or three times as testing drags on.

One remaining problem, though, is there's still nobody telling us which of the code tools are any good or what a given tool is specifically good at.

At this point, the work is hard, and there's just no independent body out there that can point you to the tool that will best fit your needs.

"It's kind of arduous," Oracle's Mark Fallon, senior release manager of software development, said in a chat we had before a presentation he's gave on the subject at the RSA Conference on Feb. 15.

"At the moment there are no publicly available benchmarks. There isn't a good body of knowledge [from which] to say 'It's these guys over these guys,' 'It's these guys for this particular area,' so you have to go through and do the evaluation yourself. That's fine if you have 100 lines of code. We have 50 million lines of code."

That's a huge body of code with extremely complex paths wending through it. Oracle and companies with comparatively unwieldy code sets at this point have to bring the code in, get the code scanning tool working, make sure it can scan the massive body of code, and then evaluate its results to make sure that they're real and not false positives.

Click here to read about the free debugger EnterpriseDB shipped with an update to its PostgreSQL-based database.

"With any scanner company we've worked with, we've gone through iterations of where their tool couldn't handle our code, and we've worked with them" to fine-tune the tool's ability to churn through the code set, Fallon told me.

That's why, for example, Oracle, based in Redwood Shores, Calif., worked with Fortify for a year before signing on the bottom line to use its tool. During a year of tweaking, Fortify came in to Oracle repeatedly as the developers put their heads together to optimize results.

The Fortify deal was part of Oracle's ongoing effort to knit volume code testing into its development DNA. In December, Oracle announced that it would use static code analysis technology from Fortify to hunt for bugs in C, C++, PL/SQL and Java as part of a program to improve checking for security holes during development, instead of trying to patch holes after the product's out the door.

The Fortify tool had to stand up to a brutal load: Oracle's database alone contains between 40 million to 50 million lines of code. The tool had to scale to spit out results in a reasonable amount of time and be able to work on parallel machines.

"We want to get an answer in a day, not find out that two or three people have modified the product" while it's dragged through testing, Fallon said at the time.

Next Page: How to use a code scanner effectively.