A wise old programmer once advised me that code isn't right until it's been written three times. The first time, he said, you're just trying to see if you can make the idea work in any way — a proof of concept. The second iteration is to make the code work. Finally, the third time you write the code, it's to make it work right.

And that, opined my friend, is why the best version of any software is version 3.1.

You're probably familiar with the concept of refactoring, even if you didn't have a name for it. By its Wikipedia definition, refactoring is the process of rewriting a computer program or other material to improve its structure or readability, while explicitly preserving its meaning or behavior. Or to put it another way: "You refactor your code to make a working product great," explains Jared Richardson, author of A Practical Guide to Successful Software Projects.

You'd think that's what most people want. However, according to Evans Data's latest survey of North American developers, refactoring is at the bottom of the pile. Only 22% of developers rank refactoring as "highly important" among their developer tools. I wondered: why? Are there technical reasons? philosophical reasons? simply an issue of education? (i.e. "what the heck is it, and why should I care?") So I approached a bunch of agile development experts to learn more.

It's clear that the problem isn't "I don't know what it is" (though "I misunderstand what it is" is an element) but "I don't have/can't get the company resources or buy-in to do it."

When dealing with properly refactored code developers can add new features more quickly, fix customer complaints faster, and the product tends to run faster. But, cautions Richardson, these are all intangible benefits with a difficult ROI to measure — particularly in a shop where short-term thinking wins. "Most managers want new products with a measurable impact on sales not a cleaner, more supportable product. Support costs tend to drop into the background and only new development costs are measure. This can be a death knell for practices like refactoring in a short-term shop," he says. "Instead of selling the technical teams on refactoring, we need a concerted effort to sell technical managers on the benefits of refactoring."

Developer Christopher McMahon prepared a list, citing the reasons he thinks most developers avoid refactoring. Some of them are rather cynical:

• The code works, so we don't invest in making it better.
• The code works, and the risk of making a change outweighs the benefit of making the code better.
• The code works, and I'm the only one who knows why, and if I make it better then other people will take my job.
• The process I have to undergo to make a change (approvals, signoffs, etc.) is so onerous that it's not worth going through.
• The design process (typically top-down rather than bottom-up) makes it difficult to rework existing code.
• The development process (different people or different teams working independently in the same system) makes it difficult to rework existing code.

Education — or at least understanding the basics — is an issue. Says Richardson, "Refactoring code isn't a license to go hacking around in a working product or to rewrite someone else's code. It's a set of practices, including a solid set of automated tests, that are used to solve real business problems."

But the theory isn't the major problem. "You can't refactor if you don't have a suite of automated unit tests, and many people are lacking that," says Lisa Crispin, co-author of Testing Extreme Programming. "Maybe that means the real question is, 'Why don't more people automate unit tests?' since they have so many benefits — aiding in design, promoting quality, allowing refactoring with further promotes quality... I think there are many answers to that. One is that unit testing is hard to learn and significant resources (time and training) are needed to get started. Another is that programmers 'get paid to write production code' and some managers don't feel tests are production code. I'm sure there are many more."

Other developers point out the perception that "refactoring is risky," because you can break code that already works. If you have a large product and want to refactor a basic building block, those developers believe, it may take a huge level of effort to retest all that stuff and know you didn't break anything. Hence the philosophy: if it works don't touch it!

Yet, agile developers suggest that the opposite is true: not refactoring is risky. According to one discussion participant, "Code bloat, high coupling, high duplication, etc. are not good for maintaining a system in a bug-free way.

"Once you've experienced proper refactoring, along with a solid set of automated tests verifying your work, you'll find it's a very safe and solid way to move your product forward. In fact, once you've experienced proper refactoring, you may want to use the same practices on your day-to-day 'normal' development," says Richardson.

For more discussion on this topic, visit the Agile Testing group where the original thread took place.