The Value of Developer Testing in Software
Developer testing (unit tests, for example) is a great way to improve the quality of your code and deliver more reliably. It’s tempting to say that every line of code should have a test associated with it, but testing involves tradeoffs. Some tests are easy to write while some are harder.
All testing has a cost, both in terms of time spent writing the test and in execution time, which can translate into longer build time and a longer interval between commits. Ideally, you want to write low-cost, high-value tests and manage risk by skipping the low-value tests. Unfortunately, it’s difficult to know which tests are which when you are coding.
As Elisabeth Hendrickson points out, testing does not have value in itself. Testing has value only to the extent that it helps you deliver more customer value. She points out how more testing done incorrectly can decrease value.
Business value needs to be at the core of everything you do in an agile project. This is true of the infrastructure agile teams take for granted, and it is also true of testing. Testing is a bit different from, say, setting up a continuous integration server. Rather than being a one-time task, it’s an ongoing effort, and the effort of testing affects delivery time (a cost) and also affects the risk in the future (a benefit). Stakeholders may need to be convinced that testing is part of delivering value. For more on this subject, Linda Hayes explains how to sell testing to customers.
It can be tricky to get teams to adopt a mindset that considers testing practices essential to their daily work. If you have seen the (good) results of using testing practices such as test-driven development (TDD), it’s tempting to advocate that practice for your team. Jeff Atwood explains why any test is better than no test and that trying to force a team to fully embrace TDD can be counterproductive.
Even if you are committed to testing and TDD as part of your lifecycle, you have to understand that you are using TDD as a means to an end not as an end unto itself. As Michael Feathers discusses, issues such as design and language granularity come into play when you are making changes, and while TDD is helpful, TDD by itself will not ensure quality.
People struggle with whether things are worth testing. Until you have experience, it’s to easy to dismiss a test case as “too simple to cause problems.” Likewise if you’ve adopted testing too enthusiastically, you can spend a lot of time automating a rare, low-risk case that is complicated to automate but easy to test manually. Often it’s better to start with a default position of writing tests. You should get into the habit of justifying why a test does not add value when you decide to skip a test.
Our intuitions about the relative value of testing are often wrong. Only by gaining experience writing tests will you be able to learn which tests really add value and which might be worth skipping.
Does your team write tests regularly? Has it helped reduce time to fix bugs? Has writing tests diligently caused problems with your ability to deliver?
