The Value of Testing Simply
Automated testing is valuable but often hard to do well. You can’t test everything, and even if you could, there is a cost to writing and executing the test.
While testing makes agile software development possible by providing the feedback we get from the practices of continuous integration and continuous deployment, too many low-value tests can slow integration time, lengthening the feedback cycle that makes agile work. Finding the balance can be a challenge.
Among the metrics people obsess about are the number of tests and test coverage. However, tests that cover more code don’t always improve quality. Some tests have low value and thus, implicitly, high cost.
The classic example of a low-value test is a unit test that tests a trivial setter or accessor. If a setter fails to set a value, there is a problem with the underlying language, or runtime. The role of a unit test is to verify that your application code is performing correctly, not to test the contracts that external systems provide.
And when it comes to testing, the line between trivial and valuable can be a fine one. If you err too much on the side of seemingly complex testing, you may lose opportunities to find and fix problems early.
There is a class of seemingly really dumb yet valuable tests that you can write if you are willing to observe when costly yet easily fixable problems occur. Validating that configuration files are syntactically correct and that the code in your persistence layer is mapping all the essential fields in the data store seem like the kinds of errors one could catch by inspection, but we don’t always do so.
Consider a system that performs some complicated task based on the contents of a human-written XML configuration file. An integration test will catch the syntax error when you launch the application, but by that point you are at a later stage in your integration and deployment pipeline. You could write a unit test that validates the XML to a schema to catch the most obvious errors before the code is committed to the repository. While this isn’t a perfect test (there are still logic errors it will miss), it’s a test that will save a significant amount a time and is relatively simple to write and execute.
Configuration failures are a frequent cause of problems, and while taking a “fail fast” approach in the application can mitigate problems, any validation you can do as part of your build pipeline helps you fix the problems earlier and at lower cost. Some research even shows that simple testing using validation and static analysis can prevent most critical failures.
There are many tests you might not realize you need until you discover errors later on that have things like typos as their root causes. Simple tests may not seem impressive at first glance. But the goal of testing is to ensure quality, and simple tests can help you get there faster.
