Why You Should Treat Tests as Products
In continuous pipelines—for continuous integration, deployment, or testing—many tests are not regarded as separate products. They’re bound to the application and the pipeline that creates and maintains it.
An alternative approach for at least some of the tests is to regard them as products—project deliverables in their own right, created as a business investment. "Productizing" a test means taking it and making a deliverable in the product backlog that a team needs to commit to as part of a sprint, or that is produced independently outside the sprints.
Treating tests as products can show their value to management, but more importantly, it can help them contribute more effectively to the development lifecycle.
To identify and plan for productizing tests, consider these four steps.
1. Organize tests with high-level test design
A good structure for the tests can provide a basis for productizing. Organize tests in modules, with each having a clear, well-differentiated, unambiguous scope, and use keywords to define specific actions. Structuring tests in a certain way can also help you achieve overall goals in the development lifecycle. For example, having tests well positioned can achieve the elusive in-sprint automation.
2. Define which tests to productize
When test modules have been identified, determine which are useful to view as products. These are some criteria to consider:
- If a test is large and complex and takes a substantial investment of time to create
- If the test works with functionalities that are not developed in an individual sprint, like an end-to-end test
- When tests are expected to have a long life, which tests are reusable when the application is maintained
A test that consists of business scenarios will be more of a candidate for a product-oriented treatment than tests that are close to the application and sensitive to application maintenance, like UI tests. Such tests are better to be updated along with the application and executed when source files are changed as part of the continuous process.
3. Determine when to develop
There are usually two main options for when to develop tests: as part of a sprint or in their own lifecycle. Tests that are close to the application fare well in a sprint, such as in a TDD or ATDD approach. Tests that are at a higher level, like end-to-end tests, usually don’t fit an individual sprint.
By organizing tests in modules and using action keywords, such tests can be developed well in advance, before sprints need them. Automation can then focus on implementing the actions, which will be an effort best done in an application development sprint.
4. Fit in the automation
Automation development is a different trade from test development. There are deliverables, like action implementations or Page Objects, but most of the time teams will work on them inside sprints, in sync with the application development, and not manage them as products.
Treating tests as products can establish their value as an asset in addition to the value of the application they’re testing. It can also help improve the efficiency and quality of the overall development and release lifecycles.
