Is Testing Just Looking for Bugs? A Slack Takeover with Janna Loeffler
Thought leaders throughout the software community are taking over the TechWell Hub for a day to introduce themselves, answer questions, and engage in conversations.
Janna Loeffler has more than fifteen years of software quality experience working in a variety of software engineering roles, including development, testing, quality assurance, and DevOps. Loeffler presided over our most recent Slack takeover, which led to some insightful discussions.
What Is Testing?
“By ‘testing’ do you mean ‘looking for bugs?’” — @Matt Griscom
Testing means different things to different people, especially as software and technology evolve. @JannaLoeffler likes to use a definition from Michael Bolton and James Bach’s Rapid Software Testing class: "Testing is the process of evaluating a product by learning about it through exploration and experimentation."
There’s an active discussion in the software community about the difference between testing and checking, and Loeffler suggests testing is the exploration, thought processes, and experimentation you go through in order to create test cases.
Automating Our Tests
“What and how much should we automate?” — @KuriousTester
Many testers agree that we shouldn’t automate 100 percent of our tests, and that automation decisions depend on your organization, product, team, and customer.
Angie Jones and Dot Graham developed a spreadsheet to help software professionals work through these decisions. Loeffler said she likes the spreadsheet because it takes into account some key factors:
- Your gut feeling on whether something should be automated
- The impact on the customer if the feature/scenario breaks
- How easy is it to write and maintain
- How likely it is for a broken feature/scenario to be fixed
- Historical information, such as whether something breaks often or if similar things break
Using the Test Pyramid
“What are your thoughts on the test pyramid? How can a tester be a responsible tester using that toolbox?” — @KuriousTester
The test pyramid is a tool software professionals can use to think about different kinds of automated tests and how they can create a balanced testing portfolio.
@GriffinJones explains that the point of the test pyramid is to focus most of your testing on simple checks, not end-to-end, system-level tests. Jones also argues that the pyramid is meant to be suggestive and not prescriptive, because while it is accurate, it is not intended to be precise.
Loeffer says the biggest way to use the test pyramid is to simply be aware of it. Many companies and leaders are familiar with the test pyramid, so knowing it will help you speak responsibly about its pros and cons.
