Early Testing Questions for Mobile Apps
Assuming one has considered test strategy and done some high-level test verification and validation (V&V) planning, a mobile app tester should take early actions to gain a better understanding of the product in order to answer the following questions:
- What test/V&V actions has the developer taken?
- Has the developer implemented static code-analysis testing?
- What risks exist for the product?
- Does the team communicate well and understand what software development/test actions are in play?
For the first question, I consider testing/V&V integral to what many app developers do. Programmers do things like unit testing or test-driven development. They participate in code reviews or pair programming. They conduct activities like modeling, design, or analysis, which can remove some error sources for software bugs.
Better programmers will practice more of these, but some development staffs still think testing/V&V is the job of the designated quality assurance people. This raises potential problems the test team must add to their risk list.
The second question focuses on a developer action that may be done by the test group. For years now there has been a class of analysis tools that searches for potential errors in the app code. I think of these as compilers on steroids, but they are more commonly called code static-analysis tools. Ideally, developers should use these tools as they code and before they baseline their code.
However, I’ve worked alongside many teams with strict time constraints, so I understand that programmers do not always use these tools properly—if at all. If the answer to this question is “No, but the team does have access to such tools,” then the testers can take over the analysis job and become the first-pass filter for errors. This moves testing to the left, provides early feedback to developers, and finds many errors that are hard to find by traditional testing.
Defining risks in apps means you must know things such as why the software has been created to solve a problem (this may be business rules, requirements, marketing ideas, etc.); when the software must be done (time to market, what parts of the software are needed when, etc.); who is creating the software (what skills they have, what knowledge they have, what bug history they have, etc.); what approaches are being used in creating the software (lifecycles, agile vs. traditional methods, vendors of third-party software, outsourcing of code elements, etc.); and how much money is at stake (to create the software, that can be lost by the software, etc.).
Finally, a concern I always have is communication within the development-test team. One ideal in agile is communication that is quick and meaningful. Mobile is fast. Mobile app communication often will be direct to the stakeholder, with minimal formal documentation and few or no formal written documents. If testers detect an issue or risk, this should be brought forth in a daily meeting or to stakeholders in real time.
There are other questions to ask before classical testing really starts, but if you can get good answers to these questions, your mobile project stands a better chance of success.
Jon Hagar is presenting the session Top Ten Attacks to Break Mobile Apps at STARWEST, in Anaheim, CA, October 12–17, 2014.
