Performance Testing: A Team Sport
Performance testing is fundamentally different from functional testing. In functional testing you design test cases to address specific issues and risk inherent to the product being tested, focusing on both positive (does it work?) and negative (locating defects) aspects of the product.
When executing a test case you have an expected result in mind, and each time you run the test, you expect the same result. Functional testing is about what the system does.
Performance testing is not about test cases, and we don’t have specific expected results in mind when designing a test. A performance test is the search for potential problems within the overall system, either in the software or in the environment as utilized by the software.
Performance testing is not necessarily concerned with the functionality of the software, but rather with how the software consumes system resources and where and when that consumption reaches a point that it causes problems. Performance testing is about how the system behaves, not what it does. However, to performance test a system or application, we do require that the functions be stable and reliable.
The need for technical cooperation usually is much greater for performance tests than for traditional functional testing. Many of the issues encountered in performance testing will require technical expertise to identify and correct.
To be successful, a performance test must start at the requirements, just as functional testing does, but the performance test requires a team effort. When putting together a performance test, the first task is to identify all the key players that need to be in place to ensure the success of the test.
Key players involved early in planning the performance test include:
- Test/QA
- Database administrator
- System administrator
- Network administrator
- Users/customers
- Security administrator
- Developers/technicians
- Management/marketing
- Capacity planners, operations staff, etc.
Putting together a proper performance test team is essential to success. The performance tester is dependent on people such as database administrators, system administrators, and network engineers to install probes, provide understanding of technical architecture, help interpret the performance test data collected, and tune and debug the system.
Involving all the key players early in the process is critical. Many of the issues related to performance can be identified early through static methods, often referred to as performance engineering. If the key technical players are continuously involved in the testing process, many conflicts later on can be avoided.
When problems are discovered, there is less resistance to accepting them as problems, as they are not just the results of testing, but of collaboration between all members of the team. This also lessens the feeling of blame that sometimes occurs when problems are identified.
As testers, we want to expend as much effort early in the process as possible. A well-planned and well-designed test requires combining the knowledge and skills of all the key players, makes test execution far simpler, and reduces overall costs.
Done well, many of the issues relating to performance can be addressed early in the development process. By doing this, we eliminate many of the problems associated with finding performance-related issues late in the process, when they are more expensive to correct.
