Software load testing and software performance testing are used interchangeably and are sometimes confused. Probably because there is no clear definition. Even the Wikipedia entry is not clear! In most cases, these terms are used to describe an approach of using software test automation to determine when a software application will begin to suffer and perform poorly under increasing numbers of users or transactions.
In an effort to simplify a complex topic, we distinguish software load testing from software performance testing based on the type of question to be answered or risk to be mitigated.
Two Testing Methods for Two Questions
Load Testing answers the question “At what point does my application become too slow to be usable?”
Performance Testing answers the question “Where are the bottlenecks in my software application?”
We make this distinction because most Load Testing problems can be easily solved by simply adding more computing resources – bandwidth, CPU, etc. This is largely based on the fact that most software today is cloud-based and provisioning more computing resources is a simple “flick of the switch”.
Outsourcing is Usually the Solution
Load and performance testing are niche areas in software QA. They require many years of training and expertise to do right and do well. Because of the scarcity of talent to implement successful load and performance testing project, software QA outsourcing is a very good approach to solve this scarcity of talent and insure project success.
Most Performance Testing problems tend to be much more thorny, detailed and hard to clearly distinguish. They are also generally harder to solve once discovered. But, there are still many, many non-cloud-based software applications (remember Enterprise Software?) still in production and daily use today.
Performance problems in Enterprise-class software tend to be more complex, require detailed engineering analysis and can seldom be solved by just adding additional computing resources. In fact, a lot of performance testing issues occur and can be solved with a slight change of the configuration of one of the system components – think adjusting the count or size of a database connection pool.
Building automated performance tests looks very much like building automated load tests but the outputs from those tests are designed to answer very different questions.
We’ve learned to spend a little more time asking questions about the specific questions, answers and business case for automated load and performance testing so that solve the right problem and answer the right question the first time.
