Building automated performance tests looks very much like building automated load tests but the outputs from those tests are designed to answer very different questions.
We’ve learned to spend a little more time asking questions about the specific questions, answers and business case for automated load and performance testing so that we solve the right problem and answer the right question the first time.
Here’s why knowing the right question is important! Answering a business risk question about when a website or web application becomes unusable is much less expensive than determining all of the bottleneck places within a website or web application. Not only is the process more expensive, the fixes for performance based bottlenecks is also expensive. Finding the maximum usable load and then adjusting computing hardware to increase that load is much less expensive.
In closing, a note about software testing automation and qa outsourcing. In general, you should expect that an automated software testing project will look, cost and behave more like a regular software project than a software QA project. That’s because software test automation is actually writing software to test software.
A software test automation project will also require maintenance after it is completed. The software under test will change and no matter how cleverly the software test automation was built, it will need to be updated when the software under test is changed. This will be an ongoing cost of quality and should be considered in the business case prior to starting this type of project. Here is more information about maintenance costs of automatic software testing.