Automated Regression Testing Dilemma

Nobody likes running long software regression testing cycles. It’s long, hard and boring.

And yet, nobody likes to release bugs either. NPS (Net Promoter Score) drop like a rock and the support phones light up like little annoying Christmas ornaments.

So, just grin and bear it or take the plunge into software test automation?

It depends on where you stand on these two dilemmas.

1. Business Speed Dilemma

This dilemma is about your cost and speed of producing software versus capturing more customers and larger market share before your competitors.

• Adding customers, solving real problems and growing your market share requires more features in your software.
• More software features require more software testing.
• More software testing takes more time, people and delays releasing your new features.

So, faster software production equals more customers as long as you meet their minimum quality requirements. Otherwise, they dump your software or mobile app like yesterday’s leftover tacos.

2. People Versus Machines Dilemma

This dilemma considers that people are more adaptable but more expensive than machines that are more consistent but less expensive.

• People are better at finding defects.
• People are slower and more expensive than machines but require less “programming”.
• Defects found late in the process are very expensive to fix.
• Machines are better are consistently repeating sets of activities.
• Machines are slower and less expensive than people but require more “programming”.
• Defects found early in the process are cheaper to fix.

So, how to resolve your dilemma? My friend and business mentor, Keith Cunningham says

One choice is slavery. Two choices is a dilemma. Three choices is freedom.

Hybrid Software Regression Testing That Turns a Dilemma into Freedom

Consider a hybrid approach to software regression testing.

Use people for manual regression for all features in the current and previous version.

New(ish) features are where most of the defects lie in hiding. This leverages the human advantage of finding defects in new features.
This also minimizes the amount of time & money spent on people in the regression testing cycles. You will have the benefit of faster and cheaper software regression testing cycles.

Use Test Automation for regression testing all features released prior to the previous version.

Older features are where most software developers do damage. This leverages the machine advantage of consistently repeating activities. This also minimizes the amount of time between discovering a defect and fixing the defect – which dramatically reduces the cost of fixing the defect.

You and your developers will have the benefit of real time testing so that regression defects (where new features break older existing features) are much cheaper to fix.

The Drawbacks to Hybrid Software Regression Testing

1. You may need to add Test Engineers to your software testing team. Successful test automation will require written test cases that can be easily automated. Test Engineers know how to write these types of test cases.
2.  You will have to decide on which approach to use for Test Automation – Record/Play, Scripted or built-in APIs. You’ll need an experienced Test Automation Engineer to help you determine the best path for you and your business.

At TESTCo, we can help with both – call and let’s talk about how to implement or improve your automated regression testing.

Download our report that looks at the challenges and pitfalls of creating automated regression tests in an Agile environment.