Posts

Outsource Software Testing

Outsource Software Testing – Painful Lessons

/in /by

This is the true story of a real estate company that learned about promises and quality the hard way. They learned that it’s not always a good decision to just trust a third-party software developer to deliver the quality they promised. They also learned the value of outsource software testing when trying to get a mission critical software project back on track.

One of TESTCo’s real estate clients – a company that shall go nameless for reasons you’ll soon understand – was trying to develop the ultimate software for distributing and tracking available properties in their portfolio. By ignoring the need to outsource software testing, they quickly found themselves in a pinch.

The company’s old application was cumbersome, slow to load new properties, and a giant hassle in terms of administration, updates and changes. Their clients, the agents within the brokerage, had numerous requests for new features that would help improve sales and revenue. They wanted better information and more insight into the data they were collecting.

Naturally, they got really excited when a real estate software development team they found promised them the moon. The developers said all the right things, ensuring easier data loading, improved content management and a faster path to sales nirvana. What they forgot to consider was the quality of the software delivered and the need to outsource software testing for a mission critical business application. 

Download Case Study PDF

 

Well, Not So Fast

When development ended everyone was all hyped up to get the new application into– the hands of the sales people, but there were a number of glaring issues that raised questions about the project that we can all learn from.

  1. When does regression testing of the outsourced development become a mission-critical activity?
  2. When is the right time for Outsourced Software Testing?
  3. What happens when outsourced development goes wrong?
  4. What is the best Outsourced Software Testing ROI?

For starters, the outsourced developers only tested new features they designed and were hot for. The developers also failed to perform regression testing or thought they didn’t need to. When you outsource software development, regression testing is typically the most important part of delivering high-quality software.

The real estate company had no experience with outsourced software development. Their core business is moving properties, not batches of code. They relied on the advice of the outsourced software development shop and misguidedly cut some corners by doing their own internal QA testing instead of using professional software testing. In the real estate company’s defense, the outsourced software consultants didn’t offer a professional, third-party outsource testing option. The software developer thought they tested it well enough internally – and this almost never happens! With no prior software development experience, the real estate company didn’t even know it was on the table as an option. And, the contract terms were loose enough to allow the development shop to meet deadlines with low quality deliverables.

As you can guess, when the real estate software project got closer to completion, all kinds of defects started showing up. Features that previously worked in the old software were showing up broken in the new system! The trial and beta launches were a complete disaster.

Correcting Course with Counsel from an Outsourced Software Testing Company, TESTCo

When things were looking really bleak, the real estate company decided to correct course and hire TESTCo for their outsourced software testing to see exactly what was going on.

TESTCo designed a test strategy focused on achieving high regression testing coverage in a short period of time (to match their 2-week release cycle). We brought in a Test Engineer  experienced with exploratory testing strategies, and he built an automated ad-hoc testing program that looked for defects without a specific set of test cases. We also added a software QA track to the project plan and implemented burst-mode regression testing so sprints were not delayed, which spot checked and discovered regression defects during the new feature testing and development processes.

Abandon Ship!

With thorough testing and QA procedures under way, the real estate company began to assess the risks involved with completing the project and waiting for the software developer to fix the problems.

They came to a somewhat brutal conclusion. They wouldn’t be able to complete the project. They’d have to abandon ship, cancel the project without throwing more money away, and retreat to the old system in order to claw back some of its functionality. They ended up modifying the old system, testing it with TESTCo outsource software testing services, and relaunching that solution on time without any defects within three months.

Why It can Make Sense to Hire an Outsource Software Testing Firm Before Starting Development

It was an unfortunate lesson, but at least they got back to a working system that could handle some basic property listing tasks. Sadly, this happens often in the software development world. Companies with no software design and development experience hire outside developers that can’t deliver on their promises. They get oversold and then retreat, licking their wounds, discouraged by shattered technology dreams.

The moral of the story? Hire a solid outsource testing company like TESTCo prior to launching a mission-critical software development project. We can act as consultants and help you vet developers while designing test plans that help guide the project to success.

Can we make a difference for you? Read what our clients have to say.

Download Case Study PDF

Consider the Cost

There is another reason to think carefully about how you design,build and test your software.

When a mistake, defect or error is discovered during the Design Phase of your software project, the cost to fix it is simply a little time discussing the true need and then changing the design documentation.

When a defect is noticed during the Building Phase of your software project, the cost to fix it is larger – close to 10X greater than the cost to discover it in the Design Phase. Why? A mistake in the Build Phase will require significant time and effort for the Design Team to validate the requirement and then the Build Team to go back, re-write the offending code (and any other closely related code) and then test it again. This is far greater cost than during the Design Phase where the changes were only to the requirements.

When a defect is discovered during the Test Phase of your software project, the cost is even larger – again, close to 10X greater than the cost to discover it during the Build Phase. Why? A defect discovered during the Test Phase already has countless hours of engineering time to build the feature. Additionally, the defect has cost you Testing time as well. Now, add to that, the need to re-validate the requirement, confirm that the test case and defect is accurate and then get it rebuilt by the Build Team (along with any other affected or related code because defects seldom happen by themselves) and then re-tested by the Test Team.

Lastly, when a defect is discovered by one of your customers after it has been launched, the cost is approximately 10X greater than the cost to discover it during the Test Phase. If you have a defect in your production software, it will incur all of the previous costs to find, validate and fix. It will also probably cost you a customer or prospect if they can not complete their work with your software.

Defects can be Very Expensive. Testing Early and Extra is almost always an investment with a generous return. Testing with an outsource software testing specialist like TESTCo can decrease your costs and amplify your returns. I look forward to speaking with you.

Offshore Software Testing Company Helps BetweenMarkets Deliver On Time

/in , /by

Why would a company with good internal software testing capabilities decide to hire an offshore software testing company?

BetweenMarkets, an enterprise software company specializing in distributed order fulfillment, was recognized as one of the premier development organizations in Austin, Texas. Although BetweenMarkets had released their first product with zero customer reported defects, they quickly realized that their software testing process – which consisted of a combination of in-house testing and expensive consultants – would not scale as they began the build out of their second release.

Problem

BetweenMarkets was preparing to release a new product version to market. Although the product had been successful in production environments, the management team was worried about the changing dynamics of their quality processes. As implementation of their second product release began, it became obvious that their existing software testing processes would not be effective going forward. “We had tremendous success in the quality of our first release. However, as the size and complexity of our application grew, there was just too much ground to cover,” said Erik Huddleston, CTO at BetweenMarkets. “There was no way we could cost effectively staff and train the people we needed to maintain our rigorous quality standards. Not only was there a tremendous amount of code to test, but there was also a wide variety of skills needed to effectively analyze the quality of our application and diagnose problems.”

Decision to Use An Offshore Software Testing Company

BetweenMarkets decided to turn to an  offshore software testing company in order to achieve the scalability, flexibility, and diversity of skills that were required. After interviewing and accepting bids from over a dozen offshore software testing firms, BetweenMarkets selected TESTCo. “TESTCo’s Engineered Assurance technology made it the clear choice,” Huddleston said. “Engineered Assurance, coupled with some great overseas relationships, gave us the real-time visibility and predictability that no other firm could deliver.”

Solution

At the end of each workday, a build is produced with a list of new features implemented and defects resolved. A TESTCo project manager, aided by Engineered Assurance technology, then analyzes the work and formulates a test strategy covering the next 12 hours. A team of specialists is assembled whose skills exactly match the night’s challenges. By morning, when BetweenMarkets’ development team returns to work, their issue-tracking system is fully populated with the results of the preceding day’s code analysis. BetweenMarkets immediately knows of any problems that were introduced into the product within a day of their creation. At the same time, BetweenMarkets’ management team has a full online report of the preceding day’s activities as well as analysis and recommendations going forward, all because of offshore software testing through TESTCo.

Benefits of Offshore Software Testing

In addition to enabling a new, more powerful development process, TESTCo offshore software testing provided BetweenMarkets with a number of tangible benefits. By offering a flexible usage model for testing services and a reduced per-head cost, BetweenMarkets was able to reduce its software QA testing related expenses by 50%. “The types of outsourced / offshore software testing skills you need vary over the course of a release as does the number of resources required. TESTCo was able to flexibly allocate the right people at the right time to keep the quality of our product high, and the cost of the effort low,” said Huddleston. Since BetweenMarkets is able to weave software testing into the daily development process, they are able to dramatically reduce the length of their release cycle. “TESTCo has helped enable a real-time, iterative development process that has had a dramatic impact on our time-to-market,” Huddleston said. “The introduction of real-time software testing decreased release cycle times by 20% and increased the defect discovery rate by 400%. TESTCo used their Engineered Assurance technology to provide us with accurate defect discovery within a single build, so that by the next build, the defect could be repaired. It’s hard to quantify the value of fixing bugs that are fresh on the developer’s mind instead of months later before a release, but it is significant,” said Huddleston. “It has had a tremendous impact on the quality of the code we produce.”