Mobile Application Testing Services Choices

How Much Testing Does Your Mobile Application Need?

/in /by

Mobile applications are different than web or installed applications and deserve some careful thought to make the best choices about what and how much mobile application testing services are really needed.

There is a long list to consider and surprisingly, you probably don’t need all of those mobile app testing services to achieve your goal.

Mobile application testing services can include:

  • Mobile application functional testing
  • Mobile application regression testing
  • Mobile application cross device testing
  • Mobile application test automation
  • Mobile application security and penetration testing
  • Mobile application performance testing
  • Mobile application usability testing

Goals for Mobile Application Testing Services

Here are some questions you can consider to help you determine if one of the types of mobile application testing services I listed will help you meet your needs and achieve your goal.

First, what is your goal for your mobile application?

  • How many downloads or users do you want to use it?
  • What value will your users receive from using your mobile application?
  • How will you know they have received that value?

Now that you have a basic goal for your mobile application, see if the following questions help you chose the best and most appropriate mobile application testing services.

Mobile Application Functional Testing

Does you mobile application contain complex or simple functionality?

Simple functionality with only a few inputs/outputs/screens usually doesn’t need anything other than some basic validation that the inputs/outputs and screens work and flow properly. There are few inputs/outputs/screens so the number of combinations to test will be limited and can be accomplished relatively quickly.

Complex functionality where you have more than just a few inputs, outputs and screens probably needs (and deserves) a more robust testing strategy and plan. In general, you can’t test everything (it takes too long and costs too much) so selecting a smart test strategy will be very important to make sure that all of the “Important Stuff” is tested thoroughly and frequently. Almost any qualified Test Engineer or QA Manager will have a small library of Test Strategies and can help you select the one or two that most closely match your needs and goals.

Mobile Application Regression Testing

Regression Testing helps you make sure that you didn’t break anything that was previously working well. The main goal of regression testing is to insure that you don’t disappoint or anger your mobile application users instead of delighting them with new features and functions.

Mobile applications tend to be updated frequently. One of the benefits of a mobile application is that they tend to be smaller and simpler than other types of applications and can be updated very quickly and easily. This is a tremendous benefit because it allows you to quickly add or change features based on real-time user feedback.

But, being very quick to make changes to your mobile application may also hurt you if you don’t take just a little extra time to complete an effective regression test.

Regression Testing Can Be Challenging

  1. Familiarity breeds contempt. When you’ve seen and tested your mobile applications features and functions every day for the past six months, it’s easy to let your assumptions and urgency take control. This can result in skipping or overlooking some functionality in your haste to release your new mobile application. Using an experienced and trusted third party for your regression testing will insure that nothing is skipped (or assumed to be OK) during your regression testing. “Once over lightly and quickly” is not a good regression testing strategy!
  2. Lack of test documentation. Mobile applications tend to be smaller than web applications and good test documentation is usually the first casualty. And no, your mobile application doesn’t necessarily need a full set of ISO compliant test documentation. But, a decent list of test scenarios and use cases can help you complete your regression testing quicker and with a much higher (and measurable) level of quality and customer delight. An experienced third party software test engineer can quickly and easily help you build and maintain these test documents as a natural part of helping you regression test your mobile application.

Regression testing is simple but not always easy if you aren’t an experienced software test engineer. But, getting help from a professional, at least the first couple times, can go a long way to helping you release your mobile application faster, better and cheaper.

Mobile Application Cross Device Testing

Your mobile application might be used on a new iPhone or an Android phone several years old. Almost all mobile devices behave slightly differently and the difference in behaviors between iOS and Android can be significant. If you fail to test your mobile application on a variety of mobile devices, you are inviting a catastrophe.

Cross device testing helps your mobile device safeguard against the following.

Customer loss: Cross testing is vital to ensuring quality assurance across all platforms that your application is on. If everyone was using the same device with default graphics, this wouldn’t be an issue. Users today use a variety of devices across several platforms; cross testing across platforms ensures users from all ends have the same experience. If the user interface on one platform performs poorly, customers will likely leave the app quickly.

Issues that arise from upgrades: Mobile applications are updated very frequently and these updates will eventually cause several bugs. Cross device testing will help you identify bugs and how to deal with them. Failure to recognize this might cause significant bugs that can cause your application to be unusable until it is resolved. This can also lead to compatibility issues after design changes as well.

SEO and additional rankings from Google: For mobile applications, ranking on search engines is crucial to gain traction to your site and mobile app. Since 2015, Google has taken a mobile-first indexing approach. This means that the algorithm for Google search results takes cross device compatibility into account and ranks accordingly. If a competitor to your app is compatible with all platforms and devices, they will significantly outrank you.

There are two basic methods available for cross device testing. You should consider the pros and cons of both before specifying what mobile application testing services to obtain.

Real Mobile Devices and Operating Systems

Real devices provide the best mobile application testing results since you are testing your application on the actual device. But, Real Mobile Devices are expensive to acquire and maintain. Real Mobile Devices are also slightly more time consuming to test on as they require physical handling. Finding a sufficient variety and availability of Real Mobile Devices is a big challenge as new devices and mobile operating systems are released and updated frequently – and this makes the process of building and maintaining a library of mobile devices very expensive.

Mobile Device Emulators

Emulators are usually on-line services where you can load your mobile application on a variety of specific virtual or emulated mobile devices with specific operating systems and versions. Mobile Device Emulators are very handy but have some distinct challenges. Mobile Device Emulators work well for basic functionality but are seldom useful if hardware functionality testing (GPS, Cellular Vs Wifi Networks or Device rotation) is needed to test your mobile application.

Also, mobile device emulators can’t emulate the performance of an app when it receives incoming calls, text messages or other notifications. This is a crucial issue because everyone with mobile devices gets numerous notifications every hour. If an app has to crash each time a notification occurs, this would make it unusable. Additionally, mobile device emulators can’t simulate other aspects like battery issues, touchscreen issues and more. These are some of the most fundamental aspects of using a mobile device.

But, We Think There is a Better Way

The TESTCo Blended XDT (Cross Device Testing) – at TESTCo, we take a blended approach to using Real Mobile Devices and Mobile Device Emulators. We will help you analyze your expected customer device usage and the current market share of each mobile device and operating system and produce a matrix that compares your testing needs against Real and Emulated devices. We use this to build a testing strategy that optimizes for primarily Real Mobile Device Testing with a portion of Emulator Testing so that we can give you the highest confidence that your mobile application will work flawlessly on the devices that your customers use.

The TESTCo Blended approach takes advantage of the benefits of both real mobile devices and emulators while covering the blind spots of each approach. This gives your mobile application testing a comprehensive approach that would simulate what would happen to your application in the real world. This ensures that your mobile application will work seamlessly during its first encounter with users doing normal functions on their phone like using it with low battery, taking calls while using your app and more.

Mobile Application Test Automation

Using test automation for your mobile application can allow your software development team to build and release your software quicker. But, is development speed really a problem and do you want to invest to solve that problem?

Here’s why I ask. Test automation is the process of writing computer code (for the test automation tool to use) to test your computer code (your mobile application). So, you’ll be building another computer program (to test your mobile application) that does test your software faster but is more complex and expensive to build and maintain. Go into this one with your eyes wide open for the ongoing consequences and not just the initial expected benefit of faster testing.

Another issue with “going faster” when building and testing your mobile application is the approval and update process – which is entirely out of your hands! Sure, you can build and test your mobile application faster but you still have the issue of bottlenecks getting your mobile app approved for Apple or Google store and then, even after it has been re-approved and loaded in the Store, your chances that your users will actually download and install the update are small.

So, instead of “going faster”, you might find that “building smarter” provides you with more benefits and fewer costs. Nevertheless, building and running test automation can be valuable for your mobile application in certain situations.

Mobile Application Security and Penetration Testing

Your mobile application probably has some personal and/or confidential information on it. Hackers are penetrating, stealing and selling confidential data from mobile applications and their servers on a daily basis.  Can you afford for your mobile application to be hacked and your users mad at you?

Thankfully, most mobile applications have a small number of security and penetration points so testing them is common practice.

The security of your mobile application’s login screen probably isn’t an issue. In fact, most mobile applications are naturally quite secure in their user interface as long as the login function is well built – and most are.

The real challenges with mobile application and security lie in the way your mobile application communicates with the world around it and it’s server. Mobile applications use API calls to send and receive information to and from other users and the server in most cases. The robustness, error checking and reliability of these API calls will determine how “safe and secure” your mobile application will be for your users. Mobile application security and penetration testing evaluates your mobile applications login screen and internal functions as well as your API calls to ensure that your customers’ data is safe and secure.

Penetration testing can be done against different types of code and systems in your mobile application like servers and APIs. Depending on your goals for the pen tests, there are several phases within each pen test to ensure that the security of your application is covered from all angles. After a pen test is done, a formal report will be given to you that covers all the vulnerabilities that were found and how hackers can use them to gain access to your application’s data. In contrast to many systems and processes of your mobile application, penetration testing is largely a manual process. There are testers that overlook the entire process to make sure that your mobile application cannot be breached.

Mobile Application Performance Testing

Your mobile application will need to respond and perform quickly for your customers and prospects to feel confident using your app. Poor or questionable performance can (and does) lead to poor reviews, fewer stars and rapidly decreasing downloads and usage. This probably is not your goal so performance testing may be a risk you want to consider selecting for your mobile application testing services.

There are three critical areas to consider when performance testing you mobile app.

Device Performance  – Your mobile application needs to do 2 things simultaneously – respond quickly when in use and play well among the other mobile apps on the device. Testing the first is usually straight forward. Testing the second will require a smart strategy to get accurate results quickly without wasting a lot of time with invalid test setups.

Server/API Performance – Your mobile application will need to communicate with your server to send and retrieve information. In most cases, the server application is separate from your mobile app and interacts with your mobile app through an Application Programming Interface (API). This server app will tend to behave like most server-based applications and is subject to the same performance testing with one exception. The APIs need special attention to a wide variety of performance loading criteria to make sure the mobile apps that depend on them are not needlessly bogged down. Make sure to remember to test the server under load as well – not just the server application!

Network Performance – There are almost too many mobile networks to even list anymore. Mobile devices communicate either via WiFi or the carrier’s mobile data network. In all cases, even Wifi, the network speed and capacity are going to vary wildly. 2G and 3G networks still exist in many places in the world and the continental United States. Many WiFi hotspots are still using the older 802.11(a) and (b) technology. Network performance testing is tricky and it is critical to understand the user’s environment and goals before building a network performance testing plan.

Lastly, all performance testing requires detailed data analysis to clearly identify and truly understand the performance bottlenecks. Performance testing typically produces massive amounts of data that needs to be analyzed to be understood. Performance Test Engineers specialize in this data testing and analysis and have spent many years learning and building their knowledge.

Mobile application performance testing can be tricky to plan and execute so that you get the full value of the work. There is a lot of performance testing that can be done but typically only a few of the tests are truly valuable to any certain situation. The key is to be perfectly clear on the problems you want to solve and the risks you want to reduce. This approach allows your Test Engineers to perform the exact tests you need without wasting time and money on unneeded mobile app performance tests.

To Be Continued

In the coming weeks I will shed light on the five other types of mobile application testing so you can make smarter testing decisions. In the meantime, check out this post if want tips on how to select a mobile application testing company.

If you can’t wait for me to finish this post (hey, I can’t write all day, I have a business to run), call me (888-254-9709), or request a call back using the green button on this page. I’m happy to answer your questions.

Website Testing

The Case for Website Testing

/in /by

Are you one of those companies that doesn’t conduct systematic website testing after a site refresh? By systematic website testing I’m referring to a planned and thorough testing of the site for functionality and user experience.

Too many companies, I’ve learned over the years, don’t take website testing seriously enough. Employees  will be asked to set aside a few minutes of their day to  “go through the site” and note any errors or broken links. That’s a start, but don’t mislead yourself into thinking that all the bugs are going to be found with this hit-and-miss method.

In today’s world, if your website doesn’t work properly, quickly and easily on the device the person is using then you just lost a visitor, prospect, new customer or, worst of all, a repeat customer. In every case your brand image takes a beating, too.

Most consumers won’t tolerate a business website that does not perform on their device. They will go to the next business on the list!

Ask yourself, how much is each visitor worth to your business? Each lead?  Each  new customer? Each repeating customer? Is it worth the risk NOT to test your website after you change it?

The 1-2-3 of Testing Websites

There are three critical areas to test on most websites – functionality, cross-browser/devices, and page-load speed.

  1. Functional Testing insures that each button performs the intended action properly. Some websites are complex with advanced search, grouping, recommendations and shopping carts. Some websites are simple with just a few buttons to submit a contact request. Most of the functionality will behave the same no matter which browser or device is used so testing the functionality is typically a “once and done” set of steps each time you are finished with your website changes.
  2. Cross-Browser/Device Testing insures that your website’s content, layout and presentation look and behave properly in a variety of operating systems, browsers and mobile devices. This is where it get’s a little complex. There are just too many different operating systems, browsers and mobile devices to test each one within a reasonable period of time.
  3. Page Load Speed Testing. Visitors have little tolerance for websites that take too long to appear on their device. If the page isn’t fully loaded in a few seconds, there’s a high probability the visitor will abandon the attempt and try another site.

Yes, there are tools available online that seem to make it easier to run functional testing and cross-browser/device testing. But, if you aren’t familiar with them or you don’t have a clear idea of which test cases, browsers/devices are most important to your website visitors, then you can easily waste a lot of time.

Who to Test a Website

The good news is that you have several options for website testing.

  1. Big IT Outsourcing Company – if you aren’t already a Fortune 500 company you probably won’t get much help from these guys. It’s not that they don’t want to help you. It’s just that they are so big that they can’t help you and expect to make a profit. If they could help you, they’d need a small army of managers and directors just to make your project successful.
  2. Freelance – the Gig Economy. This might work for you if you already know a freelancer that is very good at exactly what you need (website testing for functionality and cross-browser/device) and is not already booked with another gig.  If you have to go find that person for the first time, be prepared to put in some detailed work to find, select, pilot and then manage the project to success. Online freelancer boards are great and can give you a lot of good information to make your choice. But, there are a lot of non-qualified freelancers that will answer your call to help and you’ll have to spend time sorting the wheat from the chaff.
  3. DIY – Do It Yourself – This might work as well.  It could be you or it could be someone else in your business. If you take this approach, it will work but it won’t work very well. Here’s why – you simply haven’t been trained in how to think about and perform website testing. It’s not hard and you could definitely learn it but your business’s newly launched website probably isn’t the best place to get started.
  4. TESTCo – Safe. Easy. Reliable. Software and website testing is all we do. We’re highly trained in software testing methods and systems. We see hundreds of website testing projects every year. We know the right tools to use, when to use them and what to look for. We are the right size for you so we can complete your small, 1 or 2 day project as easily as we complete larger multi-month testing projects.

Is TESTCo Your Goldilocks?

If you have a website testing project or are just not confident in the website testing that has been done already, please consider that TESTCo may be your Goldilocks – just the right size and temperature!

You can talk with us and find out how we’re different and how that difference can benefit you. We work as an outsourced QA service, or on demand for projects.

QA Software Testing Company

Does Your QA Software Testing Company Do Both?

/in , , , /by

That’s not a typo in the headline. I’ll repeat. Does your QA software testing company do both?

Both? Both what?

Why, both QA (Quality Assurance) and Software Testing, of course.

QA, Quality Assurance, is a goal and process.

Software Testing is an activity, usually inside the QA process.

Is This Difference Between QA And Software Testing Important?

It depends on what you have and what is important to you, of course.

If you already have a rock-solid QA process for your software quality, then software testing help may be all you need.

After all, your rock-solid QA process should inform you exactly when (and how much) you need software testing versus when you need test strategy and planning.

Your rock-solid QA process already has a SMART Goal that drives your software QA and testing team to the right activities at the right time with the right effort to achieve your goal, right?

But, if you don’t already have a rock-solid QA process, then you may need a little bit more than just software testing. A qualified QA software testing company, LIKE TESTCo, can (and should) help you with this as a natural part of their work with you.

Why Would You Even Need a QA Process?

There are three important reasons to consider building a QA process, even a small one, before you start software testing.

  1. You can’t test everything
  2. Testing can be expensive
  3. You can’t “undo” disappointed customers

As you can see, you can’t win in this situation unless you have unlimited time, money and fresh customers.

How can you win against these three big constraints?

Build a simple QA Process with a SMART Goal, clear strategy, and simple plan to achieve your goal.

Unless you already have a QA Process, goals, strategy and plan, you should seek a QA software testing company that does BOTH – QA and Testing.

Note: a QA software testing company CAN’T help you with this unless they assign a QA Manager to your project and they collect the information needed to build a SMART QA Goal. Testers, no matter how experienced, just don’t have the skills and training to build effective QA Goals or QA Processes.  TESTCo provides a QA Manager, at no additional charge, on every software testing project.

How Much QA Process Do You Need?

We think you should have the “Goldilocks” QA Process – not too much, not too little, just enough to achieve your business and quality goals with minimal risk.

Too much QA process will slow down your software development and release process and you’ll struggle to hit your release date targets.

Too little QA process and your customer service phone lines will light up like a Christmas tree.

A “just right” QA process should produce software that meets your business and quality goals and makes your team feel proud of their accomplishments.

Knowing Your Goal

How much is “just right”? That depends on your goals and constraints. If your goal is for ZERO defects, you should know that your budget is going to need to be *very* large. This type of quality is usually reserved for nuclear power plants and military software.

If you don’t have a quality goal and just need to get it “tested” then you won’t need much budget or time. But, your customers will probably experience significant bugs and delete your mobile application based on their poor experience. This type of quality is typically used by businesses testing a market or idea rather than penetrating and competing in that market. New customers are expensive to acquire and bad mobile applications or buggy web apps are one of the easiest ways to drive them away.

If you *do* have a quality goal, then the right amount of QA process is exactly the minimum needed to achieve your goal. This usually means investing in building test cases during your specification phase, designing and building test frameworks during your build phase, quickly executing test cases and reporting defects as soon as possible after it is built (real-time functional testing), and then a brief functional and cross-device regression testing after all the fixes are completed and prior to roll out.

Reaching For The Right Assistance

Helping you design, experiment and build your QA process is something that an experienced QA Software Testing Company can help you with. A mobile software development company can help you build your development process, but they seldom do well helping you design and build a software QA process.

Why? Software developers CREATE and BUILD software to add FEATURES. Test Engineers TEST and attempt to DESTROY software to find DEFECTS in the FEATURES. Test Engineers typically have the same university-level education and training as software developers, but have chosen a different path for their career. Additionally, QA Managers have a library of QA processes and strategies so an effective strategy and process that meets your specific needs is quick and easy to find, evaluate, and implement.

How Can You Know Which Vendor Has Which Capabilities?

That’s easy – here are three questions you can ask that will quickly and easily let you find out if the QA software testing company you’re talking to has both Quality Assurance and Software Testing experience you need.

  1. How do you establish and achieve quality goals for your clients? What is a recent example? QA Goals should be SMART Goals and have a clear plan for achievement. QA Goals should also have clear constraints – especially in Budget and Time – no project runs forever for free! Lastly, QA Goals should have clear Business Value – they achieve something useful that moves the entire business forward in achieving its’ goals.
  2. What variety and type of QA strategies do you consider when evaluating a client’s needs? What is your “go-to” strategy? There is always more than one way to achieve a goal – these are strategies and help you quickly determine if your plans can work to achieve the goal within the constraints. Sadly, one of the most popular QA Strategies is “test everything” and that seldom works. It feels good because you can see and measure a lot of activity but it almost never provides the detail and coverage most modern technology applications need. In some cases, it can actually be counter productive and delay a release! A smart QA Strategy should be proven effective and simple enough for stakeholders to understand and support.
  3. How do you document, update and communicate your test plans and percentage of goal completion? What is your goal completion percentage for your clients? Documentation of technology projects can be cumbersome but accurate documentation, reporting and measurement of QA and Testing is critically important. Poorly documented defects are literally worthless if they require developer time to diagnose and recreate the defect. Test plans are mathematical equations for demonstrating test coverage and completion across a technology project. They must be updated real-time to provide the real value a testing team needs to achieve their goal and the quality goal of the technology project. Goal completion is a critical value measure for outsourced software testing providers. Without measuring Goal Completion, your outsourced software testing partner is doing the work but failing to deliver the true business value.

Asking these questions can help you avoid the painful lessons of working with an outsourced software testing provider.

Can Location Choices For A QA Software Testing Company Be Helpful Or Harmful?

Having a QA Software Testing company near you can be helpful – especially of you have hardware closely connected to your project. Technology projects are rapidly evolving to include the Internet of Things.

QA Software Testing companies span the world and you have many choices about where you want your software testing & QA services located. There are two primary considerations when considering where to locate your QA software testing services.

Timezone Considerations

If your project is rapidly evolving and the stories/requirements very fluid, then selecting a QA Software Testing Company in the same or very similar Timezone will probably work in your favor. This will help you avoid some of the communications challenges that occur when a team is working at the opposite end of your clock. If your project is relatively stable, then selecting a QA Software Testing Company in an opposite Timezone will work in your favor. This will allow your QA and Testing team the opportunity to complete their work prior to your local team arriving at work. Fresh code at night and fresh bugs in the morning!

Time zone considerations will largely depend on the needs of your business. Generally, teams with the same time zones are more standardized and work together more efficiently. This is the more popular way to work and you don’t have to accommodate other teams that work during strange hours. If your team is focused on a continuous delivery model, having different time zones can be an advantage. This is because one team can work on the project in the morning and at night the other team picks up where they left off. This allows projects to be done around the clock and can be very advantageous. Take a look at the nature of your projects and you can decide whether it’s better to share the same time zones or not.

Intellectual Property Considerations

If your company or project has new and highly valuable intellectual property in your software project, then it makes a lot of sense to avoid locating you QA Software Testing Company in a country that does not have reliable and effective IP protection laws. If you’re building a cutting edge artificial intelligence system then locating your QA Software Testing Company in another country with strong IP protection is almost a basic requirement. If you’re building a marketing mobile app, then you probably don’t have much IP to protect (in your app) and the risk of IP theft is very low and the country where you locate your QA Software Testing Company is not nearly as important.

IP considerations for your mobile app might seem tedious and unnecessary at first, but the long tail consequences are enormous. Without proper IP considerations, your business intelligence is liable to be stolen at virtually any time. Locating your business in a country with strong IP laws protects your data, branding and everything else related to your mobile app. Aspects of your mobile app like logos, patents, trademarks, etc. are protected under countries with strong IP laws. This protects you from having copycats steal your entire business model, name and brand.

The Nexus Of QA Software Testing

TESTCo is ideally located in Austin, TX, with offices in Mexico and Ukraine. This allows us to work in almost any time zone you prefer. Our offices in Mexico and Ukraine allow TESTCo to work on your project without any state-sanctioned IP risk.

Our physical locations enable your mobile app to have both quality assurance and software testing. With TESTCo you can customize your QA and testing process. You’ll be given a QA manager that will help you design SMART QA goals for your mobile app and guide you along your entire QA process. In addition, you’ll also be provided with test engineers that will identify and solve defects within your application. The several locations of TESTCo allows you to choose a location that accommodates your time zone and IP preferences. With the talented QA staff and multiple available locations, your application will have the human and technological bandwidth needed to succeed.

A QA Software Testing Company Should Give You a “Certainty of Success”

This is both a set of facts and feelings that your outsourced software testing company can deliver the value and services you need to achieve your business and software quality goals.

Seeking a software testing company that does both QA and testing will be the most cost-effective solution for your business. This prevents you from incurring additional costs associated with each service and the final invoice amount would be much larger. Also, choosing two different companies to do these services can cause a disconnect; some parts of QA and testing overlap and having one provider for both is the most optimal solution. This creates a seamless integration for both of these services and you’ll get access to all the talented staff that work on QA and testing.

When consulting with a vendor, asking the right questions provided above will enable you to see the important details in both their QA and testing processes. If your QA and testing provider can’t answer those specific questions, you should seek a different provider. A competent QA or testing provider will be able to answer all the pertinent questions related to your project and provide some consultation to help you choose the best path forward.

At TESTCo, every team consists of a Test Engineer and QA Manager to make sure you get the Quality Assurance you want and the software testing you need. It makes a difference. Just see what our clients have to say.

Give us a call at 888-254-9709, or request a call back with the form on this page if you want to explore this further with me.

Outsource Software Testing

Outsource Software Testing – Painful Lessons

/in /by

This is the true story of a real estate company that learned about promises and quality the hard way. They learned that it’s not always a good decision to just trust a third-party software developer to deliver the quality they promised. They also learned the value of outsource software testing when trying to get a mission critical software project back on track.

One of TESTCo’s real estate clients – a company that shall go nameless for reasons you’ll soon understand – was trying to develop the ultimate software for distributing and tracking available properties in their portfolio. By ignoring the need to outsource software testing, they quickly found themselves in a pinch.

The company’s old application was cumbersome, slow to load new properties, and a giant hassle in terms of administration, updates and changes. Their clients, the agents within the brokerage, had numerous requests for new features that would help improve sales and revenue. They wanted better information and more insight into the data they were collecting.

Naturally, they got really excited when a real estate software development team they found promised them the moon. The developers said all the right things, ensuring easier data loading, improved content management and a faster path to sales nirvana. What they forgot to consider was the quality of the software delivered and the need to outsource software testing for a mission critical business application. 

Download Case Study PDF

 

Well, Not So Fast

When development ended everyone was all hyped up to get the new application into– the hands of the sales people, but there were a number of glaring issues that raised questions about the project that we can all learn from.

  1. When does regression testing of the outsourced development become a mission-critical activity?
  2. When is the right time for Outsourced Software Testing?
  3. What happens when outsourced development goes wrong?
  4. What is the best Outsourced Software Testing ROI?

For starters, the outsourced developers only tested new features they designed and were hot for. The developers also failed to perform regression testing or thought they didn’t need to. When you outsource software development, regression testing is typically the most important part of delivering high-quality software.

The real estate company had no experience with outsourced software development. Their core business is moving properties, not batches of code. They relied on the advice of the outsourced software development shop and misguidedly cut some corners by doing their own internal QA testing instead of using professional software testing. In the real estate company’s defense, the outsourced software consultants didn’t offer a professional, third-party outsource testing option. The software developer thought they tested it well enough internally – and this almost never happens! With no prior software development experience, the real estate company didn’t even know it was on the table as an option. And, the contract terms were loose enough to allow the development shop to meet deadlines with low quality deliverables.

As you can guess, when the real estate software project got closer to completion, all kinds of defects started showing up. Features that previously worked in the old software were showing up broken in the new system! The trial and beta launches were a complete disaster.

Correcting Course with Counsel from an Outsourced Software Testing Company, TESTCo

When things were looking really bleak, the real estate company decided to correct course and hire TESTCo for their outsourced software testing to see exactly what was going on.

TESTCo designed a test strategy focused on achieving high regression testing coverage in a short period of time (to match their 2-week release cycle). We brought in a Test Engineer  experienced with exploratory testing strategies, and he built an automated ad-hoc testing program that looked for defects without a specific set of test cases. We also added a software QA track to the project plan and implemented burst-mode regression testing so sprints were not delayed, which spot checked and discovered regression defects during the new feature testing and development processes.

Abandon Ship!

With thorough testing and QA procedures under way, the real estate company began to assess the risks involved with completing the project and waiting for the software developer to fix the problems.

They came to a somewhat brutal conclusion. They wouldn’t be able to complete the project. They’d have to abandon ship, cancel the project without throwing more money away, and retreat to the old system in order to claw back some of its functionality. They ended up modifying the old system, testing it with TESTCo outsource software testing services, and relaunching that solution on time without any defects within three months.

Why It can Make Sense to Hire an Outsource Software Testing Firm Before Starting Development

It was an unfortunate lesson, but at least they got back to a working system that could handle some basic property listing tasks. Sadly, this happens often in the software development world. Companies with no software design and development experience hire outside developers that can’t deliver on their promises. They get oversold and then retreat, licking their wounds, discouraged by shattered technology dreams.

The moral of the story? Hire a solid outsource testing company like TESTCo prior to launching a mission-critical software development project. We can act as consultants and help you vet developers while designing test plans that help guide the project to success.

Can we make a difference for you? Read what our clients have to say.

Download Case Study PDF

Consider the Cost

There is another reason to think carefully about how you design,build and test your software.

When a mistake, defect or error is discovered during the Design Phase of your software project, the cost to fix it is simply a little time discussing the true need and then changing the design documentation.

When a defect is noticed during the Building Phase of your software project, the cost to fix it is larger – close to 10X greater than the cost to discover it in the Design Phase. Why? A mistake in the Build Phase will require significant time and effort for the Design Team to validate the requirement and then the Build Team to go back, re-write the offending code (and any other closely related code) and then test it again. This is far greater cost than during the Design Phase where the changes were only to the requirements.

When a defect is discovered during the Test Phase of your software project, the cost is even larger – again, close to 10X greater than the cost to discover it during the Build Phase. Why? A defect discovered during the Test Phase already has countless hours of engineering time to build the feature. Additionally, the defect has cost you Testing time as well. Now, add to that, the need to re-validate the requirement, confirm that the test case and defect is accurate and then get it rebuilt by the Build Team (along with any other affected or related code because defects seldom happen by themselves) and then re-tested by the Test Team.

Lastly, when a defect is discovered by one of your customers after it has been launched, the cost is approximately 10X greater than the cost to discover it during the Test Phase. If you have a defect in your production software, it will incur all of the previous costs to find, validate and fix. It will also probably cost you a customer or prospect if they can not complete their work with your software.

Defects can be Very Expensive. Testing Early and Extra is almost always an investment with a generous return. Testing with an outsource software testing specialist like TESTCo can decrease your costs and amplify your returns. I look forward to speaking with you.

Software Application Testing Company Three Questions

Software Application Testing Company Three Questions

/in /by

There are just three things to think about before contacting a software application testing company. Yep, just three little questions to ask yourself once your software is “almost” ready and it’s time to have a professional take a look to make sure there aren’t any nasty little surprises left in there.

Trust me on this–if you know the answers to these three questions, I can promise that you’ll find the perfect software application testing company for your unique needs.  I promise!

On the other hand, if you don’t know the answers to these three questions, the best any application testing company is going to do is *guess* or *assume*.

I’m pretty confident you don’t want the company you’re seeking help from to *guess* or *assume*, do you?

Answer These Questions Before Contacting Any Software Application Testing Company

1 – What are your goals for this project?

Specifically, the list of things you want done (and NOT done) and the list of things you want to see so you can be confident the goal has been achieved. Nothing complicated – even some jots on a napkin will help. What’s MOST important here is that you’ve thought about it enough to be clear about what you do and don’t want.

Why? Software quality means different things to different people. Google is OK with some defects in their software (permanent Beta) while high-end jewelry stores are not willing to tolerate even a slightly-less-than-perfect image.  Either approach is effective but only in the context of your specific needs and goals. If you know (or have at least thought through) where you stand on your goals, the application testing company you’re talking to can do a much better job of understanding and meeting your specific needs and goals!

2 – What is your budget for this project?

Yeah, the budget question….and budget is more than just dollars! Possibly more important is TIME. When do you need it to be done? You probably already know this but it’s important enough to make sure. Take a minute and confirm the dollar budget you have to spend and the date by when you need to launch the application.

Why is this important to know and SHARE? Very simply, it helps the application testing company you’re talking with do a better job of understanding and proposing a solution that meets your unique needs. If you withhold this important information under the guise of an outdated and ineffective vendor negotiation model, you’ll get quotes and proposals that are more guesses than fact. What you get is a best-guess for a solution where critical facts are missing. Further, if you award the project based on this approach, you’re merely awarding the “best guesser”.  That’s probably not what you were seeking when you went looking for a software application testing company. The good ones don’t *guess”.

3 – How will you chose the best vendor for you and your project?

Some folks have a need for speed and others have a need for deep attention to detail. Some want a very close relationship between developers and test engineers and others don’t want their developers bothered. Some people want lots of transparency in the process and some people just want the defect reports. Decide which methods and priorities best meet your needs.

Why? If you share your selection criteria with the application testing companies you’re talking to, they can customize their proposals to highlight their differences and how those differences can be valuable to YOU. If you don’t, everyone is back to guessing and that’s not going to improve your certainty of success. In some cases, time can be saved when a firm quickly determines they are NOT the best fit for your project. Wouldn’t it be better to acknowledge that and get out of the way instead of endlessly pestering you for another sales call?

A good software application testing company, like TESTCo,  is not in the guessing business. We’re in the facts and confidence business. Please help us help you achieve your goals and dreams. Think through the three questions the best you can, then give us a call.

 

Mobile Application Testing Strategy

Why You Need a Mobile Application Testing Strategy

/in /by

In This Post

App testing looks easy but you might want to think again – and I’ll help you think through a mobile application testing strategy.

Simple, but not Easy

You can very easily skip or ignore some very important risks just because you didn’t even know to think about them! Mobile application development moves very fast, is usually highly agile and covers much more than just the mobile app itself. Most mobile application testing projects spend too much time on functional testing and not enough time finding and reducing real technology and business risks.

Starting to test your mobile app without a Testing Strategy can result in a “well tested” mobile app that frustrates customers, scares off prospects and lights up the support phone and chat lines.

All because you didn’t know what you didn’t know!

And that’s probably not what you wanted or expected.

A simple mobile testing strategy will help you find the important risks, plan your testing work to focus on the important areas and reduce the time and money you spend launching your mobile application

What You Don’t Know CAN Hurt Your Project!

There are six  important subject areas in that you need to consider in your strategy for testing your mobile application.

  1. Devices – What is your list of most important devices? What devices are you willing to exclude or not test? Do you have access to those important devices?
  2. Networks – What networks will your mobile application use? Are they cell networks or wifi networks? What happens when your mobile app switches networks?
  3. Servers – How much of your mobile application functionality depends on accurate information from a server application? Most mobile apps rely heavily on back-end servers to do the heavy lifting work while the mobile app itself mostly accepts inputs and display outputs. What happens in between your server and mobile app?
  4. Performance – What are your minimum performance times for functions in your mobile application? What are the upper limits of your server? How many simultaneous users do you expect? What performance tools are provided by your server hosting company?
  5. Security – How secure does your mobile application need to be to protect your user’s information privacy? Is 2FA important to your users? How secure does your server need to be?
  6. UX (User Experience) – Will your users find your mobile application enjoyable and easy to use or frustrating and confusing to use? What UX data will you collect to help you understand this? If a user does leave, how will you recover them?

If you know the answers to these questions, you’ll be able to clarify your goals and build a mobile testing strategy to achieve them for your unique mobile application.

Curious yet about TESTCo’s mobile app testing services? Here’s more information on our capabilities.

No Mobile Application Testing Strategy? Here Comes the Pain.

If you don’t answer those questions, you can expect to see (and feel) some of these pains:

  1. Your mobile works fine on the most recent iPhone but fails with previous versions of iOS and some Android devices. And, only a small portion of your audience has the most recent iPhone.
  2. Your mobile application works great on Wifi but fails with errors on a cellular network if a user is talking on the phone and using your app.
  3. Your mobile application fails and crashes when the server is slow returning complex query results.
  4. Your mobile application works fine until there are over 900 simultaneous users – and then it displays the wrong query results.
  5. Your mobile application is easily penetrated and accounts are hijacked.
  6. Your mobile application UI works great for the engineers who built it but ordinary users struggle to understand how to use it.

Build Your Own Simple Strategy – or Let Us Help You!

The best solution is to build a simple mobile application testing strategy unique to your project. It’s easy if you follow this guide I recently published.  At TESTCo, we build and evaluate several test strategies for every software and mobile application testing project. If you don’t want to build your’s, we can build it for you!

Schedule a call with me using the green button on this page, or call me at 888-254-9709

testco-software regression testing | agile software testing | agile testing

Agile Testing Doesn’t Have To Be Complicated

/in /by

Agile Development methodologies have become widely used and accepted as industry standards across the world. The flexibility to grow and adapt software development organizations is one of the most attractive features of Agile development. The price paid for this flexibility is buggy software. While most Agile development teams are able to produce features at a higher rate than traditional development teams, they also deliver more defects than a traditional development team. Software quality for Agile testing is needed.

Agile Software Testing Services

Agile testing involves testing from the customer perspective as early as possible, testing early and often as code becomes available and stable enough from module/unit level testing. Since working increments of the software are released often in agile software development, there is also a need to test often. Taking an agile testing for software quality control approach is key.

Download Agile Testing Report

Why Have an Agile Testing Methodology?

  • Early in the lifecycle teams don’t realize the severity of existing defects, and do not put a large effort into finding them.
  • When Critical Mass is reached, customers begin using the product in unexpected ways.
  • As product development becomes steady, the effort necessary to find the defects is not constant throughout the release cycle, and a steady team of Test Engineers is too much for some stages and not enough for other stages.
  • When the product matures, automation test scripts that have been relied on in the past no longer provide sufficient coverage, and cannot find all of the defects.

On Demand Agile testing solves these problems, it keeps product quality in step with your engineering team. TESTCo’s QA testing will make this happen.

Software Test Outsourcing

The Costly Learning Curve of Software Test Outsourcing

/in /by

There are three things you can do to avoid the costly learning curve of software test outsourcing initiatives. Why does a learning curve exist? Software is getting more complex every day.  This makes software testing even harder. The learning curve for any software testing team can be steep and it is even steeper and more costly for software test outsourcing teams.

When the learning curve to begin testing software is steep due to complexity, everybody on the team feels the cost – especially the project manager! Software engineers experience interruptions which degrades their concentration, business analysts get frustrated covering old ground and project managers churn through project timelines and budgets trying to get everything to match up.

You can’t make software less complex and you can’t roll back time (unless you have a time machine that you’re keeping secret from the rest of us!). If you can’t compress the size of the problem or flatten the learning curve then all you’re left with is tools.

Three Key Tools to Shorten the Learning Curve of Software Test Outsourcing

Here are the things we use regularly at TESTCo that have proven effective in reducing the learning curve of software test outsourcing.

  1.  Use a Functional Map – create an index of user stories/scenarios as compared to primary function points in the software where the intersection of a user story and a function point denotes a test case or collection of test cases. This index provide multiple benefits starting from Day 1. Contact Us to see how a Functional Map could help your software testing team.
  2. Build a Value Proposition – when your entire software test outsourcing team understands and is aligned behind BOTH your technical and business goals, your QA team becomes a key strategic lever to move your software project directly towards your business and technical goals every day. Reach Out to Us to learn more about our Value Proposition Process.
  3. Add a QA Manager for your software test outsourcing team. At TESTCo, we believe that managers manage and engineers engineer. Engineers almost always perform at a higher level when they are led and managed by an experienced technical manager. We believe so strongly in the value and benefit of a QA Manager that we provide one, at no additional cost, for every TESTCo client.

If the learning curve of software test outsourcing is holding you back, we’d enjoy the opportunity to share how we’re different and more successful for our clients.

Automatic Software Testing | Automated Software Testing | Software Test Automation

Calculating the Maintenance Costs of Automatic Software Testing Suites

/in /by

Is maintaining automatic software testing* an investment, expense, or just a problem? You’ve paid a lot of money to automate software testing and all was working just fine,  but as time goes on and new features are added to your application, the automatic software testing suite is breaking more often and you’re getting frustrated.

Software Test Automation Tools are Changing Rapidly – Double Check

Before you jump off into a test automation rebuild project, it’s worthwhile to do a little bit of due diligence to make sure your platform and software test automation tool will continue to serve you into the future.

  • Your choices for software to automate your testing activities is growing and changing rapidly.  There are literally new software test automation tools announced every month. They tend to be born or created from a need to solve a unique test automation problem. Make sure your software test automation tool is synchronized with your technical needs and supports your technology platform.
  • Software test automation tools change versions and compatibility frequently. As new software technologies are built and release, software test automation tools change to accommodate those new technologies.  In some cases, these upgrades can cause havoc with the scripts you’ve already built for the previous version. How stable and “future proof” is your selected tool?
  • Software test automation tools also fail frequently as businesses. How can you tell if the company you purchase your tool from will be in business next year? There are many open source choices for software test automation but those tools have a risk as well. Will the open source community continue to be interested enough in that test automation tool to keep it working with the current changes in technology?

Maintaining Software Test Automation is Not Easy

  • You probably paid a lot of money to get your software test automation working properly but the return on your initial investment is fading as time goes by – surely that’s not how it is supposed to go.
  • Your software test automation suite worked just fine when it was originally built but today, it’s throwing more and more false errors – and that doesn’t help you achieve your business and technical goals.
  •  Your software development team is too busy writing new features to keep the test automation updated and, they aren’t really interested in working with those tools.
  • Your internal QA Team has trouble finding, selecting and keeping someone talented enough to maintain your software test automation suite – the learning curve is steep and maintaining code is much more challenging than just writing new code.
  • You’ve looked for outsourced help (again) but your software test automation maintenance project just isn’t quite big enough to interest the big offshore technology firms.

Push the Reset Button for Automatic Software Testing

  1. Build a business case – nothing complex – back of the napkin calculations should be sufficient. Gain a deeper understanding of your specific technology and business goals and how they relate to continuing to invest in software test automation. Once you are clear on your specific goals and expected returns, your choices will become much more distinct and easy to evaluate.
  2. Treat test automation false alerts as defects – just like you do with your application software. Use a defect tracker just like you do with your application software. Then, simply cross check application defects reported by your software test automation with the software test automation defects and automatically rule them as false and then use manual testing to verify. This won’t eliminate the time waiting problem but it will reduce it and stop the confidence erosion.
  3. Move your test automation engineer into the software development team. When your test automation work is aligned with the development project timeline and not the QA team timeline, your software test automation will be ready when the software is ready, not a few weeks later.
  4. Start building a software Test Automation Team. Hire a Jr Test Automation engineer to learn from your current Test Automation engineer. Not only is this a great strategy for succession, your software test automation suite will suffer fewer issues and grow quicker to match your software application.
  5. Find an outsourcing partner (not a”vendor”) to work with you over time to specifically meet your business and technical goals. Most outsourcing companies try to be as large and broad as possible to meet every need possible. A few outsourcing companies specialize in automatic software testing and test automation. We specialize so all of our talent will be perfectly suited to solving your software test automation maintenance problem.
^Most readers will dislike my use of the term, “automatic software testing suites”, in this post. The term is not accurate. “Automated software testing suites” is the accepted phrase used in the software testing world. Accepted or not, our research indicates that “automatic software testing” is used by a large number of folks as they search for information on the topic. Therefore, I’ve devoted this post to the folks who prefer “automatic” over “automated”. We do a great job for both. -JCH
Web Application Testing

Web Application Testing Case Study

/0 Comments/in , /by

How Comprehensive, Tightly Managed Web Application Testing Enabled PDS to Keep a Critical Commitment to Customers.

On This Page

About Pharmacy Development Services

Pharmacy Development Services helps pharmacies develop critical and fundamental business principles that allow them to operate successfully. Their services, systems and software allow pharmacy owners to strategically plan and execute pharmacy-specific operational, training and marketing programs.

Event Deadline and a Project in Shambles

Pharmacy Development Services was in a difficult situation. They were trying to create a new member services platform to improve the way their clients access key information, cooperative resources, communities and connections.

The software project, dubbed PDS Advantage, was to replace an antiquated seven-year-old platform that wasn’t living up to the needs of their pharmacy clients.

After months of development, the look and feel of the new web application was on target, but the functionality wasn’t there yet. “It seemed like there were a lot of elements working . .  until they didn’t,” said Kelley Babcock, Chief Operating Officer at PDS.

“It was difficult for me to sift through all the technical stuff and determine who needed to be responsible for what and who was letting us down on our team – both internally and externally.” The company’s marketing agency should have been conducting web application testing during the development process, but they overlooked some key people and process issues.

Adding to the stress, was that the new software needed to launch at their yearly business development conference in Orlando, Florida. The February conference was only a month away, and they had scheduled their CEO to be on stage, demonstrating the new software and promoting its virtues. Client attendees were already looking forward to the big reveal.

The crisis ramped up even more in early January, just a month away from the show. “It became painfully apparent to me that we didn’t have an internal project plan, and we lacked a scope of work that would allow us to confidently hit the milestones necessary for a show date launch.” explained Babcock. They also wanted to have PDS members try the solution right after the conference.

Our credibility and the reputation of our CEO and founder were on the line.

“If we failed to meet the show deadline, the level of customer dissatisfaction level would have been astronomical,” said Babcock. “That may sound melodramatic, but even if 20% to 30% of our members were highly dissatisfied, it would have been catastrophic. If we lost 20% of our member base, for example, that kind of outcome was just unthinkable to us.”

Download This Case Study (PDF)

TESTCo Rescue

In early January, PDS sent up an SOS signal to TESTCo. “Our credibility and the reputation of our CEO and founder Dan Benamoz was on the line,” said Babcock. “Fortunately, the minute I raised the alarm, Dan said, ‘I know who we have to call immediately!’” Benamoz and TESTCo’s founder, Jeff Hotz, belong to the same business strategy group, so the timing and support relationship were impeccable.

The companies made the proper introductions, and things went into crisis management mode. Babcock led daily meetings to navigate TESTCo employees through the challenges at hand. “I was driving those meetings,” said Babcock. “In non-crisis times, the COO is usually not the one drilling down to that level.”

“I was protecting our dedication to the user experience,” she continued. TESTCo’s Cristina Nistor led a team of developers, strategists and web application testing experts to manage the project.

“Cristina and her team worked continually behind the scenes to make sure the member experience was seamless and flawless,” said Babcock. “Cristina was our white knight in shining armor.”

A Comprehensive Web Application Testing Solution

TESTCo made sure every link, every function and every click behaved exactly how it should in the PDS Advantage application. They tested the application on every known browser and device, including Firefox, Chrome, Safari, Internet Explorer, desktops, tablets, iPads, iPhones, Androids and Windows Phones.

“I felt like the testing process was extremely comprehensive,” said Babcock. The project management software includes message boards, a resource library, implementation tools and internal communication features that each pharmacy can use. It’s pre-loaded with more than 40 turnkey programs designed specifically for pharmacists. If one of the pharmacies wants to implement a free children’s’ vitamin program, for example, the system makes it really easy. They search for ‘free children’s vitamins,’ and up comes a template with every step and resource needed to launch a comprehensive program. The resources include team training processes, marketing collateral, policies and more. All the user needs to do is search, download, assign tasks to their team and put a due date on the project.

“Our clients went to school to be pharmacists,” explained Babcock. “They didn’t go to school to become trainers, marketers, and operations managers. With our software solution, they don’t have to reinvent the wheel and do all heavy lifting on their own. That’s a huge benefit. They get to do what they do best – pharmacology.”

Just In Time

Even though they were working on a tight deadline and the tasks were highly technical, Babcock was easily able to follow their progress. “TESTCo put things into a language that was easy to understand and describe,” she said. “We were getting into the weeds, but our collaboration was delightful. You never thought about it as drudgery. I liked the personal connection.” That personal connection even went so far as to include changing languages. “When TESTCo found out that our marketing strategist for service delivery, Omar Avila, spoke Spanish, they started holding those meetings in Spanish, which was nice.”

When the PDS business development conference finally arrived, they were ready to go. CEO Benamoz took the stage and introduced the app, their marketing lead paced through some of the new features, and customers were able to test drive the new application right there at the show. “The confidence we felt was a direct result of TESTCo’s efforts,” said Babcock.

“Without that final push, we would have had to apologize, iterate and continue developing the software well past the conference,” said Babcock. “Because of TESTCo, we didn’t have to do that.”

Key: A Strong Project Lead

Babcock credits a lot of the project’s success to TESTCo QA Manager Cristina Nistor. “She’s highly competent with great attention to detail, follow through, and execution,” said Babcock. “She could look at our processes and people – all the moving parts of the project – and isolate, identify and advise on next steps. Her level of expertise allowed me to have confidence that we’d get it done right before the show.”

The weeks leading up to the business development conference were tense for all involved, but everything worked out. “It was an unparalleled, exceptional experience in terms of service, competency, and advisory guidance,” said Babcock. “We couldn’t have done it without the TESTCo’s web application testing team.”

Download This Case Study (PDF
Software Test Automation

Software Test Automation Case Study

/0 Comments/in , , /by

Journyx has customers worldwide and have been improving their products with software test automation for the past seven years. The company offers customers project, time and expense tracking software, and resource management software for forecasting projects, budget status, and employee time and availability.

Challenges/Pains

Before hiring TESTCo to manage software testing and develop best practices, Journyx was in a world of hurt. Early on, they used company staff and even some family members to bang on new software releases, but that didn’t work so well. This was about 15 years ago. “It’s difficult, repetitive work,” said Journyx CEO Curt Finch. “No one really likes it, and it’s not really a core competency.”

The company wasn’t able to hire more people for testing, but they were always under pressure to release new versions of their products. “We couldn’t get a handle on when our next release would show up and what level of quality it would be. We could say we’re going to ship this come hell or high water by May 1st, but we’d be in danger of shipping broken software.”

“Back then, customers hated us and wanted to shoot us,” said Finch.

Journyx’s director of product management, Meredith Zachritz, had to manage a lot of the chaos at the time. “In the early days, our developers were responsible for testing their own code,” she said. “There were no robust peer code reviews. It was very messy. We just about drove ourselves out of business.”

The company didn’t have an official product manager back then, and they didn’t have the budget to hire testing resources in-house. “I was spending more than 50% of my time on testing and related fire drills,” said Zachritz. “It affected customer retention, and we lost confidence in our product’s ability to handle customer needs.” When mobile apps came along, complexity increased, and it was even more difficult to keep up.

Download This Case Study (PDF)

 

Decision

The Journyx team eventually decided on outsourced testing, but they struggled with how to do it. Initially they hired contractors in India, but that failed. “We tried a couple of different firms,” said Zachritz. “It wasn’t apparent that they had QA expertise. If they did, the language barriers and time zone limitations were too difficult to overcome, and we couldn’t make it work. Our QA didn’t improve, and they didn’t understand what our goals were. They had no certifications or formal expertise, and no one was managing the entire project.”

Journyx ended up with nothing to show for their efforts – no test scripts, documentation or processes they could hand over to their internal development team. “We got what we paid for,” said Zachritz.

After that fiasco, Journyx hired a dedicated employee to be QA manager. They built up a set of test scripts – nothing automated – and some manual testing methods. “We had to train her,” said Zachritz. “She was only one person. When she was out, she wasn’t doing QA. And when she found a more attractive offer, she left. When you have someone in house, those are always issues.” That employee didn’t offer up any strategic guidance or best practice recommendations either.

Local, Accountable and Low-Cost Solution Helps Drive Revenue

Journyx eventually found TESTCo and solved everything in one fell swoop. They found a local company with the expertise to manage complex software testing and low-cost offshore testing labor. “What I like about TESTCo is that they’re an intermediary to different cultures,” said Finch. “At one point they switched testers from Romania to Ukraine, and we didn’t notice the change.”

“They’re here in Austin, and I can take them to lunch and hold them accountable or beg for forgiveness if we’ve messed something up on our end,” he continued. “We trust them, and the costs are very reasonable for what we get.”

Finch categorized the value of what TESTCo brings in terms of revenue, cost and risk. “TESTCo helps us understand exactly what we’re shipping,” said Finch. “We know customers will be happy with it right away, and our sales people have confidence in the software. That translates to increased sales.”

“The sales team is no longer afraid to demo stuff in presentations,” added Zachritz.

“Our costs are reduced because the testing talent is in another country where it’s less expensive to employ them,” continued Finch. “If I were to replace TESTCo with people here in the U.S., I’d have to spend about $5,000 to $10,000 more per month.”

As Journyx rolls out new product features and new software, they reduce risk by leveraging TESTCo resources. “For one thing, I have reliable, detailed insight into the status of everything,” said Finch. “With TESTCo we’re able to have a known level of quality in a consistent, predictable way.”

Journyx can also examine potential costs and complexities when considering the addition of new software features. “We can ask TESTCo how much it’s going to cost to add something to the testing,” said Finch. “That’s a double-check on the complexity question. It’s like a reverse estimation tool.”

Virtual In-House Team

Zachritz meets with TESTCo for bi-weekly progress calls and has daily email status updates from the TESTCo team. “We coordinate as though TESTCo is an in-house team,” she said. “The TESTCo QA process is run for me. I don’t have to do anything but review status and answer questions. I contact them with adjustments. They come to me with best practices and next steps guidance. All of that is very powerful for me.”

She now spends about 5% of her time on testing chores (as opposed to the 50% she mentioned earlier). “Now I can spend my time on planning and execution, and avoid all that fire fighting,” said Zachritz.

With TESTCo, Journyx can also scale up and scale back resources depending on their release cycles. “We can really manage it closely in terms of what we need at a given time,” said Zachritz. “And it doesn’t hurt us when someone takes a two-week vacation during testing. The TESTCo replacements are seamless.”

Software Test Automation Moving Forward

The TESTCo solution allows Journyx to automate software testing processes, as well. The goal is to reduce regression testing cycle times and get new releases out in a shorter amount of time.

Initially, Journyx didn’t tackle software test automation when starting out with TESTCo, but they eventually decided to move ahead with it in order to further enhance productivity and free up resources. “We tried to implement an automation suite earlier,” said Zachritz, “but coverage was bad, the platform was difficult to maintain, it was open source, and it wasn’t possible to upgrade it.”

TESTCo recommended changing out the platform. The two companies went through a new platform evaluation process, and found a new system that fit Journyx’s budget.

“We ended up automating the most time consuming manual tests and regression test cycles,” said Zachritz. “That speeds up our release cycle and in effect drives sales.”

 The process helps them improve code quality, close the gaps between fixes and generally move faster as an organization. “Our support teams can now work on bug reports and patching custom solutions for customers,” she continued. “They can also work on the videos they create to help people configure their solutions. Software test automation allows us to work on more strategic projects because of the quality of releases.”

“All of it has transformed our business,” said Zachritz. “It’s really incredible how it works.”

Download This Case Study (PDF)

Read more about TESTCo’s software test automation services.
QA Testing Services

How QA Testing Services Elevate the Role of QA Manager

/0 Comments/in , , /by

In this Post

It’s safe to say that QA Managers are under-appreciated in their companies. Too often they’re seen by the rest of the organization as the person who prevented the product from launching on time. Ouch!

It’s completely unfair to be seen as the goat, and it’s certainly no fun to always be in firefighting mode.

What if I explained how using the right QA testing services can change the perception from goat to hero? Or, from “the head of testing”, to “our #1 brand ambassador?” I believe that QA can deliver a lot more value to the organization than simply testing software and reporting bugs. Consider this axiom:

“Testing” happens when you tell your tester what to test.

“Software Quality” happens when your team knows the goals and constraints, has a well thought out strategy and plan to achieve them, and a unique person to support and hold everyone accountable to their part in achieving the goals.

I talk with QA Managers every day who want their team to be perceived as contributing more to the business. They just aren’t quite sure how to go about it. Permit me to present a perspective I’ve developed since founding TESTCo in 2002 (yep, that’s a lot of QA experience).

Identify and Communicate the Real Business Value of QA

The quality of your software is reflected in:

  • Customer satisfaction or complaints
  • Engineering staff retention or turnover
  • Business goal success or failure
  • Your personal delight or embarrassment

In fact, software quality probably means more than you realized! At the core of what QA does is mitigate risk that threatens to rob profits and dilute brand loyalty.

If your business builds software, mobile apps or websites (for yourself or others) then your business risk is high because of technology. In these cases, QA testing services become a very necessary and strategic asset for the organization. After all, it’s your software, mobile app or website that brings you new customers. What could be more critical to your business than that? Answer: Nothing.

Anxiety in the Face of Risk

It’s understandable if your anxiety is growing because your testing team (internal or outsourced) is giving you the run-around.

You know, those weekly Sprint meetings where you’re supposed to help Engineering get everyone on the same page, but you wind up feeling disgruntled? Like you didn’t get the information you really need from the testers.

Or, you’re having to calm down a frustrated engineering manager because of a fix-one, break-one scenario where software defects get fixed but something else breaks. Yeah, and what about the angry phone call from the product manager who’s hearing about surprising bugs that users are finding after release? There’s a way out of this nightmare.

The Best QA Testing Services Start with the Right Strategy

You’ve heard it before – there is more than one way to skin a cat. There is also more than one way to test your software. The key (and our preliminary goal for you) is to get as much QA testing services benefit as possible at the lowest possible cost.

Effective strategies start with a clear goal and list of constraints. These are the boundaries of the strategy and there are a variety of different ways to combine QA testing services in order to achieve your goal within your constraints.

The best strategy will provide you with the most benefit for the least cost. At TESTCo, we’ll provide you with several strategies we’ve considered and a recommendation for the one that suits your goals the best.

With a defined strategy you’re able to better communicate with Engineering and Product Management. You can set expectations more accurately.

Best Practices Beyond the Testing Tool Stack

We’ve been testing software since 2002. In that time we’ve perfected a process and philosophy that work really, really well for delivering business value beyond simply finding defects. You see, we don’t just find bugs, we help the development team deliver software that thrills and excites your customers. It’s not the tools we use, it’s the brain power and discipline we put into every assignment.

As QA Manager, when you have access to accurate information and clarity of purpose you’re in a stronger position to deliver business value that is broadly recognized and appreciated. The following is a list of “must-haves” if you seek to elevate QA’s contribution to the business.

  1. An understandable report on the exact features completed and tested every day – twice daily. Not just completed. Completed and Tested.
  2. A detailed understanding of your business and technical goals and priorities and their current daily status during your Sprint. Your priorities, not whatever the Dev team can get done.
  3. A strong foundation of regression testing and measurement of coverage. Yes, good old-fashioned test cases built over time and then run as a final regression prior to releasing. No more surprise defects after you launch.
  4. Weekly product quality meetings with the project QA Manager to review your software development quality measures and processes with recommendations for small improvements that make a big difference. Say goodbye to bad or missing estimates and buggy code that causes missed dates. You’ll see your software quality metrics in real time and be able to make smart adjustments to achieve your goals.
  5. Daily software testing by a Test Engineer.

Do our best practices pay off? Read what other QA Manager have to say about working with TESTCo.

 

Working with TESTCo actually makes us more efficient.

Having you on my team, working together and helping me reach my goals, is invaluable.

Sandy S
Salient Systems

Indispensable to meeting delivery dates

We would not be able to commit to the release schedule we have without TESTCo’s help.

Yelena G.
Salient Systems

Increased Productivity of Engineers

By using TESTCo to drive our software QA testing, we’ve dramatically increased the productivity of our engineers. Now they can spend more time focused on product development and receive almost immediate feedback on fixes needed.

Joseph P.
Convio

Peace of Mind

We needed to be MS OEM certified ASAP and the TESTCo team was able to jump in and get it done.

Carrie D.
Carbonite

Integrated Into Our Culture

TESTCo provided staff with strong technical skills and the personality to propose creative solutions without waiting for direction.

Guy L.
Lombardi Software

Fantastic Job

TESTCo has done a fantastic job managing this, oftentimes haphazard and disorganized, testing project.

Moriah C.
Bluespace

Objective Perspective

Having the objective perspective that TESTCo contributes is supporting our goal of providing a high quality end product to our customers.

Ivor S.
SKYLIST

A Pool of Trained Resources

TESTCo provides us with a pool of trained resources that gives us the flexibility to ramp up our software QA testing efforts on a very short notice.

Maarten M.
Lombardi Software

An Extension of Our Team

TESTCo’s software testing services are very effective in uncovering software defects prior to release.

Bill B.
Journyx (2014)

Drive Costs Down

The value of their staff extends well beyond the positive measurable ROI I receive each month.

Brian W.
Surgient

Great Help

They quickly picked up familiarity with the product line and helped us get through our testing.

Jeff T.
Anue Systems

Synchronize Testing

They allow me to get a lot of manual testing done at once, and have helped synchronize testing efforts with the workflow in development.

Stephen S.
AG Mednet

More Than Pleased

We have completed our Forms application and are more than pleased with the work TESTCo performed for us.

Wendy J.
Accubuild

Brains Matter

Notice in the previous section that I mentioned QA Manager and Test Engineers. I’ll expand on this important point.

Sure, you’re a QA Manager – in role if not in title. But imagine if you had on the project testing team a like-minded QA Manager to help you get the strategy right, develop the test plan, and report to you daily? This proposition is even more attractive when the QA Manager is available at no additional cost, speaks English fluently, and is based in the Central time zone (as is the case with TESTCo).

For example, if my company works for you, here are the things you can count on your QA Manager for:

  1. Analyze and select the best QA Strategy for your unique situation, goals and constraints
  2. Building and tracking the QA Plan to make sure Real Value is delivered every day
  3. Constantly confirm or replan so that the Goals are achievable within the Constraints
  4. Coach and support the Test Engineers to peak performance
  5. Negotiate and resolve blocking issues
  6. Lead post-mortem meetings to improve the next Sprint

I deliberately differentiate between Test Engineers and Testers. Software Test Engineer or Software Tester. Test Engineers are engineers who love to test. They have the mindset of engineers: “Let’s define the problem set a goal, and make it happen.” In other words, Test Engineers don’t have to be told what to do. They’ll figure out the best path to take once they understand the goals and constraints. Plus, because they’re engineers they know how to communicate effectively with the Dev team. Testers are generally limited to doing whatever you tell them to do.

We Know There’s More to QA Than Just Testing Software

I hope my perspective on QA is helpful to you. Maybe it has inspired you to elevate the role QA plays in the organization. I’d love the opportunity to help you.

We’re experts that diligently and consistently test and measure the quality and readings of your product relative to your business and technical goals.

We do the QA Testing work necessary for effective business and technical decisions. If you don’t have a daily report on your goals and quality, you’re missing key information that you need to build a stronger and more profitable business.

TESTCo provides everything you need in QA Testing Services to easily and successfully launch your website, mobile app or software.

Say good-bye to frustration, buggy software, disappointed customers and missed prospects. Say hello to a stronger and more profitable business because your software is an asset rather than a liability.

If this makes sense to you let’s talk about the state of software QA in your organization. Just click the green button on this page to schedule a time to chat with me.