Software App Testing cCompany

Three Criteria to Select the Best Software App Testing Company

It seems there is a new software app testing company born every week. That can be good or bad depending on your situation.

It’s good in that newer companies can bring innovation and higher levels of customer service to a very mature market. This can give you a broader set of choices to meet your unique needs.

It’s bad in that there are so many choices! Pages and pages of them and then more websites to help you evaluate their particulars. It’s been proven that too many choices can actually make it harder to make a wise choice.

Why Bother? The Risks of Making a Bad Choice for your Software App Testing Company

There are many risks in business and technology. Testing your app properly will identify these risks so you can make a smart decision about what to do with the risk. If you fail to identify and eliminate the risk, your business will suffer the consequences.

Selecting the best software app testing company also has risks. The three biggest risks you face when selecting a software app testing company are:

  1. Your software app testing company is not really a testing company. They primarily build apps but do offer testing when they don’t have enough work building apps. Your risk is that your app won’t be properly tested by trained, professional Test Engineers. Rather, your app will be tested by junior developers who’d rather be building something than testing it.
  2. Your software app testing company provides a “project manager” but doesn’t provide a QA Manager. Your project manager will make sure your “testers” are working but won’t have much information on how to focus or optimize their work. Poor work, poorly led seldom produces outcomes that you’ll be confident in.
  3. Your software app testing company can’t offer an effective QA Strategy or Plan. They don’t specialize in software testing and QA. They can “test it for you” but you won’t be able to see and measure how deep or how broad your app was tested.

At TESTCo, we are built on top of these three foundations – Test Engineers, QA Managers and proven/tested QA Strategies. If your software app testing company is missing any of these foundations, your risk of failure is higher.

Three Criteria for Selecting a Software App Testing Company

You’re going to need at least three really important things from your software app testing company if you want to be successful with your app launch or update.

  1. You will want to Set and Achieve Your Goal. You may not have a goal yet – or you do and don’t realize it. “Just test it” isn’t a goal – that’s an activity. “Zero customer found defects” is a goal. “Zero fatal crashes” is also a goal. You’re building a software app for some business purpose, right? What is that purpose and how can your software app testing company help you achieve it?
  2. You will want to achieve your goal Within Your Constraints. You *do * have constraints! Most probably you have limited time and limited budget to achieve your goal. If you’ll share your constraints with you software app testing company, you’ll avoid a nasty surprise and/or the run around trying to make an overly large proposal fit into a tiny budget and timeframe.
  3. You will want Real and Lasting Value. You probably want defects – bug reports. Yes, those are valuable but only if they are accurate, unique and developer-ready to fix. You can also capture even more value that last much longer than a bug report. That additional value might be a small process change that improves your software development speed or it may be an open source requirements management tool that help you shave a couple days off your product specification process. Insist on and capture that extra long-term value, too!

You have a huge variety of choices when you select a software app testing company to help you complete and release your software project. Making a wise choice will save you countless hours and thousands of dollars that a poor choice will cost you.

Ask us these questions at TESTCo. I’m confident you’ll be pleased with our answers.  I’d enjoy speaking with you about your project. Call me at 888.254.9709 or schedule a call back using the green button on this page.

We’re different from most outsourced qa testing companies in refreshing ways. Browse our case studies and testing tips to learn why.

Bonus – Three Traps to Avoid when Selecting a Software App Testing Company

So, you’re clear on your goals and constraints. You’re on the alert for how your choice of a software app testing company can improve your situation now and in the future.

Now, be on the alert for these three common traps.

1. Price Vs. Cost

Seeking the lowest hourly rate – their price – can lead you into a trap where the hourly rate is low but your project cost is high. This can happen when your software app testing company struggles to attract and find top quality software testing talent. They might also possibly be testers that were originally software developers but weren’t good enough. Be on the alert for “too good to be true” hourly pricing.

2. Missing Manager

software app testing companies can also reduce their hourly rates by “body shopping”. This occurs when you rent or lease a person’s time to perform tasks for you. One of the most common ways to reduce the price of a tester is to eliminate any management oversight or coaching. You get the tester and that’s it. This may or may not work well for you. If you’re an experienced QA Manager with plenty of experience managing remote teams, this might work for you. If you’re not experienced with outsourced testing or QA, you might be disappointed when you find out that you’re going to need to manage and task this person every day.

3. Testers Vs Test Engineers

it is very common to find “testers” available to help you at a relatively low cost. This might work for you if your software app is simple and you already have a QA Strategy, Test Plan and Test Cases. A “tester” can probably run those test cases for you. If you have anything other than a very simple app or don’t have a QA Strategy, Test Plan and Test Cases, hiring a “tester” will lead you to disappointment. Only a Test Engineer is educated and experienced in the art and science of software testing and quality assurance.

If you are on the alert for these Three Software Testing Traps, you’ll make a better decision with a higher degree of certainty. That’s what you’re looking for, right – a Certainty of Success?

Double Bonus – Three Tests Plus One Guarantee to Consider When Selecting a Software App Testing Company

So, now you’re clear on your software testing goals and constraints and you’re on the alert for the traps to avoid.

You’ve “pre selected” one or two testing companies that you think can help you achieve your goals.

You’ve talked with them in depth and shared all of your relevant project information.

You’ve received a “proposal” that tells you what they will do, what it will cost and how long it will take.

Here are four “tests” you can use to make sure you’re getting everything you need to be successful with the software app testing company you are going to select.

1. Repeat Your Goals and Constraints

Your proposal should clearly explain your goal and constraints. This is evidence that the software app testing company *knows* and *understands” your goals. If this is missing from your proposal, you should be concerned that they don’t clearly understand your project or goals.

2. At Least One Strategy and A Plan

Your proposal should show “the math” – just like your grammar school mathematics teachers required. Software and software testing projects are complex and there are many different ways to achieve your goals. Your proposal should show the software QA Strategy that was selected as the best approach to achieve your goals within your constraints. It also goes without saying that your proposal should also include a plan that implements the selected Strategy. Double check that the Plan actually follows the Strategy. Award bonus points if your proposal shows more than one QA Strategy and the thinking behind how the best Strategy was selected.

3. A “Fixed Price” Offer?

You probably won’t see a Fixed Price offer in your Proposal unless you ask for one. So, ask and see what their response is. This will give you an indication of how confident the software app testing company is in their Strategy, Plan, and Estimates. Be aware though, that Fixed Price projects can be more contentious to manage to completion. That said, it never hurts to ask.

4. A Success Guarantee

You almost definitely won’t see a Success Guarantee in your Proposal. But, what types of success guarantees will your software app testing company offer you? Ask and see what their response is. The is a powerful conversation to have with your software app testing company and here’s why. Not every project goes according to plan and your’s may not either. Wouldn’t you rather know how your software app testing company is going to respond now, before the crisis happens, than later when everyone is caught by surprise? You deserve to know if your testing partner is committed to your project and how they are going to help you resolve the crisis – if and when it occurs.

If a software app testing company can pass these tests, you’ve found a partner to help you achieve your goals.

If they can’t pass these tests, you’re taking on additional risks needlessly.

Share This:
Mobile Application Regression Testing

Why Regression Testing Must Be Included in Mobile Application Testing

Are your mobile apps prone to breaking after implementing a new feature or update? If so I encourage you to make mobile application regression testing a regular part of your test plan?

You see, after an update or new feature has been added there’s a really good chance that you have some new defects in your older, previously working code. Why might that happen? There are several reasons that your previously “good” mobile application now has new defects after your last update.

Why Defects Appear in “Good” Mobile Applications

  1. Changes in the mobile operating system – it is not uncommon for changes and improvements in a mobile operating system to have a negative impact on your mobile application. This isn’t intentional nor does it happen every time. Testing on a variety of mobile operating systems and devices can help you find these regression defects quickly. Knowing which operating systems and devices to test is where professional mobile application regression testing can help you save time and money.
  2. Changes in code module dependencies – most mobile applications are built by assembling a set of basic software components and then customizing how they interact with each other and your customer. These basic software components are almost always being changed and improved. When those new changes and improvements are included in your new mobile application version, errors are likely to occur. Writing test plans to specifically test mobile software component interaction will help you find these types of regression defects. Professional software test engineers know and use effective test strategies and plans to help you identify and test these “risky” areas.
  3. Human error in development or testing – humans make mistakes – sometimes knowingly and sometimes without knowing. Most regression defects caused by human error are not intentional – in fact, just the opposite! Human errors typically occur when an improvement in one area of the code has a negative impact on some other area of the code that wasn’t foreseen by your software developer. For example, improving the user experience for logging in may have an unexpected impact on how a password reset request works that cause it to fail. QA managers, as an integral part of a mobile application regression testing team, can help you better understand how your requirements/user stories can show you clues to find these types of regression defects.

Smart Mobile Application Regression Testing Begins with a Smart Test Plan

Regression testing doesn’t fit easily into an Agile software development process. And yet, going without regression testing for your mobile application can lead to loss of prospects, customers, and revenue. Working with the right outsourced mobile app testing company certainly goes a long ways to removing the risk associated with rolling out mobile app enhancements.

Go ahead, call me. 888-254-9709. Even if you just want to pick my brain, I want to hear from you. Or, if you prefer, click the blue button on this page to schedule a call back.

 

Share This:
Mobile Application Testing Company Selection

How to Select a Mobile Application Testing Company

Mobile application testing is arguably more important than regular software testing.

Here’s why. Most mobile applications are free and most software is paid for. Additionally, most mobile applications are designed to attract new customers and retain your current customers by making it easier to buy from or work with your company. Most software is used within a working environment. See the difference? Mobile applications primarily attract and retain customers where most software is used to accomplish a task.

Now, defects or bugs in ANY software are a problem but defects and bugs in mobile applications have a much higher cost than most defects in regular software.

Why?

Because people seeking help or a new source to purchase from have ZERO TOLERANCE for defects that make it hard to begin or continue doing business with your company. If they find a defect in software they’ve purchased from you, you’ll get a help desk call. If they can’t get your mobile application to work, what do they do? Think about it for a second – what happened the last time you encountered a defect in a mobile application. What did you do? Yeah, you probably closed the mobile application and went somewhere else. I do.

The cost of a defect in a mobile application is higher than the cost of a defect in most software. The cost is a lost customer or new prospect

Mobile application testing is little different from software testing in a couple of  other key areas, as well.

“Walled Garden” Software Distribution Challenges

Releasing your mobile software requires “approval” from a third party prior to listing on Apple Store or Google Play. How will this requirement impact your release dates and software testing schedule? What do you need to know to avoid Apple or Google declining your mobile app?

Inputs & Navigation Vs Display & Calculation

The intense focus on Inputs and Navigation in mobile apps requires different software testing strategies to avoid wasted time and effort. How will you know you’ve tested the most vulnerable areas in your mobile app?

Devices, Browsers, Networks and Emulators

So many and growing every day! Your mobile software will operate slightly differently on various device, browser and network combinations. Which combinations are required and which are optional? Which combinations need testing on a real device or network and which can be tested on an emulator? Making the wrong choice can cost you prospects and customers!

Test Engineers & QA Managers Vs Testers

Almost anyone that has used a mobile device “can” test your mobile app. But should they? Test Engineers coupled with a QA Manager can give you full QA coverage and Risk analysis. Testers can usually do what you tell them to.

Not every software testing company will know these key areas well enough to meet all your needs as a mobile application testing company.

Where to start?

There are many mobile application testing company choices available to you. What should you consider and where should you start to make your selection?

  1. Clearly define your quality goal and your budget for time and dollars. This will help whoever you select to build an accurate proposal/estimate. There will always be more QA work that is needed but business decisions drive mobile software projects. Time and money are key constraints that limit what can be done.
  2. Mobile application testing experience is critical. Mobile application testing is different than testing regular software and requires different tools, patterns and methods. Likewise, mobile application developers are seldom a good choice for testing your mobile application. Software test engineers are your best choice.
  3. The size and recency of the device and emulator library will dictate how successful a mobile application testing company can be for you. The wide variety of mobile devices and operating systems dictates that you make two wise choices.
    • Which devices and operating systems are most important to you and your customers?
    • Which devices does your mobile application testing company actually have? When a particular device/operating system is not available, an emulator must be used instead. Emulators are useful for some testing but almost always fail to provide the complete testing coverage that most modern mobile applications require

The Cost of a Bad Choice

You must already be convinced that testing your mobile application is important and needed – as evidenced by your visiting this page.

Congrats! Some of your competitors either don’t bother to test their mobile application or do it poorly.

Don’t believe me? Just download one of them and see how long it takes before you can crash it. Not long, right?

Here are the big costs you need to avoid when selecting your mobile application testing company.

  1. Price Vs Cost – in software testing, you really do get what you pay for. Low rates/prices are the teaser and longer projects, poor defects, missed coverage and blown out deadlines are the true cost. Compare their calculated project cost instead of the hourly rates. Then,  get a money-back guarantee to make double sure. Even better, get a fixed price for your project.
  2. Talent Vs Availability – not everyone is a talented Test Engineer. In fact, most “testers” aren’t engineers at all. If your problem can be solved by “just get it tested” then almost anyone will do – and there are plenty of those folks out there. If your goal is “on time with zero defects” then you’ll need talent – Test Engineer talent and QA Manager Talent. And yes, the price for a Test Engineer is higher but see #1 above.

The Keys to Successful Mobile Application Testing

The three key areas where mobile app testing differs from software testing are:

  1. Faster and Simpler – mobile application projects are usually faster to start and complete since they are smaller and more focused. Applications on mobile devices are also usually simpler in functionality than software applications due to limited screen real estate and uncertain connectivity.
  2. Experience and Completion – mobile apps typically focus more heavily on the user experience than most software applications. Also, clear evidence that the transactions made in your mobile application have been completed are more important due to the tendency of mobile applications to lose network connectivity.
  3. All Those Different Devices – software applications and (most) web applications run in a web browser on a desktop operating system. There are a finite number of operating systems and web browsers that need to be tested against a software or web browser application. There is an almost infinite number of mobile operating systems/browser combinations – definitely more than you could possibly test.

With these three big differences in mind, can you see how selecting the right mobile application testing company might be important in the success of your mobile application project?

Questions for Your Next Mobile Application Testing Company

Here are some great questions to ask when you’re shopping around for a company to test your mobile application.

  1. With our very fast sprints, how will you design your testing strategy to help us increase our speed?
  2. Since we have very simple functionality, how much and what type of testing do you propose to make sure we achieve our goal of Zero Defects?
  3. What strategy do you propose to test the usability of our mobile application?
  4. How will you test and validate transaction completion and notification? How will you test graceful failures?
  5. What is your strategy to help us figure out which devices to test on and which devices that an emulator should be OK?
  6. Is hard device testing necessary or will a tool or emulator be sufficient? Which tools/emulators do you recommend and why?
  7. Where does test automation fit into your strategy and why do you recommend it?

Mobile application testing is similar to but different enough from traditional software testing that a slightly different approach to selecting the best mobile application testing company can mean the difference between your success and failure.

Want to talk with someone at TESTCo about how we’d test your mobile app? Most clients, at first, aren’t even sure how much testing will be necessary. We will give very practical advice in the first call. Call and talk to me personally. 888.254.9709 Or, click the green button on this page to schedule a call back.

Share This:
Mobile Application Testing Services Choices

How Much Testing Does Your Mobile Application Need?

Mobile applications are different than web or installed applications and deserve some careful thought to make the best choices about what and how much mobile application testing services are really needed.

There is a long list to consider and surprisingly, you probably don’t need all of those mobile app testing services to achieve your goal.

Mobile application testing services can include:

  • Mobile application functional testing
  • Mobile application regression testing
  • Mobile application cross device testing
  • Mobile application test automation
  • Mobile application security and penetration testing
  • Mobile application performance testing
  • Mobile application usability testing

Goals for Mobile Application Testing Services

Here are some questions you can consider to help you determine if one of the types of mobile application testing services I listed will help you meet your needs and achieve your goal.

First, what is your goal for your mobile application?

  • How many downloads or users do you want to use it?
  • What value will your users receive from using your mobile application?
  • How will you know they have received that value?

Now that you have a basic goal for your mobile application, see if the following questions help you chose the best and most appropriate mobile application testing services.

Mobile Application Functional Testing

Does you mobile application contain complex or simple functionality?

Simple functionality with only a few inputs/outputs/screens usually doesn’t need anything other than some basic validation that the inputs/outputs and screens work and flow properly. There are few inputs/outputs/screens so the number of combinations to test will be limited and can be accomplished relatively quickly.

Complex functionality where you have more than just a few inputs, outputs and screens probably needs (and deserves) a more robust testing strategy and plan. In general, you can’t test everything (it takes too long and costs too much) so selecting a smart test strategy will be very important to make sure that all of the “Important Stuff” is tested thoroughly and frequently. Almost any qualified Test Engineer or QA Manager will have a small library of Test Strategies and can help you select the one or two that most closely match your needs and goals.

Mobile Application Regression Testing

Regression Testing helps you make sure that you didn’t break anything that was previously working well. The main goal of regression testing is to insure that you don’t disappoint or anger your mobile application users instead of delighting them with new features and functions.

Mobile applications tend to be updated frequently. One of the benefits of a mobile application is that they tend to be smaller and simpler than other types of applications and can be updated very quickly and easily. This is a tremendous benefit because it allows you to quickly add or change features based on real-time user feedback.

But, being very quick to make changes to your mobile application may also hurt you if you don’t take just a little extra time to complete an effective regression test.

Regression Testing Can Be Challenging

  1. Familiarity breeds contempt. When you’ve seen and tested your mobile applications features and functions every day for the past six months, it’s easy to let your assumptions and urgency take control. This can result in skipping or overlooking some functionality in your haste to release your new mobile application. Using an experienced and trusted third party for your regression testing will insure that nothing is skipped (or assumed to be OK) during your regression testing. “Once over lightly and quickly” is not a good regression testing strategy!
  2. Lack of test documentation. Mobile applications tend to be smaller than web applications and good test documentation is usually the first casualty. And no, your mobile application doesn’t necessarily need a full set of ISO compliant test documentation. But, a decent list of test scenarios and use cases can help you complete your regression testing quicker and with a much higher (and measurable) level of quality and customer delight. An experienced third party software test engineer can quickly and easily help you build and maintain these test documents as a natural part of helping you regression test your mobile application.

Regression testing is simple but not always easy if you aren’t an experienced software test engineer. But, getting help from a professional, at least the first couple times, can go a long way to helping you release your mobile application faster, better and cheaper.

Mobile Application Cross Device Testing

Your mobile application might be used on a new iPhone or an Android phone several years old. Almost all mobile devices behave slightly differently and the difference in behaviors between iOS and Android can be significant. If you fail to test your mobile application on a variety of mobile devices, you are inviting a catastrophe.

There are two basic methods available for cross device testing. You should consider the pros and cons of both before specifying what mobile application testing services to obtain.

Real Mobile Devices and Operating Systems

Real devices provide the best mobile application testing results since you are testing your application on the actual device. But, Real Mobile Devices are expensive to acquire and maintain. Real Mobile Devices are also slightly more time consuming to test on as they require physical handling. Finding a sufficient variety and availability of Real Mobile Devices is a big challenge as new devices and mobile operating systems are released and updated frequently – and this makes the process of building and maintaining a library of mobile devices very expensive.

Mobile Device Emulators

Emulators are usually on-line services where you can load your mobile application on a variety of specific virtual or emulated mobile devices with specific operating systems and versions. Mobile Device Emulators are very handy but have some distinct challenges. Mobile Device Emulators work well for basic functionality but are seldom useful if hardware functionality testing (GPS, Cellular Vs Wifi Networks or Device rotation) is needed to test your mobile application.

But, We Think There is a Better Way

The TESTCo Blended XDT (Cross Device Testing) – at TESTCo, we take a blended approach to using Real Mobile Devices and Mobile Device Emulators. We will help you analyze your expected customer device usage and the current market share of each mobile device and operating system and produce a matrix that compares your testing needs against Real and Emulated devices. We use this to build a testing strategy that optimizes for primarily Real Mobile Device Testing with a portion of Emulator Testing so that we can give you the highest confidence that your mobile application will work flawlessly on the devices that your customers use.

Mobile Application Test Automation

Using test automation for your mobile application can allow your software development team to build and release your software quicker. But, is development speed really a problem and do you want to invest to solve that problem?

Here’s why I ask. Test automation is the process of writing computer code (for the test automation tool to use) to test your computer code (your mobile application). So, you’ll be building another computer program (to test your mobile application) that does test your software faster but is more complex and expensive to build and maintain. Go into this one with your eyes wide open for the ongoing consequences and not just the initial expected benefit of faster testing.

Another issue with “going faster” when building and testing your mobile application is the approval and update process – which is entirely out of your hands! Sure, you can build and test your mobile application faster but you still have the issue of bottlenecks getting your mobile app approved for Apple or Google store and then, even after it has been re-approved and loaded in the Store, your chances that your users will actually download and install the update are small.

So, instead of “going faster”, you might find that “building smarter” provides you with more benefits and fewer costs. Nevertheless, building and running test automation can be valuable for your mobile application in certain situations.

Mobile Application Security and Penetration Testing

Your mobile application probably has some personal and/or confidential information on it. Hackers are penetrating, stealing and selling confidential data from mobile applications and their servers on a daily basis.  Can you afford for your mobile application to be hacked and your users mad at you?

Thankfully, most mobile applications have a small number of security and penetration points so testing them is common practice.

The security of your mobile application’s login screen probably isn’t an issue. In fact, most mobile applications are naturally quite secure in their user interface as long as the login function is well built – and most are.

The real challenges with mobile application and security lie in the way your mobile application communicates with the world around it and it’s server. Mobile applications use API calls to send and receive information to and from other users and the server in most cases. The robustness, error checking and reliability of these API calls will determine how “safe and secure” your mobile application will be for your users. Mobile application security and penetration testing evaluates your mobile applications login screen and internal functions as well as your API calls to ensure that your customers’ data is safe and secure.

Mobile Application Performance Testing

Your mobile application will need to respond and perform quickly for your customers and prospects to feel confident using your app. Poor or questionable performance can (and does) lead to poor reviews, fewer stars and rapidly decreasing downloads and usage. This probably is not your goal so performance testing may be a risk you want to consider selecting for your mobile application testing services.

There are three critical areas to consider when performance testing you mobile app.

Device Performance  – Your mobile application needs to do 2 things simultaneously – respond quickly when in use and play well among the other mobile apps on the device. Testing the first is usually straight forward. Testing the second will require a smart strategy to get accurate results quickly without wasting a lot of time with invalid test setups.

Server/API Performance – Your mobile application will need to communicate with your server to send and retrieve information. In most cases, the server application is separate from your mobile app and interacts with your mobile app through an Application Programming Interface (API). This server app will tend to behave like most server-based applications and is subject to the same performance testing with one exception. The APIs need special attention to a wide variety of performance loading criteria to make sure the mobile apps that depend on them are not needlessly bogged down. Make sure to remember to test the server under load as well – not just the server application!

Network Performance – There are almost too many mobile networks to even list anymore. Mobile devices communicate either via WiFi or the carrier’s mobile data network. In all cases, even Wifi, the network speed and capacity are going to vary wildly. 2G and 3G networks still exist in many places in the world and the continental United States. Many WiFi hotspots are still using the older 802.11(a) and (b) technology. Network performance testing is tricky and it is critical to understand the user’s environment and goals before building a network performance testing plan.

Lastly, all performance testing requires detailed data analysis to clearly identify and truly understand the performance bottlenecks. Performance testing typically produces massive amounts of data that needs to be analyzed to be understood. Performance Test Engineers specialize in this data testing and analysis and have spent many years learning and building their knowledge.

Mobile application performance testing can be tricky to plan and execute so that you get the full value of the work. There is a lot of performance testing that can be done but typically only a few of the tests are truly valuable to any certain situation. The key is to be perfectly clear on the problems you want to solve and the risks you want to reduce. This approach allows your Test Engineers to perform the exact tests you need without wasting time and money on unneeded mobile app performance tests.

To Be Continued

In the coming weeks I will shed light on the five other types of mobile application testing so you can make smarter testing decisions. In the meantime, check out this post if want tips on how to select a mobile application testing company.

If you can’t wait for me to finish this post (hey, I can’t write all day, I have a business to run), call me (888-254-9709), or request a call back using the green button on this page. I’m happy to answer your questions.

Share This: