Mobile Testing Strategy

The Difference Between a Mobile Testing Strategy and a Mobile Testing Plan

In This Post

 

There is a big difference between a mobile testing strategy and a mobile testing plan. If you know the difference, your mobile software testing will be much easier and more valuable.

There are a number of ways of defining a strategy. One of the definitions that we’ve found useful is:

“Identify assets that have advantages that you can put into coordinated action to achieve your goal

Or, said another way, list and organize everything you think you’ll need to achieve your mobile testing goal and why you think you need it.

Your goal should be clear enough that you can quickly see what is, and is not, relevant to achieving your goal.

For example, if your mobile testing goal is to release your game every week with new features and fixes, your strategy might be different from a goal to release a zero defect trading app. Already, you can start to see that you might need different Assets for each of those goals.

Your Goal Will Dictate Strategy

One of the first challenges in building your mobile testing strategy will be to clarify and tighten your mobile testing goal. Your goal should clearly show what is and is not important to include in your mobile testing strategy.

The biggest and best tool you can use to uncover gaps or weaknesses in your mobile testing goal will be relentless Critical Thinking Questions.

“Why is this important and what, exactly does it mean?” is one of the most powerful questions you can ask about your mobile testing goal.

For example, which devices should you test against? The answer depends on which devices your current customers or prospects are most likely to be using. You know that mobile apps behave differently on different devices and operating systems, right? You won’t be able to afford to test against every device out there – you’re going to have to make some assumptions and then test their validity.

The more you can refine your mobile testing goal by clearly understanding what is directly relevant (need to have) and which things are “nice to have”, the more effective your mobile testing strategy will be.

Wait Just a Second – Why Bother with a Goal?

You don’t really have to have a goal to build a mobile testing strategy.

Your strategy could be “just test it”,or “get folks to bang on it a bunch”, or even “meh, just test it enough to make sure it doesn’t crash”.

Those are all valid mobile testing strategies and they have even been successful strategies – a couple times.

Here’s the problem that happens when you don’t have a goal.

You Don’t Know When You Are Done.

So, unless you want to test forever or don’t really care about your app or your customers, you’ll probably benefit from the 10-15 minutes it will take you to build a good goal for your mobile app.

A List of Things to Decide for Your Mobile Testing Strategy

Since you’ve made it this far, you probably realize that your questions are far more important than your answers!

Here is a list of questions that you can use to help you narrow your focus and build an effective mobile testing strategy:

  1. Which devices are important to my target audience?
  2. Do I have to have the real device or can I use an emulator?
  3. Can I use a tool for testing and do I have the experience to use it properly? Which tool will meet my needs and how can I tell that?
  4. What network connectivity is required for my app? Wifi? Cellular Data?
  5. Which parts should be automated testing and which should be manual testing? What’s the proper balance and how will I know?
  6. Will I need performance testing for my mobile app? Why would I need performance testing and which parts of my mobile app are subject to network congestion?
  7. What are the security requirements for my mobile app and how do they need to be tested? What is the probability and impact of a security breach of my mobile application?
  8. What sort of UX testing do I need and who is best qualified to give UX feedback on my mobile app?

Whew! Those are not easy questions!

And yet, you do need to know those answers if you want to avoid a “just get it tested” type of strategy.

So, is there an easy way to get started and build a great mobile testing strategy?

Additional Considerations when Selecting a Mobile Application Testing Company

When customers are selecting a mobile application testing company to work with, there are several key factors they should evaluate; failing to do due diligence on your mobile application testing company of your choice can lead to bad quality control and an overall negative experience with your app. A good mobile testing company will improve the functionality, usability and security of your app in general.

First, you should ensure your mobile testing provider has trained mobile testers or test engineers. This should be a given, but dealing with companies that don’t have trained mobile testers/engineers can be very expensive in the short and long-term health of your app. A company with competent mobile testers will have expertise in functional and usability testing. Additionally, companies with competent mobile testers will ensure that test optimization is done based on the current mobile device market.

Also, the company you work with should have the correct resources for your app’s testing. This can include a lab for device, emulator and simulator based testing. It’s important to talk with your provider about the specific tests you’re looking for and ask about any other testing they provide and how that can benefit your application. You should mention the platforms and operating systems that your app primarily runs on to ensure all testing is done across all verticals. The most important resource that a provider should have is test engineers, QA managers and other staff. For many businesses, it doesn’t make financial sense to hire more team members to their QA team so working with a provider that has all of the expertise you need is vital. With an expert QA team at your disposal, you can get daily status reports on your mobile testing and solve any issues that arise in the meantime.

Lastly, it’s key to work with a provider that understands your business goals and tailors their service to fit your needs. This seems intuitive, but when you get on a phone call with a provider, they should take some time to figure out your business model, specific application needs, etc. This will allow your provider to form the right QA coverage for your product and you can have a rough outline of how their services can fit your needs

A Simple Strategy is Simple!

One of the most common mistakes about strategies is that they are complex. Sure, your strategy can be complex but it doesn’t have to be complex. It depends mostly on the complexity and difficulty of your goal.

We prefer Simple Strategies and we can help you have one too.

For example, if your goal is to build a full-featured trading app for mobile devices, your strategy might be quite complex. But, if your goal is to launch an app that trades only 1 stock, your strategy might be simpler.

Our rule of thumb is that a strategy can be easily explained with three sentences or less. Sort of like this:

  1. we’re going to achieve THIS GOAL
  2. with these CONSTRAINTS
  3. by doing A and B at the same time and then C next and D last.

It should be clear and understandable to everyone involved in your project that doing A and B at the same time, followed by C then D, will achieve the GOAL within the CONSTRAINTS.

If it is not clear and understandable, then dig a little deeper, find the unknown or un-agreed upon aspect, and build a tighter, clearer strategy for your mobile testing project.

If you skimp on your strategy, you can expect to have problems with your plan.

If you invest wisely, and build a powerful but simple strategy, your plan will be obvious and tracking your mobile testing project will be very easy.

And, one more note about building a simple strategy – expect to rebuild it at least once.

Here’s why – the most unknown and uncontrollable part of your strategy will be the Environment. The Environment is everything around you that you can’t control. These can be as simple as a misunderstood requirement or an emergency family leave for a critical team member. They can also be as complex as a major business setback or the cancellation of a critical software license. These are all aspects of the “Environment” you’re working in and can have a big impact on your strategy. When the Environment changes, re-evaluate your strategy and rebuild it if the changes impact your mobile testing strategy .

Start with a Free Scoping Session with Me

I know, that sounds generous and it is.

And, we can do that for and with you in one of our standard 60-90 minute Scoping Sessions.

Most of our clients and prospects don’t have a quality goal prior to meeting with us. Only one or two have ever had a quality strategy when they first came to us.

So, we’ve become very adept at helping our clients and prospects define their quality goal and build an effective testing strategy.

We do this by gently asking you a lot of questions. Sure, we want to know the details about your mobile app and how you want it tested. We can and will do that for you.

And, we want to make sure we provide you with everything you need to achieve your goal. So, the first thing we do for you is clearly define your goal and then propose a couple of different strategies for you to consider. We’ll always recommend at least one of the strategies but we want you to know that there are different ways to achieve your goals and that you have the last word on how we work for you to achieve your goal.

If you’re curious about how a mobile testing strategy can improve your mobile app testing, achieve your goals and build your business stronger, please reach out and let’s talk about how we might help you. Send the form on this page to schedule a call, or give me a shout now at 888.254.9709 .

QA as a Service | QAaaS

QA-As-A-Service, The Missing Link For Product Managers

In This Post

QA-as-a-Service for Product Owners is more than software testing. QAaaS provides fast and easy development testing plus your business goals delivered on time and with eliminated or reduced risks.

QAaaS is a flexible way to bridge the gap between freshly coded features and your customers’ needs and desires. QAaaS is capable of testing a wide range of software systems, platforms, and applications in real-world environments and using real-world user and test cases. It uses agile and flexible processes across the entire software development life cycle from Alpha and Beta phases, through production. It is highly focused on needs and encompasses large-scale system testing as necessary.

TESTCo was founded in 2002 to serve the unique requirements of Software Product Managers. Our QA Managers and Test Engineers know the services and procedures needed to support every aspect of Quality Assurance as a Service.

You can take advantage of this experience right now, just by downloading the Product Manager’s Check List for QA.

Quality as a Service Check List

Find and Eliminate Risks Before They Become a Problem

Risk remediation is a biggie. Product Managers (and their bosses) don’t like surprises. Therefore, our QA-as-a-service reduces surprises by cutting risk. Here’s how:

  • We simplify complexity so that everyone can understand it.
  • We measure real progress towards goals and release dates.
  • We trap and control blocking issues before they stop your project.

A good example is regression testing: Nobody wants to wait through a long regression testing cycle, but releasing without one is very risky. We recommend fast regression testing through release, so there aren’t big surprises on the eve of the target release date. 

Regression testing allows you to:

  • Prevent functionality crashes: When new features are added to a product or platform, several difficulties can arise due to misalignment between the new features and old features. With regression testing, functionality crashes that can be detrimental to your product are prevented
  • Detect all bugs and defects: Regression testing also identifies all bugs and defects before they make an impact on the product deployment. These harmful bugs can be prevented, assessed and fixed before deployment. This leads to less time and money on efforts related to resolving bugs; this directly reduces the time to market for the product.
  • Continuous integration: If you have a platform or product that needs continuous integration, regression testing is vital. Performing regression tests after incorporating new code ensures that any new changes don’t affect the current performance of a product. As a product has more code integrated over time, regression tests become more beneficial.

What Concerns Product Managers

Product Managers care about more than “testing,” and some haven’t had the opportunity yet to see how the advantages of a QAaaS company can help them achieve their customer goals.

Here’s what concerns Product Managers.

  • “Oh, we should have caught that in grooming.”
  • “Our stories (storyboards) are missing something but I’m not sure what.”
  • “What are the real acceptance criteria?”
  • “QA is just an expense or tax that we want to minimize.”
  • “We’re not sure what DONE really means yet.”
  • “We need more QA, not just testing.”
  • “Are we testing the user experience enough?”

Additional concerns for product managers include doing enough customer validation to ensure the product being built is truly useful and can be validated by the customer. Customer validation drives product and platform success because it provides valuable insights before launch. Without customer validation, it’s easy to overlook features and other requirements that the customer wants.

In addition to customer validation, tracking KPIs is crucial. Tracking KPIs allows product managers to assess if new features or changes have improved or weakened any aspects of the product. Tracking KPIs exposes strengths and weakness while setting expectations and improving overall management. KPIs are some of the few ways product managers can quantify progress, so tracking it is extremely beneficial.

Lastly, it’s important to ensure that QA has all the possible use cases and protocols to verify no additional bugs are introduced. With this, QA is well-equipped with a plan of action to handle any difficulties that emerge like bugs.

When QA as a Service is Missing

Each of these Product Managers was missing something (or someone) in the QA function that if present could have had a very positive impact on the product’s goals. QAaaS provides everything you need to bridge the gap and ensure your product delight your customers.

Some problems that can arise from lack of QA include bugs and “fire” based problems. These problems can be detrimental because it can halt a customer from using a product or platform. By using QA, any bugs can be detected before they cause significant issues to a product.

QA can also be more value-oriented. It’s known that QA can detect bugs and perform similar tasks, but QA can power work in solutions architecture. This can turn QA into a deploy able utility for sales teams and customers. This helps shift QA past being a part of the product definition phase to being more utilizing in other aspects of the business. This in turn makes QA directly generate value independently (from a non-functional requirement to a functional requirement).

A Product Manager’s Dream Come True

QA-as-a-service comes through in our complete understanding of software company priorities, goals, and constraints. By gathering information upfront, we form and make recommendations, test plans, and feedback targeted to the client’s needs. Product Managers get their products delivered faster, cheaper, and better.

With TESTCo, project managers can easily communicate with QA teams. In the case of an urgent bug that stops the platform, project managers can let QA teams know exactly what problems they are experienced and the QA team will focus on fixing the bug until the platform works properly. QA teams can start addressing the specific bug at hand within a matter of hours after contact.


Testing What Users Really Want

There is much more to accurately and effectively testing a software project than just running the tests! Our Product Manager clients consistently tell us that their top priority is a valuable user experience. Yes, this does require testing to make sure nothing goes wrong – the app doesn’t crash and it provides useful information. But, there’s another aspect to user testing that’s frequently overlooked. User Value Testing – does the software project truly deliver on the promise and prompt them to return for more? TESTCo provides this as a regular part of our strategy.


Easy and Painless

TESTCo never outsources software QA testing work. Keeping the job in-house makes reaching the results both easy and painless.

Projects Delivered on Time

Test plans are accurate, followed with care and precision, and projects are delivered on time. Sprint Goals, exit criteria, and key performance indicators are clear to prevent the team’s attention and efforts from wandering. Everyone knows when they are “done” because that point is clearly defined.

Faster Development Cycles

The developers are fast when they have two things: crystal-clear stories and requirements, along with inline and real-time validation. TESTCo Test Engineers test and validate work as soon as it is completed, based on the stories and requirements.

Achieve Business Goals

Our QA-as-a-Service is designed to support the business goals of the product. You can expect us to ask what the goals are before we start. QA helps Product Managers meet the expectations of the business and their customers. QA drives the testing before the processes start and are ready when you are. 

Find and Eliminate Risks Before They Become a Problem

Risk remediation is a biggie. Product Managers (and their bosses) don’t like surprises. Therefore, our QA-as-a-service reduces surprises by cutting risk. Here’s how:

  1. We simplify complexity so that everyone can understand it. 
  2. We measure real progress towards goals and release dates.
  3. We trap and control blocking issues before they stop your project. 

A good example is regression testing: Nobody wants to wait through a long regression testing cycle, but releasing without one is very risky. We recommend fast regression testing through release, so there aren’t big surprises on the eve of the target release date.  

When the Phone Rings

When a problem occurs, or the situation gets tense and time is rapidly sliding by, Product Managers give us a call to line up a Test Engineer/QA Manager team to step in and help get the hot-spot cooled off. They appreciate TESTCo’s ability to staff a Test Engineer and QA Manager for as little as a day and as quickly as tomorrow. 

Why let those problems linger?  And least you think our being extra valuable to Product Managers means a higher price tag, you can relax. We offer comprehensive solutions that increase ROI. 

How QAaaS Can Fit into Your Team

Your goals are our goals! Bringing an outsourced team into your software project team can be a big challenge. At TESTCo, our goal is to fit seamlessly into your team and processes without any disruptions. Our scoping process helps us uncover all the information we need to build a strategy to work with your team without disruptions! At a minimum, you’ll hear from us twice daily. But, if your team has stand up meetings, our Test Engineers and QA Managers sure like to be included.

Must-Haves To Elevate QA’s Contribution

When QA managers have access to accurate information and general business objectives, they are in a better position to provide value. Some of the ways to improve QA’s contribution to your business includes:

  • An understandable report with the exact features completed and tested every day. It’s important to emphasize that the features must be completed and tested twice daily.
  • A detailed understanding of your business and technical goals; this should include the current daily status of your sprint and your specific priorities in your business
  • A good foundation of regression testing along with measurement of coverage. This should be old-fashioned test cases built over time and then consequently run as a final regression prior to release. This will prevent surprise defects after launch.
  • Weekly product quality meetings with project QA manager: During these meetings, software development, quality measures and processes will be reviewed. This will prevent missing estimates and buggy code.
  • Daily software testing by a test engineer.

Maximizing Your QA Manager

Having great communication with the QA manager is vital because they are deeply ingrained into the projects and testing teams. QA managers play different roles and can serve many functions for your business. Incorporating your QA manager in decision-making when it comes to QA teams, hiring, etc. can be very beneficial due to their insights. Some avenues of maximizing your QA manager include:

  • Project leadership: QA managers can help you understand risk areas, identify goals and give you an estimate of what a QA team can produce for your business. The QA manager is involved in almost all projects, so they are the best resource for project leadership. QA managers are knowledgeable on client fears, risk areas and any technical concerns that come with certain projects. By utilizing the QA manager for input on projects, you’ll always have a good idea how a project is going and what are some potential difficulties that can arise.
  • Education: QA managers are a great resource when educating engineers, testers and any other members of the QA team. QA managers know what skills and knowledge the QA team has and how new training can be incorporated into the team while performing normal day to day functions for clients. Consulting with QA managers prior to any education or training programs for your business is crucial. Their insight can provide you the most cost-effective ways to educate your staff
  • Recruiting: Additionally, QA managers can help recruit talented engineers to reach your business objectives. QA managers are the best resource at assessing talent for your QA team. They know the specific tasks that are demanded from the QA team on a day to day basis; this is very beneficial in interviews because the QA manager can assess talent on the spot. Assessing talent is not only limited to technical skills, aspects like culture-fit and soft skills are important to finding the best fit. The QA manager also knows where to find the best talent for your QA team locally and remotely; this can include professional organizations, schools, etc.

How might we help you deliver your software faster, better and cheaper? I’d love to hear from you. Give me a call at 888-254.9709, or request a call back using the green button on this page.


Thanks to Richard Howes for contributing to this post - JCH