Mobile Application Testing Services Choices

How Much Testing Does Your Mobile Application Need?

Mobile applications are different than web or installed applications and deserve some careful thought to make the best choices about what and how much mobile application testing services are really needed.

There is a long list to consider and surprisingly, you probably don’t need all of those mobile app testing services to achieve your goal.

Mobile application testing services can include:

  • Mobile application functional testing
  • Mobile application regression testing
  • Mobile application cross device testing
  • Mobile application test automation
  • Mobile application security and penetration testing
  • Mobile application performance testing
  • Mobile application usability testing

Goals for Mobile Application Testing Services

Here are some questions you can consider to help you determine if one of the types of mobile application testing services I listed will help you meet your needs and achieve your goal.

First, what is your goal for your mobile application?

  • How many downloads or users do you want to use it?
  • What value will your users receive from using your mobile application?
  • How will you know they have received that value?

Now that you have a basic goal for your mobile application, see if the following questions help you chose the best and most appropriate mobile application testing services.

Mobile Application Functional Testing

Does you mobile application contain complex or simple functionality?

Simple functionality with only a few inputs/outputs/screens usually doesn’t need anything other than some basic validation that the inputs/outputs and screens work and flow properly. There are few inputs/outputs/screens so the number of combinations to test will be limited and can be accomplished relatively quickly.

Complex functionality where you have more than just a few inputs, outputs and screens probably needs (and deserves) a more robust testing strategy and plan. In general, you can’t test everything (it takes too long and costs too much) so selecting a smart test strategy will be very important to make sure that all of the “Important Stuff” is tested thoroughly and frequently. Almost any qualified Test Engineer or QA Manager will have a small library of Test Strategies and can help you select the one or two that most closely match your needs and goals.

Mobile Application Regression Testing

Regression Testing helps you make sure that you didn’t break anything that was previously working well. The main goal of regression testing is to insure that you don’t disappoint or anger your mobile application users instead of delighting them with new features and functions.

Mobile applications tend to be updated frequently. One of the benefits of a mobile application is that they tend to be smaller and simpler than other types of applications and can be updated very quickly and easily. This is a tremendous benefit because it allows you to quickly add or change features based on real-time user feedback.

But, being very quick to make changes to your mobile application may also hurt you if you don’t take just a little extra time to complete an effective regression test.

Regression Testing Can Be Challenging

  1. Familiarity breeds contempt. When you’ve seen and tested your mobile applications features and functions every day for the past six months, it’s easy to let your assumptions and urgency take control. This can result in skipping or overlooking some functionality in your haste to release your new mobile application. Using an experienced and trusted third party for your regression testing will insure that nothing is skipped (or assumed to be OK) during your regression testing. “Once over lightly and quickly” is not a good regression testing strategy!
  2. Lack of test documentation. Mobile applications tend to be smaller than web applications and good test documentation is usually the first casualty. And no, your mobile application doesn’t necessarily need a full set of ISO compliant test documentation. But, a decent list of test scenarios and use cases can help you complete your regression testing quicker and with a much higher (and measurable) level of quality and customer delight. An experienced third party software test engineer can quickly and easily help you build and maintain these test documents as a natural part of helping you regression test your mobile application.

Regression testing is simple but not always easy if you aren’t an experienced software test engineer. But, getting help from a professional, at least the first couple times, can go a long way to helping you release your mobile application faster, better and cheaper.

Mobile Application Cross Device Testing

Your mobile application might be used on a new iPhone or an Android phone several years old. Almost all mobile devices behave slightly differently and the difference in behaviors between iOS and Android can be significant. If you fail to test your mobile application on a variety of mobile devices, you are inviting a catastrophe.

Cross device testing helps your mobile device safeguard against the following.

Customer loss: Cross testing is vital to ensuring quality assurance across all platforms that your application is on. If everyone was using the same device with default graphics, this wouldn’t be an issue. Users today use a variety of devices across several platforms; cross testing across platforms ensures users from all ends have the same experience. If the user interface on one platform performs poorly, customers will likely leave the app quickly.

Issues that arise from upgrades: Mobile applications are updated very frequently and these updates will eventually cause several bugs. Cross device testing will help you identify bugs and how to deal with them. Failure to recognize this might cause significant bugs that can cause your application to be unusable until it is resolved. This can also lead to compatibility issues after design changes as well.

SEO and additional rankings from Google: For mobile applications, ranking on search engines is crucial to gain traction to your site and mobile app. Since 2015, Google has taken a mobile-first indexing approach. This means that the algorithm for Google search results takes cross device compatibility into account and ranks accordingly. If a competitor to your app is compatible with all platforms and devices, they will significantly outrank you.

There are two basic methods available for cross device testing. You should consider the pros and cons of both before specifying what mobile application testing services to obtain.

Real Mobile Devices and Operating Systems

Real devices provide the best mobile application testing results since you are testing your application on the actual device. But, Real Mobile Devices are expensive to acquire and maintain. Real Mobile Devices are also slightly more time consuming to test on as they require physical handling. Finding a sufficient variety and availability of Real Mobile Devices is a big challenge as new devices and mobile operating systems are released and updated frequently – and this makes the process of building and maintaining a library of mobile devices very expensive.

Mobile Device Emulators

Emulators are usually on-line services where you can load your mobile application on a variety of specific virtual or emulated mobile devices with specific operating systems and versions. Mobile Device Emulators are very handy but have some distinct challenges. Mobile Device Emulators work well for basic functionality but are seldom useful if hardware functionality testing (GPS, Cellular Vs Wifi Networks or Device rotation) is needed to test your mobile application.

Also, mobile device emulators can’t emulate the performance of an app when it receives incoming calls, text messages or other notifications. This is a crucial issue because everyone with mobile devices gets numerous notifications every hour. If an app has to crash each time a notification occurs, this would make it unusable. Additionally, mobile device emulators can’t simulate other aspects like battery issues, touchscreen issues and more. These are some of the most fundamental aspects of using a mobile device.

But, We Think There is a Better Way

The TESTCo Blended XDT (Cross Device Testing) – at TESTCo, we take a blended approach to using Real Mobile Devices and Mobile Device Emulators. We will help you analyze your expected customer device usage and the current market share of each mobile device and operating system and produce a matrix that compares your testing needs against Real and Emulated devices. We use this to build a testing strategy that optimizes for primarily Real Mobile Device Testing with a portion of Emulator Testing so that we can give you the highest confidence that your mobile application will work flawlessly on the devices that your customers use.

The TESTCo Blended approach takes advantage of the benefits of both real mobile devices and emulators while covering the blind spots of each approach. This gives your mobile application testing a comprehensive approach that would simulate what would happen to your application in the real world. This ensures that your mobile application will work seamlessly during its first encounter with users doing normal functions on their phone like using it with low battery, taking calls while using your app and more.

Mobile Application Test Automation

Using test automation for your mobile application can allow your software development team to build and release your software quicker. But, is development speed really a problem and do you want to invest to solve that problem?

Here’s why I ask. Test automation is the process of writing computer code (for the test automation tool to use) to test your computer code (your mobile application). So, you’ll be building another computer program (to test your mobile application) that does test your software faster but is more complex and expensive to build and maintain. Go into this one with your eyes wide open for the ongoing consequences and not just the initial expected benefit of faster testing.

Another issue with “going faster” when building and testing your mobile application is the approval and update process – which is entirely out of your hands! Sure, you can build and test your mobile application faster but you still have the issue of bottlenecks getting your mobile app approved for Apple or Google store and then, even after it has been re-approved and loaded in the Store, your chances that your users will actually download and install the update are small.

So, instead of “going faster”, you might find that “building smarter” provides you with more benefits and fewer costs. Nevertheless, building and running test automation can be valuable for your mobile application in certain situations.

Mobile Application Security and Penetration Testing

Your mobile application probably has some personal and/or confidential information on it. Hackers are penetrating, stealing and selling confidential data from mobile applications and their servers on a daily basis.  Can you afford for your mobile application to be hacked and your users mad at you?

Thankfully, most mobile applications have a small number of security and penetration points so testing them is common practice.

The security of your mobile application’s login screen probably isn’t an issue. In fact, most mobile applications are naturally quite secure in their user interface as long as the login function is well built – and most are.

The real challenges with mobile application and security lie in the way your mobile application communicates with the world around it and it’s server. Mobile applications use API calls to send and receive information to and from other users and the server in most cases. The robustness, error checking and reliability of these API calls will determine how “safe and secure” your mobile application will be for your users. Mobile application security and penetration testing evaluates your mobile applications login screen and internal functions as well as your API calls to ensure that your customers’ data is safe and secure.

Penetration testing can be done against different types of code and systems in your mobile application like servers and APIs. Depending on your goals for the pen tests, there are several phases within each pen test to ensure that the security of your application is covered from all angles. After a pen test is done, a formal report will be given to you that covers all the vulnerabilities that were found and how hackers can use them to gain access to your application’s data. In contrast to many systems and processes of your mobile application, penetration testing is largely a manual process. There are testers that overlook the entire process to make sure that your mobile application cannot be breached.

Mobile Application Performance Testing

Your mobile application will need to respond and perform quickly for your customers and prospects to feel confident using your app. Poor or questionable performance can (and does) lead to poor reviews, fewer stars and rapidly decreasing downloads and usage. This probably is not your goal so performance testing may be a risk you want to consider selecting for your mobile application testing services.

There are three critical areas to consider when performance testing you mobile app.

Device Performance  – Your mobile application needs to do 2 things simultaneously – respond quickly when in use and play well among the other mobile apps on the device. Testing the first is usually straight forward. Testing the second will require a smart strategy to get accurate results quickly without wasting a lot of time with invalid test setups.

Server/API Performance – Your mobile application will need to communicate with your server to send and retrieve information. In most cases, the server application is separate from your mobile app and interacts with your mobile app through an Application Programming Interface (API). This server app will tend to behave like most server-based applications and is subject to the same performance testing with one exception. The APIs need special attention to a wide variety of performance loading criteria to make sure the mobile apps that depend on them are not needlessly bogged down. Make sure to remember to test the server under load as well – not just the server application!

Network Performance – There are almost too many mobile networks to even list anymore. Mobile devices communicate either via WiFi or the carrier’s mobile data network. In all cases, even Wifi, the network speed and capacity are going to vary wildly. 2G and 3G networks still exist in many places in the world and the continental United States. Many WiFi hotspots are still using the older 802.11(a) and (b) technology. Network performance testing is tricky and it is critical to understand the user’s environment and goals before building a network performance testing plan.

Lastly, all performance testing requires detailed data analysis to clearly identify and truly understand the performance bottlenecks. Performance testing typically produces massive amounts of data that needs to be analyzed to be understood. Performance Test Engineers specialize in this data testing and analysis and have spent many years learning and building their knowledge.

Mobile application performance testing can be tricky to plan and execute so that you get the full value of the work. There is a lot of performance testing that can be done but typically only a few of the tests are truly valuable to any certain situation. The key is to be perfectly clear on the problems you want to solve and the risks you want to reduce. This approach allows your Test Engineers to perform the exact tests you need without wasting time and money on unneeded mobile app performance tests.

To Be Continued

In the coming weeks I will shed light on the five other types of mobile application testing so you can make smarter testing decisions. In the meantime, check out this post if want tips on how to select a mobile application testing company.

If you can’t wait for me to finish this post (hey, I can’t write all day, I have a business to run), call me (888-254-9709), or request a call back using the green button on this page. I’m happy to answer your questions.

Share This:
Website Testing Services

Website Testing Services for the Average Bear

When it comes to website testing services, you might not be smarter than the average bear. Or, you might not know if you are.

That’s OK, I can help you sort through some of the techno-babble about getting some help with website testing so you can quickly become smarter than the average bear.

You see, when it comes to software testing, hunting, farming, being a Dad, or training a dog I am smarter than the average bear.

If you ask me about psychology, event management or finance then I’m definitely not smarter than the average bear. Best we stick to the subject of website and web app testing right now, okay?

Cutting Through the Clutter of Website Testing Services

As I scanned internet articles to see what was being written on the subject, I noticed a few things that could be confusing or misleading to the average bear.

Let me sort through those things now to help you find an outsourced website testing company that work best for your unique need and business goals.

Set the Right Goal

First, set a goal. Know what you want.  A SMART goal  is great – even if you don’t know all of it.

But, you may not have a “goal” and that’s OK, too. Use these simple questions to figure out what you really want from your website testing services.

  1. Do I want website testing because it’s something I “should” do or because I’m concerned about the cost of losing customers and prospects to bugs and glitches? This answer will help you (and us) understand why you want website testing services.
  2. What does it cost me to get a new prospect to visit my website or eCommerce store? What is the lifetime value of one of my customers? This answer will help you understand the risk of loss from bugs and glitches that website testing can help you prevent.
  3. Do I have time before I need to release my website to fix the bugs and glitches that the website testing finds? This answer will help you understand if you will get real value from your website testing work.

There are not any right or wrong answers to these questions. The answers are important but the thinking and reasoning you do while answering those questions will shed light on what you really want and we can easily help you turn that into a website testing services goal.

Here’s why knowing your goal is important. Testing is an activity – and it can be performed for as long as you want it performed – regardless and without respect to your goal.  You’re probably more interested in the outcomes of testing – defects, bugs, test plans, test cases, etc – than just knowing someone is testing it. If you know your goal before you start looking for help with website testing services, you’ll get the outcomes you’re looking for. Otherwise, you’ll definitely get some “testing” but you won’t know if it is helping you achieve what you actually want.

Don’t Be Over-Sold by Tool Vendors

Second, Tools! (Hat tip to Tim the Toolman, who always got excited about tools and machines.

Almost everything I read on the internet about website testing services seems to be about how a great tool can solve your problem.

Yeah, like having a hammer get’s you a birdhouse. Not so much. Tools are only as good as the person using them. So, this begs two questions.

  1. How long will it take to learn how to use the tool?  They seem easy in the demo but have you actually tried using it?  It may not be as easy as you think.
  2. Even if you know how to use the tool, will you know where to use it most effectively? Just because you can easily and quickly run a bunch of tests, how will you know they are the right tests applied to the right place in your website?

Humans are more important than tools. Tools help humans be more useful.

The Human Element in Website Testing Services

Third, Testers! Now we’re getting to the heart of the matter.

I’ve learned that there is a difference between a Tester and a Test Engineer. We don’t hire testers and you should look very carefully before you hire them yourself.

Here’s why.

Testers do very well when they are told what to do. They don’t do so well when given a testing problem to solve.  They just don’t have the mindset, education and ongoing training to come up with good alternatives or judgment to select the best course of action.

Test Engineers are seekers, finders, and solvers of quality problems. They have university level degrees in math, engineering, and computer science and are actively engaged in ongoing training in quality assurance and control. Sure, they cost a little more – so does a nice car – for a great reason. They both deliver more value per dollar than a less expensive choice – like a tester.

Name Dropping in the Buzzword Jungle

Fourth, Buzzwords like Services, Technology, Systems, Industry, Functional, Performance, Security, Usability can lure you into a trap. Remember, you’re hiring website testing services, not a copywriter.

When it comes to website testing services, having deep knowledge and experience in some set of buzzwords is far less valuable than having seen and completed hundreds of website testing projects.

TESTCo is focused solely on providing the best website testing services for your business. The buzzwords used to sell you on testing services won’t help fix bugs in your code or ensure your application works well when it’s released. We have decades of experience when it comes to solving testing problems that are similar to yours. Instead of writing about buzzwords to entice you to select TESTCo for services, we are focused on providing real testimonials and consultations so you can see our services firsthand.

If you’re looking for a match between your situation and what the website testing services company can provide, look to the references and testimonials and give them about 10 times the weight you’d give to their buzzwords matching your buzzwords. Even better, reach out and talk to the reference/testimonial yourself!

Trust Your Brand to a Crowd of Strangers? Really?

Fifth, there’s the whole crowdsourcing thing.

Sure, some things might be better, faster and cheaper when crowdsourced. But, when what you’re interested in crosses the line from “opinion” to “expert advice”, you’re probably going to get better and more valuable results from an expert rather than “the crowd”.

I know (and so do our clients) that a dedicated QA Team with a QA Manager and a Test Engineer, even for a tiny project of a day or so, can deliver a far more valuable set of results than a semi-random group of internet users.

Crowdsourcing your brand can lead to unpleasant outcomes.

Customer Churn: The entire focus of testing is to make sure that your product is as smooth as possible when it’s released. Relying on crowdsourcing can lead to significant customer churn due to unresolved bugs. All of the marketing efforts it took to make users choose your product will be wasted if there are bugs whenever your release a new version of your product.

Technical issues: Crowdsourced testers are compensated if they find bugs or issues within your application. Generally, most testers are not as qualified as test engineers and cannot deduce the core root issues within your application. Hiring crowdsourced testers may lead to you overlooking the fundamental issues that are causing bugs. This will continue repeatedly until the core issues are resolved.

Confidentiality: One of the biggest drawbacks of crowdsourcing your testing is the breach and lack of confidentiality. When you crowdsource your testing, there’s no incentive for testers to be loyal to your brand. This can lead to the leakage of sensitive business information and other trade secrets. A competitor can offer one of your crowdsourced testers money in exchange for critical information. This can lead to the demise of your business.

Automate All the Things!

Lastly, there’s automation. Automation really means using test automation, AI or Machine Learning software to perform a task. If you believe what you read, then literally everything can be automated – especially “low skill” activities like data scrubbing, sending emails and software testing.

There are 2 challenges to successfully automating anything with these tools.

  1. Automation Tools don’t run themselves. Someone has to instruct or configure the tools. Even then, an incorrect or old instruction can cause a bucket load of False Positives. This might cause you to spend more time diagnosing and fixing your automation tool than just performing the task with a human in the first place. If you’re going to use Test Automation, you might want to consider that you’re writing more software to test your current software. Who’s going to watch the watcher?
  2. Website Testing is a “low skill” activity. I beg to disagree with this idea. Yes, Testers, doing what they are told and manually running a known set of regression tests for the 32nd time is a low skill activity that should be automated. But, that’s not a common problem and probably not your problem. Most website testing challenges aren’t known and aren’t covered by a test case that you know about ahead of time. Most of the defects discovered during website testing aren’t covered by a test case. Testers will seldom find these defects because they aren’t obvious. Test Engineers, however, will definitely find them! Quality and Test Engineering are definitely not “low skill” activities.

Automated testing provides several hurdles for business prior to even getting testing started.

Initial cost: The initial cost of using automated testing is very high. For automated testing to work properly for your products, it will have to be configured and tailored to your business operations. This configuration along with the staff needed to make this happen often costs more than the budget for testing. In large corporations this may be cost-effective, but for the average business this is very detrimental.

Maintenance: Automated testing isn’t a one-time service, it requires continuous upgrades and maintenance just like many parts of your business. The key difference in maintaining automated testing is that it will likely require additional personnel on top of your current payroll. The hardware, software and staff needed to maintain automated systems will easily add five figures to your current payroll each month.

Expertise: In-house automated testing will require the additional staff you hire to have technical backgrounds. Basic programming skills and knowledge will be required because scripts will need to be completely understood. The tools that are used for automated testing may have their own defects, so staff must be able to resolve that as well.

So, before you drink the flavored punch that say’s “automate all the things”, double check your goals and constraints. You might not have the time and budget to “automate all the things” and you probably won’t be happy with your first set of outcomes, either.

The Things You Really Want

What you really want with website testing services is confidence and speed. You’re at the end of a long journey building a new website and you’re not in the mood for a long, drawn out, testing phase just when you’re ready to launch you new “look”.

You’re probably not sure but suspect that there might be a few bugs or defects in your website. You’ve done your testing but still aren’t sure. You want to know that your website will be flawless for everyone that visits.

You’re also probably tired from all the hard work and concentration that it’s taken to get your website to this stage. You’re ready to be done and you’re not in the mood for a bunch of back-and-forth with a website testing company that just doesn’t understand where you are and what you want.

When we talk with you about your project, we’ll get a keen understanding of your risks and concerns and build a plan and proposal to complete our work quickly and efficiently so that you can move your website project forward and claim your new customers and prospects.

At TESTCo, we believe in honoring promises. We take the time to make sure our testing work focuses on and delivers the value you need to achieve your business goals. Here’s a page about our website and web app testing services.

We want to help you achieve your business and technology goals.

Share This:
Website Testing

The Case for Website Testing

Are you one of those companies that doesn’t conduct systematic website testing after a site refresh? By systematic website testing I’m referring to a planned and thorough testing of the site for functionality and user experience.

Too many companies, I’ve learned over the years, don’t take website testing seriously enough. Employees  will be asked to set aside a few minutes of their day to  “go through the site” and note any errors or broken links. That’s a start, but don’t mislead yourself into thinking that all the bugs are going to be found with this hit-and-miss method.

In today’s world, if your website doesn’t work properly, quickly and easily on the device the person is using then you just lost a visitor, prospect, new customer or, worst of all, a repeat customer. In every case your brand image takes a beating, too.

Most consumers won’t tolerate a business website that does not perform on their device. They will go to the next business on the list!

Ask yourself, how much is each visitor worth to your business? Each lead?  Each  new customer? Each repeating customer? Is it worth the risk NOT to test your website after you change it?

The 1-2-3 of Testing Websites

There are three critical areas to test on most websites – functionality, cross-browser/devices, and page-load speed.

  1. Functional Testing insures that each button performs the intended action properly. Some websites are complex with advanced search, grouping, recommendations and shopping carts. Some websites are simple with just a few buttons to submit a contact request. Most of the functionality will behave the same no matter which browser or device is used so testing the functionality is typically a “once and done” set of steps each time you are finished with your website changes.
  2. Cross-Browser/Device Testing insures that your website’s content, layout and presentation look and behave properly in a variety of operating systems, browsers and mobile devices. This is where it get’s a little complex. There are just too many different operating systems, browsers and mobile devices to test each one within a reasonable period of time.
  3. Page Load Speed Testing. Visitors have little tolerance for websites that take too long to appear on their device. If the page isn’t fully loaded in a few seconds, there’s a high probability the visitor will abandon the attempt and try another site.

Yes, there are tools available online that seem to make it easier to run functional testing and cross-browser/device testing. But, if you aren’t familiar with them or you don’t have a clear idea of which test cases, browsers/devices are most important to your website visitors, then you can easily waste a lot of time.

Who to Test a Website

The good news is that you have several options for website testing.

  1. Big IT Outsourcing Company – if you aren’t already a Fortune 500 company you probably won’t get much help from these guys. It’s not that they don’t want to help you. It’s just that they are so big that they can’t help you and expect to make a profit. If they could help you, they’d need a small army of managers and directors just to make your project successful.
  2. Freelance – the Gig Economy. This might work for you if you already know a freelancer that is very good at exactly what you need (website testing for functionality and cross-browser/device) and is not already booked with another gig.  If you have to go find that person for the first time, be prepared to put in some detailed work to find, select, pilot and then manage the project to success. Online freelancer boards are great and can give you a lot of good information to make your choice. But, there are a lot of non-qualified freelancers that will answer your call to help and you’ll have to spend time sorting the wheat from the chaff.
  3. DIY – Do It Yourself – This might work as well.  It could be you or it could be someone else in your business. If you take this approach, it will work but it won’t work very well. Here’s why – you simply haven’t been trained in how to think about and perform website testing. It’s not hard and you could definitely learn it but your business’s newly launched website probably isn’t the best place to get started.
  4. TESTCo – Safe. Easy. Reliable. Software and website testing is all we do. We’re highly trained in software testing methods and systems. We see hundreds of website testing projects every year. We know the right tools to use, when to use them and what to look for. We are the right size for you so we can complete your small, 1 or 2 day project as easily as we complete larger multi-month testing projects.

Is TESTCo Your Goldilocks?

If you have a website testing project or are just not confident in the website testing that has been done already, please consider that TESTCo may be your Goldilocks – just the right size and temperature!

You can talk with us and find out how we’re different and how that difference can benefit you. We work as an outsourced QA service, or on demand for projects.

Share This: