Automated Regression Testing for Software | TESTCo

Automated Regression Testing Dilemma

/in , /by
Download Agile Testing Report

Nobody likes running long software regression testing cycles. It’s long, hard and boring.

And yet, nobody likes to release bugs either. NPS (Net Promoter Score) drop like a rock and the support phones light up like little annoying Christmas ornaments.

So, just grin and bear it or take the plunge into software test automation?

It depends on where you stand on these two dilemmas.

1. Business Speed Dilemma

This dilemma is about your cost and speed of producing software versus capturing more customers and larger market share before your competitors.

  • Adding customers, solving real problems and growing your market share requires more features in your software.
  • More software features require more software testing.
  • More software testing takes more time, people and delays releasing your new features.

So, faster software production equals more customers as long as you meet their minimum quality requirements. Otherwise, they dump your software or mobile app like yesterday’s leftover tacos.

2. People Versus Machines Dilemma

This dilemma considers that people are more adaptable but more expensive than machines that are more consistent but less expensive.

  • People are better at finding defects.
  • People are slower and more expensive than machines but require less “programming”.
  • Defects found late in the process are very expensive to fix.
  • Machines are better are consistently repeating sets of activities.
  • Machines are slower and less expensive than people but require more “programming”.
  • Defects found early in the process are cheaper to fix.

So, how to resolve your dilemma? My friend and business mentor, Keith Cunningham says

One choice is slavery. Two choices is a dilemma. Three choices is freedom.

Hybrid Software Regression Testing That Turns a Dilemma into Freedom

Consider a hybrid approach to software regression testing.

Use people for manual regression for all features in the current and previous version.

New(ish) features are where most of the defects lie in hiding. This leverages the human advantage of finding defects in new features.
This also minimizes the amount of time & money spent on people in the regression testing cycles. You will have the benefit of faster and cheaper software regression testing cycles.

Use Test Automation for regression testing all features released prior to the previous version.

Older features are where most software developers do damage. This leverages the machine advantage of consistently repeating activities. This also minimizes the amount of time between discovering a defect and fixing the defect – which dramatically reduces the cost of fixing the defect.

You and your developers will have the benefit of real time testing so that regression defects (where new features break older existing features) are much cheaper to fix.

The Drawbacks to Hybrid Software Regression Testing

  1. You may need to add Test Engineers to your software testing team. Successful test automation will require written test cases that can be easily automated. Test Engineers know how to write these types of test cases.
  2.  You will have to decide on which approach to use for Test Automation – Record/Play, Scripted or built-in APIs. You’ll need an experienced Test Automation Engineer to help you determine the best path for you and your business.

At TESTCo, we can help with both – call and let’s talk about how to implement or improve your automated regression testing.

agile software testing reportDownload our report that looks at the challenges and pitfalls of creating automated regression tests in an Agile environment.

Download Agile Testing Report

QA Services

QA Services for the A-Team. A Founder’s Dream Comes True.

/0 Comments/in , /by

 

In This Post

As a founder myself I know first-hand how agonizingly difficult it is to build and retain an A-team of talent to run and grow my business – to make my dream a reality. If you’ve started a company that develops mobile apps, software, or websites you’re feeling the same pain every day. The pain is especially acute in the area of QA services for small companies.

It’s not just the shortage of talent that’s the problem. It’s the shortage of smart talent. You know, the proactive problem solvers and opportunity achievers. The type of people that bring a lot to the table. In short, the A-team.

Putting All-Stars At Every Position

In 80% of the conversations I have with founders and CEOs they point with pride at the skills and innovativeness of their engineers. But when the conversation turns to their QA services team, whether internal or outsourced, the mood changes. By and large most founders believe they have competent people in the QA slots, but they don’t have enough A-players in those roles to take the QA function to the next level.

The really savvy founders I know understand that the dev cycle is streamlined in direct relation to just how sharp their QA team is and how well they work with Engineering. Those same founders know that the quality of their product becomes their brand and defines their dream. On the other hand,  bad reviews, poor customer loyalty, extensive support costs threaten to destroy their dream. No wonder a lack of QA talent causes more than a little anxiety for these folks.

Your Dream is My Dream

The dream I’ve built–my company–actually helps founders build and maintain their dreams by providing the smartest, most goal-driven QA service teams on the planet. Give me a shot at a project and if we don’t perform according to plan, you pay nothing. You understand correctly. That’s an old-fashioned money-back guarantee.

I’m not desperate for your business. I’m confident. Confident in the ability of my teams to delight.

You don’t hear the word “delight” associated with software QA much, do you? Well, that word and many like it surface in our customer reviews. Here are a few reviews from founders of small companies like you.

We maintained Zero customer-found defects while cutting the QA budget in half

This was not an easy thing to do, but TESTCo management in general, and the QA team more specifically, found a way to make this happen.

Meredith Z.
Journyx: www.journyx.com

They ask thoughtful questions, bringing clarity

TESTCo team is diligent, asks thoughtful questions, and they bring clarity.

Chris S.
Praxis Technology Escrow

I can focus now on customer acquisition and customer satisfaction.

I can focus now on customer acquisition and customer satisfaction.

Ivan Z.
PennyWorks

Take product to market

We want to thank you for your services over the last 6 months. The UI, functional, and regression testing you’ve provided has REALLY helped us take this product to market.

John S.
Accubuild

Simply Delighted.

I couldn’t be more pleased. TESTCo exceeded my expectations.

Charles B.
Besondy Consulting

Consistently Delivered

I have to say that our experience with TESTCo has been very positive for the two years we’ve been working with them.

Phil G.
Lombardi Software

Easy and Painless

Our company has never outsourced software QA testing work before and TESTCo made it both easy and painless.

Ryan T.
Draconis Software

Fleshed Out the QA Process

But working with TESTCo has been great! They ramped quickly and within two weeks TESTCo fleshed out the QA process for us.

Brian D.
Optive Research

Instantaneous Expertise

It is obvious that TESTCo measures their success by our success. TESTCo’s process dovetails with our agile development methods, which facilitates a ‘develop today and test tonight’ methodology.

Michael H.
Hayes Software Systems

Fewer Customer Complaints

Now that we have TESTCo in place, we’re experiencing fewer customer complaints.

Zeeshan M.
IntoSales

Extremely Valuable

On behalf of our customers, we thank you. We look forward to a long and productive relationship together.

Curt F.
Journyx (2016)

Real-Time Testing

TESTCo has helped us enable a real-time, iterative development process that has had a dramatic impact on our time-to-market.

Erik H.
Between Markets

Saving 30 to 50%

We’re saving probably 30% to 50% of what we were paying previously, and we get things done faster and more reliably with TESTCo’s software testing services.

Jim F.
Guard-IT

A New Tool

My view is we now have “a new tool in the bag” for meeting our testing requirements.

Pat M
Bluespace

Rigorous Review

Having a rigorous third party review to ensure quality, but didn’t want to make their clients wait longer for their deliverables.

Michael T
Bluefish Development Group

An Indispensable Factor

Put simply, the TESTCo team was an indispensable factor in transforming the quality and reliability of our software products.

Matt R.
NYASS

Three Reasons Why TESTCo

The three reasons I’m thrilled with our decision to use TESTCo: communication, comprehensiveness and team unity.

Jonathan D.
Umphub

A Great Partner

Your team has become a great partner to our software development teams.

Sebastian G
Expero/Palladium

Attitude, Consistency, Intelligence - The TESTCo Difference

What makes TESTCo different? After all, the world is filled with software testers and outsourced QA services companies. Some of which even proclaim to serve small companies.

First, I like to describe us as a boutique software QA company. We’re not a QA factory. I have no interest in running that kind of business. We provide highly personalized services, crystal-clear communication, and top-drawer project leadership. (At affordable rates I should add). TESTCo's nimbleness and size dovetail well with small and mid-sized companies.

Second, I employ only A-team caliber people. My QA Managers (provided with every project at no additional cost) are brilliant at setting the right testing strategies to deliver value to you every day. They are true leaders.  Our Test Engineers are bonafide engineers who love to test. They don’t sit back and wait to be told what to do. No way. They work closely with the QA Manager to develop and implement the perfect test plan to achieve the project goal.

Third, we are awesome communicators. You receive status reports twice a day. My QA Managers participate in (and often lead) project postmortems. The feedback they provide to Engineering actually helps shorten the Dev cycle. The QA Managers are fluent in English. Plus we’re available to chat, email or talk when you are. TESTCo QA Managers are located in the Central time zone.

Fourth, we deliver expected results because the results are expected. That’s a clever way of saying we really excel at understanding the business goals (not just the testing goals) and creating test plans that support the business goals within the known restraints. Your team knows what to expect by when. And we deliver.

When was the last time you talked to a software tester who wanted to know about the business goals associated with the project? Never, right?

On-Demand, or On-Going QA Services for Small Companies on a Budget

Here's the other good thing about TESTCo QA services. On very little notice (often as little as one day's notice), we can parachute in to rescue a project. Even for small projects. The other way we provide value is to be an on-going extension of the QA team, or be the QA team for your organization. I have many clients who have relied on TESTCo for all of their testing requirements for years. They receive value, reliability, and intelligence all for 30% - 50% less than an internal team.

A-Players for QA. No Risk. Make the Call.

And that’s the primary difference between B- and C-level QA people and the A-team at TESTCo. We care a lot about your dream.

Earlier in the post I made an offer. I’ll make it again. Try us for a project. If we fail to perform according to the goals of the project, the work is free.

Go ahead, schedule a call with me to see if I stand behind my promises. I look forward to speaking with you – founder to founder. Just click the green button on this page to schedule a conversation. Talk to you soon.

Mobile App Performance Testing

Do I Need Mobile App Performance Testing?

/0 Comments/in /by

In This Post

Mobile app performance testing is the process of testing and reporting the time it takes your application to process a request or take an action – especially when large groups of users perform the same or similar action simultaneously.

Performance testing answers the question, “When and where does the performance and speed of my application degrade the user’s experience?” It also answers the question, “How many users can I have before my application crashes?”

Performing “At Scale”

“At Scale” is the current lingo for “many, many, many users and transactions”. Performing “At Scale” is a big concern when building software and especially mobile apps. Mobile app performance testing helps you understand the scale your mobile app can withstand before degrading or crashing. Most software development teams invest significant time and money engineering their mobile apps to perform at scale from the very beginning. This is generally a wise investment but only if you are willing to apply mobile app performance testing throughout the software development process. Otherwise, the software development team will not have the feedback they need during the software development process to measure and correct the scaling assumptions they had to make while building your software.

Performing “Good Enough”

If “At Scale” is not a critical business and technology driver for your mobile app, you still might need to be concerned about performance and mobile app performance testing.

Here’s why:

  • All mobile applications have performance limits
  • When mobile application performance degrades and slows down, users abandon mobile apps.

You’ve invested serious time and money building your mobile application. Don’t you want to know how many people you can satisfy with your mobile application before they become dissatisfied?

There’s a big difference between engineering and performance “At Scale” and “Good Enough”.

Engineering and mobile app performance testing “At Scale” is an ongoing endeavor to insure that your mobile application will always be available to as many users as possible. This is an expensive process but worth it when your goal is performing “At Scale”.

Mobile app performance testing  for “Good Enough” is an end-of-cycle process that determines “How many users can I handle before I need to turn on more servers?” This is a much simpler and less expensive process that gives you a finite number and an alert for when you need to add more servers in your AWS or Rackspace cloud.

If you aren’t required to engineer, build and deliver “At Scale”, then “Good Enough” may work best for you. A short engagement to determine “Your Maximum Number” will help you recognize and avoid a disaster where your mobile application performance degrades and you lose customers. You’ll use “Your Maximum Number” to alert you to add more server capacity before your known performance threshold and prior to any perceived slow downs by your customers.

What Does Mobile App Performance Testing Look Like?

Mobile app performance testing is a simple process but it’s not easy.  Here’s one way of looking at the process:

  1. Build your mobile app features
  2. Add “Instruments” at locations within your mobile app that will measure throughput
  3. Build or find a “Pump” to push specific user scenarios through your mobile app
  4. “Load” the “Pump” with one or more test cases or user scenarios
  5. Tell the “Pump” the volume and timing of test cases to push through your mobile app
  6. Record the readings from the “Instruments” as the “Pump” pushes a measured volume of test cases through your mobile app.
  7. Analyze the readings from the “Instruments” to determine if
    • throughput performance was acceptable – add more volume and retest
    • throughput performance was not acceptable – determine bottlenecks to discuss with your mobile app developers to fix for greater throughput.

Simple But Not Easy

Yeah, it’s simple but it’s not easy. Generally speaking, finding the “Pump”, placing the “Instruments” and building the “Load” of test cases is straightforward. You’ll wind up with a big pile of data from the instruments and you’ll have some early anecdotal evidence of how your mobile application performs under a load.

But, it’s seldom actually that simple because mobile applications almost never just fail due to a high load. It’s the nuances of what happens under one set of load circumstances that look just fine and another set of load circumstances that crashes your mobile app. This is when a trained and experienced Performance Testing Engineer can really help you and your software development team quickly find, test and fix the performance bottlenecks in your mobile application.

Mobile app performance testing can be a critical component of testing and launching your mobile app. Not every mobile app needs performance testing, but when you expect a large volume of users or heavy usage, mobile app performance testing by experts can save you from a major crash and downtime where you can lose customers and prospects.

Look Before You Leap

You want good testing results, of course. Or put another way, you want good testing value. That’s why it’s crucial to define the mobile testing strategy first, which includes performance testing, before thinking about a test plan.

Want to discuss your situation? Call me 888-254-9709. Or, click the blue and green button on this page to request a call back.

Checklist for Finding the Best Software Testing Company

Checklist for Finding the Best Software Testing Company

/in /by

After your web search for a software testing company reveals a long list of company names, how do you select the one that is right for your business? Here’s some guidelines and a check list.

How to Initially Screen Software Testing Companies

You may or may not know exactly what you need.  If you do, then please skip to How to Select a Software Testing Company. If you’re not sure what you need, then please read on.

First, do your choices have references and testimonials from clients where they solved a similar problem? Take note of where they solved a business problem, not just a software testing problem.

Next, do they share their knowledge? Knowledge sharing is an important indicator of where the software testing company believes their value lies. If they share software testing knowledge and know-how with you then they are likely a company that values their people and commitments more than their intellectual property. If they don’t share their knowledge of software testing with you, then they probably value their intellectual property more than their people and commitments.

Lastly, is software testing their primary focus? Many companies provide a wide variety of technical services – everything from ITIL to software development to business processing outsourcing. Unfortunately, software testing is seen as a commodity by many people.  We have found that many outsourced technical companies that offer software testing along with a variety of other services tend to see software testing as a commodity as well.  This leads to their top talent and most attention going to somewhere other than software testing. That may be fine to build a business but it probably won’t be your best choice to solve your current software testing and business problem.

Ready to put TESTCo on the list of software testing companies to consider? Just send the form from this page to schedule a call.

How to Select a Software Testing Company

By now you should have a handful of good choices that you are confident in. According to your research each company on your short list

  • Has solved similar problems successfully in the past,
  • Has shared their knowledge about software testing with you,
  • Has focused on software testing as a priority

Now, how to select the winner.

A Check List for Selecting the Best Software Testing Company

  1. Do they perform an effective needs analysis? Do they ask you and then clearly understand why you need help, your business goals related to this problem and the constraints you have? This needs analysis should take no more than about 30 minutes over the phone. If you aren’t convinced they clearly understand your problem then you should move on with another choice.
  2. Do they offer smart choices and benefits? Since they understand your problem, goals and constraints, do they offer and explain the selection of choices they can offer and how each can solve your problem? If not, then you should move on.
  3. Do they answer all your questions? Are the answers clear and relevant to your situation? Do they seek additional questions from you or avoid them trying to close the deal? If you aren’t convinced these are the most curious creatures on the planet (all great software Test Engineers are hugely curious!) then you should move on.
  4. Do they make it easy to get started? Software Testing is one of the few technical activities that can begin almost immediately. But, without a good understanding of your business and technical problems, that testing may not produce the outcome you need. Gaining this understanding has to be done quickly and effectively so that time and budget constraints are met. They can’t go off to “train on your app” for a couple days. Are you confident that they know how to get your team producing real value as quickly as possible without taking a bunch of your time? If not, then you should move on.
  5. Do they clearly distinguish the value? Business is a game of value creation and transfer. Your software produces real problem-solving value for your customers. That’s why they keep buying it. Software testing is not just maid service for the coders! Software testing and quality assurance activities confirm and deliver the real value of your software application. If your software testing company prospect can’t clearly explain how this happens so that you see real value, then you should probably move on.

It is a simple checklist but it’s a tough one to pass.  If you make it through every step with a software testing company your odds are high that you’ve of identified the best software testing company for your business.

Ready to put TESTCo on the list of software testing companies to consider? Just send the form from this page to schedule a call.

Is On Demand Software Testing Different from Offshore Software Testing?

/in /by

On Demand Software Testing is simply a smart implementation of offshore software testing. The traditional offshore software testing model consisted of the complex and risky process finding, selecting and managing a team of people in an offshore testing company another country with the hope that you could save some money. Most research shows that the odds of meeting your quality, production and cost savings goals were slim to none.

Should We Go for On Demand Software Testing or Offshore Testing?

Nevertheless, if you are big enough and have a sufficient number of people with the skills and time to find, select and manage an offshore software testing team, then this is probably a good strategy – you can negotiate lower costs and then use your stateside team to put pressure on meeting the quality and productivity goals. If you aren’t big enough to have that many people with that much extra time then On Demand Software Testing may be a better approach. On Demand Software Testing does typically use offshore software testing teams but is delivered in a different manner where we are responsible for finding, selecting and managing the offshore software testing team and you only need to tell us what you want and how you want it delivered. Does it cost more? Sure it does if you only look at the hourly rate. Is it more cost effective? We think so, especially when measured by total dollars spent Vs real value delivered and your time spent.

The TESTCo On Demand Software Testing Difference

If you find yourself needing help with offshore testing but not having enough people to turn that need into a big project, then TESTCo’s On Demand Software Testing may be the best answer for your offshore software testing needs.

Mobile Application Testing Services Choices

How Much Testing Does Your Mobile Application Need?

/in /by

Mobile applications are different than web or installed applications and deserve some careful thought to make the best choices about what and how much mobile application testing services are really needed.

There is a long list to consider and surprisingly, you probably don’t need all of those mobile app testing services to achieve your goal.

Mobile application testing services can include:

  • Mobile application functional testing
  • Mobile application regression testing
  • Mobile application cross device testing
  • Mobile application test automation
  • Mobile application security and penetration testing
  • Mobile application performance testing
  • Mobile application usability testing

Goals for Mobile Application Testing Services

Here are some questions you can consider to help you determine if one of the types of mobile application testing services I listed will help you meet your needs and achieve your goal.

First, what is your goal for your mobile application?

  • How many downloads or users do you want to use it?
  • What value will your users receive from using your mobile application?
  • How will you know they have received that value?

Now that you have a basic goal for your mobile application, see if the following questions help you chose the best and most appropriate mobile application testing services.

Mobile Application Functional Testing

Does you mobile application contain complex or simple functionality?

Simple functionality with only a few inputs/outputs/screens usually doesn’t need anything other than some basic validation that the inputs/outputs and screens work and flow properly. There are few inputs/outputs/screens so the number of combinations to test will be limited and can be accomplished relatively quickly.

Complex functionality where you have more than just a few inputs, outputs and screens probably needs (and deserves) a more robust testing strategy and plan. In general, you can’t test everything (it takes too long and costs too much) so selecting a smart test strategy will be very important to make sure that all of the “Important Stuff” is tested thoroughly and frequently. Almost any qualified Test Engineer or QA Manager will have a small library of Test Strategies and can help you select the one or two that most closely match your needs and goals.

Mobile Application Regression Testing

Regression Testing helps you make sure that you didn’t break anything that was previously working well. The main goal of regression testing is to insure that you don’t disappoint or anger your mobile application users instead of delighting them with new features and functions.

Mobile applications tend to be updated frequently. One of the benefits of a mobile application is that they tend to be smaller and simpler than other types of applications and can be updated very quickly and easily. This is a tremendous benefit because it allows you to quickly add or change features based on real-time user feedback.

But, being very quick to make changes to your mobile application may also hurt you if you don’t take just a little extra time to complete an effective regression test.

Regression Testing Can Be Challenging

  1. Familiarity breeds contempt. When you’ve seen and tested your mobile applications features and functions every day for the past six months, it’s easy to let your assumptions and urgency take control. This can result in skipping or overlooking some functionality in your haste to release your new mobile application. Using an experienced and trusted third party for your regression testing will insure that nothing is skipped (or assumed to be OK) during your regression testing. “Once over lightly and quickly” is not a good regression testing strategy!
  2. Lack of test documentation. Mobile applications tend to be smaller than web applications and good test documentation is usually the first casualty. And no, your mobile application doesn’t necessarily need a full set of ISO compliant test documentation. But, a decent list of test scenarios and use cases can help you complete your regression testing quicker and with a much higher (and measurable) level of quality and customer delight. An experienced third party software test engineer can quickly and easily help you build and maintain these test documents as a natural part of helping you regression test your mobile application.

Regression testing is simple but not always easy if you aren’t an experienced software test engineer. But, getting help from a professional, at least the first couple times, can go a long way to helping you release your mobile application faster, better and cheaper.

Mobile Application Cross Device Testing

Your mobile application might be used on a new iPhone or an Android phone several years old. Almost all mobile devices behave slightly differently and the difference in behaviors between iOS and Android can be significant. If you fail to test your mobile application on a variety of mobile devices, you are inviting a catastrophe.

Cross device testing helps your mobile device safeguard against the following.

Customer loss: Cross testing is vital to ensuring quality assurance across all platforms that your application is on. If everyone was using the same device with default graphics, this wouldn’t be an issue. Users today use a variety of devices across several platforms; cross testing across platforms ensures users from all ends have the same experience. If the user interface on one platform performs poorly, customers will likely leave the app quickly.

Issues that arise from upgrades: Mobile applications are updated very frequently and these updates will eventually cause several bugs. Cross device testing will help you identify bugs and how to deal with them. Failure to recognize this might cause significant bugs that can cause your application to be unusable until it is resolved. This can also lead to compatibility issues after design changes as well.

SEO and additional rankings from Google: For mobile applications, ranking on search engines is crucial to gain traction to your site and mobile app. Since 2015, Google has taken a mobile-first indexing approach. This means that the algorithm for Google search results takes cross device compatibility into account and ranks accordingly. If a competitor to your app is compatible with all platforms and devices, they will significantly outrank you.

There are two basic methods available for cross device testing. You should consider the pros and cons of both before specifying what mobile application testing services to obtain.

Real Mobile Devices and Operating Systems

Real devices provide the best mobile application testing results since you are testing your application on the actual device. But, Real Mobile Devices are expensive to acquire and maintain. Real Mobile Devices are also slightly more time consuming to test on as they require physical handling. Finding a sufficient variety and availability of Real Mobile Devices is a big challenge as new devices and mobile operating systems are released and updated frequently – and this makes the process of building and maintaining a library of mobile devices very expensive.

Mobile Device Emulators

Emulators are usually on-line services where you can load your mobile application on a variety of specific virtual or emulated mobile devices with specific operating systems and versions. Mobile Device Emulators are very handy but have some distinct challenges. Mobile Device Emulators work well for basic functionality but are seldom useful if hardware functionality testing (GPS, Cellular Vs Wifi Networks or Device rotation) is needed to test your mobile application.

Also, mobile device emulators can’t emulate the performance of an app when it receives incoming calls, text messages or other notifications. This is a crucial issue because everyone with mobile devices gets numerous notifications every hour. If an app has to crash each time a notification occurs, this would make it unusable. Additionally, mobile device emulators can’t simulate other aspects like battery issues, touchscreen issues and more. These are some of the most fundamental aspects of using a mobile device.

But, We Think There is a Better Way

The TESTCo Blended XDT (Cross Device Testing) – at TESTCo, we take a blended approach to using Real Mobile Devices and Mobile Device Emulators. We will help you analyze your expected customer device usage and the current market share of each mobile device and operating system and produce a matrix that compares your testing needs against Real and Emulated devices. We use this to build a testing strategy that optimizes for primarily Real Mobile Device Testing with a portion of Emulator Testing so that we can give you the highest confidence that your mobile application will work flawlessly on the devices that your customers use.

The TESTCo Blended approach takes advantage of the benefits of both real mobile devices and emulators while covering the blind spots of each approach. This gives your mobile application testing a comprehensive approach that would simulate what would happen to your application in the real world. This ensures that your mobile application will work seamlessly during its first encounter with users doing normal functions on their phone like using it with low battery, taking calls while using your app and more.

Mobile Application Test Automation

Using test automation for your mobile application can allow your software development team to build and release your software quicker. But, is development speed really a problem and do you want to invest to solve that problem?

Here’s why I ask. Test automation is the process of writing computer code (for the test automation tool to use) to test your computer code (your mobile application). So, you’ll be building another computer program (to test your mobile application) that does test your software faster but is more complex and expensive to build and maintain. Go into this one with your eyes wide open for the ongoing consequences and not just the initial expected benefit of faster testing.

Another issue with “going faster” when building and testing your mobile application is the approval and update process – which is entirely out of your hands! Sure, you can build and test your mobile application faster but you still have the issue of bottlenecks getting your mobile app approved for Apple or Google store and then, even after it has been re-approved and loaded in the Store, your chances that your users will actually download and install the update are small.

So, instead of “going faster”, you might find that “building smarter” provides you with more benefits and fewer costs. Nevertheless, building and running test automation can be valuable for your mobile application in certain situations.

Mobile Application Security and Penetration Testing

Your mobile application probably has some personal and/or confidential information on it. Hackers are penetrating, stealing and selling confidential data from mobile applications and their servers on a daily basis.  Can you afford for your mobile application to be hacked and your users mad at you?

Thankfully, most mobile applications have a small number of security and penetration points so testing them is common practice.

The security of your mobile application’s login screen probably isn’t an issue. In fact, most mobile applications are naturally quite secure in their user interface as long as the login function is well built – and most are.

The real challenges with mobile application and security lie in the way your mobile application communicates with the world around it and it’s server. Mobile applications use API calls to send and receive information to and from other users and the server in most cases. The robustness, error checking and reliability of these API calls will determine how “safe and secure” your mobile application will be for your users. Mobile application security and penetration testing evaluates your mobile applications login screen and internal functions as well as your API calls to ensure that your customers’ data is safe and secure.

Penetration testing can be done against different types of code and systems in your mobile application like servers and APIs. Depending on your goals for the pen tests, there are several phases within each pen test to ensure that the security of your application is covered from all angles. After a pen test is done, a formal report will be given to you that covers all the vulnerabilities that were found and how hackers can use them to gain access to your application’s data. In contrast to many systems and processes of your mobile application, penetration testing is largely a manual process. There are testers that overlook the entire process to make sure that your mobile application cannot be breached.

Mobile Application Performance Testing

Your mobile application will need to respond and perform quickly for your customers and prospects to feel confident using your app. Poor or questionable performance can (and does) lead to poor reviews, fewer stars and rapidly decreasing downloads and usage. This probably is not your goal so performance testing may be a risk you want to consider selecting for your mobile application testing services.

There are three critical areas to consider when performance testing you mobile app.

Device Performance  – Your mobile application needs to do 2 things simultaneously – respond quickly when in use and play well among the other mobile apps on the device. Testing the first is usually straight forward. Testing the second will require a smart strategy to get accurate results quickly without wasting a lot of time with invalid test setups.

Server/API Performance – Your mobile application will need to communicate with your server to send and retrieve information. In most cases, the server application is separate from your mobile app and interacts with your mobile app through an Application Programming Interface (API). This server app will tend to behave like most server-based applications and is subject to the same performance testing with one exception. The APIs need special attention to a wide variety of performance loading criteria to make sure the mobile apps that depend on them are not needlessly bogged down. Make sure to remember to test the server under load as well – not just the server application!

Network Performance – There are almost too many mobile networks to even list anymore. Mobile devices communicate either via WiFi or the carrier’s mobile data network. In all cases, even Wifi, the network speed and capacity are going to vary wildly. 2G and 3G networks still exist in many places in the world and the continental United States. Many WiFi hotspots are still using the older 802.11(a) and (b) technology. Network performance testing is tricky and it is critical to understand the user’s environment and goals before building a network performance testing plan.

Lastly, all performance testing requires detailed data analysis to clearly identify and truly understand the performance bottlenecks. Performance testing typically produces massive amounts of data that needs to be analyzed to be understood. Performance Test Engineers specialize in this data testing and analysis and have spent many years learning and building their knowledge.

Mobile application performance testing can be tricky to plan and execute so that you get the full value of the work. There is a lot of performance testing that can be done but typically only a few of the tests are truly valuable to any certain situation. The key is to be perfectly clear on the problems you want to solve and the risks you want to reduce. This approach allows your Test Engineers to perform the exact tests you need without wasting time and money on unneeded mobile app performance tests.

To Be Continued

In the coming weeks I will shed light on the five other types of mobile application testing so you can make smarter testing decisions. In the meantime, check out this post if want tips on how to select a mobile application testing company.

If you can’t wait for me to finish this post (hey, I can’t write all day, I have a business to run), call me (888-254-9709), or request a call back using the green button on this page. I’m happy to answer your questions.

Website Testing

The Case for Website Testing

/in /by

Are you one of those companies that doesn’t conduct systematic website testing after a site refresh? By systematic website testing I’m referring to a planned and thorough testing of the site for functionality and user experience.

Too many companies, I’ve learned over the years, don’t take website testing seriously enough. Employees  will be asked to set aside a few minutes of their day to  “go through the site” and note any errors or broken links. That’s a start, but don’t mislead yourself into thinking that all the bugs are going to be found with this hit-and-miss method.

In today’s world, if your website doesn’t work properly, quickly and easily on the device the person is using then you just lost a visitor, prospect, new customer or, worst of all, a repeat customer. In every case your brand image takes a beating, too.

Most consumers won’t tolerate a business website that does not perform on their device. They will go to the next business on the list!

Ask yourself, how much is each visitor worth to your business? Each lead?  Each  new customer? Each repeating customer? Is it worth the risk NOT to test your website after you change it?

The 1-2-3 of Testing Websites

There are three critical areas to test on most websites – functionality, cross-browser/devices, and page-load speed.

  1. Functional Testing insures that each button performs the intended action properly. Some websites are complex with advanced search, grouping, recommendations and shopping carts. Some websites are simple with just a few buttons to submit a contact request. Most of the functionality will behave the same no matter which browser or device is used so testing the functionality is typically a “once and done” set of steps each time you are finished with your website changes.
  2. Cross-Browser/Device Testing insures that your website’s content, layout and presentation look and behave properly in a variety of operating systems, browsers and mobile devices. This is where it get’s a little complex. There are just too many different operating systems, browsers and mobile devices to test each one within a reasonable period of time.
  3. Page Load Speed Testing. Visitors have little tolerance for websites that take too long to appear on their device. If the page isn’t fully loaded in a few seconds, there’s a high probability the visitor will abandon the attempt and try another site.

Yes, there are tools available online that seem to make it easier to run functional testing and cross-browser/device testing. But, if you aren’t familiar with them or you don’t have a clear idea of which test cases, browsers/devices are most important to your website visitors, then you can easily waste a lot of time.

Who to Test a Website

The good news is that you have several options for website testing.

  1. Big IT Outsourcing Company – if you aren’t already a Fortune 500 company you probably won’t get much help from these guys. It’s not that they don’t want to help you. It’s just that they are so big that they can’t help you and expect to make a profit. If they could help you, they’d need a small army of managers and directors just to make your project successful.
  2. Freelance – the Gig Economy. This might work for you if you already know a freelancer that is very good at exactly what you need (website testing for functionality and cross-browser/device) and is not already booked with another gig.  If you have to go find that person for the first time, be prepared to put in some detailed work to find, select, pilot and then manage the project to success. Online freelancer boards are great and can give you a lot of good information to make your choice. But, there are a lot of non-qualified freelancers that will answer your call to help and you’ll have to spend time sorting the wheat from the chaff.
  3. DIY – Do It Yourself – This might work as well.  It could be you or it could be someone else in your business. If you take this approach, it will work but it won’t work very well. Here’s why – you simply haven’t been trained in how to think about and perform website testing. It’s not hard and you could definitely learn it but your business’s newly launched website probably isn’t the best place to get started.
  4. TESTCo – Safe. Easy. Reliable. Software and website testing is all we do. We’re highly trained in software testing methods and systems. We see hundreds of website testing projects every year. We know the right tools to use, when to use them and what to look for. We are the right size for you so we can complete your small, 1 or 2 day project as easily as we complete larger multi-month testing projects.

Is TESTCo Your Goldilocks?

If you have a website testing project or are just not confident in the website testing that has been done already, please consider that TESTCo may be your Goldilocks – just the right size and temperature!

You can talk with us and find out how we’re different and how that difference can benefit you. We work as an outsourced QA service, or on demand for projects.

Outsource Software Testing

Outsource Software Testing – Painful Lessons

/in /by

This is the true story of a real estate company that learned about promises and quality the hard way. They learned that it’s not always a good decision to just trust a third-party software developer to deliver the quality they promised. They also learned the value of outsource software testing when trying to get a mission critical software project back on track.

One of TESTCo’s real estate clients – a company that shall go nameless for reasons you’ll soon understand – was trying to develop the ultimate software for distributing and tracking available properties in their portfolio. By ignoring the need to outsource software testing, they quickly found themselves in a pinch.

The company’s old application was cumbersome, slow to load new properties, and a giant hassle in terms of administration, updates and changes. Their clients, the agents within the brokerage, had numerous requests for new features that would help improve sales and revenue. They wanted better information and more insight into the data they were collecting.

Naturally, they got really excited when a real estate software development team they found promised them the moon. The developers said all the right things, ensuring easier data loading, improved content management and a faster path to sales nirvana. What they forgot to consider was the quality of the software delivered and the need to outsource software testing for a mission critical business application. 

Download Case Study PDF

 

Well, Not So Fast

When development ended everyone was all hyped up to get the new application into– the hands of the sales people, but there were a number of glaring issues that raised questions about the project that we can all learn from.

  1. When does regression testing of the outsourced development become a mission-critical activity?
  2. When is the right time for Outsourced Software Testing?
  3. What happens when outsourced development goes wrong?
  4. What is the best Outsourced Software Testing ROI?

For starters, the outsourced developers only tested new features they designed and were hot for. The developers also failed to perform regression testing or thought they didn’t need to. When you outsource software development, regression testing is typically the most important part of delivering high-quality software.

The real estate company had no experience with outsourced software development. Their core business is moving properties, not batches of code. They relied on the advice of the outsourced software development shop and misguidedly cut some corners by doing their own internal QA testing instead of using professional software testing. In the real estate company’s defense, the outsourced software consultants didn’t offer a professional, third-party outsource testing option. The software developer thought they tested it well enough internally – and this almost never happens! With no prior software development experience, the real estate company didn’t even know it was on the table as an option. And, the contract terms were loose enough to allow the development shop to meet deadlines with low quality deliverables.

As you can guess, when the real estate software project got closer to completion, all kinds of defects started showing up. Features that previously worked in the old software were showing up broken in the new system! The trial and beta launches were a complete disaster.

Correcting Course with Counsel from an Outsourced Software Testing Company, TESTCo

When things were looking really bleak, the real estate company decided to correct course and hire TESTCo for their outsourced software testing to see exactly what was going on.

TESTCo designed a test strategy focused on achieving high regression testing coverage in a short period of time (to match their 2-week release cycle). We brought in a Test Engineer  experienced with exploratory testing strategies, and he built an automated ad-hoc testing program that looked for defects without a specific set of test cases. We also added a software QA track to the project plan and implemented burst-mode regression testing so sprints were not delayed, which spot checked and discovered regression defects during the new feature testing and development processes.

Abandon Ship!

With thorough testing and QA procedures under way, the real estate company began to assess the risks involved with completing the project and waiting for the software developer to fix the problems.

They came to a somewhat brutal conclusion. They wouldn’t be able to complete the project. They’d have to abandon ship, cancel the project without throwing more money away, and retreat to the old system in order to claw back some of its functionality. They ended up modifying the old system, testing it with TESTCo outsource software testing services, and relaunching that solution on time without any defects within three months.

Why It can Make Sense to Hire an Outsource Software Testing Firm Before Starting Development

It was an unfortunate lesson, but at least they got back to a working system that could handle some basic property listing tasks. Sadly, this happens often in the software development world. Companies with no software design and development experience hire outside developers that can’t deliver on their promises. They get oversold and then retreat, licking their wounds, discouraged by shattered technology dreams.

The moral of the story? Hire a solid outsource testing company like TESTCo prior to launching a mission-critical software development project. We can act as consultants and help you vet developers while designing test plans that help guide the project to success.

Can we make a difference for you? Read what our clients have to say.

Download Case Study PDF

Consider the Cost

There is another reason to think carefully about how you design,build and test your software.

When a mistake, defect or error is discovered during the Design Phase of your software project, the cost to fix it is simply a little time discussing the true need and then changing the design documentation.

When a defect is noticed during the Building Phase of your software project, the cost to fix it is larger – close to 10X greater than the cost to discover it in the Design Phase. Why? A mistake in the Build Phase will require significant time and effort for the Design Team to validate the requirement and then the Build Team to go back, re-write the offending code (and any other closely related code) and then test it again. This is far greater cost than during the Design Phase where the changes were only to the requirements.

When a defect is discovered during the Test Phase of your software project, the cost is even larger – again, close to 10X greater than the cost to discover it during the Build Phase. Why? A defect discovered during the Test Phase already has countless hours of engineering time to build the feature. Additionally, the defect has cost you Testing time as well. Now, add to that, the need to re-validate the requirement, confirm that the test case and defect is accurate and then get it rebuilt by the Build Team (along with any other affected or related code because defects seldom happen by themselves) and then re-tested by the Test Team.

Lastly, when a defect is discovered by one of your customers after it has been launched, the cost is approximately 10X greater than the cost to discover it during the Test Phase. If you have a defect in your production software, it will incur all of the previous costs to find, validate and fix. It will also probably cost you a customer or prospect if they can not complete their work with your software.

Defects can be Very Expensive. Testing Early and Extra is almost always an investment with a generous return. Testing with an outsource software testing specialist like TESTCo can decrease your costs and amplify your returns. I look forward to speaking with you.

Mobile Application Testing Strategy

Why You Need a Mobile Application Testing Strategy

/in /by

In This Post

App testing looks easy but you might want to think again – and I’ll help you think through a mobile application testing strategy.

Simple, but not Easy

You can very easily skip or ignore some very important risks just because you didn’t even know to think about them! Mobile application development moves very fast, is usually highly agile and covers much more than just the mobile app itself. Most mobile application testing projects spend too much time on functional testing and not enough time finding and reducing real technology and business risks.

Starting to test your mobile app without a Testing Strategy can result in a “well tested” mobile app that frustrates customers, scares off prospects and lights up the support phone and chat lines.

All because you didn’t know what you didn’t know!

And that’s probably not what you wanted or expected.

A simple mobile testing strategy will help you find the important risks, plan your testing work to focus on the important areas and reduce the time and money you spend launching your mobile application

What You Don’t Know CAN Hurt Your Project!

There are six  important subject areas in that you need to consider in your strategy for testing your mobile application.

  1. Devices – What is your list of most important devices? What devices are you willing to exclude or not test? Do you have access to those important devices?
  2. Networks – What networks will your mobile application use? Are they cell networks or wifi networks? What happens when your mobile app switches networks?
  3. Servers – How much of your mobile application functionality depends on accurate information from a server application? Most mobile apps rely heavily on back-end servers to do the heavy lifting work while the mobile app itself mostly accepts inputs and display outputs. What happens in between your server and mobile app?
  4. Performance – What are your minimum performance times for functions in your mobile application? What are the upper limits of your server? How many simultaneous users do you expect? What performance tools are provided by your server hosting company?
  5. Security – How secure does your mobile application need to be to protect your user’s information privacy? Is 2FA important to your users? How secure does your server need to be?
  6. UX (User Experience) – Will your users find your mobile application enjoyable and easy to use or frustrating and confusing to use? What UX data will you collect to help you understand this? If a user does leave, how will you recover them?

If you know the answers to these questions, you’ll be able to clarify your goals and build a mobile testing strategy to achieve them for your unique mobile application.

Curious yet about TESTCo’s mobile app testing services? Here’s more information on our capabilities.

No Mobile Application Testing Strategy? Here Comes the Pain.

If you don’t answer those questions, you can expect to see (and feel) some of these pains:

  1. Your mobile works fine on the most recent iPhone but fails with previous versions of iOS and some Android devices. And, only a small portion of your audience has the most recent iPhone.
  2. Your mobile application works great on Wifi but fails with errors on a cellular network if a user is talking on the phone and using your app.
  3. Your mobile application fails and crashes when the server is slow returning complex query results.
  4. Your mobile application works fine until there are over 900 simultaneous users – and then it displays the wrong query results.
  5. Your mobile application is easily penetrated and accounts are hijacked.
  6. Your mobile application UI works great for the engineers who built it but ordinary users struggle to understand how to use it.

Build Your Own Simple Strategy – or Let Us Help You!

The best solution is to build a simple mobile application testing strategy unique to your project. It’s easy if you follow this guide I recently published.  At TESTCo, we build and evaluate several test strategies for every software and mobile application testing project. If you don’t want to build your’s, we can build it for you!

Schedule a call with me using the green button on this page, or call me at 888-254-9709

testco-software regression testing | agile software testing | agile testing

Agile Testing Doesn’t Have To Be Complicated

/in /by

Agile Development methodologies have become widely used and accepted as industry standards across the world. The flexibility to grow and adapt software development organizations is one of the most attractive features of Agile development. The price paid for this flexibility is buggy software. While most Agile development teams are able to produce features at a higher rate than traditional development teams, they also deliver more defects than a traditional development team. Software quality for Agile testing is needed.

Agile Software Testing Services

Agile testing involves testing from the customer perspective as early as possible, testing early and often as code becomes available and stable enough from module/unit level testing. Since working increments of the software are released often in agile software development, there is also a need to test often. Taking an agile testing for software quality control approach is key.

Download Agile Testing Report

Why Have an Agile Testing Methodology?

  • Early in the lifecycle teams don’t realize the severity of existing defects, and do not put a large effort into finding them.
  • When Critical Mass is reached, customers begin using the product in unexpected ways.
  • As product development becomes steady, the effort necessary to find the defects is not constant throughout the release cycle, and a steady team of Test Engineers is too much for some stages and not enough for other stages.
  • When the product matures, automation test scripts that have been relied on in the past no longer provide sufficient coverage, and cannot find all of the defects.

On Demand Agile testing solves these problems, it keeps product quality in step with your engineering team. TESTCo’s QA testing will make this happen.

Software Test Outsourcing

The Costly Learning Curve of Software Test Outsourcing

/in /by

There are three things you can do to avoid the costly learning curve of software test outsourcing initiatives. Why does a learning curve exist? Software is getting more complex every day.  This makes software testing even harder. The learning curve for any software testing team can be steep and it is even steeper and more costly for software test outsourcing teams.

When the learning curve to begin testing software is steep due to complexity, everybody on the team feels the cost – especially the project manager! Software engineers experience interruptions which degrades their concentration, business analysts get frustrated covering old ground and project managers churn through project timelines and budgets trying to get everything to match up.

You can’t make software less complex and you can’t roll back time (unless you have a time machine that you’re keeping secret from the rest of us!). If you can’t compress the size of the problem or flatten the learning curve then all you’re left with is tools.

Three Key Tools to Shorten the Learning Curve of Software Test Outsourcing

Here are the things we use regularly at TESTCo that have proven effective in reducing the learning curve of software test outsourcing.

  1.  Use a Functional Map – create an index of user stories/scenarios as compared to primary function points in the software where the intersection of a user story and a function point denotes a test case or collection of test cases. This index provide multiple benefits starting from Day 1. Contact Us to see how a Functional Map could help your software testing team.
  2. Build a Value Proposition – when your entire software test outsourcing team understands and is aligned behind BOTH your technical and business goals, your QA team becomes a key strategic lever to move your software project directly towards your business and technical goals every day. Reach Out to Us to learn more about our Value Proposition Process.
  3. Add a QA Manager for your software test outsourcing team. At TESTCo, we believe that managers manage and engineers engineer. Engineers almost always perform at a higher level when they are led and managed by an experienced technical manager. We believe so strongly in the value and benefit of a QA Manager that we provide one, at no additional cost, for every TESTCo client.

If the learning curve of software test outsourcing is holding you back, we’d enjoy the opportunity to share how we’re different and more successful for our clients.

Automatic Software Testing | Automated Software Testing | Software Test Automation

Calculating the Maintenance Costs of Automatic Software Testing Suites

/in /by

Is maintaining automatic software testing* an investment, expense, or just a problem? You’ve paid a lot of money to automate software testing and all was working just fine,  but as time goes on and new features are added to your application, the automatic software testing suite is breaking more often and you’re getting frustrated.

Software Test Automation Tools are Changing Rapidly – Double Check

Before you jump off into a test automation rebuild project, it’s worthwhile to do a little bit of due diligence to make sure your platform and software test automation tool will continue to serve you into the future.

  • Your choices for software to automate your testing activities is growing and changing rapidly.  There are literally new software test automation tools announced every month. They tend to be born or created from a need to solve a unique test automation problem. Make sure your software test automation tool is synchronized with your technical needs and supports your technology platform.
  • Software test automation tools change versions and compatibility frequently. As new software technologies are built and release, software test automation tools change to accommodate those new technologies.  In some cases, these upgrades can cause havoc with the scripts you’ve already built for the previous version. How stable and “future proof” is your selected tool?
  • Software test automation tools also fail frequently as businesses. How can you tell if the company you purchase your tool from will be in business next year? There are many open source choices for software test automation but those tools have a risk as well. Will the open source community continue to be interested enough in that test automation tool to keep it working with the current changes in technology?

Maintaining Software Test Automation is Not Easy

  • You probably paid a lot of money to get your software test automation working properly but the return on your initial investment is fading as time goes by – surely that’s not how it is supposed to go.
  • Your software test automation suite worked just fine when it was originally built but today, it’s throwing more and more false errors – and that doesn’t help you achieve your business and technical goals.
  •  Your software development team is too busy writing new features to keep the test automation updated and, they aren’t really interested in working with those tools.
  • Your internal QA Team has trouble finding, selecting and keeping someone talented enough to maintain your software test automation suite – the learning curve is steep and maintaining code is much more challenging than just writing new code.
  • You’ve looked for outsourced help (again) but your software test automation maintenance project just isn’t quite big enough to interest the big offshore technology firms.

Push the Reset Button for Automatic Software Testing

  1. Build a business case – nothing complex – back of the napkin calculations should be sufficient. Gain a deeper understanding of your specific technology and business goals and how they relate to continuing to invest in software test automation. Once you are clear on your specific goals and expected returns, your choices will become much more distinct and easy to evaluate.
  2. Treat test automation false alerts as defects – just like you do with your application software. Use a defect tracker just like you do with your application software. Then, simply cross check application defects reported by your software test automation with the software test automation defects and automatically rule them as false and then use manual testing to verify. This won’t eliminate the time waiting problem but it will reduce it and stop the confidence erosion.
  3. Move your test automation engineer into the software development team. When your test automation work is aligned with the development project timeline and not the QA team timeline, your software test automation will be ready when the software is ready, not a few weeks later.
  4. Start building a software Test Automation Team. Hire a Jr Test Automation engineer to learn from your current Test Automation engineer. Not only is this a great strategy for succession, your software test automation suite will suffer fewer issues and grow quicker to match your software application.
  5. Find an outsourcing partner (not a”vendor”) to work with you over time to specifically meet your business and technical goals. Most outsourcing companies try to be as large and broad as possible to meet every need possible. A few outsourcing companies specialize in automatic software testing and test automation. We specialize so all of our talent will be perfectly suited to solving your software test automation maintenance problem.
^Most readers will dislike my use of the term, “automatic software testing suites”, in this post. The term is not accurate. “Automated software testing suites” is the accepted phrase used in the software testing world. Accepted or not, our research indicates that “automatic software testing” is used by a large number of folks as they search for information on the topic. Therefore, I’ve devoted this post to the folks who prefer “automatic” over “automated”. We do a great job for both. -JCH