DIY Website Testing

You have a website – an important website –it represents your company (or is your company), and you want to make certain it works properly. You know the site should be tested, but should you consider a website testing service, or gather a few employees, friends and relatives together for an afternoon of testing while munching on pizza? One option costs money, the other doesn’t (okay, you have to spring for a few pizzas).

You might ask yourself, what do other companies do? The sad truth is that too many businesses either don’t test their websites or just do it poorly.

They pay for it on the back end — high bounce rates, abandoned shopping carts, customer complaints and not enough lead forms or converted sales.

Some folks just semi-test it on their own computer/phone and call it done. Who needs outsourced website testing services, right?

While that’s  better than nothing, it leaves you wide open for delivering a user experience that hurts your business. Let me make it simpler and easier for you with this report.

What follows is great advice for setting up a test plan for your website. I’ve even included a link to a free report, “Website Testing Cheat Sheet”.

What Does It Take To Test A Website Yourself? The DIY Method.

First, you’ll need a Goal, a Strategy, and a Plan.

Sure, you can just sit down and start testing but you probably won’t get what you want. Why? Because you’re just performing semi-random testing to see if you can find some “bugs”. Sure, that’s helpful but not very economical or efficient. Additionally, if you aren’t careful with documenting and reporting the “bug” you find during your website testing, you won’t be able to determine if and/or when that “bug” was fixed.

Set The Goal For Your Website Testing Project

Zero defects? Works okay for 90% of users/visitors? No critical defects but some annoyances are okay? You’re about to spend your time or money in a business activity, so you DO want a return or positive outcome for your time and money, right?

If you know what your quality goal is, then you can easily determine how much time and money it’s wise to spend in achieving that goal so that it does produce the outcome or return that you desire.

This holds true for doing it yourself or using a website testing service. Knowing your goal helps you determine the best budget for your website testing project and achieving your quality and business goals.

Make The Goal Realistic

Your quality goal for your website testing project needs to be realistic and relative to your budget for time and money. If you have little time and little money, you won’t be able to achieve a stringent quality goal. A “Zero Defects” quality goal for your website testing project can be expensive and time consuming to achieve. This type of goal is worthwhile for moon landings and space shuttle docking but doesn’t always make sense for a website.

A better goal for your website testing project may be “Works on 90% of known customer browsers and devices”. With this type of goal, you can easily calculate the number and variety of web browsers and devices that need to be tested within a small website testing project budget.

Now That You Have Your Goal, What Is Your Strategy?

A Quality or Testing Strategy is a collection of Assets (people, resources, time, money, etc.), each with a unique Advantage that can be used or perform a certain Action to achieve your goal.

One strategy is DIY Website Testing – Use your time (Asset) where you are experienced with your application (Advantage) and is already paid for (Advantage) to run ad-hoc tests (Action).

Another strategy is Outsourced Website Testing Services – Use trained professionals that you pay (Asset) who have and use an industry standard testing strategy (Advantage) to build and run a regression testing suite (Action).

These are two very simple examples among countless quality and testing strategies. The best strategy is the one that achieves your quality goal for the lowest cost with the highest outcome or return.

It might help to think of strategies as various ways to optimize Faster, Better and Cheaper!

Every strategy needs a plan to coordinate the Actions of the Assets with Advantages (Strategy) that you’ve selected.

Is It The Plan Or The Planning?

Plans are important for every website testing project. But, at TESTCo, we’ve learned that the planning is much more valuable than the plan. Why? Most plans blow up within the first 10-20% of the time allotted. The requirements might or might not change but some of your assumptions are almost guaranteed to be wrong or change. The environment or situation around you is the most common change that impacts a plan for website testing projects. Someone always seems to need something sooner than they thought previously. Or, your best and most important technical person goes on an unexpected vacation. What happens to the plan when something changes or goes wrong?

You replan–and quickly!

But, the second (and third and fourth, sometimes) time you replan, you’re smarter and more effective in your planning. Why? Because you’ve already thought through most of the requirements and assumptions and you’re simply factoring in changes. You get a little smarter and a little better every time you need to replan. And, almost every replan provides better outcomes and greater accuracy.

At TESTCo, our website testing service plans tend to be very accurate, but only because we’ve done so many of them and already have backup plans and test engineers ready to go if (and when) a change happens.

So, Why Plan (or Replan) If It’s Going To Blow Up Anyway?

Great question – because your project and team need your leadership and management. They (maybe just you sometimes) need and want to be told what to do to achieve the goal they’ve agreed is important. They need to know you have some level of confidence and certainty in the project and your plan is a primary component to communicate your confidence.

Website testing services appears to be quite simple. But, with mobile devices and browsers changing and upgrading frequently, a plan to achieve your specific goals will always your best friend.

Planning is even more powerful. A plan just communicates the timeline and the outcome. The process of planning involves (or should involve) almost everyone on the team. Planning provides the opportunity to raise, discuss and resolve issues and doubts about achieving the goal. Talking and discussing how you will achieve a goal will almost always trump a Gantt chart or project plan. Sure, you need the plan but the gold is mined in the plannING!

So, go ahead and replan that rascal if your current plan isn’t on track!

A Quick Checklist To Cover the Basics Of Website Testing

Does it work properly?

• Does every input form correctly save the data and send the appropriate emails?
• Does every link go to the right place?
• Does every page and form work properly, even when you use the Back button on the browser?

Does it display properly?

• Does every page load quickly and fully?
• Does every graphic render fully and completely?
• Does every animation display properly?

OK, that’s it.  Congrats, you’re done.

Oh, Wait a Minute. What About Mobile?

Which browser did you use when you tested?

And that browser was running on which operating system?

Oh, you used your phone…great!

OK, which mobile browser did you use?

And what type of mobile device do you have? Apple or Android?

And what operating system version does it have?

Wait! Why are you asking me all these questions??????

Well, here’s a dirty secret about websites and today’s web browsers on desktop and mobile devices.

They’re all different and they all have different “issues” with how they try to display your web designer’s newest and coolest graphics and display treatments.

What does that even mean???

It means that your website may look perfect on a Mac with Safari but render terribly on an Android phone or iPad running the previous operating system.

There are literally hundreds of different possible combinations.

Selecting The Right Browsers And Devices To Test

There are two references to consider when deciding which browsers and devices to test in your website testing project.

1. Browser and mobile device market share information is frequently published on the web.  Find the most recent research report (I won’t link here because they change frequently, just Google it) and note the most common browsers and devices.
2. Check your website server logs (or a web analytics application) and find the one that tells you which operating systems and browsers your visitors are using.  No need to test for browsers that aren’t popular and aren’t visiting your site.

Match list #1 against list #2 and pick the combinations that are most popular in the market and most used by your website visitors.

Now, you have your list of devices and browsers (we call them flavors).

You should probably have about 6-12 flavors.

Now, you simply run the same tests on this list of browsers and devices that you ran on your own browser. You can begin to understand why hiring an outsourced website testing service makes sense in a lot of cases.

But I Don’t Have All Those Flavors!

What, you don’t have all those devices and computers on which to test?

Not a problem, there are a bunch of “Cross Browser Testing” tools available. They are very handy and while most use emulators instead of the real devices, you will be able to see and correct most of the defects that occur on the actual devices.

Now you’re ready to test your website.

How Long Will It Take? Counting The Hours.

How much time will you need to complete your website testing project?

The math is easy.

Count or estimate the number of pages and forms on your website.

On average, it takes about five minutes to fully evaluate and test a single web page on a single device.

Now, multiply number of pages and forms by five minutes and then divide by 12 to get the number of hours it should take to test your website with EACH flavor.

Then, multiply those hours by the number of flavors you want to test.

Now, you have a good rough estimate of the amount of time you’re going to be spending at night or on the weekend testing your website.

A Small Note On Regression Testing And What It Means

Regression testing means “testing it again” after fixes have been completed from the bugs you found during your last round of testing.

Yes, you will find defects when you test – almost every time. If you don’t find any defects, you should be very suspicious about the quality of the testing.

After you find the defects, your developer or software engineers will read your defect (write it very carefully and thoroughly so they can quickly reproduce it and then fix it) and make changes to prevent that defect from happening.

This will need to be rested to confirm that it has been fixed. This is called validation testing – you are validating that the defect you found has been accurately fixed.

There is also the possibility (or even probability) that your developer or software engineers accidentally caused another defect somewhere else when they were fixing the defect you found. Make sense? They fixed something but that fix caused another defect.

You need to find these and chasing them down can be maddening sometimes. This is called regression testing – testing again to make sure nothing else was broken during the latest round of fixes.

And, theoretically, you can keep doing this until you have Zero Defects. But, you should consider that performing regression testing until you achieve Zero Defects will probably be very expensive. Think ahead and plan for at least one round of regression testing (they do tend to go faster than your first round of testing) and possibly two rounds depending on the number and severity of the defects you find in each round of testing.

That’s Too Long!

Yeah, I’m sorry.  I knew you’d probably be surprised when you actually did the detailed math for testing and regression testing.

Website testing is relatively simple but it’s not easy. There’s a LOT of ground to cover, a lot of repetition and it requires an intense attention to detail – especially when you’re looking at the same pages over and over, just on slightly different devices.

If you have a small(ish) website and/or you aren’t very busy, you can certainly do this yourself.

Be careful, though, and make sure you’re paying close attention as you do it – website errors can be sneaky and hard to notice. Maybe get a pot of coffee ready?

Rather than consume more caffeine, seriously consider outsourcing your website testing project to a website testing service that can run the necessary tests while you and your team focus on other priorities that build and grow your business.

The Ultimate Question – Can I Afford A Website Testing Service?

Building a website or a business is partly the process of making wise investments – particularly with your scarcest two resources – time and money.

Just within your website project, you have to decide several times whether you’re going to invest AT ALL or if you’re going to spend your scarce time and/or money.

If you don’t have a solid goal for your business or website project, your decision on whether to invest is going to be hard. After all, how can you decide how much to invest if you don’t know what you really want?

But, let’s say you do have a goal for your website. Let’s say it’s to increase your sales by $50k/month within six months. That’s a solid and worthy goal!

And, let’s say you spent around $10k with a website design agency to get your website built to achieve your goal.

You’re small and that was a big chunk of money to spend but you knew you couldn’t do all the research, design and coding yourself. Sure, you can roll a little HTML/CSS but a modern production/commerce ready website is beyond just HTML/CSS. You made the smart call and got expert help.

And, they’ve completed their work and your website is new and shiny. You can clearly see that your investment of $10k for a monthly return of $50k in the near future is going to be achieved. You’re excited!

Your website design agency says they tested it and it looks good to you. You’re ready to launch it.

But wait – how well did they test it?

You’re not sure so you decide to spend an evening testing it yourself.  Just to see.

And, yup, there it is – a small error on your order screen when someone clicks the dreaded “Back” button. Yeah, they’re going to do this no matter how many times you tell them not to.

Now, you’re a little concerned. If there’s that bug that they didn’t catch, might there be others?

You’re going to invest another $10k in advertising and promotion when you launch your website. That’s another $10k investment but it’s needed to get the word out so you can achieve your goal.

You can’t afford to have any website errors after you’ve spent $20k on building and promoting it.

Should you spend more time or money testing?

Let’s Do The Math

If you do it yourself, it will cost your direct time and it will also cost lost opportunities that you could have been pursuing while you spent time testing your website.

And, you’re a business owner or manager and you’re not an experienced website tester. You’re going to be only marginally efficient and partially productive simply because you don’t really know what you’re doing!

So, if your average annual earnings are near $75k and you’re only half as productive as a trained Test Engineer, your cost to test your website is nearly $75/hr and it’s going to take you twice as long to finish. And that doesn’t even count the other business opportunities (and problems, we have those, too) that you missed while you were trying to get your website tested.

Roughly speaking, it will take  about six days to thoroughly test your website for full functionality and across all the browsers and mobile devices that your users have.

The math says $75/hr x (6 days @ 8 hrs/day) = $3,600.

If you asked TESTCo to scope this project our proposal would be closer to three days (because we do this all the time and know what we’re doing) and the cost would be closer to $56/hr x (3 days @ 8 hrs/day) = $1,344 and in half the time! Much less than half of your direct cost and without the lost business opportunities!

Save time.

Save money.

Achieve your most important goals.

A Website Testing Service Is Sounding Pretty Good Right Now

I understand. We have a number of clients who came to us after they tried testing their site in-house.

We can help in a number of different ways:

1. We can answer your questions. Just ask, and we’ll be happy to answer.  No charge!
2. We can help with your research. Don’t want to figure out the right flavors? Let us tell you what we’d propose!
3. We can test it for you. You’ll be surprised at how affordable we can be – we definitely cost less than what it would cost to do it yourself!

Here’s additional information about the TESTCo website testing service.

At Least Do Something!

Please, at least do a little testing on your website.  The rest of us who visit your site will be thankful.

If you can’t do it yourself, we can help you with our website testing services. It doesn’t hurt to talk. Click the green button in upper right of this page to request a call back, or reach out to me by phone right now. 888-254-9709