QA Outsourcing Scorecards
QA Outsourcing is usually an expense – and that’s wasteful. But, it could be a valuable investment with a substantial return if you do a couple simple things.
Every software project needs to be tested. You know that already. How you handle the outcomes and outputs from QA Outsourcing will determine if you get the most for your time, money and effort.
The Value you Receive from QA Outsourcing Starts at the Beginning – Your Expectations
Did you ever wonder why most people are disappointed with outsourced work? In most cases, it’s because they weren’t clear about the value they actually wanted before they started. If you don’t know what you want, how can you ever be satisfied?
Start with these questions to help you uncover and capture every ounce of value that you can from your QA Outsourcing partners.
– Is your QA outsourcing partner helping you track software defects?
– Do you know the status of your test plan on a daily basis?
– By track I mean are they providing a concise scorecard that is meaningful to you and the development team?
Tracking your software defects is important and valuable, but just tracking the features and defects (you are using your defect tracking for new features, aren’t you?) is not enough to improve how you build software.
Is your QA outsourcing partner providing you with a clear way to understand your software quality so you can make a good decision about when to release your software?
Releasing your software too soon with too many defects can backfire on you!
Is your QA Outsourcing partner helping you build better software and a stronger business that attracts and retains valuable customers?
When Scorecards Help Development and Testing
Onboarding someone outside of your team can be very challenging. Where do you even start to help them understand the size and scope of your project? One method is to learn by doing based on a clear list of actions, deliverables and dates. This simple methods quickly leads to a clear focus and a simple scorecard that monitors your biggest challenges.
Here are some potential problems that you might be struggling with that can be solved with a software quality scorecard.
The QA Outsourcing Team Doesn’t Know Your Software Quality or Production Goals
A lack of focus and clarity is the #1 reason most goals are not achieved. A scorecard can quickly uncover who doesn’t know or understand your most critical criteria – your goals! This same scorecard can also show you, every day, exactly who is focusing their effort to achieve your goal. The goals should be clear and concise and re-written in the scorecard every day. Each person’s plan should show you the steps and outcomes that will be accomplished each day in pursuit of achieving your goal. You should be able to read and understand each step towards the achievement of your goals. If it doesn’t make sense, a scorecard makes it easy to see and make adjustments that day so you can get the project or person back on track to achieve your software quality and production goals.
Your Team Doesn’t Have a Daily Plan
A scorecard can record each person’s daily plan. A daily scrum or stand-up meeting is not a plan. The true value in a plan lies in the work and thinking that created the plan. A plan is simply a measure of the amount and quality of thinking that produced the plan. Without a daily plan, you are relying on luck and fleeting good intentions to accomplish your goals.
A scorecard serves as a measurable and reviewable plan for each individual on the QA team. The scorecard contains the tasks that each person is supposed to accomplish and continuously using a scorecard to measure progress leads to self-improvement. This is because a scorecard asks every team member for areas to improve on or suggestions after each day. Continuously addressing weaknesses using scorecards will lead to individual improvement and team improvement.
Your Team Doesn’t Have a Daily Summary
A scorecard can record each person’s daily summary. A daily summary can provide every person on the team with an insight into coordinating their work with everyone else on the team. Without a daily summary, the rest of the team is left to guess (or infer from the daily standup the next morning) exactly what was accomplished and if their dependent work can proceed.
After recording each person’s daily summary, the QA team can review the scorecards cumulatively and see where the team’s strengths and weaknesses are. This exercise can serve as a team daily summary and the team can brainstorm plans to address any weaknesses within the individual scorecards.
Your Team Finds Blocking Issues
Your QA outsourcing partner’s work can’t continue until someone else clears some sort of blockage. Blockers occur every day – it is the nature of fast paced software development. But, without score carding blockers, their frequency, resolution and cost can’t be determined – and you’re doomed to continue facing them at inconvenient moments!
Your Team “Pushes” to Meet the Goal
This frequently occurs in the world of software development and is probably the biggest reason software teams make mistakes! A simple scorecard from every member of the team could have alerted everyone else to the time crunch issue way before it occurred. A little forward notice could have prevented the “push” and the corresponding wasted energy by the rest of the team.
The solution is simple – use a daily scorecard from each member of your software development team – at a minimum from your software QA team since they are the prime “measures” of your software quality and production capacity.
The daily scorecard can highlight issues that teams make when they are rushing to release a product. By having scorecards to quantify the work that each member of the development team is doing, teams can find the root cause of any mistakes made and prevent future instances from occurring. Even if your team is pushing to meet a deadline, using scorecards lets every individual know their responsibility on a daily basis and teams can avoid the routine mistakes that occur while rushing.
The Problem with Scorecards and QA Outsourcing
Are you uncomfortable measuring and judging someone’s performance? Does giving “feedback” give you the “willies”? Using a scorecard is a very easy way to clearly identify performance problems without the uncomfortable feedback.
Most people do not want to be measured or judged. And yet, measurement, analysis and correction is the cornerstone of ANY production process or quality assurance method. Think about it – if you can’t (or won’t) measure it, how can you ever be sure – of anything? Would you want to drive a car, bake a cake or raise your family in a home that wasn’t inspected and measured/scored for quality while it was being built? Yeah, not really, so why would you tolerate that in your software project?
Most software QA outsourcing companies and their testers do not want to be measured. The perception is that bad things will happen so ignoring measurement extends the work for another day – in the shadows.
TESTCo’s team is always ready to share scorecards because transparency is a part of the work culture. For other QA outsourcing vendors, this can be very different to their normal work protocols and their staff resist using scorecards. The transparency provided by scorecards is a net positive influence on most QA teams. Although it might be uncomfortable at first to be evaluated so transparently, once teams adjust, it will be normal and it will allow teams to improve their weaknesses.
When selecting a vendor for your QA needs, asking for their scorecards can be a great way to vet potential vendors. Vendors who are experienced in QA know what scorecards are and will readily provide their team’s scorecards. Ask your software QA Outsourcing vendor for their scorecards. If you’re talking with TESTCo, we’ll happily share our scorecards with you. If you’re talking to someone else – expect crickets
The QA Outsourcing Team Should be Providing the Scorecard
Most folks will either run the other way or try to fight you when asked to measure themselves. At TESTCo, we THRIVE on being measured! In fact, if you aren’t measuring our performance, we’re doing it anyway and we’ll bring one with us when we come help you!
At TESTCo, we issue report/scorecards twice daily. The morning report/scorecard reports the daily plan and establishes the most important metrics and outcomes for the day. It also has target values for each metric or outcome. The evening report/scorecard reports the daily actual production metrics, plan completions and adjustments, any blockers waiting resolution and statistics on the most important daily metrics and outcomes. These twice-daily reports are distributed to the QA Manager and anyone else that may be interested or connected to the workflow.
The true value of a plan is the planning. Planning is the process of moving from your current state to a desired state – a goal. Moving from one state to another requires measurement. Daily measurement of critical outcomes is required to achieve your goal as efficiently as possible.
A final thought on twice-daily reports. I always get push back from my Test Engineers when I initially ask them for twice-daily reports. It looks like a big hassle! But, being perfectly clear on goals and status, twice each day, quickly turns them into big fans of daily scorecards because they know they are highly valuable and efficient engineers – the twice daily scorecard give them the opportunity to demonstrate their talent and drive to their customers and managers. TESTCo Test Engineers very quickly become fans of scorecards because it gives them the feedback they seek to become even better at delivering the value they are committed to. Just ask one if you don’t believe me! For other client comments, check out our testimonials page.
What sort of project should you assign to a new outsourced software QA company? I invite you to read my post on this topic; the recommendation may surprise you.
What to Look for in a QA Outsourcing Software Quality Scorecard
A Software Quality Scorecard should be customized so that it meets the following criteria.
- Takes less than 5 minutes to create.
- Contains your software production and quality goals (these don’t change) for the current period.
- Contains brief On-Plan, Ahead-of-Plan or Behind-Plan indicator and short explanation of why.
- Contains 3-5 quality metrics related to your production and quality goals that clearly tell you where you stand relative to your goals.
- Contains any other information you need to make decisions on releasing your software project and achieving your goals.
- Contains 1-2 recommendations for improvement from the person creating the scorecard.
These criteria allow your scorecard to have enough information and it makes it relatively easy to create. If your QA team is just spending 5 minutes per day on creating scorecards, it creates a standard for each individual in the group and it doesn’t take significant time out of anyone’s schedule. Additionally, scorecards don’t have to follow this exact guideline, if your team has specific restrictions or advantages you should incorporate them.
The criteria above serve as a general guideline to make scorecards simple, yet effective for each team that utilizes them. With clearly defined and simple scorecards, your QA team can start implementing the scorecards today and see the benefits. Once ingrained in your work routine, scorecards will be invaluable tools for project management. If you’d like to learn more about creating scorecards and how your team can implement them, talk to TESTCo today for a free consultation.
