In response to rising user expectations, quality software that is consistently usable, secure and available is a necessity for businesses. Different industry sectors, as well as public sectors, rely on their IT Systems to support their core business functions. Defects on software systems can cost them time and money, potential reputational damage or even provoke serious injury.
Users have raised the bar regarding quality expectations and have less tolerance to failing technologies. Incorporating quality assurance (QA) best practices into the delivery process at the beginning of every project is imperative. In order to provide quality products, there must be a great emphasis on quality assurance activities.
QA Scorecards are supporting tools that monitor QA activities and their outcome throughout the entire lifecycle of each project. By monitoring and analyzing them, your technology partner can better deliver products that are worthy of your brand. Taking quality into consideration from the beginning of each project and monitoring results against project goals allows your project to be set up for success. These goals must be defined based on industry standards regarding quality expectations.
Gathering, maintaining and analyzing measures can be costly and also time consuming. Therefore, it is very important to define the correct set of measures to be gathered. A measure set should be complete enough to guide each team through the self-learning and continuous improvement path, but without introducing undesired bureaucracy into the process.
In the right time and the right place
After years of evolving QA measures sets, there is now a defined a set of scorecards that allows monitoring quality against the expectations that have been set with clients. It is imperative to keep accurate and up-to-date information, which can be done by automatically gathering information from a unified project management system. If accurate information is not feeding the measures, even the best measurement system will not be reliable. Therefore, it is highly important to accurately complete the information required to gather the measures right in the moment where the activity is being performed (i.e. The defect is being reported).
Where do we want to be?
It is as important to have a defined goal for each of the measures, as it is to have a well-defined measure set. Having goals is very useful in the analysis of the measure, raising a preventive alert, which is not necessarily a problem, in case the goal has not been met. By identifying a potential problem early on in the process, you can easily minimize the impact of it without affecting the product quality or project budget.
What do we need to know?
A robust QA measure set must include a complete set of data that guides a continuous improvement effort, based on numbers and not only perceptions. Some of the measures that should be gathered and monitored are the Intercept Rate, which represents a rate between defects detected internally by your technology partner and defects detected by you, the client; Detected Before Released, which compares the defects detected inside each Sprint against defects found after the Sprint; QA Effort %, Defects By Category, showcasing overall defects detected internally and detected by you, the client, etc.
Complementary measures are also a key when taking advantage of the scorecards. Once the measures’ results have been determined, the complement information provides a deeper understanding around the measures and the ability to execute couse-effects analysis. For example, comparing QA Effort % vs. External Defect Density helps identify common patterns. Measuring defect intercept rates and complementing this measure with client-detected defects analyzed by category empowers choice when deciding where to invest the QA effort.
Same information, multiple focuses
Having the same piece of information available for individuals who hold different positions at various levels within an organization has been a key success factor for the QA Scorecards implementation. For example, at the team level, a team member can get benefit from the measure set to guide their Retrospective session meetings. At an Executive level, the same information should be analyzed to plan improvement actions across the company, to understand where the weakest points are, and determine where to focus improvement efforts.
What are the measures telling us?
Measures should be analyzed as a snapshot of the current status and based on their trend lines. From reviewing trend lines, it is possible to obtain an evolution of the measures’ results and analyze the repercussions of improvement action items. It also allows you to identify outliers, understand them and determine if a specific action is needed against them.
Quality Assurance measure sets must guide continuous improvement efforts, without introducing extra effort to gather them. Effort must be invested in analyzing what has been gathered, comparing different results and establishing patterns in order to understand possible weak points and improvement opportunities. This guides us to consistently delivering to our clients with the expected quality level and allows us to keep raising our quality bar project by project.
