From Goal to Metric – value-focused metrics

If you look at the projects in your company, my guess is that you will find several metrics which you as a loyal employee measure from sprint to sprint, from project to project. Whether you work in an agile context or in the more traditional development methods, metrics are something you encounter on a regular basis. Someone has a few individual metrics or KPIs, some have MANY. In some projects and organizations, it seems almost as if metrics are sprinkled over the projects with a gentle hand and change from day to day because someone has read or seen something smart since yesterday. But just try to stop and ask; why do we measure this? What value does it give? and who is it that needs this measurement?

Too often, metrics originate from one of several things:

• It’s the one we usually measure
• It’s the one we get “for free” when we use the XYZ tool
• It’s the latest bright idea from someone😊
• It is something that someone has read about in some book in the top 10 sold books

But what about using a method that will give you a common thread from your goals to your metrics? The method “Goal – Question – Metric” or GQM, originally defined as part of a PHD by Dr. David M. Weiss and later used by NASA’s Goddard Space Center can give you that. The method is not so new, but apparently not very well known. GQM is used as a driver to ensure a goal-oriented approach to metrics in the organization and is simple (at least in theory):

1. Identify what the goal is for quality of your project or organization

2. Identify the questions you want to ask to find out if the goal has been achieved

3. Identify the metrics needed to answer that question.

It sounds very simple, right? three quick activities and you are there. However, it often turns out that especially the first activity is proven difficult, because here you need to get your stakeholders involved, and have put into words exactly what it is for goals we must strive for in the organization. That is, it is often high-level definitions, but concrete enough to use as a basis for the next activities. It could be something like:

1. Validate the fulfillment of requirements for new functionality and new systems before they go into production.

2. Identify errors that may affect the daily use of the systems and provide the developer with sufficient information to correct the errors before they go into production

3. Use a risk-based approach to testing to reduce the risk of critical issues after release

4. Give project management and release management valuable information in support of go / no go decision

Now you have an overall picture of “what are we here for” – what are the goals we should strive to fulfill. But to be able to say something concrete about whether we have met these goals, we must have made them more measurable. Therefore, quickly move on to activity 2. Here you take the individual goal and consider; What questions should I ask to find out if the goal is met? In my example the questions could be something like:

1. Are all requirements covered by a test?

2. Have all test cases specified to meet requirements been successfully implemented?

3. Are the end users satisfied with the functionality provided?

Or for goal two:

1. How many errors have we found in production with high severity after release?

2. How many errors are rejected because they cannot be reproduced?3. Do we have many errors in production after a major release?

4. Does the end user experience a system of good quality after a major release?

Now you are heading for something more concrete, and of course you must clarify with your stakeholders whether it is the right questions you have identified – maybe even have identified them together. With these questions, you are now able to identify the concrete metrics that will add value in your context – to be able to answer whether the goals your stakeholders have set for testing have been met. But then comes the next challenge; Do you have the data and data quality you need to use the metrics…. But it’s a completely different story.

Now you can drive the other way:

1. Make the measurements
2. Answer to the questions
3. Assess whether your goals are met.

So all in all a method that looks something like this

This method can be used at several levels; Whether you are working on developing your quality policy for the company, whether you are on a large program with many stakeholders, or whether you are a small project with 10 participants. You will always have stakeholders; they will always have an expectation about what they want from you. The most important thing is that you reach a state where you measure what gives value to you, numbers in can actually use to control and that can help you improve both quality and working methods – not just those things that is easiest to measure or is up in time.