Meaningful Agile Metrics
People like to set deadlines and to measure things, this is how we roll. In my classes, I hear many questions about how to measure software quality in an agile context. Many teams who switched to agile continue to measure quality like they used to do it before: number of found bugs, number of closed bugs, number of automated tests, stories rejected by PO etc. But after some time they found it was not helpful.
I am very cautious with metrics. First of all, because out of context they are misleading. Secondly, people tend to focus on what they are measuring, because they want to improve that thing. If we focus only on the numbers, it can be very dangerous.
If you measure “stories rejected by PO” you will work with those numbers, but not address the real reason behind the problem, so in the majority of cases, it will not change. Just like in Dilbert comics or more current issue with Trump who did not want to test people for Corona, because then the numbers will sky rocket. He “likes the numbers where they are”.
OK, let’s assume I convinced you – metrics are tricky. But what can we do? My first suggestion is to evaluate if we really need a metric. If yes – for what and why? Sometimes the answer to those questions can give the solution. “Because we’ve always measured something!”, “Our management / stakeholders want to know it!”, “Our boss wants to know to whom he should pay a bonus”. Some of those are indicators of bad management, others of bad agile approach and I hope you understand that no metric will solve those problems.
If your challenge is nothing like this, then for the second round, I have a story to tell. Two years ago I took an Agile Testing Fellowship TrainTheTrainer course. I had two major highlights in the training, two things that changed my agile approach. I remember so vividly the moment when Janet Gregory announced: “forget about traceability in agile! We don’t need it”!
It came as a shock. How?! Why?? No traceability??? But, but, but… how will I work without traceability? Traceability always was a part of software development – starting from requirement elicitation until support and maintenance. I needed some time to process it. In the end I had to agree with Janet. If everything from beginning to end works, if testing is a part of development, if the team is the one who leads, if stakeholders support the team and give freedom of a decision then of course we don’t need a traceability. It becomes one and the same as the process.After reevaluation of traceability, I started to revise other old “religions”. As you can guess, one of them was metrics. My idea was – if I would remove everything I used to know about measuring quality and assume that it is baked in, what would I like to measure then? I will save you the whole rethinking process. At the end, I decided to focus on the whole team. For that I use 5 values of eXtreme Programming: Simplicity, Communication, Feedback, Respect, Courage. There are several ways to measure each of them. I will give you few examples that you know what I mean: simplicity besides known methods can be measured by checking if everyone on the team can read and understand code from somebody else. Respect I prefer to measure with closed team dot voting. For courage I find it very important to monitor how strong a team feels about pushing back or questioning (unnecessary?) user stories.
It took some courage (ha -ha) to leave old practices in the past and I must admit not with every team and company this will be possible. If you practise fragile (bad agile), if the company does not really live by agile values, it will not work.
For a dessert I want to share a video of a TED talk about missing baby dinosaurs. Maybe it gives you another perspective on known things.