Metrics Matter... Testing Parallels between the Covid19 crisis and my experiences in Software Engineering.

Covid19 Vs A Software Quality Pandemic

I’m drawing a parallel here between the health care capacity to operate and that of a large engineering program. In order to thrive we need to have capacity and culture to focus on delivering customer centric solutions that are founded in innovation and quality.

What are folks on your programme discussing ???

When it comes to engineering excellence the good news is you don’t need to find perfection, you just need to level set as a programme where you want to get to, explain and discuss what your goals are and enable your talent to understand how you achieve it. Ideally with a dashboard that’s the trusted source of truth. Then keep iterating... easy right?

Exec… if we turn the quality gates on it will slow us down. Engineer… yes, we may have to go slow to go faster than if we leave them disabled.

How do you flatten the curve?

Creating the culture

As with any project, we will have some challenges to solve. When I first started this culture shift with my latest partner. I found that onboarding of teams, having time dedicated to proper team norms and test strategies are non negotiable. I always tell engineers, don’t worry how we write a test, how would you test this manually?

Winning conversations with execs

The moral of the story, borne out in the data, is this: improvements in software delivery are possible for every team and in every company, as long as leadership provides consistent support — including time, actions, and resources — demonstrating a true commitment to improvement, and as long as team members commit themselves to the work.

Nicole Forgren, Jez Humble and Gene Kim coin four key metrics:

Lead Time: the time it takes to go from a customer making a request to the request being fulfilled.

Deployment Frequency: frequency as a proxy for batch size since it is easy to measure and typically has low variability. In other words, (desirable) smaller batches correlate with higher deployment frequency.

Mean Time to Restore (MTTR): as software failures are expected, it makes more sense to measure how quickly teams recover from failure than how often they happen.

Change Fail Percentage: a proxy measure for quality throughout the process.

These KPI’s are powerful because they focus on global outcomes and capture many intangibles (such as “quality”) throughout the Software Development Lifecycle (SDLC). Accelerate proposes these KPIs, but does not offer a method of capturing them throughout the SDLC. This post suggests possible approaches and solutions for measuring these KPI’s.

Other engineering metrics such as code coverage, big, vulnerabilities, code duplication, etc are all great metrics and should be set as enabling constraints within any decent pipeline, but if you want to frame to an exec who doesn’t get these, they will understand lead time. Accelerate metrics help us frame to leaders engineering metrics which matter.

Testing has never been more relevant in today’s market where it’s all about enabling business outcome and speed to market, here’s to my engineering peers who want to flatten the tech debt curve.

Stay safe, avoid change approval boards and follow me on: @belfast_nerd

Belfast born, proud father and husband to Noelle living in Connemara. Enabling Creativity is my passion. Opinions are my own…