Metrics Matter... Testing Parallels between the Covid19 crisis and my experiences in Software Engineering.
Covid19 Vs A Software Quality Pandemic
How the worlds governments have reacted to testing of Covid19 reminds me of certain execs in large enterprises. Some sponsor active leadership and look to empower their teams, destroy hierarchical reporting structures in a psychologically safe environment which Foster’s innovation from engineering teams, whilst other utter nonsense about herd immunity or in engineering that might be a status chart, whilst passing critical conversations over core engineering practices (TDD, BDD, Code Quality, etc).
I’m drawing a parallel here between the health care capacity to operate and that of a large engineering program. In order to thrive we need to have capacity and culture to focus on delivering customer centric solutions that are founded in innovation and quality.
What are folks on your programme discussing ???
When it comes to engineering excellence the good news is you don’t need to find perfection, you just need to level set as a programme where you want to get to, explain and discuss what your goals are and enable your talent to understand how you achieve it. Ideally with a dashboard that’s the trusted source of truth. Then keep iterating... easy right?
Exec… if we turn the quality gates on it will slow us down. Engineer… yes, we may have to go slow to go faster than if we leave them disabled.
How do you flatten the curve?
It’s a cultural shift in getting everyone to understand why testing is important and what’s in it for them. Execs who invest in enabling cultural shifts in this space may have to explain why they are moving slow in order to go faster. But they will win the war. Though if you leave it too late you will end up with a lack of confidence in your main develop branch as you watch lead time plummet off a cliff.
Creating the culture
A culture of testing is based on the premise that no single tool, test, or person can guarantee quality. For quality to be a solid part of the company culture, everyone needs to support and contribute to it—from management and the business to developers, architects, and product managers.
As with any project, we will have some challenges to solve. When I first started this culture shift with my latest partner. I found that onboarding of teams, having time dedicated to proper team norms and test strategies are non negotiable. I always tell engineers, don’t worry how we write a test, how would you test this manually?
Winning conversations with execs
So how do you frame your conversations better with execs? With Data! Anyone ever heard of accelerate metrics which focus on high performance engineering?
The moral of the story, borne out in the data, is this: improvements in software delivery are possible for every team and in every company, as long as leadership provides consistent support — including time, actions, and resources — demonstrating a true commitment to improvement, and as long as team members commit themselves to the work.
Nicole Forgren, Jez Humble and Gene Kim coin four key metrics:
Lead Time: the time it takes to go from a customer making a request to the request being fulfilled.
Deployment Frequency: frequency as a proxy for batch size since it is easy to measure and typically has low variability. In other words, (desirable) smaller batches correlate with higher deployment frequency.
Mean Time to Restore (MTTR): as software failures are expected, it makes more sense to measure how quickly teams recover from failure than how often they happen.
Change Fail Percentage: a proxy measure for quality throughout the process.
These KPI’s are powerful because they focus on global outcomes and capture many intangibles (such as “quality”) throughout the Software Development Lifecycle (SDLC). Accelerate proposes these KPIs, but does not offer a method of capturing them throughout the SDLC. This post suggests possible approaches and solutions for measuring these KPI’s.
Other engineering metrics such as code coverage, big, vulnerabilities, code duplication, etc are all great metrics and should be set as enabling constraints within any decent pipeline, but if you want to frame to an exec who doesn’t get these, they will understand lead time. Accelerate metrics help us frame to leaders engineering metrics which matter.
Testing has never been more relevant in today’s market where it’s all about enabling business outcome and speed to market, here’s to my engineering peers who want to flatten the tech debt curve.
Stay safe, avoid change approval boards and follow me on: @belfast_nerd
