A Measures & Metrics Hierarchy?


This session has been inspired by following the journey of Solar Impulse 2 (SI2), an aeroplane powered only by solar power, as it circumnavigates the globe.

In June this year SI2 flew non-stop 5,663km from Japan to Hawaii. It took 6 days, using only solar power. The pilot sleeping in 20 minute naps, flying above and around weather systems, whilst travelling at around 30 knots, between 4,000 and 30,000 feet in altitude.


SI2 Landing in Hawaii

It sounds remarkable now describing the feat which smashed all solar powered flight records.

What was even more amazing was as the plane flew it transmitted real time data back to the control centre in Monaco, and the information was then published, live and real time, on the internet at solarimpulse.com.

This got me thinking. Firstly, wouldn’t it be amazing if our software testing projects could broadcast live, real-time metrics and measures just like Solar Impulse? And this must be easier for us to do because we are not high over the middle of the Pacific Ocean, linked only by a sat phone?

Images used by kind permission of SolarImpulse.com

Secondly, of the 14 key measures and metrics that were being displayed, some were fundamental flight and safety related. Always good in an aeroplane to know your; altitude, speed, and attitude. Some were flight plan related, in terms of direction and timing. And others were more inspirational, such as a view of the overall world circumnavigation. Not needed on a second by second basis, but giving overall context to the journey.

And that was when I had the thought that where we often struggle when producing software testing measures and metrics is that we mix up the safety with the inspirational, the planning with the performance.  Which may be why some people are against measurement, because they are stuck in a loop of safety measurement driven by fear. And others think that measurement and metrics can only be achieved as a complete end-to-end program of works, putting off all but the keenest of measurement disciples.

What software testers may find helpful is a hierarchy of software testing measures and metrics, along the Maslow lines of; Physiological, Safety, Belonging, Esteem, and Self-Actualisation.


Maslow’s hierarchy of needs – Wikipedia

Knowing which measures address which needs on our projects may help save us from; life in the dark with no measures, producing the wrong ones at the wrong time, or being driven by measures and metrics rather than using them as tools to help us test software.

That could give us a new appreciation of what each of our favourite measures and metrics, S-Curve, Bug Count, Velocity, DDP, Coverage, Defect Heat Map, Burn Down Charts, <insert your favourite measure here>, etc., is achieving for us on the Maslow Hierarchy. And maybe an understanding of what each measure or metric we use is actually doing for us, where we may be misusing them, or where we may wish to use them in the future

Of course this may not work. But I think it is better to try and understand what our software testing measures and metrics are doing for us, than to either blindly generate screeds of impenetrable charts and numbers, or worse still, not produce any at all!


 PowerPoint Show    PowerPoint show (6.6mb)

Presented at:

1. UK TMF, London, Oct 2015.

UK TMF Session Outputs:

Below are the output flips from the session and the resultant hierarchy.

PNG  Flip 1     PNG  Flip 2    pdf  Hierarchy