Metrics that Matter …for Cloud
July 9, 2015 § Leave a comment
Years ago, I had the opportunity of leading one of our Project Management Offices (PMO) in IBM. For anyone that has worked with project managers, one quickly finds out how much they like metrics. In fact, it is more than just project management. We all like metrics and most every leader in IT at some level relies on metrics to make decisions and gauge how they are doing.
I’ve always had a fascination (obsession perhaps) with metrics (ask me sometime about my fantasy football league if you want to see my obsession in action J). One of the challenges we face with metrics is an overload of too many metrics and not all the metrics are necessarily important. With all the metrics out there, I’ve worked hard to “reduce the noise” and find the metrics that really matter in my job.
Recently, I’ve had several conversations with clients on their cloud adoption strategy that led me to an epiphany related to metrics – The epiphany is that many of the key metrics I’ve come to value over the years are 100% applicable in cloud as well.
The classic way we usually categorize metrics is using the “Project Management Triangle” as shown below:
The classic thought behind the Project Management Triangle is that we need to try and optimize a project based on these three opposing drivers of cost, time, and quality. The common belief is that you can optimize to two of these points but not all three (e.g. I can optimize on cost and time but at the sacrifice of quality).
I’ve found that the classic Project Management Triangle a very useful way to categorize and discuss metrics related to Cloud as well. In several client conversations I’ve had this year, we have used the categories of Cost (Budget), Time (Speed), and Quality (Reliability) as the categories to group the metrics we want to measure. The metrics that I’ve found that “matter” in these cloud discussions are shown below:
|Cost / Budget||Innovation Ratio||How much are we spending on reducing technical debt vs. adding new capabilities?|
|Cost / Budget||Budget underage/overage||How well are we at predicting costs and estimating?|
|Time / Speed||On-time Delivery %||# of projects delivered on time / total # of projects|
|Time/ Speed||Velocity||What is the velocity of our team as well as historical Velocity?|
|Time / Speed||Turnaround time for Changes||How long does it take to handle a change request for a new feature or to fix a defect/issue?|
|Quality / Reliability||Uptime/MTBF||What is our mean time between failures?|
|Quality / Reliability||Test Escapes||How many defects do we have that were not found during testing?|
|Quality / Reliability||System Utilization||What is our resource utilization during stress (performance, # of users, etc)?|
As I explored some of the root causes of issues for some of these metrics with clients, it was interesting to map how Cloud would address and improve the issues that were causing some of the poor numbers in some of these metrics. In the end, these were the metrics that “mattered” in my cloud discussions.
The other insight I had regarding Cloud and Metrics is whether we can take the classic Project Management Triangle and break the notions around optimization? What if we were able to optimize across all three dimensions instead of two? I believe Cloud is a game changer and we can indeed optimize across cost, time, and quality! I’ll share some of my ideas on how this might be possible in my next blog.