On the back of my comments regarding metrics, a natural question arises. Namely, provided you are able to identify the correct measurements to take in the course of assessing the success or failure of any particular program, what is there to be said about interpretation of those measurements?
This is a great question for two reasons. First, it was exactly the question I was asking myself as I was writing about Death by Metrics. Second, it brings to mind the whole issue of measuring value. That is, how do you determine the value that a program is delivering? This really has nothing at all to do with collection of data for incorporation into a pie chart visual dashboard. It isn’t about what the data says. It is about what the data means. So what if you are increasing metric x by y percent over timeframe z? Is that good or bad? If it is good, why is it good? How is it good?
Perhaps I’m oversimplifying, but at the end of the day, there really needs to be some relation to dollars in this equation. Call me silly. I don’t know how else a company with restless shareholders would identify something of value.
When attempting to identify the dollar-denominated value being delivered by a program, there are several standards that are commonly used: does the program lower current cost; does the program eliminate future cost; does the program generate revenue. In consulting parlance, we might refer to these as cost-reduction, cost-avoidance and income generation, respectively.
To the extent that any effort leads to income generation (better sales practices or marketing tools, for example), the value proposition is fairly clear and a simple cost-benefit analysis would demonstrate that. Furthermore, we know what we want to measure in those cases and we know what those measurements mean. I’ll leave it to all my finance friends to further elaborate on the varying interpretations of revenue metrics, but suffice it to say that more is better. A visual dashboard with an ever increasing line graph is what we want to our execs to see over morning coffee each day.
When we start talking about the other two standards to measure by, cost-reduction and cost-avoidance, I have to say that things become less clear (on the surface).
Cost-reduction is about lowering the cost of doing business in the here and now. To the extent that any program reduces the cost of business operations without impacting revenue generation, either by reducing headcount or increasing throughput or by any other myriad means, this is less money being sent down the proverbial hole and should have a demonstrable positive impact on profit margin. Your typical IT department should be falling into this category. Information Technology exists to enable the business. If it can effectively enable the business AND reduce the overall cost of doing business, you have a properly functioning IT department. If it enables the business at such a huge overhead cost that it actually reduces revenue and/or profit, then it is not functioning properly. If it becomes the stereotypical black hole into which millions are sucked and precious little benefit ever emerges, then…well, you my friend have an issue that dashboards can’t solve. But in any of these three scenarios, it is a case of whether or not your IT program is reducing costs and this should be a measurable thing that is able to be interpreted. Yes there are lots of factors and yes many things are open to interpretation, but at the end of the day, an IT program can be shown to have either a net positive or net negative impact on cost of operations. If something can be shown to reduce my current costs without sacrificing other things such as innovation, creativity, efficiency, market-share, etc., then this is a good place for an IT program to be.
Things get real murky with cost-avoidance. Unfortunately, I see too many IT programs using this third standard as justification for large efforts. That is, they promise new technology will deliver benefit by avoiding costs in the future that would otherwise impact the bottom line. This is real sticky. Right off the bat, any time you’re dealing with the future (the future Conan? Yes that’s right, all the way to the year 2000…), you’re dealing with an immediate set of known unknowns and unknown unknowns. Sure you can forecast, you can predict, you can read the Farmer’s Almanac, but you are merely guessing. In some cases we have educated guesses. In most cases we have simple wild-ass guesses. Either way we have a situation where we are justifying actions today, justifying real money to be spent today, by conjuring the specter of some future cost that we may incur if we don’t implement program xyz. Really? Is this the best we can do in the name of our program? If the program was that good, that able to measurably deliver value, shouldn’t it increase revenue or lower cost? Should we, as IT practitioners, be hanging our hats on future cost-avoidance as a business justification?
This practice of cost-avoidance is widespread and championed in many places. <politics>The United States Congress comes to mind. Surely we don’t want to be compared to that! </politics> It may have legitimate uses. I can’t think of any. Let me know if you do. I’m sorry, but if your program doesn’t positively and demonstrably impact my bottom line in some way, I’m not funding it. Why? Because it delivers no measurable value to my business in the here and now and is competing with programs that do.
The whole realization here should be that everything is exactly opposite my initial question and exactly not what it should be. We should be figuring out what adds value to our enterprises and then measuring for it. Instead we’re stuck in a system that emphasizes gathering of metrics first and divining what they say second. This leads to situations where it is acceptable to promise future delivery of benefit based on hazy predictions of the future business environment. Architecture programs tend to fall into this trap. They also tend to be in a constant battle of defining themselves and the value they bring to the table. Almost never are these programs justified by increasing revenue. Too infrequently are they occupied with cost-reduction. They are mostly off in the squishy, nebulous world of cost-avoidance, justifying their existence by cloaking themselves in the pretense that without them you might face future costs. In an age of budget pressures, those programs that promise you’ll avoid a cost you may or may not incur in 5 years will be unfunded and money will go to programs that bring in revenue or lower operating expenses in the nearest fiscal year. The emperor’s clothes will be seen for what they are.
Given the choice, I want programs that deliver value to my business in concrete ways wherein I know what to measure for and what those measurements mean. In my world, if the measurement doesn’t ultimately tie to a dollar-based number, then it is not a useful metric for establishing value.