Beware of Vanity Metrics
I was really happy to read how the view of dollar-value ROI metrics is changing. In fact, the death of these page-view metrics started in 2008. One thing for sure, the products that succeed will be the ones that really satisfy customers. It is important for companies to realize this sooner rather than later and start concentrating on building products that customers love while measuring the appropriate metrics.
Instead of giving some hypothetical examples, I want to share my own experience with this.
Recently, I blogged about an incident in which I dealt with a Toyota salesperson; I don’t think such interactions, like the one I had, leave a good impression on customers. If Toyota keeps measuring the total count of customers visited without measuring their end satisfaction, Toyota will continue on the wrong path.
Similarly, my wife and I purchased two cell phones of different brands a few years ago. My iPhone, in spite of heavy usage, is still in a great shape. However, my wife’s phone crashes twice a day and its icons get stuck. As such, we are planning to retire her phone and switch over to another iPhone. I am not doing any sales pitch for Apple here; I’m just sharing my real experience. As a metric, It is important to measure customer satisfaction as well as customer retention.
Metrics like recording the number of visits or the number of downloads are called vanity metrics. One should start measuring the actionable metrics and stop getting fooled by the vanity.
Here is another true story. In 2011, DoSomething.org posted a sales video in order to encourage young people to donate their used sports equipment. While the organization received 1.5 million views on YouTube, it received zero donations. Companies that only measure metrics like page views or visits are fooling themselves.
The discussion about metrics is not something new to agile software development either. We have our own do’s and don’t’s about metrics. On a lighter note, the best way to derail any conversation and waste time is to get five coaches in a room and start a discussion on metrics.
In this discussion about whether or not velocity is a vanity metric, Jim Benson bring out a key point: A metric is always context dependent. Personally, I believe that metrics are necessary, but they get misused in many places. Instead of generalizing their ill effects, one should choose the right metrics and measure them for the company's benefit.
Can you list any vanity metrics you have come across?
