jump to navigation

Wasting Time On Meaningless Data May 28, 2014

Posted by Tim Rodgers in baseball, Management & leadership, Process engineering.
Tags: , ,

One of the things I love about baseball is that it’s amenable to assessing and studying performance using numbers, and that appeals to my analytical side. Since I started following the game as a boy there’s been a lot of questioning of the traditional measures of player performance, particularly with the emergence of sabermetrics as a serious area of study. Older metrics such as pitcher wins and batter RBIs have been challenged as inadequate and sometimes misleading because they don’t reflect the isolated contribution of a single player. Just because they’re easy to measure doesn’t mean that they contribute to a better understanding.

One of the breakthroughs was when people started looking for ways of associating the performance of a player with the objectives of the team, which is to win games. The team wins games when they score more runs than the other team, so a player who significantly improves the likelihood of scoring runs or preventing runs is valuable. That may seem obvious, but the older, traditional metrics don’t reflect that. It’s a complicated empirical model, and the sabermetric community has been experimenting with new metrics based on data that has never been systematically collected before. As a devoted fan with an analytical disposition I’ve been watching this with great interest.

All of this reminds me of organizations that collect and report data that are disconnected from or unrelated to the real drivers of success. I’ve written about this before, arguing that if the data isn’t being used to manage the business, then the business should step back and re-visit their metrics. More data and more metrics doesn’t always lead to better insight and decision-making.

Here are some key questions that should be asked when evaluating business metrics:

1. Can you control it? Can you “move the needle?” If you can’t control it, you can’t manage it. This is worth knowing. It may still be worth tracking because it provides some interesting background information that could be used as input for future decisions.

2. Is there a correlation between this metric and business success? If your deliberate action helps to move the needle in the right direction, is the business significantly better off? Conversely, if your lack of action leads to no change, or a change in the wrong direction, is the business worse off? I realize that we’re talking about complex interactions with imperfect models, but if there isn’t some kind of causation and correlation, then the metric has no value.

Finally, a word about balanced scorecards. I’m all for simplicity and focus in our communications. We should seek a minimum set of metrics that describes status and progress. In our desire to provide a complete picture we run the risk of ambiguity and confusion.

Metrics should be regularly reviewed for their effectiveness. If they’re not helping to manage the business, we should find new ones.




No comments yet — be the first.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: