jump to navigation

Decisions Based on Psuedo-Quantitative Processes December 28, 2012

Posted by Tim Rodgers in Management & leadership, Organizational dynamics, Process engineering, Quality, strategy.
Tags: , , , , , ,

I’ve spent a lot of time working with engineers and managers who used to be engineers, people who generally apply objective analysis and logical reasoning. When faced with a decision these folks will look for ways to simplify the problem by applying a process that quantitatively compares the possible options. The “right” answer is the one that yields the highest (or lowest) value for the appropriate performance measure.

That makes sense in many situations, assuming that improvement of the performance measure is consistent with business strategy. You can’t argue with the numbers, right? Well, maybe we should. In our rush to reduce a decision to a quantitative comparison we may overlook the process used to create those numbers. Is it really as objective as it seems?

There’s a common process for decision making that goes by several different names. Some people call it a Pugh diagram or a prioritization matrix. A more sophisticated version called a Kepner Tragoe decision model includes an analysis of possible adverse effects.

These all follow a similar sequence of steps. The options are listed as rows in a table. The assessment criteria are listed as columns, and each criterion is given a weighting factor based on its relative importance. Each row option is evaluated on how well it meets each column criterion (for example, using a scale from 1 to 5), and this assigned value is multiplied by the weighting factor for the column criterion. Finally, the “weighted fitness” values are summed for each row option, and the option with the highest overall score is the winner.

At the end there’s a numerical ranking of the options, and one will appear to be the best choice, but the process is inherently subjective because of the evaluation criteria, the weighting factors, and the “how well it meets the criteria” assessment. It’s really not that hard to game the system and skew the output to provide any desired ranking of the options.

I’m not saying this is a bad process or that the result is automatically invalid. What I am saying is that this isn’t like weighing two bags of apples. The value of a decision analysis process isn’t just the final ranking, it’s the discussion and disagreements between the evaluators, which are obviously subjective. We shouldn’t consider the process to be an infallible oracle that delivers an indisputable answer just because there’s math involved.

I’m sure there are other examples of psuedo-quantitative processes that shouldn’t be accepted at face value. Leaders should question assumptions, listen to dissenting opinions, and check for biases. It’s rarely as cut-and-dried as it seems.



No comments yet — be the first.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: