Derived Importance Measures Will Lead You to the Wrong Decision

From antiquity to the late 19th Century, blood-letting, in modern medical parlance referred to as phlebotomy, was an extremely popular practice involving the withdrawal of large amounts of blood from a patient in the hopeful belief that it would cure the patient from disease. It never helped and sometimes ended up killing the patient.

Today, in marketing, a method grounded in correlation analysis is widely practiced which never does any good and can sometimes kill a brand, a product, a service, maybe even an entire company. The approach is called "Derived Importance" and its purpose is to identify the dimensions which drive behavior in any B2C or B2B category.

Here’s a very different kind of example:

For thirty years, academic researchers have published over 60 studies on the relationship between marital status and personal happiness. The conclusion? Marriage works wonders. Since married people have been found to be happier than singles, the argument is made the state of marriage must be a "causal agent" contributing to a couple's bliss.

But now a large-scale, 15-year longitudinal study involving more than 24,000 people challenges this long-held belief that “The state of marriage” makes people happy. This new study shows that most married people were happy and satisfied with their lives long before they actually got married.

Now some might question overturning a long-held belief supported by over 60 studies based on the results of one opposing study, but we wouldn’t. You see, those 60 studies were all based on cross-sectional correlation analysis—essentially comparing all marrieds versus all singles at a single point in time and observing who’s happier. Seeing a high correlation between marriage and happiness, researchers “derived” the importance of marriage to happiness and incorrectly concluded that marriage leads to happiness.

But as anyone who has ever taken an introductory statistics class will tell you (because professors drill it into your head), correlation does not equal causation. It’s a plain fact that evidence of a relationship reveals nothing about the cause of the relationship; derived importance is by no means an exception to this inescapable truth. Nevertheless, researchers not only in the social sciences, but also in marketing, continue to act like it is, particularly in studies of brand positioning and of customer satisfaction and loyalty.

Too many marketing researchers have been telling companies that they don’t have to query respondents to find out what’s really motivating: all positioning and satisfaction research requires is some form of correlation analysis. It could be simple correlations, regression analysis, or even structural equation modeling. They're all forms of correlation.

As part of a sample survey researchers pull out a list of 30 or more product and/or service attributes and benefits and ask respondents to rate different brands on each dimension. For a soft drink, as an example, a researcher might ask how satisfied a respondent is with taste, carbonation, youthfulness etc. The survey then goes on to ask about the likelihood of purchase and/or overall preference for each of the same set of brands.

The next step is to run the correlation analysis between each attribute/benefit (or factor) and the dependent variable of purchase interest or behavior. Ceteris paribus, the attributes and benefits that yield the highest correlations with likelihood of purchase and/or overall preference or loyalty (positive or negative) are labeled the “most important.” Thus, they’ve derived the importance of the attributes and benefits.

Yet the winning attributes from this analysis may really be losers and vice versa.

As examples, for a soft drink, the presence or absence of a high level of carbonation or an energy stimulant such as taurine, the image of youthfulness or a health claim would be revealed by correlation analysis to be unimportant (when, in fact, they might be very powerful), while patriotic red and blue packaging and a consumer hotline--characteristics associated with the biggest brands in the category--might be (incorrectly) interpreted as important drivers of preference.

Base a strategy on these findings and you’re taking your brand to the graveyard.

Conclusion: Beware of traditional correlation-based measures of derived importance. They’re liable to bring you toward a contrived disaster. Like 19th century blood-letting, it can be a path to the wrong strategy.

For more evidence of the shocking truth about Derived Importance download the academic paper Problems with Derived Importance Measures in Brand Strategy and Customer Satisfaction Studies, by Kevin J. Clancy, Ph.D., Paul D Berger, Ph.D. and Peter Krieg.