As a practitioner and researcher in the arena of marketing accountability, I am impressed with the latest articles in PRWeek, and other places, about the industry’s growing stress on measurement. It is clear that PR must be concerned about accountability measurement if it is to retain its proper share of corporate budgets.

While a recent guide to measuring the impact of PR on sales (Council of Public Relations Firms, 2005) has helped establish some informational groundwork, the buzz around complex statistical approaches to measurement is evident. Having career-long experience in measuring marketing effects, I would like to outline the strengths and limitations of emerging tools.

Market Mix Modeling (MMM)

This approach came to us from economists. It is actually a form of linear regression, a basic technique used in many industries for years. It works by looking at a number of potential variables that could impact something (e.g., sales) and simultaneously considering their individual contributions.

Unfortunately, it is also misused and abused, because it is a straightforward technique and can be utilized by people who do not understand the requirements that have to be in place for it to yield accurate, unbiased results.

Marketing practitioners use MMM to measure the relative effects and return on investment of marketing communications. There are, however, a number of limitations:

• MMM is based on linear (straight-line) assumptions, but marketing is multidimensional and non-linear.
• MMM is only as good as the variables included in the model: so-called ‘specification’ errors can produce highly accurate, but misleading results.
• MMM has difficulty accounting for intervening variables; that is, something that is not in the model equation that is affecting a variable that is in the model, which produces biased results.
• MMM usually requires an extensive collection of historical data and an integrated data set that is often quite expensive to build, particularly if critical variables have not been measured yet.
• Finally, models work on aggregate data (using averages), which smoothes over individual differences at the customer-centric level.

The Achilles Heel of MMM is not taking into account the effect of intangible impacts, such as the power of advertising on our attitudes and emotions. The bottom line is that while models are quite useful, they still are models of reality – and the results tend to become ‘black box’ measures that are difficult to explain or understand.

Alternative to Modeling: Experimental Design (MROI)

There are other ways to approach the measurement of PR to sales that deserve better understanding. The primary alternative is the use of experimental design, now commonly known as Marketing ROI (MROI).

Professional associations, such as the Advertising Research Foundation, are clear in their recommendation of the use of experimental design, when possible, to measure the return on investment of marketing and advertising. As such, it makes sense for the PR world to consider this as well. A simple example of experimental design is the use of test and control groups to determine the differential effects of PR on some outcome, such as purchase or intent.

Experimental design is more than a statistical technique; it is a process to get a cause-and-effect answer. It uses the principles of the scientific method to develop research designs and isolate variables of interest for study. There are many types of efficient research designs that don’t cost a lot of money or take a lot of time. Interestingly, Six Sigma proponents have built many of their ‘black belt’ techniques based on experimental design principles.

The biggest hindrance to the use of experimental design is our unfamiliarity with it and lack of understanding of its principles. Yet, that should not be an excuse for overlooking an essential and viable alternative to measuring the impact of PR on sales.

In the end, there is no short cut when it comes to conducting effective research and measurement. PR is important to an organization’s overall communications objectives. It is critical that the PR function gets its research done right. Wise companies will overcome the ‘silver bullet syndrome’ by carefully examining the approach that best meets their unique needs, situations, and challenges.

Dr. Raymond Pettit is a friend of The Bivings Group and a leading research thinker and practitioner in the US, with quantitative and research consultancy experience on both client and agency side. Having co-authored “Market Research in the Internet Age” in 2002, he was commissioned by the Advertising Research Foundation to write “Learning from Winners” (available fall, 2006),
based on the David Ogilvy Research Excellence Award case study collection on advertising effectiveness. Ray received his Bachelors’ degree from the University of Michigan and both Masters and Doctoral degrees from the University of Illinois. He is the president of ERP Associates.