Your Public Relations and Communications Community

Conference Recap: Measurement Matters (a lot); But How Do You Do It?


November 15, 2013

Mark Weiner, CEO of PRIME Research
Mark Weiner, CEO of PRIME Research

Anyone who has studied for the APR exam, or read the Barcelona Declaration of Measurement Principles, released in 2010, knows that research, goal setting and measurement are key components of an effective strategic communications plan.

Meaningful measurement begins with developing communications objectives that answer who, what, when and how much the PR program is intended to affect.  A useful acronym many PR professionals are familiar with is SMART: specific, measurable, attainable, results-driven and time-based.

Yet many of us still struggle to apply measurement in our everyday practice.

For those seeking guidance, PRSA and the International Association for the Measurement and Evaluation of Communications (AMEC) teamed up to present a pre-Conference measurement symposium on Oct. 27 at the PRSA 2013 International Conference.

“Data is a competitive advantage,” said Allyson Hugley, chair of the AMEC North America chapter and executive vice president of measurement, analytics and insights at Weber Shandwick, in her opening remarks. “Companies that are data driven are more productive and more profitable.”

Implementing change

Marguerite Marston, commercial PR manager for IKEA U.S., described her department’s struggle to develop a new measurement mindset.

The catalyst for change was a new chief marketing officer who used analytics to drive the business forward. Prior to that, Marston said, the PR department was measuring campaigns, but not yearly progress.

“PR was more of an afterthought, with the smallest budget in the marketing department,” she said.

IKEA’s PR team members began by educating themselves about the Barcelona Principles, and having an ongoing dialogue about how to move towards integrated measurement. Marston said they then took “baby steps” towards implementing changes. “We were talking about outputs and impressions and now we’re seeing a shift towards reporting behavior change,” she said.

Marston advises PR teams taking their own baby steps to keep the process simple and affordable. Make sure your approach is aligned with your client’s or organization’s needs.

Measuring what matters

For the greatest insight, Mark Weiner, CEO of PRIME Research and author of “Unleashing the Power of PR,” said PR professionals must measure both ouputs and outcomes.

Outputs include all the tactics deployed by the agency or organization, whereas outcomes include shifts in stakeholder awareness, comprehension, attitude and behavior.

“Measuring outcomes without measuring outputs is the slowest path to victory,” he said, because outcomes are measured infrequently — usually only once or twice a year. And since these outcomes reside in the mind of your target audience, the primary way to measure them is to conduct a well-designed survey.

“[But] that’s not agile enough to guide day-to-day decision making,” said Weiner. As an example, he said, knowing which press releases were picked up (and which ones weren’t) immediately informs your ongoing media relations decisions and activities because you will know what kind of content appeals to journalists.

“On the other hand, measuring outputs without measuring outcomes is the noise before defeat,” Weiner added, because without outcome evaluation you will never know enough about your performance to make improvements.

And merely reporting clip counts and impressions or comparing the space of earned media to paid media isn’t enough.

The Barcelona Principals state: “Media measurement requires quantity and quality” and “AVEs [Advertising Value Equivalents] are not the value of public relations.”

PR professionals must measure such things as tone, credibility, relevance, key message delivery and prominence within the medium.

The principle method for measuring these qualities is through content analysis. Weiner noted that content analysis methods have evolved through three distinct trends.

The first wave was manual content analysis by communications professionals.  “This was appropriate and possible when the pace and volume of media was slower and manageable,” he said. This produced high-quality content analysis, but the process was time-consuming.

The second wave was fully automatic content analysis. This came about in an effort to keep up with social media, which moved at a faster pace than human eyes could monitor. So, software solutions were developed to track such things as keywords.

“These were fast, but they were stupid,” Weiner said. “They couldn't tell the difference between such statements as: ‘Ford makes great cars and Ford makes anything but great cars.’ This high tech approach to content analysis generated lots of irrelevant and inaccurate data, and resulted in a lot of false, misleading insight.”

Now, he said, a third wave is evolving that’s a hybrid of both technology and human measurement, which Weiner says is far superior.

This approach is shaped around letting tech do what it does best — delivering speed and consistency — and letting humans do what they do best, which is uncovering relevancy and insight.

“If it was [just] about technology, everyone with a keyboard would be Tolstoy,” Weiner said.

Demonstrating business value

But measurement without context isn’t enough. According to the Barcelona Principles, “The effect on business results can and should be measured where possible.”

For many PR professionals, this is the most challenging — and most expensive — analysis to undertake because it involves marketing mix modeling and statistical analysis. That’s fine if you’re Wal-Mart, Apple or General Motors, but not so much if you’re, well, nearly anybody else.

Yet there is broad consensus that public relations must demonstrate value and show a positive return on investment.

“Measurement is essential to the ROI/value conversation,” Weiner said. But he cautioned that PR professionals should avoid using those terms interchangeably.

“ROI is measurable,” Weiner explained. “It’s a quantitative financial measure, which relates dollars spent with business results. Value, on the other hand, is subjective and relates only to perceptions of expectation, worth and importance.”

Measuring what matters

Valuable metrics answer the questions executives care about.

Mark Stouse, vice president, Global Connect of BMC Software explained the Influence Scoring System (ISS), a measurement platform his company developed after a pivotal conversation with a CEO who asked some tough questions, such as: “I know this is important, but how important is it?” and “How much of what you do is luck vs. skill?”

Stouse said this motivated his company to change the conversation about communications metrics and start with “the big three — the Crown Jewels in the CEO world: revenue, margin and cash flow.” ISS was developed for business leaders, with their collaboration and guidance. The system was field tested for eight years in three public companies and later with five PR agencies.

ISS integrates tactical, operational, and strategic outputs to calibrate marketing and communications activities. First, all tactics are given a point value, then an “environmental difficulty index” is applied, which Stouse said is analogous to driving a car on a paved road at one end of the spectrum versus a dirt road in the rain at the other end.

The third overlay assesses each tactic for demand generation, deal expansion and sales velocity. These affect revenue, margin and cash flow. “All three levels are always fully exposed, based on principles of transparency,” he said. This ensures all stakeholders know and understand all three perspectives.

“ISS creates a logic path that chief executive officers recognize and respect, and data correlations they need to make decisions,” said Stouse.

He added that the platform also “enables agencies to operate their organization like an investment fund, to accurately predict performance and impact across multiple functions.”

It also eliminates client subjectivity around performance and allows agencies to automatically adjust performance objectives based on increases and decreases in spend, he added.

Beverly M. Payton, M.A., APR Beverly M. Payton, M.A., APR, is principal of Payton Communications LLC, a certified B Corporation, focused on helping nonprofits, healthcare organizations and socially minded businesses accomplish their missions. Twitter: @BevPayton
Email: ev at paytonpr.com



Comments

Mark Weiner says:

Beverly, Thank you for a very thoughtful summary of the PRSA/AMEC pre-conference. Even though my career is devoted to providing research-based communication consulting, there was so much to learn from so many speakers. Mark

November 15, 2013

Tim Robinson says:

Beverly - I thought your article was very good...very helpful. It was a concise, targeted summary of the Barcelona Principles and measurement. I've needed a great summary like that. I do, however, feel that the final segment -- Measuring What Matters -- was a whole lotta stuff packed into a few sentences. I'd like to see that last segment broken down into a piece as long as the entire article. But, again, your piece was terrific. Thank you....

November 19, 2013

Jesse Radonski says:

This article is so helpful. I've recognized this for the past year since I've started working with the agency I'm currently at, but didn't know how to put it into words. I'm definitely sharing this with my colleagues and want to thank you all for providing your insights.

January 15, 2014

Post a Comment

Editor’s Note: Please limit your comments to the specific post. We reserve the right to omit any response that is not related to the article or that may be considered objectionable.

Name:
Email:
Comment:
Validation:

To help us ensure that you are a real human, please type the total number of circles that appear in the following images in the box below.

(image of four circles) + (image of six circles) + (image of five circles) =

 

Online PR Training: FREE With PRSA Membership

Broaden your skill set with access to an extensive library of live and on-demand professional development webinars — one of PRSA's premier member benefits.