Ten Questions to Ask a Media Measurement Firm
By Mark Weiner, CEO of PRIME Research-AmericasA study sponsored in part by the Institute for Public Relations and The Arthur Page Society revealed that while executives who fund PR programs desire measurement to gauge performance and return-on-investment, they don’t know enough about public relations to provide direction. At the same time, it revealed that PR executives want to measure their performance but don’t know enough about PR research to initiate the process. As a result and too frequently, no progress is made.While media relations is only one form of public relations, media evaluation is one of the most common applications of research in public relations. The purpose of the following ten questions, organized by phases, is to stimulate the media measurement process and to provide guidance for getting the best return on your investment in media relations research.Fundamentals
- Do you provide content for the media analysis or do I? An “all-in-one” solution is convenient and may be cost-effective but if the content is not included, be sure to factor the incremental cost of content in your budget. There should be no “surcharges” or hidden costs when using an all-in-one solution.
- Is your firm’s offering based on computerized coding or human coding? Computerized coding relies on artificial intelligence software to divine the meaning of a news story. Humans may be slower but we tend to be more accurate in recognizing the underlying meaning of complex issues, nuance and sarcasm. Computerized coding systems may be consistent but less precise and they usually require significantly more time in establishing and adjusting the criteria by which the media are analyzed.
- How do you check for quality and consistency? Accuracy is a critical element in any research program. With so much data to be uncovered, entered and produced, each media analysis program may have hundreds-of-thousands quality elements (accurate circulation, accurate message analysis, accurate reporting…even simple tasks like spelling need to be checked). It is reasonable to expect that the service provider’s software to eliminate simple errors, that the coders have strict guidelines (known commonly as “the code book”), that the coders are trained and encouraged to openly discuss judgment-calls, that the coders are supervised and evaluated on the quality and accuracy of their work, that the people involved in report preparation look for abnormalities when they transpose data to charts and graphs, and so on. With this in mind, you should expect at least six levels of quality control.
- If our program is international, do you do the work “in-country?” Rather than romance-language majors at the local university, “in-country” coders and analysts understand local priorities and nuance. CSR themes resonate much more strongly, for example, in France than they do among the general population in the United States. If your program is international, you have a right to expect native-language speakers to perform your coding and analysis. Even if the French coder works in an office in Brisbane, the coder should be French and familiar with French culture to accurately reflect local perceptions.
Getting Started
- How much time is required to set up the program? Initial program set-up requires a thorough examination of objectives, strategies and measures-of-success. A simple program can be set up “same-day” while more sophisticated programs may take longer. This is an important factor to consider when choosing a media measurement partner generally and an automated or human-based coding solution in particular.
- Do I have input on designing my own report or do you use templates? One consideration in determining cost and turn-around is the degree of standardization which can be leveraged when designing your media measurement. Some programs are simple enough to be standardized but others may require custom elements. Keep in mind that your initial needs may be standard but your future needs may require customization. Flexibility is an important attribute. Also, be certain to weight the impact of customization as it may greatly increase your cost and delay turn-around in exchange for elements which may not be as important as the related costs.
- May I meet the people who will be working on my account? Establishing a relationship with your research team is a good idea in the long term but when it comes time to choose a measurement firm, it is important to profile the team in terms of whether they are employees or part-timers, on-site versus work-at-home personnel, and whether they have industry expertise in your category. These elements translate to better research, greater flexibility and enhanced efficiency.
Reporting and Analysis
- How long does it take to produce a report after the end of the reporting period? Do-it-yourself options provide for instantaneous reporting but if you intend to use an outside firm, the reporting should be delivered while there’s still time to act upon the findings: Content sources can affect delivery as some digital content feeds are faster than manual press clippings, but monthly reports should be delivered within ten days following the end of the month; quarterly reports should be delivered within the following month. Faster turn-around times are usually available for an extra cost.
- Do you provide interpretive analysis and guidance? Some measurement solutions resemble a report card: they tell you how you’ve done. Other solutions resemble a tutor: they go beyond “how you did” to answer “why” and “what can be done about it.” It is important to know in advance whether you are looking for: every media measurement firm provides data but only a handful can provide meaningful interpretive analysis and salient strategic guidance.
- How much time is required to adjust my media analysis program when message or media changes are required? Your media relations programs change at “the speed of business” which is to say “moment-by-moment.” Your media measurement program must be just as flexible and responsive.
As in any serious pre-purchase process, be sure to ask about the firm’s cost-basis, their experience in media measurement, their expertise in your industry and don’t forget to request—and check—references.
To read more learn more about corporate communications research and evaluation, visit the PRIME blog at https://www.prime-research.com/
[author]About the Author: Mark Weiner is CEO of PRIME Research Americas. PRIME is among the world’s largest PR research firm with 500 analysts and consultants employed in ten service centers around the globe. Mark is the author of “Unleashing the Power of PR” published by John Wiley & Sons. To learn more about PRIME, visit www.prime-research.com for additional information. [/author]
