I’ve recently fallen in love with a data science book by Seth Stephens-Davidowitz, Everybody Lies – really, buy a copy, or at least read his summary. Among the countless examples is that there are always better – often free – ways of measuring and evaluating something: from the effect of racism on the US elections (see base), people’s actual intimate desires (and how they’re afraid to tell anyone them), the actual liberal bias in US media / how it is purely owners giving readers what they want, or how, contrary to popular opinion, the best NBA players come from middle class families, rather than from ‘the projects’.
The topic that comes out time and again is not the need for big data, but to have the right data. And this is certainly true for PR.
As an industry we (mostly) know that we shouldn’t be using AVEs – aka advertising value equivalents. Yes, that doesn’t always stop us using them though, last year a survey by PR Week / the PRCA showed 35% of agencies were still using them. As were 23% of in house teams.
Coverage should never be the end goal, it is just an intermediate step to help you achieve your goal – create awareness, increase salience, drive desire, make it easier to find you. And a piece in the NY Times, or the EE Times doesn’t guarantee anything. If just having coverage on a site were enough to achieve this, then there wouldn’t be a difference in effect between editorial (earned) and wire-fed (bought) articles on sites such as Bloomberg (see chart below).
Bloomberg uses a different URL tag for editorial / wire coverage from this we can see editorial coverage is significantly more influential in effecting a change in behaviour (in this case sharing it on social networks)
AVE value and effect of coverage doesn’t correlate well. This is especially true for B2B clients, where more of their customers read Telecoms.com, EE Times or Print Week (/ insert trade mag) than read the Guardian, Mail or NY Times. In short, overall readership (and therefore the amount a company charges to buy a page in it) is a useless metric. And even for B2B titles it doesn’t hold true as AVE is based on subscriptions, but there is little incentive for readers to stop receiving them as they’re free.
A director at a major tech agency that I’ve done evaluation for told me, he’d used AVEs as it was a simple way to compare (year on year) the effect that you’ve had. The MD of small tech agency I’ve collaborated with used them as they couldn’t always get access to Google Analytics. Another MD of a small travel agency told me she simply didn’t have the time to learn alternative measures, but at the same time needed a way to demonstrate to her client that what they’ve achieved is good.
Taking this last one as an example (for Abode PR, a UK-based travel PR agency) it didn’t take long to show that there was a significant upswing in interest as a result of their efforts. We may not have had access to their Analytics account, but we could see that year on year searches were up significantly since their 4-month promotion began.
The headline figure was simply a mean 91.1% increase in salience (measured used Google Trends – see below) versus 2016 and 2015 for the same time period*. with each of the spikes below correlating with coverage appearing. And while there was a small spike due to a funding round closure 3 months before meaning values were a little up for the year, the average for
*Travel is a seasonal business, so we’ve taken it solely for the same period in each year. There had been a spike earlier in the year due to TechCrunch coverage, however, even with this taken into account the four months were 23.9% up versus what you’d expect once you’d taken this into account.
Can you measure PR in terms of sales?
Not really – there are too many variables that affect sales.
Did the PR create awareness but the SEO wasn’t effective and people couldn’t find the website? Was the website poorly engineered, reducing conversions? Was pre-sales support not in place, preventing people asking questions crucial to the buying process? Were product stock levels too low, removing the ability for people to buy? Were stores (I know, who still buys in the real world?) tatty?
What, for most of my clients at least, PR (product PR) is there to do is help take the customer to the point of sales. This may be achieved through improving SEO (through on site content, creating links in high-domain authority sites and creating coverage that will be prominent in searches). This may be through creating awareness with key individuals. That doesn’t necessarily mean you can’t measure financial performance – share price over time is a good one – but sales shouldn’t (in my opinion) be tracked.
For example, in a previous agency I worked with Dialog Semiconductor, at the time it was the fastest growing public semiconductor company, and the success of a release was measured solely in the effect on the share price. The below represents a shift from €0.48 to a max in 2015 of €52.07.
A note on metrics
Very quickly, PR should be run like a scientific experiment, with objectives set and ways to measure against these as well as conditions for success well defined at the start. In my experience, while very few agencies have this written down, there is at least an understanding of what they’re trying to achieve, which lets retrospective analysis work.
And a lack of an effect is not a failure of the campaign – if it’s a crisis campaign it’s a great result, and even if it’s a standard awareness campaign, by understanding what isn’t having an effect allows you to alter the campaign and adjust budget / time / effort accordingly.
Typical types of campaign that I’ve worked with:
Product / brand awareness Crisis management
SEO Expert positioning
I’m going to talk about these, rather than say political campaigns (votes / polling data), internal HR-based campaigns (staff retention / applications). And here are some of my favourite metrics that can be used in the majority of projects:
Coverage social shares:
Do people like what they’ve read about you (in earned coverage) enough to go out of their way and talk about it on social media? The APIs of LinkedIn. Facebook and Google+ (and Pinterest / Digg / YouTube etc for that matter) all allow you to see how many shares your coverage generated. You may have seen this if you use services like CoverageBook.com (if not, you should be) other services to track this include the free tool SharedCount or the Google Sheets add on Supermetrics. From this – and using a little manipulation in Google Sheets or Excel – you’ll be able to plot this against activity to see which publications drive share count, you’ll be able to see what topics do too.
The same can be said for blog post shares. It should be said that shares are still vanity (they don’t drive sales) but are a relatively straight forward way of measuring the effectiveness of yours (and your competitors) coverage.
One project I did last year was to help my client understand how to best prepare for a major product launch, with their client being one of the products two distributors. See below for the importance in measuring more than coverage levels:
Note – I’ve removed identifying information. But the first is coverage (supplied by Meltwater, reach / AVE matched this almost exactly), right is shares (using Shared Count). Red is the disti, blue their competition and purple both, grey is just the product.
Notice the lack of blue shares? Coverage, while an important intermediate step in the process isn’t the key step, what we saw was the need for coverage to coincide with the day of the launch.
Google Analytics traffic:
Google Analytics is one of the best ways to see what effect the coverage is having. Not only can you correlate direct traffic (people typing brandx.com into the URL bar) coverage as well as brand-search coverage (people typing Brand X into Google / Bing), and correlate this with coverage or activity, you can also see the number of web hits that have come from your earned coverage (I highly recommend answertheclient.com for this – albeit it can take some time).
As I said earlier, a main aim of PR (certainly in an awareness campaign) is to drive people to your website or product page on a distributor’s website (or a real life store – albeit counters at exits will be a better metric here). And, in addition to this, you can easily configure goals to ensure you are only counting the right people (ie those spending enough time on the store, or visiting multiple pages – but be aware of how ) rather than those leaving immediately.
As per social shares – look for correlations with activity, look for what does work and (in my opinion) more importantly what doesn’t. And adjust the campaign accordingly.
Your client hasn’t shared access to Google Analytics? The alternative (and for certain things, the first choice) for me is Google Trends, which will tell you for any keyword of reasonable volume, who is searching for your brand / product, how this this changed over the past year / 5 years / a custom time period? And compare it with your competitors / other search terms. I not only use Trends for evaluation, I also use it a lot for strategy, and in keyword planning for content campaigns.
Going back to the above distributor case study from the social shares, what Google Trends showed was the effect of the shares / timely coverage. And by day 2, not only are shares down, but searches for the product and disti are down to less than 40% as they are on the day of launch.
The problem with Google Trends is that it doesn’t give absolute numbers. Merely as a percentage of the maximum number for that search / time period. Google Adwords lets you ad some context to this by giving the approximate number of people who have recently made the search. Which (to an extent) allows you to compare apples with oranges.
Coverage-site domain authority:
For SEO campaigns, placing content with an in-bound link onto the site is a crucial element in their success. A Bulk Domain Authority Checker (a rough guide of how important Google thinks a site is) will allow you to do an automated vlookup in Excel / Sheets for coverage to check the effect. It also allows you to plan and prioritise which sites to target for content placements. See below.
Tools like Meltwater will give a sentiment analysis measure in its reports. I’m not a big fan in using it for coverage analysis, but for crisis campaigns it can be crucial in both identifying the starts of an issue and measuring the progress of the crisis and its return to normal. There are several others that are worth checking out.
For blogging / social media campaigns the sentiment analysis you need to track is your customers’ perception. Again, there are tools you can use, but there’s also a rule of thumb called The Ratio that can work very well indeed. This is based on the fact it takes more effort to write something than click a button, so if there are more replies than retweets (or comments than shares) someone has done badly. Or as Luke O’Neil more succinctly wrote in Esquire “The lengthier the conversation [from the post] the surer it is that someone royally messed up,” .This ratio can help you track which writers are working well for you – for a glowing example of this in action see FiveThirtyEight’s excellent analysis of Trump and the US Senators’ tweets.
These aren’t the only ones, of course. Others include
1) Downloads (eg of a white paper),
2) Number of multi-vendor mentions PR mentions (highlights journalist awareness)
3) Email click through (tracking effect of email campaigns)
4) Follower growth rate
5) Mailing list signups
6) Financial performance (share price) over time
One of my favourite examples of how Google Trends has been used is in political polling. In polling, data is rarely timely and often skewed to what people are expected to say, for example (according to Seth Stephens-Davidowitz) Barack Obama’s polling (or more accurately how much it was off by) correlated exactly what Google Trends predicted as the levels of racism in each county.
One of the best things the author also showed is that Google Trends can predict what is likely to happen better than polling companies – for example, certain searches (how to I vote / where is my polling booth) will predict turnout. And even Google Trends predicting the outcome. While searches for one candidate will be made by those who love and hate them (I know I searched for Trump out of fear) those searches referencing both candidates are a good predictor of the outcome, with the order the candidates are mentioned giving away the mental bias.
So, for 2018, have a play with data sources and ditch the AVE.