MAKING BETA SOCIAL MEDIA METRICS BETTER

It’s a given that where consumers go marketers will surely follow, hence all the chatter around social media and the rapid growth of networks such as Facebook has unsurprisingly pricked marketers’ attention. Yet for all the current “markets are conversations” talk, that phrase is merely a slogan when there’s no common agreement about the measurement of ‘the conversation’.

It’s still early days in social media metrics land, however there are certainly tools that attempt to quantify how much online conversation is taking place about any given topic, and even how particular words or topics compare in volume over time. Yet these tools vary hugely in how much of ‘the conversation’ they actually monitor, particularly when you consider the amount of conversations that take place in password-protected areas such as discussion forums, social networks, etc.

Obviously, some kind of manual analysis can be applied to qualitative discussions behind closed doors. This process can involve examining the context of online conversations and consequently comes with a far-from-trivial overhead for anything more than a very thin slice of data, especially if any quantitative analysis is provided. Regardless, it’s easy to see how key word and topic analyses can be particularly useful for search engine marketing or optimisation initiatives.

What doesn’t exist yet is a credible way of quantifying the economic value of these online conversations, which is the bottom line for any business and therefore any marketer serving that business. So it’s all very well trying to look at new ways of measuring social media that go beyond the traditional marketing paradigm of awareness or reach, but marketers also need to show that these metrics reveal a progressive means to a financial end. Proving that there have been increases in customer conversations, satisfaction, engagement, or even loyalty is not going to be enough in anything but the short term. Ultimately, rather than replacing one set of means-based metrics with another simply for the sake of doing something new, it’s all about using metrics that show if outcomes such as increased customer conversations also deliver economic end value.

This imperative for marketers - and managers in other business disciplines such as research, business development, strategy and innovation - to develop or adopt more financially accountable metrics is not unique to the social media field. Consider how business managers rushed to embrace the Net Promoter® Score (NPS) customer loyalty metric after Bain & Co’s loyalty guru Fred Reichheld claimed it was the “One Number You Need To Grow”. Sadly, life is not that simple. There’s a big difference between buying into the general philosophy that customer focus drives healthy profits, and believing that measuring customer recommendation rates in a particular way is the Holy Grail of business metrics. The so-called science behind the NPS has been heavily criticised recently, so much so that Reichheld is no longer arguing that ‘one number fits all’, but that the NPS metric is just one aspect of an overall approach and that its importance shouldn’t be over-emphasised within that approach.

There are parallels that can be drawn here, particularly with the online quantitative analysis of sentiment based on monitoring consumer conversations.

Firstly, there’s a big difference between intention and actual behaviour, as anyone in the banking industry will tell you. Put simply: people don’t always do what they say they will.

Perhaps more importantly, the online monitoring and measurement of consumer conversations is not as rigorous as traditional market research. Companies basically eavesdrop on consumer conversations and it’s very easy for what’s being said to be taken out of context, especially if you’re relying on software to automate the gathering and sorting of data. The point being that online conversations are unstructured and undirected - they are not a set of solicited responses from selected participants to specific questions.

There’s also no way of assessing whether or not there’s any bias in this kind of analysis, because you can’t tell if the conversations are from a representative sample of consumers. Even if you involve humans in the analysis process, you’ll end up with quantitative results based on qualitative assessment, which is a research no-no.

This doesn’t seem to stop brands throwing their discretionary spend at companies offering this kind of quantitative online monitoring service, when really it’s little more than a finger in the air. That’s not to say it’s not worth doing at all, only that it’s not worth spending too much money on right now, nor placing any oracle-like emphasis on the output data. You’d be well advised to benchmark the findings against some more rigorous research before you take any major action.

From a qualitative point of view however, this buzz monitoring activity becomes a lot more interesting - though arguably only with human input into the analysis. There is definitely valuable insight to be gained about what it is about a brand or its products that is generating both negative and positive word of mouth, as well as competitive analysis such as why consumers are talking about some brands and not others. There are still limitations with this measurement process, such as the issues mentioned before relating to the context of conversations and sample bias. Then again, traditional qualitative research is not without its issues either. So it’s best not to view buzz monitoring as an alternative to other forms of consumer insight research; think of it more as a complement. In particular, buzz monitoring is very useful as an early warning of negative word of mouth for Reputation Management/Crisis PR.

For the measurement of social media to become more credible and widely used, a good place to start would be for practitioners to agree on a common framework of terminology. There’s a confusing myriad of buzzwords flying around, often with one term being used to describe very different things. (In the same way that, for example, the phrase “net promoter” is used randomly by several companies to indicate their own adapted versions of the original metric.)

There also needs to be greater collaboration between buzz monitoring companies that are immersed in the social media world, and professional researchers who understand what’s needed to produce valid, rigorous, actionable data and how to distinguish insight from noise.

Lastly, the development of a set of standards for measuring the value of social media must involve key brands who are already active in this area. On the one hand, it will help social media practitioners understand the business drivers that need to be taken into account when developing financially accountable metrics; on the other, it will help brands determine which metrics really help prove that economic value is being delivered, and which - at the risk of repeating some of the mistakes of the NPS’ experience - are bigging up the promise of a full pint and delivering short of the mark.