Tom Webster, writing and speaking

"Unaided Recall" in Social Media Research

Added on by Tom Webster.

Some of the most commented-upon posts here at Brandsavant have been about one of two topics: sentiment analysis, and social media monitoring. I suspect there are three reasons for this:

  1. There are lots of sentiment analysis folks analyzing sentiment for sentiments about sentiment analysis
  2. There are lots of social media monitoring folks monitoring social media for media about social media monitoring
  3. ....uhh...I got nothin'.

Anyway, if this popped up in your social media monitoring dashboard, welcome :).

One thing I've been thinking about recently is the process of parsing through comment threads, and how modern listening and analysis services could actually follow a sequential conversation that might take place across multiple sites and services. When I first started poking around sentiment analysis, I got loads of comments, for which I am very grateful, and some wonderful conversations started up in and around the space of that initial post.

If you were using some kind of automated monitoring and sentiment analysis software to go through that post and the comments that followed, you'd certainly have no shortage of grist for your mill. Yet, if that conversation were parsed by a machine, one thing that might be lost in your analysis is this subtlety - I poked that particular hornet's nest. All of the "sentiment" about sentiment analysis in the comments below that post were prompted - I provoked others to leave opinions that might have otherwise never surfaced online.

In my day job, we do a lot of marketing/advertising effectiveness work for brands. This entails a lot of pre- and post-campaign measurement of basic measures like awareness or intent to purchase, various metrics for engagement, and even things like "opportunity to see" for our digital out-of-home advertising clients. In almost all cases, we construct a survey that reads from general to specific, leading respondents through these various measures in a planned and logical order. Two bread-and-butter metrics for any kind of brand research are unaided recall and aided recall. An "unaided recall" question occurs near the very beginning of a survey: "Tell me all the brands of luxury automobile you can name." This sort of question is asked at the start of the process to see which brands have true top-of-mind awareness, and which ones don't. Later in a survey, we might get more specific: "Have you ever heard of 'Maybach.'" At the end of such a survey, we might even ask specific questions about the Maybach ("how much do you think it costs," etc.) or ask respondents to rank the Maybach amongst other brands on various attributes. Again, this is real meat-and-potatoes stuff, but it's absolutely essential to track, especially if you want to measure the "lift" of a given campaign.

To get back on topic, if I write a post about sentiment analysis or social media monitoring, and I happen to name a particular service, then the folks monitoring for those keywords will dutifully tally this up as a "mention" (which is, as I've noted previously, a dubious metric anyway.) However, it's actually a little bit more than that - it's an unaided mention. In other words, from a cold start, I named that particular site or service unprompted. That mention has a slightly different value than a mention later in a comment thread by people responding about that particular brand and leaving their opinions. Those mentions, in the context of that post, constitute aided recall - I prompted each and every comment about that brand, and many of those mentions/opinions might never have surfaced had I not, again, prodded the nest.

All of which leads me to a very interesting social media research question: do you track unaided vs. aided mentions? Top-of-mind opinions vs. prompted opinions? Do you see the value in those measures?

P.S. I do not drive a Maybach. If the folks at Maybach would like to send me one to "review," I promise to disclose the arrangement, deliver a fair review, and then abscond with the car, never to be seen again. You have been warned.