BrandSavant

Gaining Insight From Social Media Data

Five Ways To Improve Online Influence Measures

by Tom Webster on February 1, 2011

NewImageRegular readers of my posts here know that I have, on occasion, raised questions about various measures of online influence. No, they are not perfect, and in some cases are downright meaningless. Lately, though, I’ve been hearing that still, small voice in my head: “Don’t bring me problems, Webster – bring me solutions.” There are some positive aspects of these measures (Klout’s focus on topics of influence comes to mind), and while the easiest stance to take would simply be to dismiss these measures out of hand, I think it’s more productive to (in the words of former President Bill Clinton) “mend it, don’t end it.”

With that in mind, here are five ideas, ranging from minor tweaks to major overhauls, for improving online influence measurement. These aren’t the only five, I’ll grant you, but that’s what the comments are for, right?

1. It’s All Semantics

Some of my issues with online influence measures are largely due to semantics. I don’t believe, for instance, that Klout is the “standard for influence,” as their corporate tagline proclaims. That doesn’t mean that there isn’t a “there there.” Klout does a very good job measuring the dissemination of messages on Twitter, and differentially modeling the impact upon those messages of various messengers. Klout does not, to my mind, measure “online influence,” but they do measure the effectiveness of individuals at propagating messages on Twitter, by topic. True, there isn’t much of a sexy tagline in that previous sentence, but many of my objections to these various measures would evaporate if they only said what they did, and did what they say they do.

2. Incorporate Web Analytics

This (along with #3, below) would begin to address what I call the “Seth Godin” problem. The number of people more influential on the social web than Seth Godin can probably be counted on just a few hands, but his Klout is 69 and his PeerIndex is only 29 (mine is 45). Seth Godin is not “good at Twitter.” He is, however, a pretty fair blogger, wouldn’t you say? Again, if these various measures are going to purport to gauge online influence, then either their grasp must begin to equal their reach, or they need to adjust their reach (see #1.) Incorporating Godin’s true reach, by factoring in his blog readership, would surely vault his online influence scores towards the top, where they arguably belong.

3. Deeper Blog Insights

Along with measuring the impact of, say, Seth Godin’s actual blog analytics, it would also be interesting and productive to measure the impact of Seth’s blogging on other blogs. It’s one thing to link to his blog, which certainly implies some kind of influence (either positive or negative); it’s another, however, to write a post about Godin’s blog. This very post, in fact, has spilled a lot of E-Ink about Godin, and it could fairly be argued that this in itself is indicative of Godin’s true clout on the web. Not that Chris Brogan, for instance, needs any added online influence, but one of the truest measures of his impact on the social web is not the quantity of retweets that he manages to generate, but the number of people who spend time writing posts in response to Brogan. Retweets are effortless. If, however, I take the time to write a careful (or careless, for that matter) response to another blog post, this is surely a sign of deeper engagement, and influence.

4. Better Text Analytics

Speaking of retweets (which, as I have written before, are not a proxy for trust,) there is altogether too much weight placed upon retweets in these various measures, and not enough attention paid to the character of those retweets. For instance, note the difference between these three hypothetical tweets:

A. RT @AmberCadabra: The Three Teams You Need To Organize And Scale Social Media.

B. Just read @AmberCadabra’s take on the HR restructuring social engagement requires. Is your organization ready? (link.)

C. Wow – we are in the process of setting our HR strategy for 2011, and I just discovered @AmberCadabra’s excellent article. I laughed, I cried, it became a part of me. It was better than Cats. I’ll read it again and again. It was the feel-good post of the winter.

You get the point (and yes, I may have exceeded 140 characters with “C.”) I think text analytics are at the point where the qualitative difference between “A” and “B/C” is readily apparent, and the difference between B and C is certainly within our grasp.

5. Why Not Ask?

The most problematic assumption that online measures of influence make is the assumption of motive. If I retweet someone, why did I do this? Did I do this because they are influential? Because I disagree? Because I am playing “spot the loony?” Mining unstructured data alone will never give you the answer. There are, however, simpler inputs. One might be to incorporate a user-generated influence input to the algorithm – Digg-ing people, in other words. If I see a list of users on one of these services and I disagree with how these users are ranked, I might certainly “vote” some of them up or down to correct this perceived affront. If enough people do this, voting could become a viable input to (but not a replacement for) the various algorithms these services use.

One could also incorporate a polling mechanism. Ask enough people to name the three most influential people in their topic area of interest, and you’ll get a pretty convincing proxy for influence within those topic areas. Again, this wouldn’t negate the behavioral measures, but augment them.

These literally took me a half-hour to brainstorm, so I know you have more ideas. Why not share them in the comments?

Five Ways To Improve Online Influence Measures originally appeared as a guest post on Social Media Explorer.

Be Sociable, Share!
  • http://azeemazhar.com/ azeemazhar

    Hi Tom

    Thanks for this note – and drawing attention to one clear problem (the SethGodin problem, which we’ve spoken about before).

    Few observations:
    a. It is annoying that PeerIndex doesn’t currently capture ‘real world authroity’ – of the kind Seth Godin has. But it reminds me of the days before Google had an image search – searching for images was a pain. Then they added it, and it became less of a pain.
    b. It is always good to get feedback on particular users, but as you’ll understand a ranking process is a statistical activity, and specifically an activity that is designed to minimise errors across a large set of users. So the more appropriate test is not to find the edge cases but to find the magic middle where the model works well. Of course, it’s in our interest to extend the quality out to the tails, which is something we are doing.
    c. Your observation about better text analytics is a good one. However, I think you underestimate the power of large scale statistical processes that you have access to by looking at lots of data. It’s why large scale statistical engines like Google can get very far without being able to read sentences (the way Powerset did). At a small scale–and certainly when you get down to the tactical implementation-hand coding is valuable. When you are looking at millions of users, it doesn’t really add ver much.

    best
    Azeem
    (www.peerindex.net)

  • http://www.edisonresearch.com Tom Webster

    Thanks for the note, Azeem! I realize that all of these services are iterative, and they will continue to get better as they grow (which is why I wrote this post, and not an “influence measures suck” post:) ) You are correct to point out that statistical processes on large data sets are great at finding the magic middle – it’s kinda the law! It’s certainly the law that you operate under at PeerIndex, and the law that I operate under as a a researcher of humans (and samples of humans). But you see a lot more data along these lines than I do. I would only point out here that many of these automated influence measures are NOT being used to find the magic middle. They are, in fact, being used to find the edge cases – the top influencers in a given topic or context. The “tails” are the ones being offered Klout perks (or not), differential treatment, and so on. This is not a knock against you or your service. It is a knock on how this data is being used, however.

    Really appreciate the comment, Azeem. Cheers!

  • Anonymous

    I like where you are going to determine a person’s online influence to include analysis of the different levels and quality of engagements. Maybe the goal isn’t to come up with a definitive number or indicator but rather to draw out a composite of an organization or person’s online influence, which may include Web (blog) metrics, quality (manual process, for sure, at this point) of conversation, group membership or leadership in online groups (i.e. LinkedIn), etc. It may take a bit more time and require actual understanding of the nature and intent of social conversations but perhaps lead us to a more robust and sophisticated understanding of influence?

  • http://blog.ecairn.com Laurent

    I think it’s a nightmare to try to weight all the different signals from all over the virtual planet to score a person overall. We live in a world of specialists. Some specialists have become voices in their market/niche. The first step is to identify the expertise of a person. When you have a group of people with the same expertise, it becomes easier to qualify the signals. Thats what we’ve been doing here so we only take into account signals from social media marketer to evaluate the influence of a social media marketer.
    Laurent

  • http://twitter.com/CourtV Courtney

    Really great post, Tom. My company measures online influence, so we’ve been working on this topic for a very long time and I really appreciated your insight. I completely agree with your sentiment that the entire web needs to be considered for exactly the reason you presented – Seth Godin is obviously a huge online force, but his twitter account doesn’t necessarily show that very well. By incorporating all pieces of his web presence, you get a true representation of Seth Godin’s online influence. In our 3 metric scoring system, Seth has 100% reach and 98% resonance (our third metric, relevance, is based on each campaign so there is no constant percent). I’d love to give you more explanation if you’re interested!

    While we don’t focus on the sentiment of tweets and comments, it is certainly just as important in this space. Crimson Hexagon is doing a good job of measuring and identifying this.

    http://traackr.com

  • http://raulcolon.net Raul Colon

    I don’t agree with Klout’s claim a bit on the border towards deceptive advertising I would say.

    I take it as a compliment when Klout puts me up with those guys… but wait maybe they are doing that to feed your EGO which would be their target user.

    I have to agree that Re-tweets are effortless many people even RT just to get attention or to make others happy. I guess many people RT to be polite.

    I think the polling mechanism seems great. What a better way than to humanize measuring influence.

    And by the way I thought I had read this before and I had even commented. Anyway I think these comments are different from the ones I left on the Social Media Explorer post

  • http://twitter.com/webby2001 Tom Webster

    Always glad for repeat business, Raul :)

  • http://twitter.com/webby2001 Tom Webster

    Thanks, Courtney – love to check this out. Appreciate your comment.

  • http://twitter.com/webby2001 Tom Webster

    I think we will eventually get to that point – but I’m not much of a futurist. As a “presentist” my concerns are more about how these nascent measures are being used and less about the measures themselves. I do think someone will eventually crack this nut, more or less in the way you suggest.

  • S K Tagg

    I hope I can convince my students on a marketing masters option class customer/business metrics that these issues are critical.

  • http://www.friendadderelite.com/blog/ Friend Adder FAQs

    Great list here. This is such a helpful list for it makes you stand out from people who are just using the web carelessly, branding is an investment and it takes a lot of effort. Lets face it, the internet is full of hoax and useless information which we don’t really need. A list like this gives us a box where we will feel safer and can move along easily specially in branding our self. Thanks!

  • http://conniemcknight.com Connie McKnight

    You made some excellent points; the one about Seth Godin’s Klout score was so on the mark. It must mean that in order to have our influence measured accurately, it needs to be done be a real person instead of a web crawler.

  • http://twitter.com/ConnieMcKnight Connie McKnight

    You made some excellent points; the one about Seth Godin’s Klout score was so on the mark. It must mean that in order to have our influence measured accurately, it needs to be done be a real person instead of a web crawler.

Previous post:

Next post: