Tom Webster, writing and speaking

The Twitter Selection Bias

Added on by Tom Webster.

I caught myself doing something today that is ultimately counterproductive. Maybe you've done the same thing. The post I wrote yesterday on the great Forrester Analyst Blog Kerfuffle seemed to hit a chord with some people, and I was very fortunate to have the post picked up on Social Media Today. This led to a lot of people I've yet to meet retweeting this post, and I dutifully added them all to my "following" list. I'm pleased that the post got some traction, and thrilled that there were some Twitter users out there that shared my opinion.

In research, one of the primary things we deal with is the concept of selection bias. In a convenience sample, the researcher is selecting the respondents, usually with some kind of quota of mind. In a typical online survey, there is a self-selection bias--you are only getting the opinions of the folks who chose to go out of their way to respond, which often leaves the sample stacked with people who feel very strongly one way or another about the topic of the survey. What is missing from both of these examples are the opinions of the people you didn't select--either left out by the researcher, or not sufficiently motivated to respond to an online survey. We call this "non-response bias," and it is something we have to model all the time (we do this in the Exit Polls pretty rigorously, as you might imagine.)

When you follow people on Twitter because they replied to you, or retweeted your post, you are stocking your follower list in much the same way. If you aren't too careful, you'll end up with a group of followers that can tilt anywhere from generally positive/favorable to your tweets to downright sycophantic. This might be great for your ego, but it's misleading and ultimately bad for the strength of your ideas.

Of course, you should follow people interested in your ideas, and definitely strike up conversations with those who choose to engage them. That's being a solid social citizen. But as a reality check, it's also helpful to do a Twitter search on the topics you write and tweet about. Actively seek out people who may or may not have engaged with you, but who definitely would sit on the other side of a given issue or argument with you. Follow them, too.

It's all too easy to "curate" your twitter stream to what seems comfortable--to stock it, albeit unconsciously, with attaboys and kudos, retweets and replies. But you are in charge of your Twitter experience, and that means going outside your comfort zone a bit to ensure that you are seeing all sides of the issues that you care about. Maybe you'll engage, maybe you won't--but your ideas will be stronger in the end by mitigating for "Twitter selection bias." And, speaking from experience, you won't get such a swelled head, either.