BrandSavant

Gaining Insight From Social Media Data

The Context of Online Surveys

by Tom Webster on April 23, 2009

I don’t know about you, but I am getting bombarded lately with pop-up survey requests on sites I use every day. I am worried about the damage this is doing to our industry–the more ‘work’ we make our customers do, the less likely they are to return to our websites.

Today, for example, I went to FedEx Office (Formerly FedEx Kinkos) to send a large document for printing and binding. Before I could do this (i.e., at the very beginning of my transaction) I was presented with a screen-blanking popup asking me to take a survey. Now, I don’t know about you, but I don’t spend a lot of my discretionary time fooling around at FedEx Office, you know, watching funny videos and stuff. There is really only one reason why I (or anyone) would go there–to get some work done! In that time, place and context–trying to get a document printed–I am not very likely to take a survey. And, in fact, though I usually make an effort to take all of these, I clicked out of that one as quick as I could.

Now, I am sure that their survey vendor has convinced them that online surveys are just as effective, and no doubt, sheerly by traffic alone, they will get enough completes to finish the survey with a statistically reliable number. Reliable, but not representative. Self-selected online surveys are bad enough without selecting OUT really busy people like me. What do you think the non-response bias on this survey is going to be?

A simple thing would have been to present the survey AFTER I finished my transaction, when I A) had something to rate, and B) was done with the work I NEEDED to do when I got there. Even better, though, would have been to contact me when I wasn’t at the site at all and could devote a little free time and thought to the survey instrument.

Honestly, self-service, self-selected studies like this are appearing WAY too frequently online, and if the market research business isn’t careful, it’s going to pollute this channel forever.

Buffer
  • Chris MacDonald

    Is there an “abuse bias” that the collector would need to tease out? What demographic profiles gravitate toward responding in an ever increasing barrage of requests. Are cat ladies, random fringe-y people and peopke with a tremendous amount of free time informing how our products/services are designed? Inserting my bias of course, but I rarely complete these surveys.

  • Tom Webster

    Thanks for responding! Which one are you, Cat Lady or Fringe-y?

    This increasing barrage, as you aptly put it, is going to give market research a permanent black eye, and you are right to question just who is answering these things. Actually, it doesn’t even matter who is answering them, the problem is the non-response bias. Surveys have to start being more transactional, and start treating respondents’ time as valuable, or cat ladies will be designing everything. No offense, cats.

Previous post:

Next post: