Interesting post and quite thought-provoking. I think that it’s actually incorrect to look at twitter posts and blogs as “unstructured” or “random” text. Most text and sentiment analysis is done on a topic, or brand mention. That brand or theme is the linguistic frame. The posts and tweets around a particular theme are structured in reference to a frame as I see it.

Further, sentiment analysis needs more linguistic grounding to become more accurate. I have seen and treat blogs and tweets as two different types of discourse.

My point? You need to think about the function and structure of the text being analyzed.

What I mean is: Google surely is working in sentiment analysis beyond product reviews because they are in the best position to do it.

Big thumbs-up for the Borges reference, though – a big fave.

Then you need data, and exploratory tools to analyze new kind of texts rather than spending time in individual items.
Who is in a better position for that? I think it’s Google, they outperform in machine learning research too.

BTW, that was an opinion, now is near an assertion: Google’s New Review Search Option and Sentiment Analysis http://www.seobythesea.com/?p=1488

There is enormous brand exposure on YouTube and the comments provide valuable feedback from engaged users. I’ve just undertaken an analysis of the demographics of YouTube commenters and it’s most instructive, see http://www.vquence.com.au/2010/04/25/youtube-commenters-demography/ .

Excellent article! Myself and a few colleagues *do* have Computer Science degrees and *still* share your skepticism. Our answer to it is a service we’ve launched called “upprdwnr” (http://upprdwnr.com). Rather than automatically monitor everything that is said, we ask a little help of the community (just Twitter right now) – tell us if you’re thinking positive or negative.

So, in your example above – if “Toyota FTW!” is tweeted instead as: “#Toyota FTW! #uppr” upprdwnr sees this as a positive thing for #Toyota. Similarly, “If my #Toyota would stop when I pressed the brake, I’d LOVE it! #dwnr” would be seen as a negative thing for #Toyota.

Blunt? Sure. Old school? Kind of. But we think it addresses a lot of your concerns (even if it introduces a new one – mass adoption), and we think it is more in the spirit of the social space. These hashtags aren’t exactly asking you to do backbends in your tweets (in fact, we’ve found many situations where using them actually made our tweets shorter), and as a result we’ve got humans deciding what humans feel instead of robots in the ether.

Would love to hear your take on our approach given your skepticism of the automated flavor. We’re just getting rolling, so we’ll take any feedback or advice to work toward that little mass adoption issue!

The problem is, computers still can’t “understand” the nuance of human speech. People don’t communicate — especially online — in a concise, measurable fashion.

Great piece, Tom.

Obviously people want to develop relationships with people-but having a metric for the overall impact of those efforts over time is not a bad thing–it helps justify the effort to the C-level and encourage more companies to be good actors in the space. That ain’t all bad.

For the 1- no need to elaborate. People get barely 70% with a huge level of training up to the point that using sampling ans human rating is more cost effective. And most of conversations are stated neutral. They would be as close saying all conversations are neutral.

As for 2
- Take a brand like Benetton. In such fragemented industy they are ok to piss off 90% of the world if the 10 other % become fans.
This all comes to positioning. If the positive sentiment comes from the wrong people or on the wrong attribute then it’s not positive.
It’s good that exclusive brand are viewed as too pricey and discounted airlines are not expected to give champagne for free. -it shows they don’t cut on security …

As for the “social” part. When consumer say they want brands to engage, they mean people, not algorithms. You can’t develop trust and relationship with a machine.

Best

For example a sentence “I love your comfortable beds and sheets, but your front desk staff are rude, not welcoming, and extremely unfriendly.”

There is very positive sentiment about comfort of beds and sheets, and extreme negative sentiment about friendliness of staff. The only way to properly determine the sentiments in this sentence and attribute them properly to the correct objects and concepts is through a sentiment analysis engine that uses a full NLP linguistic deep sentence parse as one of its inputs.

Also users need to be able to tune sentiments based on the topics they are looking at. If I said “the sheets, walls, and soup were thin” that is negative in the hospitality industry, but “my new iPad is thin” is a positive in the electronics industry.

And correction for various forms of negation is critical. If my “iPad is too thin, I think it may snap in half”, that is negative… too much of a good thing is negative. If a hotel “used to be great” that means it probably isn’t anymore.

For social media in particular, this is critical. A blog posting may express a whole variety of sentiments about a variety of topics, positive, neutral, and negative. If you make Corn Flakes, you only care about the sentiments relating to Corn Flakes (and maybe your competitors) in a blog, but not about the rest of the blog that may talk about something entirely unrelated.

I don’t know how anyone is getting any decent precision and recall on sentiment without using full NLP deep parsing…

- Justin Langseth, President & CTO, Clarabridge, Inc.

You make some great points in your article. From the perspective of a company in this space; sample is everything. Using SM monitoring tools, it’s important to know the collection methods of the company creating the tool in order to understand what your sample will consist of. If the company collects all of Twitter and 20% of blogs, that’s going to bias your sample just as sentiment algorithms can.

By listening in to online conversations we are inherently eliminating a certain amount of bias that traditional market research methods such as focus groups and panels can’t avoid. That doesn’t mean SM Research is bias-free- which you have certainly addressed in your post and comments.

It’s important for researchers to stress transparency with their vendors, so that they can learn where the biases are in their research and how that will affect the outcome. This applies to sentiment as well as the source of the conversations. In such a new space, vendors must also be sure to prioritize the movement towards reducing bias in their data wherever possible. Just like a brand to a consumer, vendors need to mature with the needs of social media research space.

This is a really great article. Look forward to more writing from you.

Best,
Laura

I told someone the other day that Social Metrics without Facebook is like doing a research study on what sports fans in the United States think but excluding NFL fans.