Earlier today we launched Sentiment Analysis in our A-List Platform — further enhancing the A-List as the most robust & actionable influencer platform out there.  We are very excited for this particular release.  We feel that Sentiment tracking will help our clients gain additional, important, strategic insights that will guide both the development and ongoing execution of their campaign work.

HOWEVER…I want to be very clear about the Sentiment Analysis feature we released.  We DID NOT release an automated sentiment engine. We apologize to those who were hoping otherwise, but the sentiment feature we released is based on MANUAL input provided by our users. We have built a system that gives our users the ability to tag the sentiment of specific posts produced by the influencers tracked on any list.

That’s right.  MANUAL tagging.

We didn’t do this because we want to make life harder on our users or because we wanted to make more work for those managing influencer work.  We did it because we, fundamentally, don’t believe in the effectiveness/accuracy of automated sentiment engines or the actionability of the data they provide.

First of all, automated sentiment analysis is a HUGE technical challenge which….has NOT been solved.  At least not yet.  Or for the foreseeable future.  From a technical perspective, it involves all the data processing sophistication involved with any big data problem along with the added complexity of NLP (natural language processing) which has to account for and adapt to the ways humans use language.  [For example, how does a computer account for posts like:  That is one bad ass motorcycle.  OR Blue screen of death again...Love you Microsoft!]  Ultimately, we don’t feel like this is possible in any sort of automated way.  We’ve looked at several tools and have spoken to several experts about the subject and are not convinced that anyone is even close.

In fact, I recently spoke to a representative from one of the biggest general monitoring companies who proudly claimed that their sentiment engine boasted a 74% accuracy rate – compared to the 50-55% accuracy of the other big monitoring tools.  So, that means the “best” tool on the market still has a 26% margin of error.  I’m not sure of any other statistical model where this would be considered a ‘success’ (maybe someone can point me to a good example, but I won’t hold my breath).  Not only is this margin of error far too large for any serious analyst to trust, but I can’t see how anything with that kind of margin is actionable in the least.

With manual sentiment, you have 100% accuracy.  We like that number.

SENTIMENT, LIKE INFLUENCE, IS CONTEXTUAL

The second, and most important, reason why we wouldn’t consider an automated sentiment engine for our product is that sentiment, like influence, is highly contextual.  The sentiment of a statement (or post) is highly dependent on the perspective of the reader.  Yes, there are certainly cases where sentiment is very clear and not debatable (BMW sucks! is clearly negative), but these instances are far-and-few between.  In most cases, sentiment is not clear-cut.

Automated sentiment engines assume that any given post has a set sentiment that never changes.  A negative post is a negative post in any situation, at any time.  We don’t think this is the case.  At all.  We think the sentiment of a single post can change drastically depending on the perspecitve of the reader.  Take a post like:

Those new Microsoft commercials are ok, but I liked them better when Apple did them  btw – what is Jerry Seinfeld thinking?!   So stupid.  

What would an automated sentiment engine do with this?  Probably neutral.  Maybe positive because of the smiley face.  Or maybe negative because of the mention of “stupid.”  Who knows.  In any case, the automated engine can’t account for the most important part – context.

What if you were Microsoft’s brand manager, hoping to create a campaign that would help people to forget about Apple?  You’d probably consider this post pretty negative (doh!  Damn you Apple!!).  But what if you were Apple’s ad agency?  Pretty positive for you (awesome, people think of our commercials even when they watch a Microsoft ad!).  And what if you were Jerry Seinfeld?  Not great.  Negative sentiment (what’s the deal with this twitter thing anyway??).

This is just a quick, silly example, but I think you can see the issue.  Same exact post, 3 different sentiments — all accurate.  All based on the context of the beholder.  This is why, fundamentally, we don’t believe in automated sentiment analysis.  Even if the technology worked, it just doesn’t make any sense.

WITH TRAACKR, MANUAL SENTIMENT IS TOTALLY MANAGEABLE   

Don’t get me wrong – we totally understand the “value proposition” for automated sentiment as well as the need for it.  Automated sentiment was designed to help researchers make sense of an enormous amount of data.  Automated sentiment is necessary when you are trying to analyze the general web – thousands, even millions, of posts.  We get that.  It’s a task that can’t be done manually.  Here’s the problem.  The web is an ENORMOUS OCEAN OF DIRTY WATER.  Honestly, the web makes the Great Pacific Garbage Patch look like a well manicured lawn.  So, yes, there may be a million posts mentioning your brand that you want analyzed for sentiment, but 70-80% of them (unscientific estimation) are junk and not worth any attention.  So, if 70% of the posts in a given sample are junk and the sentiment engine has a 30-50% margin of error….honestly, what’s the point?

But with TRAACKR, you are able to filter the web by the people who matter most to you at any given time.  By focusing on the content from the RIGHT PEOPLE, you eliminate the problem of junk, random, non-important posts and you make the process of manually sentiment tagging completely manageable and realistic.

And this gets you real, accurate, ACTIONABLE data.  You don’t deserve anything less…do you?

DS

This entry was posted on Tuesday, September 13th, 2011 at 10:05 am and is filed under Feature Updates, For the Traackr Expert, This We Believe. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.