Thursday 9 October 2014

Sentiment And Semantic Analysis

                                              
Michalis founded DigitalMR in 2010 following a corporate career in market research with Synovate and MEMRB since 1991. This post was first published on the DigitalMR blog. Explore the blog here: www.digital-mr.com/blog

It took a bit longer than anticipated to write Part 3 of a series of posts about the content proliferation around social media research and social media marketing. In the previous two parts, we talked about Enterprise Feedback Management (December 2013) and Short -event-driven- Intercept Surveys (February 2014). This post is about sentiment and semantic analysis: two interrelated terms in the “race” to reach the highest sentiment accuracy that a social media monitoring tool can achieve. From where we sit, this seems to be a race that DigitalMR is running on its own, competing against its best score.
 
The best academic institution in this field, Stanford University, announced a few months ago that they had reached 80% sentiment accuracy; they since elevated it to 85% but this has only been achieved in the English language, based on comments for one vertical, namely movies -a rather straight-forward case of: “I liked the movie” or “I did not like it and here is why…”. Not to say that there will not be people sitting on the fence with their opinion about a movie, but even neutral comments in this case, will have less ambiguity than other product categories or subjects. The DigitalMR team of data scientists has been consistently achieving over 85% sentiment accuracy in multiple languages and multiple product categories since September 2013; this is when a few brilliant scientists (engineers and psychologists mainly) cracked the code of multilingual sentiment accuracy!
Let’s dive into sentiment and semantics in order to have a closer look on why these two types of analysis are important and useful for next-generation market research.
 
Sentiment Analysis
 
The sentiment accuracy from most automated social media monitoring tools (we know of about 300 of them) is lower than 60%. This means that if you take 100 posts that are supposed to be positive about a brand, only 60 of them will actually be positive; the rest will be neutral, negative or irrelevant. This is almost like the flip of a coin, so why do companies subscribe to SaaS tools with such unacceptable data quality? Does anyone know? The caveat around sentiment accuracy is that the maximum achievable accuracy using an automated method is not 100% but rather 90% or even less. This is so, because when humans are asked to annotate sentiment to a number of comments, they will not agree at least 1 in 10 times. DigitalMR has achieved 91% in the German language but the accuracy was established by 3 specific DigitalMR curators. If we were to have 3 different people curate the comments we may come up with a different accuracy; sarcasm -and in more general ambiguity- is the main reason for this disagreement. Some studies (such as the one mentioned in the paper Semi-Supervised Recognition of Sarcastic Sentences in Online Product Reviews) of large numbers of tweets, have shown that less than 5% of the total number of tweets reviewed were sarcastic. The question is: does it make sense to solve the problem of sarcasm in machine learning-based sentiment analysis? We think it does and we find it exciting that no-one else has solved it yet.
Automated sentiment analysis allows us to create structure around large amounts of unstructured data without having to read each document or post one by one. We can analyse sentiment by: brand, topic, sub-topic, attribute, topic within brands and so on; this is when social analytics becomes a very useful source of insights for brand performance. The WWW is the largest focus group in the world and it is always on. We just need a good way to turn qualitative information into robust contextualised quantitative information.
 
Semantic Analysis
 
Some describe semantic analysis as “keyword analysis” which could also be referred to as “topic analysis”, and as described in the previous paragraph, we can even drill down to report on sub-topics and attributes.
 
Semantics is the study of meaning and understanding language. As researchers we need to provide context that goes along with the sentiment because without the right context the intended meaning can easily be misunderstood. Ambiguity makes this type of analytics difficult, for example, when we say “apple”, do we mean the brand or the fruit? When we say “mine”, do we mean the possessive proposition, the explosive device, or the place from which we extract useful raw materials?
Semantic analysis can help:
  • extract relevant and useful information from large bodies of unstructured data i.e. text.
  • find an answer to a question without having to ask anyone!
  • discover the meaning of colloquial speech in online posts and
  • uncover specific meanings to words used in foreign languages mixed with our own
What does high accuracy sentiment and semantic analysis of social media listening posts mean for market research? It means that a 50 billion US$ industry can finally divert some of the spending- from asking questions to a sample, using long and boring questionnaires- to listening to unsolicited opinions of the whole universe (census data) of their product category’s users.
 
This is big data analytics at its best and once there is confidence that sentiment and semantics are accurate, the sky is the limit for social analytics. Think about detection and scoring of specific emotions and not just varying degrees of sentiment; think, automated relevance ranking of posts in order to allocate them in vertical reports correctly; think, rating purchase intent and thus identifying hot leads. After all, accuracy was the only reason why Google beat Yahoo and became the most used search engine in the world. 

29 comments:

  1. V P Associates Chartered Tax Advisors and Chartered Certified Accountants, represent an authoritative source on most aspects of tax and specialise in the following: Self-Assessment Tax advice, Self Employed Tax Returns, Company and Corporation Tax advice, Capital Gains Tax advice, Buy-To-Let Property Tax advice, Tax Planning and Tax Consultancy Services. With over 25 years of experience in the business of providing expert tax advice, V P Associates Chartered Tax Advisors offer valuable information to clients to help them save money, reduce and mitigate tax. V P Associates, Chartered Tax Advisors & Chartered Certified Accountants, are the first and foremost specialists in taxation matters. Visit their site for a comprehensive list of services: http://vpassociates.com/

    Accountants Crawley

    ReplyDelete
  2. V P Associates Chartered Tax Advisors and Chartered Certified Accountants, represent an authoritative source on most aspects of tax and specialise in the following: Self-Assessment Tax advice, Self Employed Tax Returns,Company and Corporation Tax advice,Capital Gains Tax advice, Buy-To-Let Property Tax advice, Tax Planning and Tax Consultancy services. With over 25 years of experience in the business of providing expert tax advice, V P Associates Chartered Tax Advisors offer valuable information to clients to help them save money,reduce and mitigate tax. V P Associates, Chartered Tax Advisors & Chartered Certified Accountants, are the first and foremost specialists in taxation matters. Visit their site for a comprehensive list of services :http://vpassociates.com/
    Accountants Lingfield

    ReplyDelete
  3. V P Associates Chartered Tax Advisors and Chartered Certified Accountants, represent an authoritative source on most aspects of tax and specialise in the following: Self-Assessment Tax advice, Self Employed Tax Returns,Company and Corporation Tax advice,Capital Gains Tax advice, Buy-To-Let Property Tax advice, Tax Planning and Tax Consultancy services. With over 25 years of experience in the business of providing expert tax advice, V P Associates Chartered Tax Advisors offer valuable information to clients to help them save money,reduce and mitigate tax. V P Associates, Chartered Tax Advisors & Chartered Certified Accountants, are the first and foremost specialists in taxation matters. Visit their site for a comprehensive list of services :http://vpassociates.com/
    Accountants Lingfield

    ReplyDelete
  4. V P Associates Chartered Tax Advisors and Chartered Certified Accountants, represent an authoritative source on most aspects of tax and specialise in the following: Self-Assessment Tax advice, Self Employed Tax Returns, Company and Corporation Tax advice, Capital Gains Tax advice, Buy-To-Let Property Tax advice, Tax Planning and Tax Consultancy Services. With over 25 years of experience in the business of providing expert tax advice, V P Associates Chartered Tax Advisors offer valuable information to clients to help them save money, reduce and mitigate tax. V P Associates, Chartered Tax Advisors & Chartered Certified Accountants, are the first and foremost specialists in taxation matters. Visit their site for a comprehensive list of services: http://vpassociates.com/

    Accountants Haywards Heath

    ReplyDelete
  5. V P Associates Chartered Tax Advisors and Chartered Certified Accountants, represent an authoritative source on most aspects of tax and specialise in the following: Self-Assessment Tax advice, Self Employed Tax Returns, Company and Corporation Tax advice, Capital Gains Tax advice, Buy-To-Let Property Tax advice, Tax Planning and Tax Consultancy Services. With over 25 years of experience in the business of providing expert tax advice, V P Associates Chartered Tax Advisors offer valuable information to clients to help them save money, reduce and mitigate tax. V P Associates, Chartered Tax Advisors & Chartered Certified Accountants, are the first and foremost specialists in taxation matters. Visit their site for a comprehensive list of services: http://vpassociates.com/

    Accountants Haywards Heath

    ReplyDelete
  6. V P Associates Chartered Tax Advisors and Chartered Certified Accountants, represent an authoritative source on most aspects of tax and specialise in the following: Self-Assessment Tax advice, Self Employed Tax Returns, Company and Corporation Tax advice, Capital Gains Tax advice, Buy-To-Let Property Tax advice, Tax Planning and Tax Consultancy Services. With over 25 years of experience in the business of providing expert tax advice, V P Associates Chartered Tax Advisors offer valuable information to clients to help them save money, reduce and mitigate tax. V P Associates, Chartered Tax Advisors & Chartered Certified Accountants, are the first and foremost specialists in taxation matters. Visit their site for a comprehensive list of services: http://vpassociates.com/

    Accountants Horley

    ReplyDelete
  7. V P Associates Chartered Tax Advisors and Chartered Certified Accountants, represent an authoritative source on most aspects of tax and specialise in the following: Self-Assessment Tax advice, Self Employed Tax Returns, Company and Corporation Tax advice, Capital Gains Tax advice, Buy-To-Let Property Tax advice, Tax Planning and Tax Consultancy Services. With over 25 years of experience in the business of providing expert tax advice, V P Associates Chartered Tax Advisors offer valuable information to clients to help them save money, reduce and mitigate tax. V P Associates, Chartered Tax Advisors & Chartered Certified Accountants, are the first and foremost specialists in taxation matters. Visit their site for a comprehensive list of services: http://vpassociates.com/

    Accountants Horley

    ReplyDelete
  8. Mileo is instant group messaging application that connects teacher-student, coach-player, leader-follower without number sharing, safe and anonymous.

    Class

    ReplyDelete
  9. Mileo is instant group messaging application that connects teacher-student, coach-player, leader-follower without number sharing, safe and anonymous.

    Class

    ReplyDelete
  10. Emails Fresh Leads and Business All Country price Low, Tools Private
    - Email Extractor Private 2016
    - Wordpress Brute Force
    - Extract Phone Number


    For details click blacklistemail

    ReplyDelete
  11. This is an online e-commerce website where customers can purchase mid to high-range fitness accessories, apparel and nutritional supplements. Our incredible team developed this e-commerce website to showcase different categories of apparel, accessories and supplements to promote ease-of-access to our customers. We offer quality products designed for results oriented and effective workout. Even when you're not working out, we have other products and accessories to keep you looking your best and staying motivated.

    Lastly, I started "Workout or Walkout" after I said similar words to someone in a gym one day when I got tired of people spending more time on their phones than they actually spend working out in the gym. I love and encourage phones in the gym for many reasons but if I need to use an equipment and it's not available simply because you choose to use it as a couch to text or chat, then we have a tiny bit of a problem my friend. Workout, Eat healthy; it's a lifetime thing! #workoutorwalkout #stayhealthymyfriends

    For more information please click at fitness apparel

    ReplyDelete
  12. Atomic Design Need Internet marketing services that get results, a web designer, or a graphic designer? Then we can help. We assist companies rank at the top of the SERPs for their targeted keywords, generating millions of dollars in sales, all through our proven SEO services. Our digital agency methodology is attract, impress, and convert. We start by getting your business to rank on major search engines. An optimized listing ATTRACTs leads to your website. Next we IMPRESS site visitors with a superior user experience. Graphics, content, and calls-to-action combine to provide value to your visitors, which increases the chance visitors will become leads and CONVERT into paying customers.

    ReplyDelete
  13. roof installation All about Roof, calculator a detail list of roof materials, roof installation and roofing contractors.

    ReplyDelete

  14. stop snoring how to stop snoring. snoring device reviews.

    ReplyDelete
  15. stop snoring device reviews how to stop snoring. stop snoring device reviews, stop snoring, stop snoring mouthpieces.

    ReplyDelete

  16. stop snoring Home workout video tips. Workouts for home.

    ReplyDelete
  17. Clickfunnels guida Cos'è clickfunnels e come guadagnare online con il Funnel Marketing. Il corso che che in 4 video ti mostrerà le potenzialità del funnel e di clickfunnels.

    ReplyDelete
  18. Forex Trendy Review Forex Trendy Review : Forex Trendy is a computer software that enables traders the privilege to easily access profits and see gains in the forex market. Forex Trendy maximizes your presence in the forex market by enabling you to know when the market is in a good spectrum for buying and selling, by simply implementing trends.

    ReplyDelete
  19. appearance Voice your opinion! Review Italojewelry now.

    ReplyDelete
  20. wart remover having problems getting rid of your warts cleanly, then go to this website to learn about a fool proof wart removal treatment.

    ReplyDelete

  21. beauty tips negative body image, healthy body image, body images, body image activities, body image positive, positive body image, healthy body image pictures, body realistic, eating dissorders quote body image, realistic body, body talk, realistic body inspiration,

    ReplyDelete

  22. body to body massage prague In salon Ilandra Best erotic massage Prague Beautiful masseuses will massage you erotic massage will completely make you relax We offers you erotic massages, body2body, nuru massages prague, massage of penis and culmination. We offer our services 24/7.

    ReplyDelete
  23. Fl Studio A review of todays best music production software.

    ReplyDelete