What Is Sentiment Analysis?

Types of sentiment analysis 

Sentiment analysis models aim at defining polarity (positive, neutral, negative), emotions (disappointed, happy, furious), intentions (interested or not, willing to buy or not), and urgency. Depending on your analysis goals, you can use various categories to interpret customer feedback and adjust them to your specific needs. Some of the most popular sentiment analysis types include:

Fine-grained sentiment analysis

If you seek to make your sentiment analysis as precise as possible, you can add additional polarity categories, such as:

• Very negative

• Negative

• Neutral

• Positive

• Very positive 

These categories correlate with five-star rating reviews, where very positive is equal to 5 stars and very negative is equivalent to 1 star.

Emotion detection 

This type focuses on emotions and feelings, e.g., frustration, happiness, and others. Many of the emotion detection approaches are lexicon-based, meaning they use systems of emotionally charged words. You can also use machine learning algorithms to detect the sentiment behind certain words.

Aspect-based sentiment analysis

When analyzing sentiments in a piece of text, brands want to know what specific features and aspects of their products customers are discussing in a positive, negative or neutral way. For example, in this review: “The camera in this phone is worse than I expected,” a negative opinion is expressed towards a particular feature of the product.

Multilingual sentiment analysis

It allows the evaluation of sentiment scores of texts in different languages. Our culture and language affect the words we choose and how we use them to explain emotions and thoughts. So, sentences don't always have the same meaning in other languages when translated word-for-word.

Multilingual sentiment analysis doesn't use translation. It's designed to recognize the subtleties of different languages; therefore, it delivers a precise sentiment interpretation.

Why is sentiment analysis important? 

Since sentiment analysis uses automated methods, it makes it possible to sort out and analyze enormous amounts of the sentiment behind social media conversations and reviews in a timely manner. As a result, companies can make better and more informed decisions based on sufficient data and in-depth analysis.

Overall, basic sentiment analysis facilitates the process of gathering and measuring social data in several ways:

• Seizing large amounts of data. According to the World Economic Forum, it was expected that the amount of data online was going to reach 44 zettabytes by 2020, which is 40 times more bytes than the stars in the observable universe. These statistics are both stunning and intimidating since there’s no way to collect and process this data manually. Therefore, you would need automated sentiment analysis tools.

• Real-time analysis. It's always crucial to stay updated on your customers’ opinions and reactions in real time to take action immediately if a severe problem arises.  

• Centralized analysis criteria. Deciding on whether a piece of text is positive, neutral, or negative can be a challenging task for humans since they may make subjective judgments based on their previous experiences and beliefs. That is why it's better to be guided by a unified sentiment analysis system that can be applied to all text data.

How does sentiment analysis work?

To understand how sentiment analysis works, we need to dig deeper into the main approaches it employs. There are three major sentiment analysis algorithms that can be implemented in sentiment analysis and opinion mining: rule-based (lexicon-based), automatic (machine learning), and hybrid.

Rule-based approach

Most of the time, rule-based sentiment analysis algorithms rely on manually crafted rules to determine polarity, subjectivity, and sentiment in a piece of text. These rules are based on different NLP sentiment analysis techniques that were initially developed in computational linguistics, including part-of-speech tagging, tokenization, stemming, etc.

In this approach, sentiment analysis makes use of sentiment analysis datasets, e.g., large libraries of adjectives (good, fantastic, disgusting, terrible) and phrases (excellent service, awful movie) that have been previously assigned particular scores by human coders.

This hand-scoring process can be tricky and inaccurate since everyone participating in it has to come to an agreement regarding the sentiment scores. For instance, if one person assigns a sentiment score of 0.5 to the word good, but another person gives the same sentiment score to the word amazing, your sentiment analysis system will perceive both words as equally positive, which will lead to subsequent confusion and wrong results. 

Let’s take a look at an example of how a rule-based sentiment analysis system works:

1. Determines two polarities with two lists of polarized and sentiment-bearing words, e.g., negative words such as horrible, bad, awful, and positive mentions such as best, good, fabulous, etc.

2. Attaches a sentiment score to each word and component.

3. Counts how many times positive and negative words appear in the text. 

4. If the number of negative words is bigger than the number of positive words, the system returns a negative sentiment and vice versa. If the numbers are equal, the total sentiment will be marked as neutral.

Supervised machine learning for sentiment analysis

In supervised ML-based sentiment analysis, a statistical model is fed a number of pre-tagged texts to analyze. After the training, the model is given un-tagged examples to analyze. Some of the most popular supervised NLP machine learning algorithms are Bayesian Networks, Support Vector Machines, Conditional Random Field, etc.

All in all, supervised machine learning involves:

• Tokenization – breaking text documents into smaller pieces, such as words, for the model to better understand. 

• Part-of-speech tagging – identifying parts of speech, e.g., nouns, verbs, adjectives. 

• Sentiment analysis itself – identifying whether the piece of text is positive, negative, or neutral and giving a specific sentiment score to each entity.

Unsupervised machine learning for sentiment analysis

In unsupervised ML, a model trains without any pre-tagging. It uses such techniques as clustering, that is, grouping similar text together, and latent semantic indexing (LSI), which aims at identifying words and phrases that often appear next to each other in sentences. 

Unsupervised machine learning can be flawed; that’s why the best solution, as always, is to combine several approaches and techniques to achieve maximum performance.

The main difference between the automatic ML-based approach and the rule-based one is that the former can analyze way more data due to the automatization. The disadvantage of the ML-based algorithm is that it makes it difficult to explain why specific texts are categorized as bearing positive or negative sentiment.

In general, to achieve the highest accuracy, it's better to use a hybrid approach, which combines lexicon-based sentiment analysis techniques with ML algorithms.

Sentiment analysis challenges

Sentiment analysis is one of the most challenging jobs in NLP since even people may struggle to identify and analyze sentiment correctly. Even though sentiment analysis models are getting more superior and accurate, there are still numerous obstacles that prevent them from being the ultimate solution.

Context

All spoken and written words are uttered in some specific circumstances, at some point in time, by some particular people and to other people. In other words, they all have context behind them. The problem is that machines cannot recognize the context if it isn’t brought up on purpose. Let’s imagine a situation where we have two responses to a survey regarding a recent conference:

1.  All of it.

2. Totally nothing.

Now suppose, these two responses answer the question “What did you dislike about the conference?” In this case, the first answer would bear negative sentiment, meaning that the respondent dislikes everything about the conference. And the second response would deliver positive sentiment, implying that the person liked everything about the event. But if we change the question to “What did you like about the conference?”, the sentiment behind these two answers will shift to the opposite polarity.

In order to capture the negative or positive sentiment in these replies, it's necessary to understand the context. However, the process of teaching a model how to understand it is not clear and straightforward.

Comparisons

Comparative sentences aren't easy to decipher. You may need a deeper knowledge of the compared objects to understand the sentiment of a text. The same applies to a system analyzing textual sentiment. Take a look at this example:

Those headphones are like concert speakers!

Since no emotions are expressed, an analysis system would be unable to tell if this sentence is positive, neutral, or negative. It would have to know that concert speakers are loud and headphones are silent; therefore, the sentiment is positive.

Tone

The tone in writing refers to a creator's attitude toward the subject of a text. It's conveyed by the relation of specific words to the subject. But to train a machine to understand the tone can be a hardship. Let's consider this example of a customer's review:

I knew it would be difficult to integrate their service into my infrastructure, but I didn't think it would be that hard. Everything their support did was hopeless, and even the manager was clueless. We spent countless hours on the phone with no resolution. It turns out my infrastructure was faulty. I managed to take care of that and accommodate their product. I am very thankful for the patience of their technical support!

It's apparent to us that the customer is positive about the company and its service. However, let's say sentiment analysis software uses a rule-based approach. It will count all the negative and positive words and then make up a sentiment score. In this case, we have seven negative words and one positive word, meaning the sentiment of this review would be considered negative by the sentiment analysis software.

Sarcasm and irony

People usually express sarcasm and irony using positive words. Machines may have hard times trying to understand the sentiment in these expressions without knowing the context. For example, on a traveling company’s website, we can find reviews answering the question “Did you enjoy traveling with us?”

1. Absolutely, the best travel agency ever!

2. Sure, the experience I got was unforgettable!

At first glance, these responses may look like positive comments, considering they contain such words as best and sure, which are usually marked as positive. However, these replies can also be interpreted as sarcastic and bear negative sentiment, and we can come up with multiple situations where it can be interpreted as such.

Emojis

According to Guibon et al., there are three types of emojis: Western emojis, e.g., :0, containing one or two characters, more complex Eastern emojis, e.g. (°レ°), and the Unicode emoji characters. Analyzing emojis and characters is just as crucial as analyzing words and other speech components, especially when it comes to interpreting tweets. Emojis can also be broken down into tokens and whitelisted – this will help enhance sentiment analysis performance. 

Customer service

Customers seek instant and stress-free interactions with brands. The way the companies provide their products and services is just as important as what they provide. In customer service, you can use customer sentiment analysis to arrange incoming client queries according to their urgency and topic and direct them to the respective departments. It makes communication with customers more efficient and ensures that the most time-sensitive matters are solved immediately.

Conclusion

As customers generate more and more reviews and comments online daily, it’s evident how important it is to process this data and draw conclusions promptly. Sentiment analysis provides an understanding of how your clients feel about your brand and product and how you can improve your services. Based on natural language processing and constantly progressing machine learning techniques, sentiment analysis serves multiple use cases, including brand monitoring and market research.

We also suggest that you check other similar articles on news scraping and online media monitoring.