Sentiment Analysis of Tweets at Sentence Level Using Hadoop

Author Name(s): Yazala Ritika Siril Paul, Dilipkumar A. Borikar
Author Email: *paulys@rknec.edu

Abstract

In the last couple of decades Sentiment Analysis has attracted considerable amount of interest from the research community across the globe. Sentiment Analysis brings to light the underlying point of view(s) in a text; for example classifying a review as positive, negative or neutral. The main aim is to extract a set of potential confident features from the review and then classify them into emotions which they strongly depict. In this paper we dig deeper into analysing the sentiments and classify them into Ekman’s six basic emotions i.e. Anger, Fear, Disgust, Sadness, Happiness and Surprise using Hadoop with the assistance of Apache Flume, Apache Hive.

Introduction

Starting from the advent of 1990s, which is marked as the beginning of transmission of modern Internet, the usage of internet has been increased in various forms. In today’s world since textual data is increasing in an unimaginable range, there are many organizations that are trying to use this large amount of data to extract people’s opinions towards their products. Social Networking Sites (SNS) proves to the best source for obtaining the data for analysis. However since the data collected from sources is huge, it is highly unfeasible to manually analyze them. Data is critical to organizations for the immense value that it offers. In the past organizations have considered data uneconomical and the rest have used solutions which limited their capabilities to derive the value from data. In modern times technologies have advanced, businesses have become more dynamic and organizations now want to derive value from the data they always had. It is important for them to identify the data that needs to be captured and stored. With the increase in data, changes in the approach to capture, manage, process and visualize data is highly required. The need to aggregate and analyze this data to derive hidden insights has grown with exponential data growth at an exponential rate.

Conclusion

In this research work, we have proposed a framework for further classifying the sentiments into Ekman’s six basic emotions using Hadoop. Language filter was applied during the data ingestion in order to reduce load in HDFS. The unstructured ingested data that is stored in HDFS, is first structured and then processed by using MapReduce algorithm. The classification of emotions is performed by using machine learning techniques like SVM, Multinomial Naïve Bayes and Bernoulli Naïve Bayes earning maximum accuracy of 81.0431% from SVM

964 total views, no views today

Download PDF File

About the author: admin