An Empirical Study: Automating e-Commerce Product Rating Through an Analysis of Customer Review

— e-Commerce today is a remarkable experience. However, finding and purchasing a right quality product based on numerous product reviews and manual rating in the e-commerce websites utilize much time among the consumers. This paper presents the problems faced by the consumers when buying products in e-commerce websites and a solution to solve the problems. Thus, the idea of an automated product rating system would be very useful for the consumers in which it rates the products automatically based on the reviews given by the buyers. To do this, a technique called Sentiment Analysis is used. It also ranks the products in order based on the product rating that is generated automatically. It would provide a way for the consumers to purchase their desired product within minutes. Surveys and interviews were conducted to find out the problems faced by consumers when purchasing a product online through e-commerce websites. There was also research conducted to study the product rating and product review section on the current e-commerce websites. To conclude, this automated product rating system eventually eases the consumers’ effort and time from reading numerous reviews and trusting inaccurate product rating to find a best quality product for them.


I. INTRODUCTION
As technology advances, everything is becoming more digital and automated.In this digitalized world, all of us now can buy anything on e-commerce websites and have it delivered without having to go to a store physically.There is almost nothing that we cannot find on e-commerce websites.However, judging a product solely based on its pictures and reviews is difficult and requires extensive research.Online buyers will always look for reviews of a product before buying to avoid purchasing low quality products.For them to look for the right product to be purchased, they must look for the products from different shops and compare those products.Reading numerous reviews and selecting the best product from a sea of similar items is a time-consuming process.Every review must undergo a thorough examination.The buyers had to scroll through and analyze thousands of reviews to buy a product online.Not only that, even when there is a rating provided for the product, it is not accurate enough as the buyer can just rate simply and not everyone knows the reason behind the rating that is given.Some reviews and ratings that are available in certain e-commerce websites are not even related.The reviews may be written in a positive way, but the rating given may be too low and vice versa.This makes the other buyers confused to look whether to refer to the reviews or the ratings.They would be frustrated in looking for that kind of unmatching reviews and ratings.Furthermore, the arrangement of the products is unstructured.Even though a product has many good reviews or ratings, it is shown at the bottom of the page.We are not sure how many people would scroll till the bottom to find a perfect product for them.They will only be going a certain mile to find the products and read the reviews.So, by the time the buyer does not reach the product that has a good review, they are losing the chances of buying the good ones.Thus, an advanced Sentiment Analysis for Product Rating system is developed to speed up finding the best products online using Sentiment Analysis technique.In addition, the system is also able to detect the hidden sentiments in the reviews and rate the product accordingly in which it will be done automatically by the system.Using the rating that is given for each review, an overall rating is assigned for each product.Based on the overall rating, it will rank the products from the highest to lowest product rating when displaying the products.By doing this, the true customer sentiment will be shown in the form of rating, and the correct and best product can be purchased by the consumer easily.To understand better, an effort was made to look for pertinent papers, research papers, reports, and documents that are related to the topic.

II. E-COMMERCE
Electronic commerce refers to the buying, selling, and exchanging of goods and services over the internet.ecommerce is a contemporary business that caters to the needs of organizations and the customers able to purchase products when the prices drop, when the product quality improves and the delivering services pace enhances [1].Moreover, Abdullah et al. [2] portrayed e-commerce as the use of Internet, computer and shared software technology to exchange the product details and visuals, offers and purchasing information and any other information that requires to be communicated to customers, suppliers or the society.Online store sales are roughly equivalent to the physical stores and this trend would continue without a stop [3].To add on to the previous point, Franco & Regi [4] claimed that the customers can easily compare the products that they wanted to be purchased www.ijacsa.thesai.orgwithout burdening themselves to drive from shop to shop.Armando & Alberto [5] mentioned that there was a massive shift in spending towards e-commerce.The shopping rate is also hugely elevated.However, the customers always wanted and always rely on the reviews or ratings that a product has before they decide to purchase the product.This is because when buying products online, the product cannot be touched or sensed physically, therefore the only way to make the customers trust and buy a product is the feedback or reviews given by the other customers who have bought the product earlier.Nellutla et al. [6] mentioned that a typical customer goes to the website, selects a product, inspects the prices and ratings, reads the reviews, and then proceeds with the transaction.However, e-commerce has its own set of issues.One of them is the rating of the products.Thus, we can tell that e-commerce is consistently growing and becoming more significant to businesses as technology constantly advances and it is something that needs to be taken advantage of and implemented.Customers are playing a vital role in purchasing products online as their honest reviews or the information that they are spreading are eventually influencing other customers who want to purchase a product online.

III. PRODUCT RATING AND REVIEW
Product rating and review are becoming more important in the customer decision making process nowadays.Buyers are relying on the reviews and ratings that are posted by the other buyers who have bought the product.As a result, the customer ratings and reviews have the power to significantly influence the sales of the product [7].According to Fawzy et al. [8], the product rating is critical in ascertaining whether a product or service is in a good quality as publicized and can be trusted.In addition, Fan & Fuel in [9] mentioned that 94% of customers read reviews before deciding to do any online purchasing.To add on, Wang & Chen [10] also stated that reviews in ecommerce websites are a valuable resource and play a vital role in purchasing a product.Customers decide either to proceed with the transaction or drop down based on the reviews in the e-commerce websites.In addition, Lackermair et al. [11] stated that 104 of customers in Germany reported that roughly 85% of the customers view and read all the product reviews very frequently before purchasing a product.To conclude, ratings and reviews aid the customers as they help them to get a better and clear idea of a product before buying it.Before deciding whether a product is worth purchasing, the customers analyze the reviews and ratings in the comment section and make themselves clear.Unfortunately, the product ratings are inaccurate to resemble the quality of the products as some of the customers simply put some random ratings without having a proper rationale behind it.

IV. SENTIMENT ANALYSIS TECHNIQUE
To solve the problems such as unmatched product ratings with the reviews and inaccurate product ratings, Natural Language Processing (NLP) was decided to use as it has the power of measuring sentiment from a text.NLP is a branch of machine learning (more specifically the branch of Artificial Intelligence) that is concerned with a computer's ability to comprehend, analyze, manipulate, and possibly generate human language [12].In NLP, there is a technique called Sentiment Analysis which suits well to achieve the objective of this topic.To use the Sentiment Analysis technique, some other NLP techniques or processes needed to be used so that the objective of Sentiment Analysis can be achieved (will be discussed in the upcoming section).Based on the research that is made, there are no other techniques that are used to analyze a review.The only technique that has been introduced to analyze the sentiment of a review is Sentiment Analysis.There are many reasons why Sentiment Analysis is decided to be used for analyzing the product reviews and generating product rating in accordance with the reviews.Those are described in the next subsection.

V. SENTIMENT ANALYSIS IN E-COMMERCE WEBSITE
Nowadays, people place a lot of importance on online shopping because it allows them to complete their purchases faster and with less effort.According to Jha et al. [13], Sentiment Analysis is used to ascertain what customers think of the product.This helps other customers in making decisions on buying a product.Textual reviews, star ratings, and emojis are all used to express opinions.Thus, Sentiment Analysis is used to examine the vast amounts of data that assist retailers or the buyers in meeting their objectives.Sentiment Analysis, a cognitive process for eliciting a user"s feelings and emotions.As the internet-based applications have increasingly evolved, it has resulted in a huge number of personalized reviews for many types of information on the Web.Sentiment Analysis is also known as a mighty tool for extracting relevant and needed information including aggregating the sentiments of the reviews for the users.It requires a training set for its performance and its quality is utmost important in evaluating a text accurately [14].Besides that, Jabreel et al. [15] stated that the main goal of Sentiment Analysis is to predict whether a text"s overall sentiment is positive, negative, or neutral.It has numerous variations.One method is to assign a rating scale from the reviews, such as 1 = "worst" to 5 = "best".According to Vyas & Vijayasundaram [16], big data from customer reviews, e-commerce sites and other sites are nearly impossible to manage.An automated system that calculates the overall tendency of belief and intensity to units like agencies, manufactured items, events, and their components is Sentiment Analysis.Sentiment Analysis is necessary for a greater understanding of the product.Furthermore, the simplest way to analyze the reviews is by calculating the feedback rating using Sentiment Analysis with word count.The rating can be predicted by the customer feedback.After receiving the sentiment analysis output, the customer can read all the feedback as fast and efficiently as possible in terms o [17].
Not only that, Haque et al. [18] mentioned that there are so many reviews that an effective method of analysis is needed.Customers must read through thousands of reviews manually to purchase a product in e-commerce.The volume of reviews is stored like a mountain which requires some effective classifier to identify valuable information from text.Sentiment Analysis can be useful in determining customer behavior by analyzing and examining customer reviews in e-commerce.Customers express their feelings by providing a subjective judgment about the products in e-commerce [19].www.ijacsa.thesai.orgAdditionally, Sentiment Analysis aids in categorizing the unstructured text as positive, negative, or neutral, which summarizes customer opinions and helps us better understand how other customers feel about a given product and retailer [20].To summarize, Sentiment Analysis plays an essential role in analyzing the hidden and true sentiment of customers based on the feedback or reviews given by them.

VI. COMPARISON ON EXISTING E-COMMERCE SYSTEM
A comparison on some existing e-commerce websites such as Shopee, Lazada and Amazon in some aspects of their product ratings are also made.Those are shown in Table I.

VII. METHOD
Interviews and surveys were conducted to find out the problems faced by consumers when purchasing a product online through e-commerce websites.

A. Interview
The interview was conducted with two frequent users of ecommerce websites who are X, regular user of Shopee and she is a student who"s running a small jewelry business and another interviewee is Y, active user of Lazada and she is a businesswoman who"s having her own boutique.The interviews were conducted by asking open-ended questions without providing any options.This is because answering open-ended questions will reveal the truth of interviewees" problems so that all their pains and challenges of purchasing products online can be understood well.By using qualitative method, the interviewees" problems were clear-cut.It creates an opportunity to understand the interviewees in depth.Not only that, but the behavior of the interviewees was also easily observed during the interviews.This was helpful to understand them especially their pains and gains because those were expressed through their expressions and their body languages.In addition, "WH" questions were asked to the interviewees to uncover deeper meaning and to identify the problems that the interviewees had but never realized its existence.Furthermore, it was effective enough to immerse in the interviewees" experience by purchasing products online through the e-commerce website together with them.The prepared questions were asked and there were some extra questions added based on the response given by the interviewees.The online interview sessions were conducted for 20 to 45 minutes.

B. Survey
Besides, a simple online short survey was conducted through Google form with a set of questionnaires.Since this issue involves e-commerce website users which are also known as consumers, the survey has been conducted among the people who actively use e-commerce websites to purchase their desired products.Based on the survey, there are a total of 20 respondents that have answered the questionnaire.

C. Sentiment Analysis Process
Since it involves the Sentiment Analysis technique, there are some processes that have performed so that it automatically analyzes the reviews for the sentiment of the consumers and rate according to it.Those processes are as below.

D. Data Collection
Dataset was collected from the Kaggle platform.The dataset is about clothing reviews with its product ratings and some relevant information.It is the raw data.And it is in the .csvformat.

E. Data Reading
After collecting the dataset from the Kaggle platform, the data has to be read for the upcoming process.Then, the number of rows, the number of columns and the number of data were read.For this project, the amount of data for every star-rating has been read and displayed which will be useful for the upcoming processes.

F. Data Cleaning
Data cleaning was done so that the data will be accurate, consistent and complete.Data cleaning or cleansing is the process of cleaning datasets by accounting for missing values, removing outliers, and smoothing noisy data (removing the meaningless data, renaming the meaningless column names and removing duplicated data) [21].At the end, only the product reviews column and product ratings column were retained.www.ijacsa.thesai.org

G. Data Preprocessing
Data preprocessing is the process of transforming raw data into a useful, understandable format.Raw data usually has inconsistent formatting, human errors, and can also be incomplete [22].In data preprocessing, firstly, tokenization process was implemented.Tokenization is a process that will break the raw text into small chunks which helps in understanding the context or developing the model for the NLP [23].It also helps in interpreting the meaning of the text by analyzing the sequence of the words.Then, special characters such as punctuations were eliminated as those characters are less important for training and testing models later on.Next, stop words were removed in which it removes the words that occur commonly across all the documents in the corpus.Typically, articles and pronouns are generally classified as stop words.Moreover, stemming process was also performed.It is also known as data filtering.It is a process of reducing a word to its word stem that affixes to suffixes and prefixes or to the roots of words known as a lemma.

H. Feature Extraction
Feature extraction is a process that will convert text into a matrix (or vector) of features.For this, we will be using a technique called TF-IDF technique in which it stands for term frequency-inverse document frequency.It highlights a specific issue which might not be too frequent in our corpus but holds great importance [24].The TF-IDF value increases proportionally to the number of times a word appears in the document and decreases with the number of documents in the corpus that contain the word.It is composed of two sub-parts, which are Term Frequency (TF) and Inverse Document Frequency (IDF).Term Frequency (TF) specifies how frequent a term appears in the entire document while the Inverse Document Frequency (IDF) is a measure of whether a term is rare or frequent across the documents in the entire corpus.The formula of TF-IDF is: TF-IDF (t, d) = TF (t, d) * IDF (t) (3) where, d refers to a document, N is the total number of documents, df is the number of documents with term t.TF-IDF is word frequency scores that highlight the words that are more interesting.The scores have the effect of highlighting words that are unique in a given document.

I. Data Balancing
Data balancing is used to balance the imbalanced data.There are two methods to be used to solve the imbalanced data.One is a random under sampling method, and another is a random oversampling method.Random under sampling method aims to randomly choose and eliminate samples from the majority class, thereby reducing the number of examples in the majority class in the transformed data whereas random oversampling involves choosing random examples from the minority class with replacement and adding multiple copies of this instance to the training data, so it's possible that a single instance will be chosen more than once [25].

J. Model Building (Sentiment Classification Analysis)
There are several Machine Learning based classification algorithms available using supervised and unsupervised learning approaches.The supervised methods make use of a large number of labeled training documents while the unsupervised methods are used when it is difficult to find these labeled training documents.For this proposed system, supervised learning method is chosen as it has a labeled dataset and a training process, therefore, several models were used such as Gaussian Naïve Bayes Classifier, Multinomial Naïve Bayes Classifier, Bernoulli Naïve Bayes Classifier, Random Forest Classifier, Logistic Regression Classifier, Decision Tree Classifier, Support Vector Classifier and Kneighbors Classifier.All these mentioned models were decided to use as they are capable of scalability.Using all these models, accuracy of each model was then identified.From there, the highest accuracy model was chosen and the dataset using the chosen model was trained.

K. Sentiment Analysis
With the highest accuracy model, Sentiment Analysis is then carried out to analyze the sentiments in each review and provide appropriate rating to it.

VIII. RESULTS AND DISCUSSION
The results of the interviews and survey are discussed in this section.

A. Interview
In the first interview, X mentioned that she uses Shopee because it is user-friendly.In Shopee, she normally browses the same products in distinct shops and compares the price and the quality of the products by looking at the reviews and ratings.She often does online shopping in Shoppe (twice a week) to buy things for herself and for her business.She said it takes almost two to three days to view all the product features and compare them with other sellers because the same product from other shops has different kinds of reviews.Before she decides to purchase a product, she normally goes through the product features, product rating and the product reviews.As for her, purchasing a product online consumes her time as it takes time to compare the products with different shops and choose the right product.She also mentioned that she will not purchase a product if she found the product has a huge number of negative reviews and even if it has an equal number of positive and negative reviews.Furthermore, she found some of the ratings are displayed as high but portrayed bad about the product when reading the reviews.She said that seeing and reading a ton of reviews are really time consuming, as a result, she will not be in the mood to shop most of the time.
In the second interview, Y said that she uses Lazada because it is easier for her to use, and it was recommended by her friend.In Lazada, she usually buys her desired products, reads reviews of her desired product before purchasing and sees the ratings of the product.Y mentioned that she does online shopping in Lazada once a week mainly to buy some www.ijacsa.thesai.orgnecessary items for her business.She said it takes almost a day to make a purchasing decision.This is because she always trusts the people"s views, thus, she will always read the reviews before purchasing anything and she also said that some of the products have a ton of reviews which is time consuming.Before she wants to buy anything, she usually looks for people's thoughts in the review section.As for her, purchasing a product online consumes more time because she must go through a lot of reviews.She also mentioned that if she found the product has a huge number of negative reviews and even it has an equal number of positive and negative reviews, she normally will not buy the product in that shop and will find another shop that sells the same product which has more positive reviews.In addition, trusting too much on the product reviews including the product rating before purchasing a product became a bigger problem for her because she often confused with the product reviews and ratings as they both will not be correlated and she does not know which one to trust, therefore, she will move to other shop to purchase the same product.Furthermore, she found that there are unmatching reviews and ratings of a product in which the rating is low for positive reviews and vice versa.She said that seeing and reading a ton of reviews are really time consuming.As a result, her excitement of buying the product will be lessened in that shop and will decide to shop in different shops.

B. Survey
There are roughly 15 questions in the survey, but the result of this survey will be displayed only for some questions that are important for achieving the objective of this project.Those are as below: Fig. 1 shows a bar chart of the e-commerce websites that are used by the respondents daily.Based on this bar chart, 18 of the respondents use Shoppe websites, 20 of the respondents use Lazada websites and 11 of them use Amazon websites.Since Shopee, Lazada and Amazon are the three highest ecommerce websites used by the respondents, that"s why a comparison of those three e-commerce websites was made in the previous section (see Table I).Fig. 2 shows a bar chart of the activities that are carried by the respondents in the e-commerce websites.It is clearly be seen that reading other users" reviews of a product and looking at the rating of a product have the highest counts which means the respondents always look for the reviews and ratings when they shop.Fig. 3 shows a pie chart on the time taken to make a purchasing decision while shopping online by the respondents.Around 40% of the respondents took more than five hours to decide while 25% of the respondents took more than 1 day to decide and followed by the rest.From here, it is obvious that spending more than five hours or one day is too much for a person to decide on a product to purchase.Even the respondents would not spend that much time when they shop physically.It would take only 10-15 minutes to buy physically.Tons of reviews to be read might be one of the reasons why most of the respondents spend too much of their time to make decision.Fig. 4 shows a bar chart on the criteria(s) that are considered by the respondents before they proceed with purchasing a product.From this chart, reading the product reviews and looking at the product ratings have the same count which is 19 and both are the highest among the other criteria.Thus, it is crystal clear that reviews and ratings are playing a major role when purchasing a product in ecommerce websites.Fig. 5 shows a pie chart on the time taken the respondents took to read all the related reviews of a product.60% of the respondents took more than one hour to read the reviews.www.ijacsa.thesai.orgThere are also some respondents who voted for 10 -30 mins and 30 mins -1 hour.This is mainly because of the number of reviews a product has.The more the reviews a product has, the more the time taken to read the reviews one by one.Fig. 6 shows a pie chart to know whether the respondents experience unmatched product reviews and ratings.All the respondents voted for "Yes" which means all of them experienced seeing unmatched product reviews together with manual product ratings.This is meant by, either they have seen a good review with low product rating or a bad review with a high product rating which both do not make sense at all.The reviews and ratings should be related with each other to avoid creating distrust among the other consumers who firstly visited a particular shop.Fig. 7 and Fig. 8 show pie charts on difficulty and time consumption in finding and purchasing a right product.All 20 respondents agreed that finding and buying the best product for them is really time consuming and very difficult.This is maybe because finding a product that has the highest product rating on an unordered products arrangement consumes more time and a mixture of positive and negative reviews of all the similar products from different shops make the consumers think in depth to proceed with the purchasing decision.

C. Sentiment Analysis Process
The result of each process mentioned in the previous section is discussed as below.

D. Data Collection
Clothing Reviews dataset was collected from Kaggle platform in .csvformat that has the columns named product review and product rating.

E. Data Reading
The number of rows, the number of columns and the number of data counts of every rating value were read using pandas library (python)

F. Data Cleaning
The null values, duplicated values and unwanted columns are removed to make the data accurate and consistent.Additionally, the column names were renamed with meaningful names.All these were done using the drop (columns = […]) method, rename (columns= [….]) method, dropna () method and drop_duplicates () method.

G. Data Pre-Processing
In this process, label encoder is used to encode the rating value of 1,2,3,4,5 to 0,1,2,3,4,5.Then, using the NLTK library, stop words are imported so that it can remove the stop words found in the dataset.All the unwanted characters like special characters were removed.And made the reviews all in small characters.Moreover, the review was also tokenized into small chunks.And finally, the suffixes and prefixes of a word in the review were removed and only holds the root word.

H. Feature Extraction
Feature extraction technique was used to find the frequency and the importance of the words that are present in the corpus.Here, the TF-IDF technique was implemented from the sklearn library by importing TfidfVectorizer from the feature extraction.textmodule.

IX. DATA BALANCING
Upon visualizing the product review with its rating in the dataset, it is found that the number of product reviews for all the product ratings (1-star to 5-star) is not balance as shown in Fig. 9. www.ijacsa.thesai.orgThis is known as imbalanced data.When the machine learning models are trained using this imbalanced data, the models will be well trained for only 5-star rating reviews as it holds a bigger amount of product reviews compared to the other star ratings.In this case, even a bad review could make the models to interpret the review as a good one and could create a good rating since the product rating of 1-star, 2-star, 3-star have a smaller amount of review data compared to 5star reviews and it will not be trained as how 5-star product review do.Thus, out of two methods that were mentioned in the previous section which are random oversampling and random under sampling, the random under sampling method is chosen because the random under sampling method gives the accurate result compared to the random oversampling method.This was identified by running both methods and observed the accuracy of the product rating that was generated by the system upon analyzing the reviews that were submitted in the review section.

A. Model Building (Sentiment Classification Analysis)
To find out which classifier suits the dataset the best, we implemented our dataset on multiple classification models namely Multinomial Naive Bayes Classifier, Bernoulli Naive Bayes Classifier, Gaussian Naive Bayes Classifier, Support Vector Classifier, K Nearest Neighbour Classifier, Decision Tree Classifier, Logistic Regression Classifier and Random Forest Classifier.The accuracy and time taken of each machine learning model were identified to train the model using each classifier that was mentioned, as shown in Fig. 10.In Fig. 10, it is clear that the Multinomial Naïve Bayes model holds the highest accuracy compared to other models.

B. Sentiment Analysis
The highest accuracy model, which is Multinomial Naïve Bayes model, was selected to analyze the sentiments on each review given and provide an appropriate rating to it.

X. CONCLUSION
It is true that reading numerous reviews, finding a right product for a longer time, and having unmatched reviews and ratings are the top issues among the consumers and there is a dire need to solve them.These are the key reasons to have an automated product rating system and different from the existing e-commerce websites.It uses Sentiment Analysis technique as it has an ability to analyze the hidden sentiment of the consumers in the review section.In addition, this system can also rank the products based on the rating in which it will reduce the time taken for the consumers to scroll the page down till they find the right one for them.On top of that, the unmatched reviews and ratings can be avoided as the rating will be given automatically based on the reviews, hence helps the consumers shorten their time searching for the best products within minutes.

FUTURE WORKS
A fundamental challenge in e-commerce research comprehends the idea of Artificial Intelligence (AI).According to our research, e-commerce product ratings based on customer reviews have primarily been taken into consideration by researchers.Researchers can contribute AI to e-commerce research in the future.This will assist in precisely analyzing data and forecasting e-commerce activity through the use of AI platform algorithms.Having this clarification would help avoid misunderstandings between AI and business analytics and intelligence in e-commerce.Additionally, it would make it easier to distinguish between AI as a social actor and AI as a computing technology capable of cognitive tasks.Context is a second fundamental problem with AI research in e-commerce.Based on the output of an ecommerce system that is relevant in the real world, an AI system would be able to interpret the message that the user communicated or sought using the same data.Hence, rather than making broad assertions about product prices, researchers would need to work with practitioners to better understand and define contexts of inquiry.Lastly, to provide more information about the methodology, such as sample size, demographics, and questions asked during interviews and surveys.

Fig. 3 .
Fig. 3. Time taken to make a purchasing decision.

Fig. 4 .
Fig. 4. Criteria(s) that are considered before purchasing a product.

Fig. 5 .
Fig. 5. Time taken to read all the reviews.

Fig. 6 .
Fig. 6.Experience of seeing unmatched product reviews and ratings.

Fig. 8 .
Fig. 8. Time consumption in finding and purchasing a product.

Fig. 9 .
Fig. 9. Pie chart of the number of product reviews for all the product ratings in Clothing review dataset.

Fig. 10 .
Fig. 10.Accuracy and time taken of each classifier.