Customer Satisfaction Assessment System on Transactions E-commerce Product Purchases Using Sentiment Analysis

Amil Ahmad Ilham, Anugrayani Bustamin, Eugenius Wahyudiarto

Abstract


Currently, more products appear, and various services that offer similar products make it difficult for buyers to decide to buy before seeing reviews from other users. The growth of different e-commerce platforms exacerbates this. Users spent more time choosing products on each platform with many alternative considerations, such as looking at ratings, prices, and reviews from other buyers. This study conducted the optimization process of selecting e-commerce products so that users do not have to spend a long time reading every review when they want to buy a product. This research is expected to provide a comprehensive assessment of the purchase transaction of a product from the reviews provided. The data is sourced from product reviews on e-commerce in Indonesia, which are then classified into positive, negative, and neutral sentiments. The data is divided into 10 folds of data using stratified k-fold cross-validation, consisting of training and testing data with ratios of 90% and 10% of the total data. Our research proposed a system that implemented our modified Naive Bayes model to calculate a product's Customer Satisfaction (CSAT) score and compare it with the Google Cloud NLP model. In our model, the log prior and log-likelihood formulas are modified in the algorithm, adding the prefix "NOT_" after the negation words in the preprocessing. This doubled our model’s F1 score and increased the accuracy by 32%, from 59% to 91%, when compared to the Naive Bayes algorithm without modification.

Keywords


Customer satisfaction score; product purchase recommendation; sentiment analysis; Naive Bayes

Full Text:

PDF

References


Z. Shahbazi and Y.-C. Byun, “Product Recommendation Based on Content-based Filtering Using XGBoost Classifier,†Int. J. Adv. Sci. Technol., vol. 29, no. 4, pp. 6979-6988, 2020.

G. Khanvilkar and Prof. Deepali Vora, “Sentiment Analysis for Product Recommendation Using Random Forest,†Int. J. Eng. Technol., vol. 7, no. 3.3, pp. 87-89, Jun. 2018, doi: 10.14419/ijet.v7i3.3.14492.

H. Parveen and S. Pandey, “Sentiment analysis on Twitter Data-set using Naive Bayes algorithm,†in 2016 2nd International Conference on Applied and Theoretical Computing and Communication Technology (iCATccT), Bangalore, India, 2016, pp. 416–419. doi: 10.1109/ICATCCT.2016.7912034.

R. Alatrash, R. Priyadarshini, H. Ezaldeen, and A. Alhinnawi, “Augmented language model with deep learning adaptation on sentiment analysis for E-learning recommendation,†Cognitive System Research, vol. 75, pp. 53-69, Sep. 2022, doi.org/10.1016/j.cogsys.2022.07.002.

M. Bibi, W. A. Abbasi, A. Aziz, S. Khalil, M. Uddin, C.Iwendi, and T. R. Gadekallu, “A novel unsupervised ensemble framework using concept-based linguistic methods and machine learning for twitter sentiment analysis,†Pattern Recognition Letters, vol. 158, pp. 80-86, June 2022, doi.org/10.1016/j.patrec.2022.04.004

H. Deng, D. Ergu, F. Liu, Y. Cai and B. Ma, “Text sentiment analysis of fusion model based on attention mechanism,†Procedia Computer Science, vol 199, pp. 741-748, 2022, doi.org/10.1016/j.procs.2022.01.092

A.R. Rahmanti, C.Chien, A.A.Nursetyo, A. Husnayain, B. S. Wiratama, A. Fuad, H- Yang and Y.J. Li, “Social media sentiment analysis to monitor the performance of vaccination coverage during the early phase of the national COVID-19 vaccine rollout,†Computer Methods and Programs in Biomedicine, vol 221, June 2022, doi.org/10.1016/j.cmpb.2022.106838

I. N. K. Bayu, I. M. A. Suarjaya and P. W. Buana, "Classification of Indonesian Population's Level Happiness on Twitter Data Using N-Gram, Naïve Bayes, and Big Data Technology," Int. J on Adv Scie Eng Inf Tech, vol. 12, no. 5, pp. 1944-1949, 2022

S. Farzadnia and I. R. Vanani, “Identification of opinion trends using sentiment analysis of airlines passengers' reviews,†Journal of Air Transport Management, vol. 103, August 2022, doi.org/10.1016/j.jairtraman.2022.102232

G. Hanane and N. El Habib, “Semisupervised neural biomedical sense disambiguation approach for aspect-based sentiment analysis on social networks,†Journal of Biomedical Inf, vol. 135, 2022, doi: 10.1016/j.jbi.2022.104229.

L. Chucu, F. Fan, L. Xu, C. Tie, T. Xu, L. Jianguo, and L. Xin, “Improving sentiment analysis accuracy with emoji embedding,†Journal of Safety Scie and Resilience, vol. 2, no. 4, pp. 246-252, 2021.

A. Shaha, A. Allulo, A. Asma, A. Amany, A. Sara, A. Nada & A. Aisha, "Customer Satisfaction Measurement using Sentiment Analysis." International Journal of Advanced Computer Science and Applications, vol. 9, no 10, 2018, doi:14569/IJACSA.2018.090216.

L. R. Shreyas, "Sentiment Analysis of Customer Satisfaction using Deep Learning," International Research Journal of Computer Science, vol.6, no.12, 2019.

A. Nair, C. Paralkar, J. Pandya, Y. Chopra, and D. Krishnan, “Comparative Review on Sentiment analysis-based Recommendation system,†in 6th Int. Conf. Converg. Technol. I2CT 2021, pp. 2–7, 2021, doi: 10.1109/I2CT51068.2021.9418222.

T. Hariguna and V. Rachmawati, “Community Opinion Sentiment Analysis on Social Media Using Naive Bayes Algorithm Methods,†IJIIS Int. J. Informatics Inf. Syst., vol. 2, no. 1, pp. 33–38, 2019, doi: 10.47738/ijiis.v2i1.11.

E. Asani, H. Vahdat-Nejad, and J. Sadri, “Restaurant recommender system based on sentiment analysis,†Mach. Learn. with Appl., vol. 6, no. July, p. 100114, 2021, doi: 10.1016/j.mlwa.2021.100114.

M. D. Prasetio, R. Y. Xavier, H. Rachmat, W. Wiyono, and D. S. E. Atmaja, “Sentiment analysis on myindihome user reviews using support vector machine and naïve bayes classifier method,†Int. J. Ind. Optim., vol. 2, no. 2, p. 141, 2021, doi: 10.12928/ijio.v2i2.4449.

M. Ahmad, M. Octaviansyah, A. Kardiana, and K. Prasetyo, "Sentiment Analysis System of Indonesian Tweets using Lexicon and Naïve Bayes Approach." in Fifth International Conf. on Informatics and Computing, pp.1-4, 2020.

J. Kim, D. Hwang, and H. Jung, “Product recommendation system based user purchase criteria and product reviews,†Int. J. Electr. Comput. Eng. IJECE, vol. 9, no. 6, p. 5454, Dec. 2019, doi: 10.11591/ijece.v9i6.pp5454-5462.

A. R. Hanni, M. M. Patil, and P. M. Patil, “Summarization of customer reviews for a product on a website using natural language processing,†in 2016 International Conference on Advances in Computing, Communications and Informatics (ICACCI), Jaipur, India, Sep. 2016, pp. 2280–2285. doi: 10.1109/ICACCI.2016.7732392.

P. Thota and E. Ramez, “Web Scraping of COVID-19 News Stories to Create Datasets for Sentiment and Emotion Analysis,†in the 14th PErvasive Technologies Related to Assistive Environments Conference, New York, NY, USA, Jun. 2021, pp. 306–314. doi: 10.1145/3453892.3461333.

L. Irshad, L. Yan, and Z. Ma, “Schema-Based JSON Data Stores in Relational Databases,†J. Database Manag. JDM, vol. 30, no. 3, pp. 38–70, Jul. 2019, doi: 10.4018/JDM.2019070103.

H.-T. Duong and T.-A. Nguyen-Thi, “A review: preprocessing techniques and data augmentation for sentiment analysis,†Comput. Soc. Netw., vol. 8, no. 1, p. 1, Jan. 2021, doi: 10.1186/s40649-020-00080-x.

F. Alzazah, X. Cheng, and X. Gao, “Predict Market Movements Based on the Sentiment of Financial Video News Sites,†in 2022 IEEE 16th International Conference on Semantic Computing (ICSC), Jan. 2022, pp. 103–110. doi: 10.1109/ICSC52841.2022.00022.

M. Wongkar and A. Angdresey, “Sentiment Analysis Using Naive Bayes Algorithm Of The Data Crawler: Twitter,†in 2019 Fourth International Conference on Informatics and Computing (ICIC), Oct. 2019, pp. 1–5. doi: 10.1109/ICIC47613.2019.8985884.

N. H. Khun and H. A. Thant, “Visualization of Twitter Sentiment during the Period of US Banned Huawei,†in 2019 International Conference on Advanced Information Technologies (ICAIT), Nov. 2019, pp. 274–279. doi: 10.1109/AITC.2019.8921014.

Z. Yang, S. Vijlbrief, and N. Okazaki, “TokyoTech_NLP at SemEval-2019 Task 3: Emotion-related Symbols in Emotion Detection,†in Proceedings of the 13th International Workshop on Semantic Evaluation, Minneapolis, Minnesota, USA, Jun. 2019, pp. 350–354. doi: 10.18653/v1/S19-2061.

H. M. Keerthi Kumar and B. S. Harish, “Classification of Short Text Using Various Preprocessing Techniques: An Empirical Evaluation,†in Recent Findings in Intelligent Computing Techniques, Singapore, 2018, pp. 19–30. doi: 10.1007/978-981-10-8633-5_3.

H.-H. CHEN, “The Contextual Analysis of Chinese Sentences with Punctuation Marks,†Lit. Linguist. Comput., vol. 9, no. 4, pp. 281–289, Jan. 1994, doi: 10.1093/llc/9.4.281.

M. Nabil, M. Aly, and A. Atiya, “LABR: A Large-Scale Arabic Sentiment Analysis Benchmark,†Computing and Language, May 2015. doi: 10.48550/arXiv.1411.6718.

D. Jurafsky and J. H. Martin, Speech and Language Processing, 3d Edition draft. web.stanford.edu, 2020. Accessed: Aug. 02, 2021. [Online]. Available: https://web.stanford.edu/~jurafsky/slp3/

G. Forman and M. Scholz, “Apples-to-apples in cross-validation studies: pitfalls in classifier performance measurement,†ACM SIGKDD Explor. Newsl., vol. 12, no. 1, pp. 49–57, Nov. 2010, doi: 10.1145/1882471.1882479.

Y. Huang and L. Li, “Naive Bayes classification algorithm based on small sample set,†in 2011 IEEE International Conference on Cloud Computing and Intelligence Systems, Sep. 2011, pp. 34–39. doi: 10.1109/CCIS.2011.6045027.




DOI: http://dx.doi.org/10.18517/ijaseit.13.3.18273

Refbacks

  • There are currently no refbacks.


Published by INSIGHT - Indonesian Society for Knowledge and Human Development