Analysis of User Reviews for The Mytelkomsel App Using Naïve Bayes and Random Forest Methods

Abstract

While sentiment analysis of local application reviews predominantly utilizes native Indonesian data, these datasets frequently suffer from colloquial ambiguities and informal structures that degrade classifier performance. This study addresses this gap by implementing a language-filtering mechanism to separate and analyze English and Indonesian user opinions from the MyTelkomsel application, specifically justifying the inclusion of English reviews due to their superior grammatical structure and syntactic consistency, which inherently enhances feature extraction. A systematic methodology was employed, encompassing data collection from the Google Play Store, comprehensive pre-processing (case folding, tokenization, stopword removal, and stemming), and Term Frequency-Inverse Document Frequency (TF-IDF) vectorization. Evaluated using Naïve Bayes and Random Forest algorithms on 25,000 customer feedbacks, the models were compared across accuracy, precision, recall, and F1-score. The empirical results demonstrated that Random Forest outperformed Naïve Bayes, achieving a higher accuracy of 86.85% compared to 86.36%. This superiority stems from Random Forest’s robust capability to mitigate class imbalance and minimize error distribution across sentiment categories. Ultimately, this approach provides precise, actionable insights into service quality, enabling Telkomsel to effectively distinguish user satisfaction, target operational improvements, and mitigate customer churn.