Sentiment Analysis of the Mobile Legends: Bang Bang Application Using a Hybrid CNN-LSTM Model

The increasing number of user reviews on the Google Play Store is a challenge in understanding user opinions and experiences with apps. One of the most discussed apps is Mobile Legends: Bang Bang (MLBB), a popular game with millions of downloads and reviews from Indonesian users. The problem faced is the limitation of conventional sentiment analysis models in understanding sentences and context simultaneously, making it less than optimal in analyzing user sentiment. This study proposed a comprehensive sentiment analysis system for MLBB application reviews, utilizing a hybrid CNN-LSTM architecture with a systematic optimization approach. A dataset comprising 30,000 balanced Indonesian user reviews was extracted from the Google Play Store using web scraping techniques and then processed through an extensive pre-processing pipeline, which included data cleaning, case folding, stopword removal, and stemming. Five experimental scenarios were conducted to optimize model performance through feature engineering and algorithmic enhancement. The baseline CNN-LSTM model achieved 71.97% accuracy, which was progressively improved through TF-IDF vectorization with optimal N-gram (1,2) configuration, max features optimization reaching 10,000 features, FastText embedding feature expansion using a 300-dimensional Indonesian pre-trained model, and optimizer selection experiments across five algorithms. The final optimized hybrid CNN-LSTM model, using the RMSprop, demonstrated a breakthrough performance of 88.84% accuracy with remarkable consistency (standard deviation of 0.000754), representing a 23.4% improvement over the baseline. This research contributes to the field of sentiment analysis, especially for game applications, by proving that a combined approach can produce a more accurate and reliable system for understanding user opinions.

Keywords: Classification, FastText , Hybrid CNN-LSTM, Sentiment Analysis, TF-IDF

[1] V. H. Pranatawijaya, N. N. K. Sari, R. A. Rahman, E. Christian, and S. Geges, “Unveiling user sentiment: Aspect-Based analysis and topic modeling of Ride-Hailing and Google Play app reviews,” Journal of Information Systems Engineering and Business Intelligence, vol. 10, no. 3, pp. 328–339, Oct. 2024, doi: 10.20473/jisebi.10.3.328-339.

[2] P. H. C. Samanmali and R. a. H. M. Rupasingha, “Sentiment analysis on google play store app users’ reviews based on deep learning approach,” Multimedia Tools and Applications, vol. 83, no. 36, pp. 84425–84453, Apr. 2024, doi: 10.1007/s11042-024-19185-w.

[3] A. S. Chan, F. Fachrizal, and A. R. Lubis, “Outcome prediction using Naïve Bayes algorithm in the selection of role Hero Mobile Legend,” Journal of Physics Conference Series, vol. 1566, no. 1, p. 012041, Jun. 2020, doi: 10.1088/1742-6596/1566/1/012041.

[4] Y. Chandra and A. Jana, “Sentiment Analysis using Machine Learning and Deep Learning,” 2022 9th International Conference on Computing for Sustainable Global Development (INDIACom), Mar. 2020, doi: 10.23919/indiacom49435.2020.9083703.

[5] A. Yadav and D. K. Vishwakarma, “Sentiment analysis using deep learning architectures: a review,” Artificial Intelligence Review, vol. 53, no. 6, pp. 4335–4385, Dec. 2019, doi: 10.1007/s10462-019-09794-5.

[6] M. Sivakumar and S. R. Uyyala, “Aspect-based sentiment analysis of mobile phone reviews using LSTM and fuzzy logic,” International Journal of Data Science and Analytics, vol. 12, no. 4, pp. 355–367, 2021, doi: 10.1007/s41060-021-00277-x.

[7] S. A. Thampy and J. J. R. Angelina, “Deep Learning Architectures Based Sentiment Analysis Systematic Literature Review,” 2023 International Conference on Control, Communication and Computing (ICCC), May 2023, doi: 10.1109/iccc57789.2023.10164943.

[8] N. M. A. Raihan and N. E. B. Setiawan, “Aspect Based Sentiment Analysis with FastText Feature Expansion and Support Vector Machine Method on Twitter,” Jurnal RESTI (Rekayasa Sistem Dan Teknologi Informasi), vol. 6, no. 4, pp. 591–598, Aug. 2022, doi: 10.29207/resti. v6i4.4187.

[9] H. Mohaouchane, A. Mourhir and N. S. Nikolov, "Detecting Offensive Language on Arabic Social Media Using Deep Learning," 2019 Sixth International Conference on Social Networks Analysis, Management and Security (SNAMS), 2019, pp. 466-471, doi: 10.1109/SNAMS.2019.8931839.

[10] N. Kalchbrenner, E. Grefenstette, and P. Blunsom, “A Convolutional Neural Network for Modelling Sentences,” In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Jan. 2014, doi: 10.3115/v1/p14-1062.

[11] S. Minaee, N. Kalchbrenner, E. Cambria, N. Nikzad, M. Chenaghlu, and J. Gao, “Deep learning--based text classification,” ACM Computing Surveys, vol. 54, no. 3, pp. 1–40, Apr. 2021, doi: 10.1145/3439726.

[12] S. Santhanam, “Context based Text-generation using LSTM networks,” arXiv, Apr. 30, 2020. https://arxiv.org/abs/2005.00048.

[13] A. H. Ombabi, W. Ouarda, and A. M. Alimi, “Improving Arabic sentiment analysis across context-aware attention deep model based on natural language processing,” Language Resources and Evaluation, Apr. 2024, doi: 10.1007/s10579-024-09741-z.

[14] A. U. Rehman, A. K. Malik, B. Raza, and W. Ali, “A hybrid CNN-LSTM model for improving accuracy of movie reviews sentiment analysis,” Multimedia Tools and Applications, vol. 78, no. 18, pp. 26597–26613, Jun. 2019, doi: 10.1007/s11042-019-07788-7.

[15] N. Hossain, Md. R. Bhuiyan, Z. N. Tumpa, and S. A. Hossain, “Sentiment Analysis of Restaurant Reviews using Combined CNN-LSTM,” 2022 13th International Conference on Computing Communication and Networking Technologies (ICCCNT), pp. 1–5, Jul. 2020, doi: 10.1109/icccnt49239.2020.9225328.

[16] B. M. G. A. Awienoor, and E. Setiawan, “Movie Recommendation System Based on Tweets Using Switching Hybrid Filtering with Recurrent Neural Network,” International Journal of Intelligent Engineering and Systems, vol. 17, no. 2, pp. 277–293, Feb. 2024, doi: 10.22266/ijies2024.0430.24.

[17] F. Alharbi, L. Ouarbya, and J. A. Ward, “Comparing sampling strategies for tackling imbalanced data in human activity recognition,” Sensors, vol. 22, no. 4, p. 1373, Feb. 2022, doi: 10.3390/s22041373.

[18] F. Rifaldy, Y. Sibaroni, and S. S. Prasetiyowati, “Effectiveness of Word2VEC and TF-IDF in sentiment classification on online investment platforms using support Vector Machine,” JIPI (Jurnal Ilmiah Penelitian Dan Pembelajaran Informatika), vol. 10, no. 2, pp. 863–874, Mar. 2025, doi: 10.29100/jipi. v10i2.6055.

[19] A. I. Kadhim, “Term weighting for feature extraction on Twitter: A comparison between BM25 and TF-IDF,” 2019 International Conference on Advanced Science and Engineering (ICOASE), pp. 124–128, Apr. 2019, doi: 10.1109/icoase.2019.8723825.

[20] N. I. R. Illahi and E. B. Setiawan, “Sentiment Analysis on Social Media Using Fasttext Feature Expansion and Recurrent Neural Network (RNN) with Genetic Algorithm Optimization,” International Journal on Information and Communication Technology (IJoICT), vol. 10, no. 1, pp. 78–89, Jun. 2024, doi: 10.21108/ijoict. v10i1.905.

[21] N. Nedjah, I. Santos, and L. De Macedo Mourelle, “Sentiment analysis using convolutional neural network via word embeddings,” Evolutionary Intelligence, vol. 15, no. 4, pp. 2295–2319, Apr. 2019, doi: 10.1007/s12065-019-00227-4.

[22] M. Z. F. Rizky, Y. Sibaroni, and S. S. Prasetiyowati, “Sentiment Analysis on TikTok App using Long Short-Term Memory (LSTM) with Stochastic Gradient Descent (SGD) Optimization,” JURNAL MEDIA INFORMATIKA BUDIDARMA, vol. 8, no. 3, p. 1292, Jul. 2024, doi: 10.30865/mib. v8i3.7699.

[23] N. Uly, H. Hendry, and A. Iriani, “CNN-RNN hybrid model for diagnosis of COVID-19 on X-Ray imagery,” Digital Zone Jurnal Teknologi Informasi Dan Komunikasi, vol. 14, no. 1, pp. 57–67, May 2023, doi: 10.31849/digitalzone. v14i1.13668.

[24] A. Ishaq, S. Asghar, and S. A. Gillani, “Aspect-Based sentiment analysis using a hybridized approach based on CNN and GA,” IEEE Access, vol. 8, pp. 135499–135512, Jan. 2020, doi: 10.1109/access.2020.3011802.

[25] B. Liu, “Text sentiment analysis based on CBOW model and deep learning in big data environment,” Journal of Ambient Intelligence and Humanized Computing, vol. 11, no. 2, pp. 451–458, Oct. 2018, doi: 10.1007/s12652-018-1095-6.

[26] M. Ihsan, N. B. S. Negara, and N. S. Agustian, “LSTM (Long Short-Term Memory) for Sentiment COVID-19 vaccine classification on Twitter,” Digital Zone Jurnal Teknologi Informasi Dan Komunikasi, vol. 13, no. 1, pp. 79–89, May 2022, doi: 10.31849/digitalzone. v13i1.9950.

[27] N. Dr. G. S. N. Murthy, N. S. R. Allu, N. B. Andhavarapu, and N. M. B. M. Belusonti, “Text based Sentiment Analysis using LSTM,” International Journal of Engineering Research, vol. V9, no. 05, May 2020, doi: 10.17577/ijertv9is050290.

[28] C. Sahoo, M. Wankhade, and B. K. Singh, “Sentiment analysis using deep learning techniques: a comprehensive review,” International Journal of Multimedia Information Retrieval, vol. 12, no. 2, Nov. 2023, doi: 10.1007/s13735-023-00308-2.

[29] J. Wang, L.-C. Yu, K. R. Lai, and X. Zhang, “Tree-Structured Regional CNN-LSTM Model for dimensional Sentiment Analysis,” IEEE/ACM Transactions on Audio Speech and Language Processing, vol. 28, pp. 581–591, Dec. 2019, doi: 10.1109/taslp.2019.2959251.

[30] W. Li, L. Zhu, Y. Shi, K. Guo, and E. Cambria, “User reviews: Sentiment analysis using lexicon integrated two-channel CNN–LSTM family models,” Applied Soft Computing, vol. 94, p. 106435, Jun. 2020, doi: 10.1016/j.asoc.2020.106435.

[31] A. B. Syahputri and Y. Sibaroni, “Comparative Analysis of CNN and LSTM Performance for Hate Speech Detection on Twitter,” 2023 11th International Conference on Information and Communication Technology (ICoICT), pp. 190–195, Aug. 2023, doi: 10.1109/icoict58202.2023.10262656.

[32] A. Bekkar, B. Hssina, S. Douzi, and K. Douzi, “Air-pollution prediction in smart city, deep learning approach,” Journal of Big Data, vol. 8, no. 1, Dec. 2021, doi: 10.1186/s40537-021-00548-1.

[33] A. U. Rehman, A. K. Malik, B. Raza, and W. Ali, “A hybrid CNN-LSTM model for improving accuracy of movie reviews sentiment analysis,” Multimedia Tools and Applications, vol. 78, no. 18, pp. 26597–26613, Jun. 2019, doi: 10.1007/s11042-019-07788-7.

[34] S. H. Eom, “Developing sentimental analysis system based on various optimizer,” The International Journal of Internet, Broadcasting and Communication, vol. 13, no. 1, pp. 100–106, Feb. 2021, doi: 10.7236/ijibc.2021.13.1.100.

[35] N. Fatima, “Enhancing performance of a deep neural network: A comparative analysis of optimization Algorithms,” ADCAIJ ADVANCES IN DISTRIBUTED COMPUTING AND ARTIFICIAL INTELLIGENCE JOURNAL, vol. 9, no. 2, pp. 79–90, Jun. 2020, doi: 10.14201/adcaij2020927990.

[36] T.-T. Wong and P.-Y. Yeh, “Reliable Accuracy Estimates from k-Fold Cross Validation,” IEEE Transactions on Knowledge and Data Engineering, vol. 32, no. 8, pp. 1586–1594, Apr. 2019, doi: 10.1109/tkde.2019.2912815.

[37] O. Chamorro-Atalaya et al., “K-Fold Cross-Validation through Identification of the Opinion Classification Algorithm for the Satisfaction of University Students,” International Journal of Online and Biomedical Engineering (iJOE), vol. 19, no. 11, Aug. 2023, doi: 10.3991/ijoe. v19i11.39887.