Comparison of Deep Learning Models for Sentiment Analysis of IPOT Financial App Reviews Using Convolutional Neural Network (CNN) and IndoBERT

I Gusti Ngurah Agung Pawana; Made Widya Jayantari; I Gusti Ngurah Agung Bagus Aryawana

doi:10.55606/jitek.v6i1.10901

Authors

I Gusti Ngurah Agung Pawana Warmadewa University
Made Widya Jayantari Udayana University
I Gusti Ngurah Agung Bagus Aryawana Institute of Technology and Business STIKOM

DOI:

https://doi.org/10.55606/jitek.v6i1.10901

Keywords:

Sentiment Analysis, Deep Learning, CNN, IndoBERT, Financial App Reviews

Abstract

The rapid expansion of mobile-based financial applications has generated a large volume of user reviews that contain valuable insights into user satisfaction and system performance. IPOT (Indo Premier Online Trading) is a widely used financial application in Indonesia, making sentiment analysis of its user reviews essential for evaluating its service and improving it. This study applies an experimental methodology to compare the performance of two deep learning architectures, Convolutional Neural Network (CNN) and IndoBERT, for sentiment analysis of financial application reviews. User review data were collected from the Google Play Store. Sentiment labels were automatically assigned based on user ratings, and the dataset was balanced using stratified sampling to obtain 15,000 reviews. Text preprocessing included case folding, removal of punctuation and special characters, tokenization, stopword removal, and stemming. The dataset was then split into training, validation, and testing sets, with oversampling applied only to the training data to prevent data leakage. The comparison between Convolutional Neural Networks (CNNs) and IndoBERT for sentiment analysis of IPOT financial application reviews shows that both models perform sentiment classification effectively, with different strengths across sentiment categories. The CNN model achieved higher overall accuracy (0.8113) compared to IndoBERT (0.7880), indicating strong performance in detecting dominant sentiment patterns, particularly positive sentiment. Meanwhile, IndoBERT achieved superior performance in negative and neutral sentiment classification, as evidenced by higher recall and F1 scores. The confusion matrix and error analysis results further indicate that IndoBERT is more effective at understanding contextual and nuanced language, whereas CNN is more sensitive to explicit lexical sentiment indicators.

References

[1] M. Singh, “The Future of Investing : Analysing the Influence of Trading Applications on Investors Trends in Demat Accounts in 2022 ( in mn ),” vol. 07, no. 04, pp. 1755–1762, 2024, doi: 10.47191/ijmra/v7-i04-38.

[2] D. Das, R. Shenoy, and L. D, “The Role of Fintech in Increasing Retail Participation in the Indian Stock Market,” INTERNATIONAL J. Sci. Res. Eng. Manag., vol. 08, pp. 1–7, Nov. 2024, doi: 10.55041/IJSREM38135.

[3] N. Kurniasari and W. Wibowo, Sentiment analysis of IPOT application reviews using naïve Bayes method, vol. 2668. 2022. doi: 10.1063/5.0116704.

[4] R. Fuad Armansyah, “A Study Of Investor Financial Behavior on Online Trading System in Indonesian Stock Exchange: E-Satisfaction, E-Loyalty, And E-Trust,” J. Econ. Business, Account. Ventur., vol. 23, pp. 69–84, Jul. 2020, doi: 10.14414/jebav.v23i1.2176.

[5] A. Yasin, R. Fatima, A. N. Ghazi, and Z. Wei, “Python data odyssey: Mining user feedback from Google Play Store,” Data Br., vol. 54, p. 110499, 2024, doi: https://doi.org/10.1016/j.dib.2024.110499.

[6] S. F. Fahim, S. A. Sounok, N. Shaeed, M. H. Orpa, and N. T. Niloy, “Analyzing User Sentiment in Google Play Store Reviews: A Natural Language Processing Approach,” in 2024 15th International Conference on Computing Communication and Networking Technologies (ICCCNT), 2024, pp. 1–5. doi: 10.1109/ICCCNT61001.2024.10725684.

[7] C. Zucco, B. Calabrese, G. Agapito, P. Guzzi, and M. Cannataro, “Sentiment analysis for mining texts and social networks data: Methods and tools,” Wiley Interdiscip. Rev. Data Min. Knowl. Discov., vol. 10, Aug. 2019, doi: 10.1002/widm.1333.

[8] F. Khanum, P. S. Lakshmi, and H. V. R. K, “Sentiment Analysis Using Natural Language Processing, Machine Learning and Deep Learning,” in 2024 5th International Conference on Circuits, Control, Communication and Computing (I4C), 2024, pp. 113–118. doi: 10.1109/I4C62240.2024.10748425.

[9] K. Taghandiki and E. Ehsan, Types of Approaches, Applications, and Challenges in the Development of Sentiment Analysis Systems. 2023. doi: 10.48550/arXiv.2303.11176.

[10] S. Awasthi, “A Comprehensive Analysis of the Current Methods, Challenges and Innovations in Sentiment Analysis,” J. Inf. Syst. Eng. Manag., vol. 10, pp. 382–401, Mar. 2025, doi: 10.52783/jisem.v10i23s.3712.

[11] S. Redhu, “Sentiment Analysis Using Text Mining: A Review,” Int. J. Data Sci. Technol., vol. 4, p. 49, Jun. 2018, doi: 10.11648/j.ijdst.20180402.12.

[12] T. Purnomo and J. Sutopo, “Comparison of Pre-Trained BERT-Based Transformer Models for Regional Language Text Sentiment Analysis in Indonesia,” Int. J. Sci. Technol., vol. 3, pp. 11–21, Nov. 2024, doi: 10.56127/ijst.v3i3.1739.

[13] F. Wang, “Comparative Evaluation of Sentiment Analysis Methods : From Traditional Techniques to Advanced Deep Learning Models,” vol. 0, pp. 23–29, 2024, doi: 10.54254/2755-2721/105/2024TJ0056.

[14] P. Sharma, P. Poonam, and A. Yadav, “TRANSFORMER SVS.CNN SIN MEDICAL IMAGING: A COMPARATIVE REVIEW,” Int. J. Adv. Res., vol. 13, pp. 8–11, Jul. 2025, doi: 10.21474/IJAR01/21300.

[15] Z. Zhu, “BERT and Its Applications in Natural Language Understanding,” Appl. Comput. Eng., vol. 175, pp. 99–105, Aug. 2025, doi: 10.54254/2755-2721/2025.AST26090.

[16] V. Balakrishnan, Z. Shi, C. L. Law, R. Lim, L. L. Teh, and Y. Fan, “A deep learning approach in predicting products’ sentiment ratings: a comparative analysis,” J. Supercomput., vol. 78, no. 5, pp. 7206–7226, 2022, doi: 10.1007/s11227-021-04169-6.

[17] C. Lal and Z. Nasir, “Comparative Analysis of Deep Learning Methods in the Realm of Sentiment Analysis,” in 2023 International Multi-disciplinary Conference in Emerging Research Trends (IMCERT), 2023, pp. 1–3. doi: 10.1109/IMCERT57083.2023.10075107.

[18] E. Subowo, “Implementasi Pembelajaran Mendalam dalam Klasifikasi Sentimen Ulasan Aplikasi: Evaluasi Model BERT, LSTM, dan CNN,” SURYA Inform., vol. 14, no. 2, pp. 66–70, 2024.

[19] A. A. Alfin, I. Kurniasari, and I. Yanuartanti, “Analisis Klasifikasi Sentimen Berbasis Topik pada Ulasan Layanan Dana dan Sakuku dengan Convolutional Neural Network,” Inf. (Journal Inform. dan Sist. Informasi), vol. 15, no. 2, pp. 225–236, 2023.

[20] I. P. A. A. Mahendra and K. Kusrini, “ANALISIS KOMPARATIF KLASIFIKASI SENTIMEN PENGGUNA APLIKASI INVESTASI MENGGUNAKAN ALGORITMA HYBRID CNN-LSTM, CNN-GRU DENGAN IMPLEMENTASI SMOTE (Comparative Analysis of Investment Application User Sentiment Classification Using Hybrid CNN- LSTM, CNN-GRU Alg,” TEKNIMEDIA, vol. 6, no. 1, pp. 112–118, 2025.

[21] C. Ramadhan, V. Atina, and H. Permatasari, “Analisis Perbandingan Model CNN dan IndoBERT Dalam Sentimen Berita Politik Indonesia,” in Prosiding Seminar Nasional Teknologi Informasi dan Bisnis, Jul. 2025, pp. 110–118. doi: 10.47701/v1r9ka69.

[22] E. Yulianti and N. K. Nissa, “ABSA of Indonesian customer reviews using IndoBERT : single- sentence and sentence-pair classification approaches,” Bull. Electr. Eng. Informatics, vol. 13, no. 5, pp. 3579–3589, 2024, doi: 10.11591/eei.v13i5.8032.