Analysis of User Complaints for Telecommunication Brands on X (Twitter) using IndoBERT and Deep Learning

Valianda Farradillah Hakim; Dwiza Riana

doi:10.23887/janapati.v13i2.76497

Authors

Valianda Farradillah Hakim Universitas Nusa Mandiri
Dwiza Riana Nusa Mandiri University

DOI:

https://doi.org/10.23887/janapati.v13i2.76497

Keywords:

Twitter, IndoBERT, CNN, Bi-LSTM, Tellkomsel

Abstract

Tweeting on different official accounts is what users of Twitter (X) do most frequently. These tweets ranging from compliments to critiques. One of the official accounts that gets a lot of tweets from its customers is Telkomsel, an Indonesian telecom company. This study aims to find the maximum accuracy that can be obtained by combining CNN and Bi-LSTM algorithms with IndoBERT embeddings. A considerable accuracy level above 90% is demonstrated by the study, with CNN obtaining the greatest accuracy of 99% at a learning rate of 6*10^^-5, along with scores of 98%, 97%, and 97% for precision, recall, and F1 correspondingly.

References

“Essential Twitter statistics and trends for 2023,” 2023. https://datareportal.com/essential-twitter-stats (accessed Jul. 20, 2023).

S. Kemp, “Twitter users in Indonesia in 2023,” 2023. https://datareportal.com/reports/digital-2022-indonesia (accessed Mar. 12, 2023).

A. Ahdiyat, “Operator Seluler yang Digunakan Responden (Januari 2023),” 2023. https://databoks.katadata.co.id/datapublish/2023/06/23/ini-operator-seluler-dengan-pengguna-terbanyak-di-indonesia-awal-2023 (accessed Jul. 21, 2023).

D. Novianty and D. Prastya, “Riset Counterpoint: Telkomsel Jadi Operator Seluler Terbesar di Indonesia,” 2022. https://www.suara.com/tekno/2022/07/17/160231/riset-counterpoint-telkomsel-jadi-operator-seluler-terbesar-di-indonesia?page=all (accessed Jul. 31, 2023).

K. S. Nugroho, I. Akbar, A. N. Suksmawati, and Istiadi, “Deteksi Depresi Dan Kecemasan Pengguna Twitter Menggunakan Bidirectional Lstm,” arXiv, no. Ciastech, pp. 287–296, 2023, doi: 10.48550/arXiv.2301.04521.

Y. Widhiyasana, T. Semiawan, I. Gibran, A. Mudzakir, and M. R. Noor, “Penerapan Convolutional Long Short-Term Memory untuk Klasifikasi Teks Berita Bahasa Indonesia (Convolutional Long Short-Term Memory Implementation for Indonesian News Classification),” J. Nas. Tek. Elektro dan Teknol. Inf. |, vol. 10, no. 4, pp. 354–361, 2021.

I. Ayu Shafirra N, “Klasifikasi Sentimen Ulasan Film Indonesia dengan Konversi Speech-to-Text (STT) Menggunakan CNN,” J. sains dan seni ITS, vol. 9, no. 1, pp. 2301–9271, 2020.

L. Geni, E. Yulianti, and D. I. Sensuse, “Sentiment Analysis of Tweets Before the 2024 Elections in Indonesia Using IndoBERT Language Models,” J. Ilm. Tek. Elektro Komput. dan Inform., vol. 9, no. 3, pp. 746–757, 2023, doi: 10.26555/jiteki.v9i3.26490.

D. Arista, Y. Sibaroni, and S. Prasetyowati, “Sentiment Analysis on Twitter(X) Related to Relocating the National Capital using the IndoBERT Method using Extraction Features of Chi-Square,” J. Media Inform. Budidarma, vol. 8, pp. 403–411, 2024, doi: 10.30865/mib.v8i1.7198.

A. Kurniasih and L. P. Manik, “On the Role of Text Preprocessing in BERT Embedding-based DNNs for Classifying Informal Texts,” Int. J. Adv. Comput. Sci. Appl., vol. 13, no. 6, pp. 927–934, 2022, doi: 10.14569/IJACSA.2022.01306109.

I. R. Hidayat and W. Maharani, “General Depression Detection Analysis Using IndoBERT Method,” Int. J. Inf. Commun. Technol., vol. 8, no. 1, pp. 41–51, 2022, doi: 10.21108/ijoict.v8i1.634.

A. Candra, Wella, and A. Wicaksana, “Bidirectional encoder representations from transformers for cyberbullying text detection in indonesian social media,” Int. J. Innov. Comput. Inf. Control, vol. 17, no. 5, pp. 1599–1615, 2021, doi: 10.24507/ijicic.17.05.1599.

F. Koto, A. Rahimi, J. H. Lau, and T. Baldwin, “IndoLEM and IndoBERT: A Benchmark Dataset and Pre-trained Language Model for Indonesian NLP,” in Proceedings of the 28th COLING, 2020.

“IndoBERT,” 2020. https://indolem.github.io/IndoBERT/ (accessed May 12, 2023).

M. J. Hamayel and A. Y. Owda, “A Novel Cryptocurrency Price Prediction Model Using GRU, LSTM and bi-LSTM Machine Learning Algorithms,” Ai, vol. 2, no. 4, pp. 477–496, 2021, doi: 10.3390/ai2040030.

F. Shahid, A. Zameer, and M. Muneeb, “Predictions for COVID-19 with deep learning models of LSTM, GRU and Bi-LSTM,” Chaos, Solitons and Fractals, vol. 140, p. 110212, 2020, doi: 10.1016/j.chaos.2020.110212.

Z. Li, F. Liu, W. Yang, S. Peng, and J. Zhou, “A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospects,” IEEE Trans. Neural Networks Learn. Syst., vol. 33, no. 12, pp. 6999–7019, 2022, doi: 10.1109/TNNLS.2021.3084827.

M. Umer et al., “Impact of convolutional neural network and FastText embedding on text classification,” Multimed. Tools Appl., vol. 82, no. 4, pp. 5569–5585, 2023, doi: 10.1007/s11042-022-13459-x.

R. K. G. D. & K. T. Rikiya Yamashita, Mizuho Nishio, “Convolutional neural networks: an overview and application in radiology https://doi.org/10.1007/s13244-018-0639-9,” Springer, vol. 195, pp. 21–30, 2018.

A. Zafar et al., “A Comparison of Pooling Methods for Convolutional Neural Networks,” Appl. Sci., vol. 12, no. 17, pp. 1–21, 2022, doi: 10.3390/app12178643.

L. Alzubaidi et al., Review of deep learning: concepts, CNN architectures, challenges, applications, future directions, vol. 8, no. 1. Springer International Publishing, 2021. doi: 10.1186/s40537-021-00444-8.

Keras, “Probabilistic losses.” https://keras.io/api/losses/probabilistic_losses/ (accessed Jul. 20, 2023).

R. Haque, S. B. Ho, I. Chai, and A. Abdullah, “Parameter and Hyperparameter Optimisation of Deep Neural Network Model for Personalised Predictions of Asthma,” J. Adv. Inf. Technol., vol. 13, no. 5, pp. 512–517, 2022, doi: 10.12720/jait.13.5.512-517.

R. Lohith, K. E. Cholachgudda, and R. C. Biradar, “PyTorch Implementation and Assessment of Pre-Trained Convolutional Neural Networks for Tomato Leaf Disease Classification,” 2022 IEEE Reg. 10 Symp. TENSYMP 2022, pp. 1–6, 2022, doi: 10.1109/TENSYMP54529.2022.9864390.