Implementation of IndoRoBERTa to Improve the Clarity of the Context of Homograph Words in the Text-to-Speech System for Education Chatbot Early Marriage in Lombok

This study presents the implementation of IndoRoBERTa, a pre-trained Indonesian language model, to improve the contextual clarity of homograph words in Text-to-Speech (TTS) systems, particularly for virtual chatbot applications addressing early marriage education in Lombok. The proposed system integrates IndoRoBERTa into the TTS pipeline to classify the context of homographs prior to grapheme-to-phoneme (G2P) conversion, ensuring accurate pronunciation based on meaning. The research was conducted in two fine-tuning phases: the first utilized 500 manually labeled conversational samples, achieving 96% test accuracy, while the second expanded the dataset with 2,000 auto-labeled samples and yielded 88% accuracy. Evaluation metrics including precision, recall, and F1-score demonstrated the model’s effectiveness across 20 homograph categories. Despite strong results, the study acknowledges limitations in data authenticity and challenges in underrepresented classes. Future work is recommended to incorporate real-world dialogue data and enhance the system’s generalization in more complex linguistic settings. This research contributes to the advancement of Indonesian NLP in TTS systems, particularly in socially impactful educational contexts.

Keywords: IndoRoBERTa, text-to-speech, homograph, early marriage, Indonesian NLP

[1] S. Fan and A.jiteki.v9i3.26490th consequences of child marriage: a systematic review of the evidence,” BMC Public Health,

vol. 22, no. 1, p. 309, Feb. 2022, doi: 10.1186/s12889-022-12707-x.

[2] S. O. Gunawan and S. Bahri, “Impacts of Early Childhood Marriage in Indonesia Viewed from Child Protection Laws

Perspectives,” El-Usrah: Jurnal Hukum Keluarga, vol. 6, no. 2, p. 362, Dec. 2023, doi: 10.22373/ujhk.v6i2.20262.

[3] P. Hariyanti, I. Darmawan, and D. P. Mayangsari, “Child Marriage: An Exploratory Study in Aik Mual, West Lombok, West

Nusa Tenggara,” Proceedings of International Conference on Communication Science, vol. 3, no. 1, pp. 196–201, Jan. 2024,

doi: 10.29303/iccsproceeding.v3i1.453.

[4] Supi Yanti, “PENCEGAHAN PERNIKAHAN DINI DAN EDUKASI DIRI,” ALAINA: Jurnal Pengabdian Masyarakat, vol. 1,

no. 1, Jan. 2024, doi: 10.61798/alaina.v1i1.54.

[5] M. D. H. Rahiem, “COVID-19 and the surge of child marriages: A phenomenon in Nusa Tenggara Barat, Indonesia,” Child

Abuse Negl., vol. 118, p. 105168, Aug. 2021, doi: 10.1016/j.chiabu.2021.105168.

[6] S. Aminah, “RELIGIOUS AND CULTURAL CONSTRUCTS OF THE SASAK COMMUNITY AGAINST CHILD

MARRIAGE PRACTICES,” SANGKéP: Jurnal Kajian Sosial Keagamaan, vol. 6, no. 2, pp. 167–178, Dec. 2023, doi:

10.20414/sangkep.v6i2.8496.

[7] W. Huang, K. F. Hew, and L. K. Fryer, “Chatbots for language learning—Are they really useful? A systematic review of

chatbot‐supported language learning,” J. Comput. Assist. Learn., vol. 38, no. 1, pp. 237–257, Feb. 2022, doi:

10.1111/jcal.12610.

[8] L.-W. Chen, S. Watanabe, and A. Rudnicky, “A Vector Quantized Approach for Text to Speech Synthesis on Real-World

Spontaneous Speech,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 37, no. 11, pp. 12644–12652, Jun.

2023, doi: 10.1609/aaai.v37i11.26488.

[9] D. Hussen Maulud, S. R. M. Zeebaree, K. Jacksi, M. A. Mohammed Sadeeq, and K. Hussein Sharif, “State of Art for Semantic

Analysis of Natural Language Processing,” Qubahan Academic Journal, vol. 1, no. 2, pp. 21–28, Mar. 2021, doi:

10.48161/qaj.v1n2a44.

[10] Y. O. Sihombing, R. Fuad Rachmadi, S. Sumpeno, and Moh. J. Mubarok, “Optimizing IndoRoBERTa Model for Multi-Class

Classification of Sentiment & Emotion on Indonesian Twitter,” in 2024 IEEE 10th Information Technology International

Seminar (ITIS), IEEE, Nov. 2024, pp. 12–17. doi: 10.1109/ITIS64716.2024.10845566.

[11] M. S. Ribeiro, G. Comini, and J. Lorenzo-Trueba, “Improving grapheme-to-phoneme conversion by learning pronunciations

from speech recordings,” Jul. 2023, [Online]. Available: http://arxiv.org/abs/2307.16643

[12] A. R. Hanum et al., “Analisis Kinerja Algoritma Klasifikasi Teks Bert dalam Mendeteksi Berita Hoaks,” Jurnal Teknologi

Informasi dan Ilmu Komputer, vol. 11, no. 3, pp. 537–546, Jul. 2024, doi: 10.25126/jtiik.938093.

[13] A. Arisusanto, N. Suarna, and G. Dwilestari, “Analisa Klasifikasi Data Harga Handphone Menggunakan Algoritma Random

Forest Dengan Optimize Parameter Grid,” Jurnal Teknologi Ilmu Komputer, vol. 1, no. 2, pp. 43–47, 2023, doi:

10.56854/jtik.v1i2.51.

[14] Y. O. Sihombing, R. Fuad Rachmadi, S. Sumpeno, and Moh. J. Mubarok, “Optimizing IndoRoBERTa Model for Multi-Class

Classification of Sentiment & Emotion on Indonesian Twitter,” in 2024 IEEE 10th Information Technology International

Seminar (ITIS), IEEE, Nov. 2024, pp. 12–17. doi: 10.1109/ITIS64716.2024.10845566.

[15] Shanty Natalia, I. Sekarsari, F. Rahmayanti, and N. Febriani,"Promiscuity and Early Marriage Affect Reproductive Health in Adolescents ," in Journal of Community Engagement in Health, vol. 4, no. 1, 2021, doi: 10.30994/jceh.v4i1.113.

[16] R. Susilawati, “Upaya Pencegahan Pernikahan Dini Meningkatkan Generasi Berkualitas di Lombok Timur (Studi Kasus UPTD

PPA Lombok Timur),” attaujih, vol. 1, no. 1, pp. 40–48, Dec. 2022, doi: 10.37216/taujih.v1i1.755.

[17] N. Fitria Aprianti et al., “Nomor 1 Januari,” Indonesian Journal of Community Dedication, vol. 5, 2023.

[18] T. D. Chala, A. C. Guta, and M. H. Asebel, “Design and Development of a Text-to-Speech Synthesizer for Afan Oromo,” SN

Comput. Sci., vol. 3, no. 5, Sep. 2022, doi: 10.1007/s42979-022-01306-7.

[19] W. Suwarningsih, R. A. Pratama, F. Y. Rahadika, and M. H. A. Purnomo, “RoBERTa: language modelling in building

Indonesian question-answering systems,” Telkomnika (Telecommunication Computing Electronics and Control), vol. 20, no. 6,

pp. 1248–1255, Dec. 2022, doi: 10.12928/TELKOMNIKA.v20i6.24248.

[20] Y. Zhang, A. Warstadt, H.-S. Li, and S. R. Bowman, “When Do You Need Billions of Words of Pretraining Data?,” Nov. 2020,

[Online]. Available: http://arxiv.org/abs/2011.04946

[21] E. Yulianti, N. Bhary, J. Abdurrohman, F. W. Dwitilas, E. Q. Nuranti, and H. S. Husin, “Named entity recognition on

Indonesian legal documents: a dataset and study using transformer-based models,” International Journal of Electrical and

Computer Engineering, vol. 14, no. 5, pp. 5489–5501, Oct. 2024, doi: 10.11591/ijece.v14i5.pp5489-5501.

[22] M. Saeful, D. Ayu andhirah, P. Pendidikan Guru Sekolah Dasar, and F. Keguruan dan Ilmu Pendidikan, “Representasi Makna

Ganda (Homograf)dalam Bahasa Makassar: Studi Linguistik pada Masyarakat di Kelurahan Pattenne Kecamatan Polong

Bangkeng Selatan Kabupaten Takalar,” 2024, doi: 10.62383/dilan.v1i1.2120.

[23] H. Bichri, A. Chergui, and M. Hain, “Investigating the Impact of Train / Test Split Ratio on the Performance of Pre-Trained

Models with Custom Datasets,” 2024. [Online]. Available: www.ijacsa.thesai.org

[24] Y. Liu et al., “RoBERTa: A Robustly Optimized BERT Pretraining Approach,” Jul. 2019, [Online]. Available:

http://arxiv.org/abs/1907.11692

[25] E. Yulianti and N. K. Nissa, “ABSA of Indonesian customer reviews using IndoBERT: single- sentence and sentence-pair

classification approaches,” Bulletin of Electrical Engineering and Informatics, vol. 13, no. 5, pp. 3579–3589, Oct. 2024, doi:

10.11591/eei.v13i5.8032.

[26] D. M. Aprilla, F. Bimantoro, and I. G. P. Suta Wijaya, “The Palmprint Recognition Using Xception, VGG16, ResNet50,

MobileNet, and EfficientNetB0 Architecture,” JURNAL MEDIA INFORMATIKA BUDIDARMA, vol. 8, no. 2, p. 1065, Apr.

2024, doi: 10.30865/mib.v8i2.7577.

[27] Y. Li, X. Ren, F. Zhao, and S. Yang, “A Zeroth-Order Adaptive Learning Rate Method to Reduce Cost of Hyperparameter

Tuning for Deep Learning,” Applied Sciences, vol. 11, no. 21, p. 10184, Oct. 2021, doi: 10.3390/app112110184.

[28] Y. O. Sihombing, N. V. Situmorang, B. K. Negara, and J. M. Sutoyo, “Prediksi Sentimen Pada Teks Media Sosial Corporate University Menggunakan RoBERTa,” 2024.

[29] M. Nanni, J. Sjons, and F. Von Kartaschew, “Disambiguating Italian homographic heterophones with SoundChoice and testing ChatGPT as a data-generating tool,” 2023.

[30] T. T. A. Putri, S. Sriadhi, R. D. Sari, R. Rahmadani, and H. D. Hutahaean, “A comparison of classification algorithms for hate speech detection,” IOP Conf. Ser. Mater. Sci. Eng., vol. 830, no. 3, p. 032006, Apr. 2020, doi: 10.1088/1757-

899X/830/3/032006.

[31] L. Geni, E. Yulianti, and D. I. Sensuse, “Sentiment Analysis of Tweets Before the 2024 Elections in Indonesia Using Bert

Language Models,” Jurnal Ilmiah Teknik Elektro Komputer dan Informatika, vol. 9, no. 3, pp. 746–757, Aug. 2023, doi:

10.26555/jiteki.v9i3.26490.