Exploring The Quality of AI Translation in Roblox Chat
DOI:
https://doi.org/10.24256/ideas.v14i1.11458Keywords:
Translation Quality Assessment (TQA), 'Machine Translation', Neural machine translation, roblox, TranslationAbstract
This study evaluates the translation quality of Roblox’s AI translation system by combining translation quality assessment (TQA) with syntactic clause analysis in a multilingual online gaming environment. The study aims to examine how the system translates different clause types based on Nababan’s theory, which is assessed by parameters of accuracy, acceptability, and readability. A qualitative approach was employed using 15 representative Indonesian–English translation pairs, selected from an initial dataset of 34 chat conversations collected from real-time Roblox gameplay. The data were classified into declarative, interrogative, and imperative clauses based on the syntactic framework of Huddleston and Pullum before being evaluated using the TQA model. The results show that the Roblox AI translation system performed best on interrogative clauses, while imperative and informal declarative clauses produced lower translation quality due to frequent lexical, syntactic, and semantic errors. The overall translation quality score was 1.98 on a 1–3 scale, indicating moderate-to-low translation quality. These findings demonstrate that combining TQA with syntactic clause analysis provides a more comprehensive approach to gaming communication and highlights the need for translation models that better accommodate informal language, gaming-specific expressions, and context-dependent interactions.
References
Cahyaningrum, I. O. (2021). Quality of Automatic Translation for Post on Facebook. Retrieved from http://publikasi.dinus.ac.id/index.php/estructural
Creswell, John. W., & Creswell, J. D. (2018). Research Design: Qualitative, Quantitative, and Mixed Methods Approaches.
Dostert, L. E. (1963). MACHINE TRANSLATION AND AUTOMATIC LANGUAGE DATA PROCESSING.
Haikal, F., & Ali, M. (2025). Translation Error in Auto Translation Feature in Song “Higuchi Ai-Akuma No Ko” The First Take. In International Journal of Computer in Humanities (Vol. 5).
Huddleston, R., & Pullum, G. K. (2002). THE CAMBRIDGE GRAMMAR OF THE ENGLISH LANGUAGE.
Kanoko, N. P., & Sutrisno, A. (2024). Translation Evaluation of Social Media Auto-Translate Feature toward News Headline. Sasdaya: Gadjah Mada Journal of Humanities, 8(2), 235–255. https://doi.org/10.22146/sasdaya.13180
Karyani, L., Nababan, M. R., & Marmanto, S. (2020). Translation Analysis on Dayak Cultural Terms From Dayak Ngaju to Indonesian and English. Langkawi: Journal of The Association for Arabic and English, 6(1), 41. https://doi.org/10.31332/lkw.v6i1.1676
Koka, N., Akan, Md. F., Kana’n, B. H. I., Khan, M. R., Zulfiquar, F., & Jan, N. (2023). IMPACT OF ARTIFICIAL INTELLIGENCE (AI) ON TRANSLATION QUALITY: ASSESSMENT AND EVALUATION. Journal of Southwest Jiaotong University, 58(4). https://doi.org/10.35741/issn.0258-2724.58.4.72
Ma’arif, S., Salma, A., & Afyudiin, M. S. (2025). The Quality of ChatGPT’s Arabic-Indonesian Translation of Matan al-Ghayah wa at-Taqrib: Implications for Arabic Language Learning in Pesantren. ATHLA : Journal of Arabic Teaching, Linguistic and Literature, 6(2), 81–97. https://doi.org/10.22515/athla.v6i2.12555
Nababan, M., Nuraeni, A., & Sumardiono. (2012). Pengembangan Model Penilaian Kualitas Terjemahan. Kajian Linguistik dan Sastra.
Newmark, P. (1988). A TEXTBOOK OF TRANSLATION W *MRtt SHANGHAI FOREIGN LANGUAGE EDUCATION PRESS.
Poibeau, T. (2017). The MIT Press Essential Knowledge Series.
Saehu, A., & Hkikmat, M. M. (2025). The quality and accuracy of AI-generated translation in translating communication based topics: Bringing translation quality assessments into practices. In Role of AI in Translation and Interpretation (pp. 237–265). IGI Global. https://doi.org/10.4018/979-8-3373-0060-3.ch009
Umbar, K., Aisyah Adilah, Rahma Raihana Romadona, & Muhammad Rafli. (2023). Analisis Akurasi Penerjemahan Bahasa Arab Lewat Fitur Auto Translate pada Aplikasi Twitter. Jurnal Naskhi: Jurnal Kajian Pendidikan Dan Bahasa Arab, 5(1), 21–33. https://doi.org/10.47435/naskhi.v5i1.1460
Wicaksono, R. D., Anwar, K., & Asmara, C. H. (2025). Students Perception of Roblox and Language Barrier in ESL. Indonesian Teaching English to Speakers of Other Languages Journal, 2(2). https://doi.org/10.30587/inatesol.v2i2.9139
Wu, D., Wang, M., & Li, X. (2022). Automatic Scoring for Translations Based on Language Models. Computational Intelligence and Neuroscience, 2022. https://doi.org/10.1155/2022/2171206
Downloads
Published
Issue
Section
Citation Check
License
Copyright (c) 2026 Siti Nabila Amira Wangsi, Erfan Muhamad Fauzi, Fourus Huznatul Abqoriyyah
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under an Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See the Effect of Open Access)
