Translation Quality Assessment in Conventional and AI-Assisted Practices: Evidence from EFL Student Translators
DOI:
https://doi.org/10.24256/ideas.v14i2.13018Keywords:
AI-assisted translation; EFL learners; student translators; traditional translation; translation quality.Abstract
The emergence of artificial intelligence technology has changed translation techniques in the English as a Foreign Language (EFL) learning, particularly through ChatGPT, Google Translate and DeepL. The aim of this study was to find the differences in the translation quality of EFL students when translating texts without AI support and with it, and to find out how far AI can help support the process of academic translation. This study employed qualitative descriptive design with 10 students of the sixth semester in the English Literature Study Program at Universitas Bumigora West Nusa Tenggara. Data was acquired via two translation tasks, one using the traditional way, and the other using AI aid. Larson’s translation quality metrics were then used to assess the data. The results indicated that both strategies yielded relatively satisfactory translations. However, AI assisted translation scored higher in all respects. The best score in category 4 earned an overall score of 78.6% in the traditional way, with an accuracy of 82.9%, clarity of 78.6%, and naturalness of 74.3%. Meanwhile, the AI-assisted translation obtained an overall score of 83.33%, with an accuracy of 87.14%, clarity of 82.86% and naturalness of 80.0%. The results showed that the AI-assisted condition obtained higher observed scores than the conventional condition across accuracy, clarity, and naturalness. However, these differences should be interpreted cautiously because the two tasks were not fully equivalent and the AI-assisted task followed the conventional task. The findings therefore indicate an observed difference in translation quality rather than a definitive causal effect of AI assistance. These findings suggest that AI-assisted translation may serve as a useful learning resource when accompanied by critical evaluation and human post-editing.
References
Al Farisi, M. Z. (2018). The Impact of techniques and translation ideology on the clarity of pragmatic meanings translation of the Qur’anic imperative verses. Indonesian Journal of Applied Linguistics, 7(3), 676–686. https://doi.org/10.17509/ijal.v7i3.9818
Algaraady, J., & Mahyoob, M. (2025). Exploring ChatGPT’s potential for augmenting post-editing in machine translation across multiple domains: challenges and opportunities. Frontiers in Artificial Intelligence, 8, 1526293. https://doi.org/https://doi.org/10.3389/frai.2025.1526293
Ayyaz, S. (2025). AI Translation Tools in Language Learning: Bridging Gaps and Enhancing Skills. In AI Applications for English Language Learning (pp. 43–72). IGI Global Scientific Publishing.
Belhassen, S., Hakami, A., Alzobidy, S., & Hamda, A. (2025). Navigating the complexities of AI-driven literary translation: Challenges and perspectives across diverse user groups.
Boonmoh, A., & Kulavichian, I. (n.d.). Enhancing Thai Pre-Service Teachers’ Translation Skills Through AI Tools, Social Media, and Public Feedback.
Castilho, S., Doherty, S., Gaspari, F., & Moorkens, J. (2018). Approaches to Human and Machine Translation Quality Assessment. In J. Moorkens, S. Castilho, F. Gaspari, & S. Doherty (Eds.), Translation Quality Assessment: From Principles to Practice (pp. 9–38). Springer International Publishing. https://doi.org/10.1007/978-3-319-91241-7_2
Dinh, C. T. (2025). EFL Students’ Perspectives on ChatGPT in Translation: Exploring AI Assistance, Motivation, and Learning Outcomes. Electronic Journal of E-Learning, 23(2), 99–116.
Hadi, M. Z. P., Yuliarsi, I., Pratama, H., & Yulianti, Y. (2024). Literacy in translation: between strategic competence and AI assistance. Proceedings of Fine Arts, Literature, Language, and Education, 830–839. https://proceeding.unnes.ac.id/icoella/article/view/3788
Hadi, M. Z. P., Yuliasri, I., Pratama, H., & Yuliati. (2025). Exploring Translation Techniques and Ideological Tendencies in AI-Assisted English Indonesian Texts by EFL Learners. International Journal of Learning, Teaching and Educational Research, 24(10), 85–105. https://doi.org/10.26803/ijlter.24.10.4
Karasimos, A. (2025). Artificial Intelligence and Machine Translation: Post-editing of Machine Translated Texts by Large Language Models vs Human Translation.
Larson, M. L. (1997). Meaning-based translation: A guide to cross-language equivalence. University press of America.
Li, R., Han, S., & Guo, Y.-K. (2025). Simulated Justice: How AI Alignment Replaces Conflict with Coherence. Available at SSRN 5940998.
Lin, G. H., & Chien, P. S. C. (2009). Machine Translation for Academic Purposes. International Conference on TESOL and Translation.
Lysova, I., Ahmed, L., Cunningham, B., Huang, Y., & Wiltschko, M. (2026). Do chatbots dream of AI sheep? A semantic–pragmatic investigation of “naturalness” in human–AI interaction. AI & SOCIETY, 41(2), 1053–1062. https://doi.org/10.1007/s00146-025-02595-1
Meyer, A. H. (2024). AI-Driven Quality Assurance in Translation: Tools and Techniques. Translating the Future: Exploring the Impact of Technology and AI on Modern Translation Studies, 53–64. https://doi.org/10.1007/978-3-319-91241-7_2
Moulavinafchi, A., Hosseini, S. M., Shadpour, R. S., & Qasim, Y. H. Q. (2025a). AI and the Next Era of Translation: A Critical Inquiry on Advantages, Risks, and Future Venues. Gonbad-Cals, 2(1), 1–34. https://doi.org/10.22034/jcals.2.1.1
Moulavinafchi, A., Hosseini, S. M., Shadpour, R. S., & Qasim, Y. H. Q. (2025b). AI and the Next Era of Translation: A Critical Inquiry on Advantages, Risks, and Future Venues. Critical Applied Linguistics Studies, 2(1), 1–34.
Otaif, F. A. (2026). From Input to Narrative: Authoring Processes in AI-generated Discourse. Journal of Arabic Sciences and Humanities, 5(2), 120–146.
Patton, M. Q. (2002). Two decades of developments in qualitative inquiry: A personal, experiential perspective. Qualitative Social Work, 1(3), 261–283. https://doi.org/https://doi.org/10.1177/1473325002001003636
Răducea, L., & Leotta, P. C. (2026). Traditional AI vs Generative AI vs Human Translation in Education-Related Texts, Translated from English into Italian. International Journal of English Linguistics, 16(1), 84. https://doi.org/10.5539/ijel.v16n1p84
Sayadani, A., Heydarpour, Y., & Asgharpour, S. (2019). The Quality of Meaning-based Translation According to Larson’s Theory in Yasrebis Translation of Quran: Case Study of Kahf Surah. Translation Researches in the Arabic Language And Literature, 9(20), 59–92. https://doi.org/https://doi.org/10.22054/rctall.2019.36367.1348
Way, A. (2018). Quality Expectations of Machine Translation. In J. Moorkens, S. Castilho, F. Gaspari, & S. Doherty (Eds.), Translation Quality Assessment: From Principles to Practice (pp. 159–178). Springer International Publishing. https://doi.org/10.1007/978-3-319-91241-7_8
Weda, S. (2014). Transtool versus conventional translation in digital technology era. International Journal on Studies in English Language and Literature (IJSELL), 2(8), 149–153.
Wu, W. (2023). Automatic Machine Translation Quality Evaluation Model under Feature Extraction Algorithm. Procedia Computer Science, 247(C), 689–695. https://doi.org/10.1016/j.procs.2024.10.083
Yuliasri, I., Purwanto, B., Hadi, M. Z. P., Nurcahyo, A. D., Rochmahwati, P., Rifqi, M., Kumara, K. A. D., & Ma’sumah, F. (2025). Student poetry translations with and without the use of AI: An evaluation of translation quality. LITE: Jurnal Bahasa, Sastra, Dan Budaya, 21(2), 452–468. https://doi.org/10.33633/lite.v21i2.14336
Yuxiu, Y. (2024). Application of translation technology based on AI in translation teaching. Systems and Soft Computing, 6. https://doi.org/10.1016/j.sasc.2024.200072
Zouhar, V., Kocmi, T., & Sachan, M. (2025). AI-Assisted Human Evaluation of Machine Translation. ArXiv Preprint ArXiv. https://doi.org/https://doi.org/10.48550/arXiv.2406.12419Focustolearnmore
Downloads
Published
Issue
Section
Citation Check
License
Copyright (c) 2026 Muhammad Zaki Pahrul Hadi, Issy Yuliasri, Hendi Pratama
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under an Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See the Effect of Open Access)
