The The Measurable Effects of Machine Translation on L2 Writing Performance: A Systematic Literature Review
DOI:
https://doi.org/10.24256/ideas.v14i2.12617Keywords:
machine translation, L2 writing, writing performance, learner proficiency, instructional guidanceAbstract
This systematic review investigates the measurable effects of machine translation (MT) on second language (L2) writing performance. It asks how MT is used in EFL/ESL writing instruction and revision, what effects have been reported, and how learner proficiency and instructional guidance shape these effects. Scopus was searched for peer-reviewed studies published in 2022–2026, and a procedure adapted from PRISMA was followed. Fourteen studies met the criteria. Studies were coded, appraised for quality, and counted by outcome. MT served as free access, a revision tool, feedback, a base for post-editing instruction, and a comparison with other tools. Nine of the 10 studies that measured accuracy reported better accuracy. Four of the six studies that measured fluency reported gains. Only four of nine studies reported gains in syntactic complexity, and each gain had limits. Lexical results were mixed to positive (five of nine studies reported clear gains), but only one study tested vocabulary directly. In the studies that tested this, gains made with MT fell when MT was removed, but stayed above pretest levels in the clearest case. Studies that added explicit instruction, such as post-editing training, reported gains mainly in accuracy. Guidance seems to shape the effect of MT, but no study clearly isolated guided from unguided MT use. Only three studies compared proficiency levels, and effects were not simple. Lower-proficiency learners gained most in syntactic complexity in one study, while higher-proficiency learners kept vocabulary gains longer in another. Study quality was high in two, moderate in five, and low in seven. The evidence shows short-term writing performance more than long-term language development. Therefore, MT should be taught with guidance and assessed with and without MT. Future studies need longitudinal designs with delayed tests without MT, guided versus unguided comparisons, and comparisons across proficiency levels and contexts.
References
Alharbi, W. (2023). The use and abuse of artificial intelligence-enabled machine translation in the EFL classroom: An exploratory study. Journal of Education and e-Learning Research, 10(4), 689–701. https://doi.org/10.20448/jeelr.v10i4.5091
Almashy, A., Ahmed, A. S. M. M., Jamshed, M., Ansari, M. S., Banu, S., & Warda, W. U. (2024). Analyzing the impact of CALL tools on English learners’ writing skills: A comparative study of errors correction. World Journal of English Language, 14(6), 657–667. https://doi.org/10.5430/wjel.v14n6p657
Alones, J. (2025). The role of machine translation in English language learning: A systematic review. Journal of English Language and Pedagogy (JELPA), 3(1), 46–67. https://doi.org/10.51826/jelpa.v3i1.1388
Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. https://doi.org/10.1191/1478088706qp063oa
Campbell, M., McKenzie, J. E., Sowden, A., Katikireddi, S. V., Brennan, S. E., Ellis, S., Hartmann-Boyce, J., Ryan, R., Shepperd, S., Thomas, J., Welch, V., & Thomson, H. (2020). Synthesis without meta-analysis (SWiM) in systematic reviews: Reporting guideline. BMJ, 368, l6890. https://doi.org/10.1136/bmj.l6890
Chang, P., Chen, P. J., & Lai, L. L. (2024). Recursive editing with Google Translate: The impact on writing and error correction. Computer Assisted Language Learning, 37(7), 2116–2141. https://doi.org/10.1080/09588221.2022.2147192
Correa, M. (2011). Academic dishonesty in the second language classroom: Instructors’ perspectives. Modern Journal of Language Teaching Methods, 1(1), 65–79.
Correa, M. (2014). Leaving the “peer” out of peer-editing: Online translators as a pedagogical tool in the Spanish as a second language classroom. Latin American Journal of Content & Language Integrated Learning, 7(1), 1–20. https://doi.org/10.5294/3568
Ducar, C., & Schocket, D. H. (2018). Machine translation and the L2 classroom: Pedagogical solutions for making peace with Google Translate. Foreign Language Annals, 51(4), 779–795. https://doi.org/10.1111/flan.12366
Erisyerico, M. L., Perdana, I., & Karani, E. (2025). Students' perceptions of Grammarly application feedback on writing skill at SMAN 4 Palangka Raya. PROJECT (Professional Journal of English Education), 8(2), 292–305. https://journal.ikipsiliwangi.ac.id/project/article/view/26252
Fan, Y., Tang, L., Le, H., Shen, K., Tan, S., Zhao, Y., Shen, Y., Li, X., & Gašević, D. (2025). Beware of metacognitive laziness: Effects of generative artificial intelligence on learning motivation, processes, and performance. British Journal of Educational Technology, 56(2), 489–530. https://doi.org/10.1111/bjet.13544
Farkas, A., & Németh, R. (2022). How to measure gender bias in machine translation: Real-world oriented machine translators, multiple reference points. Social Sciences & Humanities Open, 5(1), 100239. https://doi.org/10.1016/j.ssaho.2021.100239
Hutchins, J. (2010). Machine translation: A concise history. Journal of Translation Studies, 13(1–2), 29–70.
Jiang, L., Yu, R., & Zhao, Y. (2024). Theoretical perspectives and factors influencing machine translation use in L2 writing: A scoping review. Journal of Second Language Writing, 64, 101099. https://doi.org/10.1016/j.jslw.2024.101099
Jolley, J. R., & Maimone, L. (2022). Thirty years of machine translation in language teaching and learning: A review of the literature. L2 Journal, 14(1), 26–44. https://doi.org/10.5070/L214151760
Kang, D.-M. (2026). The effects of machine translation (MT) use on the writing quality and perceptions of L2 writers across proficiency levels: A longitudinal examination. Language Teaching Research. Advance online publication. https://doi.org/10.1177/13621688261439276
Kim, M., & Chon, Y. V. (2025). The impact of self-revision, machine translation, and ChatGPT on L2 writing: Raters’ assessments, linguistic complexity, and error correction. Assessing Writing, 65, 1–18. https://doi.org/10.1016/j.asw.2025.100950
Kim, M., & Han, S. (2023). A case study on integrating Google Translate into college EFL writing instruction. Korean Journal of English Language and Linguistics, 23, 1111–1135. https://doi.org/10.15738/kjell.23..202312.1111
Kim, T., & Chang, S. M. (2024). Machine translation and EFL writing development: A focus on complexity, accuracy, and fluency. The Sociolinguistic Journal of Korea, 32(4), 185–220. https://doi.org/10.14353/sjk.2024.32.4.07
Klimova, B., & Carvalho, A. N. P. (2026). A systematic review of innovative methodologies in foreign language pedagogy in the era of artificial intelligence. Social Sciences & Humanities Open, 14, 103170. https://doi.org/10.1016/j.ssaho.2026.103170
Lee, J., & Oh, S.-Y. (2025). Effects of Google Translate instruction on young learners’ L2 writing revisions. The Journal of AsiaTEFL, 22(1), 41–64. https://doi.org/10.18823/asiatefl.2025.22.1.3.41
Lee, S.-M. (2023). The effectiveness of machine translation in foreign language education: A systematic review and meta-analysis. Computer Assisted Language Learning, 36(1–2), 103–125. https://doi.org/10.1080/09588221.2021.1901745
Lee, S.-M., & Kang, N. (2024). Effects of machine translation on L2 writing proficiency: The complexity, accuracy, lexical diversity, and fluency. Language Learning & Technology, 28(1), 1–19. https://doi.org/10.64152/10125/73585
Lo, S. (2025). Neural machine translation in EFL classrooms: Learners’ vocabulary improvement, immediate vocabulary retention and delayed vocabulary retention. Computer Assisted Language Learning, 38(3), 592–611. https://doi.org/10.1080/09588221.2023.2207603
Long, M. H. (1996). The role of the linguistic environment in second language acquisition. In W. C. Ritchie & T. K. Bhatia (Eds.), Handbook of second language acquisition (pp. 413–468). Academic Press.
Loock, R., & Holt, B. (2024). Augmented linguistic analysis skills: Machine translation and generative AI as pedagogical aids for analyzing complex English compounds. Technology in Language Teaching & Learning, 6(3), 1–27. https://doi.org/10.29140/tltl.v6n3.1489
Masruddin, M., Amir, F., Langaji, A., & Rusdiansyah, R. (2023). Conceptualizing linguistic politeness in light of age. International Journal of Society, Culture & Language, 11(3), 41-55.
Masruddin, Hartina, S., Arifin, M. A., & Langaji, A. (2024). Flipped learning: facilitating student engagement through repeated instruction and direct feedback. Cogent Education, 11(1), 2412500.
McCarthy, B. (2004). Does online machine translation spell the end of take-home translation assignments? Computer-Assisted Language Learning Electronic Journal, 6(1), 26–39.
Namfah, N. (2024). “Should I submit a blank assignment?!!”: The effect of machine translation on the writing process and performance among Thai EFL students with low English proficiency. LEARN Journal: Language Education and Acquisition Research Network, 17(2), 134–162. https://doi.org/10.70730/PRNM2789
Niño, A. (2020). Exploring the use of online machine translation for independent language learning. Research in Learning Technology, 28, 1–32. https://doi.org/10.25304/rlt.v28.2402
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., Chou, R., Glanville, J., Grimshaw, J. M., Hróbjartsson, A., Lalu, M. M., Li, T., Loder, E. W., Mayo-Wilson, E., McDonald, S., . . . Moher, D. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. International Journal of Surgery, 88, 105906. https://doi.org/10.1016/j.ijsu.2021.105906
Poláková, P., & Klimová, B. (2023). Using DeepL translator in learning English as an applied foreign language–An empirical pilot study. Heliyon, 9(8), e18595. https://doi.org/10.1016/j.heliyon.2023.e18595
Qin, Y. (2025). Semantic role labeling in neural machine translation addressing polysemy and ambiguity challenges. Journal of Technology Informatics and Engineering, 4(1), 41–55. https://doi.org/10.51903/jtie.v4i1.274
Resende, N. (2023). Investigating syntactic priming cumulative effects in MT-human interaction. Open Research Europe, 1, 93. https://doi.org/10.12688/openreseurope.13902.2
Sasaki, M., Mizumoto, A., & Matsuda, P. K. (2025). Machine translation as a form of feedback on L2 writing. International Review of Applied Linguistics in Language Teaching, 63(4), 2301–2326. https://doi.org/10.1515/iral-2023-0223
Savoldi, B., Gaido, M., Bentivogli, L., Negri, M., & Turchi, M. (2021). Gender bias in machine translation. Transactions of the Association for Computational Linguistics, 9, 845–874. https://doi.org/10.1162/tacl_a_00401
Schmidt, R. W. (1990). The role of consciousness in second language learning. Applied Linguistics, 11(2), 129–158. https://doi.org/10.1093/applin/11.2.129
Song, C., & Song, Y. (2023). Enhancing academic writing skills and motivation: Assessing the efficacy of ChatGPT in AI-assisted language learning for EFL students. Frontiers in Psychology, 14, 1260843. https://doi.org/10.3389/fpsyg.2023.1260843
Steding, S. (2009). Machine translation in the German classroom: Detection, reaction, prevention. Die Unterrichtspraxis/Teaching German, 42(2), 178–189. https://doi.org/10.1111/j.1756-1221.2009.00052.x
Swain, M. (1995). Three functions of output in second language learning. In G. Cook & B. Seidlhofer (Eds.), Principle and practice in applied linguistics: Studies in honour of H. G. Widdowson (pp. 125–144). Oxford University Press.
Vold, E. T. (2018). Using machine-translated texts to generate L3 learners’ metalinguistic talk. In A. Haukås, C. Bjørke, & M. Dypedahl (Eds.), Metacognition in language learning and teaching (pp. 67–97). Routledge. https://doi.org/10.4324/9781351049146-5
Wang, J., & Ke, X. (2022). Integrating machine translation into EFL writing instruction: Process, product and perception. Journal of Language Teaching and Research, 13(1), 125–137. https://doi.org/10.17507/jltr.1301.15
Wu, Y., Schuster, M., Chen, Z., Le, Q. V., Norouzi, M., Macherey, W., Krikun, M., Cao, Y., Gao, Q., Macherey, K., Klingner, J., Shah, A., Johnson, M., Liu, X., Kaiser, Ł., Gouws, S., Kato, Y., Kudo, T., Kazawa, H., ... Dean, J. (2016). Google’s neural machine translation system: Bridging the gap between human and machine translation. arXiv. https://doi.org/10.48550/arXiv.1609.08144
Yang, Y., Wei, X., Li, P., & Zhai, X. (2023). Assessing the effectiveness of machine translation in the Chinese EFL writing context: A replication of Lee (2020). ReCALL, 35(2), 211–224. https://doi.org/10.1017/S0958344023000022
Yoon, C. W., & Chon, Y. V. (2022). Machine translation errors and L2 learners’ correction strategies by error type and English proficiency. English Teaching, 77(3), 153–175. https://doi.org/10.15858/engtea.77.3.202209.153
Downloads
Published
Issue
Section
Citation Check
License
Copyright (c) 2026 Mimo Lalito Erisyerico, Ni Made Ratminingsih, Made Hery Santosa, Putu Indra Kusuma
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under an Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See the Effect of Open Access)
