Evaluating Multimodal Feedback Systems in AI-Powered Writing Tools

Abstract

Writing constitutes one of the most formidable abilities for EFL learners, necessitating proficiency in grammar, vocabulary, coherence, and organization. input is essential for writing development; unfortunately, instructor input is sometimes limited by time constraints and high-class sizes. As a result, Automated Writing Evaluation (AWE) systems like Grammarly and ChatGPT are extensively utilized to deliver prompt remedial comments. Although the majority of AI feedback is text-centric, multimodal AI feedback that incorporates text, audio, or visual elements may improve engagement and understanding; yet its efficacy in English as a Foreign Language context is still little investigated. This study studied the efficacy of multimodal AI feedback versus text-only AI feedback on the writing performance of EFL students, as well as students' perceptions and learning outcomes. A quasi-experimental mixed-methods approach was employed, involving 45 undergraduate EFL students enrolled in an academic writing course who completed pre- and post-test writing tasks, a perception questionnaire, and interviews. Paired-sample t-tests indicated a substantial enhancement in both groups, with the text-only group exhibiting a somewhat superior mean gain (2.652) compared to the multimodal group (2.591). Qualitative studies revealed that while students regarded multimodal AI feedback as more interesting, this choice did not result in improved writing ability. These data indicate pedagogical significance for EFL writing instruction.