Improving SMK Students' Speaking Skills with AI-Based Learning in Vocational Contexts: A Quasi-Experimental Study
DOI:
https://doi.org/10.24256/ideas.v14i2.10887Keywords:
AI-based learning; Gemini; speaking skills; vocational context; quasi-experimentalAbstract
English speaking proficiency is essential for vocational high school (SMK) graduates entering the workforce, yet speaking instruction often lacks authentic workplace contexts and scalable feedback mechanisms. Although AI-based language learning and vocational contextualized approaches are each documented in the literature, their synergistic integration remains empirically under-researched in Indonesian SMK pharmacy settings. This study examined whether AI-based learning embedded in vocational pharmacy tasks significantly improves the English speaking skills of Grade XI students at SMKN 1 Polewali. A quasi-experimental nonequivalent pretest–posttest control group design was employed with 31 Grade XI Pharmacy students (experimental n = 15; control n = 16). The experimental group received Gemini-assisted patient-counseling role-play instruction; the control group received conventional vocational-contextualized instruction without AI, across eight sessions over four weeks. Speaking performance was assessed using Heaton's (1988) rubric covering accuracy, fluency, and comprehensibility, and analyzed using paired-samples t-tests, an independent-samples t-test, and Cohen's d. The experimental group demonstrated significant pre–post improvement across all speaking aspects (p < .001) with large effect sizes (average d = 2.97), and significantly outperformed the control group at posttest (p = .006, d = 1.06). Comprehensibility showed the greatest gain. Theoretically, the study extends Swain's (1995) Output Hypothesis into AI-assisted ESP contexts by showing how chatbot-mediated role-play supports pushed output and immediate corrective feedback in resource-constrained SMK settings. The novelty lies in the synergistic integration of AI-based learning and vocational contextualized instruction—an approach not previously validated in Indonesian SMK pharmacy classrooms. AI-based learning in vocational contexts effectively improved speaking skills. Integrating accessible AI tools with workplace-relevant tasks offers a scalable model for English instruction in SMK programmes.
References
Anthony, L. (2018). Introducing English for specific purposes. Routledge.
Arora, B., Al-Wadi, H., & Afari, E. (2024). Scaffolding instruction for improvement in learning English language skills. International Journal of Evaluation and Research in Education (IJERE), 13(2), 1265–1275. doi:10.11591/ijere.v13i2.26659
Campbell, D. T., & Stanley, J. C. (1966). Experimental and quasi-experimental designs for research. Rand McNally.
Chen, J., Lai, P., Chan, A., Man, V., & Chan, C.-H. (2023). AI-assisted enhancement of student presentation skills: Challenges and opportunities. Sustainability, 15(1), Article 196. doi:10.3390/su15010196
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum Associates.
Creswell, J. W., & Creswell, J. D. (2018). Research design: Qualitative, quantitative, and mixed methods approaches (5th ed.). SAGE.
Dudley-Evans, T., & St. John, M. J. (1998). Developments in English for specific purposes: A multi-disciplinary approach. Cambridge University Press.
Efrizah, D., Fadly, Y., & Putri, V. O. (2024). English speaking barriers in vocational education: A study of SMK SPP SNAKMA students. Journal of Language, 6(2), 271–278. e-ISSN: 2685-8878 | p-ISSN: 2655-9080. Retrieved from https://jurnal.uisu.ac.id/index.php/journaloflanguage
Fathi, J., Rahimi, M., & Derakshan, A. (2024). Improving EFL learners' speaking skills and willingness to communicate via artificial intelligence-mediated interactions. System, 121, Article 103254. doi:10.1016/j.system.2024.103254
Field, A. (2013). Discovering statistics using IBM SPSS Statistics (4th ed.). SAGE.
Heaton, J. B. (1988). Writing English language tests. Longman.
Kementerian Pendidikan dan Kebudayaan. (2017). Peraturan Menteri Pendidikan dan Kebudayaan Republik Indonesia nomor 3 tahun 2017 tentang penilaian hasil belajar oleh pemerintah dan penilaian hasil belajar oleh satuan pendidikan. Berita Negara Republik Indonesia Tahun 2017 nomor 117.
Kementerian Pendidikan Dasar dan Menengah. (2025). Peraturan Menteri Pendidikan Dasar dan Menengah Republik Indonesia Nomor 13 Tahun 2025 tentang Perubahan atas Peraturan Menteri Pendidikan, Kebudayaan, Riset, dan Teknologi Nomor 12 Tahun 2024 tentang Kurikulum pada Pendidikan Anak Usia Dini, Jenjang Pendidikan Dasar, dan Jenjang Pendidikan Menengah. Berita Negara Republik Indonesia Tahun 2025 Nomor 503. Retrieved from https://jdih.kemendikdasmen.go.id
Koo, T. K., & Mae, A. Y. (2016). A guideline of selecting and reporting intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine, 15(2), 155–163. doi:10.1016/j.jcm.2016.02.012
Lee, S.-E. (2023). Otherwise than teaching by artificial intelligence. Journal of Philosophy of Education, 57, 553–570. https://doi.org/10.1093/jopedu/qhad019
Lima, L. A. de O., Gomes, L. P., Silva e Silva, P. H. da, Oliveira, E. F. da S., Nascimento, M. do, Tourem, R. V., Gonçalves, J. N. de A., Lima, A. da S., Sobral, R., & Santos, I. da M. P. dos. (2024). Artificial intelligence and its use in the educational process. Navigating Through the Knowledge of Education, 43, 1–20. doi:10.56238/sevened2024.002-043
Long, M. H. (Ed.). (2005). Second language needs analysis. Cambridge University Press.
Newton, J. M., & Nation, I. S. P. (2021). Teaching ESL/EFL listening and speaking (2nd ed.). Routledge.
Nguyen, H. A. (2024). Harnessing AI-based tools for enhancing English speaking proficiency: Impacts, challenges, and long-term engagement. International Journal of AI in Language Education, 1(2), 18–29. doi:10.54855/ijaile.24122
Nhan, L. K., Hoa, N. T. M., & Quang, L. V. N. (2025). Revolutionizing speaking skills improvement: AI's role in personalized language learning. International Journal of Innovative Research and Scientific Studies, 8(2), 4637–4649. doi:10.53894/ijirss.v8i2.6408
Nuryadin, R., & Marlina. (2023). The use of artificial intelligence in education (literature review). Indonesian Journal of Primary Education, 7(2), 143–156. Retrieved from https://ejournal.upi.edu/index.php/IJPE/index
Pallant, J. (2020). SPSS survival manual: A step by step guide to data analysis using IBM SPSS (7th ed.). Routledge.
Pishadast, A. (2022). Developing the speaking ability of EFL learners through scaffolding. Journal of Contemporary Language Research (JCLR), 1(2), 60–64. doi:10.58803/jclr.v1i2.8
Rahman, S., Yani, A., & Prasetyo, Z. K. (2022). Vocational English contextualization in Industry 4.0. Journal of Vocational Education, 18(2), 90–104.
Safira, L., & Azzahra, N. F. (2022). Meningkatkan kesiapan kerja lulusan SMK melalui perbaikan kurikulum bahasa Inggris. Center for Indonesian Policy Studies. doi:10.35497/558654
Safitri, E. I., Hidayati, S., & Ciptaningrum, D. S. (2025). The impact of AI chatbots on English language learners' speaking proficiency: A systematic review. Journal of Research on English and Language Learning (J-REaLL), 6(2), 317–329. https://doi.org/10.33474/j-reall.v6i2.23866
Sawilowsky, S. S. (2009). New effect size rules of thumb. Journal of Modern Applied Statistical Methods, 8(2), 597–599. doi:10.22237/jmasm/1257035100
Seidlhofer, B. (2011). Understanding English as a lingua franca. Oxford University Press.
Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Houghton Mifflin.
Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86(2), 420–428. doi:10.1037/0033-2909.86.2.420
Skarpaas, K. G., & Hellekjær, G. O. (2021). Vocational orientation—A supportive approach to teaching L2 English in upper secondary school vocational programmes. International Journal of Educational Research Open, 2, Article 100064. doi:10.1016/j.ijedro.2021.100064
Statistics Indonesia. (2021). Unemployment rate by education level, 2019–2021. Retrieved from https://www.bps.go.id/indicator/6/1179/1/unemployment-rate-by-education-level.html
Swain, M. (1995). Three functions of output in second language learning. In G. Cook & B. Seidlhofer (Eds.), Principle and practice in applied linguistics: Studies in honour of H. G. Widdowson (pp. 125–144). Oxford University Press.
Sweller, J. (1994). Cognitive load theory, learning difficulty, and instructional design. Learning and Instruction, 4(4), 295–312. doi:10.1016/0959-4752(94)90003-5
Thompson, F., & Lim, G. (2021). Reaping the benefits of industry 4.0 through skills development in high-growth industries in Southeast Asia: Insights from Cambodia, Indonesia, the Philippines, and Viet Nam (Report No. SPR-200328). Asian Development Bank. doi:10.22617/SPR200328
Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019). Systematic review of research on artificial intelligence applications in higher education—Where are the educators? International Journal of Educational Technology in Higher Education, 16, Article 39. doi:10.1186/s41239-019-0171-0
Zou, B., Du, Y., Wang, Z., Chen, J., & Zhang, W. (2023). An investigation into artificial intelligence speech evaluation programs with automatic feedback for developing EFL learners' speaking skills. SAGE Open, 13(3), 1–9. doi:10.1177/21582440231193818
Downloads
Published
Issue
Section
Citation Check
License
Copyright (c) 2026 Sitti Maesura Yahya, Munir, Amra Ariyani
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under an Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See the Effect of Open Access)
