Developing Assessment Instruments for Procedural Writing of Managing Flight Emergency and Urgency Situations: Examining Indonesian Aviation English Instructors’ Perceptions
DOI:
https://doi.org/10.24256/ideas.v13i2.7775Keywords:
Assessment instruments, aviation English, flight emergency and urgency, procedural writingAbstract
Despite the critical need for written communication and emergency management proficiency in aviation, existing research in aviation English education has primarily emphasized speaking and listening skills, neglecting cadets’ writing abilities. This gap is particularly concerning given the International Civil Aviation Organization (ICAO) Annex 10 requirements, which demand accurate and coherent documentation in emergency scenarios. Addressing this void, the present study aimed to develop reliable and valid assessment instruments for assessing aviation cadets’ procedural writing skills in managing flight emergency and urgency situations. This study also examined the Indonesian Aviation English instructors’ perceptions on the developed assessment instruments. Employing a Research and Development (R&D) design using the ADDIE model—Analysis, Design, Development, Implementation, and Evaluation—the study constructed test instructions and an analytic scoring rubric based on learning indicators derived from ICAO Annex 10. The instruments underwent expert validation (CVI = 1.00) and were trialed with 24 third-semester aeronautical communication cadets. The inter-rater reliability test showed good consistency (ICC = 0.802–0.890), confirming the rubric’s reliability in practical use. Results revealed that the instruments effectively assessed both linguistic skills and operational understanding. Aviation English instructors found the assessment instruments pedagogically valuable as they promoted integrated language learning and critical thinking skills through the application of a sequenced procedure in managing flight emergency and urgency situations. However, limitations include the small sample size and the absence of multimodal performance assessments. Future research should expand implementation across diverse aviation academies and explore integration with AI-driven tools or scenario-based simulations. This study contributes to English for Specific Purposes (ESP) pedagogy by providing a validated framework for enhancing and assessing domain-specific writing in high-stakes professional contexts.
References
Balaji, R., & Venkadasalam, S. (2017). Developing a marine engineering centre of excellence for competency-based training. WMU Journal of Maritime Affairs, 16(2), 293-311. https://doi.org/10.1007/s13437-017-0124-1
Barshi, I., & Farris, C. (2016). Misunderstandings in ATC communication: Language, cognition, and experimental methodology. Routledge. https://shorturl.at/oWCkT
Bechinie, C., Zafari, S., Kroeninger, L., Puthenkalam, J., & Tscheligi, M. (2024). Toward human-centered intelligent assistance system in manufacturing: challenges and potentials for operator 5.0. Procedia Computer Science, 232, 1584-1596. https://doi.org/10.1016/j.procs.2024.01.156
Bjornestad, A., Olson, S., & Weidauer, L. (2021). Improving officer-soldier communication through active listening skills training with Army ROTC cadets. Military Psychology, 33(3), 205-215. https://doi.org/10.1080/08995605.2021.1902179
Borkers, P. (2024). Measuring service quality during and after in-flight incidents: A case study of Alaska Airlines Flight 1282 (Doctoral dissertation, University of Hamburg). https://elib.dlr.de/208385/
Brightwell, A., & Grant, J. (2013). Competency-based training: Who benefits?. Postgraduate medical journal, 89(1048), 107-110. https://doi.org/10.1136/postgradmedj-2012-130881
Chen, S., Zhang, C., & Li, W. (2022). The effects of competency-based training model in the training of new nurses: A meta-analysis and systematic review. PLoS One, 17(11), e0277484. https://doi.org/10.1371/journal.pone.0277484
Demirdöken, G., & Atay, D. (2024). Enhancing aviation English competency: A simulation-based approach for aspiring pilots. English for Specific Purposes, 76, 106-121. https://doi.org/10.1016/j.esp.2024.08.001
Drayton, J. (2021). The vocabulary of aviation radiotelephony communication in simulator emergencies and the contradictions in air traffic controller beliefs about language use (Doctoral dissertation, Open Access Te Herenga Waka-Victoria University of Wellington). https://shorturl.at/2kS2r
Emery, H. (2025). Role play in aviation language training and assessment. Teaching and Assessment in Global Aviation English, 55. https://shorturl.at/fV8wB
Estival, D. (2016). Aviation English: A linguistic description. In Aviation English (pp. 22-53). Routledge. https://doi.org/10.4324/9781315661179
Fagbohun, O., Iduwe, N. P., Abdullahi, M., Ifaturoti, A., & Nwanna, O. M. (2024). Beyond traditional assessment: Exploring the impact of large language models on grading practices. Journal of Artificial Intelligence, Machine Learning and Data Science, 2(1), 1-8. https://doi.org/10.51219/JAIMLD/Oluwole-fagbohun/19
Fatmawati, F., & Rochmawati, L. (2022). TOEIC-readiness and TOEIC mock tests: measuring Civil Aviation Polytechnic of Surabaya Cadets’ Preparedness for the TOEIC Test. Jurnal Penelitian, 7(4), 312-322. https://doi.org/10.46491/jp.v7i4.1187
Forcella, C. (2021). Fatigue and human factor in aviation (Doctoral dissertation, Politecnico di Torino). http://webthesis.biblio.polito.it/id/eprint/18368
Fountoulakis, M. S. (2024). Optimizing police and military academy training: Implementing a tiered education program through comprehensive cadet assessments. International Journal of Scientific Research in Science and Technology, 11(3), 705-721. https://dx.doi.org/10.2139/ssrn.4930069
Hayes, P., Bearman, C., Butler, P., & Owen, C. (2021). Non‐technical skills for emergency incident management teams: A literature review. Journal of Contingencies and Crisis Management, 29(2), 185-203. https://doi.org/10.1111/1468-5973.12341
Helal, H. (2022, June). Incorporating virtual reality into maritime safety training to enhance competency-based learning outcomes. In Maritime Transport Conference (No. 9). Universitat Politècnica de Catalunya. Iniciativa Digital Politècnica. http://dx.doi.org/10.5821/mt.11414
Herasymenko, L., Muravska, S., Lomakina, M., Martynenko, N., & Mazurenko, I. (2021). Reticence is not a virtue in aviation: improving English speaking proficiency of future aviation specialists. Revista Romaneasca Pentru Educatie Multidimensionala, 13(2), 498-515. https://dspace.sfa.org.ua:80/handle/123456789/1494
Hutchinson, T., & Waters, A. (1987). English for specific purposes. Cambridge university press. https://shorturl.at/wbofu
Keller, J., Mendonca, F., Cutter, J., Suckow, M., & Dillman, B. (2020). Justification and development of competencies to transform a collegiate aviation flight program. The Journal of Competency‐Based Education, 5(3), e01216. https://doi.org/10.1002/cbe2.1216
Klír, R., & Semančíkovâ, M. (2024, November). ICAO standards for language skills and current language ability of students of aeronautical faculty. In 2024 New Trends in Aviation Development (NTAD) (pp. 84-90). IEEE. https://doi.org/10.1109/NTAD63796.2024.10850349
Knoch, U. (2014). Using subject specialists to validate an ESP rating scale: The case of the International Civil Aviation Organization (ICAO) rating scale. English for Specific Purposes, 33, 77-86. https://doi.org/10.1016/j.esp.2013.08.002
Koo, T. K., & Li, M. Y. (2016). A guideline of selecting and reporting intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine, 15(2), 155–163. https://doi.org/10.1016/j.jcm.2016.02.012
Kusumayati, L. D., Putra, I., Rochmawati, L., Wiguna, I. W. Y. M., & Febiyanti, H. (2023, March). Dictogloss technique in aviation English II: Basis for developing cadet listening skills (a classroom action research at air transportation management batch VII Bravo, Civil Aviation Polytechnic of Surabaya). In International Conference of Advance Transportation, Engineering, and Applied Science (Vol. 1, No. 1, pp. 10-16). https://doi.org/10.46491/icateas.v1i1.1320
Lawshe, C. H. (1975). A quantitative approach to content validity. Personnel psychology, 28(4). https://shorturl.at/PZ4yv
Molenda, M. (2003). In search of the elusive ADDIE model. Performance improvement, 42(5), 34-37. https://bit.ly/3T1SIN3
Monteiro, A. L. T., & Bullock, N. V. (2020). A broader view of communicative competence for aeronautical communications: Implications for teaching and high-stakes testing. The ESPecialist, 41(3). https://doi.org/10.23925/2318-7115.2020v41i3a4
Park, M. (2020). Investigating target tasks, task phases, and indigenous criteria for military aviation English assessment. Language Assessment Quarterly, 17(4), 337-361. http://dx.doi.org/10.1080/15434303.2020.1799221
Pichaiyutthakit, C., & Tangkiengsirisin, S. (2023). Challenges and needs in English communication development among air cadets in Thailand (Doctoral dissertation, Thammasat University). https://bit.ly/43mZQtv
Pratama, R.D., & Imaniar, F. (2024). Elementary English for aviation community. AE Publishing
Reale, C., Salwei, M. E., Militello, L. G., Weinger, M. B., Burden, A., Sushereba, C., ... & Anders, S. (2023). Decision-making during high-risk events: a systematic literature review. Journal of cognitive engineering and decision making, 17(2), 188-212. https://doi.org/10.1177/15553434221147415
Rifenburg, J. M. (2022). Drilled to write: becoming a cadet writer at a senior military college. University Press of Colorado. https://bit.ly/3FzPbCG
Robinson, P. A. (2019). Writing and designing manuals and warnings. CRC Press.
Rochmawati, L., Fatmawati, & Sukma, M. M. (2023). Motivation, anxiety, and self-efficacy in learning aviation English: a study of Indonesian aviation cadets. Asian-Pacific Journal of Second and Foreign Language Education, 8(1), 40. https://doi.org/10.1186/s40862-023-00212-6
Rochmawati, L., Sylvia, T., Kusumayati, L. D., & Silvia, M. A. (2024). Communicative language teaching in aviation vocational college context: the effect of students’ learning perception and attitude on their EFL learning performance. Journal of Foreign Language Teaching and Learning, 9(1), 19-40. https://doi.org/10.18196/ftl.v9i1.20789
Siraranghom, W. (2020). The needs of air cadets in an English speaking course. The New English Teacher, 14(2), 139-139. https://bit.ly/454wx02
Sirikanjanawong, N., & Wasanasomsithi, P. (2018). Relationship between the ICAO Language Proficiency Requirements (LPRs) and Test of English for International Communication (TOEIC) Scores of Flight Attendants in Thailand. LEARN Journal: Language Education and Acquisition Research Network, 11(1), 64-86. https://so04.tci-thaijo.org/index.php/LEARN/article/view/135870
Sulistyorini, D. (2023). Machine images as media to improve cadet’s ability in writing descriptive texts. Education and Linguistics Knowledge Journal, 5(1), 104-115. https://doi.org/10.32503/edulink.v5i1.3593
Suprapti, F. (2024). The influence of competency-based training on the readiness of nautical cadets in the maritime industry. Journal of Education Research, 5(4), 6122-6130. https://doi.org/10.37985/jer.v5i4.1941
Tetiana, P. (2024, January). Enhancing military English learning through multimodal memory approaches. In The 2nd International scientific and practical conference “Innovations in education: prospects and challenges of today”(January 16-19, 2024) Sofia, Bulgaria. International Science Group. 2024. 389 p. (p. 200). https://bit.ly/4kjpvsW
Whyte, S. (2019). Revisiting communicative competence in the teaching and assessment of language for specific purposes. Language Education & Assessment, 2(1), 1-19. https://dx.doi.org/10.29140/lea.v2n1.33
Widiyanto, S. (2023, November). The implementation of ICAO and civil aviation regulation in vocational higher education. In Proceeding of International Conference of Advanced Transportation, Engineering, and Applied Social Science (Vol. 2, No. 1, pp. 895-899). https://doi.org/10.46491/icateas.v2i1.1757
Zhang, Y., & Luo, S. (2021). The impact of scenario-based settings on cognitive reading-to-write processes. International Journal of TESOL Studies, 3(2), 23-40. https://doi.org/10.46451/ijts.2021.06.03
Downloads
Published
Issue
Section
Citation Check
License
Copyright (c) 2025 Raga Driyan Pratama, Nur Mukminatien, Fatmawati, Suhono
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under an Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See the Effect of Open Access)
