Abstracts
Abstract
This experimental study investigates the impact of ChatGPT-simplified authentic texts on university students’ reading comprehension, inferencing, and reading anxiety levels. A within-subjects design was employed, and 105 undergraduate English as a foreign language (EFL) students engaged in both original and ChatGPT-simplified text readings, serving as their own controls. The findings reveal a significant improvement in reading comprehension scores and inferencing scores following ChatGPT intervention. However, no significant change in reading anxiety levels was observed. Results suggest that ChatGPT simplification positively influences reading comprehension and inferencing, but its impact on reading anxiety remains inconclusive. This research contributes to literature on the use of artificial intelligence (AI) in education and sheds light on ChatGPT’s potential to influence language learning experiences within higher education contexts. The study highlights the practical application of ChatGPT as a tool for helping students engage in authentic text readings by making text more comprehensible. Based on the findings, several multifaceted implications that extend to various stakeholders in the field of language education are provided.
Keywords:
- artificial intelligence,
- ChatGPT,
- simplification,
- reading,
- language teaching
Appendices
Bibliography
- AbdulHussein, F. R. (2014). Investigating EFL college teachers’ and learners’ attitudes toward using authentic reading materials in Misan. Procedia—Social and Behavioral Sciences, 136, 330–343. https://doi.org/10.1016/j.sbspro.2014.05.338
- Bayat, N., & Çetinkaya, G. (2020). The relationship between inference skills and reading comprehension. Education and Science, 45(203), 177–190. https://doi.org/10.15390/EB.2020.8782
- Bin-Hady, W. R. A., Al-Kadi, A., Hazaea, A., & Ali, J. K. M. (2023). Exploring the dimensions of ChatGPT in English language learning: A global perspective. Library Hi Tech. Advance online publication. https://doi.org/10.1108/LHT-05-2023-0200
- Brown, H., Lee, K., Mireshghallah, F., Shokri, R., & Tramèr, F. (2022, June 21–24). What does it mean for a language model to preserve privacy? In FaccT ’22: Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency (pp. 2280–2292). Association for Computing Machinery. https://doi.org/10.1145/3531146.3534642
- Cain, K., Oakhill, J. V., Barnes, M. A., & Bryant, P. E. (2001). Comprehension skill, inference-making ability, and their relation to knowledge. Memory & Cognition, 29(6), 850–859. https://doi.org/10.3758/BF03196414
- Chen, H.-R., & Hsu, W.-C. (2022). Do flipped learning and adaptive instruction improve student learning outcome? A case study of a computer programming course in Taiwan. Frontiers in Psychology, 12, Article 768183. https://doi.org/10.3389/fpsyg.2021.768183
- Chen, X., Xie, H., Zou, D., & Hwang, G.-J. (2020). Application and theory gaps during the rise of artificial intelligence in education. Computers and Education: Artificial Intelligence, 1, Article 100002. https://doi.org/10.1016/j.caeai.2020.100002
- Chen, Y., Chen, Y., & Heffernan, N. (2020). Personalized math tutoring with a conversational agent (arXiv preprint: arXiv:2012.12121).
- Chow, B. W.-Y., Mo, J., & Dong, Y. (2021). Roles of reading anxiety and working memory in reading comprehension in English as a second language. Learning and Individual Differences, 92, Article 102092. https://doi.org/10.1016/j.lindif.2021.102092.
- Corpus of Contemporary American English (COCA). (n.d.). Retrieved January 27, 2024, from https://www.english-corpora.org/coca/
- Crossley, S. A., McCarthy, P. M., Louwerse, M. M., & McNamara, D. S. (2007). A linguistic analysis of simplified and authentic texts. The Modern Language Journal, 91, 15–30. https://doi.org/10.1111/j.1540-4781.2007.00507.x
- Crum, A. J., Salovey, P., & Achor, S. (2013). Rethinking stress: The role of mindsets in determining the stress response. Journal of Personality and Social Psychology, 104(4), 716–733. https://doi.org/10.1037/a0031201
- DeBacker, T. K., Heddy, B. C., Kershen, J. L., Crowson, H. M., Looney, K., & Goldman, J. A. (2018). Effects of a one-shot growth mindset intervention on beliefs about intelligence and achievement goals. Educational Psychology, 38(6), 711–733. https://doi.org/10.1080/01443410.2018.1426833
- Dewi, P. H. R., & Pramerta, G. P. A. (2021). Correlation between anxiety and reading comprehension: A study in a secondary school. Academic Journal on English Studies, 1(2), 152–161. https://e-journal.unmas.ac.id/index.php/ajoes/article/view/4606
- Dida, H. A., Chakravarthy, D. S. K., & Rabbi, F. (2023). ChatGPT and big data: Enhancing text-to-speech conversion. Mesopotamian Journal of Big Data, 2023, 33–37. https://doi.org/10.58496/MJBD/2023/005
- Dwivedi, Y. K., Kshetri, N., Hughes, L., Slade, E. L., Jeyaraj, A., Kar, A. K., Baabdullah, A.M., Koohang, A., Raghavan, V., Ahuja, M., Albanna, H., Albashrawi, M.A., Al-Busaidi, A.S., Balakrishnan, J., Barlette, Y., Basu, S., Bose, I., Brooks, L., Buhalis, D., Carter, L., & Wright, R. (2023). “So what if ChatGPT wrote it?” Multidisciplinary perspectives on opportunities, challenges and implications of generative conversational AI for research, practice and policy. International Journal of Information Management, 71, 102642. https://doi.org/10.1016/j.ijinfomgt.2023.102642
- FeedSpot. (2024, May 20). 100 best life blogs and websites in 2024. Lifestyle Bloggers Database. https://lifestyle.feedspot.com/life_blogs/
- Gashti, Y. B. (2018). The effect of authentic and simplified literary texts on the reading comprehension of Iranian advanced EFL learners. Iranian Journal of English for Academic Purposes, 7(2), 32–44. https://e-journal.unmas.ac.id/index.php/ajoes/article/view/4606
- Graesser, A. C., Singer, M., & Trabasso, T. (1994). Constructing inferences during narrative text comprehension. Psychological Review, 101(3), 371–395. https://doi.org/10.1037/0033-295X.101.3.371
- Haastrup, K. (2008). Lexical inferencing procedures in two languages. In D. Albrechtsen, K. Haastrup, & B. Henriksen (Eds.), Vocabulary and writing in a first and second language: Processes and development (pp. 67–111). Palgrave Macmillan. https://doi.org/10.1057/9780230593404_3
- Holmes, W., Bialik, M., & Fadel, C. (2023). Artificial intelligence in education. In C. Stückelberger & P. Duggal (Eds.), Data ethics: Building trust: How digital technologies can serve humanity (pp. 621–653). Globethics Publications. https://doi.org/10.58863/20.500.12424/4276068
- Hong, W. C. H. (2023). The impact of ChatGPT on foreign language teaching and learning: Opportunities in education and research. Journal of Educational Technology and Innovation, 5(1), 37–45. https://jeti.thewsu.org/index.php/cieti/article/view/103
- Hu, H. C. M., & Nassaji, H. (2014). Lexical inferencing strategies: The case of successful versus less successful inferencers. System, 45, 27–38. https://doi.org/10.1016/j.system.2014.04.004
- Hwang, G. J., Xie, H., Wah, B. W., Gasevic, D. (2020). Vision, challenges, roles and research issues of artificial intelligence in education. Computers and Education: Artificial Intelligence, 1, Article 100001. https://doi.org/10.1016/j.caeai.2020.100001
- Jalongo, M. R., & Hirsh, R. A. (2010). Understanding reading anxiety: New insights from neuroscience. Early Childhood Education Journal, 37(6), 431–435. https://doi.org/10.1007/s10643-010-0381-5
- Jeon, J., Lee, S., & Choe, H. (2023). Beyond ChatGPT: A conceptual framework and systematic review of speech-recognition chatbots for language learning. Computers & Education, 206, Article 104898. https://doi.org/10.1016/j.compedu.2023.104898
- Kasneci, E., Seßler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., Krusche, S., Kutyniok, G., Michaeli, T., Nerdel, C., Pfeffer, J., Poquet, O., Sailer, M., Schmidt, A., Seidel, T., Stadler, M., … Kasneci, G. (2023). ChatGPT for good? On opportunities and challenges of large language models for education. Learning and Individual Differences, 103, Article 102274. https://doi.org/10.1016/j.lindif.2023.102274
- Keren, G. (2014). Between- or within-subjects design: A methodological dilemma. In G. Keren & C. Lewis (Eds.), A handbook for data analysis in the behavioral sciences: Vol. 1. Methodological issues (pp. 257–272). Psychology Press. https://doi.org/10.4324/9781315799582-10
- Kim, S., Park, J., & Lee, H. (2019). Automated essay scoring using a deep learning model. Journal of Educational Technology Development and Exchange, 2(1), 1–17.
- Kispal, A. (2008). Effective teaching of inference skills for reading: Literature review (Research report DCSFRR031). National Foundation for Educational Research. https://eric.ed.gov/?id=ED501868
- Kohnke, L., Moorhouse, B. L., & Zou, D. (2023). ChatGPT for language teaching and learning. RELC Journal, 54(2), 537–550. https://doi.org/10.1177/00336882231162868
- Laufer, B. (2020). Lexical coverages, inference unknown words and reading comprehension: How are they related? TESOL Quarterly, 54(4), 1076–1085. https://doi.org/10.1002/tesq.3004
- Lottridge, S. M., Nicewander, W. A., & Mitzel, H. C. (2011). A comparison of paper and online tests using a within-subjects design and propensity score matching study. Multivariate Behavioral Research, 46(3), 544–566. https://doi.org/10.1080/00273171.2011.569408
- Lund, B. D., & Wang, T. (2023). Chatting about ChatGPT: How may AI and GPT impact academia and libraries? Library Hi Tech News, 40(3), 26–29. https://doi.org/10.1108/LHTN-01-2023-0009
- Meymeh, M. H., Rashtchi, M., & Mohseni, A. (2019). Anxiety effect: A case of text modification and the effect of high and low anxiety levels on medical students’ comprehension performance. Journal of Paramedical Sciences, 10(1), 20–26. https://doi.org/10.22037/jps.v10i1.24111
- Mhlanga, D. (2023, February 11). Open AI in education, the responsible and ethical use of ChatGPT towards lifelong learning. SSRN. https://doi.org/10.2139/ssrn.4354422
- Mikami, H. (2019). Reading anxiety scales: Do they measure the same construct? Reading in a Foreign Language, 31(2), 249–268. https://eric.ed.gov/?id=EJ1232213
- Oakhill, J., & Cain, K. (2007). Issues of causality in children’s reading comprehension. In D. S. McNamara (Ed.), Reading comprehension strategies: Theories, interventions, and technologies (pp. 47–72). Lawrence Erlbaum Associates Publishers.
- Ouyang, F., & Jiao, P. (2021). Artificial intelligence in education: The three paradigms. Computers and Education: Artificial Intelligence, 2, Article 100020. https://doi.org/10.1016/j.caeai.2021.100020
- Pang, G., Shen, C., Cao, L., Hengel, A. V. D. (2021). Deep learning for anomaly detection: A review. ACM Computing Surveys, 54(2), 1–38. https://doi.org/10.1145/3439950
- Pavlik, J. V. (2023). Collaborating with ChatGPT: Considering the implications of generative artificial intelligence for journalism and media education. Journalism & Mass Communication Educator, 78(1), 84–93. https://doi.org/10.1177/10776958221149577
- Pogla, M. (2023, April 24). ChatGPT: Optimizing language models for dialogue. AutoGPT. https://autogpt.net/chatgpt-optimizing-language-models-for-dialogue/
- Rospigliosi, P. A. (2023). Artificial intelligence in teaching and learning: What questions should we ask of ChatGPT? Interactive Learning Environments, 31(1), 1–3. https://doi.org/10.1080/10494820.2023.2180191
- Saito, Y., Garza, T. J., & Horwitz, E. K. (1999). Foreign language reading anxiety. Modern Language Journal, 83(2), 202–218. https://doi.org/10.1111/0026-7902.00016
- Soma, R., Mukminin, A., & Noprival, A. M. (2015). Toward a better preparation of student teachers’ reading skill: The SQ3R strategy with authentic and simplified texts on reading literacy and vocabulary mastery. Journal of Education and Learning, 9(2), 125–134. https://doi.org/10.11591/edulearn.v9i2.1527
- Sullivan, M., Kelly, A., & McLaughlan, P. (2023). ChatGPT in higher education: Considerations for academic integrity and student learning. Journal of Applied Learning and Teaching, 6(1), 31–40. https://doi.org/10.37074/jalt.2023.6.1.17
- Susnjak, T., & Maddigan, P. (2023). Forecasting patient flows with pandemic induced concept drift using explainable machine learning. EPJ Data Science, 12, Article 11. https://doi.org/10.1140/epjds/s13688-023-00387-5
- Taecharungroj, V. (2023). What can ChatGPT do? Analyzing early reactions to the innovative AI chatbot on Twitter. Big Data and Cognitive Computing, 7(1), Article 35. https://doi.org/10.3390/bdcc7010035
- Tlili, A., Shehata, B., Adarkwah, M. A., Bozkurt, A., Hickey, D. T., Huang, R., & Agyemang, B. (2023). What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education. Smart Learning Environments, 10, Article 15. https://doi.org/10.1186/s40561-023-00237-x
- Tomlinson, B. (2004). Materials development in language teaching (2nd ed.). Cambridge University Press. https://doi.org/10.1017/9781139042789
- Walton, G. M., & Cohen, G. L. (2011). A brief social-belonging intervention improves academic and health outcomes of minority students. Science, 331(6023), 1447–1451. https://doi.org/10.1126/science.1198364
- Wesche, P. M. B., & Paribakht, P. T. S. (2009). Lexical inferencing in a first and second language: Cross-linguistic dimensions. Multilingual Matters. https://doi.org/10.21832/9781847692245
- Yang, S. J., Ogata, H., Matsui, T., & Chen, N. S. (2021). Human-centered artificial intelligence in education: Seeing the invisible through the visible. Computers and Education: Artificial Intelligence, 2, Article 100008. https://doi.org/10.1016/j.caeai.2021.100008
- Yi, Y., Cho, S., & Jang, J. (2022). Methodological innovations in examining digital literacies in applied linguistics research. TESOL Quarterly, 56(3), 1052–1062. https://doi.org/10.1002/tesq.3140