Abstracts
Abstract
Understanding factors promoting or preventing participants’ completion of a massive open online course (MOOC) is an important research topic, as attrition rates remain high for this environment. Motivation and digital skills have been identified as aspects promoting student engagement in a MOOC, and they are considered necessary for success. However, evaluation of these factors has often relied on tools for which the psychometric properties have not been explored; this suggests that researchers may be working with potentially inaccurate information for judging participants’ profiles. Through a set of analyses (t-test, exploratory factor analysis, correlation), this study explores the relationship between information collected by administering valid and reliable pre and post instruments to measure traits of MOOC attendees. The findings from this study support previously reported outcomes concerning the strong relationships among motivation, previous knowledge, and perceived satisfaction factors for MOOC completers. Moreover, this study provides evidence of the feasibility of developing valid assessments for evaluation purposes.
Keywords:
- MOOC assessment,
- exploratory factor analysis,
- assessment validity
Appendices
Bibliography
- Admiraal W, Huisman B., & Pilli, O. (2015). Assessment in massive open online courses. The Electronic Journal of e-Learning, 13(4), 207-216.
- Aesaert, K., van Nijlen, D., Vanderlinde, R., & van Braak, J. (2014). Direct measures of digital information processing and communication skills in primary education: Using item response theory for the development and validation of an ICT competence scale. Computers & Education, 76, 168-181. doi: 10.1016/j.compedu.2014.03.013
- Aiken, J. M., Lin, S., Douglas, S. S., Greco, E. F., Thoms, B. D., Schatz, M. F., & Caballero, M. D. (2013). The initial state of students taking an introductory physics MOOC. Proceedings of the 2013 Physics Education Research Conference, Portland, OR. doi: 10.1119/perc.2013.pr.001
- Alario-Hoyos, C., Pérez-Sanagustín, M., Delgado-Kloos, C., Parada, H. A., & Muñoz-Organero, M. (2014). Delving into participants' profiles and use of social tools in MOOCs. IEEE Transactions on Learning Technologies, 7(3), 260-266. doi: 10.1109/TLT.2014.2311807
- Barak, M., Watted. A., & Haick, H. (2016). Motivation to learn in massive open online courses: Examining aspects of language and social engagement. Computers & Education, 94, 49-60. doi: 10.1016/j.compedu.2015.11.010
- Barcena, E., Martin-Monje, E., & Read, T. (2015). Potentiating the human dimension in Language MOOCs. Proceedings of the European MOOC Stakeholder Summit, Mons, Belgium, 46-54.
- Blackmore, K. (2014). Measures of success: Varying intention and participation in MOOCs. Paper presented at the ACS Annual Canberra Conference, Canberra, Australia.
- Castaño-Muñoz, J., Kreijns, K., Kalz, M., & Punie, Y. (2017). Does digital competence and occupational setting influence MOOC participation? Evidence from a cross-course survey. Journal of Computing in Higher Education, 29, 28-46. doi: 10.1007/s12528-016-9123-z
- Chen, J.C. (2013). Opportunities and challenges of MOOCS: Perspectives from Asia. Proceedings of the IFLA World Library and Information Congress 79th IFLA General Conference and Assembly (pp. 1-16), Singapore: The International Federation of Library Associations and Institutions.
- Chudzicki, C., Chen, Z., Zhou, Q., Alexandron, G., & Pritchard, D. E. (2015). Validating the pre/post-test in a MOOC environment. In A. D. Churukian, D. L. Jones, & L. Ding (Eds.), Proceedings of the 2015 Physics Education Research Conference (pp. 83-86). Maryland: American Association of Physics Teachers. doi: 10.1119/perc.2015.pr.016
- Daradoumis, T., Bassi, R., Xhafa, F., & Caballe, S. (2013). A review of massive e-learning (MOOC) design, delivery and assessment. Paper presented at the Eighth International Conference on P2P, Parallel, Grid, Cloud and Internet Computing, Compiegne, France. doi: 10.1109/3PGCIC.2013.37
- de Barba, P. G., Kennedy, G. E., & Ainley, M. D. (2016). The role of students' motivation and participation in predicting performance in a MOOC. Journal of Computer Assisted Learning, 32, 218-231. doi: 10.1111/jcal.12130
- Diver, P., & Martinez, I. (2015). MOOCs as a massive research laboratory: Opportunities and challenges. Distance Education, 36(1), 5-25. doi: 10.1080/01587919.2015.1019968
- Dray, B. J., Lowenthal, P. R., Miszkiewicz, M. J., Ruiz-Primo, M. A., & Marczynski, K. (2011). Developing an instrument to assess student readiness for online learning: A validation study. Distance Education, 32(1), 29-47. doi: 10.1080/01587919.2011.565496
- Durksen, T. L., Chu, M., Ahmad, Z. F., Radil, A. I., & Daniels, L. M. (2016). Motivation in a MOOC: A probabilistic analysis of online learners' basic psychological needs. Social Psychology of Education, 19, 241-260. doi: 10.1007/s11218-015-9331-9
- Farid, A. (2014). Student online readiness assessment tools: A systematic review approach. Electronic Journal of e-Learning, 12(4), 375-382.
- Field, A. (2009). Discovering statistics using SPSS (3rd ed.). London, UK: SAGE Publications.
- Gaebel, M. (2013). MOOCs - Massive open online courses. Report of the European University Association (EUA). Retrieved from http://www.eua.be/Libraries/publication/EUA_Occasional_papers_MOOCs.pdf?sfvrsn=2
- Garrido, M., Koepke, L., Andersen, S., Mena, A., Macapagal, M., & Dalvit, L. (2016). An examination of MOOC usage for professional workforce development outcomes in Colombia, the Philippines, & South Africa. Seattle, WA: Technology & Social Change Group, University of Washington Information School.
- Horzum, M. B., Kaymak, Z. D., & Gungoren, O. C. (2015). Structural equation modeling towards online learning readiness, academic motivations, and perceived learning. Educational Sciences: Theory & Practice, 15(3), 759-770. doi: 10.12738/estp.2015.3.2410
- Jordan, K. (2015). Massive open online course completion rates revisited: Assessment, length and attrition. International Review of Research in Open and Distributed Learning, 16(3), 341-358.
- Kaiser, H. F. (1974). An index of factorial simplicity. Psychometrika, 39, 31-36.
- Kennedy, G., Coffrin, C., & de Barba, P. (2015.). Predicting success: How learners' prior knowledge, skills and activities predict MOOC performance. Proceedings of the Fifth International Conference on Learning Analytics and Knowledge (pp. 136-140). Poughkeepsie, NY: ACM Publications. doi: 10.1145/2723576.2723593
- Kennedy, J. (2014). Characteristics of massive open online courses (MOOCs): A research review, 2009-2012. Journal of Interactive Online Learning, 13(1), 1-16.
- Kerr, J., Houston, S., Marks. L., & Richford, A. (2015). Building and executing MOOCs: A practical review of Glasgow's first two MOOCs (Massive Open Online Courses ). Glasgow, Scotland: University of Glasgow. Retrieved from http://www.gla.ac.uk/media/media_395337_en.pdf
- Kilgore, W., Bartoletti, R., & Freih, M. A. (2015). Design intent and iteration: The #HumanMOOC. Proceedings of the European MOOC Stakeholder Summit (pp. 7-12). Mons, Belgium: 2015 eMOOCS conference committee.
- Kline, R. B. (2005). Principles and practice of structural equation modeling. New York, NY: Guilford Press.
- Li, Q., & Baker, R. (2016). Understanding engagement in MOOCs. In T. Barnes, M. Chi, & M. Feng (Eds.) Proceedings of the 9th International Conference on Educational Data Mining (pp. 605-606). Raleigh, NC; International Conference on Educational Data Mining (EDM) 2016.
- Littlejohn, A., Hood, N., Milligan, C., & Mustain, P. (2016). Learning in MOOCs: Motivations and self-regulated learning in MOOCs. Internet and Higher Education, 29, 40-48. doi: 10.1016/j.iheduc.2015.12.003
- Lui, J., & Li, H. (2017). Exploring the relationship between student pre-knowledge and engagement in MOOCs using polytomous IRT. In X. Hu, T. Barnes, A. Hershkovitz, & L, Paquette (Eds.) Proceedings of the 10th International Conference on Educational Data Mining (pp. 410-411). Wuhan, China: International Conference on Educational Data Mining (EDM) 2017.
- Magen-Nagar, N., & Cohen, L. (2017). Learning strategies as a mediator for motivation and a sense of achievement among students who study in MOOCs. Education and Information Technologies, 22, 1271-1290. doi: 10.1007/s10639-016-9492-y
- Mazoue, J. G. (2013). The MOOC model: Challenging traditional education. Formamente, 8(1-2), 161-174.
- McGreal, R., Mackintosh, W., & Taylor, J. (2013). Open educational resources university: An assessment and credit for students initiative. In R. McGregor, W. Kinutia, & S. Marshall (Eds.), Open educational resources: Innovation research and practices (pp. 47-59). Vancouver, BC: Commonwealth of Learning.
- Onah, D.F., Sinclair, J., & Boyatt, R. (2014). Dropout rates of massive open online courses: Behavioural patterns. Proceedings of the 6th International Conference on Education and New Learning Technologies (EDULEARN14), Barcelona, Spain. doi: 10.13140/RG.2.1.2402.0009
- Pegler, C. (2012). Reuse and repurposing of online digital learning resources within UK higher education: 2003-2010. (Unpublished doctoral dissertation). The Open University. United Kingdom. Retrieved from http://oro.open.ac.uk/32317/1/Pegler_PhD_final_print_copy.pdf
- Perna, L. W., Ruby, A., Boruch, R. F., Wang, N., Scull, J., Ahmad, S., & Evans, C. (2014). Moving through MOOCs: Understanding the progression of users in massive open online courses. Educational Researcher, 43(9), 421-432. doi: 10.3102/0013189X14562423
- Petronzi, D., & Hadi, M. (2016). Exploring the factors associated with MOOC engagement, retention and the wider benefits for learners. European Journal of Open, Distance and e-Learning, 19(2), 129-146.
- Pursel, B. K., Zhang, L., Jablokow, K. W., Choi. G. W., & Velegol, D. (2016). Understanding MOOC students: Motivations and behaviours indicative of MOOC completion. Journal of Computer Assisted Learning, 32, 202-217. doi: 10.1111/jcal.12131
- Rayyan, A., Fredericks, C., Colvin, K. F., Liu, A., Teodorescu, R., Barrantes, A., ... Pritchard, D. E. (2016). A MOOC based on blended pedagogy. Journal of Computer Assisted Learning, 32, 190-201. doi: 10.1111/jcal.12126
- Rivera, N., & Ramirez, M. S. (2015). Digital skills development: MOOC as a tool for teacher training. Proceedings from International Conference of Education, Research, and Innovation (ICERI2015), Seville, Spain.
- Sanchez-Acosta, E., Escribano-Otero, J. J., & Valderrama, F. (2014). Motivación en la educación masiva online. Desarrollo y experimentación de un sistema de acreditaciones para los MOOC. [Motivation in massive education online: Development and testing of a system of accreditation badges for MOOC]. Digital Education Review, 25, 18-35.
- Shapiro, H. B., Lee, C. H., Wyman Roth, N. E., Li, K., Cetinkaya-Rundel, M., & Canelas, D. A. (2017). Understanding the massive open online course (MOOC) student experience: An examination of attitudes, motivations, and barriers. Computers & Education, 110, 35-50. doi: 10.1016/j.compedu.2017.03.003
- Stevanovic, N. (2014). Effects of motivation on performance of students in MOOC. Paper presented at Sinteza 2014 - Impact of the Internet on Business Activities in Serbia and Worldwide, Belgrade, Serbia. doi: 10.15308/sinteza-2014-418-422
- Tecnológico de Monterrey (Producer). (2017). La reforma energetica y sus oportunidades: MOOC. [The energy reform and its opportunities: MOOC]. Retrieved from https://energialab.tec.mx/es/la-reforma-energetica-y-sus-oportunidades
- Thille, C., Scheneider, R.F., Piech, C., Halawa, S.A., & Greene, D.K. (2014). The future of data-enriched assessment. Research and Practice in Assessment, 9, 5-16.
- Valdivia, J.A., Valenzuela, J.R., & Ramírez-Montoya, M.S. (2017). Pilotaje de contenido de instrumentos MOOC [Content pilot for MOOC instruments]. Retrieved from https://repositorio.itesm.mx/bitstream/handle/11285/622345/170210-Reporte%20Pilotaje%20Instrumentos.pdf?sequence=1&isAllowed=y
- Valdivia Vázquez, J.A., Valenzuela, J.R., & Ramirez-Montoya, M.S. (2017). Encuesta inicial sobre intereses, motivaciones y conocimientos previos en MOOC: Reporte de validación y confiabilidad. [Initial assessment to evaluate interests, motivation and previous knowledge: Validity and reliability report]. Retrieved from https://repositorio.itesm.mx/bitstream/handle/11285/622442/Encuesta_inicial_motivacioonesMOOC.pdf?sequence=1&isAllowed=y
- Valenzuela, J.R., Mena, J.J., & Ramírez-Montoya, M.S. (2017a). Encuesta inicial sobre intereses, motivaciones y conocimientos previos en MOOC [Initial assessment to evaluate interests, motivation and previous knowledge]. Retrieved from https://repositorio.itesm.mx/bitstream/handle/11285/622349/170210-EncuestaInicial.pdf?sequence=1
- Valenzuela, J.R., Mena, J.J., & Ramírez-Montoya, M.S. (2017b). Encuesta final sobre intereses, motivaciones y conocimientos previos en MOOC [Final assessment to evaluate interests, motivation and previous knowledge]. Retrieved from https://repositorio.itesm.mx/bitstream/handle/11285/622348/170210-EncuestaFinal.pdf?sequence=1&isAllowed=y
- Xiong, Y., Li, H., Kornhaber, M. L., Suen, H. K., Pursel, B., & Goins, D. D. (2015). Examining the relations among student motivation, engagement and retention. Global Education Review, 2(3), 23-33.
- Xu, B., & Yang, D. (2016). Motivation classification and grade prediction for MOOCs learners. Computational Intelligence and Neuroscience, 2016, 1-7. doi: 10.1155/2016/2174613
- Yuan, L., & Powell, S. (2013). Open education: Implications for higher education. Retrieved from https://www.researchgate.net/publication/265297666_MOOCs_and_Open_Education_Implications_for_Higher_Education
- Zhang, D., & Nunameker, J. F. (2003). Powering e-learning in the new millennium: Information Systems Frontiers, 5(2), 207-2018.