What If It’s All an Illusion? To What Extent Can We Rely on Self-Reported Data in Open, Online, and Distance Education Systems?

Abstract

Online surveys are widely used in social science research as well as in empirical studies of open, online, and distance education. However, students’ responses are likely to be at odds with their actual behavior. In this context, we examined the discrepancies between self-reported use and actual use (i.e., learning analytics data) among 20,646 students in an open, online, and distance education system. The ratio of consistent responses to each of the 11 questions ranged from 43% to 70%, and the actual access to learning resources was significantly lower than self-reported use. In other words, students over-reported their use of learning resources. Females were more likely to be consistent in their responses. Frequency of visits to the open, online, and distance education system, grade point average, self-reported satisfaction, and age were positively correlated with consistency; students’ current semester was negatively correlated with consistency. Although consistency was not maintained between actual use and self-reported use, consistency was maintained between some of the self-report questionnaires (i.e., use vs. satisfaction). The findings suggested that system and performance data should be considered in addition to self-reported data in order to draw more robust conclusions about the accountability of open, online, and distance education systems.

Keywords:

• open and distance learning,
• higher education,
• self-report,
• inconsistent responding,
• learning analytics