Corps de l’article

Introduction: The Awful Reputation of Educational Research

The quality of educational research has come under considerable scrutiny in recent decades in the United Kingdom and the United States, intensifying in the mid-1990s (Oancea, 2005). Undertaking research in the US in the early 1990s, Kaestle (1993, p. 136) observed, “following interviews with 33 educational researchers and federal agency officials, that education research has ‘an awful reputation’” (as cited in Hyslop-Marginson & Nasseem, 2007, p. 5). As Alis Oancea notes of the UK context, in 1998 alone “about 60 newspaper articles were published on the alleged ‘irrelevance,’ ‘waste of public money’ and ‘low quality’ of educational research” (p. 161). In the UK, the tone of the debate was largely set by the publication of three critical reports, commissioned by the then-Department for Education and Employment (DfEE) and the then-Teacher Training Agency: (1) D. H. Hargreaves’s Teaching as a Research-Based Profession: Possibilities and Prospects (1996/2007), a published version of his Teacher Training Agency Annual Lecture; (2) Tooley and Darby’s Educational Research: a Critique: A Survey of Published Educational Research (1998), written for Ofsted (the Office for Standards in Education, which sets and inspects standards for statutory education and childcare settings and teacher education providers in England); and (3) Excellence in Research on Schools, a report for the DfEE by Hillage, Pearson, Anderson and Tamkin, also in 1998.  In part as a result of these criticisms, and due to the changes that have shaped higher education since that time—for example, increasingly competitive funding regimes, increased focus on the measurable, societal impact of research, and a shift in the role of research, wherein “the production of knowledge” is seen “as an engine of economic prosperity and growth” (Hasselberg et al., 2013, p. 4)—educational research has been at pains to prove its validity as a field, both in terms of its scientific legitimacy and its relevance to policy and practice.

This is not an unprecedented phenomenon. When teacher education moved into universities in the UK in the 1960s, questions about the epistemological basis of educational theory and its relationship to practice were raised. The influence of the logical positivism dominant at the time were invoked by D. J. O’Connor in his 1957 Introduction to Philosophy of Education: “The standards and criteria used to determine what is to count as a genuine theory in science can and should be used to judge the value of the various (and often conflicting) theories that are put forward by writers of education’” (p. 76; as cited in Carr, 2006, p. 140). Philosophy was no longer seen to be a sufficient underpinning for education, and the intellectual legitimacy of the field that would become known as educational studies was shored up—in Anglophone contexts at least—by the more scientific of the foundation disciplines: psychology and sociology. As the existence of university faculties of education was often closely related to the provision of teacher education, changes to the teacher education curriculum have seen philosophy and history—the other foundation disciplines—sidelined in terms of what is deemed valuable for the study of education. 

Shifts in the mode of governance since the late twentieth century, shaped by New Public Management, marketisation, and the development of the global knowledge economy, have further reshaped the role of research and thus of higher education. In this context, parts of educational research shifted from identifying themselves in terms of their disciplinary basis to focus on bringing educational policy and practice in line with wider political and economic thinking. Hence, we saw the development of school effectiveness research, the “what works” agenda, and so on (see, e.g., Ball, 1995 for a critical account of these developments), which continue to shape how the quality and value of educational research is understood. For example, the training of teachers in England currently is shaped by the Teachers’ Standards (Department for Education, 2011), which are underpinned by a body of evidence, the Core Content Framework (CCF), recently updated to incorporate the programme of development set out for newly qualified teachers, or Early Career Teachers (ECTs), to become the ECTCCF (2024). In disciplinary terms, the literature cited in the CCF derives from the fields of economics and psychology, with some subject-specific literature, for example on teaching English (Murdock-Perriera & Sedlacek, 2018) or maths (Jerrim & Vignoles, 2016). As such, methodologically, the research is often based on quantitative data and analysis in relation to correlations between practices and outcomes, or the effectiveness of particular practices. Educational philosophy and small-scale, contextual qualitative research are less visible.

The concern with effectiveness and outcomes shapes not only how we think about school education and the research that ought to inform it, of course, but also the governance of research itself. The trends we have seen in the UK context echo those in the US where, following the National Research Council report on Scientific Research on Education (2002), Congress opted to restrict funding to research deemed “scientifically rigorous, as defined narrowly by the use of randomized experimental or field trial designs” (Phillips, 2006, p. 19), seen as the “gold standard” methodology for research (p. 20). Other forms of research were not entirely written off but were seen as “subsidiary” to that which employs such methods. It was conceded that “even throat-clearing essays at times contribute to understanding” (Mosteller & Boruch, 2002, p. 2; as cited in Phillips, 2006, p. 21).

In the past twenty years, the privileging of particular forms of educational research has only intensified, with qualitative or theoretical approaches competing according to criteria not necessarily appropriate to their theoretical or educational commitments. Important critiques have been developed, not least within philosophy of education, of the flawed logic of trying to understand education through a “what works” approach (Biesta, 2007, 2010), the limits of empirical inquiry for understanding the existential and ethical complexity of education (e.g., Standish, 2002), the scientism that underpins such a focus, and the preoccupation with methods that has ensued. In what follows, I will explore some of the implications of this focus on method in educational research. Rather than offering a further straightforward defence of the need for philosophical inquiry in education, I suggest that it is perhaps not the privileging of empirical research that is the problem for education, but the particular form this takes. To refocus educational research on educational questions, I turn to Martin Packer’s (2010) account of constitution as an approach to educational research that also brings forth the shared sensibilities of education and anthropology.

Methodolatry

Phillips (2006) captures the privileging of particular methodologies, and the attribution of an inherent rigour and quality to research that uses them, using Arthur Kaplan’s (1964) term “methodolatry.” Phillips identified this as an emerging trait of educational research in the early 2000s, and it has since been commented on in the literature numerous times (e.g., Macfarlane, 2022), and in fields other than education (e.g., Chamberlain, 2000 in the field of health). Changes in the governance of research have in turn impacted the way in which students are introduced to it, such that research methods training is now commonplace and often compulsory for many postgraduate students. Gustavsson (2013) gives the issue a slightly different term from Kaplan: “methodomania.” He attributes it in part to the massification of education necessitating the use of research methods courses and handbooks in order to be able to train increased numbers of students. Such texts and courses present research as a matter of “mastery of a certain set of instruments and methods” on which these fields’ “claims to academic legitimacy” are based (p. 158). Such legitimacy is, he argues, “provided by the methods employed rather than a theoretical question regarding a scientific problem arising in the discipline itself” (p. 158). This is perhaps a particular issue for inherently interdisciplinary or applied fields of study such as education. Gustavsson also identifies nursing and social work programmes as leading students to “think that there exists a scientific methodology, which contains a number of methods … They are rarely aware that most of these methods have arisen within a theoretical context but tend rather to see theory as a product of method” (p. 159).

Many graduate students in philosophy of education will have experienced compulsory research methods courses that assume not only that research will be empirical but also that that is where it derives its value. The concern here is not so much that students might find these courses irrelevant to their own projects, but that questions regarding the disciplinary and epistemological basis of such methods and their application are not given space. It is not that epistemology or ontology are not present in the discussion, but that such discussion is unmoored from the conditions that give rise to them.

Weaver and Snaza (2017), inspired by the work of Edward Said, coin a related term, “‘methodocentrism’: the belief that particular, pre-formed methods can guarantee the validity of an intellectual investigation into the world by factoring out the vicissitudes of the observer’s entanglement with the world” (p. 1056). Drawing on Stengers (1997), Weaver and Snaza argue that methodocentrism—“the belief that the method one chooses to guide research determines its truth, its legitimacy, its validity, and its trustworthiness” (2017, p. 1056)—is about trying to minimize the “risk” of intellectual investigation, an avoidance that ultimately produces bad science. These issues are not, as indicated, restricted to educational research and are illustrative of wider changes to higher education and the governance of research. The focus on learning and the effectiveness of particular pedagogies should be seen in part as “a consequence of a reorganisation of knowledge production that opens the university sector to market mechanisms, with the result of making universities, in effect, suppliers of knowledge within a global knowledge economy” (Hasselberg et al., 2013, p. 2). Elzinga (1985, 1997) has characterised the effect of this as a form of epistemic drift to “a supply and demand definition of quality” (as cited in Hasselberg et al., 2013, p. 4).  

This is not to suggest that small-scale qualitative research is not taking place, nor that it is absent from research methods courses. Indeed, recent research attests to the diversity of the field today, but also to its uncertainty. The British Educational Research Association (BERA) published the outcomes of its State of the Discipline project in 2023. Its summary statement refers to the breadth of the methodological expertise in the field today, but also the persistence of “disagreement, sometimes over very long-standing issues, such as contestation about the merits of qualitative methods versus quantitative methods, and the intersections of these methods with government policies” (BERA, 2023). Of course, the decision between quantitative and qualitative approaches is not one of better or worse but of appropriateness to the commitments and questions at hand, although BERA’s findings suggest that it is this contestation, rather than specifically educational commitments and questions, that often preoccupy educational research. This preoccupation is referred to by Ingold (2018) as a “methodological arms race” powered by “relentless competition for ‘innovation’ and ‘excellence’” (p. 70). 

The rationale for the privileging of particular forms of research and a focus on method in relation to educational research is understandable, both in terms of ensuring that school-focused research provides evidence of “what works” and that graduate students are trained to produce risk-free work within a set time period, if we accept the embeddedness of the idea of the instrumental, governmental purpose of research and of education. As Ingold (2018) notes, this “hardening” of science can be attributed to “its marketisation as the engine of a global knowledge economy” (p. 70). Beyond the binaries of empirical/non-empirical, qualitative/quantitative, and positivist/interpretivist that shape and direct discussion of educational research methods, a further shift seems to have taken place, one that reinforces the risk-aversion and further disables educational research from addressing the complexity of its object of enquiry.

The Adisciplinarity of Educational Research

As Ball (1995) noted, changes in the rationality of governance that saw the introduction of New Public Management in the 1980s shifted attention towards approaches to research that responded to this—for example, school effectiveness research, “what works,” and so on—and away from a discipline-led focus. This has been much lamented, often for good reason, not least by those critical of an instrumentalist, neoliberal understanding of education and by philosophers of education, whose discipline is perhaps seen to be of least value in improving educational outcomes. Such critiques notwithstanding, the response of the field seems to have been a reinforcement of methodolatry and, rather than embracing education’s inherent interdisciplinarity (as it has developed in Anglophone contexts), an embedded adisciplinarity. Packer (2010) hints at the roots of this in reference to the U.S. National Research Council proposals for a balanced, middle way for educational research wherein “research design was driven by one’s question rather than by a priori commitments to a specific methodology” and “where there was room for both quantitative and qualitative methodologies” (p. 19). This middle way, however, was based on “conventional and stereotyped conceptions” of what these methods can do and on a limited range of questions focused on identifying causality; the privileging of particular methodologies in educational research is based on a “very narrow conception of what science is and should be” (p. 19) that derives from logical positivism and the assumption that the metaphysics of science can be removed by sound methodology. This scientific understanding of research as testing hypotheses through the control and measurement of variables also divorces method from theory.

We can also see a shift in the way in which research is framed, away from its disciplinary organisation to a thematic focus. In a 2005 iteration of the guidelines of the Economic and Social Research Council (ESRC), the particular disciplines within its remit are defined individually. The distinction between Education and Social Anthropology was seen to be indicative of a narrowness to the conception of education as a field of research (Hodgson and Standish, 2006). Educational research was defined as including “any enquiry which promotes theoretical and/or empirical social science understanding of educational and/or learning processes and settings, or which informs judgements about educational policy and practice” (ESRC, 2005, F5, 1.1).

While this includes both theoretical and empirical research, it is possible to see that the focus on processes and settings and on informing policy and practice may be narrowly interpreted. The description of “The Nature of the Area” continues:

Educational inquiry draws upon a broad range of theoretical and methodological resources including philosophy and social science disciplines. It may involve specific methods and techniques appropriate to the distinctive nature of educational knowledge and theories and the generation of new methods may itself be a focus of educational research.

ESRC, 2005, F5, 1.2

This appears to be a usefully open definition that enables research in education to be conceived according to multidisciplinary theoretical and methodological approaches and with appreciation of what is distinctive about education. When considered in relation to the guidelines for Social Anthropology, however, the above outline seems limited.

Social Anthropology works with a creative tension between empirical particularity and attention to the broadest theoretical questions about what it means to be a human social agent. Its theory, method and analysis are mutually constitutive. The discipline is noted for its fine-grained empirical detail. Its researchers achieve high levels of linguistic and cultural competence through long periods of fieldwork, complemented by ancillary sources of documentary information. Social anthropologists locate their evidence in as broad a context as possible, and the data they collect usually extend beyond the original focus of interest and specific research topic

ESRC, 2005, F13, 1.3

The contrast suggests limitations on how educational research is seen. The description of Social Anthropology suggests an intensity of experience, of allowing oneself to pursue uncharted directions, and an expectation of scholarly rigour not as evident in the definition of educational research.

In the intervening twenty years, shifts in the governance of research have led to a reorientation in the way research councils identify and represent their rationale for allocating funds, from a disciplinary focus to a thematic focus that requires interdisciplinarity; crudely put, from “What do you need funding for?” to “Can you tell us about what we want to know about?” This is not to suggest that such funding streams were not competitive previously, but this has intensified in the context of a greater focus on accountability, heightened levels of global challenge, and straitened public finances. Hence, today, while each Research Council is still responsible for particular disciplines and subject areas, the focus is on thematic priorities. Those of the ESRC, under whose remit education sits, for example, are the following: climate change and sustainability; data and analysis for decision making; health, wellbeing and social care; politics; population and society; public services; the economy; and understanding the impact of COVID-19.

The question here is not whether a thematic focus is better or worse than a disciplinary one or whether these are the thematic areas we should focus on. The competitive funding model and its privileging of empirical, scientifically recognisable methods, and the need to determine in advance how the research will impact society, have been extensively and rightly criticised from various perspectives. Within philosophy of education, critiques find that such approaches cannot explore questions of value and meaning, that they presuppose an instrumental understanding of education, and, at times, this oversight is attributed to the researchers not having availed themselves of the work of the appropriate classical philosopher. Such critiques can provide important insights into what is left out of the picture by dominant approaches, but too often rest on the assumption of the inherent value of philosophy as such. As Vlieghe (2020) notes, for “more conservative philosophers of education … philosophy is not just a foundational endeavour, but the foundational discipline par excellence” (p. 151). In this assumption, and in the broader concern with method that characterises educational research, what is left out of the picture, often, are educational questions. Vlieghe highlights how the focus on the defence of disciplines in Anglophone education studies, for example, the philosophy of education, or the sociology of education, places the focus on the discipline, not on the practices of education, and entails that theorists “look for a justification of a practice by referring to a standard that is found outside of this practice” (p. 152). Ingold (2018) makes a similar point in Anthropology as Education: “Whenever we invoke the anthropology of this or that, it is as though we run rings around the thing in question, turning the places and the paths from which we observe into circumscribed topics of inquiry” (p. 61). The discipline-led approach that shapes educational studies is particular to Anglophone contexts, in contrast to the specific discipline of pedagogy (though it does not translate directly) as a distinct field of study on a par with other -ologies (Vlieghe, 2020) that developed in Continental Europe. Although the thinkers constitutive of the discipline of pedagogy found in continental European contexts have had some influence on Anglophone educational research, this is not, unsurprisingly, where mainstream educational research sought inspiration when faced with the barrage of criticisms of relevance and quality in the 1990s. As noted above, research shifted with the changing mode of governance to a concern with effectiveness, and quality and rigour were shored up by a focus on methods, resulting in the diagnoses of methodolatry, methodomania, and so on.

Educational Practices and Constitution

The identification of themes by research councils, and the concern among philosophers that empirical research leaves important questions unacknowledged, serves to highlight that perennial questions of how to live, and pressing contemporary questions of how to do so sustainably and justly, are not within the purview of any particular discipline, nor do they immediately seem answerable by reliance on randomised controlled trials. As Packer (2010) puts it:

Today we are in a much stronger position than at any time in the past to articulate the logic of a program of research that explores a more fundamental level of educational phenomena than can be studied using clinical trials. Important theoretical and empirical work across the social sciences and in the humanities—in history, philosophy, linguistics, and literary theory—now enables us to define a program of investigation that is focused on “constitution.”

p. 17

Packer defines constitution “as a relationship of mutual formation between people and their form of life. Neither can exist without the other” (p. 18). The educational questions that the study of constitution is concerned with are not the “what” or “why” of Randomised Control Trials and the like, but the “how.” Packer sets out two conceptions of constitution as it has developed throughout the humanities and social sciences. The first has become a common-sense understanding since its emergence in Kant’s conclusion that “the individual mind constitutes reality by imposing on experience the universal categories of space, time, causality, and object” (p. 21). In the social sciences, this conception would later be taken up by Berger and Luckmann in The Social Construction of Reality (1966). Packer highlights that the issue with such a conception is its epistemological focus on individual knowledge of reality and its aim to be “ontologically mute” (Gergen, 2001, as cited in Packer, 2010, p. 21). The second conception derives from Hegel’s critique of Kant and conceives of “human beings as enmeshed in a material and social world. Each of us is thrown into a world that predates our existence, that offers certain ways to be and that is inherently social” (pp. 21–22). This second conception, which also moved from philosophy into the social, psychological, and learning sciences,

implies that constitution is a visible phenomenon. Neither an occult mental process nor a causal mechanism, it is an aspect of everyday practical activity. As such it can be the object of scientific investigation. This insight provides the basis for a fresh conception of social scientific investigation, using the traditional tools of qualitative inquiry—fieldwork, the detailed study of episodes of interaction, and interviews—but in new ways. They are employed to produce a detailed account of the three aspects—order, ordering, and orderers—that explains how constitution is taking place.

p. 22

This approach to studying the “how” of a form of life, the practices through which it is constituted, also entails attention to what is said, involving “an approach to language that emphasises pragmatics and sees talk as a form of action,” following “Wittgenstein’s notion that ‘language games’ are central aspects of any ‘forms of life’” (p. 23).

Packer articulates the study of constitution with reference to ethnographic fieldwork. The example is drawn upon here not as a random example of how the qualitative methods of another discipline (anthropology) might enliven the theoretical concerns of educational philosophy, interchangeably with any other, but rather to explore how a focus on constitution framed by educational concerns is brought into particular relief by the commitments of anthropology as a discipline. The understanding of ethnography has undergone considerable change since Malinowski’s study of Trobriand Island culture at the turn of the twentieth century (1922/1961). Culture itself is no longer conceived of as “bounded, systematic, and integrated” but as “a dispersed, dynamic, and contested form of life” (Faubion, 2001, as cited in Packer, 2010, pp. 23–24) and the notion of participation that underpins the idea of “participant observation” has been rethought in terms of “complicity” (Marcus, 1997, as cited in Packer, 2010, p. 24): “what the ethnographer witnesses is always a reaction to the foreign place that she represents” (Packer, 2010, p. 24). Rather than obtaining entry to a culture, the researcher continually negotiates access.

To study the order and ordering of a form of life as Packer suggests is not to imply a stability and neatness, for its “objects and concepts … are multiple and contested even though they exist in a common space” (p. 24). Packer illustrates this using the example of the study of a lesson:

A classroom event as simple as “the end of a lesson,” for example, will be dispersed. Does the lesson end when the bell rings, when the teacher dismisses the students, or when the students put their books away? No one doubts that the lesson ends, but the exact moment will be open to debate. Once we get to something as complex as “a learning disability” (e.g., McDermott, 1993), the dispersion will be even more evident.

pp. 24–25

In seeking to understand the constitution of, for example, school as a form of life today, everyday practices are turned from objects of policy or diagnosis to processes, through a focus on “the practices that create it, the work that goes into producing and reproducing this order” (p. 25). For Packer, this focus on constitution is not purely descriptive. Rather, he suggests, educational research at its best is an example of scientific inquiry that arises from the human interest in emancipation, in “increasing human freedom”; this can be described as critical “in the twofold sense that it explores the conditions for a phenomenon (its constitution) and it diagnoses inequity” (pp. 26–27). This is not, I would argue, to take the critical perspective that presupposes the lines along which inequity exists and seeks to reveal them, but rather looks at how inequity is constituted through particular discourses and practices. Both the tacit understandings of participants and theoretical understandings of the researcher are put to the test of what is said and done.

Conclusion

In this article, I have sought to revisit the development of the field of educational research, as it has broadly taken shape in Anglophone contexts, as a starting point for seeking to overcome a seeming impasse in its ability to gain traction on what it is beyond a means of determining causality and improving learning outcomes. As a field with a clear applied focus, it has been reshaped by changing conditions of knowledge production throughout the late twentieth and early twenty-first centuries, to focus on its direct relevance to policy and practice and affirm its rigour through a concern with methodology. This methodomania in turn can be seen to have effected a shift from foundation disciplines to adisciplinarity, unmooring educational research from not only its theoretical but also its educational concerns. Rather than providing a further defence of the necessity of philosophy of education as such, this article questions whether it is the empiricism of educational research that is the issue or the particular form this has taken in light of the concern with “what works” and based on a narrow conception of what science is. To refocus on educational questions, both current and recurrent, I have (re)turned to anthropology, whose sensibilities complement those of educational philosophy and theory, through the work of Martin Packer and his articulation of the study of constitution.

A similar articulation of the shared concerns and sensibilities of anthropology and education is provided by Ingold (2018), and there exists a rich vein of scholarship in anthropology drawing insight from Wittgenstein (e.g., Das, 2020). Such work inherently affirms the value of theoretically grounded scholarship, acutely aware of the historical and contemporary shortcomings of its methods, but not seeking to unmoor itself from them or becoming preoccupied with methodology. It points the way to scholarship that is, yes, relevant, rigorous, and innovative, and responsive to the mundane yet challenging conditions in which we find ourselves, but in ways that frame the questions according to its object of inquiry: an anthropologically informed, educational philosophy.