Search results

This chapter explores how the case studies were ‘messy’ research. Because we were researching in contexts with many unknowns, the research process was unpredictable. ‘Tidying up’ the research in advance and working within clearly defined parameters was not usually possible. Across the case studies, mess occurred at different points and in different ways in the research process. For some projects, the design itself was subject to uncertainty and change; sometimes what had been planned was not possible; sometimes what had been planned was not the best course of action as the project progressed; and sometimes the design itself was emergent, requiring creativity and flexibility to meet the project outcomes. Some projects faced messiness when trying to combine methods and data. Others encountered messiness when collecting data, deciding what counted as data, and interpreting data. The real-world nature of our research and our need to be responsive to dynamic and often unknown out-of-school contexts meant that our methods could not fit into the neatly structured shorthand that is often used to think about (and teach about) methods. As researchers, we were constantly dealing with fluid and changing identities, as our relationships with participants and spaces developed during the project. This also means that tidying up our research could be counter-productive. The chapter concludes that making sense of mess in research can reveal understandings that are sometimes hidden. Mess and complexity, then, is something to be held on to, celebrated and engaged with, rather than tidied away.

The chapter will explore a collaborative theory of change approach that the authors used to evaluate three projects. The three projects worked with young people out of school in different ways to enable the young people to become agents of change in tackling the causes of alcohol misuse in their local Scottish communities. A theory of change approach provides a way of conceptualising programmes from inception, through to implementation and the evaluation of outcomes, in order to develop an understanding of how they work, for whom and in what circumstances. Using a collaborative model of this approach challenged prevailing notions of evaluation being the job of the evaluator and situated evaluation as a shared endeavour with the project staff. We outline the key attributes of such a collaborative model of theory of change and reflect on how this model can contribute to the evaluation of out-of-school activity.

This chapter describes methodologies used in the project ‘Out-of-school activities and the education gap’. The project explored how the out-of-school environment affects children, whether it impacts on primary school attainment and whether it reinforces existing socioeconomic differences. A mixed-methods approach combined three areas of research: statistical analysis of the Millennium Cohort Study (MCS) linked to the National Pupil Database (NPD); a qualitative study through interviews with key stakeholders in 10 schools in London and the North East and the articulation of theories of change for how out-of-school activities may affect attainment. Patterns in how children spend their time, and whether and how this affects attainment, were investigated by analysis of the MCS linked to the NPD. Qualitative research with parents, teachers, pupils and activity providers from schools in London and the North-East afforded an in-depth understanding of drivers and barriers influencing how children spend their time and pathways by which activities may affect children's learning and development. The qualitative research also provided a narrative intersectional analysis of responses in terms of class, gender, ethnicity, religion and disability. Mixing quantitative and qualitative research was made difficult by the volume of data and the time needed to analyse and report each area separately, the different nature of data in the three areas of research and the timing of each phase of data collection. However, meaningful combining of methods occurred at the level of research questions and contributed to a more critical analysis of children's out-of-school activities than had been possible before.

This chapter discusses the different types of researcher/participant relationship described in the case study chapters, alongside the extent to which the projects were (and could be) pre-defined in terms of structure and expected outcomes. The case studies ranged from secondary data analysis methods with no researcher/participant contact, those with structured one-off interviews, those with more ongoing, but still researcher-led, relationships between researcher and participant, to more ethnographic and participatory research where relationships were negotiated between researcher and participant and, in some cases, led by the participants. This chapter highlights that researcher/participant relationships lie parallel to the structure of the project and the extent to which the outcomes are pre-defined. Despite the range of types of relationship, however, all the case studies highlighted the value of trust in those relationships, for participants to feel happy to share the details of their personal lives beyond that which is usually visible in the formal education setting of school. Edwards' (2017a) concepts of relational agency, relational expertise and common knowledge are used to help explain why these relationships matter in research on out-of-school learning – to understand activities that we do not know about, which take places in spaces that we are unfamiliar with.

Proponents of robust research design and methodology (particularly, although not exclusively, in more positivist-leaning epistemology) have often suggested that the role of the researcher should be as invisible, or distanced, as possible in the research process. Many of the case studies presented in this book take a more qualitative, interpretative approach, reflecting the often complex, situated, local and dynamic contexts in which out-of-school learning occurs. This raises particular challenges relating to the researcher role, especially when the researcher's presence materially changes the context and phenomena that are being researched. Some of the case studies describe the tensions and affordances of the researcher as insider/outsider and demonstrate how this role can develop and change as a project progresses and the implications this has for research practice, research quality and research governance.

This chapter reflects on the concept of knowledge – or perhaps, more accurately, the multiple knowledges – generated in this field of study. We consider, through drawing on some of the examples of ways in which knowledge about out-of-school learning is constructed in the case studies, issues such as the authenticity and value of knowledges pertinent to this field, the power structures and knowledge hierarchies involved and the localised sites of such knowledge production. We conclude with some thoughts about how researchers can manage the tensions involved in making decisions about whether to try to integrate or to keep separate such multiple forms of knowledge.

Ethics work in research is often conceived of as a process of research governance. The case study chapters, however, provide evidence of a much more sophisticated engagement with ethical dilemmas arising in research and an enactment of ‘everyday ethics’, in other words, a concern with our relationships with, and responsibilities to, other people (Banks, 2016). This emphasis on relationality can often lead to what Cook (2009) describes as ‘mess’ in research, which needs to be made sense of. This is in contrast to the notion of ‘well-ordered’ research, which underpins many of the ethical frameworks, principles and guidelines that are produced for research. The chapters also indicate the opening up of new spaces for research that raise new challenges in respect of ethical practice, including, for example, digital spaces (Case Study 4 – Minecraft Club). Case Study 8 – Democratic Engagement also demonstrates that both researchers and participants in the research process find ways in which to challenge conformity and research norms in order to access knowledge, and this is not always a harmonious process. The following sections try to make sense of the implications of these issues for the ethical practice of research. This chapter pulls together three key themes emerging from the case studies of research governance, ethical relationality and ethical spaces, presenting an analytical overview of all three areas using the concept of ‘willful subjects’.