Towards a Comprehensive and Multidimensional Evaluation of Social Impact in Community-Based Initiatives
Understanding the social impact of community-based programmes requires far more than measuring attendance rates or producing statistical outcomes. Social impact is multidimensional and deeply embedded in participants’ lived experiences, interpersonal relationships, and the wider social contexts in which programmes operate (Ivankova, 2017). Initiatives centred on social inclusion, wellbeing, and personal development often foster subtle yet transformative forms of change that cannot be fully captured through quantitative indicators alone. As a result, meaningful social impact research requires both a carefully constructed evaluative framework and a diverse range of quantitative and qualitative methodologies capable of exploring the complexity, depth, and subjectivity of human experience, accompanied by a continual reflective approach before and during research implementation (Nikidehaghani et al., 2023).
In 2023, our six-week HeppCare research trial on the effect of circus-based movement therapy on low to medium levels of anxiety and depression, involving 54 participants, exemplifies how a comprehensive mixed-methods research strategy can be adopted to adequately evaluate the social impact of a community-based programme. The trial combined structured evaluation tools, quantitative and qualitative methods and phenomenological approaches, which have proven to be beneficial to qualitative research (Fernandez, 2020). This knowledge snippet outlines the framework, methodologies, and key learnings that emerged from the HeppCare trial, offering insights into how social impact can be more holistically understood and evaluated within community-based initiatives.
Firstly, before the research starts, before any data is collected, it is essential to develop a thorough understanding of what a programme intends to achieve through the creation of a Theory of Change (ToC). A Theory of Change provides a structured framework for identifying how and why a programme is expected to generate meaningful change. It enables organisations to clearly articulate the relationship between programme activities and their intended social impact, while also making explicit the assumptions underlying this process (Wilkinson et al., 2021). A Theory of Change serves several important functions. In terms of strategic planning, it helps to map the pathways through which change is expected to occur and supports the practical implementation of programme activities. Within monitoring and evaluation, a ToC establishes clear expected processes and outcomes that can be reviewed over time, allowing to assess contribution to change and refine approaches where necessary (Stein & Valters, 2012). It also functions as a communication tool by enabling the explanation of goals, methods, and intended impacts. Furthermore, a ToC acts as a valuable learning tool, encouraging critical reflection and helping practitioners clarify and strengthen the theoretical foundations underpinning their programme or organisation.
A Theory of Change is generally composed of three core components. The first is impacts, which refer to the broader long-term changes or goals the programme seeks to achieve. The second component is strategies, which encompass the activities, interventions, and approaches implemented to bring about these changes (Vogel, 2012). The third component consists of connections, namely the logical pathways that explain how specific interventions are expected to lead to desired outcomes and impacts. Through this process, the relationships between interventions, outputs, outcomes, and long-term impacts become clearly defined, creating a coherent framework for both programme delivery and social impact evaluation (Wilkinson, 2021).
HeppCare Trial Theory of Change
Secondly, a comprehensive evaluation of the social impact of a programme requires the selection of an appropriate analytical framework that is tailored to the specific context of the initiative under study. In the case of the HeppCare trial, programme quality and social impact were assessed through a multidimensional framework consisting of the Growth and Mastery Scale, the Responsive Instruction Scale, and the Participant Engagement Scale. This framework was informed by the research of Smith et al. (2017), who identified three key dimensions necessary for the design of impactful circus-based programmes.
The first dimension, the Growth and Mastery Scale, examined whether participants were exposed to new experiences, engaged in progressively more complex tasks across sessions, received recognition for their achievements, and had opportunities to identify and develop their individual strengths. The second dimension, the Responsive Instruction Scale, evaluated staff instructional practices in three areas: the extent to which a structured and supportive environment was facilitated through guidance and encouragement; the opportunities provided for leadership and collaboration; and the programme’s capacity to foster planning and reflection among participants. The third dimension, Participant Engagement, focused on participants’ levels of enjoyment and interest in the activities, as well as whether they experienced an appropriate degree of positive challenge.
In addition to selecting a suitable evaluative framework, the use of diverse data collection methods is essential for capturing the complexity of social impact. Reliance on a single form of data, such as interviews or surveys, may provide only a partial understanding of participants’ experiences (Ivankova, 2017). Consequently, the HeppCare trial adopted a mixed qualitative approach incorporating interviews, testimonials, observation logs, and phenomenological insights.
Semi-structured in-depth interviews were conducted to gather rich and nuanced data regarding participants’ individual experiences within the programme. In-depth interviews are a qualitative research method designed to explore participants’ perspectives and experiences in detail. Within the HeppCare trial, interviews were conducted individually with twelve participants in a private room on site. The interviews were transcribed using the application Otter, enabling accurate documentation and subsequent thematic analysis.
Observation logs also formed a central component of the data collection process. Observation logs are research documents used to systematically record observations made during the course of a study. They contribute to the accuracy and consistency of qualitative data while also providing contextual information that supports the interpretation of findings derived from other methods. In the HeppCare trial, an observation log was completed by the Hepp team following each session.
A phenomenological approach further enriched the study by focusing on the subjective experiences and perceptions of participants (Finlay, 2012). Phenomenological research seeks to identify common themes, meanings, and structures emerging from individuals’ lived experiences. As part of this approach, the researcher also participated directly in the HeppCare trial, enabling the recording of phenomenological insights within the observation logs. These reflections contributed additional depth to the interpretation of participant experiences and programme dynamics.
Finally, a rigorous social impact evaluation must include a clear limitations section. Identifying methodological or contextual limitations is essential for transparency and for strengthening future research and programme delivery. Equally important is the inclusion of corresponding actions or recommendations that address these limitations. Reflecting on key learnings and outlining recommended actions ensures that the evaluation process contributes not only to academic understanding, but also to the continuous improvement and long-term effectiveness of the programme.
Limitations Table: HeppCare Trial
In conclusion, evaluating the social impact of programmes such as the HeppCare trial requires a holistic and context-sensitive approach that combines structured evaluation frameworks with diverse qualitative research methods. The integration of the Growth and Mastery Scale, Responsive Instruction Scale, and Participant Engagement Scale enabled a multidimensional understanding of programme quality and participant development. Simultaneously, the use of interviews, observation logs, and phenomenological insights provided a deeper exploration of participants’ lived experiences and the subtle forms of change generated through the programme. By acknowledging limitations and translating findings into actionable recommendations, social impact research can move beyond assessment alone and contribute meaningfully to the ongoing development, sustainability, and effectiveness of socially engaged initiatives.
References
Ivankova, NV. (2017) ‘Applying mixed methods in community-based participatory action research: a framework for engaging stakeholders with research as a means for promoting patient-centredness’, Journal of Research in Nursing, 22(4):282-294. doi:10.1177/1744987117699655
Nesta (2011) ‘I want to clarify my priorities by defining my goals and the path’, Theory of Change, Practical tools to trigger & support social innovation, Development Impact & You
Smith, C., Roy, L., Peck, S., Macleod, C. (2017) ‘Evaluation of Program Quality and Social and Emotional Learning in American Youth Circus Organization Social Circus Programs’, American Youth Circus Organization, Research Gate
Stein, D. & Valters, C. (2012) ‘Understanding Theory of Change in International Development’, JSRP, The Asia Foundation
Vogel, I. (2012) ‘Review of the use of ‘Theory of Change’ in international development‘, UK Department of International Development
Wilkinson, H., Hills, D., Penn, A., & Barbrook-Johnson, P. (2021). Building a system-based Theory of Change using Participatory Systems Mapping. Evaluation, 27(1), 80-101.
Finlay, L. (2012) ‘Embodying Research’ https://doi.org/10.1080/14779757.2013.855133