Explore Evaluation Session - November 2022

As part of Youth Music’s learning and development programme, Exchanging Notes, we launched the Explore Evaluation sessions (originally Grantholder Drop-in Sessions) in early 2021 to help funded partners with their evaluation practices. We ran a session on November 2022 that explored evaluation based questions that were submitted by funded partners before the session, covering a wide range of topics.

In this blog, we’ll cover the session content, including some general top tips and new tools that were suggested by our funded partners.

General evaluation guidance:

Evaluation is primarily for you and organisation learning: think about what learnings you can gather that you can embed into your organisation’s practices moving forward.
The data Youth Music require you to collect can be seen in our reporting templates on the Youth Music Network website: any additional data you collect should be bespoke to you to support your organisational learning.
Youth Music does not require a specific way for you to conduct your evaluation: choose something that works best for your organisation. We do not require a specific way to conduct or present your evaluation, choose what works best for you. If you’re unsure where to start, use our Plan, Do, Review guidance.
There is no “best” way to present qualitative data: case studies are powerful for exploring an individual journey and work well when complementing quantitative data. If you want to use them, there is no minimum or maximum amount expected. But there are other ways to conduct qualitative data analysis that may work better for you and can be seen in our Different evaluation techniques guidance.
Triangulate your data: triangulation is the process of comparing and contrasting multiple viewpoints (e.g. young people, music leaders, parents) to form a conclusion from your data. Using triangulation leads to your findings being more robust.
Safeguarding is always the priority: safeguarding should come before the research. Some ways to do this are encouraging safe spaces and ensuring participants can opt-out at any point in the process. See our Ethics and data protection guidance for further support.

Below are some other topics we covered during the session, including suggestions from our funded partners.

External evaluators

External evaluators are a useful option that bring additional expertise to your evaluation work and can also assist in easing staff workload. If you choose to do this, think about what it is that you need the specialist for (data collection, data analysis, team support, etc.)

Be sure to get a ball-park figure for the work you would require before you budget.

Embedding evaluation throughout your project

Embedding evaluation into your project practice can be beneficial, particularly if you have time constraints.

Some funded partner suggestions included:

Video observations
A form with a set of questions you can fill out on the go
Voice notes in-between sessions
Budgeting pay for practitioners to stay behind after sessions

Co-creating with children and young people (CYP)

Having a youth-led and designed space can increase the comfort and engagement of CYP. It helps to ensure they do not feel like something is being done to them instead of with them.

Using creative methods can often be effective in implanting youth voice in your work and making use of direct quotes from CYP, images or videos. Qualitative data collection tools can also be embedded into the project, such as collecting lyrics at project touchpoints.

One way to demonstrate how youth voice is embedded in your project is by outlining your organisational learning: how did CYP lead; what your learnings were from this; and then how you actioned these recommendations.

Data collection from parents and carers

Choosing to collect data from parents and carers can be particularly useful as a data collection point for triangulation or for when you cannot directly speak to the CYP e.g., Early Years participants.

It’s important to acknowledge the capacity of parents and carers, so think about the frequency and moment of collection. One method is to use open diaries set up in the workspace that they can use at their leisure.

Measuring musical development

One way musical development can be measured is through assessing skill progression. For example, this could be done by the attainment of music accreditations or comparing videos of a participant’s performance taken at the start and end of your project.

However, there are other outcomes for musical development other than technical skill development such as:

Musical awareness. Developing knowledge and understanding of musical worlds and roles e.g. musical cultures, the music business, music leading
Musical understanding and communication. Developing music understanding and communication e.g. expression, creativity, listening, interpretation and evaluation
Personal outcomes. Developing intrinsic outcomes like emotional and psychological capabilities e.g. communication, confidence, agency, self-efficacy, creativity, resilience, motivation managing feelings, empathy, self-awareness

General data collection tools

Our funded partners suggested some tools that can be used for data collection:

Mentimeter, an interactive presentation software
Sounds of intent, a framework of musical development for CYP with SEN/D
Transformative evaluation, a method to evaluate impact that uses aspects of appreciative inquiry, participatory evaluation and transformative learning, while involving the whole organisation

Other evaluation guidance

Collecting diversity monitoring data: Youth Music only ask you to collect this data from your core participants, i.e., those who engaged with three or more sessions. This data is also useful for your organisational learning. It can help you to understand if you are reaching your target participant group and take informed steps for the future.

Capturing progression data: What’s next for the participants and how has their involvement in the project impacted their goals for the future? Incorporate timelines into your questioning. For example, at the end of the project, ask participants, “Where to you see yourself in six months’ time?” Here are some examples of questions we have used:

To what extent do you feel the project has impacted your career progression? [Not at all – To little extent – To moderate extent – To good extent – To great extent] Why? (open text response)
Did you learn anything or gain any transferable skills that you might take forward into future opportunities?
Can you give us an example of how you learnt these skills?

We conducted an exit poll at the end of the session, with 87% saying they felt more confident in their evaluation skills, while 87% also indicated they would attend another Explore Evaluation session. As well as this, 93% would recommend attending to other funded partners.

If you have any feedback or further questions, please don’t hesitate to get in touch with your Grants & Learning Officer, or the Insights team at insights@youthmusic.org.uk.