Explore Evaluation with Soundabout (Learning Disabilities) - May 2023

In November last year, Youth Music ran an Explore Evaluation session with Take Art that focussed on evaluating Early Years music projects. Following the success of the session, we decided to take this format forward and ran a session concentrating on evaluating music projects that work with children and young people (CYP) with Learning Disabilities. The session was co-led by Sam Williams (Insights Officer at Youth Music), and Holly Radford-James (CEO at Soundabout), Jo Colton (Director of Programmes at Soundabout) and Will Tuggart (Music Practitioner at Soundabout).

 

Youth Music’s approach to evaluation

Your evaluation is primarily for your learning. We believe you can learn from what didn’t go to plan just as much as what did. Here are some key principles we use within our approach to evaluation at Youth Music:

Flexibility. There are some standard questions you will have answer, but the main part of your report can be submitted in whatever format you choose (e.g., written report, webpage, PowerPoint, podcast, video). We do not require a specific way to conduct or present your evaluation, choose what works best for you.

Multi-purpose outputs. You can create something to use beyond reporting to us (e.g., reporting to other funders, promoting your project, as a training tool, case studies for your website). What audiences and wider communities can benefit from sharing your learning?

Proportionality. Your evaluation doesn’t have to cover every part of your project. It’s better to be focussed and do something really well than do too much and overstretch yourself.

Budget/time. Include budget for evaluation time and roles, and build it into your project timeline. This includes building in reflection time along the way to see if you need to adapt your approach.

Essential monitoring data. There are some data Youth Music asks you to collect and we have tried to make this as light-touch as possible. Make sure you know what it is before you start delivery. You can see which data we require you to collect by looking at our reporting templates here. Other data you collect should be to support your organisational learning and is bespoke to you. It's not solely about providing evidence but how the sources can demonstrate change and progress.

If you’re unsure where to start, take a look at our Trailblazer Evaluation guidance, or if you’re using an outcomes approach use our Plan, Do, Review guidance.

 

Getting started with your evaluation

Sometimes deciding where to begin can be the hardest part of your evaluation! Here are some key questions to ask yourself to start designing your evaluation:

• Why does this project exist?
• What type of project are you running?
• Who are you working with?
• What need are you trying to fulfil?
• What is the purpose, and how are you trying to measure it?

Always remember that you are the expert in your own project and evaluation is about demonstrating the impact that you know matters.

 

Project evaluation with non-verbal CYP

There are some common evaluation techniques that may not be an accessible format for non-verbal CYP. However, it’s important to remember in situations like these to think about what you can do, not what you can’t.

One method that you could use is observations. During your sessions, observe closely what your participants are doing and how they are reacting to session content. If you have the resources, have a trusted person in the room who is there to just observe, or alternatively set up the sessions to allow the leading practitioner space to observe.

You could also record your sessions and review them afterwards. If you decide to do this, ensure you have consent from the relevant people involved.

To support yourself in what to look for when observing, consult with the CYP’s parent[s] and/or carer[s]. Every CYP is unique and may display how they are feeling in a different way.

 

Data collection from parents and carers

Along with consulting with parents and carers, they are also a useful data source for your evaluation. They can be particularly helpful if you’re unable to collect feedback directly from the CYP, or to use in triangulating your data to have more robust findings.

It is important to acknowledge the capacity of parents and carers, so think about the frequency and collection point of your data.

One way to do this is to formalise any anecdotal feedback you receive. For example, if you regularly get emails from parents and carers talking about the impact your project has had, record and log them to use in your evaluation.

 

Youth Voice in evaluation

Your evaluation is a key opportunity to listen to the voices of CYP and help both your project and organisation grow.

Soundabout formed the Soundabout Participant Advisory Group (SPAG) where their participants were invited to provide feedback. The group helps to foster a continuous feedback loop: listen; make change; communicate change; review how changes go; and repeat. One way Soundabout approached this was ensuring the SPAG met before and after trustee meetings.

Soundabout also developed an Emerging Leaders programme, where young people take on roles such as supporting in session leadership, participating in the interview process for Soundabout roles, and operating as a sounding board for new ideas.

You can find out more about implementing Youth Voice into your projects by using Youth Music’s #ShareTheMic Resource Hub.

 

Evaluation from a practitioner’s perspective

The session involved a Q&A presentation with Soundabout practitioner, Will Tuggart. Here are some of the key points for approaching evaluation with CYP with Learning Disabilities from Will’s perspective:

• Create an environment where CYP can give feedback and generate responses: try and build a trusted relationship with the CYP so they feel able to give feedback, while also ensuring they have the space to do so.
• The importance of silence: allow space for silence that the CYP can fill. Fight the urge to fill it yourself!
• Keep it simple: make use of repetition, chants, and silence.
• Live music vs recorded music: while both have their place, live music allows you to stop and start up again with ease. This enables that space to observe and get feedback.
• Take time at the end of a session for evaluation and don’t just record musical outcomes: ensure you are factoring in time to complete evaluation work throughout your project. Make notes straight after a session while your thoughts are fresh. Note communication and social outcomes, as well as areas where more interest was shown.
• Give CYP a leadership role: always look for opportunities to give participants a leadership role in the session. This could just be a CYP taking the lead musically and others following, instead of always being led by the practitioner.
• Have different resources for different needs: ensure you have lots of options for resources to ensure everyone can participate. Some examples can be seen in the section below.
• Finish with a recap: at the end of the session, encourage CYP and adults in the room to recall what you covered in the session – this could be done in the format of a song! This is another opportunity to see which parts of the session people enjoyed the most.

 

Resources and tools

• Sound of Intent: an inclusive framework of musical engagement which sets out how everyone engages with music, including those with profound learning difficulties.
• Widget symbols: a system of symbols used to support text and make them easier to understand.
• Talking Tiles: small electronic devices that can record and playback up to 80 seconds of audio.
• Skoog Skratch: an accessible sampling app for iOS devices.

 

We conducted an exit poll after the session, with 88% saying they felt more confident in conducting their evaluation work as a result of taking part in the session. Along with this, 100% said they would recommend the session to another funded partner, and would be interested in attending another Explore Evaluation session.

If you have any feedback or further questions, please don’t hesitate to get in touch with your Grants & Learning Officer, or the Insights team at insights@youthmusic.org.uk.

 

The Explore Evaluation sessions are part of Youth Music’s learning and development programme, Exchanging Notes. Check out this page to see our upcoming Exchanging Notes sessions.