Evaluation Drop-In Session Write-Up - August 2021
In February and March 2021 we launched our new evaluation resources (Plan, Do and Review guidance) and reporting templates for our Fund A and Fund B grants.
As part of the launch, Youth Music began hosting online evaluation drop-in sessions for grantholders: the first was held in April 2021 and the second in August. In the sessions, we talk about some of the key changes in our approach to evaluation, and answer any questions that our grantholders have. To see the topics covered in the first session, visit this blog. The following is a summary of what we covered in the second session.
In the session we covered a range of questions submitted by grantholders prior to the session. As a result, we have updated several of the existing evaluation resources with information you might need, including:
- Common evaluation challenges: We have updated the list of frequently asked evaluation questions and answers you might find helpful.
- Different evaluation techniques: We have added a section about doing evaluation with young people who are harder to engage in traditional data collection methods.
We also led a breakout group discussion about some of the best approaches to take when presenting your evaluation data to us and other audiences. One of the main reasons for Youth Music refining the evaluation requirements was to make it more flexible for our grantholders. In the session we facilitated a conversation inviting grantholders to share their best approaches and successes when reporting and presenting evaluation data. Here are some top tips that came back from that discussion:
- Focusing on the learning: several attendees told us about how evaluation isn’t just about reporting on the project’s outcomes, but thinking about what they’re learning from the project. We always want to know about what didn’t go to plan as well as the successes, and what you’ve learned from the experience.
- Planning and consistency – little and often: there were also a lot of comments from grantholders who make their reporting easier on themselves by planning time in for noting down findings regularly throughout the project, as opposed to trying to remember everything retrospectively at the end. Some did this by blogging or keeping diaries after every session, whereas others had regular debriefs with music-leaders and other staff and recorded it so they could remember later. One attendee observed that collecting evidence and feedback along the way, rather than just at the end, can also enable you to learn and adapt from what your participants are telling you before the project’s over.
- Using mixed media: we have always welcomed photo, audio and video documentation to support your evaluation reports in the past, however, the change to making our reporting more flexible means you can now submit your entire evaluation in audio or video format if you wish! Filming a documentary-style video report, recording a podcast, or submitting a presentation deck are now all very welcome ways of telling us about your project. We recommend no longer than 10 minutes’ worth of footage, and it should still cover your progress towards your intended outcomes, as well as what you learned from the project. Unless you give us express permission to share the information externally, these videos and photos will still remain anonymous.
- Balancing the individual and the collective: collecting and aggregating data from all (or a representative sample) of your participants using scales and forms is a brilliant way to get an understanding of the overall experience and progress of your group. But we also know that this isn’t always the easiest or most exciting way to engage young people in the evaluation process! Many attendees told us that although it may sometimes take longer, spending quality time talking to participants informally and sincerely gave the most valuable data. Collecting qualitative data can be a great way to provide some contextual quotes when reporting your evidence, and it can also enable you to write case studies of individual young people’s stories that you can submit too. If you have the relevant permissions, case studies also double up as brilliant content for your own website and social media channels.
- Involving young people in the process: we ended the session with a short discussion on how to make evaluation an enjoyable and meaningful part of the process. We heard some excellent examples from attendees of how to involve young people in the data collection process: working with staff to decide what questions to be asking, and carrying out peer interviews with each other are great ways to include the young people in the process in a way that’s engaging and relevant to them. Think about ways you could also involve young people in the reporting and presenting stage: could they play a role in analysing findings, presenting them in a video or podcast, or using their music to communicate the outcomes of your project? The evaluation process should be a beneficial and fulfilling space for participants. Peer-led reflection can highlight experiences or perspectives that may not have been captured in a leader-led space.
In an end of session poll, the majority (78%) of attendees felt more confident about their own evaluation as a result of the session and most (75%) indicated that they would like to attend another one in the future. If you would like to attend the next grantholder drop-in session, keep an eye out for the invitation in your inbox towards the end of the year.
If you have any feedback or further questions, please don’t hesitate to get in touch with your Grants & Learning Officer, or the Research & Evaluation team at researchandevaluation@youthmusic.org.uk .