Top tips for project evaluation reporting

From application to final evaluation report, Youth Music's funding programme is firmly rooted in an outcomes focused approach, and we're not the only ones using it. But whilst it’s all very well us waxing lyrical about it at head office, what does it realistically mean for a delivery organisation? We discuss some top tips for evaluation reporting.

Whether you’re in the midst of your project’s final concert, or discussing the nuances of a growling bassline on a mentee’s new track, your application probably seems like a distant memory – but chances are it contains the key to demonstrating the value and impact of your hard work. The application process at Youth Music is not just designed to assure us that your programme of work deserves our backing, but to make sure you are able to capture all the positive changes your organisation brings about and help share that knowledge with the sector. The intended outcomes, indicators and evaluation plan you specified in your application should help you to take the right measurements, and ask the right questions to show exactly how well your project is going. With this in mind, the Grants & Learning Team have got their heads together to offer some of their top tips for writing milestone and evaluation reports - a process that hopefully should serve not as a chore, but as valuable reflection time – and if our recent stakeholder survey is anything to go by, 77% of respondents have found it is doing just that!

Do your preparation from the outset

You’ve been awarded the grant. The funding agreement needs to be signed, the venues need to be booked, and the music leaders need to be briefed. There are probably a thousand other pressing things that need to be done to get your project going. And I know it is easy to say and harder to action, but if you can set aside just a couple of hours before the project begins to get your head around the reporting template (and of course your intended outcomes and stated indicators) then it makes the final write up process more of a process and less of a panic!

Taking this time allows you to know exactly what statistics you will need to report on, and so you can collect this information from the young people from the very start (their age, ethnicity, how many sessions they attend etc.).

It also allows you to gather meaningful baseline information. When the end of project report asks you to describe your progress towards your intended outcomes, you can be confident that you know exactly the distance travelled by each young person, because you asked them the same questions at the beginning and the end.

Brief the whole team

As project manager it shouldn’t be your job alone to collect all the information needed to evidence progress towards your outcomes. If you can brief all your team on the indicators, and encourage them to note down anything they notice that contributes to them, your evidence will be that bit stronger.

If your music leader is engaged in helping you to gather information, and is aware that their perception of each young person’s musical ability is crucial to useful reporting, then perhaps they can build into their sessions some time to make notes on each young person’s progress to hand to you as the project goes along.

Reflecting on the indicators could even form a part of any regular supervision or 1:1 sessions they might have with yourself or a line manager.

Read and follow the guidance

The information and guidance we provide should help to make your reporting easier. We've made the reporting templates as simple as possible with each question clearly explained. It's a good idea to get familiar with these templates at the outset to get an idea of the diferrent areas you will be reporting on- we don't want reporting to take over your life (!) but having an awareness of what the report is asking for should help you to work smart and ensure nothing takes you by surprise when you sit down to write the report.

Check out the Youth Music Outcomes Guidance on the Youth Music Network, where you'll find advice on measuring and evaluating the changes (outcomes) your project brings about: https://network.youthmusic.org.uk/sites/default/files/Taking_an_Outcomes_Approach_from_planning_to_evaluation.pdf

Refer back to the application

Before you begin to write your report it is necessary to interpret the data you have gathered to transform it into evidence. This can seem like a daunting task but remember that you have already clearly specified your project aim, intended outcomes and outcome indicators in planning for your project. You have also selected the appropriate sources of evidence needed to capture relevant information. It’s essential that you refer back to these plans and use them as a guide for your evaluation. Reflect on your outcome indicators, collate all the data you have collected and identify strong themes within the data.

Think outcomes not outputs

When we ask the question ‘please tell us about your experiences of meeting your project outcomes…’ we’re not asking you to list all of the activities you delivered (we ask about that elsewhere in the form when we ask about the progress you have made in achieving your outputs). What we’re looking for on the outcomes question is an honest reflection of how far you’ve met the outcomes. We don’t need to know how many sessions you’ve delivered or what the content of these were. What we would like to see is something like the following …. ‘we think we’ve met this outcome fairly well. The young people have reported an increase in their musical skills, as well as the music leaders in their feedback from each session. In addition, a comparison of filmed performances from the beginning of the project would also indicate an increase in musical skills, with young people being able to play more complex pieces of music and a move away from the use of backing tracks (at the beginning) to their own compositions (at the end).’

List your outcomes and describe the progress against each one

When writing your report and presenting your evidence, you may want to use the following guide:

For each of your intended outcomes:

1. State the outcome: e.g. ‘to increase the musical skills of participants’.
2. Outline the activities undertaken to achieve this outcome (e.g. Running weekly workshops for participants during school term time; Employing a skilled music leader to deliver high quality music-making activities.) State the outputs associated with these activities (e.g. 35 sessions, 5 music leaders). Briefly note what worked about these activities, and any changes or challenges you encountered.
3. State your first indicator and associated source of evidence (e.g. Young people’s own assessment of musical skills, using Youth Music ‘musical development scale’). Outline your findings, including any charts, graphs, pictures, songs or any other evidence illustrating your findings.
4. State your second indicator and associated source of evidence (e.g. the Music leader’s rating of young people’s musical skills, using a log completed by the Music leader at the end of each workshop, noting progress made by each participant. Outline and illustrate your findings.
5. Repeat step 3 for each subsequent indicator, if you used them.
6. Sum up in one or two sentences, on balance, the overall extent to which this outcome was achieved.

Here is an excerpt from a recent milestone report that we received that follows some of these steps:

Outcome: to improve the quality and standards of music delivery for children and young people

Indicator: Using participant's feedback and young music leaders views to influence the quality of our music offered to children and young people.

We have seen many examples of participant’s feedback and young music leaders views leading to influence the quality of our music offer across the board. One clear example of this came from the young music leaders who have been involved in many of our workshops. They suggested that one workshop, which has a focus on music technology and production, could also incorporate more instrument tuition and work towards live performances.  Through regular conversations at the music mentors workshop plus development chats the young music leaders highlighted to us that they wanted to see the playing of instruments and live performances become part of these regular workshops.

For us this is a good example of how facilitating participant’s feedback and young music leaders views has led to influencing the quality of our music offer to children and young people. There are now a broader range of development opportunities within the workshop itself and also for the young music leaders to be part of facilitating this development. An unintended outcome of this journey is also the CPD opportunities it has opened up.

Evidence 1: Attached is a video showing some of the young people from this particular workshop performing with instruments and one of our music mentors explaining the process … and expressing his thanks to another music mentor who had helped … lots of young people … playing an instrument.

Be honest

We love to hear about all the things that are going right with the project, but we also want to hear about any issues/problems, whether you’ve found solutions and what you’ve learnt from the process. If you feel on reflection that you might have over-promised in your applications then don’t be afraid to address this in your reports and share your learning with us. Similarly, avoid ‘over-claiming’ when reporting on your outcomes.

Get the team involved

Writing a report can seem like a dull task that requires you to sit in a quiet room, staring at the report’s questions for a long time… BUT! You could take a more creative and inclusive approach and get your team involved.

No matter whether your ‘team’ consists of several full-time staff, a bunch of committed volunteers, a couple of trainee music leaders or a mixture of all of the above, they were part of the project you want to shout about and will have insights and learning to share. Maybe you could get some of the key questions from the report on flip chart paper around a room and ask your team to add sticky notes with their comments as part of a half hour brain storming exercise. Alternatively you could interview your staff about the various aspects of the project to gain additional insights which would help you to complete the form.

There are many ways for getting your team involved but our advice is to ensure that the breadth and depth of your work can be heard.

Resources

Youth Music Evaluation Builder - online tool to help you create a personalised evaluation plan for your project

Youth Music Funding Page - information about Youth Music funding

Youth Music Existing Grantholders Page - to access Youth Music milestone and evaluation reporting templates