Calculating distance travelled with the people we work with - how do we know we are impacting?
Article written for Soundsense 'Sounding Board' magazine about calculating the distance traveled for the people we work with. There are loads of other interesting things in the magazine so if you'd like more info on anything to do with supporting Community musicians then I suggest you look Soundsense up (link below the photograph)
Soundsense produce the 'Sounding off' magazine quarterly if you are involved in community music then it's well worth getting in touch with them to see what value they can bring to you or your organisation http://www.soundsense.org
Distance travelled
Recently, in conversation with someone involved the world of community music, I had to repeat the phrase “non grant-funded, profitable social enterprise.” Not just because it didn’t trip off the tongue, but because people aren’t used to hearing those words in the same sentence. Personally I ascribe our ability to say profitable social enterprise to good collection and use of impact data. It’s the thing that means we can confidently state that what we do is effective. When we were starting Noise Solution we got lucky, and received some of the best business advice available in the social enterprise field. The main thrust of that early advice – which still rings true every day – was that to sell services to customers it is crucial to be able to prove that what you do is effective.
I know that words like sell, profit, product, customers etc are anathema to many people in the sector but these words are merely tools that have proved effective in enabling organisations to work out how to best achieve their goals. I see no reason not to appropriate those tools for Community music’s goals.
If you can’t prove you are good then what’s the point, in today’s climate you’re probably not going to be around very long to have the positive impacts we know we can achieve. In simple terms without proof you will always struggle to convince people to pay for the work we do. It’s advice that’s repeated by Tony Brown in Whatever happened to community music? It’s not that surprising maybe, but you’d be amazed how many organisation can’t do this.
Noise Solutions starting point is qualitative data. We use social media interaction in every set of sessions we embark on, capturing through blogging every session we do with every client. It enables us to bring into our work family, peers, professionals – all those that should be involved when taking a holistic approach to working with clients. We want to be able to tell and show and shout about the stories of transformation we are seeing. Blogging has been a core part of our model, enabling us to capture evidence of soft success and share that success easily. It’s been instrumental in our ability to raise client’s levels of confidence, a great way to capture and demonstrate how effective our work is, and a great source of stories and real life examples. It’s also used a great source of evidence for clients who are working towards Arts Awards qualifications.
The blogs make great stories, engaging narratives and occasionally gripping reading. But where commissioners are concerned we also needed more than just qualitative stories to demonstrate our value and secure increasingly large contracts. We also need quantitative data to show succinctly that what we do is achieving tangible outcomes; these are the outcomes that we measure against.
• Increase the client’s confidence • Foster a greater feeling of self determination • Provide a successful educational experience with qualification where appropriate • Increase musical skills • Facilitate a positive progression to education, volunteering, etc.
Importantly when finding funders (customers) we look for organisations who are trying to achieve some or all of these same outcomes. This alignment is important – our methodology is designed to achieve these outcomes and that’s what it does best. We don’t change our model to fit funders or chase the money by trying to sell a different story. We find people who our outcomes (that we value) will suit.
For our qualitative data we use the following equation to work out how far our clients have travelled. The system is called POPI or Percentage of Potential Increase. I came across it from reading a case study of an organisation in London working with disadvantaged young people and it struck us as being a simple client centred approach to demonstrating distance travelled. Here is how it works
*POPI Score: Clients are asked to score all questions between 1-5 before and after working with Noise Solution. From this we are able to establish how much the learners’ perception of their progress has increased. POPI (Percentage of potential increase) is calculated by taking the initial learner score (e.g. 2), calculating the gap to the maximum score (5), then comparing the actual increase vs. the potential increase and converting this to a percentage.
FORMULA: ACTUAL INCREASE / SUM [MAXIMUM SCORE - STARTING SCORE] = POPI %
It works like this: a client presents initially with a score of say 2 for their recent confidence levels; their potential for increase (to get to the maximum of 5) is therefore 3. If at the end of the sessions they report a confidence level of 4 we have seen an actual increase of 2. Remember the maximum possible increase was 3. If we divide the actual increase by the possible increase we can generate the POPI score. In this case there was an 67% increase of the potential increase available.
Our tutors can complete these questions verbally with the clients. They normally ask the same set of questions at the start of the first session and the end of the last. We are not asking for life histories – just the relevant information that we need to prove our point. The data is recorded in a tick box questionnaire that is hosted in our cloud based content management system designed by Substance (another social enterprise).
When we pull together this simple easily collected data across a cohort it creates a powerful picture – not only for the individual clients, whose successes we can feed back to their case workers but also for the organisation as a whole, importantly acting as an effective marketing tool. The box xx shows the sort of “outcomes table” the simple data collection described can create. What it demonstrates is what you and I already know, that music can have a dramatic, positive effect on people’s lives – and it demonstrates that simply and succinctly in a way that is easily understood by those holding the purse strings.
The table enabled us to show the mental health service what their money had bought. The figures are relevant, impressive, and presented simply – and have directly led to the mental health trust awarding us a new contract, to work in an acute ward.
This is a work in progress and it’s not a perfect solution. In particular Noise Solution happens consecutively with other things in a client’s life, which will also affect how they are feeling. Isolating that which is our attributable impact is extremely hard without masses of data and sophisticated statistical analysis. Or in real terms someone’s move from a 2 to a three might represent a more massive shift than another clients move from 1 to 5 in these cases we rely on communication with the referrer. Even so, our early attempts at presenting both qualitative and quantitative data coupled with our transparent processes, are received well by our potential customers. And so we are able to be a “non grant funded, profitable social enterprise."
LINKS Simon Glenister founded Noise Solution as a social enterprise in 2009 to use music technology to create social impact E: simon@noisesolution.org W: www.noisesolution.org/ CAMHS pilot project W: http://network.youthmusic.org.uk/learning/blogs/simon-glenister/impact-report-noise-solutions-child-adolescent-mental-health-pilot-su