Reviewing evaluation evidence in an early years music project
Wiggle & Shake is a year-long early years project in First Steps nurseries and children’s centres in Bath.
Before the summer break we reviewed our evaluation evidence to see how we’re doing with meeting the project outcomes. From the reflective conversations between myself and practitioners after every session, and the more formal review meetings, it felt like we knew how well the project was progressing – which areas of the project were working well at meeting the outcomes for children, and which areas were not working so well. But, taking time to gather the evidence together and look at it systematically, has highlighted a few things which need more thought and discussion.
In one of the nursery settings, we’re pleased with the impact the project has had on most children’s communication and language development, and social, emotional and behavioural development.
One significant source of evidence of children’s progress is my journal. I make brief notes about each child after each session. When collated together these notes shown the progress children have made over time. For some children it shows a fascinating and remarkable progression. For others, much less so.
I’ve been thinking a lot about one child, R, who finds it emotionally very difficult to cope with some of the group activities we do. The project has been carefully planned to develop children’s understanding of the social conventions of taking turns and sharing instruments. For most children they have developed these skills during the project, but R still finds this difficult. If, for example, the basket of instruments is tipped out onto the floor and another child chooses the instrument she wants she becomes tearful and withdraws from the circle, lying on the floor and crying. Most children, when faced with this situation overcome their distress when an adult helps them to choose another instrument or explains that they will be able to have a turn with their chosen instrument soon - but not R. Her distressed emotional reactions seem to be getting worse.
The notes from my journal about R in the first few weeks of the project include:
Session 1: Confident & joined in very well. Speaking and singing to familiar songs. Counting on fingers for Zoom, Zoom.
Session 2: V. confident. Liked hiding. Smiling and mostly engaged. Did wander towards the end.
Session 3: Participation OK but doesn’t follow instructions and distracted by other children not participating.
Session 6: Less distracted and wandering this week. Improved participation.
Session 9: Speaking and suggesting ideas. Really good participation.
Recently, however, my notes have included:
Very emotional and tearful at times. Lots of running around the room. Upset at being asked to swap instruments.
Still finding it difficult to share. Distressed when she can’t have what she wants.
Participated well at first – singing and making lots of suggestions for choosing animals for ‘Fishes are coming’. When I said it was someone else’s turn to choose an animal she withdrew from the circle and didn’t want to join in.
There are several children who have participated in the project who have struggled with a 30 minute adult-led group session and we’ve adapted the structure to meet their needs. In one nursery we have a ‘small group’. This is two children who have speech and language delay and emotional and behavioural difficulties who weren’t benefiting from the 30 minute group sessions because they couldn’t sustain their attention for very long. These two boys have really benefitted from the small group. We’ve measured their progress by timing how long they engage with the music making before one of them wants to leave the room or starts wandering around. At the first small group it was less than 5 minutes before one boy wanted to leave the room. They are now both able to take part in a 15 minute music session where they are both engaged throughout – singing, speaking, exploring instruments and taking turns. Sometimes they now take part in the larger group sessions and join in for much longer periods.
So, going back to R – why are we repeatedly presenting her with a situation which she can’t cope with, emotionally? Should we be adapting activities so that she is not getting distressed? Or, is R showing normal 2-3 year old behaviour and we just need to persevere with supporting her to learn to share, take turns and manage her emotions?
In trying to understand if there’s a reason for R’s distress, one practitioner said she suspected that, as the youngest of several siblings, R is indulged and allowed to have her own way a lot at home. Another practitioner felt that R’s moods and behaviour at nursery were affected by difficult circumstances her family were experiencing.
Looking at our evaluation evidence, it seems that we can’t say that we’ve achieved the outcome of ‘To improve the social, emotional and behavioural development of young children at higher risk of delay through participation in creative musical activity’ for R. So, has the project failed?
Well, let’s think about what the project has achieved for R. There are many notes in my journal about R enjoying, participating, singing, confidently speaking, suggesting ideas and listening really well. In many aspects of communication and language, and social and emotional development, R has made progress. Perhaps I’m unduly focussing on the negative because her emotional episodes impact on the group activities and as adults we find it difficult to deal with.
So, my point is: I think all this highlights the complexity of children’s development - how a child can thrive with some areas of emotional development and struggle with other aspects, and how children’s home lives can impact on their learning in an early years setting. It also highlights the complexity of evaluating a young child’s experience. This is a phrase which has stuck in my mind from Susan Young’s 2007 article, Early childhood music education in England: changes, choices, and challenges,
"Although all grant-funded work requires evaluation and thereby accountability to the funding body, this process is largely based on the traditions of arts evaluation. These processes are derived from the need to monitor participation and to assess the quality of the participants’ experience. They lack mechanisms for improving practice in educational areas and an understanding of the highly skilled processes of collecting and interpreting data accurately and objectively". (p.24)
Young also wrote that recipients of project funding lend towards “good news evaluation reporting” and produce reports which can be used to promote their work, a practice which she felt was detrimental to “open, unbiased and rigorous appraisal” (p.24).
I think that this is a really valid comment, and something I’ve always tried to avoid. I feel satisfied that we haven’t strayed into ‘good news evaluation reporting’ with Wiggle & Shake and that we’ve utilised evaluation evidence to improve practice and adapt the project to better meet children’s needs.
References
Young, S. (2007) ‘Early childhood music education in England: changes, choices, and challenges’, Arts Education Policy Review, 109(2), pp. 19-2