by Author simonsteptoe

Published on

You are here:

Sound Control - Workshop Diaires

"Sound Control" is one of our projects operating under the umbrella of our Fund B programme, "Music Forge", is a recipient of one of the Paul Hamlyn Foundation's Explore and Test grants, and part of our partnership with Dr Rebecca Fiebrink at the Computing Department of Goldsmiths London. The project itself is a combination of machine-learning software and off-the-shelf hardware devices (e.g. sensors, controllers) to assist in the rapid prototyping and development of digital musical instruments with a specific focus on working with PMLD children. 

We posted a few articles about the project itself over the past few weeks... so if you want to find out more about the technologies involved please visit: 

However, the purpose of this blog is just to share some of the best practice our music leaders have been developing with the young people involved. 

Rebecca Price is one of our Musical Inclusion Team and is involved in two of our "Music Forge" projects: "Relaxed Singalong" and "Interactive Piano" - both of which are worth finding out more about:

Over the past few months, Rebecca has rapidly become an expert in using the Sound Control software.... and has combined this knowledge with her experience of working with SEND children at the Rowan Gate Special School campuses in Wellingborough and Rushden... to really help the project move forward. In addition to this work she's also been out and about demonstrating the software at Kingsley Special Academy in Kettering, Corby Business Academy DSP unit, and with Youth Chaos (SEND over-16s) in Rutland (NB: There's a post about Youth Chaos HERE together with a great video from the group about their music work).

Along the way, she had put together a brilliant workshop diary, which is a great insight into her practice and something we felt worthy of sharing more widely. No doubt, the diaries need to be read in conjunction with the other posts listed above, but we feel they are a good example of reflective practice nonetheless. Also, in addition to being her personal reflections, there were a primarily tool to communicate with the researchers at Goldsmiths and help them tweak the programme and the user interface. 

If anyone wants to get in touch with Rebecca and find out more about her work, both on Sound Control and other projects, drop me a line HERE, and I'll pass the message on. 

 

Workshop Diaries – Rebecca Price

“I worked mainly with the colour recognition camera, finding that primary colours are useful, and also a plain background, clothing, and good lighting! I worked with three students on this, setting up individual instruments one at a time, and then combining them. 

We ended up using four different objects, and the students then started to associate these as different instruments, and pick each one up when wanting that sound.

I was thinking it would be useful to have an imprint left on the B&W camera screen when a point has been recorded there, as we encountered some problems with overlapping. When they looked at the screen, they were able to control the positions they recorded, but we also used blindfolds, and they were less precise then.

All three students liked the sounds available, with them all showing a preference for the Looper with the loop samples. Two of them requested some more “Rocky” sounds!

For some reason, the Gametrak has not worked when I have used it, although Jan has used it with no issues... I must be me doing something wrong!

The BBCMicrobit is brilliant, really reactive.  I have tried attaching it to a wristband and a headband with great results. The only thing is again, on the overlap, the sounds sometimes 'glitch' into each other on the border between positions and using a blindfold this made it trickier to locate a sound & position.”   (November 2017)

 

“Today I've mostly worked with the Gametrak, tying it to a stretchy band/'bracelet' that the child can wear. Both E, R and D showed a preference for the pitch shifting FM synth, understanding their part in the control straight away. With each of these, I also set up a few instruments on the table as a visual, asking them to point at drums and then recording drum sounds there etc. They all grasped this really well and enjoyed pointing and 'conducting' the instruments they wanted to play.

I then combined the two with D  -  she had one on each wrist. I found some discrepancies here, the two sides of the Gametrak controller don't always seem to work independently of each other? Is it possible for this to be done?

This afternoon I've worked with M, who has limited movements but can make big movements with his arms. I used the bracelets in the same way, and he definitely increased how much he was moving when the sounds were on - he seemingly understood that he was controlling it straight away.

I also spent some time experimenting with the Microbit. I stretched a band around a euphonium, tucking the Microbit inside, and setting some brass sounds, so that as soon as it is lifted slightly it makes a sound. My speaker was too big to fit, but a small speaker inside the instrument would work really well. I also tried suspending it, so that only a slight amount of effort could move it, thus making it very easy to access notes through a heavy instrument. I also tried handing this to M, I helped him to lift it at first, and then he tried hard to tilt it by himself and was pleased when it made a sound.

There is still a little glitch/click sound in between the samples when used on the Microbit – i.e. in the overlap between one position and the other.” (January 2018)

 

“I worked again with E, mostly repeating what we did last week, but using her feet instead of her hands to control the GameTrak with the FM synth and the Looper (one on each side). This encouraged lots of movement, which is excellent for D; she is encouraged to use her legs more as she is in a chair, but is sometimes reluctant to do so. So, a result that is not necessarily the main purpose of Sound Control, but a brilliant result for the student in a different way

With W, we used the GameTrak and a bracelet on each wrist, initially starting with right hand and FM synth with different pitches of sound effects, which he was very animated with, and seemed to really enjoy. Trying the same on his left hand was interesting, as he has less movement, and the recordings had to be more precise, but still worked well. We tried some musical sound effects and some drum samples on his left wrist bracelet although he was less engaged by these.

M initially explored some of the variations on the FM synth with the Gametrak on his right wrist. We moved on to some sampled flute sounds using the Looper. I could use the Sampler to achieve the same effect here, but I find the Looper easier to use. He was very interested in playing different notes from the scale. I wonder if it is possible to choose a range of maybe 5 notes, and set in only the highest and lowest at two points, and the software could map in--between notes? This would make it easier to set in, especially with M or W, where their movements are sometimes sudden, and difficult for them to hold in place whilst we record each one.

I also put the Microbit in a headband for M to wear. It worked effectively, and so that very quickly he understood he was controlling the samples and showing this through deliberate movements of his head. When using more than one instrument at a time, it would be useful to have a box on each window on the software to label them, to make it easier to know which is which.I also trialed a way that might work with R, who is very limited in her movements. Putting a camera on a mount so it could be close to my face (it needed to be within two feet), the Colour Tracker was able to pick up my chin movements like R makes. Today I used red electrical tape and this could maybe work as well with Micropore tape - I am waiting to hear from her class what is safe to use on her skin. I think coloured tape, stuck on the skin, could work in a few other situations with other children too. (February 2018)

 

“All went well at Kingsley today.  The Music Coordinator, X, was really supportive and stayed with us for the session. We worked with Butterflies class (I think), a class with a range of needs, from PMLD to autism and, I think all Reception and Year 1.

Some really positive interaction with the equipment, a definite awareness and engagement from all but one of the children. Some of the children were showing a really good awareness just from the short session today, which was great to see.

X herself had a go at setting up the software at the end of the session and had some really good feedback for us to know how user-friendly it is. She came up with some very enthusiastic ideas about how she might implement it in school and seemed keen for us to go back at some point if that's possible.

I wasn't taking notes today as we were constantly working with the children… so this is from memory. I hope it’s enough detail!” (March 2018)

 

“Similar work to before with 'The Hulk' (!), I introduced the “fart” sounds, which he moved around more for, and lots of him sticking out his tongue and making raspberries! (Thanks Sam [Goldsmith’s researcher/programmer], the “fart” noises have the approval!

M similar to before, but I added in a scale on the midi mapper. He seemed to be repeating a similar movement with his arm that created a certain note pattern. And then, when I sang that particular pattern back to him, he gave a big smile every time!

I gave W the volume control on the Mixer to start; he was showing a preference for staying below a certain level. I then matched that volume level into the Looper, and gave him a choice of sounds. He also used the red mitten to control sounds today.

I was finally able to work with R today. I set the colour tracker camera up on the end of a boom arm mic stand, so I could position it close to her face. We were able to attach a red sticker to her chin, and it picked up her breathing movements to control the sound on or off. Whether she was aware of the cause and effect today we were unsure, but she did make some murmuring sounds. K from her class asked if she could use the sound control every week!

I also had some chance to investigate that BBC Sound Library - I've added some sea and seagull sounds to the sound folder to use with the Music Festival over at the Royal and Derngate later this term”. (April 2018)