Please Enter a Search Term

Where the music is



Dr. Ellen Waterman, director of the School of Music, demonstrates how a computer camera can be turned into a musical instrument through AUMI, the adaptive use musical instrument.

By Meaghan Whelan


Where does music lie in the body? Does it reside in fine motor skills and vibrating vocal cords? What about bodies that cannot speak, or bodies that cannot manipulate a musical instrument? Should access to musical expression be a human right?

Ellen Waterman asks questions like this every day.

Dr. Waterman is the director of the School of Music and a scholar with a keen interest in musical improvisation as a social practice. She is a part of a collaborative research team that works with Adaptive Use Musical Instruments (AUMI), computer software that allows individuals with extremely limited mobility to express themselves and connect with others through improvisational music.

AUMI is a free computer program that uses the motion tracking technology built into an ordinary computer camera. Users look at an image of themselves on the screen and their movements trigger sound samples. The movement that can be tracked by AUMI ranges from an arm sweeping across the body to the slight movement of a mouth or tongue. It was developed by Pauline Oliveros who is a composer, performer and Distinguished Research Professor of Music at Rensselaer Polytechnic Institute in New York, and Leaf Miller, a drummer and occupational therapist at Abilities First.

Abilities First is a school for students with moderate-to-severe disabilities in New York. Students at Abilities First were the first to try AUMI and the response has been resoundingly positive. Anne-Marie is a 16-year-old student at Abilities First with cerebral palsy. Non-verbal and confined to a wheelchair, Anne-Marie’s voluntary movement is limited to moving one arm and her head. Dr. Waterman was at Abilities First and saw Anne-Marie using AUMI to improvise with Ms. Miller on drums.

“It was music; beautiful improvised music. It was very clear that Anne-Marie is a musician,” said Dr. Waterman.

“If music resides deep within our humanness, unlocking that for people who can’t speak or hold an instrument is extraordinarily powerful,” she explained.
During an open house musical event at Abilities First in April 2010, children demonstrated that they could use AUMI to play music together.

“About 50 kids were in a circle and there were three AUMIs set up. Students with the ability to do so drummed in the circle and kids like Anne-Marie took turns using AUMI. They were interacting with the drums to make music and were able to participate as opposed to sitting on the sidelines and watching. It was very powerful and it was also an amazing jam session,” Dr. Waterman explained.
AUMI has led to three primary research questions for scholars involved with the project: how to advance the technology of the instrument to allow further independence by users; what the potential physical and psycho-social therapeutic benefits are for users; and what are the sociological implications for creating musical community across a wider range of bodies.

The third question is of primary interest to Dr. Waterman. She is a core researcher in the SSHRC-funded Major Collaborative Research Initiative on Improvisation, Community and Social Practice.

“Childrens’ joyful response to AUMI has made us eager to explore the potential of improvisational music and technology to provide greater social interaction, and a creative voice for children with disabilities.”

Dr. Waterman says music and improvisation are very powerful tools for fostering community. “In improvisation, everyone has a voice, all contributions are valued, and we are all listening to one another. This is essential for creating empathy and building a sense of community.”

Dr. Waterman is now in consultations with community organizations and music therapists to bring AUMI to the local community.

Share