Thursday, May 10, 2007

Enabling musical expression

Rachel's graphical representation of Dan's musical composition

During the H2.0 event, Adam Boulanger, researcher at the Hyperinstruments group with Professor Tod Machover presented their most impressive latest research: enabling musical expression.
Dan Ellsey, who has cerebral palsy, has been composing with the Hyperscore environment for almost two years. Adam developed a head-set and supporting software so that Dan could expressively perform his hyperscore compositions as a soloist. Dan worked closely with the team to perform his “eagle song” using head movements during the event. Rachel Roppolo worked on translating the sensor inputs from Dan into minimal graphics. Zach Watts worked on the curve algorithm to determine the type of shape Dan was making.
The performance was clearly impressive, very detailed and empowered the public.

I later talked to Adam about the project. He explained that the project necessitated a number of important steps. The team initially built on the commercial head tracker that Dan usually uses. However the commercial tracker was unsuccessful for a performance because it is not fast enough and is not reporting continuous data. Adam had to create a continuous head tracker.
Regarding the design aspects of the system, Adam had to work closely with Dan to come up with the algorithm that he needed to get clean data from Dan’s head movements, the team later calibrated the sensors for Dan.
The other step consisted in designing a controller that could run a performance. The team designed a new software that could take Dan’s range of motions and velocity of movement. They also had to do curve analysis to determine the type of shape Dan was making. Adam had to also figure out, based on his piece and movement, how this would be represented as an expression of the piece. The whole process was a discussion back and forth between Dan and the team. The system being done, they went through the piece of music and defined the type of movement needed for expression.

The performance was incredible. I cried, impressed by what the team had achieved.
The speakers were all impressive, I missed a few of them that I have to catch up on the archive. The webcast of the event is available online.

Adam, Dan and Tod

1 comment:

Anonymous said...

This is some impressive software. I enjoyed the video footage of the conference immensely.