Robotic Percussionist

Dr. Gil Weinberg interacts with Haile during an improvisational live performance

Georgia Tech’s music technology program is pushing the envelope in artificial intelligence research and human-robot interaction. Director of Music Technology Gil Weinberg has created a robot that is not only changing the way we think about music, but also advancing research about the human mind and the way in which we may interact with robots in the future.

The robot, Haile, analyzes music based on human perception models and improvises algorithmically using unparalleled cognitive and physical abilities. Haile can therefore create a novel kind of human-machine interaction that can lead to new insights about music perception, improvisation, and collaboration.

Haile's uniqueness lies in its ability to play acoustically with a vibrant sound while utilizing perceptual and improvisational algorithms. The robot can listen to live players, analyze their music in real-time, and use the product of this analysis to play back in an improvisational manner. It is designed to combine the benefits of computational power with the richness, visual interactivity, and expression of acoustic playing.

“We believe that when collaborating with live players, Haile can facilitate a musical experience that is not possible by any other means, inspiring players to interact with it in novel and expressive manners, leading to new musical outcomes,” said Weinberg.

The robotic percussionist is the result of research that crosses several disciplines and combines Weinberg's passions for music and technology to produce novel and innovative music. The research has created a harmony of sorts for Weinberg, who started the research about three years ago.

Haile was created in collaboration with researchers and graduate assistants from Music Technology, Mechanical Engineering, College of Computing, Industrial Design and the Advanced Wood Products lab, making it an extremely unique and interdisciplinary project.

“Haile allows us to develop models for human-robot collaboration in timing-critical missions. Music is a perfect medium to design and evaluate such models due to its emphasis on timing, collaboration and improvisation. The project is also designed to shed light on humans' cognitive and physical ability to enjoy and create music. This is one of the most unique human traits that has not been explained by science as of yet.”

The robot, which started as a drum player, has been recently adapted to play a xylophone. To this end, new algorithms have been developed at the Music Technology Group that allows it to perceive, analyze and improvise melodic and harmonic music.

Haile has been featured in dozens of concerts and workshops worldwide, including cities such as Barcelona, Jerusalem, Paris, Copenhagen, Odense, Boston and Atlanta. The robot has also had its fair share of publicity, being featured in stories on CNN, Discovery Channel, Computer Magazine, PC Magazine, and the New Scientist among others.

The research that led to the creation of Haile has already paid extreme dividends for the Music Technology Group, leading to National Science Foundation support, new graduate research assistant positions, and global exposure.

“I am particularly excited about the new version of the robot which is currently under development in collaboration with Georgia Tech’s GVU center and the Robotic and Intelligent Machines Center. In this project, we plan to study and develop sophisticated models for human-robot interaction in expressive and social contexts. The project can also help shed more light on the role of music in human culture both cognitively and physically.”