Where Science and Art Meet: The Mind Music Machine

By Allison Mills


Performance artist Tony Orrico creates a graphite drawing mirrored by sound and visuals through the Immersive Interactive Sonification Platform (ilSoP).

Brainwaves, meet sketching. Dance, meet computer programming. Performance artist Tony Orrico teamed up with researcher Philart Jeon last year to explore the science and art of making a room-sized graphite drawing and what that might mean for the minds of artists. To dig into the research, Jeon brings together a whole team, an interdisciplinary mix of cognitive and learning science students alongside engineers and computer programmers. Check out how they turn movement into sound, make better railroad signs, help robots learn, and measure the mind of an artist.

Mind Music Machine

Jeon's given name is Myounghoon, but he goes by a nickname--Philart. Literally, love-art. And that love of art is what binds together his interdisciplinary work in computer science and cognitive science. In the most general sense, Jeon researches how to make machines more human. Or at least how to bring more human features, like emotions and learning, into computer programs and robots. Under that large umbrella, Jeon works on everything from helping kids with autism learn social skills to making better warning systems in cars to prevent railroad accidents. In order to better understand the minds of artists, the team quantifies physiological responses while doing art--like measuring the heart rate and brainwave activity of creatives like Tony Orrico while he draws. That data can also be turned back into art. Take a listen to this student-created track based on Orrico's physiological measurements.

And ICYMI here's a piece about the Mind Music Machine featured on the Distillations podcast put on by the Chemical Heritage Foundation. You can also keep updated on happenings in the lab by subscribing to their YouTube channel (TrimLab) and following them on Twitter (@Tri_MLab)



Michigan Technological University is a public research university founded in 1885 in Houghton, Michigan, and is home to more than 7,000 students from 55 countries around the world. Consistently ranked among the best universities in the country for return on investment, Michigan’s flagship technological university offers more than 120 undergraduate and graduate degree programs in science and technology, engineering, computing, forestry, business and economics, health professions, humanities, mathematics, social sciences, and the arts. The rural campus is situated just miles from Lake Superior in Michigan's Upper Peninsula, offering year-round opportunities for outdoor adventure.

Last Modified 9:19 AM, February 4, 2023