2016 Research Magazine

Emotions and Technology

By Allison Mills

People’s individual response to colored lights vary, and Shiyan Hu and Sarah Sun are developing medical wearable tech incorporating both light and music.

Red is angry. Red is passionate. Red is happy, lively, spicy and soothing. Wait, soothing? Well, for someone it might be. And one person's chill-out cyan could be another's jealousy trigger, while green can be neutral, enlightening, or nauseating. Our emotional and physical responses to color are as varied as the paint-chip selection at the hardware store. Instead of deterring Sarah Sun and Shiyan Hu, that variety helps inspire their research on wearable medical tech. Sun, an assistant professor of mechanical engineering, and Hu, an associate professor of electrical and computer engineering, are pushing the edges of the spectrum in this new technology field.

Shiyan Hu is an associate professor of electrical and computer engineering, specializing in cybersecurity.
Shiyan Hu is an associate professor of electrical and computer engineering, specializing in cybersecurity.

Beyond the Hospital

Imagine this: You like cello music and your favorite color is teal. One afternoon—it's been a crummy day—a panic attack creeps up. Your heart rate picks up and your skin gets clammy. But before you start fighting for air, a soft teal light envelops you and a Yo-Yo Ma lullaby picks up. The cue to start the soothing music and light came from a small sensor, or maybe several, strapped to your wrist, ribcage, or head. Possibly, the whole room could join in with lights, speakers, and sensors along the walls.

The vision for technology like that started as a project that Sun and Hu wanted to do to help new mothers deal with post-partum depression. Once they started, they realized the devices could have broad applications for a number of medical conditions, spanning both mental and physical illnesses.

"Many people live with chronic diseases," Sun says, pointing out that the Centers for Disease Control acknowledges that nearly half of the US population live with at least one chronic illness like diabetes or cardiovascular disease and one in four adults have multiple chronic diseases. "And in 2010, 86 percent of all healthcare expenditures was spent on chronic diseases."

Heartbeats and Brainwaves
A: LIGHTING--People's individual response to colored lights vary, and Shiyan Hu and Sarah Sun are developing medical wearable tech incorporating both light and music.

B: MUSIC--Using five different songs in their research, Sun and Hu are analyzing how the music affects physiological metrics in order to build better medical devices and software algorithms.

C: BLOOD PRESSURE--Using Galvanic Skin Response (GSR) sensors, the researchers monitor telltale signs like blood pressure.

The cost is high, estimated to be more than $320 billion for heart disease and stroke alone in 2015 and annually about one in every six US healthcare dollars is spent on cardiovascular disease. Add hundreds of billions more each for obesity, smoking, excessive drinking, arthritis, and cancer. To lower those costs, Sun says we need a transition from traditional in-hospital healthcare to preventative, proactive, out-of-hospital care. Monitoring devices that pick up on the telltale signs of chronic diseases—such as rising blood pressure or low blood sugars—could help people be more independent, too.

"They could wear this hardware without having it influence their daily life and they could drive, walk, run, swim, and do what they want to," Sun says, adding that the tech helps people become more aware in the moment of what their body is doing. This kind of real-time intervention could prevent more expensive and invasive treatments later.

Sarah Sun is an assistant professor of mechanical engineering, specializing in designing wearable technology.
Sarah Sun is an assistant professor of mechanical engineering, specializing in designing wearable technology.

Heartbeats and Brainwaves

A lot of people already have some sort of health-related app, wear Fitbits and digitally track their wellbeing. But good health is more than 10,000 steps per day. To meet hospital standards, the tech needs to precisely replicate diagnostic tests like an electrocardiogram (ECG—heart health) or electroencephalogram (EEG—brain health).

It needs to reliably read blood pressure, blood sugar, temperature, and a number of other physiological factors.

There is no silver bullet, although one method that Sun designs around shows promise: Galvanic Skin Response (GSR) sensors. GSR shows that people don't wear their hearts on their sleeves, but rather their skin. The sensors measure electric conductivity through moisture and tension on the skin—a nice way of saying nervous sweat. With GSR and other wearable devices, Sun and Hu act as a hardware- software team and are taking health monitoring to the next level by increasing accuracy, personalizing the tech, and refining the algorithm models.

"However, there are a lot of uncertainties," Hu cautions. "That's what makes this a pioneering field."

Heartbeats and Brainwaves
A: HEART RATE--Consistently and accurately reading physiological metrics such as heart rate is a key part of Sun's hardware development.

B: COLOR--Building in responses to color requires a lot of personalization. A cheery teal and enlivening orange for one person may not be soothing to another.

C: TEMPERATURES--A challenge for the devices and software is dealing with environmental factors; measuring body temperature reliably can be difficult with changing temps and humidity.

The challenge is rooted in the device itself and the data models. A major obstacle is the huge variability in individual responses, which can influence the ideal location for the sensors. (Suzie Q can get away with a GSR wrist sensor, but Jane Doe needs the accuracy of armpit sweat.) The data also have to account for environmental factors like room temperature and humidity.

The challenge makes Sun and Hu a perfect research pair. Sun focuses on the hardware—ensuring that the heartbeat is getting picked up accurately, for example—and Hu focuses on the software—making sure that heartbeat gets interpreted correctly. The next challenge is what to do with all those heartbeats and brainwaves.

Crafting Algorithms

There's a fine balance between intervention and independence in using tech to help us step out of an anxiety storm or call attention to dangerously rising blood pressure or give us a gentle reminder to workout. The software needs to be agile, responsive, and smart, Hu says, and definitely not bossy or intrusive.

In a way, it's like the online music program Pandora. Each station has a seed, a music genre, band or artist, but the program wouldn't be around if the music player simply repeated the same album.

"Eventually this model will learn each individual's behavior. Then the software will know what's your favorite music—it can adapt and think like you."Shiyan Hu

Instead, it relies on a massive database to catalogue similar music, refining that process with each thumbs up or down. Sun and Hu didn't use a Pandora's worth of tunes for their research; they narrowed it down to five specific songs, so they can hone in on people's very specific emotional and physical responses measured with the GSR sensors. Similarly, they relied on the extensive body of literature detailing the most common emotional reactions to color.

"Also, our software is different from Pandora since we rely exclusively on quantitative data," Hu says. "You don't get to like or dislike your heart rate, it just is."

Based on their research so far, teal and cello is a safer bet to calm down than orange and Metallica. But as the user you get to set your baseline. That's the end goal of personalizing medical technology. Continuous feedback then helps teach the software algorithms where your like/neutral/dislike boundaries are as well as what different music and colors change your physiological readings.

Whether red is your soother, or spa-ready flute music sets you on edge, doesn't matter. The whole spectrum is available. Or at least it will be as Sun and Hu keep rolling the color wheel into the wearable world of medical tech.

Michigan Technological University (www.mtu.edu) is a leading public research university developing new technologies and preparing students to create the future for a prosperous and sustainable world. Michigan Tech offers more than 120 undergraduate and graduate degree programs in engineering; forest resources; computing; technology; business; economics; natural, physical and environmental sciences; arts; humanities; and social sciences.