If computing opened the door to assistive communication, artificial intelligence has now kicked the door wide open, turning slow, character-by-character typing into fast, fluent text prediction. But critical gaps remain. At Michigan Tech, Keith Vertanen, the Dave House Professor of Computing, and his lab's student researchers take on the challenges, designing open-source, on-device interfaces to ensure tomorrow's technology belongs to everyone.
In Vertanen's office in the College of Computing's Rekhi Hall, third-year computer science major Adyn Skowronski dons an augmented reality headset and begins making gestures that resemble an elaborately slow game of air piano. This is navigation at the front line of communication accessibility.
Through the headset, Skowronski sees his physical surroundings overlaid with a three-dimensional digital environment. As his gaze shifts across a virtual keyboard floating in midair, specialized tracking cameras capture his subtle eye movements as other sensors chart the rapid trajectories of his hands.
The interface of human and machine demonstrates the lab's goal to engineer multimodal input methods that make computing seamless for everyone—including individuals with severe motor and speech impairments.
Skowronski, whose study focus is video game development, plays VALORANT for Michigan Tech's Esports team. Even though gaming comes naturally to him, spelling words correctly through eye and hand movements requires patience and concentration.
Doctoral student Soufia Bahmani joins the conversation as the demo continues. Her focus leverages large language models to predict character-by-character entries and develop smart personalization systems that live entirely on a user's local device.
"Communication is such a fundamental part of independence, connection, education, work, and self-expression," Bahmani says. "That's one of the reasons this work means so much to me. I'm motivated by the possibility that better technology can remove some of the barriers people face in everyday life. Even small improvements in speed, accuracy, or ease of use can make communication feel less frustrating and more natural. When I see that something I've worked on could make someone's life a little easier, it makes me really happy, and it gives me even more motivation to keep going."
Bahmani is building an open-source Python library named TextSlinger, designed to level the playing field for assistive communication technologies while also protecting user autonomy and privacy.
Commercial text-prediction software tends to be proprietary, making it difficult for independent clinicians, researchers, or specialty hardware companies to adapt the code for specific disabilities. Lack of privacy is also a problem, since standard consumer algorithms continually push user data up to giant commercial cloud servers.
When people use an alternative communication interface due to conditions like late-stage amyotrophic lateral sclerosis (ALS), their device records every intimate conversation, password, and private medical update. If that data is funneled to an external corporate cloud, it introduces an immense security risk. TextSlinger aims to provide a sophisticated predictive model that can adapt to a user's unique shorthand, personal abbreviations, and distinct writing style dynamically and securely on-device.
Bahmani says working in the lab has been a supportive and inspiring experience that she'll carry into future career endeavors. "I hope to continue doing research that combines AI, accessibility, and human-computer interaction in a way that can have a real impact on people's daily lives."
How AI Accelerates Advances
Designing intelligent interactive systems that leverage uncertain output technologies is nothing new in Vertanen's lab. Past projects have involved supporting the social communication and community engagement of autistic adults, sensing tap location using acoustic sensors, a brain-computer interface system that allows people with severe impairments to type, and participatory design for people who use augmentative and alternative communication.
Supported by an active $1.2 million collaborative research grant from the National Science Foundation (NSF), the lab is working with institutions like MIT and Oregon Health & Science University to integrate these predictive intelligence systems into specialized physical interfaces.
As he describes his team's work, Vertanen reflects on how rapidly the research landscape has shifted beneath their feet, fundamentally transformed by the rise of artificial intelligence.
Harkening back to his 2018 NSF CAREER Award for the five-year project "Technology Assisted Conversations," Vertanen observes that "the CAREER research was all built as we were starting to look at the early neural models—called recurrent neural net language models."
AI advances in neural network-based language models proved to be a dramatic game-changer. "The large language models really started to explode around 2019," Vertanen recalls. "I spent quite a lot of effort training a very good statistical model during my early research. And out of the box, leveraging one of these pre-trained models immediately beat it. It was night and day."
Massive AI acceleration has allowed the team to power interfaces like Nomon, a single-switch selection interface designed for individuals with extremely limited physical mobility. Users who can only manage a single reliable movement—like a button click, head tilt, or eyeblink—rely on Nomon to make selections on a screen. By pairing Nomon with TextSlinger's underlying neural language modeling, the system can more accurately anticipate a user's intent, drastically shrinking the time required to type out a single word.
In 2024, Vertanen received $129,000 in National Institutes of Health funding to help optimize a Brain Computer Interface-Functional Implementation Toolkit, subcontracting with collaborators at Oregon Health & Science University and Northeastern University. In addition to his cross-institutional collaborative work, he's also received individual industry support, including $75,000 in funding from digital giant Meta and $107,000 from Google in support of the lab's ongoing work to enhance mobile typing, speech recognition, and augmentative and alternative communication devices.
Designing for the Physical World
Back in the headset, Skowronski navigates the physical hurdles that make typing in augmented reality a challenge, including the complete absence of tactile feedback. When typing on a virtual, floating keyboard, there's no physical touch sensation that signals a key has been activated.
"Without any tactility, it's incredibly difficult for your body to be able to figure out where your hands are located in the 3D space," Skowronski says. "Our goal with the research is to find what modifications increase the ease and speed at which typing can be done—this might mean combining eye-tracking data with word prediction to more accurately guess what word a user is trying to type."
To accomplish their goal, researchers blend eye- and finger-tracking with mathematical error-correction decoders. When a user attempts to strike a virtual key and misses, the system looks at where their eyes were focused, combines that location with the language model's context, and automatically corrects the letter to what the user meant to type.
Skowronski works with both Vertanen and Scott Kuhl, associate professor of computer science, whose research focus includes improving virtual reality systems and who has advised Michigan Tech's Husky Game Development Enterprise since 2011.
"Working with them has been a great privilege. Both are wonderful educators and have taught me a lot. Keith has allowed me to set new standards for myself with both the quality of work that I achieve and how I hold myself in a professional setting," says Skowronski, who is also working on a minor in business.
He enjoys being part of research with the power to reshape how society perceives technology.
"This work is important to me because it's an example of how we can use newly developed technology for good," Skowronski says. "There is a lot of discussion revolving around the harms that new technological progress has brought. As someone very interested in said technology, it's gratifying to prove that there is a lot of good that can be done with it as well. The idea of my work also being used as a reference point one day when designing said technology is also very cool and rewarding to think about."
Michigan Technological University is an R1 public research university founded in 1885 in Houghton, and is home to nearly 7,500 students from more than 60 countries around the world. Consistently ranked among the best universities in the country for return on investment, Michigan's flagship technological university offers more than 185 undergraduate and graduate degree programs in science, technology, engineering, mathematics, computing, forestry, business, health professions, robotics, psychology, social sciences, humanities, and the arts. The rural campus is situated just miles from Lake Superior in Michigan's Upper Peninsula, offering year-round opportunities for outdoor adventure.
