Main Content

Smart acoustic devices: coming soon to a screen near you?

A University of Rochester project uses flat panel technology to build a more cost-effective device that can also function as a touch interface.
Over the past decade, smart devices have become essential components of daily life: doorbells, watches, keychains—even thermostats.

But many smart devices, such as televisions, laptops, smartphones, and car “infotainment” dashboard consoles, rely on visual presentation of information. That can be a problem when a person’s visual attention is focused on something other than the device.

One way to address this problem is to let people use sound or touch for interactions. Of course, adding separate microphones, loudspeakers, and vibration mechanisms using conventional means can impose extra costs and reduce the device’s durability and aesthetics.

Audio and music engineering major Ben Kevelson ’22 is part of a University of Rochester team working to design flat surfaces that can function as acoustic and tactile interfaces—no external microphones or speakers needed.

The project involves developing alternative technologies for recording and reproducing spatial audio through bending vibrations on flat surfaces ranging from smartphones to video walls. These vibrations allow the smart acoustic surface to serve as a touch interface. The advantage is that the surface is already part of the device, allowing the device to maintain its durability and aesthetics while incorporating these new features.

Microphone and speaker all in one
“Our goal is to make a display—or any flat surface—serve as a loudspeaker, microphone, and touch interface,” says project head Michael Heilemann ’18 (PhD), an assistant professor in the Department of Electrical and Computer Engineering. “By vibrating the device itself to produce sound, we can make a good-sounding speaker that fits well in a thin device.”

The technology uses a flat surface to create and receive sound, making it different from a typical speaker: it can be a microphone and a speaker all in one, and the direction of the sound source can easily be detected.

The goal of this project is to help improve smart assistants—think Amazon Alexa or Google Assistant—that use this technology. “The benefits and applications are pretty broad,” says Kevelson, who works closely with Tre DiPassio ’19 (MS), a Rochester doctoral student focusing on musical acoustics and signal processing. “But it has particularly promising applications in home entertainment, consumer electronics, and general music and audio technology.”

Current smart assistants use between four and eight microphones to detect the direction of the sound source. This new flat panel technology would use only one, making it a cheaper alternative.

“The speakers on my TV are bad, and if I buy external speakers, they’re usually clunky,” Heilemann says. “Even a sound bar takes up room. By using the TV panel itself as the speaker, you don’t need external speakers to get good sound out of the TV.”

An easier way to localize sound
Heilemann received funding for the project through the National Science Foundation in June 2021. Kevelson was a student in Heilemann’s acoustics course and asked if there were any projects he could help with over the summer. “He was an excellent student, so there was no question he would do well working on this project,” Heilemann says.

Kevelson worked full time on the project last summer and up to 10 hours per week once the school year began. He was accepted into the master’s program in electrical engineering at Rochester and is working on the project this semester for both his senior design project and graduate research credit. He has helped create massive data sets of recorded audio signals, allowing the team to test their systems and determine how different panel sizes and materials affect recording quality.

If successful, the technology would localize sound to specific areas of the screen. For example, if someone is on a Zoom call with several others, the audio would come from the area where the person was speaking. “No more searching your screen to see who is talking,” Heilemann says.

One potential application in the testing phase involves deploying the flat panel audio technology to make commercial devices more durable. “They get beaten up pretty good, and a lot of damage can occur due to things like weather, smoke, or anything that can get into the unit and damage the insides,” Heilemann says. “If you use a microphone or speaker, even in a cell phone, you need some kind of penetration in the unit so the mike can probe the acoustic environment or the speaker can radiate sound.”

The need for such penetration would be eliminated if the display itself is the speaker, he says, which makes the unit more waterproof and airtight.

Century-old technology gets a Rochester upgrade
For his part, Kevelson has been thrilled to “get my hands dirty” with the project. “It’s very different compared to other things that are research-based in audio,” he says.

According to Heilemann, the idea of using panels as speakers has been around for nearly a century. “It never caught on because the sound quality was always much worse than that of conventional speakers,” he says.

Kevelson is proud to be part of a research team that may finally make some noise in that field.

“It was dismissed as completely unviable,” he says. “We’ve brought it back to life at the University of Rochester, and we’re at the forefront of flat panel technology. It will be exciting to see where it leads.””

Link to article