Malek Itani
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Malek Itani has a total of 17 co-authors. Here are the top co-authors by number of shared publications:
- Shyamnath Gollakota
University of Washington, Seattle, USA
9 shared publications - Tuochao Chen
School of Computer Science & Engineering University of Washington
5 shared publications - Takuya Yoshioka
Microsoft Redmond WA USA
4 shared publications - Bandhav Veluri
Paul G. Allen Center for Computer Science & Engineering, University of Washington, United States
3 shared publications - Justin Chan
Carnegie Mellon University, Pittsburgh, USA
2 shared publications - Antonio Glenn
Department of Biomedical Engineering Case Western Reserve University Cleveland Ohio USA
1 shared publication
Latest Publications
- Open Access
- 20 June 2026
- Published by Association for Computing Machinery (ACM)
- p. 371-391
Fine-grained Soundscape Control for Augmented Hearing
- 20 June 2026
- Published by Association for Computing Machinery (ACM)
- p. 371-391
Open AccessFine-grained Soundscape Control for Augmented Hearing
- Open Access
- 3 November 2025
- Published by Association for Computing Machinery (ACM)
- p. 863-877
Wireless Hearables With Programmable Speech AI Accelerators
- 3 November 2025
- Published by Association for Computing Machinery (ACM)
- p. 863-877
Open AccessWireless Hearables With Programmable Speech AI Accelerators
- Open Access
Neural Speech Extraction with Human Feedback
We present the first neural target speech extraction (TSE) system that uses human feedback for iterative refinement. Our approach allows users to mark specific segments of the TSE output, generating an edit mask. The refinement system then improves the marked sections while preserving unmarked regions. Since large-scale datasets of human-marked errors are difficult to collect, we generate synthetic datasets using various automated masking functions and train models on each. Evaluations show that models trained with noise power-based masking (in dBFS) and probabilistic thresholding perform best, aligning with human annotations. In a study with 22 participants, users showed a preference for refined outputs over baseline TSE. Our findings demonstrate that human-in-the-loop refinement is a promising approach for improving the performance of neural speech extraction.
Open AccessNeural Speech Extraction with Human Feedback
We present the first neural target speech extraction (TSE) system that uses human feedback for iterative refinement. Our approach allows users to mark specific segments of the TSE output, generating an edit mask. The refinement system then improves the marked sections while preserving unmarked regions. Since large-scale datasets of human-marked errors are difficult to collect, we generate synthetic datasets using various automated masking functions and train models on each. Evaluations show that models trained with noise power-based masking (in dBFS) and probabilistic thresholding perform best, aligning with human annotations. In a study with 22 participants, users showed a preference for refined outputs over baseline TSE. Our findings demonstrate that human-in-the-loop refinement is a promising approach for improving the performance of neural speech extraction.
- 31 January 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Pervasive Computing
- Vol. 23 (4) , 53-55
Creating Super-Hearing Capabilities With Real-Time AI
Sound is a fundamental medium through which we perceive our environment. However, today we are surrounded by a cacophony of sounds that can overwhelm our senses. Indeed, human auditory perception can be limited in noisy environments. Imagine being in a crowded room with a cacophony of sounds and having the ability to focus on or remove sounds from a specific person or based on their semantic descriptions. This requires understanding and manipulating an acoustic scene, isolating each sound, and associating a spatial context or semantic meaning with each constituent sound or speaker—a challenging set of tasks even for the human brain in noisy environments.
- 31 January 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Pervasive Computing
- Vol. 23 (4) , 53-55
Creating Super-Hearing Capabilities With Real-Time AI
Sound is a fundamental medium through which we perceive our environment. However, today we are surrounded by a cacophony of sounds that can overwhelm our senses. Indeed, human auditory perception can be limited in noisy environments. Imagine being in a crowded room with a cacophony of sounds and having the ability to focus on or remove sounds from a specific person or based on their semantic descriptions. This requires understanding and manipulating an acoustic scene, isolating each sound, and associating a spatial context or semantic meaning with each constituent sound or speaker—a challenging set of tasks even for the human brain in noisy environments.
- 14 November 2024
- Published by Springer Nature in Nature Electronics
- Vol. 7 (11) , 1047-1058
Hearable devices with sound bubbles
- 14 November 2024
- Published by Springer Nature in Nature Electronics
- Vol. 7 (11) , 1047-1058
Hearable devices with sound bubbles
- Top Cited PapersOpen Access
- 11 May 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-16
Look Once to Hear: Target Speech Hearing with Noisy Examples
In crowded settings, the human brain can focus on speech from a target speaker, given prior knowledge of how they sound. We introduce a novel intelligent hearable system that achieves this capability, enabling target speech hearing to ignore all interfering speech and noise, but the target speaker. A naïve approach is to require a clean speech example to enroll the target speaker. This is however not well aligned with the hearable application domain since obtaining a clean example is challenging in real world scenarios, creating a unique user interface problem. We present the first enrollment interface where the wearer looks at the target speaker for a few seconds to capture a single, short, highly noisy, binaural example of the target speaker. This noisy example is used for enrollment and subsequent speech extraction in the presence of interfering speakers and noise. Our system achieves a signal quality improvement of 7.01 dB using less than 5 seconds of noisy enrollment audio and can process 8 ms of audio chunks in 6.24 ms on an embedded CPU. Our user studies demonstrate generalization to real-world static and mobile speakers in previously unseen indoor and outdoor multipath environments. Finally, our enrollment interface for noisy examples does not cause performance degradation compared to clean examples, while being convenient and user-friendly. Taking a step back, this paper takes an important step towards enhancing the human auditory perception with artificial intelligence.
- 11 May 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-16
Top Cited PapersOpen AccessLook Once to Hear: Target Speech Hearing with Noisy Examples
In crowded settings, the human brain can focus on speech from a target speaker, given prior knowledge of how they sound. We introduce a novel intelligent hearable system that achieves this capability, enabling target speech hearing to ignore all interfering speech and noise, but the target speaker. A naïve approach is to require a clean speech example to enroll the target speaker. This is however not well aligned with the hearable application domain since obtaining a clean example is challenging in real world scenarios, creating a unique user interface problem. We present the first enrollment interface where the wearer looks at the target speaker for a few seconds to capture a single, short, highly noisy, binaural example of the target speaker. This noisy example is used for enrollment and subsequent speech extraction in the presence of interfering speakers and noise. Our system achieves a signal quality improvement of 7.01 dB using less than 5 seconds of noisy enrollment audio and can process 8 ms of audio chunks in 6.24 ms on an embedded CPU. Our user studies demonstrate generalization to real-world static and mobile speakers in previously unseen indoor and outdoor multipath environments. Finally, our enrollment interface for noisy examples does not cause performance degradation compared to clean examples, while being convenient and user-friendly. Taking a step back, this paper takes an important step towards enhancing the human auditory perception with artificial intelligence.
- Open Access
- 29 October 2023
- Published by Association for Computing Machinery (ACM)
- p. 1-15
Semantic Hearing: Programming Acoustic Scenes with Binaural Hearables
- 29 October 2023
- Published by Association for Computing Machinery (ACM)
- p. 1-15
Open AccessSemantic Hearing: Programming Acoustic Scenes with Binaural Hearables
- Open Access
- 21 September 2023
- Published by Springer Nature in Nature Communications
- Vol. 14 (1) , 1-16
Creating speech zones with self-distributing acoustic swarms
Imagine being in a crowded room with a cacophony of speakers and having the ability to focus on or remove speech from a specific 2D region. This would require understanding and manipulating an acoustic scene, isolating each speaker, and associating a 2D spatial context with each constituent speech. However, separating speech from a large number of concurrent speakers in a room into individual streams and identifying their precise 2D locations is challenging, even for the human brain. Here, we present the first acoustic swarm that demonstrates cooperative navigation with centimeter-resolution using sound, eliminating the need for cameras or external infrastructure. Our acoustic swarm forms a self-distributing wireless microphone array, which, along with our attention-based neural network framework, lets us separate and localize concurrent human speakers in the 2D space, enabling speech zones. Our evaluations showed that the acoustic swarm could localize and separate 3-5 concurrent speech sources in real-world unseen reverberant environments with median and 90-percentile 2D errors of 15 cm and 50 cm, respectively. Our system enables applications like mute zones (parts of the room where sounds are muted), active zones (regions where sounds are captured), multi-conversation separation and location-aware interaction.
- 21 September 2023
- Published by Springer Nature in Nature Communications
- Vol. 14 (1) , 1-16
Open AccessCreating speech zones with self-distributing acoustic swarms
Imagine being in a crowded room with a cacophony of speakers and having the ability to focus on or remove speech from a specific 2D region. This would require understanding and manipulating an acoustic scene, isolating each speaker, and associating a 2D spatial context with each constituent speech. However, separating speech from a large number of concurrent speakers in a room into individual streams and identifying their precise 2D locations is challenging, even for the human brain. Here, we present the first acoustic swarm that demonstrates cooperative navigation with centimeter-resolution using sound, eliminating the need for cameras or external infrastructure. Our acoustic swarm forms a self-distributing wireless microphone array, which, along with our attention-based neural network framework, lets us separate and localize concurrent human speakers in the 2D space, enabling speech zones. Our evaluations showed that the acoustic swarm could localize and separate 3-5 concurrent speech sources in real-world unseen reverberant environments with median and 90-percentile 2D errors of 15 cm and 50 cm, respectively. Our system enables applications like mute zones (parts of the room where sounds are muted), active zones (regions where sounds are captured), multi-conversation separation and location-aware interaction.
- Open Access
- 18 June 2023
- Published by Association for Computing Machinery (ACM)
- p. 84-95
Wireless earbuds for low-cost hearing screening
We present the first wireless earbud hardware that can perform hearing screening by detecting otoacoustic emissions. The conventional wisdom has been that detecting otoacoustic emissions, which are the faint sounds generated by the cochlea, requires sensitive and expensive acoustic hardware. Thus, medical devices for hearing screening cost thousands of dollars and are inaccessible in low and middle income countries. We show that by designing wireless ear-buds using low-cost acoustic hardware and combining them with wireless sensing algorithms, we can reliably identify otoacoustic emissions and perform hearing screening. Our algorithms combine frequency modulated chirps with wideband pulses emitted from a low-cost speaker to reliably separate otoacoustic emissions from in-ear reflections and echoes. We conducted a clinical study with 50 ears across two healthcare sites. Our study shows that the low-cost earbuds detect hearing loss with 100% sensitivity and 89.7% specificity, which is comparable to the performance of a $8000 medical device. By developing low-cost and open-source wearable technology, our work may help address global health inequities in hearing screening by democratizing these medical devices. Open-source hardware and code can be found here: https://github.com/uw-x/OAEbuds
- 18 June 2023
- Published by Association for Computing Machinery (ACM)
- p. 84-95
Open AccessWireless earbuds for low-cost hearing screening
We present the first wireless earbud hardware that can perform hearing screening by detecting otoacoustic emissions. The conventional wisdom has been that detecting otoacoustic emissions, which are the faint sounds generated by the cochlea, requires sensitive and expensive acoustic hardware. Thus, medical devices for hearing screening cost thousands of dollars and are inaccessible in low and middle income countries. We show that by designing wireless ear-buds using low-cost acoustic hardware and combining them with wireless sensing algorithms, we can reliably identify otoacoustic emissions and perform hearing screening. Our algorithms combine frequency modulated chirps with wideband pulses emitted from a low-cost speaker to reliably separate otoacoustic emissions from in-ear reflections and echoes. We conducted a clinical study with 50 ears across two healthcare sites. Our study shows that the low-cost earbuds detect hearing loss with 100% sensitivity and 89.7% specificity, which is comparable to the performance of a $8000 medical device. By developing low-cost and open-source wearable technology, our work may help address global health inequities in hearing screening by democratizing these medical devices. Open-source hardware and code can be found here: https://github.com/uw-x/OAEbuds