Bandhav Veluri
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Bandhav Veluri has a total of 17 co-authors. Here are the top co-authors by number of shared publications:
- Shyamnath Gollakota
University of Washington, Seattle, USA
6 shared publications - Malek Itani
University of Washington, Seattle, USA
3 shared publications - Ali Saffari
University of Washington, Seattle, United States of America
2 shared publications - Collin Pernu
University of Washington, Seattle, USA
2 shared publications - Takuya Yoshioka
Microsoft Redmond WA USA
2 shared publications - Tuochao Chen
School of Computer Science & Engineering University of Washington
2 shared publications
Latest Publications
- 31 January 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Pervasive Computing
- Vol. 23 (4) , 53-55
Creating Super-Hearing Capabilities With Real-Time AI
Sound is a fundamental medium through which we perceive our environment. However, today we are surrounded by a cacophony of sounds that can overwhelm our senses. Indeed, human auditory perception can be limited in noisy environments. Imagine being in a crowded room with a cacophony of sounds and having the ability to focus on or remove sounds from a specific person or based on their semantic descriptions. This requires understanding and manipulating an acoustic scene, isolating each sound, and associating a spatial context or semantic meaning with each constituent sound or speaker—a challenging set of tasks even for the human brain in noisy environments.
- 31 January 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Pervasive Computing
- Vol. 23 (4) , 53-55
Creating Super-Hearing Capabilities With Real-Time AI
Sound is a fundamental medium through which we perceive our environment. However, today we are surrounded by a cacophony of sounds that can overwhelm our senses. Indeed, human auditory perception can be limited in noisy environments. Imagine being in a crowded room with a cacophony of sounds and having the ability to focus on or remove sounds from a specific person or based on their semantic descriptions. This requires understanding and manipulating an acoustic scene, isolating each sound, and associating a spatial context or semantic meaning with each constituent sound or speaker—a challenging set of tasks even for the human brain in noisy environments.
- Open Access
- 11 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-16
IRIS: Wireless ring for vision-based smart home interaction
Integrating cameras into wireless smart rings has been challenging due to size and power constraints. We introduce IRIS, the first wireless vision-enabled smart ring system for smart home interactions. Equipped with a camera, Bluetooth radio, inertial measurement unit (IMU), and an onboard battery, IRIS meets the small size, weight, and power (SWaP) requirements for ring devices. IRIS is context-aware, adapting its gesture set to the detected device, and can last for 16-24 hours on a single charge. IRIS leverages the scene semantics to achieve instance-level device recognition. In a study involving 23 participants, IRIS consistently outpaced voice commands, with a higher proportion of participants expressing a preference for IRIS over voice commands regarding toggling a device’s state, granular control, and social acceptability. Our work pushes the boundary of what is possible with ring form-factor devices, addressing system challenges and opening up novel interaction capabilities.
- 11 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-16
Open AccessIRIS: Wireless ring for vision-based smart home interaction
Integrating cameras into wireless smart rings has been challenging due to size and power constraints. We introduce IRIS, the first wireless vision-enabled smart ring system for smart home interactions. Equipped with a camera, Bluetooth radio, inertial measurement unit (IMU), and an onboard battery, IRIS meets the small size, weight, and power (SWaP) requirements for ring devices. IRIS is context-aware, adapting its gesture set to the detected device, and can last for 16-24 hours on a single charge. IRIS leverages the scene semantics to achieve instance-level device recognition. In a study involving 23 participants, IRIS consistently outpaced voice commands, with a higher proportion of participants expressing a preference for IRIS over voice commands regarding toggling a device’s state, granular control, and social acceptability. Our work pushes the boundary of what is possible with ring form-factor devices, addressing system challenges and opening up novel interaction capabilities.
- Top Cited PapersOpen Access
- 11 May 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-16
Look Once to Hear: Target Speech Hearing with Noisy Examples
In crowded settings, the human brain can focus on speech from a target speaker, given prior knowledge of how they sound. We introduce a novel intelligent hearable system that achieves this capability, enabling target speech hearing to ignore all interfering speech and noise, but the target speaker. A naïve approach is to require a clean speech example to enroll the target speaker. This is however not well aligned with the hearable application domain since obtaining a clean example is challenging in real world scenarios, creating a unique user interface problem. We present the first enrollment interface where the wearer looks at the target speaker for a few seconds to capture a single, short, highly noisy, binaural example of the target speaker. This noisy example is used for enrollment and subsequent speech extraction in the presence of interfering speakers and noise. Our system achieves a signal quality improvement of 7.01 dB using less than 5 seconds of noisy enrollment audio and can process 8 ms of audio chunks in 6.24 ms on an embedded CPU. Our user studies demonstrate generalization to real-world static and mobile speakers in previously unseen indoor and outdoor multipath environments. Finally, our enrollment interface for noisy examples does not cause performance degradation compared to clean examples, while being convenient and user-friendly. Taking a step back, this paper takes an important step towards enhancing the human auditory perception with artificial intelligence.
- 11 May 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-16
Top Cited PapersOpen AccessLook Once to Hear: Target Speech Hearing with Noisy Examples
In crowded settings, the human brain can focus on speech from a target speaker, given prior knowledge of how they sound. We introduce a novel intelligent hearable system that achieves this capability, enabling target speech hearing to ignore all interfering speech and noise, but the target speaker. A naïve approach is to require a clean speech example to enroll the target speaker. This is however not well aligned with the hearable application domain since obtaining a clean example is challenging in real world scenarios, creating a unique user interface problem. We present the first enrollment interface where the wearer looks at the target speaker for a few seconds to capture a single, short, highly noisy, binaural example of the target speaker. This noisy example is used for enrollment and subsequent speech extraction in the presence of interfering speakers and noise. Our system achieves a signal quality improvement of 7.01 dB using less than 5 seconds of noisy enrollment audio and can process 8 ms of audio chunks in 6.24 ms on an embedded CPU. Our user studies demonstrate generalization to real-world static and mobile speakers in previously unseen indoor and outdoor multipath environments. Finally, our enrollment interface for noisy examples does not cause performance degradation compared to clean examples, while being convenient and user-friendly. Taking a step back, this paper takes an important step towards enhancing the human auditory perception with artificial intelligence.
- Open Access
- 29 October 2023
- Published by Association for Computing Machinery (ACM)
- p. 1-15
Semantic Hearing: Programming Acoustic Scenes with Binaural Hearables
- 29 October 2023
- Published by Association for Computing Machinery (ACM)
- p. 1-15
Open AccessSemantic Hearing: Programming Acoustic Scenes with Binaural Hearables
- Open Access
- 2 October 2023
- Published by Association for Computing Machinery (ACM)
- p. 1-17
NeuriCam: Key-Frame Video Super-Resolution and Colorization for IoT Cameras
We present NeuriCam, a novel deep learning-based system to achieve video capture from low-power dual-mode IoT camera systems. Our idea is to design a dual-mode camera system where the first mode is low power (1.1 mW) but only outputs grey-scale, low resolution and noisy video and the second mode consumes much higher power (100 mW) but outputs color and higher resolution images. To reduce total energy consumption, we heavily duty cycle the high power mode to output an image only once every second. The data for this camera system is then wirelessly sent to a nearby plugged-in gateway, where we run our real-time neural network decoder to reconstruct a higher-resolution color video. To achieve this, we introduce an attention feature filter mechanism that assigns different weights to different features, based on the correlation between the feature map and the contents of the input frame at each spatial location. We design a wireless hardware prototype using off-the-shelf cameras and address practical issues including packet loss and perspective mismatch. Our evaluations show that our dual-camera approach reduces energy consumption by 7x compared to existing systems. Further, our model achieves an average greyscale PSNR gain of 3.7 dB over prior single and dual-camera video super-resolution methods and 5.6 dB RGB gain over prior color propagation methods. Open-source code: https://github.com/vb000/NeuriCam.
- 2 October 2023
- Published by Association for Computing Machinery (ACM)
- p. 1-17
Open AccessNeuriCam: Key-Frame Video Super-Resolution and Colorization for IoT Cameras
We present NeuriCam, a novel deep learning-based system to achieve video capture from low-power dual-mode IoT camera systems. Our idea is to design a dual-mode camera system where the first mode is low power (1.1 mW) but only outputs grey-scale, low resolution and noisy video and the second mode consumes much higher power (100 mW) but outputs color and higher resolution images. To reduce total energy consumption, we heavily duty cycle the high power mode to output an image only once every second. The data for this camera system is then wirelessly sent to a nearby plugged-in gateway, where we run our real-time neural network decoder to reconstruct a higher-resolution color video. To achieve this, we introduce an attention feature filter mechanism that assigns different weights to different features, based on the correlation between the feature map and the contents of the input frame at each spatial location. We design a wireless hardware prototype using off-the-shelf cameras and address practical issues including packet loss and perspective mismatch. Our evaluations show that our dual-camera approach reduces energy consumption by 7x compared to existing systems. Further, our model achieves an average greyscale PSNR gain of 3.7 dB over prior single and dual-camera video super-resolution methods and 5.6 dB RGB gain over prior color propagation methods. Open-source code: https://github.com/vb000/NeuriCam.
- Open Access
NeuriCam: Key-Frame Video Super-Resolution and Colorization for IoT Cameras
We present NeuriCam, a novel deep learning-based system to achieve video capture from low-power dual-mode IoT camera systems. Our idea is to design a dual-mode camera system where the first mode is low-power (1.1 mW) but only outputs grey-scale, low resolution, and noisy video and the second mode consumes much higher power (100 mW) but outputs color and higher resolution images. To reduce total energy consumption, we heavily duty cycle the high power mode to output an image only once every second. The data for this camera system is then wirelessly sent to a nearby plugged-in gateway, where we run our real-time neural network decoder to reconstruct a higher-resolution color video. To achieve this, we introduce an attention feature filter mechanism that assigns different weights to different features, based on the correlation between the feature map and the contents of the input frame at each spatial location. We design a wireless hardware prototype using off-the-shelf cameras and address practical issues including packet loss and perspective mismatch. Our evaluations show that our dual-camera approach reduces energy consumption by 7x compared to existing systems. Further, our model achieves an average greyscale PSNR gain of 3.7 dB over prior single and dual-camera video super-resolution methods and 5.6 dB RGB gain over prior color propagation methods. Open-source code: https://github.com/vb000/NeuriCam.
Open AccessNeuriCam: Key-Frame Video Super-Resolution and Colorization for IoT Cameras
We present NeuriCam, a novel deep learning-based system to achieve video capture from low-power dual-mode IoT camera systems. Our idea is to design a dual-mode camera system where the first mode is low-power (1.1 mW) but only outputs grey-scale, low resolution, and noisy video and the second mode consumes much higher power (100 mW) but outputs color and higher resolution images. To reduce total energy consumption, we heavily duty cycle the high power mode to output an image only once every second. The data for this camera system is then wirelessly sent to a nearby plugged-in gateway, where we run our real-time neural network decoder to reconstruct a higher-resolution color video. To achieve this, we introduce an attention feature filter mechanism that assigns different weights to different features, based on the correlation between the feature map and the contents of the input frame at each spatial location. We design a wireless hardware prototype using off-the-shelf cameras and address practical issues including packet loss and perspective mismatch. Our evaluations show that our dual-camera approach reduces energy consumption by 7x compared to existing systems. Further, our model achieves an average greyscale PSNR gain of 3.7 dB over prior single and dual-camera video super-resolution methods and 5.6 dB RGB gain over prior color propagation methods. Open-source code: https://github.com/vb000/NeuriCam.