RANF: Neural Field-Based HRTF Spatial Upsampling with Retrieval Augmentation and Parameter Efficient Fine-Tuning
Open Access
- 4 December 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Open Journal of Signal Processing
- Vol. PP, 1-10
Abstract
This paper gives an in-depth description of our submission to Task 2 of the Listener Acoustic Personalization (LAP) challenge 2024, which aims to reconstruct head-related transfer functions (HRTFs) with dense spatial grids from sparse measurements. Neural fields (NFs) with parameter-efficient fine-tuning (PEFT) have led to dramatic performance improvements in HRTF spatial upsampling and personalization. Despite these advances, spatial upsampling performance remains limited in scenarios with very sparse measurements. Our proposed system, named retrieval-augmented NF (RANF), incorporates HRTFs retrieved from a dataset as auxiliary inputs. We leverage multiple retrievals via transform-average-concatenate and adopt a PEFT technique tailored for retrieval augmentation. Furthermore, we capitalize on the results of a signal-processing-based spatial upsampling method as optional inputs. By incorporating these auxiliary inputs, our system demonstrated state-of-the-art performance on the SONICOM dataset and placed first in Task 2 of the LAP challenge 2024.Keywords
This publication has 39 references indexed in Scilit:
- Retrieval-Augmented Layout Transformer for Content-Aware Layout Generation2024
- Retrieval Augmented End-to-End Spoken Dialog Models2024
- Retrieval-Augmented Text-to-Audio Generation2024
- Implicit HRTF Modeling Using Temporal Convolutional Networks2021
- On the Predictability of Hrtfs from Ear Shapes Using Deep Networks2021
- End-to-end Microphone Permutation and Number Invariant Multi-channel Speech Separation2020
- Experiments on localization accuracy with non-individual and individual HRTFs comparing static and dynamic reproduction methodsbioRxiv, 2020
- HRTF personalization based on weighted sparse representation of anthropometric features2017
- Interpolation and range extrapolation of HRTFs [head related transfer functions]2004
- Perceptual consequences of interpolating head-related transfer functions during spatial synthesis2002