Stefan Weinzierl

Accurate free-field acoustic simulations require non-reflective boundary conditions to suppress spurious reflections at the computational domain boundaries. Although several characteristic-based formulations for direct (forward) simulations have been proposed in recent decades, adjoint formulations of such characteristic-based boundary conditions (CBCs) have received limited attention in the literature and lack a comprehensive analysis. [Formula: see text]This paper presents the derivation and evaluation of adjoint CBCs complementing existing direct CBCs. Both direct and adjoint CBCs are applied to the nonlinear Euler equations and linear acoustic equations in time-domain simulations. The resulting boundary treatments are compared in terms of accuracy, reflection behavior, and direct-adjoint consistency. [Formula: see text]The CBCs are implemented using a single-point approach with both static and dynamic wave-vector estimates and are optionally combined with a sponge layer. Over the evaluated frequency range of 350–4000[Formula: see text]Hz, spurious reflections are effectively attenuated in both direct and adjoint simulations. The adjoint CBCs show reflection behavior consistent with their direct counterparts, demonstrating that the proposed formulation provides a consistent open-boundary treatment for adjoint time-domain acoustics.

Source directivity is an integral part of room acoustic simulation, affecting not only the intensity and frequency balance of the direct sound depending on source orientation, but also room-acoustic parameters, such as the direct-to-reverberant energy ratio. While the directivity of a loudspeaker is time-invariant, the directivity of other sound sources, such as the human voice, varies over time. Whether these time-dependent directivity changes must be fully considered for a plausible binaural rendering of speech remains an open question, requiring rendering solutions that can reproduce time-varying directivity. This work provides three key contributions. First, it introduces a feature for liveRAZR—a real-time, six-degrees-of-freedom room acoustic simulator—to render sound sources with time-varying directivity from Ambisonics input. Second, it proposes a method to generate speech material with time-varying directivity from monaural recordings using phoneme-dependent directivity filters. Third, it presents a listening experiment comparing fluent speech with multiple time-varying and static directivities regarding transfer-plausibility and perceptual qualities. Time-varying directivity did not significantly improve transfer-plausibility or liveliness and source-orientation clarity ratings compared to static, measurement-based directivity in the tested scenario. However, analytically obtained spherical head model directivity reduced these perceptual qualities, indicating that measurement-based voice directivity is preferable for demanding immersive extended reality applications.

The preferences of music ensembles for the architectural and acoustic features of the stages on which they perform have primarily been examined through field studies. Although these studies provide high ecological validity, they are limited in terms of the architectural variations that can be studied and by the confounding effects resulting from the unique features of the halls in which the experiments are conducted. This study took an experimental approach in which six typical architectural features of stage enclosures were first identified through discussions with concert hall design experts. These features were then systematically varied to produce a set of 256 virtual halls. Using latency-free dynamic binaural synthesis in an anechoic environment, chamber music ensembles of up to five musicians were able to perform simultaneously in the same virtual space. The results revealed the influence of architectural features such as stage width and height on perceived Support and Reverberance. Of the many stage acoustic parameters analyzed, only the top-to-horizontal and top-to-side energy ratios were significantly correlated with perceived quality, Transparency, Support, Brightness, and Bass on stage.

This study investigates whether the visual color design of concert halls influences the room acoustic impression during musical performances. While previous research has failed to show effects on the perceived loudness and reverberance of music venues, the present audiovisual experiment explores the cross-modal impact of vision on a broader set of room acoustic properties, including timbre-related attributes such as brilliance, warmth, clarity, and roughness. For this purpose, 48 participants rated eight room acoustic attributes while listening to motion-tracked musical performances in virtual concert halls with systematically varied color schemes. Musical experience and expertise of the participants were assessed as moderating variables using the Goldsmiths Musical Sophistication Index. The results showed no significant effects of color on perceived loudness or reverberance, suggesting that these attributes remain predominantly unimodal. However, a significant visual influence was observed on the perceived acoustic “Warmth” and overall “Liking” of the performance. These effects were significantly moderated by the participants' musical experience. Thus, this study suggests that concert hall color design can affect auditory timbre perception through semantically mediated cross-modal interactions, highlighting the interplay between visual aesthetics and auditory experience during musical performances.

Functional sounds—typically brief, nonverbal audio cues used in the interfaces of electronic devices—play a critical role in human–machine interaction but remain largely unexplored within music information retrieval (MIR). This study proposes a data-driven framework that uses musically informed audio features to predict the perceived semantic expression of functional sounds. Our three-stage pipeline first uses unsupervised feature extraction to transform 805 functional sounds into high-level topic distributions for timbre, chroma, and loudness using Gaussian mixture models and latent Dirichlet allocation. Second, these features train multi-output regression models to predict 19 perceptual dimensions from the FBMUX framework, with a random forest regressor achieving the best performance. Finally, a listening experiment assesses how well the model predictions align with user perceptions. Interpretability analyses further reveal how individual features contribute to model predictions. This work contributes to MIR by expanding its scope to the domain of functional, non-musical audio. It presents a novel application of MIR techniques, demonstrating that structured, musically informed descriptors can support perceptual modeling in domains with limited data and high subjective variance. It contributes a transferable approach and highlights the potential of MIR to inform human–machine interaction and sound design.

This paper presents a novel approach to neural instrument sound synthesis using a two-stage semi-supervised learning framework capable of generating pitch-accurate, high-quality music samples from an expressive timbre latent space. Existing approaches that achieve sufficient quality for music production often rely on high-dimensional latent representations that are difficult to navigate and provide unintuitive user experiences. We address this limitation through a two-stage training paradigm: first, we train a pitch-timbre disentangled 2D representation of audio samples using a Variational Autoencoder; second, we use this representation as conditioning input for a Transformer-based generative model. The learned 2D latent space serves as an intuitive interface for navigating and exploring the sound landscape. We demonstrate that the proposed method effectively learns a disentangled timbre space, enabling expressive and controllable audio generation with reliable pitch conditioning. Experimental results show the model's ability to capture subtle variations in timbre while maintaining a high degree of pitch accuracy. The usability of our method is demonstrated in an interactive web application, highlighting its potential as a step towards future music production environments that are both intuitive and creatively empowering: https://pgesam.faresschulz.com

Differentiable rendering has become a key ingredient in solving challenging inverse problems in computer graphics and vision. Existing systems can simulate and differentiate the spatial propagation of light. We exploit the duality of light transport simulations and geometric acoustics to apply differential rendering techniques to established acoustic simulation methods. The resulting system is capable of simulating sound according to the geometrical acoustics model and computing derivatives of the output energy spectrograms with respect to arbitrary parameters of the scene, including materials, emitters, microphones, and scene geometry. Contrary to current differentiable transient rendering, we can handle arbitrary simulation depths and achieve constant memory and linear execution times by presenting a temporal extension of Path Replay Backpropagation [Vicini et al. 2021]. We verify our model against established simulation software, and demonstrate the capabilities of optimization with gradients at examples of inverse acoustics and optimizing room parameters. This opens up a new field of research for acoustic optimization that could be as impactful for the acoustic community as differentiable rendering was for the graphics community.