Andrew McPherson

Interest is growing in how relational ontologies and more-than-human design methodologies – so-called ‘entanglement’ theories – might inform digital music research. To what extent does the growing popularity of such theories portend a change in technical or musical practices versus putting new gloss on long-standing ideas? Can or should entanglement theories be distilled into concrete design frameworks? This paper starts from the opposite premise: rather than offering an unambiguous roadmap for designers, entanglement theories are at their most powerful in destabilising ideas and worldviews that have become so ingrained as to become invisible. In digital musical instrument (DMI) design, this barely visible infrastructure consists partly of an ecosystem of stable, context-agnostic concepts about music: analytical descriptors such as notes, pitches, onsets and gestures which get inverted into the building blocks of sound-producing technical systems. However, design is not as simple as inverted analysis. This paper argues that treating familiar musical concepts as authoritative is responsible for several long-standing conundrums facing DMI research. The paper juxtaposes ideas from entanglement literature with brief vignettes concerning instruments new and old, arguing that musical instruments enact the very concepts they take as pre-existing, and that everything about musical discourse and practice should remain up for grabs.

In this article, a thematic analysis of interviews with 20 visually impaired and blind (VIB) people working creatively with sound highlights three prevalent gaps that impact access to music software: gaps in technology, education and information. The authors highlight the pivotal role of online communities of VIB people in bridging these gaps. The authors argue that it is impossible to fully anticipate how an artefact will be used until it is, in fact, in use, and in order for a digital audio workstation (DAW) to reach its full potential regarding accessibility, it needs to be malleable, responding to the evolving requirements of interdependent communities of visually impaired and blind sound creatives (VIBSCs). Furthermore, the authors provide three guidelines for accessible music software design drawing on notions of convivial tools.

This article examines how digital systems designers distil the messiness and ambiguity of the world into concrete data that can be processed by computing systems. Using Karen Barad's agential realism as a guide, we explore how data is fundamentally entangled with the tools and theories of its measurement. We examine data-enabled artefacts acting as Baradian apparatuses: they do not exist independently of the phenomenon they seek to measure but rather collect and co-produce observations from within their entangled state: the phenomenon and the apparatus co-constitute one another. Connecting Barad's quantum view of indeterminacy to the prevailing HCI discourse on the opportunities and challenges of ambiguity, we suggest that the very act of trying to stabilise a conceptual interpretation of data within an artefact has the paradoxical effect of amplifying and shifting ambiguity in interaction. We illustrate these ideas through three case studies from our own practices of designing digital musical instruments (DMIs). DMIs necessarily encode symbolic and music-theoretical knowledge as part of their internal operation, even as conceptual knowledge is not their intended outcome. In each case, we explore the nature of the apparatus, what phenomena it co-produces, and where the ambiguity lies to suggest approaches for design using these abstract theoretical frameworks.

Timbre is a primary mode of expression in diverse musical contexts. However, prevalent audio-driven synthesis methods predominantly rely on pitch and loudness envelopes, effectively flattening timbral expression from the input. Our approach draws on the concept of timbre analogies and investigates how timbral expression from an input signal can be mapped onto controls for a synthesizer. Leveraging differentiable digital signal processing, our method facilitates direct optimization of synthesizer parameters through a novel feature difference loss. This loss function, designed to learn relative timbral differences between musical events, prioritizes the subtleties of graded timbre modulations within phrases, allowing for meaningful translations in a timbre space. Using snare drum performances as a case study, where timbral expression is central, we demonstrate real-time timbre remapping from acoustic snare drums to a differentiable synthesizer modeled after the Roland TR-808.

This paper investigates sound and music interactions arising from the use of electromyography (EMG) to instrumentalise signals from muscle exertion of the human body. We situate EMG within a family of embodied interaction modalities, where it occupies a middle ground, considered as a “signal from the inside” compared with external observations of the body (e.g., motion capture), but also seen as more volitional than neurological states recorded by brain electroencephalogram (EEG). To understand the messiness of gestural interaction afforded by EMG, we revisit the phenomenological turn in HCI, reading Paul Dourish’s work on the transparency of “ready-to-hand” technologies against the grain of recent posthumanist theories, which offer a performative interpretation of musical entanglements between bodies, signals, and representations. We take music performance as a use case, reporting on the opportunities and constraints posed by EMG in workshop-based studies of vocal, instrumental, and electronic practices. We observe that across our diverse range of musical subjects, they consistently challenged notions of EMG as a transparent tool that directly registered the state of the body, reporting instead that it took on “present-at-hand” qualities, defamiliarising the performer’s own sense of themselves and reconfiguring their embodied practice.

Entanglement theories are well established in HCI discourse. These involve a commitment to view human experience in encounters with technology as relational and contingent, and research apparatuses as co-producers rather than passive observers of phenomena. In this paper, we argue that sound is the sensory modality best suited to the investigation of entanglements. Materialist theories of sound and listening guide both the design of a novel interactive sound installation and the methodological approach of a participant study exploring the experience of listening. We present a diffractive analysis whereby micro-phenomenological interview data is read with sonic theories, generating accounts that might otherwise remain mute: the temporal fluctuation and physical feeling of proximity in listener entanglements with sound, somatic intention setting, and plural interpretations of interactivity. Finally, we offer a series of provocations for HCI to embrace qualities of the sonic and consider epistemological positions grounded in other sense modalities.

Tone Transfer is a novel deep-learning technique for interfacing a sound source with a synthesizer, transforming the timbre of audio excerpts while keeping their musical form content. Due to its good audio quality results and continuous controllability, it has been recently applied in several audio processing tools. Nevertheless, it still presents several shortcomings related to poor sound diversity, and limited transient and dynamic rendering, which we believe hinder its possibilities of articulation and phrasing in a real-time performance context. In this work, we present a discussion on current Tone Transfer architectures for the task of controlling synthetic audio with musical instruments and discuss their challenges in allowing expressive performances. Next, we introduce Envelope Learning, a novel method for designing Tone Transfer architectures that map musical events using a training objective at the synthesis parameter level. Our technique can render note beginnings and endings accurately and for a variety of sounds; these are essential steps for improving musical articulation, phrasing, and sound diversity with Tone Transfer. Finally, we implement a VST plugin for real-time live use and discuss possibilities for improvement.