TangiMap: Context-Aware Generative Function Mapping for Tangible Input Devices
Open Access
- 2 November 2026
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 1-13
Abstract
While tangible input devices with physical controls such as knobs and sliders enable efficient interaction, they often lack the flexibility to adapt to dynamic contexts, limiting their use to predefined applications with fixed functionalities. In this paper, we introduce TangiMap, a proof-of-concept system that bridges tangible input devices with dynamic software contexts by generating adaptive function mappings for hardware controls. To achieve this, TangiMap analyzes the user’s declared hardware specifications, the available controls, and the target software context, and uses large language models (LLMs) to assign suitable functions to each control element. To support diverse contexts, TangiMap presents multiple strategies such as API queries and screen parsing. We demonstrate the system through four scenarios: interactive adjustment of natural language parameters in 3D modeling, live music performance, task-aware mapping of on-screen controls in image editing, and dynamic puppeteering in animation.Keywords
This publication has 39 references indexed in Scilit:
- BISCUIT: Scaffolding LLM-Generated Code with Ephemeral UIs in Computational Notebooks2024
- Dynamic Tangible User Interface PalettesLecture Notes in Computer Science, 2013
- Stane2008
- VoodooSketch2008
- Automatically generating user interfaces adapted to users' motor and vision capabilities2007
- Siftables2007
- The reacTable2007
- Design research & tangible interaction2007
- Huddle2006
- The SensOrg: a musical cyberinstrument with a cognitive ergonomic touch2002