Davide Rocchesso

RhyGlyph is a radial visualization system that represents rhythmic interaction as three pairwise spacetime trajectories within a compact three-wedge glyph, supporting both overview and detailed reading. Building on this representation, RhyDiff is a generative framework that learns aligned latent spaces for symbolic drum patterns and the encoded trajectory features through paired VQ-VAEs and FiLM-conditioned diffusion. Users edit relational trajectories and metadata to generate new drum patterns by treating visualization itself as a control surface.

We investigate navigation in two dimensions with velocity control, using a single-button interface. Users adjust the controlled-object speed through rhythmic tapping and its direction by pressing and tilting, releasing the button to finalize the rotation. Feedback to control action and object motion is provided by integration of multiple sensory modalities. Tactile pulses are delivered at 30-degree intervals during rotation, emulating the detents of a rotary encoder. Simultaneously, a sonic glissando accompanies rotation, thus rendering upward or downward rotation. Both visual and auditory cues are used to provide absolute positional feedback. Discrete notes are rhythmically played, whose pitch indicates vertical position, while stereo audio panning follows horizontal position. The rhythmic pace aligns with on-screen object speed, as dictated by tapping time intervals. Two studies were designed around a target-following task, under different sensory conditions. Study 1 has shown that target tracking can be effectively performed with multisensory rhythmic interaction. This is true also when the controlled object is intermittently hidden to view, although it was not possible to measure the advantage provided by auditory-tactile feedback. In study 2, no significant performance differences were observed between auditory and tactile conditions in situations of intermittent visual feedback, indicating that, if the two kinds of non-visual feedback are effective, they are essentially equivalent.

We demonstrate two-dimensional navigation with velocity control on a single button. Users can vary the speed of the controlled object by rhythm tapping, and can control direction by pressing and tilting, releasing the button once the desired rotation is achieved. Feedback is multisensory. Tactile pulses are being delivered at 30-degree intervals during rotation, simulating the detents of a rotary encoder. Simultaneously, a sonic glissando accompanies rotation, raising or lowering pitch according to the change of direction. Absolute positional feedback is provided visually as well as auditorilly, with an intermittent auditory tone whose pitch conveys vertical position, panned to the left or to the right depending on horizontal position. The rhythmic pace corresponds directly to the on-screen element speed, defined by the tapping interval. Participants will be engaged in a target-following task, thus being able to appreciate the precise speed and direction control of the multisensory navigation experience.