SYSongyuan Yang

Songyuan Yang

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Songyuan Yang has a total of 13 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    MoViS: Interactive 3D Human Motion Authoring via Zero-3D Visual Prompts

    by Huibin Tan, Songyuan Yang, Jianing Yang, Yize Sui, Meng Zhang, Jintao Huang, Xiaoming Yan, Ligong Cao, Nan Hu, Kedi Zhang, et al.

    Generating spatially-aware 3D human motions from natural language remains a fundamental challenge, as text inherently lacks precise trajectory control. While recent methods condition generation on explicit 3D scenes to enforce physical grounding, they impose a severe human-computer interaction (HCI) bottleneck, demanding specialized 3D expertise and cumbersome user inputs. To democratize spatial control, we propose a paradigm shift: an interactive, zero-3D multimodal authoring framework named MoViS. MoViS empowers users to dictate complex spatial intent simply by drawing a 2D planar sketch (Visual Prompt) over a scene image, complemented by a text instruction. However, inferring 3D physical geometry from ambiguous 2D sketches presents extreme challenges in perspective scale and long-horizon intent drift. To overcome these, MoViS introduces a Physics-Grounded Cross-Modal Alignment mechanism, uniting 3D landmark distillation and continuous 2D perspective projection to embed true spatial awareness into a discrete generative backbone. Furthermore, we propose a cognitive-inspired Sketch-Driven Compositional Decoding strategy to robustly execute complex, multi-stage tasks. To catalyze this new paradigm, we construct IndoorViS, a large-scale multimodal dataset featuring 2,015 daily tasks across 1,173 diverse scenes. Extensive experiments demonstrate that MoViS strictly maintains the physical fidelity of explicit 3D-aware models while reducing human authoring overhead by over 90%, establishing a new standard for accessible and expressive embodied agent control.

  • AnyUser: Translating Sketched User Intent Into Domestic Robots

    by Songyuan Yang, Huibin Tan, Kailun Yang, Wenjing Yang, Shaowu Yang

    We introduce AnyUser, a unified robotic instruction system for intuitive domestic task instruction via free-form sketches on camera images, optionally with language. AnyUser interprets multimodal inputs (sketch, vision, language) as spatial-semantic primitives to generate executable robot actions requiring no prior maps or models. Novel components include multimodal fusion for understanding and a hierarchical policy for robust action generation. Efficacy is shown via extensive evaluations: (1) Quantitative benchmarks on the large-scale dataset showing high accuracy in interpreting diverse sketch-based commands across various simulated domestic scenes. (2) Real-world validation on two distinct robotic platforms, a statically mounted 7-DoF assistive arm (KUKA LBR iiwa) and a dual-arm mobile manipulator (Realman RMC-AIDAL), performing representative tasks like targeted wiping and area cleaning, confirming the system's ability to ground instructions and execute them reliably in physical environments. (3) A comprehensive user study involving diverse demographics (elderly, simulated non-verbal, low technical literacy) demonstrating significant improvements in usability and task specification efficiency, achieving high task completion rates (85.7%-96.4%) and user satisfaction. AnyUser bridges the gap between advanced robotic capabilities and the need for accessible non-expert interaction, laying the foundation for practical assistive robots adaptable to real-world human environments.