Songyuan Yang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Songyuan Yang has a total of 13 co-authors. Here are the top co-authors by number of shared publications:
- Huibin Tan
College of Computer Science and Technology, National University of Defense Technology, Changsha, China
2 shared publications - Jianing Yang
National University of Defense Technology, Changsha, Hunan, China
1 shared publication - Jintao Huang
College of Computer Science and Technology, National University of Defense Technology, Changsha, China
1 shared publication - Kailun Yang
Hunan University, Changsha, China
1 shared publication - Kedi Zhang
National University of Defense Technology, Changsha, Hunan, China
1 shared publication - Ligong Cao
National University of Defense Technology, Changsha, Hunan, China
1 shared publication
Latest Publications
- Open Access
- 9 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 9363-9372
MoViS: Interactive 3D Human Motion Authoring via Zero-3D Visual Prompts
Generating spatially-aware 3D human motions from natural language remains a fundamental challenge, as text inherently lacks precise trajectory control. While recent methods condition generation on explicit 3D scenes to enforce physical grounding, they impose a severe human-computer interaction (HCI) bottleneck, demanding specialized 3D expertise and cumbersome user inputs. To democratize spatial control, we propose a paradigm shift: an interactive, zero-3D multimodal authoring framework named MoViS. MoViS empowers users to dictate complex spatial intent simply by drawing a 2D planar sketch (Visual Prompt) over a scene image, complemented by a text instruction. However, inferring 3D physical geometry from ambiguous 2D sketches presents extreme challenges in perspective scale and long-horizon intent drift. To overcome these, MoViS introduces a Physics-Grounded Cross-Modal Alignment mechanism, uniting 3D landmark distillation and continuous 2D perspective projection to embed true spatial awareness into a discrete generative backbone. Furthermore, we propose a cognitive-inspired Sketch-Driven Compositional Decoding strategy to robustly execute complex, multi-stage tasks. To catalyze this new paradigm, we construct IndoorViS, a large-scale multimodal dataset featuring 2,015 daily tasks across 1,173 diverse scenes. Extensive experiments demonstrate that MoViS strictly maintains the physical fidelity of explicit 3D-aware models while reducing human authoring overhead by over 90%, establishing a new standard for accessible and expressive embodied agent control.
- 9 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 9363-9372
Open AccessMoViS: Interactive 3D Human Motion Authoring via Zero-3D Visual Prompts
Generating spatially-aware 3D human motions from natural language remains a fundamental challenge, as text inherently lacks precise trajectory control. While recent methods condition generation on explicit 3D scenes to enforce physical grounding, they impose a severe human-computer interaction (HCI) bottleneck, demanding specialized 3D expertise and cumbersome user inputs. To democratize spatial control, we propose a paradigm shift: an interactive, zero-3D multimodal authoring framework named MoViS. MoViS empowers users to dictate complex spatial intent simply by drawing a 2D planar sketch (Visual Prompt) over a scene image, complemented by a text instruction. However, inferring 3D physical geometry from ambiguous 2D sketches presents extreme challenges in perspective scale and long-horizon intent drift. To overcome these, MoViS introduces a Physics-Grounded Cross-Modal Alignment mechanism, uniting 3D landmark distillation and continuous 2D perspective projection to embed true spatial awareness into a discrete generative backbone. Furthermore, we propose a cognitive-inspired Sketch-Driven Compositional Decoding strategy to robustly execute complex, multi-stage tasks. To catalyze this new paradigm, we construct IndoorViS, a large-scale multimodal dataset featuring 2,015 daily tasks across 1,173 diverse scenes. Extensive experiments demonstrate that MoViS strictly maintains the physical fidelity of explicit 3D-aware models while reducing human authoring overhead by over 90%, establishing a new standard for accessible and expressive embodied agent control.
- 21 April 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Robotics
- Vol. 42 (15523098) , 1921-1941
AnyUser: Translating Sketched User Intent Into Domestic Robots
We introduce AnyUser, a unified robotic instruction system for intuitive domestic task instruction via free-form sketches on camera images, optionally with language. AnyUser interprets multimodal inputs (sketch, vision, language) as spatial-semantic primitives to generate executable robot actions requiring no prior maps or models. Novel components include multimodal fusion for understanding and a hierarchical policy for robust action generation. Efficacy is shown via extensive evaluations: (1) Quantitative benchmarks on the large-scale dataset showing high accuracy in interpreting diverse sketch-based commands across various simulated domestic scenes. (2) Real-world validation on two distinct robotic platforms, a statically mounted 7-DoF assistive arm (KUKA LBR iiwa) and a dual-arm mobile manipulator (Realman RMC-AIDAL), performing representative tasks like targeted wiping and area cleaning, confirming the system's ability to ground instructions and execute them reliably in physical environments. (3) A comprehensive user study involving diverse demographics (elderly, simulated non-verbal, low technical literacy) demonstrating significant improvements in usability and task specification efficiency, achieving high task completion rates (85.7%-96.4%) and user satisfaction. AnyUser bridges the gap between advanced robotic capabilities and the need for accessible non-expert interaction, laying the foundation for practical assistive robots adaptable to real-world human environments.
- 21 April 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Robotics
- Vol. 42 (15523098) , 1921-1941
AnyUser: Translating Sketched User Intent Into Domestic Robots
We introduce AnyUser, a unified robotic instruction system for intuitive domestic task instruction via free-form sketches on camera images, optionally with language. AnyUser interprets multimodal inputs (sketch, vision, language) as spatial-semantic primitives to generate executable robot actions requiring no prior maps or models. Novel components include multimodal fusion for understanding and a hierarchical policy for robust action generation. Efficacy is shown via extensive evaluations: (1) Quantitative benchmarks on the large-scale dataset showing high accuracy in interpreting diverse sketch-based commands across various simulated domestic scenes. (2) Real-world validation on two distinct robotic platforms, a statically mounted 7-DoF assistive arm (KUKA LBR iiwa) and a dual-arm mobile manipulator (Realman RMC-AIDAL), performing representative tasks like targeted wiping and area cleaning, confirming the system's ability to ground instructions and execute them reliably in physical environments. (3) A comprehensive user study involving diverse demographics (elderly, simulated non-verbal, low technical literacy) demonstrating significant improvements in usability and task specification efficiency, achieving high task completion rates (85.7%-96.4%) and user satisfaction. AnyUser bridges the gap between advanced robotic capabilities and the need for accessible non-expert interaction, laying the foundation for practical assistive robots adaptable to real-world human environments.