Hsiao-Yu Chen
Research Subjects
Hsiao-Yu Chen has a total of 20 co-authors. Here are the top co-authors by number of shared publications:
- Tuur Stuyck
Meta Reality Labs USA
7 shared publications - Egor Larionov
Meta Reality Labs USA
6 shared publications - Gene Wei-Chin Lin
Meta Reality Labs Canada
6 shared publications - Doug Roble
Meta, Menlo Park, USA
4 shared publications - Ladislav Kavan
Meta Reality Labs
3 shared publications - Philipp Herholz
Independent, Zurich, Switzerland
2 shared publications
Latest Publications
- Open Access
- 30 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 1-11
HyperBones: Realtime Bone-driven Neural Garment Simulation with Hypernetwork Conditioning
- 30 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 1-11
Open AccessHyperBones: Realtime Bone-driven Neural Garment Simulation with Hypernetwork Conditioning
- 14 April 2026
- Published by Wiley in Computer Graphics Forum
SkinCells: Sparse Skinning using Voronoi Cells
For decades, real‐time skinning has been the cornerstone of character animation in visual effects and games. Despite its importance, the creation of animatable digital assets remains a labor‐intensive manual process. Existing automated tools frequently struggle with intricate geometries, often necessitating significant manual refinement to reach production standards. We present a robust, fully automated method for generating high‐quality skinning weights from a standard mesh and skeleton in a canonical A‐ or T‐pose. Unlike traditional approaches, our framework offers direct sparsity controls to limit bone influences per vertex – a critical requirement for maintaining performance in large‐scale mobile environments. Furthermore, we address the challenge of Level‐of‐Detail (LoD) management by optimizing weights within a continuous spatial volume rather than on discrete vertices. This allows a single optimization pass to be applied seamlessly across multiple asset resolutions and variations. Central to our approach is a novel parameterized family of functions, we call SkinCells. We demonstrate that our method consistently produces stable, high‐quality results even in complex scenarios where standard biharmonic weight computations fail.
- 14 April 2026
- Published by Wiley in Computer Graphics Forum
SkinCells: Sparse Skinning using Voronoi Cells
For decades, real‐time skinning has been the cornerstone of character animation in visual effects and games. Despite its importance, the creation of animatable digital assets remains a labor‐intensive manual process. Existing automated tools frequently struggle with intricate geometries, often necessitating significant manual refinement to reach production standards. We present a robust, fully automated method for generating high‐quality skinning weights from a standard mesh and skeleton in a canonical A‐ or T‐pose. Unlike traditional approaches, our framework offers direct sparsity controls to limit bone influences per vertex – a critical requirement for maintaining performance in large‐scale mobile environments. Furthermore, we address the challenge of Level‐of‐Detail (LoD) management by optimizing weights within a continuous spatial volume rather than on discrete vertices. This allows a single optimization pass to be applied seamlessly across multiple asset resolutions and variations. Central to our approach is a novel parameterized family of functions, we call SkinCells. We demonstrate that our method consistently produces stable, high‐quality results even in complex scenarios where standard biharmonic weight computations fail.
- 30 March 2026
- Published by Wiley in Computer Graphics Forum
Neuralocks: Real‐Time Dynamic Neural Hair Simulation
Real‐time hair simulation is a vital component in creating believable virtual avatars, as it provides a sense of immersion and authenticity. The dynamic behavior of hair, such as bouncing or swaying in response to character movements like jumping or walking, plays a significant role in enhancing the overall realism and engagement of virtual experiences. Current methods for simulating hair have been constrained by two primary approaches: highly optimized physics‐based systems and neural methods. However, state‐of‐the‐art neural techniques have been limited to quasi‐static solutions, failing to capture the dynamic behavior of hair. This paper introduces a novel neural method that breaks through these limitations, achieving efficient and stable dynamic hair simulation while outperforming existing approaches. We propose a fully self‐supervised method which can be trained without any manual intervention or artist generated training data allowing the method to be integrated with hair reconstruction methods to enable automatic end‐to‐end methods for avatar reconstruction. Our approach harnesses the power of compact, memory‐efficient neural networks to simulate hair at the strand level, allowing for the simulation of diverse hairstyles without excessive computational resources or memory requirements. We validate the effectiveness of our method through a variety of hairstyle examples, showcasing its potential for real‐world applications.
- 30 March 2026
- Published by Wiley in Computer Graphics Forum
Neuralocks: Real‐Time Dynamic Neural Hair Simulation
Real‐time hair simulation is a vital component in creating believable virtual avatars, as it provides a sense of immersion and authenticity. The dynamic behavior of hair, such as bouncing or swaying in response to character movements like jumping or walking, plays a significant role in enhancing the overall realism and engagement of virtual experiences. Current methods for simulating hair have been constrained by two primary approaches: highly optimized physics‐based systems and neural methods. However, state‐of‐the‐art neural techniques have been limited to quasi‐static solutions, failing to capture the dynamic behavior of hair. This paper introduces a novel neural method that breaks through these limitations, achieving efficient and stable dynamic hair simulation while outperforming existing approaches. We propose a fully self‐supervised method which can be trained without any manual intervention or artist generated training data allowing the method to be integrated with hair reconstruction methods to enable automatic end‐to‐end methods for avatar reconstruction. Our approach harnesses the power of compact, memory‐efficient neural networks to simulate hair at the strand level, allowing for the simulation of diverse hairstyles without excessive computational resources or memory requirements. We validate the effectiveness of our method through a variety of hairstyle examples, showcasing its potential for real‐world applications.
- Open Access
- 8 August 2025
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Computer Graphics and Interactive Techniques
- Vol. 8 (4) , 1-18
Self-supervised Learning of Latent Space Dynamics 57
Modeling the dynamic behavior of deformable objects is crucial for creating realistic digital worlds. While conventional simulations produce high-quality motions, their computational costs are often prohibitive. Subspace simulation techniques address this challenge by restricting deformations to a lower-dimensional space, improving performance while maintaining visually compelling results. However, even subspace methods struggle to meet the stringent performance demands of portable devices such as virtual reality headsets and mobile platforms. To overcome this limitation, we introduce a novel subspace simulation framework powered by a neural latent-space integrator. Our approach leverages self-supervised learning to enhance inference stability and generalization. By operating entirely within latent space, our method eliminates the need for full-space computations, resulting in a highly efficient method well-suited for deployment on portable devices. We demonstrate the effectiveness of our approach on challenging examples involving rods, shells, and solids, showcasing its versatility and potential for widespread adoption.
- 8 August 2025
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Computer Graphics and Interactive Techniques
- Vol. 8 (4) , 1-18
Open AccessSelf-supervised Learning of Latent Space Dynamics 57
Modeling the dynamic behavior of deformable objects is crucial for creating realistic digital worlds. While conventional simulations produce high-quality motions, their computational costs are often prohibitive. Subspace simulation techniques address this challenge by restricting deformations to a lower-dimensional space, improving performance while maintaining visually compelling results. However, even subspace methods struggle to meet the stringent performance demands of portable devices such as virtual reality headsets and mobile platforms. To overcome this limitation, we introduce a novel subspace simulation framework powered by a neural latent-space integrator. Our approach leverages self-supervised learning to enhance inference stability and generalization. By operating entirely within latent space, our method eliminates the need for full-space computations, resulting in a highly efficient method well-suited for deployment on portable devices. We demonstrate the effectiveness of our approach on challenging examples involving rods, shells, and solids, showcasing its versatility and potential for widespread adoption.
- Open Access
- 8 August 2025
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Computer Graphics and Interactive Techniques
- Vol. 8 (4) , 1-17
Dress Anyone : Automatic Physically-Based Garment Pattern Refitting 56
Well-fitted clothing is essential for both real and virtual garments to enable self-expression and accurate representation for a large variety of body types. Common practice in the industry is to provide a pre-made selection of distinct garment sizes such as small, medium and large. While these may cater to certain groups of individuals that fall within this distribution, they often exclude large sections of the population. In contrast, individually tailored clothing offers a solution to obtain custom-fit garments that are tailored to each individual. However, manual tailoring is time-consuming and requires specialized knowledge, prohibiting the approach from being applied to produce fitted clothing at scale. To address this challenge, we propose a novel method leveraging differentiable simulation for refitting and draping 3D garments and their corresponding 2D pattern panels onto a new body shape. This enables a workflow where garments only need to be designed once, in a single size, and they can be automatically refitted to support numerous body size and shape variations. Our method enables downstream applications, where our optimized 3D drape can be directly ingested into game engines or other applications. Our 2D sewing patterns allow for accurate physics-based simulations and enables manufacturing clothing for the real world.
- 8 August 2025
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Computer Graphics and Interactive Techniques
- Vol. 8 (4) , 1-17
Open AccessDress Anyone : Automatic Physically-Based Garment Pattern Refitting 56
Well-fitted clothing is essential for both real and virtual garments to enable self-expression and accurate representation for a large variety of body types. Common practice in the industry is to provide a pre-made selection of distinct garment sizes such as small, medium and large. While these may cater to certain groups of individuals that fall within this distribution, they often exclude large sections of the population. In contrast, individually tailored clothing offers a solution to obtain custom-fit garments that are tailored to each individual. However, manual tailoring is time-consuming and requires specialized knowledge, prohibiting the approach from being applied to produce fitted clothing at scale. To address this challenge, we propose a novel method leveraging differentiable simulation for refitting and draping 3D garments and their corresponding 2D pattern panels onto a new body shape. This enables a workflow where garments only need to be designed once, in a single size, and they can be automatically refitted to support numerous body size and shape variations. Our method enables downstream applications, where our optimized 3D drape can be directly ingested into game engines or other applications. Our 2D sewing patterns allow for accurate physics-based simulations and enables manufacturing clothing for the real world.
- Open Access
- 21 November 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-12
Estimating Cloth Elasticity Parameters From Homogenized Yarn-Level Models
Virtual garment simulation has become increasingly important with applications in garment design and virtual try-on. However, reproducing garments faithfully remains a cumbersome process. We propose an end-to-end forward pipeline for estimating parameters of shell material models corresponding to real fabrics with minimal input. In contrast to prior work that relies on complex and often expensive capture systems, our method determines yarn model parameters from Young’s moduli determined during standard yarn stretch tests. We use an extended homogenization method to match yarn-level and shell-level hyperelastic energies with respect to a range of surface deformations represented by the first and second fundamental forms, including anisotropic bending. We optimize the parameters of a shell material model involving uncoupled bending and membrane energies. This allows the simulated shell model to exhibit deformation modes motivated by yarn-level physics in real fabrics. Finally, we validate our results with quantitative and visual comparisons against real world fabrics through stretch tests and drape experiments. Using the homogenized parameters, the shell models are capable of capturing the characteristics of underlying yarn patterns and exhibiting distinct behaviors for different yarn materials.
- 21 November 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-12
Open AccessEstimating Cloth Elasticity Parameters From Homogenized Yarn-Level Models
Virtual garment simulation has become increasingly important with applications in garment design and virtual try-on. However, reproducing garments faithfully remains a cumbersome process. We propose an end-to-end forward pipeline for estimating parameters of shell material models corresponding to real fabrics with minimal input. In contrast to prior work that relies on complex and often expensive capture systems, our method determines yarn model parameters from Young’s moduli determined during standard yarn stretch tests. We use an extended homogenization method to match yarn-level and shell-level hyperelastic energies with respect to a range of surface deformations represented by the first and second fundamental forms, including anisotropic bending. We optimize the parameters of a shell material model involving uncoupled bending and membrane energies. This allows the simulated shell model to exhibit deformation modes motivated by yarn-level physics in real fabrics. Finally, we validate our results with quantitative and visual comparisons against real world fabrics through stretch tests and drape experiments. Using the homogenized parameters, the shell models are capable of capturing the characteristics of underlying yarn patterns and exhibiting distinct behaviors for different yarn materials.
- Open Access
- 16 August 2023
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Computer Graphics and Interactive Techniques
- Vol. 6 (3) , 1-14
DiffXPBD
We present DiffXPBD, a novel and efficient analytical formulation for the differentiable position-based simulation of compliant constrained dynamics (XPBD). Our proposed method allows computation of gradients of numerous parameters with respect to a goal function simultaneously leveraging a performant simulation model. The method is efficient, thus enabling differentiable simulations of high resolution geometries and degrees of freedom (DoFs). Collisions are naturally included in the framework. Our differentiable model allows a user to easily add additional optimization variables. Every control variable gradient requires the computation of only a few partial derivatives which can be computed using automatic differentiation code. We demonstrate the efficacy of the method with examples such as elastic cloth and volumetric material parameter estimation, initial value optimization, optimizing for underlying body shape and pose by only observing the clothing, and optimizing a time-varying external force sequence to match sparse keyframe shapes at specific times. Our approach demonstrates excellent efficiency and we demonstrate this on high resolution meshes with optimizations involving over 26 million degrees of freedom. Making an existing solver differentiable requires only a few modifications and the model is compatible with both modern CPU and GPU multi-core hardware.
- 16 August 2023
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Computer Graphics and Interactive Techniques
- Vol. 6 (3) , 1-14
Open AccessDiffXPBD
We present DiffXPBD, a novel and efficient analytical formulation for the differentiable position-based simulation of compliant constrained dynamics (XPBD). Our proposed method allows computation of gradients of numerous parameters with respect to a goal function simultaneously leveraging a performant simulation model. The method is efficient, thus enabling differentiable simulations of high resolution geometries and degrees of freedom (DoFs). Collisions are naturally included in the framework. Our differentiable model allows a user to easily add additional optimization variables. Every control variable gradient requires the computation of only a few partial derivatives which can be computed using automatic differentiation code. We demonstrate the efficacy of the method with examples such as elastic cloth and volumetric material parameter estimation, initial value optimization, optimizing for underlying body shape and pose by only observing the clothing, and optimizing a time-varying external force sequence to match sparse keyframe shapes at specific times. Our approach demonstrates excellent efficiency and we demonstrate this on high resolution meshes with optimizations involving over 26 million degrees of freedom. Making an existing solver differentiable requires only a few modifications and the model is compatible with both modern CPU and GPU multi-core hardware.