Yifan Peng

Reconstructing realistic, physically plausible garments from a single image remains a fundamental challenge. Template-free methods capture surface geometry but lack explicit sewing structure for simulation; while programmatic systems are simulation-ready but constrained by predefined templates. This reveals a fundamental representation gap between geometric reconstruction and structured garment construction. We present PatternGSL, a structured garment representation in the form of a template-free and learnable specification language that encodes complete sewing patterns, including panel boundaries, parameterized seams, and explicit stitch topology, in a compact and standardized form. PatternGSL preserves the physical rigor of pattern-based models while removing template dependence, elevating sewing structure as a first-class target for generative modeling. We further propose a vision-language framework that predicts PatternGSL specifications directly from a single image and decodes them into garments using lightweight deterministic validity handling, without optimization-based refinement or manual cleanup. In addition, we introduce PatternGSLData, the first large-scale image-to-GSL paired dataset comprising 300K samples with complete sewing pattern annotations, enabling supervised VLM training for structured garment reconstruction. Experiments demonstrate improved pattern accuracy over prior baselines, explicit sewing-structure recovery, reliable cloth simulation, and pattern-level editing through the same deterministic decoding pipeline. Code and data-processing scripts will be released at https://github.com/PatternGSL/PatternGSL.

Holography offers unique advantages for delivering perceptual realism while preserving compact form factors in VR/AR. Its perceptual quality, however, hinges on encoding rich wavefronts of photorealistic scenes into interference patterns and then incoherently multiplexing the resulting wave fields for perception. Existing CGH paradigms decouple radiance estimation from wave propagation by pre-rendering radiance on discretized scene sectors. This separation between radiometric and wave-optical computation inherently limits the range of focus cues and visual effects that can be faithfully reproduced, including depth- and view-continuity, and physically based material behaviors such as glossy or mirror-like reflection and refraction. We present a physically accurate yet computationally efficient wave optics rendering framework leveraging path tracing to encode full 3D visual cues into phase holograms. Specifically, we employ a Monte Carlo method to solve both the rendering equation and the Rayleigh-Sommerfeld integral simultaneously. Our algorithm is fully compatible with modern graphics techniques and can generate multiple time-multiplexed random holograms with minimal additional time cost via Path Reuse. By employing a fast approximation with an ambient radiance cache, we realize an order of magnitude convergence speed improvement. The resulting coherent wave fields that inherently encode comprehensive visual effects are converted into phase-only holograms under complex-amplitude supervision. Through extensive simulations and experimental validations on a spatial light modulator-based display prototype, we demonstrate faithful holographic reconstructions of natural 3D cues and complex materials, including realistic defocus blur, view-dependent effects, as well as appearance highlights and reflections.

Holographic displays are a transforming technology for immersive virtual and augmented reality systems. Exploring accurate yet efficient computer-generated holography (CGH) algorithms for three-dimensional (3D) content is a valuable research field. Recent advancements in layer-based CGH may exhibit limited capacity to convey comprehensive 3D information in accurately representing tilted angular spectrum and realizing realistic defocus blur. Alternative approaches based on point clouds and light fields may demand significant computational resources for preparing adequate target data for optimization. In addition, most existing CGH algorithms rely on heuristics to encode complex amplitudes into phase-only holograms for display, which can be highly ill-posed. Here we investigate an innovative CGH framework that overcomes these challenges using a unique combination of mesh-based representation, tilt-angle tailored wave propagation modeling, and complex-valued optimization, alongside a learning-empowered display calibration scheme using camera feedback. The resulting expanded hologram encoding capabilities enable the delivery of natural 3D depth cues, including smooth defocus blur and view-dependent effects. Experimental results conducted on our holographic near-eye display prototype demonstrate unprecedented full 3D visual quality, representing a significant advancement in creating immersive visualization experiences.

Simulating event streams from 3D scenes has become a common practice in event-based vision research, as it meets the demand for large-scale, high temporal frequency data without setting up expensive hardware devices or undertaking extensive data collections. Yet existing methods in this direction typically work with noiseless RGB frames that are costly to render, and therefore their simulations are often unrealistically low in temporal resolution. In this work, we propose EventTracer,, a path tracing-based rendering pipeline that simulates high-fidelity event sequences from complex 3D scenes in an efficient and physics-aware manner. Specifically, we speed up the rendering process via low sample-per-pixel (SPP) path tracing, and train a lightweight event spiking network to denoise the resulting RGB videos into realistic event sequences. Our EventTracer pipeline runs at a speed of $\sim$1 minutes per second of 360p video, and it inherits the merit of accurate spatiotemporal modeling from its path tracing backbone. We show through the Real2Sim and Sim2Real tests that EventTracer captures higher-fidelity scene details and demonstrates a greater similarity to real-world event data than alternative event simulators, which establishes it as a potential tool for creating large-scale event-RGB datasets, narrowing the sim-to-real gap in event-based vision, and boosting various downstream applications. Our code and dataset will be released upon publication.

Wave propagation modeling is fundamental to optics, facilitating diverse applications including lens design, computational imaging, and optical computing. However, existing approaches encounter a critical trade-off between computational accuracy and efficiency, primarily limited by numerical precision constraints. In this study, we establish the explicit numerical limits in wave propagation modeling and demonstrate that accurate simulations are achievable with reduced precision. Our analysis reveals that the limited significant decimal digits and dynamic range of floating-point arithmetic compromise phase and amplitude representation accuracy, thereby constraining the dimensions and space-bandwidth product of optical systems that can be reliably modeled. To address this challenge, we introduce a differentiable modeling scheme that maintains accurate phase representation through wrapping from double precision and appropriately prescales the amplitude to ensure the integral result remains within the representable dynamic range. We validate our approach by simulating point spread functions of optical systems, solving phase retrieval problems, and synthesizing holograms for light shaping. Our method achieves, on average, ∼20× acceleration in diffraction modeling while retaining accuracy comparable to double-precision implementations. Optical experiments further demonstrate our approach’s effectiveness. Specifically, our method successfully reconstructs complex amplitudes of laser beams from coded measurements and designs phase-only holograms for desired diffraction patterns. We envision that this technique will advance research in computational optics with enhanced computational efficiency.

Holographic displays offer the promise of realistic 3D visualization for virtual and augmented wearable solutions. Nevertheless, existing computer-generated holography (CGH) methods often struggle with either a high computational burden or limited display realism. While the emerging cloud-edge computing mechanism can enable the real-time streaming of holograms, classic image compression techniques struggle to efficiently encode and decode the substantial high-frequency information inherent in hologram data. In light of these challenges, we present a display-aware and lightweight CGH framework, leveraging implicit neural representations (INRs) and camera-calibrated wave propagation, to generate and compress high-fidelity phase-only holograms. Specifically, our approach interprets hologram generation as a continuous function approximation problem, enabling the network, with reduced parameters, to effectively learn the inherent periodicity and high-frequency components of 2D and 3D hologram data. To enable efficient deployment, we further incorporate quantization-aware training, followed by entropy coding. Experimental results evaluated on an unfiltered holographic display prototype demonstrate that the proposed INR-CGH retains image quality comparable to that of existing optimization-based methods in both 2D and 3D scenarios. In addition, our compact INR representation achieves up to 11× compression rate with minimal quality degradation and can be further reduced via quantization-aware training. The resulting model enables ≥250 fps in decoding speed, paving the way towards edge holography.

Deep neural networks have demonstrated exceptional performance across various domains. However, their black‐box nature poses significant challenges to understanding their decision‐making processes. As these models increasingly underpin critical decisions in regulated sectors, explainable artificial intelligence (XAI) has emerged as a promising avenue for elucidating model behaviour. Despite advancements, this field lacks standardised protocols for rigorously evaluating the correctness of explanation methods, particularly at a granular pixel level . This article introduces Pixel Lens, a novel protocol designed to assess the correctness of saliency‐based XAI methods at the pixel level. In contrast to approaches that rely on human‐centric semantically meaningful object assessments, Pixel Lens focuses on the actual decision rules employed by classifiers. The protocol systematically embeds spatially localised shortcuts into images, which are artificial patterns that networks preferentially adopt as decision rules, while preserving the original image features. After verifying that the shortcut is the dominant rule, Shapley values are estimated for pixels to produce ground‐truth explanation maps. Through extensive evaluations of saliency‐based XAI methods across multiple architectures and datasets, our experiments reveal that existing methods often fail to provide accurate pixel‐level attributions. These findings challenge recent progress claims in XAI and establish a more robust foundation for evaluating explanation techniques.