RLRynson W. H. Lau

Rynson W. H. Lau

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Rynson W. H. Lau has a total of 348 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Glass Surface Detection Grounded in 3D Visual Geometry

    by Yiwei Lu, Ke Xu, Tao Yan, Xiaojun Chang, Radu Timofte, Rynson W. H. Lau

  • Open Access

    Steering Diffusion Models via Color-Activating Noise Patches for Controllable Image Synthesis

    by Rongjin Guo, Ke Xu, Rynson Lau

    While diffusion models have achieved remarkable progress in text-to-image (T2I) synthesis, a critical question remains: how can we achieve precise and reliable control over object colors and positions in the generated images? In this work, we first aim to answer this question by uncovering and leveraging a novel structure in the initial random Gaussian noise, which we term Color-Activating Noise (CAN) patches. These CAN patches are localized regions of the initial noise that consistently and intrinsically couple with specific colors, i.e. , they can be reliably activated by the color-related tokens in the input prompt during the denoising (generation) process. This activation, in turn, steers the position of the associated object. Based on this observation, we then propose a mining pipeline to discover CAN patches efficiently from a pre-trained UNet-based diffusion model and organize them into a plug-and-play database for retrieval and injection at inference time. We demonstrate that CAN patches successfully align the capability of UNet-based diffusion models with that of the heavily-trained Diffusion Transformers (DiTs) in reliable color-position control, particularly for atypical and multi-color generation cases, without additional optimization. We further show that CAN patches can generalize to DiT-based models, improving the generation accuracy via better object position and color control. Finally, we showcase two applications, interactive image editing and counting-aware image generation, to highlight the versatility of our CAN patches. Our method can serve as a new baseline to shed light on improving the controllability and faithfulness of diffusion-based T2I generation. Project page: https://github.com/RongjinGuo/CAN-Patches.

  • GuideMe: Multi-domain Task Guidance and Intervention in Streaming Video

    by Fang Liu, Jinpeng Chen, Ke Xu, Yuhao Liu, Huankang Guan, Xudong Lu, Yang Bo, Gerhard Hancke, Rui Liu, Rynson W. H. Lau

  • LiveMatte: Dynamic Scene Background Restoration and Selective Portrait Patch Enhancement

    by Jiayu Sun, Zhanghan Ke, Lihe Zhang, Huchuan Lu, Rynson W.H. Lau

    Real-time and accurate portrait matting in videos is a challenging problem in computer vision research. Recent approaches have explored incorporating prior conditions for accurate inference. Notably, some methods ask the user to provide the background image, which requires extra effort from the user to capture the background image and is limited to videos with static backgrounds only. We note that real-time video motion segmentation methods often train a background model to detect the foreground. Our insight of this work is that if we can directly restore the background content from the input video as a prior, we may be able to achieve more precise portrait matting. In addition, this approach could potentially work even with dynamic backgrounds, without requiring additional user input. However, automatically restoring the background content is not straightforward due to the difficulty in distinguishing between foreground and background. While it may seem that stationary pixel values represent the background, these values can vary across frames. To this end, we propose a novel dynamic scene background restoration (DSBR) module that learns a background model by accumulating background content from each input video frame. It restores the current background content, which serves as a matting prior for alpha prediction of the subsequent frame. DSBR is extremely lightweight and can be easily integrated into existing matting models. Based on it, we present a real-time portrait video matting framework, LiveMatte. To more efficiently process high-resolution videos, we also introduce a selective portrait patch enhancement (SPPE) module. Extensive experiments and user studies demonstrate that our method is better and faster than existing methods.


  • Open Access

    MDeRainNet: An Efficient Macro-pixel Image Rain Removal Network

    by Tao Yan, Weilong Huang, Weijiang He, Chenglong Wang, Cihang Wei, Yiwei Lu, Xiangjie Zhu, Yinghui Wang, Rynson W. H. Lau

    Since raining weather always degrades image quality and poses significant challenges to most computer vision-based intelligent systems, image de-raining has been a hot research topic in computer vision community. Fortunately, in a rainy Light Field (LF) image, background obscured by rain streaks in one sub-view may be visible in the other sub-views, and implicit depth information and recorded 4D structural information may benefit rain streak detection and removal. However, existing LF image rain removal methods either do not fully exploit the global correlations of 4D LF data or only utilize partial sub-views (i.e., under-utilization of the rich angular information), resulting in sub-optimal rain removal performance and no-equally good quality for all de-rained sub-views. In this article, we propose an efficient neural network, called MDeRainNet , for rain streak removal from LF images. The proposed network adopts a multi-scale encoder–decoder architecture, which directly works on Macro-pixel Images (MPIs) for improving the rain removal performance. To fully model the global correlation between the spatial information and the angular information, we propose an Extended Spatial-angular Interaction (ESAI) module to merge the two types of information, in which a simple and effective Transformer-based Spatial-angular Interaction Attention (SAIA) block is also proposed for modeling long-range geometric correlations and making full use of the angular information. Furthermore, to improve the generalization performance of our network on real-world rainy scenes, we propose a novel semi-supervised learning framework for our MDeRainNet , which utilizes multi-level KL loss to bridge the domain gap between features of synthetic and that of real-world rain streaks and introduces colored-residue image-guided contrastive regularization to reconstruct rain-free images. Extensive experiments conducted on both synthetic and real-world Light Field Images (LFIs) demonstrate that our method outperforms the state-of-the-art methods both quantitatively and qualitatively.


  • Open Access

    World-Shaper: A Unified Framework for 360° Panoramic Editing

    by Dong Liang, Yuhao Liu, Jinyuan Jia, Youjun Zhao, Rynson W. H. Lau

    Being able to edit panoramic images is crucial for creating realistic 360° visual experiences. However, existing perspective-based image editing methods fail to model the spatial structure of panoramas. Conventional cube-map decompositions attempt to overcome this problem but inevitably break global consistency due to their mismatch with spherical geometry. Motivated by this insight, we reformulate panoramic editing directly in the equirectangular projection (ERP) domain and present World-Shaper, a unified geometry-aware framework that bridges panoramic generation and editing within a single editing-centric design. To overcome the scarcity of paired data, we adopt a generate-then-edit paradigm, where controllable panoramic generation serves as an auxiliary stage to synthesize diverse paired examples for supervised editing learning. To address geometric distortion, we introduce a geometry-aware learning strategy that explicitly enforces position-aware shape supervision and implicitly internalizes panoramic priors through progressive training. Extensive experiments on our new benchmark, PEBench, demonstrate that our method achieves superior geometric consistency, editing fidelity, and text controllability compared to SOTA methods, enabling coherent and flexible 360° visual world creation with unified editing control. Code, model, and data will be released at our project page: https://world-shaper-project.github.io/

  • Open Access

    StyleSculptor: Zero-Shot Style-Controllable 3D Asset Generation with Texture-Geometry Dual Guidance

    by Zefan Qu, Zhenwei Wang, Haoyuan Wang, Ke Xu, Gerhard Petrus Hancke, Rynson. W. H. Lau


  • Open Access

    Shape-for-Motion: Precise and Consistent Video Editing With 3D Proxy

    by Yuhao Liu, Tengfei Wang, Fang Liu, Zhenwei Wang, Rynson W.H. Lau


  • Top Cited Papers
    Open Access

    Voyager: Long-Range and World-Consistent Video Diffusion for Explorable 3D Scene Generation

    by Tianyu Huang, Wangguandong Zheng, Tengfei Wang, Yuhao Liu, Zhenwei Wang, Junta Wu, Jie Jiang, Hui Li, Rynson Lau, Wangmeng Zuo, et al.

    Real-world applications like video gaming and virtual reality often demand the ability to model 3D scenes that users can explore along custom camera trajectories. While significant progress has been made in generating 3D objects from text or images, creating long-range, 3D-consistent, explorable 3D scenes remains a complex and challenging problem. In this work, we present Voyager , a novel video diffusion framework that generates world-consistent 3D point-cloud sequences from a single image with user-defined camera path. Unlike existing approaches, Voyager achieves end-to-end scene generation and reconstruction with inherent consistency across frames, eliminating the need for 3D reconstruction pipelines (e.g., structure-from-motion or multi-view stereo). Our method integrates three key components: 1) World-Consistent Video Diffusion : A unified architecture that jointly generates aligned RGB and depth video sequences, conditioned on existing world observation to ensure global coherence 2) Long-Range World Exploration : An efficient world cache with point culling and an auto-regressive inference with smooth video sampling for iterative scene extension with context-aware consistency, and 3) Scalable Data Engine : A video reconstruction pipeline that automates camera pose estimation and metric depth prediction for arbitrary videos, enabling large-scale, diverse training data curation without manual 3D annotations. Collectively, these designs result in a clear improvement over existing methods in visual quality and geometric accuracy, with versatile applications. Code for this paper are at https://github.com/Tencent-Hunyuan/HunyuanWorld-Voyager.


  • Open Access

    Unleashing the Potential of Multimodal LLMs for Zero-Shot Spatio-Temporal Video Grounding

    by Zaiquan Yang, Yuhao Liu, Gerhard Hancke, Rynson W. H. Lau

    Spatio-temporal video grounding (STVG) aims at localizing the spatio-temporal tube of a video, as specified by the input text query. In this paper, we utilize multimodal large language models (MLLMs) to explore a zero-shot solution in STVG. We reveal two key insights about MLLMs: (1) MLLMs tend to dynamically assign special tokens, referred to as \textit{grounding tokens}, for grounding the text query; and (2) MLLMs often suffer from suboptimal grounding due to the inability to fully integrate the cues in the text query (\textit{e.g.}, attributes, actions) for inference. Based on these insights, we propose a MLLM-based zero-shot framework for STVG, which includes novel decomposed spatio-temporal highlighting (DSTH) and temporal-augmented assembling (TAS) strategies to unleash the reasoning ability of MLLMs. The DSTH strategy first decouples the original query into attribute and action sub-queries for inquiring the existence of the target both spatially and temporally. It then uses a novel logit-guided re-attention (LRA) module to learn latent variables as spatial and temporal prompts, by regularizing token predictions for each sub-query. These prompts highlight attribute and action cues, respectively, directing the model's attention to reliable spatial and temporal related visual regions. In addition, as the spatial grounding by the attribute sub-query should be temporally consistent, we introduce the TAS strategy to assemble the predictions using the original video frames and the temporal-augmented frames as inputs to help improve temporal consistency. We evaluate our method on various MLLMs, and show that it outperforms SOTA methods on three common STVG benchmarks. The code will be available at https://github.com/zaiquanyang/LLaVA_Next_STVG.