HYHui Ye

Hui Ye

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Hui Ye has a total of 23 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Top Cited Papers

    MoGraphGPT : Creating Interactive Scenes Using Modular LLM and Graphical Control

    by Hui Ye, Chufeng Xiao, Jiaye Leng, Pengfei Xu, Hongbo Fu

    Creating interactive scenes often involves complex programming tasks. Although large language models (LLMs) like ChatGPT can generate code from natural language, their output is often error-prone, particularly when scripting interactions among multiple elements. The linear conversational structure limits the editing of individual elements, and the lack of graphical and precise control complicates visual integration. To address these issues, we integrate a context-aware modularization technique that processes textual descriptions for individual elements through separate LLM modules, with a central module managing interactions among elements. It defines a top-down structure to manage interactions, ensuring clear update logic and facilitating efficient collaboration while allowing for independent updates for each element. We design a graphical user interface, MoGraphGPT, which combines modular LLMs with enhanced graphical control to generate codes for 2D interactive scenes. It enables direct integration of graphical information and offers quick, precise control through automatically generated sliders. A comparative study with Cursor Composer shows MoGraphGPT significantly improves easiness, controllability, and performance in creating 2D interactive scenes with multiple visual elements in a coding-free manner. An ablation study validates the effectiveness of modularization, and an open-ended study demonstrates the usability and expressiveness of MoGraphGPT.


  • Open Access

    Aesthetic3D: Incorporating Shape Aesthetic Measures into 3D Modeling Interfaces

    by Deng Yu, Jianing Guo, Hui Ye, Pengjie Ren, Hongbo Fu, Manfred Lau

    3D Aesthetics is significant in digital design, shaping how users experience real-time 3D content in games, VR, and product design. However, creating aesthetically pleasing shapes remains challenging due to diverse subjective standards and the lack of tools that support aesthetics-driven editing. Users often rely on intuition without explicit guidance on visual appeal, making aesthetics refinement slow, inconsistent, and cognitively demanding, particularly in fast-paced, iterative workflows. To address this challenge, we conducted in-depth interviews with design experts to identify challenges in aesthetics-oriented modeling workflows. Based on the findings, we developed Aesthetic3D , a 3D modeling interface that provides real-time aesthetics scores learned from human perceptual data. Furthermore, Aesthetic3D seamlessly integrates the learned aesthetics measures into intuitive editing operations, enabling aesthetics-driven exploration and refinement of shape geometry. We evaluated Aesthetic3D through an ablation study, an open-ended study, and three generalization evaluations. Comprehensive experiments show that with Aesthetic3D , users can easily and effectively enhance the aesthetics appeal of 3D shapes.

  • GenFODrawing: Supporting Creative Found Object Drawing With Generative AI

    by Jiaye Leng, Hui Ye, Pengfei Xu, Miu-Ling Lam, Hongbo Fu

    Found object drawing is a creative art form incorporating everyday objects into imaginative images, offering a refreshing and unique way to express ideas. However, for many people, creating this type of work can be challenging due to difficulties in generating creative ideas and finding suitable reference images to help translate their ideas onto paper. Based on the findings of a formative study, we propose GenFODrawing, a creativity support tool to help users create diverse found object drawings. Our system provides AI-driven textual and visual inspirations, and enhances controllability through sketch-based and box-conditioned image generation, enabling users to create personalized outputs. We conducted a user study with twelve participants to compare GenFODrawing to a baseline condition where the participants completed the creative tasks using their own desired approaches without access to our system. The study demonstrated that GenFODrawing enabled easier exploration of diverse ideas, greater agency and control through the creative process, and higher creativity support compared to the baseline. A further open-ended study demonstrated the system's usability and expressiveness, and all participants found the creative process engaging.


  • Sketch2Human: Deep Human Generation With Disentangled Geometry and Appearance Constraints

    by Linzi Qu, Jiaxiang Shang, Hui Ye, Xiaoguang Han, Hongbo Fu

    Geometry- and appearance-controlled full-body human image generation is an interesting but challenging task. Existing solutions are either unconditional or dependent on coarse conditions (e.g., pose, text), thus lacking explicit geometry and appearance control of body and garment. Sketching offers such editing ability and has been adopted in various sketch-based face generation and editing solutions. However, directly adapting sketch-based face generation to full-body generation often fails to produce high-fidelity and diverse results due to the high complexity and diversity in the pose, body shape, and garment shape and texture. Recent geometrically controllable diffusion-based methods mainly rely on prompts to generate appearance. It is hard to balance the realism and the faithfulness of their results to the sketch when the input is coarse. This work presents Sketch2Human, the first system for controllable full-body human image generation guided by a semantic sketch (for geometry control) and a reference image (for appearance control). Our solution is based on the latent space of StyleGAN-Human with inverted geometry and appearance latent codes as input. Specifically, we present a sketch encoder trained with a large synthetic dataset sampled from StyleGAN-Human's latent space and directly supervised by sketches rather than real images. Considering the entangled information of partial geometry and texture in StyleGAN-Human and the absence of disentangled datasets, we design a novel training scheme that creates geometry-preserved and appearance-transferred training data to tune a generator to achieve disentangled geometry and appearance control. Although our method is trained with synthetic data, it can also handle hand-drawn sketches. Qualitative and quantitative evaluations demonstrate the superior performance of our method to state-of-the-art methods.


  • Open Access

    ProInterAR: A Visual Programming Platform for Creating Immersive AR Interactions

    by Hui Ye, Jiaye Leng, Pengfei Xu, Karan Singh, Hongbo Fu

    AR applications commonly contain diverse interactions among different AR contents. Creating such applications requires creators to have advanced programming skills for scripting interactive behaviors of AR contents, repeated transferring and adjustment of virtual contents from virtual to physical scenes, testing by traversing between desktop interfaces and target AR scenes, and digitalizing AR contents. Existing immersive tools for prototyping/authoring such interactions are tailored for domain-specific applications. To support programming general interactive behaviors of real object(s)/environment(s) and virtual object(s)/environment(s) for novice AR creators, we propose ProInterAR, an integrated visual programming platform to create immersive AR applications with a tablet and an AR-HMD. Users can construct interaction scenes by creating virtual contents and augmenting real contents from the view of an AR-HMD, script interactive behaviors by stacking blocks from a tablet UI, and then execute and control the interactions in the AR scene. We showcase a wide range of AR application scenarios enabled by ProInterAR, including AR game, AR teaching, sequential animation, AR information visualization, etc. Two usability studies validate that novice AR creators can easily program various desired AR applications using ProInterAR.


  • Open Access

    ProObjAR: Prototyping Spatially-aware Interactions of Smart Objects with AR-HMD

    by Hui Ye, Jiaye Leng, Chufeng Xiao, Lili Wang, Hongbo Fu

    The rapid advances in technologies have brought new interaction paradigms of smart objects (e.g., digital devices) beyond digital device screens. By utilizing spatial properties, configurations, and movements of smart objects, designing spatial interaction, which is one of the emerging interaction paradigms, efficiently promotes engagement with digital content and physical facility. However, as an important phase of design, prototyping such interactions still remains challenging, since there is no ad-hoc approach for this emerging paradigm. Designers usually rely on methods that require fixed hardware setup and advanced coding skills to script and validate early-stage concepts. These requirements restrict the design process to a limited group of users in indoor scenes. To facilitate the prototyping to general usages, we aim to figure out the design difficulties and underlying needs of current design processes for spatially-aware object interactions by empirical studies. Besides, we explore the design space of the spatial interaction for smart objects and discuss the design space in an input-output spatial interaction model. Based on these findings, we present ProObjAR, an all-in-one novel prototyping system with an Augmented Reality Head Mounted Display (AR-HMD). Our system allows designers to easily obtain the spatial data of smart objects being prototyped, specify spatially-aware interactive behaviors from an input-output event triggering workflow, and test the prototyping results in situ. From the user study, we find that ProObjAR simplifies the design procedure and increases design efficiency to a large extent and thus advancing the development of spatially-aware applications in smart ecosystems.


  • DrawingInStyles: Portrait Image Generation and Editing With Spatially Conditioned StyleGAN

    by Wanchao Su, Hui Ye, Shu-Yu Chen, Lin Gao, Hongbo Fu

    The research topic of sketch-to-portrait generation has witnessed a boost of progress with deep learning techniques. The recently proposed StyleGAN architectures achieve state-of-the-art generation ability but the original StyleGAN is not friendly for sketch-based creation due to its unconditional generation nature. To address this issue, we propose a direct conditioning strategy to better preserve the spatial information under the StyleGAN framework. Specifically, we introduce Spatially Conditioned StyleGAN (SC-StyleGAN for short), which explicitly injects spatial constraints to the original StyleGAN generation process. We explore two input modalities, sketches and semantic maps, which together allow users to express desired generation results more precisely and easily. Based on SC-StyleGAN, we present DrawingInStyles, a novel drawing interface for non-professional users to easily produce high-quality, photo-realistic face images with precise control, either from scratch or editing existing ones. Qualitative and quantitative evaluations show the superior generation ability of our method to existing and alternative solutions. The usability and expressiveness of our system are confirmed by a user study.


  • 3D Curve Creation on and Around Physical Objects With Mobile AR

    by Hui Ye, Kin Chung Kwan, Hongbo Fu

    The recent advance in motion tracking (e.g., Visual Inertial Odometry) allows the use of a mobile phone as a 3D pen, thus significantly benefiting various mobile Augmented Reality (AR) applications based on 3D curve creation. However, when creating 3D curves on and around physical objects with mobile AR, tracking might be less robust or even lost due to camera occlusion or textureless scenes. This motivates us to study how to achieve natural interaction with minimum tracking errors during close interaction between a mobile phone and physical objects. To this end, we contribute an elicitation study on input point and phone grip, and a quantitative study on tracking errors. Based on the results, we present a system for direct 3D drawing with an AR-enabled mobile phone as a 3D pen, and interactive correction of 3D curves with tracking errors in mobile AR. We demonstrate the usefulness and effectiveness of our system for two applications: in-situ 3D drawing, and direct 3D measurement.


  • Motion vector-based video steganalysis using spatial-temporal correlation

    by Hui Ye, Weiming Zhang, Yuanzhi Yao, Cong Kong, Hao Huang, Nenghai Yu

    In current media-centric society, video has become the suitable cover for steganography. However, video steganalysis remains largely unexplored compared to the mature image steganalysis. One open problem is how to design efficient rich features for video steganalysis by exploiting the correlations in video. In this paper, we propose a novel video steganalytic scheme based on the spatial-temporal correlation of motion vectors. The proposed scheme employs 324-dimension features from Markov matrix of motion vectors in each sliding window constituting of eight inter-coded frames without overlapping. Experimental results show that our proposed scheme performs better in detecting existing motion vector-based steganographic methods than previous related steganalysis schemes.