Han Yan

Diffusion models have recently shown remarkable generative and modeling capabilities across vision and audio domains. Motivated by this progress, we propose Diff-IR, a novel generative image retrieval framework that reformulates retrieval as a non-autoregressive sequence generation task. Instead of the conventional two-stage pipeline—feature extraction followed by similarity search—Diff-IR directly encodes each query image into a compact discrete sequence whose code uniquely matches database images. This end-to-end design eliminates explicit similarity computation. In Diff-IR, high-level semantics are first extracted via a pre-trained CLIP encoder. The query features and a noise sequence are then processed by a Transformer-based denoising network, which gradually reconstructs the target discrete representation. Experiments on three public benchmarks demonstrate that treating retrieval as a generative task with diffusion not only improves accuracy but also offers efficient, fully end-to-end retrieval. Our results highlight the strong potential of diffusion models for next-generation retrieval systems.

This paper presents SketchFit, a new framework for user-driven fashion image generation that integrates multi-conditional inputs to guide the learning process. SketchFit combines a Multi-Conditional Adaptive Fusion Strategy (m-CAFs), which unifies diverse input features such as sketches, editable region masks, and incomplete images into a cohesive representation. This representation is further enhanced by a Channel Adaptive Modulator (CAMod) that enables precise feature extraction from grayscale sketches through global pooling and channel-wise attention, ensuring accurate shape completion. The fused features are then mapped into multiple dimensions via a Conditional Fusion mechanism based on Stable Diffusion, improving the ability of the model to handle multi-conditional inputs for personalized fashion design. The framework empowers intuitive, user-guided interactions, enabling high-quality, customizable fashion image generation. Extensive experiments validate the effectiveness of SketchFit in applications ranging from design to customization.

While image-based inverse tone mapping (iTM) has been extensively studied, research on video-based iTM remains limited. Leveraging image-based models for video iTM presents two key challenges: (1) incorporating global operations essential for HDR video production, and (2) modeling spatial-temporal information. To address these issues, we propose integrating a kernel prediction network (KPN) with multi-frame interactions (MFI) to model spatial-temporal context. Additionally, we introduce a global color mapping network (GCMN) alongside the KPN to simulate global operations, focusing on SDR pixels near the BT.709 color gamut boundaries. The MFI module refines spatial-temporal consistency by leveraging correlations across frames. Both GCMN and MFI can be seamlessly integrated into existing image-based iTM models to extend them to video iTM. Moreover, we introduce two losses for video iTM: an inter-frame brightness consistency loss based on the Gaussian pyramid, and a differential histogram loss to capture global color distribution. Extensive experiments demonstrate our approach outperforms state-of-the-art methods in both image and video-based iTM.

Today, fashion design can be readily performed by most people due to the rapid development of design tools. However, not everyone possesses the professional skills to produce an aesthetically pleasing design. In order to assist an inexperienced user during the design process, this research explored a new fashion-related disentanglement task, with the goal of creating novel fashion items with controllable attributes. The key idea is to develop a unified framework, called CTS-GAN, by disentangling the colors, textures, and shapes of fashion items simultaneously using a generative adversarial network (GAN). Specifically, we first introduced a fashion attribute encoder to decompose input fashion items into three latent spaces, i.e., color, texture, and shape. A fashion item pattern-making module (FIPM)-based generator was then proposed to control the corresponding parameters of color and texture in FIPMs independently and combine them with the shape features in order to accomplish the final generation of new fashion items. Furthermore, three independent pathways were introduced to extract the representations of color, texture, and shape in fashion items to optimize our CTS-GAN in an unsupervised manner. Extensive experimental results demonstrate the effectiveness of our CTS-GAN and suggest that it can generate diverse, novel fashion images by taking full advantage of the controllability of the colors, textures, and shapes of different fashion items

Texture and shape in fashion, constituting essential elements of garments, characterize the body and surface of the fabric and outline the silhouette of clothing, respectively. The selection of texture and shape plays a critical role in the design process, as they largely determine the success of a new design for fashion items. In this research, we propose a texture and shape disentangled generative adversarial network (TSD-GAN) to perform “intelligent” design with the transformation of texture and shape in fashion items. Our TSD-GAN aims to learn how to disentangle the features of texture and shape of different fashion items in an unsupervised manner. Specifically, a fashion attribute encoder is developed to decompose the input fashion items into independent representations of texture and shape. Then, to learn the coarse or fine styles hidden in the features of texture and shape, a texture mapping network and a shape mapping network are proposed to disentangle the features into different hierarchical representations. The different hierarchical representations of texture and shape are then fed into a multi-factor-based generator to generate mixed-style fashion items. In addition, a multi-discriminator framework is developed to distinguish the authenticity and texture similarity between the generated images and the real images. Experimental results on different fashion categories demonstrate that our proposed TSD-GAN may be useful for assisting designers to accomplish the design process by transforming the texture and shape of fashion items.

The traditional fashion industry is heavily dependent on designers whose talent and vision have a significant impact on their innovative designs. Through taking advantage of recent advances in image-to-image translation by generative adversarial networks (GANs), marked improvement in designers efficiency is now possible. Considering both randomness and controllability in the design process, this article presents a novel artificial intelligence (AI)-based framework for fashion design. Under this framework, a sketch-generation module which is based on latent space is firstly introduced for designing various sketches. Secondly, a rendering-generation module is proposed to learn mapping between textures and sketches to complete the task of fashion design. In order to achieve effectiveness in synthesizing semantic-aware textures on sketches, a multi-conditional feature interaction module is developed in the rendering-generation model. Moreover, two different training schemes are introduced to optimize both the sketch-generation module and the rendering-generation module. In order to evaluate the performance of our proposed models, we built a large-scale dataset which consists of 115,584 pairs of fashion item images. Experimental results demonstrate the effectiveness of our proposed method, and indicate that our model can facilitate designers design process by taking full advantage of the controllability of different conditions (e.g., sketch and texture) and the randomness of latent space.