FLFan Lyu

Fan Lyu

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Fan Lyu has a total of 107 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • SIFS: From Segment Anything to Classify Anything with Few Shots

    by Wenyou Yi, Fan Lyu, Zhang Zhang, Fuyuan Hu, Liang Wang

  • A negative-anchored self-relabeling strategy for multi-label class-incremental learning

    by Kaile Du, Junzhou Xie, Fan Lyu, Yifan Zhou, Zihan Ye, Guangcan Liu

  • Elastic Multi-Gradient Descent for Parallel Continual Learning

    by Fan Lyu, Wei Feng, Yuepan Li, Qing Sun, Fanhua Shang, Liang Wan, Liang Wang

    Learning a single shared model across multiple tasks is a long-standing goal in machine learning, traditionally addressed through Multi-Task Learning (MTL) or Serial Continual Learning (SCL). However, MTL assumes fixed task availability, limiting adaptability to new tasks, while SCL suffers from delayed response due to its strictly sequential learning protocol. This paper studies the novel paradigm of Parallel Continual Learning (PCL) in dynamic multi-task scenarios, where a diverse set of tasks is encountered at different time points. PCL inherits the challenges of both MTL and SCL, namely, task conflict and catastrophic forgetting, compounded by the dynamic and asynchronous arrival of tasks. To address these challenges, we propose Elastic Multi-Gradient Descent (EMGD), which formulates PCL as a dynamic multi-objective optimization problem. EMGD introduces task-specific elastic factors to guide gradient updates toward Pareto-optimal directions, ensuring balanced learning across tasks. Additionally, we develop a gradient-guided memory editing mechanism that aligns rehearsal data with the optimized descent direction, mitigating memory-induced interference. Theoretical analysis shows that accumulation points of the EMGD update rule are Pareto critical under the proposed formulation, and extensive experiments on image classification benchmarks demonstrate that EMGD significantly outperforms existing methods, including state-of-the-art PCL, MTL, and SCL approaches.

  • MultiHuman: Leverage Multimodal Prompts for Controllable Multi-Person Image Synthesizing

    by Liuqing Zhao, Baicheng Chen, Jun He, Fan Lyu, Richang Hong

    With recent advancements in generative models, controllable human image generation has achieved significant progress. However, existing methods still struggle in complex scenes involving multiple people and intricate poses, often leading to pose ambiguity and body part mismatches. Consequently, the generated images tend to lack structural coherence and visual realism. To address these challenges, we propose MultiHuman, a novel framework for multi-person human image generation that goes beyond simple pose and interaction modeling. Our approach utilizes coarse human poses and dense depth maps as layout guidance, explicitly tackling the issues of modality alignment and feature fusion in multi-human generation. To this end, we introduce a Modality Alignment Module (MAM) that projects pose and depth representations into a shared embedding space. Spatial alignment is achieved through contrastive learning, which ensures consistent and complementary layout representations across multiple people. In addition, a Pose Modulation Module (PMM) is designed to convert hard joint masks into differentiable soft masks, which improves the precision of pose control. To support training and evaluation under challenging conditions, we construct a new dataset, MTHuman, comprising 110K images that feature multi-person occlusions and complex human poses. Experiments on LAION-Human, HumanArt, and MTHuman show that MultiHuman outperforms existing methods like StablePose, with a 2.36% improvement in controllable human image generation accuracy. MultiHuman shows better image quality and stronger structure control.

  • GAIN: Global-Atomic INteraction Graph for Few-Shot Class-Incremental Learning

    by Fan Lyu, Linglan Zhao, Chengyan Liu, Yinying Mei, Zhang Zhang, Baoqing Yu, Tao Zhang, Fuyuan Hu, Liang Wang

    Few-shot class-incremental learning (FSCIL) aims to develop models that can continually learn novel classes from only a few training samples while retaining knowledge of previously learned classes. The primary challenges of FSCIL lie in the limited available data in novel sessions, which can cause severe overfitting and exacerbate the catastrophic forgetting of novel classes. Prior studies have demonstrated the effectiveness of modeling graph relationships in few-shot learning, which can alleviate overfitting through feature aggregation. Building on these insights, we extend the concept to the incremental learning scenario and propose a Global-Atomic INteraction (GAIN) graph learning framework for FSCIL. Specifically, we introduce an Atomic-session Graph Network (AGN) to capture relationships among samples within each session, enabling the aggregation of similar samples and the extraction of more representative global-session features. Additionally, we design a Global-session Graph Network (GGN) that connects global-session features across base and novel classes, facilitating knowledge transfer between sessions and enhancing overall learning performance. Finally, we utilize a multi-stage strategy for training the GAIN model, which suppresses the training challenges posed by limited data in incremental learning sessions. The multi-stage training strategy is designed to build a GAIN graph from base to novel sessions and improve the capacity via an extra pseudo-incremental stage. Extensive experiments on three widely used FSCIL benchmarks demonstrate that our approach consistently outperforms existing methods and establishes new state-of-the-art performance. Our code will be released upon acceptance.

  • Constructing Enhanced Mutual Information for Online Class-Incremental Learning

    by Huan Zhang, Fan Lyu, Shenghua Fan, Yujin Zheng, Dingwen Wang

    Online Class-Incremental Continual Learning (OCIL) addresses the challenge of continuously learning from a single-channel data stream, adapting to new tasks while mitigating catastrophic forgetting. Recently, Mutual Information (MI)-based methods have shown promising performance in OCIL. However, existing MI-based methods treat various knowledge components in isolation, ignoring the knowledge confusion across tasks. This narrow focus on simple MI knowledge alignment may lead to old tasks being easily forgotten with the introduction of new tasks, risking the loss of common parts between past and present knowledge. To address this, we analyze the MI relationships from the perspectives of diversity, representativeness, and separability, and propose an Enhanced Mutual Information (EMI) method based on knowledge decoupling. EMI consists of Diversity Mutual Information (DMI), Representativeness Mutual Information (RMI) and Separability Mutual Information (SMI). DMI diversifies intra-class sample features by considering the similarity relationships among inter-class sample features to enable the network to learn more general knowledge. RMI summarizes representative features for each category and aligns sample features with these representative features, making the intra-class sample distribution more compact. SMI establishes MI relationships for inter-class representative features, enhancing the stability of representative features while increasing the distinction between inter-class representative features, thus creating clear boundaries between class. Extensive experimental results on widely used benchmark datasets demonstrate the superior performance of EMI over state-of-the-art baseline methods. Our code is available at https://github.com/zhwhu/emi.


  • Reconstructive Visual Tuning for Weakly Supervised Video Anomaly Detection

    by Shuangqing Zhang, Wei Xu, Yuqi Fang, Fan Lyu, Leilei Ma, Gangming Zhao, Fang Zhao, Caifeng Shan, Liang Wang

  • Mitigating Catastrophic Forgetting in Online Continual Learning With Dual-Margin Contrastive Replay

    by Fan Lyu, Gongbo Cheng, Daofeng Liu, Linglan Zhao, Zhang Zhang, Fuyuan Hu, Liang Wang

    Online Continual Learning (OCL) enables machine learning models to learn from a stream of non-stationary tasks, making it more aligned with real-world scenarios. However, OCL faces a significant challenge: catastrophic forgetting, wherein the model learned in previous tasks is substantially overwritten upon encountering new tasks, leading to a biased forgetting of prior knowledge. Among various OCL strategies, replay-based methods have proven particularly effective in mitigating catastrophic forgetting by maintaining a small buffer of past samples and retraining them alongside new data. However, due to strict memory constraints, these replay buffers often fail to adequately represent the true data distribution of previous tasks. This leads to distributional shifts in the feature space, amplifying forgetting and degrading model performance. To address the problem, in this paper, we propose a novel replay strategy, termed Dual-Margin Contrastive Replay (DMCR), to anchor the distribution of old tasks and reduce the negative transfer effects. First, we propose to select memory for more representative samples guided by constructed centroids in a data stream. Then, to keep the model from distribution chaos in biased replay, a two-level angular cross-task Contrastive Margin Loss (CML) is proposed, to encourage the intra-class and intra-task compactness, and increase the inter-class and inter-task discrepancy. Finally, to further suppress the distributional drift, we present an optional Centroid Distillation Loss (CDL) on the replay memory to anchor the knowledge in feature space for each previous old task. Extensive experimental results on five benchmark datasets validate that the proposed DMCR can effectively mitigate the catastrophic forgetting and achieve state-of-the-art (SOTA) performance in OCL.


  • MambaPTP: Exploring the Potential of Mamba for Pedestrian Trajectory Prediction

    by Shuangqing Zhang, Gangming Zhao, Fan Lyu, Songping Wang, Zhang Zhang, Fang Zhao, Jinpeng Li, Caifeng Shan, Liang Wang

    Pedestrian Trajectory Prediction (PTP) aims to predict the future trajectory of pedestrians based on a historical trajectory. Transformer-based approaches have demonstrated unparalleled performance for PTP tasks, encoding long-term temporal dependencies and heterogeneous spatial interactions of pedestrians. However, Transformer often involves redundant information and noisy interactions from irrelevant regions by considering all available trajectory features. Recently, the structured state space model, Mamba has been proposed, which captures long-range dependency in sequences with a selective mechanism to filter out redundant information. To further tap into the potential of the novel Mamba architecture for the PTP task, in this paper, we present MambaPTP, which predicts future trajectories based purely on Mamba mechanisms, to mitigate the noisy interactions of irrelevant trajectory features and avoid repetitive trajectory modeling, while maintaining high-performance trajectory prediction. Specifically, we propose a new Bidirectional Gating Mamba (BGM) module with bidirectional state space models, which leverages the sparse gate mechanism to select informative temporal patterns and spatial interactions. Moreover, we design a Bidirectional Trajectory Alignment (BTA) module towards aligning the predicted trajectory to the ground truth, ensuring that the model to learn the effective sparse feature representation of trajectories. We conduct extensive experiments on several mainstream pedestrian trajectory prediction datasets. The results demonstrate that the proposed MambaPTP achieves competitive performance compared to advanced Transformer-based models. We hope this paper can further inspire research in Mamba for the PTP task, leading to a tighter integration of the Mamba and PTP communities.


  • Few-Shot Class-Incremental Learning via Asymmetric Supervised Contrastive Learning

    by Duo Liu, Linglan Zhao, Zhongqiang Zhang, Fan Lyu, Xiangzhong Fang, Liang Wang

    Few-Shot Class-Incremental Learning (FSCIL) is to continuously learn novel classes from a few samples without forgetting previous knowledge. Adapting directly to limited novel data typically results in significant forgetting of base class knowledge. Consequently, prevailing FSCIL methods are devoted to training a strong initial model that can be frozen in incremental sessions. However, these works face a dilemma in poor generalization: they benefit mainly from base class performance, yet underperform in novel classes. To alleviate this issue, we design a two-stage training framework to simultaneously enhance generalization for novel classes and maintain base class discrimination. In the first stage, an asymmetric supervised contrastive learning (AsyCon) algorithm is proposed. AsyCon introduces a predicted feature to achieve an asymmetric alignment of positive pairs. It alleviates over-similarity within positive features, allowing the model to better transfer to new classes in incremental sessions. In the second stage, the model is finetuned for promoting its performance on base classes. To maintain the generalization obtained in the first stage, we employ an L2 normalized regularization (LR) to keep the feature consistent with the model in the first stage. The finetuned model, termed AsyCLR, effectively balances generalization and discrimination, significantly outperforming existing FSCIL works especially in novel class accuracy. Experiments on CUB200, CIFAR100, and mini-ImageNet verify the effectiveness of our method. Additionally, our method also performs well in the standard few-shot recognition scenario due to its strong generalization ability. Our codes are available at https://github.com/APORduo/AsyCLR.