Shiliang Zhang

Recent works have indicated redundancy across transformer blocks, prompting the research of depth compression to prune less crucial blocks. However, current ways of entire-block pruning suffer from risks of discarding meaningful cues learned in those blocks, leading to substantial performance degradation. As another line of model compression, channel pruning can better preserve performance, while it cannot reduce model depth and thus often brings limited latency reduction in autoregressive inference. To pursue better model compression and acceleration, this paper proposes FlattenGPT, a novel way to detect and reduce depth-wise redundancies.By flatting two adjacent blocks into one, it compresses the network depth, meanwhile enables more effective parameter redundancy detection and removal. FlattenGPT allows to preserve the knowledge learned in all blocks, and remains consistent with the original transformer architecture. Extensive experiments demonstrate that FlattenGPT enhances model efficiency with a decent trade-off to performance. It outperforms existing pruning methods in both zero-shot accuracies and WikiText-2 perplexity across various model types and parameter sizes. On LLaMA-2/3 and Qwen-1.5 models, FlattenGPT retains 90-96% of zero-shot performance with a compression ratio of 20%. It also outperforms other pruning methods in accelerating LLM inference, making it promising for enhancing the efficiency of transformers.

Cloth-changing person re-identification (CC-ReID) aims to extract cloth-irrelevant visual features to match individuals across different outfits. While images contain rich visual cues, textual descriptions often provide explicit and structured attribute information, offering a potential pathway to decouple and suppress cloth-relevant features in visual representations. In this paper, we propose a novel prompt-guided decoupled feature learning framework that leverages text prompts to guide the learning of cloth-irrelevant features for CC-ReID. Given that human-annotated descriptions are typically unavailable in CC-ReID datasets, we explore the use of prompts generated by multimodal large language models (MLLMs) as a substitute. To extract cloth-relevant features, we design a prompt-guided visual feature extractor that learns attribute-aware features aligned with the prompts. A decoupled feature learning module is then employed to suppress the cloth-relevant components from the raw visual features. Through this prompt-guided learning strategy, our framework is able to construct cloth-irrelevant representations. Extensive experiments on multiple CC-ReID benchmarks demonstrate that our method outperforms existing state-of-the-art approaches, validating its effectiveness and robustness.

Backward-compatible training enables the deployment of advanced models without requiring updates to old gallery databases. However, existing methods, including old-prototype-based (i.e., those relying on prototypes from the old model) and instance-based approaches, often overlook the impact of the old model's quality. High-quality old models exhibit compact intra-class feature distributions, which facilitate effective alignment between old and new models across various methods. In contrast, low-quality old models produce dispersed features, making it difficult for old-prototype-based methods to extract sufficient information. Additionally, instance-based methods are overly restrictive, limiting the flexibility of new models. In this work, we propose SA-BCT, an extremely simple yet effective backward-compatible training method that offers a unified framework for accommodating old models of varying quality. SA-BCT employs a single loss function applied to both old and new features, self-adaptively adjusting the constraint space for new features based on the distribution of old features. Extensive experiments in diverse settings demonstrate the effectiveness of SA-BCT. Code is available at https://github.com/yuleung/SA-BCT.

Weakly supervised group activity recognition (WSGAR) aims to identify the joint activity of a group of people without relying on hand-annotated human bounding boxes. Existing WSGAR methods typically acquire coarse human-level features by pooling from detected bounding boxes or applying human queries with cross attentions. These approaches focus on learning human relations from the acquired features. However, discriminative person-specific clues might be confused with irrelevant backgrounds, hindering the effectiveness of downstream human relation learning. To address this limitation, we propose a Human Feature Refinement framework that enhances human-level information with graph convolutional networks and self-attention. We define in-box regions as tokens and learn their spatial correspondence through GCN and self-attention. By explicitly extracting in-box details and suppressing irrelevant regions, our method acquires more discriminative human-level features for relation learning and group activity prediction. We further propose a Graph-based Token Merging algorithm to reduce the computation cost of Human Feature Refinement, while minimizing information loss and overfitting risk. Experiments show that our method outperforms previous WSGAR methods on Volleyball, NBA and JRDB-PAR benchmarks, with reduced computation cost.

Traditional approaches to adapting multi-modal large language models (MLLMs) to new tasks have relied heavily on fine-tuning. This paper introduces Efficient Multi-Modal Long Context Learning (EMLoC), a novel training-free alternative that embeds demonstration examples directly into the model input. EMLoC offers a more efficient, flexible, and scalable solution for task adaptation. Because extremely lengthy inputs introduce prohibitive computational and memory overhead, EMLoC contributes a chunk-wise compression mechanism combined with layer-wise adaptive pruning. It condenses long-context multimodal inputs into compact, task-specific memory representations. By adaptively pruning tokens at each layer under a Jensen-Shannon divergence constraint, our method achieves a dramatic reduction in inference complexity without sacrificing performance. This approach is the first to seamlessly integrate compression and pruning techniques for multi-modal long-context learning, offering a scalable and efficient solution for real-world applications. Extensive experiments on diverse vision-language benchmarks demonstrate that EMLoC achieves performance on par with or superior to naive long-context approaches. Our results highlight the potential of EMLoC as a groundbreaking framework for efficient and flexible adaptation of multi-modal models in resource-constrained environments. Codes are publicly available at https://github.com/Zehong-Ma/EMLoC.