De-Chuan Zhan

The practice of knowledge distillation, wherein pre-trained models – termed ”teachers” – serve to enhance the performance of subsequent ”student” models, has seen a surge in popularity. With a lot of pre-trained models available, there is a pressing need to investigate effective mechanisms for utilizing these resources optimally. This paper tackles two main challenges inherent in developing such mechanisms. The initial challenge lies in the effective selection of the most advantageous models from a vast pool, given that exhaustively testing each pre-trained model is impractical. The second challenge stems from the diversity of pre-trained models, as it is common for a pre-trained model to have a different label space than that of the current task. This necessitates the development of a universal model reuse approach capable of integrating any pre-trained model, regardless of its unknown label set. To address these issues, we introduce a dual-phase framework named “Selective Cross-Label Distillation.” The first phase, termed model assessment, evaluates the semantic similarity between a potential pre-trained model and the target model through optimal transport. By determining the transportation cost, we pinpoint candidate models with lower costs to serve as our source models. The second phase, knowledge reuse, concentrates on minimizing the transportation cost between the chosen source models and the target model. This dual-phase approach enables efficient model selection and bridges the semantic gap between pre-trained models and the target task, leading to enhanced performance. The effectiveness of our framework in model selection and knowledge reuse is validated by experimental results.

Hierarchical clustering produces a cluster tree with different granularities. As a result, hierarchical clustering provides richer information and insight into a dataset than partitioning clustering. However, many existing hierarchical clustering algorithms have two weaknesses: scalability and the ability to handle clusters of varying densities. We identify that existing algorithms have these weaknesses because they rely on pairwise point-based similarity calculations and the similarity measure is independent of data distribution. With this identification, we aim to overcome these weaknesses by proposing a novel efficient hierarchical clustering called StreaKHC that enables massive streaming data to be mined. The enabling factor is the use of a scalable Point-Set kernel to measure the similarity between a new point in the data stream and a cluster in the current cluster tree. It also has an efficient mechanism to update the hierarchical structure so that a high-quality cluster tree can be maintained in real time. Our extensive empirical evaluation shows that StreaKHC is more accurate and more efficient than existing hierarchical clustering algorithms. Furthermore, we demonstrate that StreaKHC can be used for anomaly detection with comparable accuracy to existing state-of-the-art anomaly detection algorithms.

Antimicrobial peptides (AMPs) are promising alternatives to conventional antibiotics, yet early translation is often hindered by the perceived coupling between antibacterial potency and mammalian toxicity. This assumption complicates prioritization: highly active candidates are frequently suspected to be hemolytic, while existing multi-task predictors rarely reveal where selectivity resides in sequence space. Here, we present Multifunctional AMP Learning Engine (MAPLE), an interpretable dual-stream framework for AMP identification and systematic category-specific functional profiling across 14 activity categories directly from peptide sequences. MAPLE combines protein language model embeddings with explicit physicochemical descriptors, enabling robust task-specific prediction under severe label imbalance. Across the benchmark dataset and a sequence-non-overlapping independent validation set, MAPLE achieves consistently well-balanced performance, including on low-prevalence but clinically relevant endpoints. Building on this predictive basis, we conduct systematic k-mer enrichment to map motif-level selectivity and show that potency–hemolysis coupling is motif-regime-dependent rather than universal. Motifs most strongly enriched for antibacterial activity exhibit reduced hemolytic overlap and occupy a physicochemical regime characterized by moderate cationicity, lower hydrophobicity, and higher amphipathicity. We further provide a proof-of-concept prioritization workflow leveraging antibacterial-selective motifs, with structural modeling yielding conformations consistent with amphipathic α-helices. Despite limitations of predominantly binary annotations and incomplete structural integration, MAPLE offers reproducible sequence-level hypotheses and prioritization principles to support the engineering of potent and safer AMPs.

Class-Incremental Learning (CIL) aims to continually learn new categories without forgetting previously acquired knowledge. Vision-language models such as CLIP offer strong transferable representations via multi-modal supervision, making them promising for CIL. However, applying CLIP to CIL poses two major challenges: (1) adapting to downstream tasks often requires additional learnable modules, increasing model complexity and susceptibility to forgetting; and (2) while multi-modal representations offer complementary strengths, existing methods have yet to fully realize their potential in effectively integrating visual and textual modalities. To address these issues, we propose BOFA (Bridge-layer Orthogonal Fusion for Adaptation), a novel framework for CIL. BOFA confines all model adaptation exclusively to CLIP's existing cross-modal bridge-layer, thereby adding no extra parameters or inference cost. To prevent forgetting within this layer, it leverages Orthogonal Low-Rank Fusion, a mechanism that constrains parameter updates to a low-rank ``safe subspace" mathematically constructed to be orthogonal to past task features. This ensures stable knowledge accumulation without data replay. Furthermore, BOFA employs a cross-modal hybrid prototype that synergizes stable textual prototypes with visual counterparts derived from our stably adapted bridge-layer, enhancing classification performance. Extensive experiments on standard benchmarks show that BOFA achieves superior accuracy and efficiency compared to existing methods.

Autoregressive (AR) image generation models are capable of producing high-fidelity images but often suffer from slow inference due to their inherently sequential, token-by-token decoding process. Speculative decoding, which employs a lightweight draft model to approximate the output of a larger AR model, has shown promise in accelerating text generation without compromising quality. However, its application to image generation remains largely underexplored. The challenges stem from a significantly larger sampling space, which complicates the alignment between the draft and target model outputs, coupled with the inadequate use of the two-dimensional spatial structure inherent in images, thereby limiting the modeling of local dependencies. To overcome these challenges, we introduce Hawk, a new approach that harnesses the spatial structure of images to guide the speculative model toward more accurate and efficient predictions. Experimental results on multiple text-to-image benchmarks demonstrate a 1.71x speedup over standard AR models, while preserving both image fidelity and diversity.

Class-Incremental Learning (CIL) aims to endow models with the ability to continuously adapt to evolving data streams. Recent advances in pre-trained vision-language models (e.g., CLIP) provide a powerful foundation for this task. However, existing approaches often rely on simplistic templates, such as "a photo of a [CLASS]", which overlook the hierarchical nature of visual concepts. For example, recognizing "cat" versus "car" depends on coarse-grained cues, while distinguishing "cat" from "lion" requires fine-grained details. Similarly, the current feature mapping in CLIP relies solely on the representation from the last layer, neglecting the hierarchical information contained in earlier layers. In this work, we introduce HiErarchical Representation MAtchiNg (HERMAN) for CLIP-based CIL. Our approach leverages LLMs to recursively generate discriminative textual descriptors, thereby augmenting the semantic space with explicit hierarchical cues. These descriptors are matched to different levels of the semantic hierarchy and adaptively routed based on task-specific requirements, enabling precise discrimination while alleviating catastrophic forgetting in incremental tasks. Extensive experiments on multiple benchmarks demonstrate that our method consistently achieves state-of-the-art performance.

In reinforcement learning (RL), agents continually interact with the environment and use the feedback to refine their behavior. To guide policy optimization, reward models are introduced as proxies of the desired objectives, such that when the agent maximizes the accumulated reward, it also fulfills the task designer's intentions. Recently, significant attention from both academic and industrial researchers has focused on developing reward models that not only align closely with the true objectives but also facilitate policy optimization. In this survey, we provide a comprehensive review of reward modeling techniques within the RL literature. We begin by outlining the background and preliminaries in reward modeling. Next, we present an overview of recent reward modeling approaches, categorizing them based on the source, the mechanism, and the reward learning paradigm. Building on this understanding, we discuss various applications of these reward modeling techniques and review methods for evaluating reward models. Finally, we conclude by highlighting promising research directions in reward modeling. Altogether, this survey includes both established and emerging methods, filling the vacancy of a systematic review of reward models in current literature.