Liang Kou

As Vision Transformer (ViT) models continue to scale, the training and fine-tuning costs for achieving advanced performance have surged, forcing us to rethink the sustainability of relying solely on model scaling. In this work, we explore a more cost-effective alternative. By effectively exploiting the complementary potential of existing frozen models, we avoid the expense of training large models from scratch. Based on this idea, we propose the Dual-ViT Fusion Transformer (DVFT). This framework explicitly aligns and fuses heterogeneous features from different frozen backbones, thereby addressing the limited representational scope and insufficient cross-scale semantic coverage of a single frozen model. Such limitations are difficult to mitigate within existing parameter-efficient fine-tuning frameworks, which are primarily designed for single-model settings. Specifically, DVFT introduces two complementary guidance paradigms. The first is Intrinsic Anchor-based calibration, which uses the frozen models’ own global semantic representations as alignment references to establish cross-model semantic consistency. The second is Extrinsic Query-based guidance, which employs learnable queries as an information bottleneck to progressively extract and integrate complementary discriminative features across models, enabling flexible feature interaction. Experimental results show that our method approaches the performance of expensively trained models with very few training epochs. These findings indicate that DVFT provides a scalable and low-cost pathway for efficiently reusing existing frozen models to achieve high-performance visual recognition.

This study tackles computational bottlenecks, training instability, and insufficient cross-modal semantic alignment in high-resolution multimodal image processing. We propose TriVLLo, an innovative multi-scale vision-language modeling framework. Our main contributions are: First, we introduce factorized 2D positional encoding and a dynamically configurable modular architecture. This approach decouples height and width position information. It improves spatial localization reliability for images with extreme aspect ratios. It also reduces computational parameters and alleviates training instability. Second, we design a unified multi-scale feature extraction and modality interaction mechanism. This uses adaptive image processing and multi-perspective feature pyramids. It enhances robustness to inputs of any resolution. It also achieves fine-grained alignment of vision-language features through a shared embedding space. Third, we build a high-quality dataset with 11K samples for fine-grained reasoning. This dataset supports improvements in visual ranking, semantic alignment, and narrative reasoning. Experiments show that TriVLLo achieves 87.4% of GPT-4V's performance on the MM-Vet benchmark. It demonstrates a 93.0 percentage-point improvement over Emu2 in spatial cognition tasks. It attains 89.2% accuracy on knowledge generation tasks. These results significantly outperform state-of-the-art methods.

Advanced Persistent Threats (APTs) pose severe challenges to Next-Generation Communication Networks (NGCNs) due to their stealthiness and NGCNs’ dynamic topology, while conventional GNN-based intrusion detection systems suffer from static aggregation and poor adaptability to unseen nodes. To address these issues, this paper proposes DAA-FPN-SAGE, a lightweight graph-based detection framework integrating Dynamic Adaptive Aggregation (DAA) and Multi-Scale Feature Pyramid Network (MSFPM). Leveraging GraphSAGE’s inductive learning capability, the framework effectively models unseen nodes or subgraphs and adapts to NGCN’s dynamic changes (e.g., elastic network slicing, online AI model updates)—a key advantage for handling NGCN’s real-time topological variations. The DAA module employs multi-hop attention to dynamically assign weights to neighbors at different hop distances, enhancing capture of hierarchical dependencies in multi-stage APT attack chains. The MSFPM module fuses local-global structural information via a gated feature selection mechanism, resolving dimensional inconsistency and enriching attack behavior representation. Extensive experiments on StreamSpot, Unicorn, and DARPA TC#3 datasets demonstrate superior performance, meeting detection requirements of large-scale NGCNs.

Time-series prediction is essential in many real-world applications, including power monitoring, traffic forecasting, and early warning of extreme weather events. While deep learning-based models have demonstrated strong performance in this domain, most existing approaches treat the input sequence holistically, failing to account for the distinct characteristics of its underlying components. This often hinders the extraction of deep, meaningful features. To address this issue, we propose a novel model, termed seasonal-trend decomposition based on loess with multigraph attention interaction (STL-MGAI), which integrates time-series decomposition with component-specific graph modeling to improve both pattern learning and feature fusion. Specifically, STL decomposition is employed to extract three orthogonal components—trend, seasonal, and residual—which are then processed individually using adaptive graph convolution, a periodic-constrained graph attention network (GAT), and dynamic graph convolution, respectively. A feature fusion module, along with orthogonality constraints, ensures effective integration of these componentwise features while preserving their independence. Additionally, embedding this architecture within an Informer-based encoder–decoder framework enhances its capacity for long-range sequence modeling. Extensive experiments on both univariate and multivariate benchmarks demonstrate that STL-MGAI achieves superior accuracy and efficiency in long-term forecasting tasks. Notably, it improves prediction accuracy by 16.38% in energy consumption forecasting and by 19.52% in weather prediction. The model’s modular design and adaptive graph convolution also contribute to enhanced generalization and interpretability.

The extremely destructive nature of malware has become a major threat to Internet security. The research on malware detection techniques has been evolving. Deep learning-based malware detection methods have achieved good results by using large-scale, pre-trained models. However, these models are complex, have large parameters, and require a large amount of hardware resources and have a high inference time cost when applied. To address this challenge, this paper proposes DistillMal, a new method for lightweight malware detection based on knowledge distillation, which improves performance by using a student network to learn valuable cueing knowledge from a teacher network to achieve a lightweight model. We conducted extensive experiments on two new datasets and showed that the student network model’s performance is very close to that of the original model and the outperforms it on some metrics. Our approach helps address the resource constraints and computational challenges faced by traditional deep learning large models. Our research highlights the potential of using knowledge distillation to develop lightweight malware detection models.

Thanks to the development of the Internet of Things, a large number of sensors have been deployed, resulting in the collection of abundant time series data. These data series contain potential space-time connection and noise at the same time. Prediction and health management (PHM) aim to provide decision support based on the health state of an entire engineering system. Additionally, the predicting future values also contribute to decision making and fall under the category of time series forecasting. While the existing methods focus on capturing the correlation between the data and reducing the impact of noise, they often fail to fully utilize the noise present in the time series data. In this article, we propose a framework for multivariate time series forecasting called GAGNN. This framework integrates the idea of a generative adversarial network and a graph neural network organically. It adopts the graph neural network as the generator and a multilayer perceptron as the discriminator. Finally, the prediction module is used to obtain the prediction results. The generator, discriminator, and prediction module are trained jointly. Our experimental results demonstrate that our model outperforms the original model on most benchmark data sets and achieves the best results on three out of six benchmark data sets.

The detection of abnormal electricity load data using big data analysis technology has garnered considerable attention from the academic community. However, traditional methods often require ample labeled data to train the model which increases the cost. This paper tackles the issues of high model training costs and poor transferability associated with traditional supervised learning methods. We propose a Contrastive Learning Network based Abnormal Electricity Load Detection Method (ED-CLN). Firstly, our model enhances training samples through data augmentation and learns the similarities and differences between samples from temporal and contextual perspectives of the sample sequence. This approach enables the acquisition of common feature representations for model training tasks. Then, the weight data of the model trained using unlabeled data is migrated to the supervised training model. Finally, the trained source model is fine-tuned for abnormal electricity load data detection tasks to improve the overall learning effectiveness of the model. The results demonstrate that ED-CLN outperforms both supervised learning methods and various classic contrastive learning methods in anomaly detection, which can effectively identify the abnormal electricity load data.

Due to the rapid development and widespread presence of malware, deep-learning-based malware detection methods have become a pivotal approach used by researchers to protect private data. Behavior-based malware detection is effective, but changes in the running environment and malware evolution can alter API calls used for detection. Most existing methods ignore API call parameters while analyzing them separately, which loses important semantic information. Therefore, considering API call parameters and their combinations can improve behavior-based malware detection. To improve the effectiveness of behavior-based malware detection systems, this paper proposes a novel API feature engineering method. The proposed method employs parameter-augmented semantic chains to improve the system’s resilience to unknown parameters and elevate the detection rate. The method entails semantically decomposing the API to derive a behavior semantic chain, which provides an initial representation of the behavior exhibited by samples. To further refine the accuracy of the behavior semantic chain in depicting the behavior, the proposed method integrates the parameters utilized by the API into the aforementioned semantic chain. Furthermore, an information compression technique is employed to minimize the loss of critical actions following truncation of API sequences. Finally, a deep learning model consisting of gated CNN, Bi-LSTM, and an attention mechanism is used to extract semantic features embedded within the API sequences and improve the overall detection accuracy. Additionally, we evaluate the proposed method on a competition dataset Datacon2019. Experiments indicate that the proposed method outperforms baselines employing vocabulary-based methods in both robustness to unknown parameters and detection rate.

Machine learning-based malware (malicious software) detection methods have a wide range of real-world applications. However, these types of approaches suffer from the fatal problem of “model aging”, in which the validity of the model decreases rapidly as the malware continues to evolve and variants emerge continuously. The model aging problem is usually solved by model retraining, which relies on lots of labeled samples obtained at great expense. To address this challenge, this paper proposes a semi-supervised continuous learning malware detection model based on Transformer. Firstly, this model improves the lifelong semi-supervised mixture algorithm to dynamically adjust the weighted combination of new sample sequences and historical ones to solve the imbalance problem. Secondly, the Learning with Local and Global Consistency algorithm is used to iteratively compute similarity scores for the unlabeled samples in the mixed samples to obtain pseudo-labels. Lastly, the Multilayer Perceptron is applied for malware classification. To validate the effectiveness of the model, this paper conducts experiments on the CICMalDroid2020 dataset. The experimental results show that the proposed model performs better than existing deep learning detection models. The F1 score has an average improvement of 1.27% compared to other models when conducting binary classification. And, after inputting hybrid samples, including historical data and new data, four times, the F1 score is still 1.96% higher than other models.