Suan Lee

Supervised fine-tuning is key for model alignment, but its mechanisms are debated, with conflicting evidence supporting either a superficial alignment hypothesis or significant task improvements. This paper examines supervised fine-tuning’s impact from the perspective of verbatim memorization. Using the open-source OLMo-2 model series and test datasets (instruction format, safety-sensitive, and factual knowledge) constructed from its pre-training corpus, we analyzed changes across memorization, linguistic styles, and task performance. We found that supervised fine-tuning significantly weakens the model’s verbatim memorization of pre-training data. Simultaneously, it improves generated text in terms of alignment objectives, such as polite expression and structured organization. However, this process also leads to performance degradation on knowledge-intensive downstream tasks. Further representation analysis reveals that these changes are mainly concentrated in the later layers of the model. We conclude that supervised fine-tuning acts as a continuation of the learning process on new data. By adjusting model representations, supervised fine-tuning induces a learning tilt toward the styles and content of the instruction-tuning dataset. This inclination successfully instills alignment objectives while consequently reducing the effective accessibility of previously learned knowledge, which indicates the observed degradation in both pre-training data memorization and factual task performance. The source code is publicly available.

This study proposes an advanced Internet fraud transaction detection method, the Temporal-aware Heterogeneous Graph Oversampling and Attention Fusion Network (THG-OAFN), designed to address the increasingly severe fraud issues in EC. The method innovatively abstracts transaction data into a heterogeneous graph structure, captures temporal dynamic features through Gated Recurrent Unit (GRU), and fuses Graph Neural Network (GNN) to process static topological relationships. To address data imbalance, an improved Graph-based Synthetic Minority Oversampling Technique (GraphSMOTE) framework is introduced, maintaining the structural integrity of fraud clusters through k-hop topological constraints. Meanwhile, a multi-layer attention mechanism (including relationship fusion, neighborhood fusion, and information perception modules) is employed to achieve active fraud prevention. Experimental results show that THG-OAFN attains an area under the curve (AUC) of 96.56% (a 7.78% improvement over the best baseline). Moreover, it achieves a recall of 95.21% (a 6.29% improvement) and an F1-score of 94.72% (a 3.96% improvement) on the Amazon dataset. On the YelpChi dataset, these three metrics reach 90.43%, 89.51%, and 90.31%, respectively, remarkably outperforming existing GNN models. This achievement provides a deployable solution for dynamic fraud detection and active defense. Our code is available at https://github.com/wei4zheng/THG-OAFN .

In time-series studies involving bearing sensor data, Gaussian noise and white noise techniques are commonly employed to evaluate model robustness. However, these conventional noise techniques are limited in their applicability to real-world industrial environments. This paper proposes three novel noise techniques—electrical interference noise, harmonic noise, and random shock noise—that more accurately reflect the complex noise encountered in industrial settings. Additionally, a new deep learning model, MultiPatchTST, is introduced, demonstrating robust performance under various noise conditions. Experimental results reveal that Gaussian noise has minimal impact on model performance, whereas the proposed noise techniques significantly affect performance, providing a more realistic evaluation of noise robustness. The proposed MultiPatchTST model achieves superior performance across all metrics in the presence of all four noise types, confirming its robustness and reliability.

In the contemporary manufacturing landscape, the advent of artificial intelligence and big data analytics has been a game-changer in enhancing product quality. Despite these advancements, their application in diagnosing failure probability and risk remains underexplored. The current practice of failure risk diagnosis is impeded by the manual intervention of managers, leading to varying evaluations for identical products or similar facilities. This study aims to bridge this gap by implementing advanced data analysis techniques on maintenance data from an aluminum extruder. We have employed text embedding, dimensionality reduction, and feature extraction methods, integrating the K-means algorithm with the Silhouette Score for risk level classification. Our findings reveal that the combination of Word2Vec for embedding and Contractive Auto Encoder for dimensionality reduction and feature extraction yields high-performance results. The optimal cluster count, identified as three, achieved the highest Silhouette Score. Statistical analysis using one-way ANOVA confirmed the significance of these findings with a p-value of 5.3213 × e⁻⁶, well within the 5% significance threshold. Furthermore, this study utilized BERTopic for topic modeling to extract principal topics from each cluster, facilitating an in-depth analysis of the clusters in relation to the extruder’s characteristics. The outcome of this research offers a novel methodology for facility managers to objectively diagnose equipment failures. By minimizing subjective judgment, this approach is poised to significantly enhance the efficacy of quality assurance systems in manufacturing, leveraging the robust capabilities of artificial intelligence.