Chi-Ho Lin

Supervised fine-tuning is key for model alignment, but its mechanisms are debated, with conflicting evidence supporting either a superficial alignment hypothesis or significant task improvements. This paper examines supervised fine-tuning’s impact from the perspective of verbatim memorization. Using the open-source OLMo-2 model series and test datasets (instruction format, safety-sensitive, and factual knowledge) constructed from its pre-training corpus, we analyzed changes across memorization, linguistic styles, and task performance. We found that supervised fine-tuning significantly weakens the model’s verbatim memorization of pre-training data. Simultaneously, it improves generated text in terms of alignment objectives, such as polite expression and structured organization. However, this process also leads to performance degradation on knowledge-intensive downstream tasks. Further representation analysis reveals that these changes are mainly concentrated in the later layers of the model. We conclude that supervised fine-tuning acts as a continuation of the learning process on new data. By adjusting model representations, supervised fine-tuning induces a learning tilt toward the styles and content of the instruction-tuning dataset. This inclination successfully instills alignment objectives while consequently reducing the effective accessibility of previously learned knowledge, which indicates the observed degradation in both pre-training data memorization and factual task performance. The source code is publicly available.

In the 21st century, the increasing complexity and uncertainty of the global economy have heightened the need for accurate economic forecasting. Per capita GDP, a critical indicator of living standards, economic growth, and productivity, plays a key role in government policy-making, corporate strategy, and investor decisions. However, predicting per capita GDP poses significant challenges due to its sensitivity to various economic and social factors. Traditional methods such as statistical analysis, regression, and time-series models have shown limitations in capturing nonlinear interactions and volatility of economic data. To address these limitations, this study develops a per capita GDP forecasting model based on deep learning, incorporating key macroeconomic variables—the Consumer Price Index (CPI) and unemployment rate (UR)—to enhance predictive accuracy. This study employs five deep-learning regression models (RNN, LSTM, GRU, TCN, and Transformer) applied to real and placebo datasets, each incorporating combinations of CPI and UR. The results demonstrate that deep learning models can effectively capture complex, nonlinear relationships in economic data, significantly improving predictive accuracy compared to traditional models. Among the models, the Transformer consistently achieves the highest R-squared and lowest error values across various metrics (MSE, RMSE, and MSLE), indicating its superior ability to model intricate economic patterns. In addition, including CPI and UR as additional predictors enhances model robustness, with the TCN and Transformer models showing particularly strong performance in capturing short-term economic fluctuations. The findings suggest that the deep learning models, especially the Transformer, offer valuable tools for policymakers and business leaders, providing reliable GDP forecasts that support economic decision-making, resource allocation, and strategic planning. Academically, this study advances the understanding of deep learning applications in economic forecasting, particularly in integrating significant macroeconomic variables for enhanced predictive performance. The developed model is a foundation for informed economic policy and strategic decisions, offering a robust and actionable framework for managing economic uncertainties. This research contributes to theoretical and applied economics, providing insights that bridge academic innovation with practical utility in economic forecasting.

Recently, the use of Automated Guided Vehicles (AGVs) at production sites has been increasing due to industrial development, such as the introduction of smart factories. AGVs utilizing the wired induction method, which is cheaper and faster than the wireless induction method, are mainly used at production sites. However, the wired guidance AGV operation system has the disadvantage of being limited to small-batch or collaboration-based production sites, since it is difficult to change the driving route. In this paper, we propose an AGV line-scan algorithm that can perform route recognition, driving commands, and operation through color-code recognition using an Arduino controller and a low-cost vision sensor, instead of the optical sensor conventionally used for these functions. When the proposed algorithm is applied to the AGV car, the CMUcam 5 Pixy2 camera identifies the driving path to follow by tracking a black line using the Otsu method. In addition, it can be confirmed that the driving command is executed using the proposed color code by applying the color recognition function of the CMUcam 5 Pixy2.

In this paper, we propose a boosted 3-D PCA algorithm based on an efficient analysis method. The proposed method involves three steps that improve image detection. In the first step, the proposed method designs a new analysis method to solve the performance problem caused by data imbalance. In the second step, a parallel cross-validation structure is used to enhance the new analysis method further. We also design a modified AdaBoost algorithm to improve the detector accuracy performance of the new analysis method. In order to verify the performance of this system, we experimented with a benchmark dataset. The results show that the new analysis method is more efficient than other image detection methods.