Weiyan Zhu

Credit risk assessment is a core component of financial decision-making. This study develops an explainable machine learning framework for modeling loan approval decisions on heterogeneous tabular data, centered on a Cross-Attentional Tabular Transformer that applies bidirectional cross-attention between numerical and categorical feature groups. The prediction target is historical loan-approval status, treated as a proxy for, not a direct measure of, borrower default risk; a supplementary validation on a dataset with an authentic default label is also reported. Class imbalance is addressed through focal loss, and post hoc interpretability is provided through SHAP analysis. Three classifiers, Random Forest, Gradient Boosting, and the proposed transformer, are evaluated on a 5000-sample credit dataset using accuracy, precision, recall, F1-score, ROC-AUC, and average precision. Gradient Boosting achieves the best performance (accuracy 0.9640, F1-score 0.9189), with Random Forest comparable; the proposed transformer reaches 0.9530 accuracy and 0.8949 F1, without surpassing the ensembles and at substantially higher computational cost. A five-split robustness comparison additionally evaluates XGBoost, LightGBM, CatBoost, and calibrated logistic regression: all three Gradient-Boosting variants and both classical ensembles exceed the transformer’s performance on every metric, while calibrated logistic regression does not. The evaluated baseline set excludes deep tabular architectures such as TabNet, FT-Transformer, SAINT, and TabPFN-style methods. Across the three primary classifiers, SHAP identifies credit score, employment status, and income as the dominant features, consistent with domain expectations. The results characterize the observed performance–efficiency trade-off between ensemble methods and attention-based tabular learning under the evaluated data conditions.

Information cascades in online social networks arise from stochastic interactions among user behavior, temporal activation, and graph-structured exposure. Early prediction of cascade outcomes remains difficult because only a short diffusion prefix is observable, while future propagation depends on sparse user-level transitions across a heterogeneous social network. This study develops a graph-conditioned stochastic modeling framework for early Twitter cascade prediction. Retweet cascades are formulated as history-dependent stochastic processes over a finite user vocabulary, and a causal dual-head Transformer is used to infer cascade virality and logarithmic final size from short observed prefixes. To incorporate social-network structure, user embeddings pretrained from the follow graph are introduced as external structural priors. A controlled ablation design separates the effects of random embeddings, graph-pretrained embeddings, frozen structural priors, and handcrafted feature fusion. Experiments on Higgs Twitter retweet cascades show that direct full-vocabulary next-user prediction is statistically fragile under sparse short-prefix observations, motivating macro-level cascade outcome prediction. Among the evaluated configurations, the frozen graph-pretrained Transformer achieves the strongest overall balance, reaching an AUC of 0.819, a Brier score of 0.151, and an RMSE of 0.192, while the causal Transformer without a graph prior already surpasses logistic regression and approaches Random Forest; however, gains over competitive baselines are modest and statistically significant only in selected pairwise comparisons. Calibration analysis, bootstrap confidence intervals, and paired statistical tests confirm that graph-derived user priors provide more reliable improvements than sequence modeling alone under short-prefix sparse observations. These findings indicate that graph-conditioned structural priors offer a promising complement to causal sequence modeling for early Twitter cascade prediction.

Short-term traffic forecasting is a critical component of intelligent transportation systems. While deep learning architectures for this task have proliferated rapidly, the sensor-level data characteristics—zero-value prevalence, distributional heterogeneity, and cross-sensor correlation structure—that drive architecture-specific failure modes remain insufficiently understood, and their implications for evidence-based model selection in real deployments have not been systematically addressed. This study addresses that question through a sensor-network diagnostic framework applied to the METR-LA dataset (Metropolitan Los Angeles; 207 inductive loop detectors, 5-min resolution). The framework integrates systematic characterization of sensor data properties, a controlled benchmark of four representative architectures—Transformer, Spatio-Temporal Graph Convolutional Network (STGCN), Diffusion Convolutional Recurrent Neural Network (DCRNN), and Gated Temporal Convolutional Network (Gated TCN)—under a unified 12→3 prediction setting, and a novel per-sensor regression analysis that quantitatively links zero-value ratios to model-specific prediction errors across all 207 sensors. Building on these findings, this study further proposes Graph-Enhanced Transformer (GETFormer), a lightweight hybrid architecture that augments the Transformer with a single-hop Graph Convolutional Network (GCN) layer and a gated residual fusion module. The diagnostic findings and condition-dependent model-selection guidelines provide an empirically grounded foundation for principled hybrid architecture development in urban traffic sensing.

Aero-engine predictive maintenance is challenged by heterogeneous operating conditions, complex degradation patterns, and the need for interpretable maintenance alerts rather than solely numerical life estimates. This study investigates a condition-aware data-driven framework for jet engine health assessment using the NASA C-MAPSS dataset, which contains four benchmark subsets (FD001–FD004) with different operating conditions and fault modes. Instead of formulating the task as conventional remaining useful life regression, this study reformulates degradation assessment as a three-class health state classification problem, including Normal, Warning, and Fault. A unified preprocessing pipeline is developed, incorporating condition-wise normalization, first-order differential feature construction, and per-unit sliding window segmentation to reduce operating-condition bias, capture degradation dynamics, and prevent data leakage. Five representative models are evaluated under the same framework, including XGBoost, LightGBM, Random Forest, a context-aware multi-scale temporal attention convolutional neural network, and a bidirectional long short-term memory network. The results show that the proposed framework achieves consistently high classification accuracy across all four subsets, with the best results of 0.9841 on FD001, 0.9764 on FD002, 0.9891 on FD003, and 0.9832 on FD004. In addition, Bi-LSTM outperforms MSTA-CNN on all subsets, for example improving accuracy from 0.9614 to 0.9747 on FD002 and from 0.9773 to 0.9806 on FD004, which is consistent with the importance of long-term temporal dependency modeling for this task. These findings suggest that the proposed framework provides an effective and maintenance-decision-aligned solution for C-MAPSS-based health monitoring, where the three-class alert output offers clearer operational meaning than a single numerical life estimate.

Robust defect classification is critical for intelligent process inspection and quality control in steel manufacturing, but it remains challenging when industrial tabular data are small, imbalanced, statistically skewed, and characterized by nonlinear inter-feature dependencies. This study proposes a robust steel plate defect classification framework based on a feature-embedded Transformer. A quantile-based transformation is first introduced to regularize heterogeneous and heavy-tailed process descriptors. Each numerical variable is then represented as a learnable feature token and processed by a Transformer encoder to model contextual interactions among positional, geometric, luminosity-related, and morphological attributes. Experiments were conducted on the Steel Plates Faults dataset, containing 1941 samples, 27 input features, and 7 defect categories. On the held-out test set, the model achieved an accuracy of 0.735, remaining competitive with XGBoost (0.794) and Random Forest (0.783). SHAP and self-attention analyses further indicate that the model captures distributed and interaction-aware defect representations, providing an interpretable solution for robust industrial defect classification.