Yike Li

Knowledge graphs (KGs) organize relational facts in a form that supports machine reasoning across many domains. In decentralized Internet of Things (IoT) environments, such graphs can describe interactions among distributed devices and services. However, privacy constraints, intermittent communication, and incomplete observations can leave the resulting graphs sparse and incomplete. This sparsity weakens path availability and makes missing-fact inference difficult. Multi-hop reasoning can recover unobserved links through explicit reasoning paths, but conventional hierarchical reinforcement learning (HRL) approaches may explore inefficiently when only a few valid paths are available. This paper presents attentiON exploratIon of cluster relatioN (ONION), a framework for sparse KG reasoning. ONION introduces a cluster relation augmentation (CRA) module to add cluster relation cues to the action space, together with a relation-level attention (RLA) mechanism to emphasize relations that are relevant to the query. These components guide hierarchical exploration toward more informative paths. ONION is evaluated on four public sparse-KG benchmarks, and the results show competitive performance with the clearest gains on highly sparse datasets.

Recent advances in Deep Learning (DL) have enhanced Aging-Related Bug (ARB) prediction for mitigating software aging. However, DL-based ARB prediction models face a dual challenge: overcoming overfitting to enhance generalization and managing the high labeling costs associated with extensive data requirements. To address the first issue, we utilize the sparse and binary nature of spiking communication in Spiking Neural Networks (SNNs), which inherently provides brain-inspired regularization to effectively alleviate overfitting. Therefore, we propose a Spiking Convolutional Neural Network (SCNN)-based ARB prediction model along with a training framework that handles the model's spatial-temporal dynamics and non-differentiable nature. To reduce labeling costs, we introduce a Bio-inspired and Diversity-aware Active Learning framework (BiDAL), which prioritizes highly informative and diverse samples, enabling more efficient usage of the limited labeling budget. This framework incorporates bio-inspired uncertainty to enhance informativeness measurement along with using a diversity-aware selection strategy based on clustering to prevent redundant labeling. Experiments on three ARB datasets show that ARB-SCNN effectively reduces overfitting, improving generalization performance by 6.65% over other DL-based classifiers. Additionally, BiDAL boosts label efficiency for ARB-SCNN training, outperforming four state-of-the-art active learning methods by 4.77% within limited labeling budgets.

Recent advances in deep learning (DL) have brought tremendous gains in signal modulation classification. However, DL-based classifiers lack transparency and interpretability, which raises concern about model's reliability and hinders the wide deployment in real-word applications. While explainable methods have recently emerged, little has been done to explain the DL-based signal modulation classifiers. In this work, we propose a novel model-agnostic explainer, Model-Agnostic Signal modulation classification Explainer (MASE), which provides explanations for the predictions of black-box modulation classifiers. With the subsequence-based signal interpretable representation and in-distribution local signal sampling, MASE learns a local linear surrogate model to derive a class activation vector, which assigns importance values to the timesteps of signal instance. Besides, the constellation-based explanation visualization is adopted to spotlight the important signal features relevant to model prediction. We furthermore propose the first generic quantitative explanation evaluation framework for signal modulation classification to automatically measure the faithfulness, sensitivity, robustness, and efficiency of explanations. Extensive experiments are conducted on two real-world datasets with four black-box signal modulation classifiers. The quantitative results indicate MASE outperforms two state-of-the-art methods with 44.7% improvement in faithfulness, 30.6% improvement in robustness, and 44.1% decrease in sensitivity. Through qualitative visualizations, we further demonstrate the explanations of MASE are more human interpretable and provide better understanding into the reliability of black-box model decisions.

Despite the broad applications of reinforcement learning in various fields, low training efficiency caused by the sparse reward problem remains crucial. In the context of pathfinding, consider the following question: given a limited number of expert trajectories, how can we find an efficient strategy that allows an agent to be trained in a shorter time achieving similar training performance in complex environments? In this paper, we propose a training strategy based on the results of inverse reinforcement learning, which not only performs well in the same environment but also reduces training time. The idea is to apply Gaussian-based inverse reinforcement learning to analyze expert trajectories, inferring reward functions and clustering. Then the framework of curriculum learning is introduced to shorten the training time and improve the training efficiency under complex tasks and extensive data. The experimental section explores the general rules for the optimal number of intermediate points. Ablation experiments were set up to demonstrate the scientific validity of the middle point selection of the curriculum. Additionally, we compare the time and reward function of our proposed method with baselines under different map sizes, numbers of obstacles, and distances to evaluate the learning performance in various scenarios. Based on the experimental results, our proposed approach outperforms the best baseline in most cases and achieves training performance similar to expert trajectories with less training time, solving the sparse reward problem.

Robust reinforcement learning (RL) has been a challenging problem due to the gap between simulation and the real world. Existing efforts typically address the robust RL problem by solving a max-min problem. The main idea is to maximize the cumulative reward under the worst-possible perturbations. However, the worst-case optimization either leads to overly conservative solutions or unstable training process, which further affects the policy robustness and generalization performance. In this paper, we tackle this problem from both formulation definition and algorithm design. First, we formulate the robust RL as a max-expectation optimization problem, where the goal is to find an optimal policy under both the worst cases and the non-worst cases. Then, we propose a novel framework DRRL to solve the max-expectation optimization. Given our definition of the feasible tasks, a task generation and sequencing mechanism is introduced to dynamically output tasks at appropriate difficulty level for the current policy. With these progressive tasks, DRRL realizes dynamic multi-task learning to improve the policy robustness and the training stability. Finally, extensive experiments demonstrate that the proposed method exhibits significant performance on the unmanned CarRacing game and multiple high-dimensional MuJoCo environments.