Yufeng Wang

Graphs are widely used to describe objects and their interactions in physically-informed real-world networking scenario including transportation, networking and energy, etc. Graph neural network (GNN) is the latest deep learning (DL) model for processing graph-structured data, widely applied in various tasks, e.g., prediction, classification/fault detection, and decision-making problems. However, GNNs based tasks still face two major challenges. First, GNN’s powerful ability in learning expressive graph representations relies on the availability and quality of graph structures, which practically, are often noisy, incomplete, or even unavailable. Second, various GNN architectures are typically designed in a manually heuristic manner, which not only requires specified knowledge of domain expert, but inevitably leads to sub-optimal performance due to insufficient exploration. To address the above issues, this survey presents a unified view of automated graph neural network design, integrating graph structure learning and graph neural architecture search (GNAS) into a common pipeline for downstream tasks. Specifically, our work first systematically categorizes the graph structure learning paradigms from the perspective of task-agnostic heuristic based strategy and task-aware (end-to-end) learning based strategy. Then, GNAS steps are comprehensively synthesized from the following aspects: micro and macro search spaces, architecture embedding, search strategies, and performance evaluation methodologies. Their applicability and limitations are thoroughly analyzed. Moreover, the interaction paradigms between graph structure learning and GNAS architecture search are conceptually outlined. Finally, open issues in automated GNAS are summarized. This survey aims to provide a thoughtful insight for the automated representation learning of graph neural networks from both perspectives of data i.e., graph structure, and models i.e., automatic GNN architectures.

Accurately forecasting wind speed is crucial for efficiently utilizing the renewable energy, stabilizing the energy system and advancing the progress of the decarbonization of our society. However, due to its inherently temporal volatility and intermittency, accurate wind speed forecasting in a wind farm is challenging. Recently, Large Language Models (LLMs) have demonstrated notable performance in abundant natural language processing and computer vision tasks. However, the conventional LLMs fail to learn the complex spatial and temporal correlations of the wind speed data at multiple turbines in a wind farm, which makes wind speed forecasting cant fully benefit from the significant breakthroughs of LLM. To fill in this gap, we propose a novel spatial-temporal cross-attention LLM framework for wind speed forecasting, namely STCA-LLM, composed of alignment phase, and fine-tuning phase. In detail, our contributions are given as follows. First, the alignment phase aligns the general-purpose LLM with task-specific data, i.e., training the LLM with wind speed data. Second, in the fine-tuning phase, two representation learning modules, i.e., convolution network and graph neural network (GNN) are respectively used to extract temporal features of intra time series in each turbine, and the correlation of inter time-series at multiple turbines in a wind farm. Moreover, the cross-attention module is innovatively proposed to establish the connections between spatial and temporal embeddings. Then, the spatial-temporal representation modules and the aligned LLM are fine-tuned in two-stage way. Finally, thorough experiments on real wind speed dataset demonstrate that our proposed STCA-LLM outperforms state-of-the-art time series forecasting models including Transformer-based models, spatial-temporal GNN-based models, and pertained LLM-based models. Our code is available at: https://github.com/Justinzzcj/STCA-LLM

The wide deployment of advanced metering infrastructure (AMI) in power systems allows utility companies to automatically and accurately collect and process the timeseries load profiles of households, but meanwhile incurs the severe electricity theft (ET) that some illegal residential users may manipulate their electricity consumptions to reduce their billings. Although, due to the powerful ability of modelling long-range dependencies in sequential data, Transformer has been widely used for time series modelling including ET detection (ETD), the significant weakpoint lies in that it only considers the attention weights between either points or pre-patched subsequences (i.e., patches) of fixed size within the input sequence, which cant fully characterize the relationships among multi-scale temporal patches, and lead to suboptimal detection performance. To address the above issue, based on self-supervised feature extraction and supervised fine-tuning, our work proposes a novel multi-scale feature fusion Transformer encoder based ETD framework, MFFTD. Specifically, our works contributions are following. First, in MFFTD, a Hierarchical Patching enhanced Transformer encoder (HPTE) is explicitly designed, in which each layer patches the input sequence with variable patch size. Then, through hierarchically stacked multiple HPTE layers, the feature combining multi-scale patches can be effectively extracted. Second, considering the constraint that only a small percentage of labelled theft samples is practically available, our work first pre-trains the structure parameters of MFFTD through a self-supervised pretext task of forecasting the randomly masked segments in time-series. Then, the small percentage of labelled anomalous samples are used to fine-tune the MFFTD model. Extensive experiments on multiple real datasets demonstrate our proposed MFFTD scheme outperforms the state-of-the-art Transformer based supervised and semi-supervised ETD methods.

Recent years have witnessed the great achievement of the AI-driven intelligent games, such as AlphaStar defeating the human experts, and numerous intelligent games have come into the public view. Essentially, deep reinforcement learning (DRL), especially multiple-agent DRL (MADRL) has empowered a variety of artificial intelligence fields, including intelligent games. However, there is lack of systematical review on their correlations. This article provides a holistic picture on smoothly connecting intelligent games with MADRL from two perspectives: theoretical game concepts for MADRL, and MADRL for intelligent games. From the first perspective, information structure and game environmental features for MADRL algorithms are summarized; and from the second viewpoint, the challenges in intelligent games are investigated, and the existing MADRL solutions are correspondingly explored. Furthermore, the state-of-the-art (SOTA) MADRL algorithms for intelligent games are systematically categorized, especially from the perspective of credit assignment. Moreover, a comprehensively review on notorious benchmarks are conducted to facilitate the design and test of MADRL based intelligent games. Besides, a general procedure of MADRL simulations is offered. Finally, the key challenges in integrating intelligent games with MADRL, and potential future research directions are highlighted. This survey hopes to provide a thoughtful insight of developing intelligent games with the assistance of MADRL solutions and algorithms.

Electricity theft, emerging as one of the severe cyberattacks in smart grids, causes significant economic losses. Due to the powerful expressive ability of deep neural networks (DNN), supervised and unsupervised DNN-based electricity theft detection (ETD) schemes have experienced widespread deployment. However, existing works have the following weak points: Supervised DNN-based schemes require abundant labeled anomalous samples for training, and even worse, cannot detect unseen theft patterns. To avoid the extensively labor-consuming activity of labeling anomalous samples, unsupervised DNNs-based schemes aim to learn the normality of time-series and infer an anomaly score for each data instance, but they fail to capture periodic features effectively. To address these challenges, this paper proposes a novel periodicity-enhanced deep one-class classification framework (PE-DOCC) based on a periodicity-enhanced transformer encoder, named Periodicformer encoder. Specifically, within the encoder, a novel criss-cross periodic attention is proposed to capture both horizontal and vertical periodic features. The Periodicformer encoder is pre-trained by reconstructing partially masked input sequences, and the learned latent representations are then fed into a one-class classification for anomaly detection. Extensive experiments on real-world datasets demonstrate that our proposed PE-DOCC framework outperforms state-of-the-art unsupervised ETD methods.