Zhenyu Wang

The dialogue policy is a fundamental component of pipeline dialogue systems, responsible for steering dialogue generation and ensuring compliance with established output constraints. Traditional approaches to dialogue policy learning typically involve sequential decision-making models based on deep reinforcement learning (RL). However, these deep RL-based dialogue policies often encounter challenges related to low sample efficiency, which can hinder their performance. Recent advancements in large language models (LLMs) have demonstrated exceptional capabilities across various domains, prompting an exploration of their potential to enhance dialogue policies. In this paper, we propose a novel framework called P3L, which synergistically integrates the fine-tuning of LLMs with on-policy RL methods to optimize dialogue policy learning. Our approach employs a robust feedback mechanism to collect and analyze interaction data, facilitating the fine-tuning of LLMs. Allowing the insights gained from LLMs to be incorporated as regularization terms within the RL-based agent’s learning process. This integration of LLM decision-making aims to address the limitations of traditional RL methods by leveraging the rich contextual understanding and generative capabilities of LLMs. The experimental results demonstrate that P3L significantly enhances the efficiency of dialogue policy learning, outperforming agents that rely solely on RL techniques.

Dialogue policy is a core module in pipeline dialogue systems as it drives conversation generation. Personalized dialogue policies aim to equip chatbots with tailored personalities, making them behave like real users, providing more accurate action responses, and improving the anthropomorphic capabilities of personal assistants. Yet existing dialogue policy approaches often overlook individual personalities because obtaining explicit user profiles is costly and time-consuming. In this article, we propose a personalized dialogue policy learning framework, named PDL. It dynamically learns implicit user profiles from successful dialogue trajectories. Specifically, we collect a lot of success histories from human–computer interactions to extract sequences of user belief states and agent actions. The extracted sequences are processed via a loop clipping operation and modeled with an autoregressive transformer to mimic human analytical behavior. After that, a new user’s latent personalized preferences are predicted based on the autoregressive transformer model. The personalized preferences are employed to implement dialogue policies via three categories of reinforcement learning algorithms, including value-based approaches, policy-based approaches, and model-based approaches. The experiments are conducted on three different task-oriented dialogue datasets, and the results show that the proposed PDL framework achieves state-of-the-art results compared to other comparative approaches.

In recent years, graph-based E-commerce Fraud detection methods have received more and more attention, but there are still some problems. Firstly, fraudulent users only account for a small part of active users, and the imbalance between classes in the graph leads to model deviation and affect the effectiveness of E-commerce Fraud detection. At the same time, the existing methods have ignored the sparse connections between fraudulent entities. In addition, most existing methods rely on the cost-sensitive matrix given by the expert experience and cannot adapt to learn the optimal cost transformation according to the data distribution. In this paper, we propose a cost sensitive graph neural network based on adaptive neighbor sampling, termed as CNSGNN. Firstly, the model designs a label-aware adaptive neighbor selector, using reinforcement learning to train a suitable sampling threshold, sampling neighbors by label similarity, thus alleviating the imbalance in the graph initially. Then, similarity scores are used to enhance the connection between fraudulent entities and help the sharing and transmission of characteristic information of fraudulent nodes. After that, GNN is used for information aggregation, and the cost-sensitive matrix optimizer is used to learn the optimal cost transformation to improve the classification effect. The experimental results show that the proposed model is superior to the current advanced baseline methods.

Recent Byzantine Fault-Tolerant (BFT) State Machine Replication (SMR) protocols increasingly focus on scalability and security to meet the growing demand for Distributed Ledger Technology (DLT) applications across various domains. Current BFT consensus algorithms typically require a single leader node to receive and validate votes from the majority process and broadcast the results, a design challenging to scale in large systems. We propose a fast-response consensus algorithm based on improvements to HotStuff, aimed at enhancing transaction ordering speed and overall performance of distributed systems, even in the presence of faulty copies. The algorithm introduces an optimistic response assumption, employs a message aggregation tree to collect and validate votes, and uses a dynamically adjusted threshold mechanism to reduce communication delay and improve message delivery reliability. Additionally, a dynamic channel mechanism and an asynchronous leader multi-round mechanism are introduced to address multiple points of failure in the message aggregation tree structure, minimizing dependence on a single leader. This adaptation can be flexibly applied to real-world system conditions to improve performance and responsiveness. We conduct experimental evaluations to verify the algorithm’s effectiveness and superiority. Compared to the traditional HotStuff algorithm, the improved algorithm demonstrates higher efficiency and faster response times in handling faulty copies and transaction ordering.

The rise of video content has significantly driven the growth of mobile data traffic, placing substantial pressure on core networks and adversely affecting user experience. To address these challenges, this paper proposes a Multi-Access Edge Computing (MEC) framework based on Deep Reinforcement Learning (DRL). By leveraging edge servers for video transmission, caching, and transcoding, this framework aims to reduce network load and latency. Our framework particularly considers the variability in video popularity and optimizes caching and transcoding strategies for different bitrate video streams. Using the Dueling Double DQN (D3QN) algorithm, it intelligently makes content caching decisions in scenarios with unknown content popularity, achieving optimal resource utilization. Experimental results demonstrate that the proposed framework enhances caching efficiency and user experience, exhibiting higher cache hit rates and lower caching costs in both the short and long term compared to traditional caching strategies and the Wolpertinger architecture.

Balancing video quality and latency in fluctuating network environments is a challenging task for live streaming. Existing Adaptive Bitrate (ABR) algorithms fail to accurately predict throughput, which impedes their ability to fully utilize bandwidth in dynamic network conditions. Additionally, most ABR algorithms inadequately consider the impact of latency on the Quality of Experience (QoE), leading to poor performance in live streaming scenarios. To address these issues, this paper proposes a new algorithm, DRALVS, which integrates the N-BEATS and deep reinforcement learning to jointly optimize video bitrate selection and playback speed control. The idea of DRALVS is to use the N-BEATS to accurately predict future available throughput and use this prediction as the state input for the next deep reinforcement learning module. This module then selects the appropriate video quality and playback speed based on the current state. Simulation results based on real network traces show that DRALVS outperforms existing rule-based and learning-based ABR algorithms in terms of average QoE and latency.