Chengxiang Yin

Offline reinforcement learning (RL), which operates solely on static datasets without further interactions with the environment, provides an appealing alternative to learning a safe and promising control policy. The prevailing methods typically learn a conservative policy to mitigate the problem of $Q$ -value overestimation, but it is prone to overdo it, leading to an overly conservative policy. Moreover, they optimize all samples equally with fixed constraints, lacking the nuanced ability to control conservative levels in a fine-grained manner. Consequently, this limitation results in a performance decline. To address the above two challenges in a united way, we propose a framework, adaptive conservative level in $Q$ -learning (ACL-QL), which limits the $Q$ -values in a mild range and enables adaptive control on the conservative level over each state–action pair, i.e., lifting the $Q$ -values more for good transitions and less for bad transitions. We theoretically analyze the conditions under which the conservative level of the learned $Q$ -function can be limited in a mild range and how to optimize each transition adaptively. Motivated by the theoretical analysis, we propose a novel algorithm, ACL-QL, which uses two learnable adaptive weight functions to control the conservative level over each transition. Subsequently, we design a monotonicity loss and surrogate losses to train the adaptive weight functions, $Q$ -function, and policy network alternatively. We evaluate ACL-QL on the commonly used datasets for deep data-driven reinforcement learning (D4RL) benchmark and conduct extensive ablation studies to illustrate the effectiveness and state-of-the-art performance compared with existing offline DRL baselines.

Deep generative models have made great progress in synthesizing images with arbitrary human poses and transferring the poses of one person to others. Though many different methods have been proposed to generate images with high visual fidelity, the main challenge remains and comes from two fundamental issues: pose ambiguity and appearance inconsistency. To alleviate the current limitations and improve the quality of the synthesized images, we propose a pose transfer network with augmented D isentangled F eature C onsistency (DFC-Net) to facilitate human pose transfer. Given a pair of images containing the source and target person, DFC-Net extracts pose and static information from the source and target respectively, then synthesizes an image of the target person with the desired pose from the source. Moreover, DFC-Net leverages disentangled feature consistency losses in the adversarial training to strengthen the transfer coherence and integrates a keypoint amplifier to enhance the pose feature extraction. With the help of the disentangled feature consistency losses, we further propose a novel data augmentation scheme that introduces unpaired support data with the augmented consistency constraints to improve the generality and robustness of DFC-Net. Extensive experimental results on Mixamo-Pose and EDN-10k have demonstrated DFC-Net achieves state-of-the-art performance on pose transfer.

Fine-grained instance segmentation is considerably more complicated and challenging than semantic segmentation. Most existing instance segmentation methods only focus on accuracy without paying much attention to inference latency, which, is critical to real-time applications, such as autonomous driving. In this paper, we aim to bridge the gap between semantic segmentation and instance segmentation by presenting a novel real-time model for instance segmentation, Sem2Ins, which effectively generates instance boundaries according to a semantic segmentation by leveraging conditional generative adversarial networks (cGANs) coupled with deep supervision and a weighted fusion layer. Specifically, supervision is imposed on each output layer, and features from different levels are fused to produce a well-generated boundary map. Sem2Ins has the following desirable features: 1) Combined with some fast semantic segmentation methods, Sem2Ins runs at a real-time speed that is fairly well-balanced against accuracy; 2) Sem2Ins works flexibly with any semantic segmentation model for instance segmentation, and if the given semantic segmentation is sufficiently good, Sem2Ins even achieves state-of-the-art in terms of accuracy; 3) deep supervision and weighted fusion can be leveraged to generate high-quality boundaries; and 4) Sem2Ins can be easily extended to panoptic segmentation. Extensive experiments performed on the Cityscapes, WildDash, KITTI and COCO benchmarks have demonstrated that 1) Sem2Ins, when combined with PSPNet and DDRNet-23-Slim, consistently outperforms the state-of-the-art real-time solution (Box2Pix) in terms of both speed and accuracy; and 2) Sem2Ins combined with DPC performs comparably to some powerful detect-and-segment approaches.

With the emergence of more and more powerful chipsets and hardware and the rise of Artificial Intelligence of Things (AIoT), there is a growing trend for bringing Deep Neural Network (DNN) models to empower mobile and edge devices with intelligence such that they can support attractive AI applications in a real-time manner. To leverage heterogeneous computational resources (such as CPU, GPU, DSP, etc.) to effectively and efficiently support the concurrent inference of multiple DNN models on a mobile or edge device, we propose a novel online Co-Scheduling framework based on deep REinforcement Learning, called COSREL. COSREL has the following desirable features: 1) it achieves significant speedup over commonly-used methods by efficiently utilizing all the computational resources on heterogeneous hardware; 2) it leverages emerging Deep Reinforcement Learning (DRL) to make dynamic and wise online scheduling decisions based on system runtime state; 3) it is capable of making a good tradeoff among inference latency, throughput, and energy efficiency; and 4) it makes no changes to given DNN models, thus preserves their accuracies. To evaluate COSREL, we conduct extensive experiments on an off-the-shelf Android smartphone. The experimental results show that COSREL consistently outperforms other baselines in terms of throughput, latency, and energy efficiency.

Cloud Radio Access Networks (CRANs) have become a key enabling technique for the next generation wireless communications. Resource allocation in CRANs still needs to be further improved to reach the objective of minimizing power consumption and meeting demands of wireless users over a long period. Inspired by the success of Deep Reinforcement Learning (DRL) on solving complicated control problems, we present a novel framework, ReCARL, for power-efficient resource allocation in CRANs with deep reinforcement learning. Specifically, we define the state space, action space and reward function for the DRL agent, apply a Deep Neural Network (DNN) to approximating the action-value function, and formally formulate the resource allocation problem (in each decision epoch) as a convex optimization problem. Under ReCARL, we propose two different DRL agents: one has a regular DNN structure trained with the basic deep Q-learning method (ReCARL-Basic); while the other has a context-aware DNN structure trained with a hybrid deep Q-learning method (ReCARL-Hybrid). We evaluated the performance of ReCARL along with the two DRL agents by comparing them with two widely-used baselines via extensive simulation. The simulation results show that ReCARL achieves significant power savings while meeting user demands, and it can well handle highly dynamic cases.

Meta-learning enables a model to learn from very limited data to undertake a new task. In this paper, we study the general meta-learning with adversarial samples. We present a meta-learning algorithm, ADML (ADversarial Meta-Learner), which leverages clean and adversarial samples to optimize the initialization of a learning model in an adversarial manner. ADML leads to the following desirable properties: 1) it turns out to be very effective even in the cases with only clean samples; 2) it is robust to adversarial samples, i.e., unlike other meta-learning algorithms, it only leads to a minor performance degradation when there are adversarial samples; 3) it sheds light on tackling the cases with limited and even contaminated samples. It has been shown by extensive experimental results that ADML consistently outperforms three representative meta-learning algorithms in the cases involving adversarial samples, on two widely-used image datasets, MiniImageNet and CIFAR100, in terms of both accuracy and robustness.

Video question answering (VideoQA) is a very important but challenging multimedia task, which automatically analyzes questions and videos and generates accurate answers. However, research on VideoQA is still in its infancy. In this article, we propose a novel memory augmented deep recurrent neural network (MA-DRNN) model for VideoQA, which features a new method for encoding videos and questions, and memory augmentation using the emerging differentiable neural computer (DNC). Specifically, we encode textual (questions) information before visual (videos) information, which leads to better visual-textual representations. Moreover, we leverage DNC (with an external memory) for storing and retrieving useful information in questions and videos, and modeling the long-term visual-textual dependence. To evaluate the proposed model, we conducted extensive experiments using the VTW data set and MSVD-QA data set, which are both Widely used large-scale video data sets for language-level understanding. The experimental results have well validated the proposed model and showed that it outperforms the state-of-the-art in terms of various accuracy-related metrics.