Xing Chen

In real-world decision-making tasks, it is critical for reinforcement learning (RL) methods to be both stable and robust. Maximum entropy RL methods typically generate a robust policy with entropy augmented reward. While incorporating entropy into the reward offers the benefit of exploration, it presents limited universal applicability and persistent convergence difficulties, such as suboptimal policy stabilization and unstable $Q$ value update. From optimization, we define these two issues as tremulous policy and spiky Q-function, investigating their underlying causes and relationships. Analysis with this, the maximum entropy principle leads to a spiky $Q$ -function update, which ultimately results in a tremulous policy. We thus introduce a beta-symmetric Kullback–Leibler (KL) divergence objective to mitigate such issues under the maximum entropy framework. With this objective function, the tremulous nature of the policy could be controlled with a large beta value. The spiky $Q$ -function could be avoided by annealing the entropy in the target $Q$ value, as the beta-symmetric KL divergence is an upper bound of the original reverse KL divergence. Theoretically, we prove that minimizing our new objective function results in a new policy that presents an improvement in the $Q$ value. Guaranteed by these results, we ultimately derive the optimal policy by iteratively updating the $Q$ value and policy, and we call this method max-entropy stable optimization (MeSO). Experimental results on the Mujoco and Roboschool platforms demonstrate that our algorithm maintains stability while offering better flexibility and overall performance.

Extracting robust features against geometric transformations and adversarial perturbations remains a critical challenge in deep learning. Although capsule networks exhibit resilience through vector-encapsulated features and dynamic routing, they suffer from computational inefficiency due to iterative routing, dense matrix operations, and extra activation scalars. To address these limitations, we propose a method that integrates (1) compact vector-grouped neurons to eliminate activation scalars, (2) a non-iterative voting algorithm that preserves spatial relationships with reduced computation, and (3) efficient weight-sharing strategies that balance computational efficiency with generalizability. Our approach outperforms existing methods in image classification on CIFAR-10 and SVHN, achieving up to a 0.31% increase in accuracy with fewer parameters and lower FLOPs. Evaluations demonstrate superior performance over competing methods, with 0.31% accuracy gains on CIFAR-10/SVHN (with reduced parameters and FLOPs) and 1.93%/1.09% improvements in novel-view recognition on smallNORB. Under FGSM and BIM attacks, our method reduces attack success rates by 47.7% on CIFAR-10 and 32.4% on SVHN, confirming its enhanced robustness and efficiency. Future work will extend vexel representations to MLPs and RNNs and explore applications in computer graphics, natural language processing, and reinforcement learning.

The popular Proximal Policy Optimization (PPO) algorithm approximates the solution in a clipped policy space. Does there exist better policies outside of this space? By using a novel surrogate objective that employs the sigmoid function (which provides an interesting way of exploration), we found that the answer is "YES", and the better policies are in fact located very far from the clipped space. We show that PPO is insufficient in "off-policyness", according to an off-policy metric called DEON. Our algorithm explores in a much larger policy space than PPO, and it maximizes the Conservative Policy Iteration (CPI) objective better than PPO during training. To the best of our knowledge, all current PPO methods have the clipping operation and optimize in the clipped policy space. Our method is the first of this kind, which advances the understanding of CPI optimization and policy gradient methods. Code is available at https://github.com/raincchio/P3O.

The popular Proximal Policy Optimization (PPO) algorithm approximates the solution in a clipped policy space. Does there exist better policies outside of this space? By using a novel surrogate objective that employs the sigmoid function (which provides an interesting way of exploration), we found that the answer is ``YES'', and the better policies are in fact located very far from the clipped space. We show that PPO is insufficient in ``off-policyness'', according to an off-policy metric called DEON. Our algorithm explores in a much larger policy space than PPO, and it maximizes the Conservative Policy Iteration (CPI) objective better than PPO during training. To the best of our knowledge, all current PPO methods have the clipping operation and optimize in the clipped policy space. Our method is the first of this kind, which advances the understanding of CPI optimization and policy gradient methods. Code is available at https://github.com/raincchio/P3O.

Medical multiobjective image segmentation aims to group pixels to form multiple regions based on the different properties of the medical images. Segmenting the 3D cardiovascular magnetic resonance (CMR) images is still a challenging task owing to several reasons, including individual differences in heart shapes, varying signal intensities, and differences in data signal-to-noise ratios. This paper proposes a novel and efficient U-Net-based 3D sparse convolutional network named SparseVoxNet. In this network, there are direct connections between any two layers with the same feature-map size, and the number of connections is reduced. Therefore, the SparseVoxNet can effectively cope with the optimization problem of gradients vanishing when training a 3D deep neural network model on small sample data by significantly decreasing the network depth, and achieveing better feature representation using a spatial self-attention mechanism finally. The proposed method in this paper has been thoroughly evaluated on the HVSMR 2016 dataset. Compared with other methods, the method achieves better performance.

Randomness is a very important resource for cryptography, algorithms, and scientific simulations. Since all classical processes are considered to be intrinsically deterministic, we must build quantum random number generators which utilize quantum processes to generate true randomness. Quantum random number generators have been realized in different quantum systems, including quantum optical systems, and trapped ions. Here we present a proof-of-concept random number generator based on a nuclear spin system for the first time. The state preparation and measurements are performed with high-fidelity operations in our system. The entropy of randomness in the experimental data is quantified by two dimension witness certification protocols, which require no detailed models to describe the experimental devices but only some general assumptions, such as the limited dimensionality and the independence of the experimental devices.