Hechang Chen
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Hechang Chen has a total of 190 co-authors. Here are the top co-authors by number of shared publications:
- Yi Chang
Jilin University, China
19 shared publications - Zhejian Yang
School of Artificial Intelligence, Engineering Research Center of Knowledge-Driven Human-Machine Intelligence, MOE, Jilin University, Changchun, China
9 shared publications - Bo Yu
School of Artificial Intelligence and Engineering Research Center of Knowledge-Driven Human-Machine Intelligence, Jilin University, Changchun, China
8 shared publications Jifeng Hu
7 shared publications- Jifeng Hu
School of Artificial Intelligence, Jilin University, Changchun, China
7 shared publications - Lichao Sun
Lehigh University, Bethlehem, USA
7 shared publications
Latest Publications
- Open Access
- 26 October 2026
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Computing for Healthcare
Reinforcement Learning-guided Discriminative Region Sampling in Histopathology
With the widespread adoption of digital pathology, computational pathology has facilitated the transition from traditional manual interpretation to intelligent-assisted diagnosis through the analysis of whole-slide images (WSIs). Nevertheless, WSIs are gigapixel in size and contain sparsely distributed diagnostically relevant information, while large portions of tissue may lack diagnostic relevance or even confound the model's decision-making. In addition, the heterogeneity of tissue structures scatters diagnostic features, necessitating models that capture long-range dependencies and integrate contextual information effectively. In this paper, we propose a novel reinforcement learning-guided Discriminative Region Sampling Framework (DRSF) for weakly supervised histopathology WSI classification. Central to DRSF is the proposed discriminative region sampling strategy, in which a reinforcement learning (RL)-based policy network sequentially samples the patches deemed most diagnostically relevant, thereby reducing redundant or interfering information. The classification module utilizes a bidirectional Mamba backbone to effectively model long-range semantic dependencies among dispersed sampled patches, while accommodating variable-length input sequences. Experimental results on WSI datasets of different staining types demonstrate that DRSF outperforms representative multiple-instance learning (MIL) frameworks. Moreover, DRSF enables direct visualization of the intermediate results of selected patches and sampling trajectories, thereby assisting pathologists in rapidly locating key regions and enhancing the interpretability of automated diagnosis.
- 26 October 2026
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Computing for Healthcare
Open AccessReinforcement Learning-guided Discriminative Region Sampling in Histopathology
With the widespread adoption of digital pathology, computational pathology has facilitated the transition from traditional manual interpretation to intelligent-assisted diagnosis through the analysis of whole-slide images (WSIs). Nevertheless, WSIs are gigapixel in size and contain sparsely distributed diagnostically relevant information, while large portions of tissue may lack diagnostic relevance or even confound the model's decision-making. In addition, the heterogeneity of tissue structures scatters diagnostic features, necessitating models that capture long-range dependencies and integrate contextual information effectively. In this paper, we propose a novel reinforcement learning-guided Discriminative Region Sampling Framework (DRSF) for weakly supervised histopathology WSI classification. Central to DRSF is the proposed discriminative region sampling strategy, in which a reinforcement learning (RL)-based policy network sequentially samples the patches deemed most diagnostically relevant, thereby reducing redundant or interfering information. The classification module utilizes a bidirectional Mamba backbone to effectively model long-range semantic dependencies among dispersed sampled patches, while accommodating variable-length input sequences. Experimental results on WSI datasets of different staining types demonstrate that DRSF outperforms representative multiple-instance learning (MIL) frameworks. Moreover, DRSF enables direct visualization of the intermediate results of selected patches and sampling trajectories, thereby assisting pathologists in rapidly locating key regions and enhancing the interpretability of automated diagnosis.
- Open Access
scRep: A Latent-Space Self-Distilled Foundation Model for Single-Cell Representation Learning
Single-cell foundation models have shown strong potential for learning transferable representations from large-scale transcriptomic data. However, many existing approaches rely on reconstructing masked gene expression values, creating a potential mismatch between observation-space reconstruction and the goal of learning stable biological representations. This challenge is particularly relevant to single-cell RNA sequencing, where sparsity, incomplete gene detection, and technical variation can obscure the underlying biological state. Here, we introduce scRep, a compact latent-space self-distillation framework for single-cell representation learning. Rather than reconstructing raw expression values, scRep aligns differently perturbed views of the same cell through a momentum-updated teacher–student architecture, with self-distillation objectives at both the cell and gene levels. This representation-centered formulation encourages the model to capture biological information that remains stable across incomplete and perturbed transcriptomic observations. Using frozen representations without task-specific fine-tuning, scRep pretrained on approximately 2.8 million cells achieves the strongest overall performance across the evaluated frozen-representation benchmarks, demonstrating strong sample efficiency. A larger-scale scRep model pretrained on 30.72 million cells further demonstrates that the framework remains effective when scaled to a substantially larger and more diverse corpus. Beyond cell identity, scRep prioritizes established marker genes, recovers transcription factor–associated gene programs with cell-type-specific activity, and preserves continuous developmental structure that supports graph-based pseudotime inference. We further show that pretraining performance is closely associated with biological diversity: reducing redundant cells while improving cell-type coverage can match or exceed the performance of larger, less balanced training corpora. Together, these results establish latent-space self-distillation as an effective alternative to expression reconstruction for single-cell foundation modeling and suggest that efficient scaling depends not only on the number of cells, but also on the learning objective and the biological diversity of the pretraining corpus.
Open AccessscRep: A Latent-Space Self-Distilled Foundation Model for Single-Cell Representation Learning
Single-cell foundation models have shown strong potential for learning transferable representations from large-scale transcriptomic data. However, many existing approaches rely on reconstructing masked gene expression values, creating a potential mismatch between observation-space reconstruction and the goal of learning stable biological representations. This challenge is particularly relevant to single-cell RNA sequencing, where sparsity, incomplete gene detection, and technical variation can obscure the underlying biological state. Here, we introduce scRep, a compact latent-space self-distillation framework for single-cell representation learning. Rather than reconstructing raw expression values, scRep aligns differently perturbed views of the same cell through a momentum-updated teacher–student architecture, with self-distillation objectives at both the cell and gene levels. This representation-centered formulation encourages the model to capture biological information that remains stable across incomplete and perturbed transcriptomic observations. Using frozen representations without task-specific fine-tuning, scRep pretrained on approximately 2.8 million cells achieves the strongest overall performance across the evaluated frozen-representation benchmarks, demonstrating strong sample efficiency. A larger-scale scRep model pretrained on 30.72 million cells further demonstrates that the framework remains effective when scaled to a substantially larger and more diverse corpus. Beyond cell identity, scRep prioritizes established marker genes, recovers transcription factor–associated gene programs with cell-type-specific activity, and preserves continuous developmental structure that supports graph-based pseudotime inference. We further show that pretraining performance is closely associated with biological diversity: reducing redundant cells while improving cell-type coverage can match or exceed the performance of larger, less balanced training corpora. Together, these results establish latent-space self-distillation as an effective alternative to expression reconstruction for single-cell foundation modeling and suggest that efficient scaling depends not only on the number of cells, but also on the learning objective and the biological diversity of the pretraining corpus.
- Open Access
- 19 August 2026
- Published by Elsevier in SSRN Electronic Journal
Exposure-Debiased Sequential Recommendation via Preference Disentanglement
Sequential recommendation aims to learn temporal dependencies from users’ interaction histories to predict future items, capturing both evolving and stable preferences. However, existing mainstream methods typically train only on observed clicks and treat unexposed or unclicked items as uniform negatives, despite the fact that many items are actually shown on the platform but not selected, thereby creating substantial exposure bias that distorts preference learning. Moreover, user exposure histories are entangled with both long-term persistent preferences and short-term transient intents; without explicitly disentanglement and structured debiasing, recommendation performance will decline significantly. To tackle these issues, we propose DELINE, an exposure-debiased sequential recommendation framework based on explicit user preference disentanglement. Specifically, we model the complete user exposure sequence as an offline reinforcement learning trajectory, deploy a dynamic convolution to extract short-term transient preferences, and employ a causal self-attention to model the evolution of long-term persistent preferences, thereby achieving structured disentanglement of mixed interests. Additionally, we design an exposure-aware debiasing mechanism, which is applied separately to the short-term and long-term preference capture modules, capable of suppressing attention bias induced by frequently exposed popular items and preventing the amplification of such bias. Furthermore, we propose a long-term and short-term constrained autoregressive augmentation strategy that generates pseudo-interaction items aligned with user preferences, thereby alleviating data sparsity and enhancing model robustness. Extensive experiments on three public benchmark datasets demonstrate that DELINE achieves the state-of-the-art performance in both recommendation accuracy and debiasing effectiveness.
- 19 August 2026
- Published by Elsevier in SSRN Electronic Journal
Open AccessExposure-Debiased Sequential Recommendation via Preference Disentanglement
Sequential recommendation aims to learn temporal dependencies from users’ interaction histories to predict future items, capturing both evolving and stable preferences. However, existing mainstream methods typically train only on observed clicks and treat unexposed or unclicked items as uniform negatives, despite the fact that many items are actually shown on the platform but not selected, thereby creating substantial exposure bias that distorts preference learning. Moreover, user exposure histories are entangled with both long-term persistent preferences and short-term transient intents; without explicitly disentanglement and structured debiasing, recommendation performance will decline significantly. To tackle these issues, we propose DELINE, an exposure-debiased sequential recommendation framework based on explicit user preference disentanglement. Specifically, we model the complete user exposure sequence as an offline reinforcement learning trajectory, deploy a dynamic convolution to extract short-term transient preferences, and employ a causal self-attention to model the evolution of long-term persistent preferences, thereby achieving structured disentanglement of mixed interests. Additionally, we design an exposure-aware debiasing mechanism, which is applied separately to the short-term and long-term preference capture modules, capable of suppressing attention bias induced by frequently exposed popular items and preventing the amplification of such bias. Furthermore, we propose a long-term and short-term constrained autoregressive augmentation strategy that generates pseudo-interaction items aligned with user preferences, thereby alleviating data sparsity and enhancing model robustness. Extensive experiments on three public benchmark datasets demonstrate that DELINE achieves the state-of-the-art performance in both recommendation accuracy and debiasing effectiveness.
- Open Access
- 15 July 2026
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Computing for Healthcare
Clinical Raman Diagnosis for Healthcare Optimization Using Reinforcement Learning and Diffusion Models
High-dimensional medical data drive clinical decisions, and Raman spectroscopy is a key modality. However, real-world data often show nonstationary distributions, peak overlap, and batch effects. Fixed policies and conservative sampling increase the risk of overfitting. We therefore need a clinical framework that can reconstruct policies and enable safe exploration. Previous methods rely on full-spectrum modeling or static feature subsets, which do not support rapid policy switching during screening or stable detection of weak peaks. The core challenges are poor policy reconstruction across scenarios and limited exploration diversity. To this end, we propose the DecisionEC framework. Our idea uses structured decision making from reinforcement learning to lock onto clinically relevant structures and a diffusion prior to maintain high-quality global search. Specifically, we present Structure-Aware Option-Q (SAOQ), which abstracts stable peak clusters as reusable options and learns budget-aware high-level actions to meet sensitivity and specificity targets. Moreover, we introduce Diffusion-Guided Generational Competition (DGGC), which uses a conditional diffusion prior to reshape the candidate distribution and employs the Diffusion Slime Mould Algorithm (DSMA) to retrieve valuable peak-cluster combinations efficiently. Extensive experiments on a hospital-derived gastric cancer Raman spectroscopy classification task and multiple high-dimensional biomedical datasets validate the effectiveness of the proposed method.
- 15 July 2026
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Computing for Healthcare
Open AccessClinical Raman Diagnosis for Healthcare Optimization Using Reinforcement Learning and Diffusion Models
High-dimensional medical data drive clinical decisions, and Raman spectroscopy is a key modality. However, real-world data often show nonstationary distributions, peak overlap, and batch effects. Fixed policies and conservative sampling increase the risk of overfitting. We therefore need a clinical framework that can reconstruct policies and enable safe exploration. Previous methods rely on full-spectrum modeling or static feature subsets, which do not support rapid policy switching during screening or stable detection of weak peaks. The core challenges are poor policy reconstruction across scenarios and limited exploration diversity. To this end, we propose the DecisionEC framework. Our idea uses structured decision making from reinforcement learning to lock onto clinically relevant structures and a diffusion prior to maintain high-quality global search. Specifically, we present Structure-Aware Option-Q (SAOQ), which abstracts stable peak clusters as reusable options and learns budget-aware high-level actions to meet sensitivity and specificity targets. Moreover, we introduce Diffusion-Guided Generational Competition (DGGC), which uses a conditional diffusion prior to reshape the candidate distribution and employs the Diffusion Slime Mould Algorithm (DSMA) to retrieve valuable peak-cluster combinations efficiently. Extensive experiments on a hospital-derived gastric cancer Raman spectroscopy classification task and multiple high-dimensional biomedical datasets validate the effectiveness of the proposed method.
- Open Access
- 26 June 2026
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Computing for Healthcare
COA-Tuning: Collaborative Adapter Tuning with Prompt Enhancement for Class-Incremental Medical Image Learning
Continual learning plays a crucial role in enabling artificial intelligence systems to accumulate knowledge over time without overwriting previously acquired information. While this capability is essential for real-world applications, especially in clinical domains where data evolves continuously, most existing approaches are designed for natural image scenarios and struggle when applied to medical imaging tasks. Medical images exhibit high intra-class similarity, making fine-grained recognition difficult. Meanwhile, adapter-based tuning with pre-trained Vision Transformers introduces task-agnostic inference challenges due to the lack of task identity at test time. To overcome these obstacles, we propose a continual learning framework tailored for medical imaging, which enhances training and inference in class-incremental settings. First, we introduce a prompt-guided adapter design that focuses the model's representation capacity on subtle disease-specific differences, improving feature discrimination. Second, we encourage knowledge reuse by facilitating communication between task-specific adapters, allowing the model to build upon prior information. Finally, during testing, we employ a collaborative inference mechanism that dynamically integrates multiple adapters, enabling robust predictions without relying on task labels. Experiments on several public medical image datasets demonstrate that our approach achieves superior performance and substantially alleviates forgetting compared to existing methods.
- 26 June 2026
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Computing for Healthcare
Open AccessCOA-Tuning: Collaborative Adapter Tuning with Prompt Enhancement for Class-Incremental Medical Image Learning
Continual learning plays a crucial role in enabling artificial intelligence systems to accumulate knowledge over time without overwriting previously acquired information. While this capability is essential for real-world applications, especially in clinical domains where data evolves continuously, most existing approaches are designed for natural image scenarios and struggle when applied to medical imaging tasks. Medical images exhibit high intra-class similarity, making fine-grained recognition difficult. Meanwhile, adapter-based tuning with pre-trained Vision Transformers introduces task-agnostic inference challenges due to the lack of task identity at test time. To overcome these obstacles, we propose a continual learning framework tailored for medical imaging, which enhances training and inference in class-incremental settings. First, we introduce a prompt-guided adapter design that focuses the model's representation capacity on subtle disease-specific differences, improving feature discrimination. Second, we encourage knowledge reuse by facilitating communication between task-specific adapters, allowing the model to build upon prior information. Finally, during testing, we employ a collaborative inference mechanism that dynamically integrates multiple adapters, enabling robust predictions without relying on task labels. Experiments on several public medical image datasets demonstrate that our approach achieves superior performance and substantially alleviates forgetting compared to existing methods.
- Open Access
- 19 May 2026
- Published by Elsevier in SSRN Electronic Journal
A Poisson-based Scalable Stochastic Block Model for Large-scale Signed Networks
Signed network analysis has attracted considerable attention for its capacity to model cooperative and antagonistic relationships in complex systems such as social and biological networks. However, existing methods typically assume homogeneous degree distributions (e.g., Gaussian) and incur high computational costs, limiting their applicability to large-scale signed networks. To address these limitations, we propose a Scalable Signed Stochastic Block Model (S3BM) built upon a Poisson-based probabilistic framework. The Poisson formulation endows S3BM with flexibility to accommodate diverse degree distributions, including Gaussian and power-law types, while reducing computational complexity from $O(n^{2})$ to $O(m)$ by restricting likelihood evaluation to observed edges. Furthermore, we develop a block-wise learning algorithm integrating the Minimum Message Length (MML) criterion with Block-wise Expectation-Maximization (BEM), enabling simultaneous model selection and parameter estimation within a principled Bayesian framework. Extensive experiments on synthetic and real-world signed networks demonstrate S3BM's superiority in scalability, adaptability, and robustness. S3BM attains an average NMI of 0.982, outperforming state-of-the-art baselines, while achieving up to $63\times$ speedup on sparse networks and reducing runtime by 91\% over variational Bayesian methods on 40{,}000-node networks. These results confirm S3BM as a practical, theoretically grounded tool for efficient signed network modeling.- 19 May 2026
- Published by Elsevier in SSRN Electronic Journal
Open AccessA Poisson-based Scalable Stochastic Block Model for Large-scale Signed Networks
Signed network analysis has attracted considerable attention for its capacity to model cooperative and antagonistic relationships in complex systems such as social and biological networks. However, existing methods typically assume homogeneous degree distributions (e.g., Gaussian) and incur high computational costs, limiting their applicability to large-scale signed networks. To address these limitations, we propose a Scalable Signed Stochastic Block Model (S3BM) built upon a Poisson-based probabilistic framework. The Poisson formulation endows S3BM with flexibility to accommodate diverse degree distributions, including Gaussian and power-law types, while reducing computational complexity from $O(n^{2})$ to $O(m)$ by restricting likelihood evaluation to observed edges. Furthermore, we develop a block-wise learning algorithm integrating the Minimum Message Length (MML) criterion with Block-wise Expectation-Maximization (BEM), enabling simultaneous model selection and parameter estimation within a principled Bayesian framework. Extensive experiments on synthetic and real-world signed networks demonstrate S3BM's superiority in scalability, adaptability, and robustness. S3BM attains an average NMI of 0.982, outperforming state-of-the-art baselines, while achieving up to $63\times$ speedup on sparse networks and reducing runtime by 91\% over variational Bayesian methods on 40{,}000-node networks. These results confirm S3BM as a practical, theoretically grounded tool for efficient signed network modeling. - 4 May 2026
- Published by Elsevier in Computerized Medical Imaging and Graphics
- Vol. 132, 102773
Semantic token-guided hierarchical adversarial knowledge distillation for 3D abdominal organ segmentation
- 4 May 2026
- Published by Elsevier in Computerized Medical Imaging and Graphics
- Vol. 132, 102773
Semantic token-guided hierarchical adversarial knowledge distillation for 3D abdominal organ segmentation
- Open Access
- 12 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 4746-4757
TGSBM: Transformer-Guided Stochastic Block Model for Link Prediction
Link prediction is a cornerstone of the Web ecosystem, powering applications from recommendation and search to knowledge graph completion and collaboration forecasting. However, large-scale networks present unique challenges: they contain hundreds of thousands of nodes and edges with heterogeneous and overlapping community structures that evolve over time. Existing approaches face notable limitations: traditional graph neural networks struggle to capture global structural dependencies, while recent graph transformers achieve strong performance but incur quadratic complexity and lack interpretable latent structure. We propose TGSBM (Transformer-Guided Stochastic Block Model), a framework that integrates the principled generative structure of Overlapping Stochastic Block Models with the representational power of sparse Graph Transformers. TGSBM comprises three main components: (i) expander-augmented sparse attention that enables near-linear complexity and efficient global mixing, (ii) a neural variational encoder that infers structured posteriors over community memberships and strengths, and (iii) a neural edge decoder that reconstructs links via OSBM's generative process, preserving interpretability. Experiments across diverse benchmarks demonstrate competitive performance (mean rank 1.6 under HeaRT protocol), superior scalability (up to 6× faster training), and interpretable community structures. These results position TGSBM as a practical approach that strikes a balance between accuracy, efficiency, and transparency for large-scale link prediction.
- 12 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 4746-4757
Open AccessTGSBM: Transformer-Guided Stochastic Block Model for Link Prediction
Link prediction is a cornerstone of the Web ecosystem, powering applications from recommendation and search to knowledge graph completion and collaboration forecasting. However, large-scale networks present unique challenges: they contain hundreds of thousands of nodes and edges with heterogeneous and overlapping community structures that evolve over time. Existing approaches face notable limitations: traditional graph neural networks struggle to capture global structural dependencies, while recent graph transformers achieve strong performance but incur quadratic complexity and lack interpretable latent structure. We propose TGSBM (Transformer-Guided Stochastic Block Model), a framework that integrates the principled generative structure of Overlapping Stochastic Block Models with the representational power of sparse Graph Transformers. TGSBM comprises three main components: (i) expander-augmented sparse attention that enables near-linear complexity and efficient global mixing, (ii) a neural variational encoder that infers structured posteriors over community memberships and strengths, and (iii) a neural edge decoder that reconstructs links via OSBM's generative process, preserving interpretability. Experiments across diverse benchmarks demonstrate competitive performance (mean rank 1.6 under HeaRT protocol), superior scalability (up to 6× faster training), and interpretable community structures. These results position TGSBM as a practical approach that strikes a balance between accuracy, efficiency, and transparency for large-scale link prediction.
- 8 April 2026
- Published by Springer Nature
- p. 154-175
PQIns: Pipeline-Driven Application-Specific Instruction-Set Architecture for Hybrid Post-quantum Cryptography Acceleration
- 8 April 2026
- Published by Springer Nature
- p. 154-175
PQIns: Pipeline-Driven Application-Specific Instruction-Set Architecture for Hybrid Post-quantum Cryptography Acceleration
- 20 March 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. PP (2162237X) , 1-15
Learning Optimal Policies With Local Observations for Cooperative Multiagent Reinforcement Learning
The cooperative multiagent reinforcement learning (MARL) has been widely used in many practical applications. Despite its success, a fundamental issue arises in MARL that agents face the dilemma of whether to select the best action to maximize rewards or to acquire more information collectively by exploring the novel states/actions due to partial observability. To solve this issue, existing methods merge exploration and exploitation methods. However, these methods are always suboptimal and may lead to failure in finishing tasks. In this article, we theoretically prove the existence of a latent state that can guarantee the optimal individual and global policies. Moreover, we prove that such a latent state can be approximately obtained by local observations. Based on the analysis, we propose a method named unified MARL (UMARL), which is a weighted value function factorization approach unifying exploitation and exploration in one framework. Specifically, we design the agent representation network (ARN) and individual weighting networks (IWNs) to learn agents’ unified representations and weights of credit. Moreover, a latent state regularizer (LSR) is designed to encourage agents’ representations to approximate the latent state. Extensive experiments show that UMARL can achieve superior performance compared with 12 state-of-the-art methods on $m$ -step matrix game, level-based foraging (LBF), StarCraft II, and Google research football (GRF). The source code is available at: https://github.com/CrazyBayes/UMARL
- 20 March 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. PP (2162237X) , 1-15
Learning Optimal Policies With Local Observations for Cooperative Multiagent Reinforcement Learning
The cooperative multiagent reinforcement learning (MARL) has been widely used in many practical applications. Despite its success, a fundamental issue arises in MARL that agents face the dilemma of whether to select the best action to maximize rewards or to acquire more information collectively by exploring the novel states/actions due to partial observability. To solve this issue, existing methods merge exploration and exploitation methods. However, these methods are always suboptimal and may lead to failure in finishing tasks. In this article, we theoretically prove the existence of a latent state that can guarantee the optimal individual and global policies. Moreover, we prove that such a latent state can be approximately obtained by local observations. Based on the analysis, we propose a method named unified MARL (UMARL), which is a weighted value function factorization approach unifying exploitation and exploration in one framework. Specifically, we design the agent representation network (ARN) and individual weighting networks (IWNs) to learn agents’ unified representations and weights of credit. Moreover, a latent state regularizer (LSR) is designed to encourage agents’ representations to approximate the latent state. Extensive experiments show that UMARL can achieve superior performance compared with 12 state-of-the-art methods on $m$ -step matrix game, level-based foraging (LBF), StarCraft II, and Google research football (GRF). The source code is available at: https://github.com/CrazyBayes/UMARL