ZUZakir Ullah
Zakir Ullah
*This is an algorithmically generated author record based on publicly available information.
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Zakir Ullah has a total of 10 co-authors. Here are the top co-authors by number of shared publications:
- Gengzhong Zheng
Department of Computer Science and Engineering, Hanshan Normal University, Chaozhou, China
1 shared publication - Liming Miao
Hanshan Normal University, Chaozhou 521041, China
1 shared publication - Mahammad Humayoo
Hanshan Normal University, Chaozhou 521041, China
1 shared publication - Naveed Ur Rehman Junejo
Hanshan Normal University, Chaozhou 521041, China
1 shared publication - Peitao Wang
Hanshan Normal University, Chaozhou 521041, China
1 shared publication - Shuwei Qiu
Hanshan Normal University, Chaozhou 521041, China
1 shared publication
Latest Publications
- Open Access
- 20 November 2025
- Published by MDPI AG in Algorithms
- Vol. 18 (11) , 729
Segmenting Action-Value Functions over Time Scales in SARSA via TD(Δ)
In numerous episodic reinforcement learning (RL) environments, SARSA-based methodologies are employed to enhance policies aimed at maximizing returns over long horizons. Traditional SARSA algorithms face challenges in achieving an optimal balance between bias and variation, primarily due to their dependence on a single, constant discount factor (). This study enhances the temporal difference decomposition method, TD(), by applying it to the SARSA algorithm, wherein the action-value function is segmented into several components based on the differences between action-value functions linked to specific discount factors. Each component, referred to as a delta estimator (D), is linked to a specific discount factor and learned independently. This modified technique is referred to as SARSA(). SARSA is a widely used on-policy RL method that enhances action-value functions via temporal difference updates. This decomposition, namely SARSA(), facilitates learning across a range of time scales. This analysis makes learning more effective and guarantees consistency, especially in situations where long-horizon improvement is needed. The results of this research show that the proposed technique works to lower bias in SARSA’s updates and speed up convergence in both deterministic and stochastic settings, even in dense-reward Atari environments. Experimental results from a variety of benchmark settings show that the proposed SARSA() outperforms existing TD learning techniques in both tabular and deep RL environments.- 20 November 2025
- Published by MDPI AG in Algorithms
- Vol. 18 (11) , 729
Open AccessSegmenting Action-Value Functions over Time Scales in SARSA via TD(Δ)
In numerous episodic reinforcement learning (RL) environments, SARSA-based methodologies are employed to enhance policies aimed at maximizing returns over long horizons. Traditional SARSA algorithms face challenges in achieving an optimal balance between bias and variation, primarily due to their dependence on a single, constant discount factor (). This study enhances the temporal difference decomposition method, TD(), by applying it to the SARSA algorithm, wherein the action-value function is segmented into several components based on the differences between action-value functions linked to specific discount factors. Each component, referred to as a delta estimator (D), is linked to a specific discount factor and learned independently. This modified technique is referred to as SARSA(). SARSA is a widely used on-policy RL method that enhances action-value functions via temporal difference updates. This decomposition, namely SARSA(), facilitates learning across a range of time scales. This analysis makes learning more effective and guarantees consistency, especially in situations where long-horizon improvement is needed. The results of this research show that the proposed technique works to lower bias in SARSA’s updates and speed up convergence in both deterministic and stochastic settings, even in dense-reward Atari environments. Experimental results from a variety of benchmark settings show that the proposed SARSA() outperforms existing TD learning techniques in both tabular and deep RL environments.