Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Amol Yerudkar has a total of 70 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • A Novel PID Design Method via Model-Based Reinforcement Learning Algorithms

    by Hozefa Jesawada, Amol Yerudkar, Yang Liu, Navdeep Singh, Carmen Del Vecchio

    This paper introduces a novel framework that bridges advanced reinforcement learning (RL) with traditional PID control by converting model-based RL policies into interpretable PID gains. By combining inverse reinforcement learning (IRL) with Kullback–Leibler divergence minimization, our method aligns sophisticated control strategies with the simplicity and robustness of PID controllers. In doing so, the proposed approach maintains the transparency and simplicity of PID controllers while incorporating the adaptability, data-driven optimization, and long-horizon planning capabilities of RL. Compatible with both model-based and model-free RL algorithms, the approach has been validated through extensive simulations on benchmark systems and real-world experiments on the Robotarium platform, demonstrating resilience against disturbances, parameter uncertainties, and noise. By blending the strengths of reinforcement learning with the practical familiarity of PID control, the proposed framework offers a data-efficient, scalable, and transparent solution for enhancing PID controller design in complex and dynamic environments.


  • Output Tracking of Periodically Time-Varying Boolean Networks: State-Flipped Control and Q -Learning Approaches

    by Xingyu Ge, Amol Yerudkar, Jianquan Lu, Bowen Li, Feng Ding, Jie Zhong

    This article investigates the output tracking problem for periodically time-varying Boolean networks (PTVBNs), motivated by rhythmic gene regulation and cyclic operating regimes in discrete systems. In such networks, the update rules change periodically, which makes tracking analysis and controller synthesis challenging, especially when only a subset of state components can be manipulated. A state-flipped control strategy is employed to address this challenge, enabling the modification of multiple nodes' states between binary values. Matrix-based representations are introduced to formalize both the output tracking problem and the state-flipped control mechanism. The article first develops an algebraic approach for analyzing output tracking in PTVBNs, establishing a comprehensive criterion for trackability. For synthesis with reduced model dependence, a model-free reinforcement learning formulation is further introduced. A two-level $Q$ -learning scheme is employed to identify the minimal state-flipping set required to successfully achieve output tracking. The effectiveness of the theoretical results is validated through extensive simulations on two biological systems: a repressilator model and a ten-node cell cycle network.


  • State estimation of stochastic temporal Boolean control networks

    by Lina Wang, Amol Yerudkar, Yang Liu, Jianquan Lu, Jinde Cao

  • Top Cited Papers

    Link Augmentation and Q-Learning for Set Stabilization in Switched Boolean Networks

    by Qinyao Pan, Jianquan Lu, Amol Yerudkar, Koichi Kobayashi, Jie Zhong

    In this article, set stabilization in switched Boolean networks is investigated through link augmentations. Link augmentation is introduced as a selective process for adding specific edges between nodes in the wiring digraph of the network. For the implementation of link augmentation, various variables are integrated into the dynamics governing networks, utilizing basic logical operators. A key aspect of this article is the incorporation of additional functions introduced by the link augmentations into the original dynamics of the system using constrained logical operators. Further, several criteria for set stabilization are formulated, and the link augmentation control strategy is innovatively designed, utilizing the Q-learning Algorithm. Finally, a biological example is presented to demonstrate the effectiveness of the proposed methods.


  • Pinning Stabilization of Logical Networks Based on Deformation of the State Transition Matrix

    by Jiayang Liu, Amol Yerudkar, Shuting Sun, Yang Liu

    Boolean Networks (BNs) are a powerful model to describe how gene work and holistically interact with one another in system biology. Based on the requirement to guide the systems to a desired state, the stabilization problem becomes an important issue in BNs. Since the normal state feedback control puts controllers to all nodes, the control cost is relatively high. In this paper, we investigate the stabilization problem of BNs under pinning control strategy. Meanwhile, to cut down the cost of control to the most, the set of pinned nodes is minimized. Specifically, a four-procedure method as well as the corresponding computationally feasible algorithms are proposed to determine a minimum set of controlled nodes based on a deformed state transition matrix and a constructed digraph. Compared with the strategy traditionally based on the state transition matrix, the approach we proposed effectively reduces the computational complexity. Finally, gene networks are discussed as simulations, which demonstrate the effectiveness of the proposed method, minimizing the number of controlled nodes with lower time complexity.


  • Pinning Control Design for Stabilizing Large-Scale Markovian Jump Boolean Networks

    by Lina Wang, Amol Yerudkar, Yang Liu, Mahmoud Abdel-Aty

    This article develops pinning control strategies to achieve global asymptotic stabilization in large-scale Markovian jump Boolean networks (MJBNs), closely linking their stability to network structural characteristics. First, the considered MJBN is divided into several sub-MJBNs, where signals switch according to irreducible transition probability matrices. Second, the stability of these sub-MJBNs is mapped onto that of deterministic periodic BNs. This transformation facilitates the formulation of a network-structure-based stability criterion for MJBNs by converting the inherently random stability characteristics of the networks into those of deterministic networks. Third, applying this criterion, static pinning control (SPC) is devised to stabilize MJBNs, where effective pinning nodes are identified by finding a feedback vertex set in the corresponding n-vertex digraph. Consequently, this method addresses the challenges associated with large-scale MJBNs effectively. To demonstrate the practical applicability of our theoretical findings, two biological examples are presented.


  • Stabilization of Boolean Networks: A Model-Free Constrained State-Flipped Control Approach

    by Zejiao Liu, Amol Yerudkar, Yang Liu, Jinde Cao

    In this paper, we explore the stabilization of Boolean Networks (BNs) under state-flipped control, incorporating three key constraints to address the challenges of disease treatment response times and the practical limitations of therapeutic interventions. Our approach includes: (i) a feasible flipped nodes constraint to ensure a subset of nodes can be realistically manipulated; (ii) a transient period constraint to align treatment with time-sensitive windows of therapeutic opportunity; and (iii) a constraint on the number of flip transitions to minimize resource consumption and potential side effects. Furthermore, an evolution selection rule is set for the BNs. Then, a necessary and sufficient condition is given for the stabilization under the above constraints. We present a modified Q-learning (QL) algorithm to design a policy for stabilization under constraints in the model-free case. Finally, BN models of metastatic melanoma network and T-cell large granular lymphocyte (T-LGL) network are considered to show the applicability of our results. Note to Practitioners—In this paper, we investigate the stabilization problem under three constraints for Boolean networks with state-flipped control. In gene regulatory networks, drug intervention usually only controls the activation and silencing of some or specific genes. Although the effect of drug intervention is significant in the short term, long-term application may cause cells to develop resistance. To solve these challenges, we consider the feasible flipped node constraint, transient period constraint and the number of flip transitions constraint. Current research on Boolean networks often overlooks various real-world constraints when considering state-flipped control, and our study addresses this oversight. We employ a model-free reinforcement learning approach to find the optimal state-flipped control policy under the given constraints, including timing and duration of drug use. The goal is to change the cell state from unstable to stable, which has important implications for improving the therapeutic effect of gene regulation. The solution proposed in this paper not only considers the dynamic characteristics of drug intervention, but also fully considers the biological feasibility and clinical application safety, which provides a new perspective and method for the state control of gene regulatory network.


  • Stabilization of Probabilistic Boolean Networks via State-Flipped Control and Reinforcement Learning

    by Yang Liu, Zejiao Liu, Amol Yerudkar, Carmen Del Vecchio

    In this paper, the state-flipped control technique is explored to investigate the stabilization of probabilistic Boolean networks (PBNs). Changing the values of many nodes from 0 to 1 (or from 1 to 0) is called the state-flipped control. The concepts of fixed point, reachable sets, and finite-time global stabilization of PBNs under state-flipped control are proposed. Several necessary and sufficient conditions for global stabilization are also derived based on the reachable sets of a given state. Furthermore, a model-free reinforcement learning (RL) algorithm, namely $Q$ -learning ( $Q$ L), is presented to design a flip sequence for any state that steers the state to a given destination state, thereby achieving finite-time global stabilization via state-flipped control. In addition, the process of finding the minimum flip set is proposed under the semi-tensor product and $Q$ L methods. Finally, the viability of the results in the paper is shown by considering a 12-gene hepatocellular cancer cell tumor network.