RJRahul Jain

Rahul Jain

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Rahul Jain has a total of 59 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Balance Equation-based Distributionally Robust Offline Imitation Learning

    by Rishabh Agrawal, Yusuf Alvi, Rahul Jain, Ashutosh Nayyar

    Imitation Learning (IL) has proven highly effective for robotic and control tasks where manually designing reward functions or explicit controllers is infeasible. However, standard IL methods implicitly assume that the environment dynamics remain fixed between training and deployment. In practice, this assumption rarely holds where modeling inaccuracies, real-world parameter variations, and adversarial perturbations can all induce shifts in transition dynamics, leading to severe performance degradation. We address this challenge through Balance Equation-based Distributionally Robust Offline Imitation Learning, a framework that learns robust policies solely from expert demonstrations collected under nominal dynamics, without requiring further environment interaction. We formulate the problem as a distributionally robust optimization over an uncertainty set of transition models, seeking a policy that minimizes the imitation loss under the worst-case transition distribution. Importantly, we show that this robust objective can be reformulated entirely in terms of the nominal data distribution, enabling tractable offline learning. Empirical evaluations on continuous-control benchmarks demonstrate that our approach achieves superior robustness and generalization compared to state-of-the-art offline IL baselines, particularly under perturbed or shifted environments.

  • Open Access

    Generalized Quantal Response Equilibrium: Existence and Efficient Learning

    by Apurv Shukla, Vijay Subramanian, Andy Zhao, Rahul Jain

    We introduce a new solution concept for bounded rational agents in finite normal-form general-sum games called Generalized Quantal Response Equilibrium (GQRE) which generalizes Quantal Response Equilibrium~\citep{mckelvey1995quantal}. In our setup, each player maximizes a smooth, regularized expected utility of the mixed profiles used, reflecting bounded rationality that subsumes stochastic choice. After establishing existence under mild conditions, we present computationally efficient no-regret independent learning via smoothened versions of the Frank-Wolfe algorithm. Our algorithm uses noisy but correlated gradient estimates generated via a simulation oracle that reports on repeated plays of the game. We analyze convergence properties of our algorithm under assumptions that ensure uniqueness of equilibrium, using a class of gap functions that generalize the Nash gap. We end by demonstrating the effectiveness of our method on a set of complex general-sum games such as high-rank two-player games, large action two-player games, and known examples of difficult multi-player games.

  • Open Access

    Markov Balance Satisfaction Improves Performance in Strictly Batch Offline Imitation Learning

    by Rishabh Agrawal, Nathan Dahlin, Rahul Jain, Ashutosh Nayyar

    Imitation learning (IL) is notably effective for robotic tasks where directly programming behaviors or defining optimal control costs is challenging. In this work, we address a scenario where the imitator relies solely on observed behavior and cannot make environmental interactions during learning. It does not have additional supplementary datasets beyond the expert's dataset nor any information about the transition dynamics. Unlike state-of-the-art (SOTA) IL methods, this approach tackles the limitations of conventional IL by operating in a more constrained and realistic setting. Our method uses the Markov balance equation and introduces a novel conditional density estimation-based imitation learning framework. It employs conditional normalizing flows for transition dynamics estimation and aims at satisfying a balance equation for the environment. Through a series of numerical experiments on Classic Control and MuJoCo environments, we demonstrate consistently superior empirical performance compared to many SOTA IL algorithms.

  • Open Access

    Markov Balance Satisfaction Improves Performance in Strictly Batch Offline Imitation Learning

    by Rishabh Agrawal, Nathan Dahlin, Rahul Jain, Ashutosh Nayyar

    Imitation learning (IL) is notably effective for robotic tasks where directly programming behaviors or defining optimal control costs is challenging. In this work, we address a scenario where the imitator relies solely on observed behavior and cannot make environmental interactions during learning. It does not have additional supplementary datasets beyond the expert's dataset nor any information about the transition dynamics. Unlike state-of-the-art (SOTA) IL methods, this approach tackles the limitations of conventional IL by operating in a more constrained and realistic setting. Our method uses the Markov balance equation and introduces a novel conditional density estimation-based imitation learning framework. It employs conditional normalizing flows for transition dynamics estimation and aims at satisfying a balance equation for the environment. Through a series of numerical experiments on Classic Control and MuJoCo environments, we demonstrate consistently superior empirical performance compared to many SOTA IL algorithms.

  • Optimal Control of Logically Constrained Partially Observable and Multiagent Markov Decision Processes

    by Krishna C. Kalagarla, Dhruva Kartik, Dongming Shen, Rahul Jain, Ashutosh Nayyar, Pierluigi Nuzzo

    Autonomous systems often have logical constraints arising, for example, from safety, operational, or regulatory requirements. Such constraints can be expressed using temporal logic specifications. The system state is often partially observable. Moreover, it could encompass a team of multiple agents with a common objective but disparate information structures and constraints. In this paper, we first introduce an optimal control theory for partially observable Markov decision processes (POMDPs) with finite linear temporal logic ( $\boldsymbol{LTL}_{\bf f}$ ) constraints. We provide a structured methodology for synthesizing policies that maximize a cumulative reward while ensuring that the probability of satisfying a temporal logic constraint is sufficiently high. Our approach comes with guarantees on approximate reward optimality and constraint satisfaction. We then build on this approach to design an optimal control framework for logically constrained multi-agent settings with information asymmetry. We illustrate the effectiveness of our approach by implementing it on several case studies.


  • Optimal Communication and Control Strategies in a Cooperative Multiagent MDP Problem

    by Sagar Sudhakara, Dhruva Kartik, Rahul Jain, Ashutosh Nayyar

    The problem of controlling cooperative multi-agent systems under different models of information sharing among agents has received significant attention in the recent literature. In this paper, we consider a setup where rather than committing to a fixed and non-adaptive information sharing protocol (e.g. periodic sharing or no sharing etc), agents can dynamically decide at each time step whether to share information with each other and incur the resulting communication cost. This setup requires a joint design of agents' communication and control strategies in order to optimize the trade-off between communication costs and the control objective. We first show that agents can ignore a big part of their private information without compromising the system performance. We then provide a common information approach based solution for the strategy optimization problem. This approach relies on constructing a fictitious POMDP whose solution (obtained via a dynamic program) characterizes the optimal strategies for the agents. We extend our solution to incorporate time-varying packet-drop channels and constraints on when and how frequently agents can communicate.


  • Probabilistic Contraction Analysis of Iterated Random Operators

    by Abhishek Gupta, Rahul Jain, Peter Glynn

    In many branches of engineering, Banach contraction mapping theorem is employed to establish the convergence of certain deterministic algorithms. Randomized versions of these algorithms have been developed that have proved useful in data-driven problems. In a class of randomized algorithms, in each iteration, the contraction map is approximated with an operator that uses independent and identically distributed samples of certain random variables. This leads to iterated random operators acting on an initial point in a complete metric space, and it generates a Markov chain. In this article, we develop a new stochastic dominance-based proof technique, called probabilistic contraction analysis, for establishing the convergence in probability of Markov chains generated by such iterated random operators in certain limiting regime. The methods developed in this article provides a general framework for understanding convergence of a wide variety of Monte Carlo methods, in which contractive property is present. We apply the convergence result to conclude the convergence of fitted value iteration and fitted relative value iteration in continuous state and continuous action MDPs as representative applications of the general framework developed here.


  • Open Access

    Probabilistic Contraction Analysis of Iterated Random Operators

    by Abhishek Gupta, Rahul Jain, Peter Glynn

    In many branches of engineering, Banach contraction mapping theorem is employed to establish the convergence of certain deterministic algorithms. Randomized versions of these algorithms have been developed that have proved useful in data-driven problems. In a class of randomized algorithms, in each iteration, the contraction map is approximated with an operator that uses independent and identically distributed samples of certain random variables. This leads to iterated random operators acting on an initial point in a complete metric space, and it generates a Markov chain. In this paper, we develop a new stochastic dominance based proof technique, called probabilistic contraction analysis, for establishing the convergence in probability of Markov chains generated by such iterated random operators in certain limiting regime. The methods developed in this paper provides a general framework for understanding convergence of a wide variety of Monte Carlo methods in which contractive property is present. We apply the convergence result to conclude the convergence of fitted value iteration and fitted relative value iteration in continuous state and continuous action Markov decision problems as representative applications of the general framework developed here.

  • Open Access

    Safe Posterior Sampling for Constrained MDPs with Bounded Constraint Violation

    by Krishna C Kalagarla, Rahul Jain, Pierluigi Nuzzo

    Constrained Markov decision processes (CMDPs) model scenarios of sequential decision making with multiple objectives that are increasingly important in many applications. However, the model is often unknown and must be learned online while still ensuring the constraint is met, or at least the violation is bounded with time. Some recent papers have made progress on this very challenging problem but either need unsatisfactory assumptions such as knowledge of a safe policy, or have high cumulative regret. We propose the Safe PSRL (posterior sampling-based RL) algorithm that does not need such assumptions and yet performs very well, both in terms of theoretical regret bounds as well as empirically. The algorithm achieves an efficient tradeoff between exploration and exploitation by use of the posterior sampling principle, and provably suffers only bounded constraint violation by leveraging the idea of pessimism. Our approach is based on a primal-dual approach. We establish a sub-linear $\tilde{\mathcal{ O}}\left(H^{2.5} \sqrt{|\mathcal{S}|^2 |\mathcal{A}| K} \right)$ upper bound on the Bayesian reward objective regret along with a bounded, i.e., $\tilde{\mathcal{O}}\left(1\right)$ constraint violation regret over $K$ episodes for an $|\mathcal{S}|$-state, $|\mathcal{A}|$-action and horizon $H$ CMDP.

  • Two-Stage Electricity Markets With Renewable Energy Integration: Market Mechanisms and Equilibrium Analysis

    by Nathan Dahlin, Rahul Jain

    In this article, we consider a two-stage market mechanism for trading electricity including renewable generation as an alternative to the widely used multisettlement market structure. The two-stage market structure allows for recourse decisions by the market operator, which are not possible in today’s markets. We allow for different conventional generation costs in the forward and the real-time stages. We have considered costs of demand response programs and black outs, and adopt a DC power flow model to account for network constraints. Our first result is to show existence (by construction) of a sequential competitive equilibrium (SCEq) in such a two-stage market. We argue social welfare properties of such an SCEq, and then design a market mechanism that maximizes social welfare when the market participants are nonstrategic. We also show that under either a congestion-free or a monopoly free condition, an efficient Nash equilibrium exists.