PGPiyush Gupta

Piyush Gupta

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Piyush Gupta has a total of 7 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Optimal Fidelity Selection for Human-Supervised Search

    by Piyush Gupta, Vaibhav Srivastava

    We study optimal fidelity selection in human-supervised underwater visual search, where operator performance is influenced by cognitive factors such as workload. In our experiments, participants perform two simultaneous tasks: detecting underwater mines in videos (primary) and responding to a visual cue to estimate workload (secondary). Videos arrive as a Poisson process and queue for review, with the operator choosing between normal fidelity (faster playback) and high fidelity. Rewards depend on detection accuracy, while penalties are tied to queue length. Workload is modeled as a hidden state using an Input-Output Hidden Markov Model, and fidelity selection is optimized via a Partially Observable Markov Decision Process. We evaluate two setups: fidelity-only selection and a version that also allows task delegation to automation for queue stability. Our approach improves performance by 26.5% without delegation and 50.3% with delegation, compared to a baseline where humans manually select fidelity.

  • Open Access

    Fostering human learning in sequential decision-making: Understanding the role of evaluative feedback

    by Piyush Gupta, Subir Biswas, Vaibhav Srivastava

    Cognitive rehabilitation, STEM (science, technology, engineering, and math) skill acquisition, and coaching games such as chess often require tutoring decision-making strategies. The advancement of AI-driven tutoring systems for facilitating human learning requires an understanding of the impact of evaluative feedback on human decision-making and skill development. To this end, we conduct human experiments using Amazon Mechanical Turk to study the influence of evaluative feedback on human decision-making in sequential tasks. In these experiments, participants solve the Tower of Hanoi puzzle and receive AI-generated feedback while solving it. We examine how this feedback affects their learning and skill transfer to related tasks. Additionally, treating humans as noisy optimal agents, we employ maximum entropy inverse reinforcement learning to analyze the effect of feedback on the implicit human reward structure that guides their decision making. Lastly, we explore various computational models to understand how people incorporate evaluative feedback into their decision-making processes. Our findings underscore that humans perceive evaluative feedback as indicative of their long-term strategic success, thus aiding in skill acquisition and transfer in sequential decision-making tasks. Moreover, we demonstrate that evaluative feedback fosters a more structured and organized learning experience compared to learning without feedback. Furthermore, our results indicate that providing intermediate goals alone does not significantly enhance human learning outcomes.


  • Open Access

    Deterministic Sequencing of Exploration and Exploitation for Reinforcement Learning

    by Piyush Gupta, Vaibhav Srivastava

    We propose Deterministic Sequencing of Exploration and Exploitation (DSEE) algorithm with interleaving exploration and exploitation epochs for model-based RL problems that aim to simultaneously learn the system model, i.e., a Markov decision process (MDP), and the associated optimal policy. During exploration, DSEE explores the environment and updates the estimates for expected reward and transition probabilities. During exploitation, the latest estimates of the expected reward and transition probabilities are used to obtain a robust policy with high probability. We design the lengths of the exploration and exploitation epochs such that the cumulative regret grows as a sub-linear function of time.

  • Towards Physically Adversarial Intelligent Networks (PAINs) for Safer Self-Driving

    by Piyush Gupta, Demetris Coleman, Joshua E. Siegel

    Neural networks in autonomous vehicles suffer from overfitting, poor generalizability, and untrained edge cases due to limited data availability. Researchers often synthesize randomized edge-case scenarios to assist in the training process, though simulation introduces the potential for overfitting to latent rules and features. Automating worst-case scenario generation could yield informative data for improving self-driving. To this end, we present a “Physically Adversarial Intelligent Network", wherein self-driving vehicles interact aggressively in the CARLA simulation. We train two agents, a protagonist, and an adversary, using dueling double deep Q networks with prioritized experience replay. The coupled networks alternately seek to collide and avoid collisions such that the “defensive” avoidance algorithm increases the mean time to failure and distance traveled under non-hostile operating conditions. The trained protagonist becomes more resilient to environmental uncertainty and less prone to corner case failures resulting in collisions than the agent trained without an adversary.


  • Incentivizing Collaboration in Heterogeneous Teams via Common-Pool Resource Games

    by Piyush Gupta, Shaunak D. Bopardikar, Vaibhav Srivastava

    We consider a team of heterogeneous agents that is collectively responsible for servicing, and subsequently reviewing, a stream of homogeneous tasks. Each agent has an associated mean service time and a mean review time for servicing and reviewing the tasks, respectively. Agents receive a reward based on their service and review admission rates. The team objective is to collaboratively maximize the number of "serviced and reviewed" tasks. We formulate a Common-Pool Resource (CPR) game and design utility functions to incentivize collaboration among heterogeneous agents in a decentralized manner. We show the existence of a unique Pure Nash Equilibrium (PNE), and establish convergence of best response dynamics to this unique PNE. Finally, we establish an analytic upper bound on three inefficiency measures of the PNE, namely the price of anarchy (PoA), the ratio of the total review admission rate (TRI), and the ratio of latency (LI).


  • Open Access

    Incentivizing Collaboration in Heterogeneous Teams via Common-Pool Resource Games

    by Piyush Gupta, Shaunak D. Bopardikar, Vaibhav Srivastava

    We consider a team of heterogeneous agents that is collectively responsible for servicing, and subsequently reviewing, a stream of homogeneous tasks. Each agent has an associated mean service time and a mean review time for servicing and reviewing the tasks, respectively. Agents receive a reward based on their service and review admission rates. The team objective is to collaboratively maximize the number of "serviced and reviewed" tasks. We formulate a Common-Pool Resource (CPR) game and design utility functions to incentivize collaboration among heterogeneous agents in a decentralized manner. We show the existence of a unique Pure Nash Equilibrium (PNE), and establish convergence of best response dynamics to this unique PNE. Finally, we establish an analytic upper bound on three measures of inefficiency of the PNE, namely the price of anarchy, the ratio of the total review admission rate, and the ratio of latency, along with an empirical study.

  • Open Access

    Towards Safer Self-Driving Through Great PAIN (Physically Adversarial Intelligent Networks)

    by Piyush Gupta, Demetris Coleman, Joshua E. Siegel

    Automated vehicles' neural networks suffer from overfit, poor generalizability, and untrained edge cases due to limited data availability. Researchers synthesize randomized edge-case scenarios to assist in the training process, though simulation introduces potential for overfit to latent rules and features. Automating worst-case scenario generation could yield informative data for improving self driving. To this end, we introduce a "Physically Adversarial Intelligent Network" (PAIN), wherein self-driving vehicles interact aggressively in the CARLA simulation environment. We train two agents, a protagonist and an adversary, using dueling double deep Q networks (DDDQNs) with prioritized experience replay. The coupled networks alternately seek-to-collide and to avoid collisions such that the "defensive" avoidance algorithm increases the mean-time-to-failure and distance traveled under non-hostile operating conditions. The trained protagonist becomes more resilient to environmental uncertainty and less prone to corner case failures resulting in collisions than the agent trained without an adversary.

  • Open Access

    Distributed resource allocation for proportional fairness in multi-band wireless systems

    by I-Hong Hou, Piyush Gupta

    A challenging problem in multi-band multi-cell self-organized wireless systems, such as multi-channel Wi-Fi networks, femto/pico cells in 3G/4G cellular networks, and more recent wireless networks over TV white spaces, is of distributed resource allocation. This involves four components: channel selection, client association, channel access, and client scheduling. In this paper, we present a unified framework for jointly addressing the four components with the global system objective of maximizing the clients throughput in a proportionally fair manner. Our formulation allows a natural dissociation of the problem into two sub-parts. We show that the first part, involving channel access and client scheduling, is convex and derive a distributed adaptation procedure for achieving Pareto-optimal solution. For the second part, involving channel selection and client association, we develop a Gibbs-sampler based approach for local adaptation to achieve the global objective, as well as derive fast greedy algorithms from it that achieve good solutions.