Sushmita Bhattacharya
Research Subjects
Sushmita Bhattacharya has a total of 14 co-authors. Here are the top co-authors by number of shared publications:
- Dimitri P. Bertsekas
M.I.T.
2 shared publications - Sahil Badyal
REACT Lab, Cambridge, MA, USA
2 shared publications - Stephanie Gil
School of Engineering and Applied Sciences, Harvard University, Allston, MA, USA
2 shared publications Akarsh Prabhakara
College of Engineering, Carnegie Mellon University, Pittsburgh, PA, USA
1 shared publication- Daniel M. Vogt
Affiliations Harvard John A. Paulson School of Engineering and Applied Sciences, Harvard University, Cambridge, Massachusetts, United States of America, Project CETI, New York, New York, United States of America and Dominica
1 shared publication - Ninad Jadhav
Harvard University
1 shared publication
Latest Publications
- 30 October 2024
- Published by American Association for the Advancement of Science (AAAS) in Science Robotics
- Vol. 9 (95) , eadn7299
Reinforcement learning–based framework for whale rendezvous via autonomous sensing robots
Rendezvous with sperm whales for biological observations is made challenging by their prolonged dive patterns. Here, we propose an algorithmic framework that codevelops multiagent reinforcement learning–based routing (autonomy module) and synthetic aperture radar–based very high frequency (VHF) signal–based bearing estimation (sensing module) for maximizing rendezvous opportunities of autonomous robots with sperm whales. The sensing module is compatible with low-energy VHF tags commonly used for tracking wildlife. The autonomy module leverages in situ noisy bearing measurements of whale vocalizations, VHF tags, and whale dive behaviors to enable time-critical rendezvous of a robot team with multiple whales in simulation. We conducted experiments at sea in the native habitat of sperm whales using an “engineered whale”—a speedboat equipped with a VHF-emitting tag, emulating five distinct whale tracks, with different whale motions. The sensing module shows a median bearing error of 10.55° to the tag. Using bearing measurements to the engineered whale from an acoustic sensor and our sensing module, our autonomy module gives an aggregate rendezvous success rate of 81.31% for a 500-meter rendezvous distance using three robots in postprocessing. A second class of fielded experiments that used acoustic-only bearing measurements to three untagged sperm whales showed an aggregate rendezvous success rate of 68.68% for a 1000-meter rendezvous distance using two robots in postprocessing. We further validated these algorithms with several ablation studies using a sperm whale visual encounter dataset collected by marine biologists.
- 30 October 2024
- Published by American Association for the Advancement of Science (AAAS) in Science Robotics
- Vol. 9 (95) , eadn7299
Reinforcement learning–based framework for whale rendezvous via autonomous sensing robots
Rendezvous with sperm whales for biological observations is made challenging by their prolonged dive patterns. Here, we propose an algorithmic framework that codevelops multiagent reinforcement learning–based routing (autonomy module) and synthetic aperture radar–based very high frequency (VHF) signal–based bearing estimation (sensing module) for maximizing rendezvous opportunities of autonomous robots with sperm whales. The sensing module is compatible with low-energy VHF tags commonly used for tracking wildlife. The autonomy module leverages in situ noisy bearing measurements of whale vocalizations, VHF tags, and whale dive behaviors to enable time-critical rendezvous of a robot team with multiple whales in simulation. We conducted experiments at sea in the native habitat of sperm whales using an “engineered whale”—a speedboat equipped with a VHF-emitting tag, emulating five distinct whale tracks, with different whale motions. The sensing module shows a median bearing error of 10.55° to the tag. Using bearing measurements to the engineered whale from an acoustic sensor and our sensing module, our autonomy module gives an aggregate rendezvous success rate of 81.31% for a 500-meter rendezvous distance using three robots in postprocessing. A second class of fielded experiments that used acoustic-only bearing measurements to three untagged sperm whales showed an aggregate rendezvous success rate of 68.68% for a 1000-meter rendezvous distance using two robots in postprocessing. We further validated these algorithms with several ablation studies using a sperm whale visual encounter dataset collected by marine biologists.
- 26 December 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Robotics
- Vol. 40 (15523098) , 2003-2023
Multiagent Reinforcement Learning: Rollout and Policy Iteration for POMDP With Application to Multirobot Problems
In this paper, we consider the computational and communication challenges of partially observable multiagent sequential decision-making problems. We present algorithms that simultaneously or sequentially optimize the agents' controls by using multistep lookahead, truncated rollout with a known base policy, and a terminal cost function approximation. In particular: 1) We consider multiagent rollout algorithms that dramatically reduce required computation while preserving the key policy improvement property of the standard rollout method. We improve our multiagent rollout policy by incorporating it in an off-line approximate policy iteration scheme, and we apply an additional ‘on-line play’ scheme enhancing off-line approximation architectures. 2) We consider the imperfect communication case and provide various extensions to our rollout methods to deal with this case. 3) We demonstrate the performance of our methods in extensive simulations by applying our method to a challenging partially observable multiagent sequential repair problem (state space size 10 37 and control space size 10 7 ). Our extensive simulations demonstrate that our methods produce better policies for large and complex multiagent problems in comparison with existing methods, including POMCP [1], MADDPG [2] and work well where other methods fail to scale up.
- 26 December 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Robotics
- Vol. 40 (15523098) , 2003-2023
Multiagent Reinforcement Learning: Rollout and Policy Iteration for POMDP With Application to Multirobot Problems
In this paper, we consider the computational and communication challenges of partially observable multiagent sequential decision-making problems. We present algorithms that simultaneously or sequentially optimize the agents' controls by using multistep lookahead, truncated rollout with a known base policy, and a terminal cost function approximation. In particular: 1) We consider multiagent rollout algorithms that dramatically reduce required computation while preserving the key policy improvement property of the standard rollout method. We improve our multiagent rollout policy by incorporating it in an off-line approximate policy iteration scheme, and we apply an additional ‘on-line play’ scheme enhancing off-line approximation architectures. 2) We consider the imperfect communication case and provide various extensions to our rollout methods to deal with this case. 3) We demonstrate the performance of our methods in extensive simulations by applying our method to a challenging partially observable multiagent sequential repair problem (state space size 10 37 and control space size 10 7 ). Our extensive simulations demonstrate that our methods produce better policies for large and complex multiagent problems in comparison with existing methods, including POMCP [1], MADDPG [2] and work well where other methods fail to scale up.
- 4 March 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 5 (3) , 3967-3974
Reinforcement Learning for POMDP: Partitioned Rollout and Policy Iteration With Application to Autonomous Sequential Repair Problems
In this paper we consider infinite horizon discounted partial observation dynamic programming problems with finite state and control spaces. We discuss an algorithm that uses multistep lookahead, truncated rollout with a known base policy, and a terminal cost function approximation. This algorithm is also used for policy improvement in an approximate policy iteration scheme, where successive policies are approximated by using a neural network classifier. A novel feature of our approach is that it is well suited for distributed computation through an extended belief space formulation and the use of a partitioned architecture, which is trained with multiple neural networks. We apply our methods in simulation to a class of sequential repair problems where a robot inspects and repairs a pipeline with potentially several rupture sites under partial information about the state of the pipeline.
- 4 March 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 5 (3) , 3967-3974
Reinforcement Learning for POMDP: Partitioned Rollout and Policy Iteration With Application to Autonomous Sequential Repair Problems
In this paper we consider infinite horizon discounted partial observation dynamic programming problems with finite state and control spaces. We discuss an algorithm that uses multistep lookahead, truncated rollout with a known base policy, and a terminal cost function approximation. This algorithm is also used for policy improvement in an approximate policy iteration scheme, where successive policies are approximated by using a neural network classifier. A novel feature of our approach is that it is well suited for distributed computation through an extended belief space formulation and the use of a partitioned architecture, which is trained with multiple neural networks. We apply our methods in simulation to a class of sequential repair problems where a robot inspects and repairs a pipeline with potentially several rupture sites under partial information about the state of the pipeline.