Sayan Mitra

We study localization and control for unstable systems under coarse, single-bit sensing. Motivated by understanding the fundamental limitations imposed by such minimal feedback, we identify sufficient conditions under which the initial state can be recovered despite instability and extremely sparse measurements. Building on these conditions, we develop an active localization algorithm that integrates a set-based estimator with a control strategy derived from Voronoi partitions, which provably estimates the initial state while ensuring the agent remains in informative regions. Under the derived conditions, the proposed approach guarantees exponential contraction of the initial-state uncertainty, and the result is further supported by numerical experiments. These findings can offer theoretical insight into localization in robotics, where sensing is often limited to coarse abstractions such as keyframes, segmentations, or line-based features.

We study localization and control for unstable systems under coarse, single-bit sensing. Motivated by understanding the fundamental limitations imposed by such minimal feedback, we identify sufficient conditions under which the initial state can be recovered despite instability and extremely sparse measurements. Building on these conditions, we develop an active localization algorithm that integrates a set-based estimator with a control strategy derived from Voronoi partitions, which provably estimates the initial state while ensuring the agent remains in informative regions. Under the derived conditions, the proposed approach guarantees exponential contraction of the initial-state uncertainty, and the result is further supported by numerical experiments. These findings can offer theoretical insight into localization in robotics, where sensing is often limited to coarse abstractions such as keyframes, segmentations, or line-based features.

This paper addresses the problem of visual target tracking in scenarios where a pursuer may experience intermittent loss of visibility of the target. The design of a Switched Visual Tracker (SVT) is presented which aims to meet the competing requirements of maintaining both proximity and visibility. SVT alternates between a visual tracking mode for following the target, and a recovery mode for regaining visual contact when the target falls out of sight. We establish the stability of SVT by extending the average dwell time theorem from switched systems theory, which may be of independent interest. Our implementation of SVT on an Agilicious drone [1] illustrates its effectiveness on tracking various target trajectories: it reduces the average tracking error by up to 45% and significantly improves visibility duration compared to a baseline algorithm. The results show that our approach effectively handles intermittent vision loss, offering enhanced robustness and adaptability for real-world autonomous missions. Additionally, we demonstrate how the stability analysis provides valuable guidance for selecting parameters, such as tracking speed and recovery distance, to optimize the SVT's performance.

Large Language Models (LLMs) have shown remarkable capabilities in natural language processing, mathematical problem solving, and tasks related to program synthesis. However, their effectiveness in long-term planning and higher-order reasoning has been noted to be limited and fragile. This paper explores an approach for enhancing LLM performance in solving a classical robotic planning task by integrating solver-generated feedback. We explore four different strategies for providing feedback, including visual feedback, we utilize fine-tuning, and we evaluate the performance of three different LLMs across a 10 standard and 100 more randomly generated planning problems. Our results suggest that the solver-generated feedback improves the LLM's ability to solve the moderately difficult problems, but the harder problems still remain out of reach. The study provides detailed analysis of the effects of the different hinting strategies and the different planning tendencies of the evaluated LLMs.

This project has led to the creation of an automated-testing infrastructure for autonomy code. The framework uses Jenkins, AWS Lambda, Docker, Kubernetes, and other open-source technologies. It was utilized and evaluated both for the Generalized Racing Intelligence Competition (GRAIC) and for evaluating student programming assignments for the principles of safe autonomy course (ECE484). This infrastructure has improved our capability to evaluate (autograde) student design assignments, and students can also receive precise feedback on their work as they progress through various design challenges. This infrastructure has significantly improved our capability in both automatic, cloud-based testing of autonomy code and making this service available to a large population of students.

Engineering safe and trustworthy autonomous systems demand formal methodologies that accommodate statistical models and assumptions. In the first part of this talk, I will present safety verification methods that use reachability analysis. This research explores a spectrum of model assumptions and the corresponding guarantees and utilities. The methods combine simulation data with sensitivity analysis. At one end, complete dynamical models offer strong guarantees, but their utility can be limited. Learning sensitivity from data comes with statistical guarantees, and the algorithms are applicable to real-world scenarios, and still confer the benefits of formal reasoning via composition and abstraction. This flexible approach informs our current work on the Verse framework which aims to make code-level analysis for autonomous multi-agent scenarios accessible to undergraduates in engineering design courses. In the second part, I will discuss a safety analysis method for learning-enabled autonomous systems using abstraction and substitutivity. The method derives the conditions when a hypothetical ideal observer can be substituted with a practical observer implemented by a machine learning model. This leads to the notion of a perception contract which approximates the machine learning model, and it can substitute a perfect observer while preserving the safety proof. I will discuss applications of this method in the analysis for vision-based lane keeping and automated landing systems.