Sanket Shah
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Sanket Shah has a total of 120 co-authors. Here are the top co-authors by number of shared publications:
- Milind Tambe
Harvard University
5 shared publications - Andrew Perrault
The Ohio State University
4 shared publications - Aparna Taneja
Google research
3 shared publications - Bryan Wilder
Machine Learning Department, Carnegie Mellon University
3 shared publications - Kai Wang
University of Pennsylvania
3 shared publications Craig Sable
Children’s National Hospital, Washington, USA
2 shared publications
Latest Publications
- Open Access
FairVizARD: A Visualization System for Assessing Multi-Party Fairness of Ride-Sharing Matching Algorithms
There is growing interest in algorithms that match passengers with drivers in ride-sharing problems and their fairness for the different parties involved (passengers, drivers, and ride-sharing companies). Researchers have proposed various fairness metrics for matching algorithms, but it is often unclear how one should balance the various parties' fairness, given that they are often in conflict. We present FairVizARD, a visualization-based system that aids users in evaluating the fairness of ride-sharing matching algorithms. FairVizARD presents the algorithms' results by visualizing relevant spatio-temporal information using animation and aggregated information in charts. FairVizARD also employs efficient techniques for visualizing a large amount of information in a user friendly manner, which makes it suitable for real-world settings. We conduct our experiments on a real-world large-scale taxi dataset and, through user studies and an expert interview, we show how users can use FairVizARD not only to evaluate the fairness of matching algorithms but also to expand on their notions of fairness.
Open AccessFairVizARD: A Visualization System for Assessing Multi-Party Fairness of Ride-Sharing Matching Algorithms
There is growing interest in algorithms that match passengers with drivers in ride-sharing problems and their fairness for the different parties involved (passengers, drivers, and ride-sharing companies). Researchers have proposed various fairness metrics for matching algorithms, but it is often unclear how one should balance the various parties' fairness, given that they are often in conflict. We present FairVizARD, a visualization-based system that aids users in evaluating the fairness of ride-sharing matching algorithms. FairVizARD presents the algorithms' results by visualizing relevant spatio-temporal information using animation and aggregated information in charts. FairVizARD also employs efficient techniques for visualizing a large amount of information in a user friendly manner, which makes it suitable for real-world settings. We conduct our experiments on a real-world large-scale taxi dataset and, through user studies and an expert interview, we show how users can use FairVizARD not only to evaluate the fairness of matching algorithms but also to expand on their notions of fairness.
- Open Access
- 11 April 2025
- Published by Association for the Advancement of Artificial Intelligence (AAAI) in Proceedings of the AAAI Conference on Artificial Intelligence
- Vol. 39 (27) , 27849-27857
Evaluating Index-based Treatment Allocation in Underresourced Communities
In many applications of AI for Social Impact (e.g., when allocating spots in support programs for underserved communities), resources are scarce and an allocation policy is needed to decide who receives a resource. Before being deployed at scale, a rigorous evaluation of an AI-powered allocation policy is vital. In this paper, we introduce the methods necessary to evaluate index-based allocation policies, which allocate a limited number of resources to those who need them the most. Such policies create dependencies between agents, rendering standard statistical tests invalid and ineffective. Addressing the arising practical and technical challenges, we describe an efficient estimator and methods for drawing valid statistical conclusions. Our extensive experiments validate our methodology in practical settings while also showcasing its statistical power. We conclude by proposing and empirically verifying extensions of our methodology that enable us to reevaluate a past randomized control trial conducted with 10000 beneficiaries for a mHealth program for pregnant women. Our new methodology allows us to draw previously invisible conclusions when comparing two different ML allocation policies.
- 11 April 2025
- Published by Association for the Advancement of Artificial Intelligence (AAAI) in Proceedings of the AAAI Conference on Artificial Intelligence
- Vol. 39 (27) , 27849-27857
Open AccessEvaluating Index-based Treatment Allocation in Underresourced Communities
In many applications of AI for Social Impact (e.g., when allocating spots in support programs for underserved communities), resources are scarce and an allocation policy is needed to decide who receives a resource. Before being deployed at scale, a rigorous evaluation of an AI-powered allocation policy is vital. In this paper, we introduce the methods necessary to evaluate index-based allocation policies, which allocate a limited number of resources to those who need them the most. Such policies create dependencies between agents, rendering standard statistical tests invalid and ineffective. Addressing the arising practical and technical challenges, we describe an efficient estimator and methods for drawing valid statistical conclusions. Our extensive experiments validate our methodology in practical settings while also showcasing its statistical power. We conclude by proposing and empirically verifying extensions of our methodology that enable us to reevaluate a past randomized control trial conducted with 10000 beneficiaries for a mHealth program for pregnant women. Our new methodology allows us to draw previously invisible conclusions when comparing two different ML allocation policies.
- Open Access
Analyzing Cost-Sensitive Surrogate Losses via $\mathcal{H}$-calibration
This paper aims to understand whether machine learning models should be trained using cost-sensitive surrogates or cost-agnostic ones (e.g., cross-entropy). Analyzing this question through the lens of $\mathcal{H}$-calibration, we find that cost-sensitive surrogates can strictly outperform their cost-agnostic counterparts when learning small models under common distributional assumptions. Since these distributional assumptions are hard to verify in practice, we also show that cost-sensitive surrogates consistently outperform cost-agnostic surrogates on classification datasets from the UCI repository. Together, these make a strong case for using cost-sensitive surrogates in practice.
Open AccessAnalyzing Cost-Sensitive Surrogate Losses via $\mathcal{H}$-calibration
This paper aims to understand whether machine learning models should be trained using cost-sensitive surrogates or cost-agnostic ones (e.g., cross-entropy). Analyzing this question through the lens of $\mathcal{H}$-calibration, we find that cost-sensitive surrogates can strictly outperform their cost-agnostic counterparts when learning small models under common distributional assumptions. Since these distributional assumptions are hard to verify in practice, we also show that cost-sensitive surrogates consistently outperform cost-agnostic surrogates on classification datasets from the UCI repository. Together, these make a strong case for using cost-sensitive surrogates in practice.
- Open Access
Finite-Horizon Single-Pull Restless Bandits: An Efficient Index Policy For Scarce Resource Allocation
Restless multi-armed bandits (RMABs) have been highly successful in optimizing sequential resource allocation across many domains. However, in many practical settings with highly scarce resources, where each agent can only receive at most one resource, such as healthcare intervention programs, the standard RMAB framework falls short. To tackle such scenarios, we introduce Finite-Horizon Single-Pull RMABs (SPRMABs), a novel variant in which each arm can only be pulled once. This single-pull constraint introduces additional complexity, rendering many existing RMAB solutions suboptimal or ineffective. %To address this, we propose using dummy states to duplicate the system, ensuring that once an arm is activated, it transitions exclusively within the dummy states. To address this shortcoming, we propose using \textit{dummy states} that expand the system and enforce the one-pull constraint. We then design a lightweight index policy for this expanded system. For the first time, we demonstrate that our index policy achieves a sub-linearly decaying average optimality gap of $\tilde{\mathcal{O}}\left(\frac{1}{ρ^{1/2}}\right)$ for a finite number of arms, where $ρ$ is the scaling factor for each arm cluster. Extensive simulations validate the proposed method, showing robust performance across various domains compared to existing benchmarks.Open AccessFinite-Horizon Single-Pull Restless Bandits: An Efficient Index Policy For Scarce Resource Allocation
Restless multi-armed bandits (RMABs) have been highly successful in optimizing sequential resource allocation across many domains. However, in many practical settings with highly scarce resources, where each agent can only receive at most one resource, such as healthcare intervention programs, the standard RMAB framework falls short. To tackle such scenarios, we introduce Finite-Horizon Single-Pull RMABs (SPRMABs), a novel variant in which each arm can only be pulled once. This single-pull constraint introduces additional complexity, rendering many existing RMAB solutions suboptimal or ineffective. %To address this, we propose using dummy states to duplicate the system, ensuring that once an arm is activated, it transitions exclusively within the dummy states. To address this shortcoming, we propose using \textit{dummy states} that expand the system and enforce the one-pull constraint. We then design a lightweight index policy for this expanded system. For the first time, we demonstrate that our index policy achieves a sub-linearly decaying average optimality gap of $\tilde{\mathcal{O}}\left(\frac{1}{ρ^{1/2}}\right)$ for a finite number of arms, where $ρ$ is the scaling factor for each arm cluster. Extensive simulations validate the proposed method, showing robust performance across various domains compared to existing benchmarks. - Open Access
Transposition of the Great Arteries with Intramural Left Main Coronary Artery—Salient Imaging Findings and Choice of Operative Technique
D-transposition of the great arteries (D-TGA) is a common cyanotic critical congenital heart disease. An arterial switch operation (ASO) with/without a ventricular septal defect (VSD) closure is the preferred surgical approach, with an added challenge when an intramural coronary artery (IMC) is present (1), with a reported increased incidence of postoperative complications and mortality (2,3). We present our recent D-TGA with intramural coronary artery (TGA-IMC) experience, focusing on the salient features identified on echocardiography, computed tomography (CT) angiography, and invasive angiograms, as well as variations in ASO surgical techniques for repair. Diagnostic imaging evaluation allowed for identification of the lesion, as well as planning for and undertaking of two different surgical approaches. While the two patients had differing immediate postoperative courses, both were asymptomatic at discharge, with normal biventricular systolic function. Our experience demonstrates that the suspicion for a coronary anomaly in TGA can be raised prenatally and confirmed postnatally with focused trans-thoracic echocardiography and ECG-gated CT angiogram evaluation while also aiding in operative planning. Moreover, suggesting further exploration of the optimal surgical technique for the repair of TGA-IMC.
Open AccessTransposition of the Great Arteries with Intramural Left Main Coronary Artery—Salient Imaging Findings and Choice of Operative Technique
D-transposition of the great arteries (D-TGA) is a common cyanotic critical congenital heart disease. An arterial switch operation (ASO) with/without a ventricular septal defect (VSD) closure is the preferred surgical approach, with an added challenge when an intramural coronary artery (IMC) is present (1), with a reported increased incidence of postoperative complications and mortality (2,3). We present our recent D-TGA with intramural coronary artery (TGA-IMC) experience, focusing on the salient features identified on echocardiography, computed tomography (CT) angiography, and invasive angiograms, as well as variations in ASO surgical techniques for repair. Diagnostic imaging evaluation allowed for identification of the lesion, as well as planning for and undertaking of two different surgical approaches. While the two patients had differing immediate postoperative courses, both were asymptomatic at discharge, with normal biventricular systolic function. Our experience demonstrates that the suspicion for a coronary anomaly in TGA can be raised prenatally and confirmed postnatally with focused trans-thoracic echocardiography and ECG-gated CT angiogram evaluation while also aiding in operative planning. Moreover, suggesting further exploration of the optimal surgical technique for the repair of TGA-IMC.
- Open Access
- 24 March 2024
- Published by Association for the Advancement of Artificial Intelligence (AAAI) in Proceedings of the AAAI Conference on Artificial Intelligence
- Vol. 38 (13) , 14902-14909
Leaving the Nest: Going beyond Local Loss Functions for Predict-Then-Optimize
Predict-then-Optimize is a framework for using machine learning to perform decision-making under uncertainty. The central research question it asks is, "How can we use the structure of a decision-making task to tailor ML models for that specific task?" To this end, recent work has proposed learning task-specific loss functions that capture this underlying structure. However, current approaches make restrictive assumptions about the form of these losses and their impact on ML model behavior. These assumptions both lead to approaches with high computational cost, and when they are violated in practice, poor performance. In this paper, we propose solutions to these issues, avoiding the aforementioned assumptions and utilizing the ML model's features to increase the sample efficiency of learning loss functions. We empirically show that our method achieves state-of-the-art results in four domains from the literature, often requiring an order of magnitude fewer samples than comparable methods from past work. Moreover, our approach outperforms the best existing method by nearly 200% when the localness assumption is broken.
- 24 March 2024
- Published by Association for the Advancement of Artificial Intelligence (AAAI) in Proceedings of the AAAI Conference on Artificial Intelligence
- Vol. 38 (13) , 14902-14909
Open AccessLeaving the Nest: Going beyond Local Loss Functions for Predict-Then-Optimize
Predict-then-Optimize is a framework for using machine learning to perform decision-making under uncertainty. The central research question it asks is, "How can we use the structure of a decision-making task to tailor ML models for that specific task?" To this end, recent work has proposed learning task-specific loss functions that capture this underlying structure. However, current approaches make restrictive assumptions about the form of these losses and their impact on ML model behavior. These assumptions both lead to approaches with high computational cost, and when they are violated in practice, poor performance. In this paper, we propose solutions to these issues, avoiding the aforementioned assumptions and utilizing the ML model's features to increase the sample efficiency of learning loss functions. We empirically show that our method achieves state-of-the-art results in four domains from the literature, often requiring an order of magnitude fewer samples than comparable methods from past work. Moreover, our approach outperforms the best existing method by nearly 200% when the localness assumption is broken.
- Open Access
Efficient Public Health Intervention Planning Using Decomposition-Based Decision-Focused Learning
The declining participation of beneficiaries over time is a key concern in public health programs. A popular strategy for improving retention is to have health workers `intervene' on beneficiaries at risk of dropping out. However, the availability and time of these health workers are limited resources. As a result, there has been a line of research on optimizing these limited intervention resources using Restless Multi-Armed Bandits (RMABs). The key technical barrier to using this framework in practice lies in the need to estimate the beneficiaries' RMAB parameters from historical data. Recent research has shown that Decision-Focused Learning (DFL), which focuses on maximizing the beneficiaries' adherence rather than predictive accuracy, improves the performance of intervention targeting using RMABs. Unfortunately, these gains come at a high computational cost because of the need to solve and evaluate the RMAB in each DFL training step. In this paper, we provide a principled way to exploit the structure of RMABs to speed up intervention planning by cleverly decoupling the planning for different beneficiaries. We use real-world data from an Indian NGO, ARMMAN, to show that our approach is up to two orders of magnitude faster than the state-of-the-art approach while also yielding superior model performance. This would enable the NGO to scale up deployments using DFL to potentially millions of mothers, ultimately advancing progress toward UNSDG 3.1.
Open AccessEfficient Public Health Intervention Planning Using Decomposition-Based Decision-Focused Learning
The declining participation of beneficiaries over time is a key concern in public health programs. A popular strategy for improving retention is to have health workers `intervene' on beneficiaries at risk of dropping out. However, the availability and time of these health workers are limited resources. As a result, there has been a line of research on optimizing these limited intervention resources using Restless Multi-Armed Bandits (RMABs). The key technical barrier to using this framework in practice lies in the need to estimate the beneficiaries' RMAB parameters from historical data. Recent research has shown that Decision-Focused Learning (DFL), which focuses on maximizing the beneficiaries' adherence rather than predictive accuracy, improves the performance of intervention targeting using RMABs. Unfortunately, these gains come at a high computational cost because of the need to solve and evaluate the RMAB in each DFL training step. In this paper, we provide a principled way to exploit the structure of RMABs to speed up intervention planning by cleverly decoupling the planning for different beneficiaries. We use real-world data from an Indian NGO, ARMMAN, to show that our approach is up to two orders of magnitude faster than the state-of-the-art approach while also yielding superior model performance. This would enable the NGO to scale up deployments using DFL to potentially millions of mothers, ultimately advancing progress toward UNSDG 3.1.
- Open Access
Leaving the Nest: Going Beyond Local Loss Functions for Predict-Then-Optimize
Predict-then-Optimize is a framework for using machine learning to perform decision-making under uncertainty. The central research question it asks is, "How can the structure of a decision-making task be used to tailor ML models for that specific task?" To this end, recent work has proposed learning task-specific loss functions that capture this underlying structure. However, current approaches make restrictive assumptions about the form of these losses and their impact on ML model behavior. These assumptions both lead to approaches with high computational cost, and when they are violated in practice, poor performance. In this paper, we propose solutions to these issues, avoiding the aforementioned assumptions and utilizing the ML model's features to increase the sample efficiency of learning loss functions. We empirically show that our method achieves state-of-the-art results in four domains from the literature, often requiring an order of magnitude fewer samples than comparable methods from past work. Moreover, our approach outperforms the best existing method by nearly 200% when the localness assumption is broken.
Open AccessLeaving the Nest: Going Beyond Local Loss Functions for Predict-Then-Optimize
Predict-then-Optimize is a framework for using machine learning to perform decision-making under uncertainty. The central research question it asks is, "How can the structure of a decision-making task be used to tailor ML models for that specific task?" To this end, recent work has proposed learning task-specific loss functions that capture this underlying structure. However, current approaches make restrictive assumptions about the form of these losses and their impact on ML model behavior. These assumptions both lead to approaches with high computational cost, and when they are violated in practice, poor performance. In this paper, we propose solutions to these issues, avoiding the aforementioned assumptions and utilizing the ML model's features to increase the sample efficiency of learning loss functions. We empirically show that our method achieves state-of-the-art results in four domains from the literature, often requiring an order of magnitude fewer samples than comparable methods from past work. Moreover, our approach outperforms the best existing method by nearly 200% when the localness assumption is broken.
- Open Access
- 4 February 2024
- Published by MDPI AG in Journal of Cardiovascular Development and Disease
- Vol. 11 (2) , 55
Prenatal Diagnosis of a Ductal-Dependent Branch Pulmonary Artery: Extra Vessels in the 3-Vessel and Trachea View
Obtaining a three-vessel (3V) and three-vessel and trachea (3VT) view from the fetal upper mediastinum is now considered to be part of standard imaging protocol for routine obstetric cardiac screening examinations. We report two fetal cases of an anomalous pulmonary artery origin, utilizing the standard 3V and 3VT views. Further imaging led to a rare diagnosis of bilateral ductus arteriosus with discontinuous branch pulmonary arteries in the absence of any other congenital heart defect. We briefly discuss the imaging features, differential diagnoses, and management of this rare entity.
- 4 February 2024
- Published by MDPI AG in Journal of Cardiovascular Development and Disease
- Vol. 11 (2) , 55
Open AccessPrenatal Diagnosis of a Ductal-Dependent Branch Pulmonary Artery: Extra Vessels in the 3-Vessel and Trachea View
Obtaining a three-vessel (3V) and three-vessel and trachea (3VT) view from the fetal upper mediastinum is now considered to be part of standard imaging protocol for routine obstetric cardiac screening examinations. We report two fetal cases of an anomalous pulmonary artery origin, utilizing the standard 3V and 3VT views. Further imaging led to a rare diagnosis of bilateral ductus arteriosus with discontinuous branch pulmonary arteries in the absence of any other congenital heart defect. We briefly discuss the imaging features, differential diagnoses, and management of this rare entity.
- Open Access
- 26 June 2023
- Published by Association for the Advancement of Artificial Intelligence (AAAI) in Proceedings of the AAAI Conference on Artificial Intelligence
- Vol. 37 (10) , 12138-12146
Scalable Decision-Focused Learning in Restless Multi-Armed Bandits with Application to Maternal and Child Health
This paper studies restless multi-armed bandit (RMAB) problems with unknown arm transition dynamics but with known correlated arm features. The goal is to learn a model to predict transition dynamics given features, where the Whittle index policy solves the RMAB problems using predicted transitions. However, prior works often learn the model by maximizing the predictive accuracy instead of final RMAB solution quality, causing a mismatch between training and evaluation objectives. To address this shortcoming, we propose a novel approach for decision-focused learning in RMAB that directly trains the predictive model to maximize the Whittle index solution quality. We present three key contributions: (i) we establish differentiability of the Whittle index policy to support decision-focused learning; (ii) we significantly improve the scalability of decision-focused learning approaches in sequential problems, specifically RMAB problems; (iii) we apply our algorithm to a previously collected dataset of maternal and child health to demonstrate its performance. Indeed, our algorithm is the first for decision-focused learning in RMAB that scales to real-world problem sizes.
- 26 June 2023
- Published by Association for the Advancement of Artificial Intelligence (AAAI) in Proceedings of the AAAI Conference on Artificial Intelligence
- Vol. 37 (10) , 12138-12146
Open AccessScalable Decision-Focused Learning in Restless Multi-Armed Bandits with Application to Maternal and Child Health
This paper studies restless multi-armed bandit (RMAB) problems with unknown arm transition dynamics but with known correlated arm features. The goal is to learn a model to predict transition dynamics given features, where the Whittle index policy solves the RMAB problems using predicted transitions. However, prior works often learn the model by maximizing the predictive accuracy instead of final RMAB solution quality, causing a mismatch between training and evaluation objectives. To address this shortcoming, we propose a novel approach for decision-focused learning in RMAB that directly trains the predictive model to maximize the Whittle index solution quality. We present three key contributions: (i) we establish differentiability of the Whittle index policy to support decision-focused learning; (ii) we significantly improve the scalability of decision-focused learning approaches in sequential problems, specifically RMAB problems; (iii) we apply our algorithm to a previously collected dataset of maternal and child health to demonstrate its performance. Indeed, our algorithm is the first for decision-focused learning in RMAB that scales to real-world problem sizes.