Michael R. Kosorok

We introduce a novel cyclic Markov decision process (MDP) framework for multi-step decision problems with heterogeneous stage-specific dynamics, transitions, and discount factors across the cycle. In this setting, offline learning is challenging: optimizing a policy at any stage shifts the state distributions of subsequent stages, propagating mismatch across the cycle. To address this, we propose a modular structural framework that decomposes the cyclic process into stage-wise sub-problems. While generally applicable, we instantiate this principle as CycleFQI, an extension of fitted Q-iteration enabling theoretical analysis and interpretation. It uses a vector of stage-specific Q-functions, tailored to each stage, to capture within-stage sequences and transitions between stages. This modular design enables partial control, allowing some stages to be optimized while others follow predefined policies. We establish finite-sample suboptimality error bounds and derive global convergence rates under Besov regularity, demonstrating that CycleFQI mitigates the curse of dimensionality compared to monolithic baselines. Additionally, we propose a sieve-based method for asymptotic inference of optimal policy values under a margin condition. Experiments on simulated and real-world Type 1 Diabetes data sets demonstrate CycleFQI's effectiveness.

Optimal individualized treatment decision‐making has improved health outcomes in recent years. The value function is commonly used to evaluate the goodness of an individualized treatment decision rule. Despite recent advances, comparing value functions between different treatment decision rules or constructing confidence intervals around value functions remains difficult. We propose a t ‐test based method applied to a test set that generates valid p ‐values to compare value functions between a given pair of treatment decision rules when some of the data are missing. We demonstrate the ease in use of this method and evaluate its performance via simulation studies and apply it to the China Health and Nutrition Survey data.

A key challenge in analyzing the behavior of change-plane estimators is that the objective function has multiple minimizers. Two estimators are proposed to deal with this non-uniqueness. For each estimator, an n-rate of convergence is established, and the limiting distribution is derived. Based on these results, we provide a parametric bootstrap procedure for inference. The validity of our theoretical results and the finite sample performance of the bootstrap are demonstrated through simulation experiments. We demonstrate the proposed methods by applying them to latent subgroup identification in precision medicine using the ACTG175 AIDS study data.

Background: Glucagon‐like Peptide‐1 Receptor Agonists (GLP1RA) may reduce asthma exacerbation (AE) risk, but it is unclear which populations benefit most. Recent pharmacoepidemiologic studies have employed iterative causal forest (iCF), a machine learning (ML) algorithm, to identify subgroups with heterogeneous treatment effects (HTEs). While iCF does not rely on prior knowledge of treatment‐variable interactions, it may be constrained by missing or poorly defined variables in pharmacoepidemiologic studies.Methods: We applied the high‐dimensional iterative causal forest (hdiCF)—a causal ML algorithm requiring predefined variables—to MarketScan 2016–2020 claims data to identify populations with asthma that might benefit most from GLP1RA in reducing AE risk. We built a GLP1RA vs. sulfonylurea new‐user cohort with ≥ 1 inpatient or two outpatient asthma encounters, excluding patients with nonasthma indications for systemic steroids. The outcome was acute AE (hospital admission or emergency department visit for asthma), assessed over 6 months using 599 high‐dimensional features from inpatient/outpatient services and pharmacy claims.Results: In the overall population, GLP1RA decreased AE risk relative to sulfonylurea: aRD −1.4% (−2.0%, −0.8%). hdiCF identified three subgroups based on the quantity of systemic steroid prescription fills (0, 1, and ≥ 2): patients with ≥ 2 prescriptions (GLP1RA: 34 events/1367 individuals; sulfonylurea: 53/1013) benefited most from GLP1RA: aRD −3.8% (−5.3%, −2.2%).Conclusions: This study demonstrates how automated feature identification can pinpoint clinically relevant subgroups with HTEs. The quantity of systemic steroid prescriptions, as a proxy for severe asthma, may guide personalized predictions of GLP1RA's short‐term benefits on acute AE.

We tested a novel high-dimensional approach (using 1 ordinal variable per code with up to four levels: zero, occurred once, sporadically, or frequent) against the standard high-dimensional propensity score (hdPS) method (up to 3 binary variables per code) for detecting heterogeneous treatment effects (HTE). Using the iterative causal forest (iCF) subgrouping algorithm, we analyzed a new-user cohort of 8075 sodium–glucose cotransporter-2 inhibitors and 7313 glucagon-like peptide-1 receptor agonists from a 20% random Medicare sample (2015–2019) with ≥1-year parts A/B/D enrollment and without severe renal disease. We extracted the top 200 prevalent codes across diagnoses, procedures, and prescriptions during the 1-year baseline. Subgroup-specific conditional average treatment effects (CATEs) were assessed for 2-year risk differences (aRD) in hospitalized heart failure using inverse-probability treatment weighting. The overall population exhibited an aRD of −0.4% (95% CI, −1.1% to 0.2%). Our high-dimensional setting identified patients with ≥2 loop diuretic prescriptions (aRD, −2.6%, 95% CI, −5.0% to -0.2%) as the subgroup with the largest CATE. In contrast, the high-dimensional setting from hdPS identified patients with chronic kidney disease (aRD, −1.7%, 95% CI, −3.6% to 0.2%). Across various sensitivity analyses, our high-dimensional approach more accurately identified expected subgroups with HTE that aligns with prior clinical evidence.

SignificancePulse oximeters are frequently used in healthcare settings to noninvasively estimate blood oxygen levels in patients, in turn influencing clinical decision-making. However, numerous studies have found pulse oximeters consistently overestimate oxygen levels in patients of color, with this overestimation increasing as oxygen levels decline. Although some reports postulate skin pigmentation may be the source of bias, to our knowledge, the precise impact of skin pigmentation on pulse oximetry has not been elucidated.AimOur objective is to experimentally characterize the impact of skin pigmentation on pulse oximeters across a broad range of oxygen saturations and determine whether skin pigmentation can explain the observed pulse oximeter bias in patients of color.ResultsUsing a live porcine model, we experimentally demonstrate the impact of skin pigmentation on pulse oximeter calculation. In particular, we show that skin pigmentation attenuates the sensitivity of red-light absorption to changes in oxygen saturation and show that the pulse oximeter output is consequently attenuated in pigmented skin.ConclusionThe systematic animal experiments confirm the confounding impact of skin pigmentation on intermediate calculations of pulse oximetry, and validate existing clinical observations of pulse oximeter bias in patients of color.