STaR-DRO: Stateful Tsallis Reweighting for Group-Robust Structured Prediction

Authors:

Abstract
Structured prediction with large language models requires more than accurate labels: outputs must satisfy ontology constraints, preserve valid structure, and ensure evidence-grounded decisions under label imbalance and heterogeneous group difficulty. To resolve these challenges, we present a unified framework for robust ontology-constrained generation. First, we introduce a modular prompt-engineering architecture combining XML-style structure, expert disambiguation rules, chain-of-thought reasoning, metadata-aware decision logic, schema contracts, and a self-validation gate. This targets recurrent in-context structured generation failures, including format drift, label ambiguity, evidence hallucination, and metadata-conditioned confusion. Second, we propose STaR-DRO, which combines Tsallis mirror ascent with sparse entmax-style primal mapback, EMA-smoothed group-loss tracking, rescaled ascent signals, and bounded excess-only multipliers. This upweights only persistently hard groups without suppressing easier ones, addressing conventional DRO’s dense Shannon-entropy exponentiated-gradient updates, high-variance stochastic reweighting, positive adversarial mass on groups that are not persistently hard, and performance costs from simplex competition. We evaluate the framework on EPPC Miner, a carefully chosen, clinically grounded representation of high-stakes structured prediction requiring hierarchical label prediction and evidence-span extraction from patient-provider secure messages. Across Llama models spanning 1B to 70B parameters, our prompt engineering improves zero-shot extraction, with an average label F1 gain of +14.46 and a +17.40 Span F1 gain. Building on supervised fine-tuning, STaR-DRO further improves accuracy and robustness, increasing average label F1 by +1.08/+2.20 and reducing average group-wise validation cross-entropy by 21.3%/14.8% relative to SFT and standard DRO, respectively. These findings significantly advance the reliability of automated communication mining for patient-centered clinical care analysis.