ARAnirban Roy

Anirban Roy

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Anirban Roy has a total of 15 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Relevance Structures Are a Necessary Prerequisite for Artificial Metacognition

    by Alexander M. Berenbeim, Ramneet Kaur, Adam Cobb, Brian Matejek, Anirban Roy, Susmit Jha, Nathaniel D. Bastian

    AI-enabled systems deployed in dynamic, open-world, safety- and mission-critical environments often lack mechanisms for reliable self-monitoring and self-regulation under shift and perturbation. Artificial Metacognition (AM) addresses this by enabling systems to assess and adapt their own inference and learning processes. We show that relevance structures, which are order-theoretic substrates for comparing generalization and specialization and for quantifying under-coverage versus overreach, are structurally unavoidable for computable self-regulation. In turn, relevance structures provide an operational control theory for metacognitive tasks. We introduce exploratory, parsimonious, and balanced relevance scores and embed them into version-space algebra to regulate hypothesis search and inductive strategy selection. The resulting framework yields competence-aware learning systems that can both correct and explain their behavior by explicitly balancing exploration and parsimony. We illustrate relevance structures and their application with a medical visual question answering study on the SLAKE dataset.

  • Open Access

    Applications of Certainty Scoring for Machine Learning Classification and Out-of-Distribution Detection

    by Alexander M. Berenbeim, Adam D. Cobb, Anirban Roy, Susmit Jha, Nathaniel D. Bastian

    Quantitative characterizations and estimations of uncertainty are of fundamental importance for machine learning classification, particularly in safety-critical settings where continuous real-time monitoring requires explainable and reliable scoring. Reliance on the maximum a posteriori principle to determine label classification can obscure the certainty of a label assignment. We develop a theoretical framework for quantitative scores of certainty and competence based on predicted probability estimates, formally prove their properties, and empirically confirm the inferential power of these properties across different data modalities, tasks, and model architectures. Our theoretical results establish that competent models have distinct distributions of certainty for true and false positives conditioned on inputs similar to training and testing data, and prove that this framework provides a reliable means to infer the quality of model predictions and detect false positives. Our empirical results bear out that there are distinct distributions of certainty scores on training and holdout data, as well as data that is a priori out-of-distribution. For expert models, at least 62.1% of false positives could be identified when using a cut-off at the bottom 5% TP threshold. Further, we found a strong negative correlation between empirical competence and the FPR95TPR rate for EnergyBased Out-of-Distribution (OOD) detectors. Finally, we developed two forms of an OOD detector that were able to reliably distinguish in-distribution data from OOD data for both frequentist and Bayesian models, performing better on average than previous state-of-the-art EnergyBased OOD detection methods, and improving upon the baseline Monte Carlo Dropout AUPR-OUT performance on average by 14.4% and 16.5%, and reducing the FPR95TPR by 54.2% and 37.6%.


  • Open Access

    Safety Monitoring for Learning-Enabled Cyber-Physical Systems in Out-of-Distribution Scenarios

    by Vivian Lin, Ramneet Kaur, Yahan Yang, Souradeep Dutta, Yiannis Kantaros, Anirban Roy, Susmit Jha, Oleg Sokolsky, Insup Lee


  • Open Access

    CODiT: Conformal Out-of-Distribution Detection in Time-Series Data for Cyber-Physical Systems

    by Ramneet Kaur, Kaustubh Sridhar, Sangdon Park, Yahan Yang, Susmit Jha, Anirban Roy, Oleg Sokolsky, Insup Lee

    Uncertainty in the predictions of learning enabled components hinders their deployment in safety-critical cyber-physical systems (CPS). A shift from the training distribution of a learning enabled component (LEC) is one source of uncertainty in the LEC's predictions. Detection of this shift or out-of-distribution (OOD) detection on individual datapoints has therefore gained attention recently. But in many applications, inputs to CPS form a temporal sequence. Existing techniques for OOD detection in time-series data for CPS either do not exploit temporal relationships in the sequence or do not provide any guarantees on detection. We propose using deviation from the in-distribution temporal equivariance as the non-conformity measure in conformal anomaly detection framework for OOD detection in time-series data for CPS. Computing independent predictions from multiple conformal detectors based on the proposed measure and combining these predictions by Fisher's method leads to the proposed detector CODiT with bounded false alarms. We illustrate the efficacy of CODiT by achieving state-of-the-art results in autonomous driving systems with perception (or vision) LEC. We also perform experiments on medical CPS for GAIT analysis where physiological (non-vision) data is collected with force-sensitive resistors attached to the subject's body. Code, data, and trained models are available at https://github.com/kaustubhsridhar/time-series-OOD