Mayank Kejriwal

We propose an iterative programmatic planning (IPP) framework that solves grid-based tasks by using large language models (LLMs) to synthesize interpretable, executable programs that act as agent policies mapping environment states to action sequences, rather than relying on traditional search or reinforcement learning. Alongside prompting strategies such as direct code generation, pseudocode conditioned refinement and curriculum-based prompting, IPP centres on an iterative refinement mechanism that updates code from coarse, task-level feedback. We evaluate our approach using six leading LLMs and two challenging grid-based benchmarks (grid-based reasoning for assessing spatial planning (GRASP) and MiniGrid). Our IPP framework demonstrates improvements over direct code generation for five of the six models, with net energy gains ranging from a few per cent to roughly threefold and, for Gemini-2.5-Pro, a sign flip from a negative baseline to a strongly positive score, establishes a new state-of-the-art result for GRASP and significantly outperforms direct elicitation from GPT-o3-mini (by 43% on the hardest MiniGrid task to 116% on GRASP). While code generation has a higher initial prompting cost compared to direct solution elicitation ($0.08 per task versus $0.002 per instance for GPT-o3-mini), the code can be reused for any number of instances, making the amortized cost significantly lower (by 400 times on GPT-o3-mini across the complete GRASP benchmark). Beyond performance gains, IPP produces reusable, human-readable policies that support transparent inspection and modification, underscoring the advantages of planning as program synthesis over per-instance inference. The code and results can be found at http://github.com/Mystic-Slice/LLM-Planning-CodeGen.

Online human trafficking investigations generate vast amounts of noisy, heterogeneous, and deliberately obfuscated data, making traditional search and analytics tools ineffective for supporting law enforcement. This paper discusses the deployment of the Domain‐Specific Insight Graphs (DIG) system, an AI‐powered investigative search engine that was operationally used by over 200 U.S. law enforcement agencies for more than five years in the pre‐COVID period. The system integrates advanced research conducted over the years in information extraction, knowledge graph construction, and entity‐centric search to enable investigators to formulate queries without technical background, aggregate evidence, and uncover latent relationships among entities such as phone numbers, emails, and locations. Beyond technical innovation, the deployment required sustained attention to usability, explainability, and policy compliance, ensuring trust in high‐stakes legal contexts. We report measurable benefits in investigative efficiency, case initiation, and prosecutorial support, as well as lessons learned from long‐term maintenance and adaptation to evolving online platforms. Since 2020, work conducted in this domain has also had significant policy and advocacy ramifications. The system's generalized design has also allowed it to be prototyped for adjacent illicit domains, including securities fraud and illegal firearm sales, demonstrating the broader applicability of AI‐driven investigative tools. We contribute a rare case study of an AI system that has transitioned from research to sustained real‐world impact in a socially critical domain.

Large language models (LLMs) now answer medical exam questions at or above expert level, but it is unclear whether such scores reflect clinical competence, or are instead affected by the format of the questions being asked (e.g., multiple-choice versus open-ended), prior exposure to the benchmarks, and the difficulty of the questions. These benchmarks are also public and widely used online, which may be resulting in LLM memorization. Models may perform better on questions that they have seen before. It is also not evident whether smaller models, which are more locally deployable in HIPAA-compliant hospital settings, have achieved human performance. In this study, we evaluate twelve current models from six leading providers, one smaller and one larger model from each provider, on 2,773 questions from three medical benchmarks (MedQA-USMLE, MedMCQA, and PubMedQA) under standard multiple choice, the same questions with the answer choices withheld, and reconstruction of a hidden question from its choices alone. In the standard multiple-choice setting, the strongest models exceed the human-expert baseline on two of three benchmarks, but the smaller models trail their larger models by 8.2 percentage points and only two of the five pass the human-expert bar on the licensing examination. Withholding the answer choices, however, makes every model struggle: mean accuracy falls from 82.8\% to 51.0\%, and the two largest drops belong to the medically specialised models. On memorization, the strongest models reconstruct one question in four from the answer choices alone, yet accuracy holds on the questions they cannot reconstruct. We further weigh accuracy against inference cost and test whether ensembling several models improves medical question answering.

In complex task environments in both nature and human society, structuralviolations of expectation (VoE) occur with non-trivial frequency. Agents that are designed to operate in such environments must be capable of open-world learning (OWL), defined as the ability to detect and accommodate out-of-distribution inputs, as well as more complex structural VoEs, without requiring extensive and offline re-training. Until recently, OWL research was relatively constrained and limited to areas such as anomaly detection and concept drift. More recently, agent-based OWL research has witnessed much interest from across the community. To support this research, not just for developing OWL algorithms, but also evaluating them, there is a need for multi-agent environments where structural VoEs can be generated, and controlled experiments can be run with relative ease. To address this need, we propose a resource called PokerOWL, a platform that is supported on the Gymnasium infrastructure, which is extensively used in the reinforcement learning and AI game-playing communities. PokerOWL supports both a rich VoE generator and a graphical interface for facilitating development and evaluation of OWL methods. Using an extensive set of experiments and a Poker-playing agent based on Deep Q-Networks, we use PokerOWL to demonstrate how even state-of-the-art agents can struggle to generalize to novel situations without additional OWL capabilities.

Computational modeling and quantifying the drivers of modern political campaign finance is an emerging area of interest among researchers, policymakers, and the general public alike. In a federal legislative body like the U.S. House of Representatives, campaign finance decisions or “money flows” between party members are legal and common practice among both political parties for allocating resources and influence. Using extensive data from the Federal Election Commission, we model these money flows as complex networks and explain their formation using exogenous factors, previously only discussed qualitatively, such as seniority, non-coordinated SuperPAC expenditures, and House leadership status. Our results show that these factors have significant and persistent effects on both parties. These findings provide an empirical basis for ongoing debates about term limits in Congress and the role of unlimited independent expenditures by SuperPACs.

This is a peer review report related to "The Performance of DeepSeek R1 and Gemini 3 on Complex Medical Scenarios: A Comparative Study"