From Regulatory Text to Structured Data: A Domain-Specific LLM Framework for Extracting Hydropower Operational Constraints
Authors:
- Faisal Bin AshrafFAFaisal Bin Ashraf
- Hong-Jun YoonHYHong-Jun Yoon
- Carly H. HansenCHCarly H. Hansen
- Thomas A. RugglesTRThomas A. Ruggles
- Debjani SinghDSDebjani Singh
- Esther S. ParishEPEsther S. Parish
- Shih-Chieh KaoSKShih-Chieh Kao
Preprint
- 9 April 2026
- preprint
- Published by Elsevier in SSRN Electronic Journal
Abstract
Operational constraints governing hydropower facilities are buried in lengthy regulatory documents with inconsistent formatting, yet no scalable method exists to extract them. Here, we present a regulatory interpretation framework that pairs domain knowledge guided prompt engineering with multi-criteria evidence ranking system, implemented using openly available large language models (LLMs) deployable on local infrastructure. The framework utilizes expert regulatory reasoning, distinguishing mandated minimums from operational capabilities, resolving conflicting values across document sections, and producing traceable, audit-ready outputs (extracted value, regulatory context, and verbatim evidence) and applies it to U.S. Federal Energy Regulatory Commission (FERC) licenses and other hydropower regulatory documents. We evaluated four open LLMs, Llama 3.2 3B, Llama 3 8B, GPT-OSS 20B, and Llama 3.3 70B using a representative set of 50 hydropower regulatory documents across ten operational variables. For simple structured variables (project names, locations, generation capacity), the 70B model achieved 90-98% accuracy, while complex variables requiring interpretation of scattered regulatory text showed substantially lower performance (0-43% for zero-shot extraction). For a complex variable tested (minimum flow requirements), a targeted multi-stage extraction approach improved accuracy from 43% (zero-shot) to 76% (targeted). Overall accuracy across all ten variables was 52.7% (Correct + Partially Correct), with substantial variation by variable complexity. Among the four models tested, only the 70B model was able to follow the complex regulatory reasoning prompts; smaller models failed systematically via false negatives and hallucinations rather than reduced accuracy. The framework shifts regulatory data extraction from document-level classification to fact-level extraction with audit trail, substantially reducing expert effort. The approach is transferable to other regulatory domains where critical information is dispersed across unstructured documents.Keywords
This publication has 7 references indexed in Scilit:
- A census of fish passage facilities at US hydropower developments across the conterminous United StatesJournal of Environmental Management, 2025
- From text to insight: large language models for chemical data extractionChemical Society Reviews, 2024
- A Decision Support System to Compile Environmental Mitigations from Hydropower Licensing Documents2024
- Climate change and its influence on water systems increases the cost of electricity system decarbonizationNature Communications, 2024
- LLM-AIx: An open source pipeline for Information Extraction from unstructured medical text based on privacy preserving Large Language ModelsmedRxiv, 2024
- Structured information extraction from scientific text with large language modelsNature Communications, 2024
- A synthesis of environmental and recreational mitigation requirements at hydropower projects in the United StatesEnvironmental Science & Policy, 2016