Jeremy T. White

Bayesian global optimization (BGO) offers an efficient way to solve expensive optimization problems, such as groundwater allocation planning, where model run times are often prohibitive. BGO uses a surrogate model, typically Gaussian Process Regression (GPR), to replace the costly process-based model, with acquisition functions maximized to select infill points that refine GPR predictions. However, standard BGO evaluates only one infill point per iteration, underutilizing available parallel computing capacity. This study introduces Batch BGO (BBGO), a framework that selects batches of infill points per iteration for parallel execution. BBGO performs an internal multi-objective optimization, using evolutionary algorithms to generate a Pareto front of diverse, informative candidates balancing exploration and exploitation. This batch process repeats iteratively, refining the surrogate until convergence. BBGO is tested on three unconstrained and one constrained benchmark problem, then applied to groundwater allocation planning in Perth, Western Australia, achieving optimal plans in as few as four parallelized iterations.

An open-source package has been developed to efficiently and non-intrusively solve high dimensional, non-linear constrained optimization problems using sequential quadratic programming (SQP). PESTPP-SQP uses an ensemble-based implementation of SQP, allowing quantification of uncertainty arising from model parameters to help facilitate reliability-based decision-making. As part of the open-source PEST++ ecosystem, PESTPP-SQP features: (i) a non-intrusive and parallelized application programming interface that allows the tool to be applied to a wide range of models, and (ii) on-demand switching to other algorithms within PEST++ such as the iterative ensemble smoother to seek/maintain feasibility. The capability and performance of PESTPP-SQP are demonstrated with a benchmark test problem and a synthetic, integrated surface-water/groundwater model.

The traditional groundwater modeling approach of manual calibration with a handful of parameters and ad hoc one-at-a-time sensitivity analysis is giving way to formal data assimilation and uncertainty estimation, where the natural very-high dimensionality of the inverse problem is embraced. In theory, this is an improvement for applied groundwater modeling, and, more importantly, the management of groundwater resources. However, this transition is not without hardship. Many new concepts, skills, and techniques must be learned to effectively and efficiently assimilate many kinds of information and to ultimately provide robust estimates of predictive uncertainty in an applied groundwater modeling setting, where time and budget pressures are real. This talk will present some foundational concepts surrounding predictive groundwater modeling, including the roles of model complexity, data, and uncertainty. The talk will include discussion of apparent trends in the groundwater modeling industry, with a few examples of modern applied predictive groundwater modeling.

Adjoint sensitivity analysis provides an efficient alternative to direct methods when evaluating the influence of many uncertain parameters on a limited number of performance measures in hydrologic and hydrogeologic models. However, most adjoint implementations are “intrusive”, requiring extensive modifications of the forward simulation code. This creates significant development and maintenance burdens that limit broad adoption. To address these needs, we present MF6‐ADJ, a “non‐intrusive” adjoint sensitivity capability for the MODFLOW 6 groundwater flow process that leverages the MODFLOW Application Programming Interface (API) to interact with the forward groundwater flow solution without altering its core code. MF6‐ADJ supports both confined and unconfined flow conditions, structured and unstructured grids, and is compatible with both the Standard and Newton–Raphson solution schemes. It computes sensitivities of a wide range of general performance measures, including hydraulic heads, boundary fluxes, and weighted residuals, with respect to key model parameters such as hydraulic conductivity, storage coefficient, injection/extraction rate, recharge rate, boundary head, and boundary conductance. Sensitivities are computed at each node, enabling fine‐grained diagnostic and calibration analysis. Validation against analytical solutions and the finite‐difference perturbation method confirms excellent agreement, while demonstrating speedups ranging from hundreds to tens of thousands of times depending on grid discretization, since the adjoint state method computes sensitivities efficiently at the grid‐block level. This non‐intrusive design makes MF6‐ADJ highly accessible and maintainable, offering efficient and scalable sensitivity analysis in complex groundwater modeling workflows.

Predictive uncertainty analysis focuses on defensible variability in model projected values after estimation of the posterior parameter distribution. Inverse-style parameter estimation selects posterior parameters through history matching where parameters are varied and resulting model simulation values are compared to observations, and parameters are selected balancing goodness-of-fit between simulated and observed values and expert knowledge. When inverse-style parameter estimation approaches are used, parameter sensitivity, which is the change in simulated outputs relative to the change in parameter values, is an important consideration. Variation in null space parameters has a limited impact on history matching skill; however, these parameters become important when they impact predictions. A new null space sensitivity analysis for ensemble methods of data assimilation (DA) using observation error models is developed and implemented for an integrated hydrological model. Empirical parameter sensitivity is estimated by comparing the spreads of prior and posterior parameter distributions. Sensitivity analysis is generated by an ensemble of models with insensitive parameters varying across the prior parameter distribution and sensitive parameters fixed to best-fit model values. The result is identification of insensitive aquifer storage parameters that change storage-related model predictions by as much as two times. This null space analysis describes uncertainty from data insufficiency. Ensemble methods using observation error models also describe predictive uncertainty from noisy measurements and imperfect models.

Artificial intelligence (AI), including machine learning (ML) and deep learning (DL), learns by training and is restricted by the amount and quality of training data. Training involves a tradeoff between prediction bias and variance controlled by model complexity. Increased model complexity decreases prediction bias, increases variance, and increases overfitting possibilities. Overfitting is a significantly smaller training prediction error relative to the trained model prediction error for an independent validation set. Uncertain data generate risks for AI–ML because they increase overfitting and limit generalization ability. Specious confidence in predictions from overfit models with limited generalization ability, leading to misguided water resource management, is the uncertainty-related negative consequence. Improved data is the way to improve AI–ML models. With uncertain water resource data sets, like stream discharge, there is no quick way to generate improved data. Data assimilation (DA) provides mitigation for uncertainty risks, describes data- and model-related uncertainty, and propagates uncertainty to results using observation error models. A DA-derived mitigation example is provided using a common-sense baseline, derived from an observation error model, for the confirmation of generalization ability and a threshold identifying overfitting. AI–ML models can also be incorporated into DA to provide additional observations for assimilation or as a forward model for prediction and inverse-style calibration or training. The mitigation of uncertain data risks using DA involves a modified bias–variance tradeoff that focuses on increasing solution variability at the expense of increased model bias. Increased variability portrays data and model uncertainty. Uncertainty propagation produces an ensemble of models and a range of predictions.

In this paper, we review the derivation of the Gauss–Levenberg–Marquardt (GLM) algorithm and its extension to ensemble parameter estimation. We explore the use of graphical methods to provide insights into how the algorithm works in practice and discuss the implications of both algorithm tuning parameters and objective function construction in performance. Some insights include understanding the control of both parameter trajectory and step size for GLM as a function of tuning parameters. Furthermore, for the iterative Ensemble Smoother (iES), we discuss the importance of noise on observations and show how iES can cope with non‐unique outcomes based on objective function construction. These insights are valuable for modelers using PEST, PEST++, or similar parameter estimation tools.