Rong Chen

This study evaluates an automatic differentiation (AD)-assisted first-order conditional estimation (FOCE) algorithm integrated into the FOCE Extended Least Squares (FOCE ELS) framework within Phoenix NLME 8.6. AD is a method commonly used in machine learning to provide exact (to machine precision) values for the partial derivatives of a function. There are two modes of AD, forward mode and reverse mode [1]. In the case of nonlinear regression, the objective function of interest is the negative twice the log-likelihood (−2LL), expressed as a function of the model parameter values. Previously released versions of NLME use finite difference (FD), a numerical approximation for the partial derivative. To examine the impact of AD compared to FD on execution speed, we constructed 72 models ranging from one to three compartments. All models included Michaelis-Menten elimination, first-order absorption, and combined additive and proportional residual error models. The models were selected to provide a range of complexity and numerical identifiability, from simple, well-identified models to complex, poorly identifiable ones. We demonstrate that AD improves performance in both the estimation and covariance steps. The AD method had no significant impact on estimation robustness or the success of the covariance step.

Automatic differentiation (AD), a key method for accurately and efficiently computing derivatives in modern machine learning, is now implemented in Phoenix® NLME™ 8.6 for the first time and applied to the first-order conditional estimation extended least squares (FOCE ELS), Laplacian, and adaptive Gaussian quadrature (AGQ) algorithms. We name the AD implementation as ‘automatic-differentiation-assisted parametric optimization’ (ADPO), which can be enabled by checking the ‘Fast Optimization’ option. We present in detail how ADPO is implemented in the frequently used FOCE ELS algorithm, and analyze its performance from the benchmarks based on four PK/PD models. We show both ADPO and traditional FOCE ELS which uses gradients obtained from finite difference (FD) are reasonably accurate and robust, while the main advantage of ADPO being that it considerably reduces computation time no matter what ODE solvers are used: in general ADPO reduces the total run time by around 20% to 50% compared to traditional FOCE ELS. In a case for the realistic voriconazole model using ‘auto-detect’ ODE solver, 95% reduction in the total run time is observed.

Inspired from quantum Monte Carlo, by sampling discrete and continuous variables at the same time using the Metropolis–Hastings algorithm, we present a novel, fast, and accurate high performance Monte Carlo Parametric Expectation Maximization (MCPEM) algorithm. We named it Randomized Parametric Expectation Maximization (RPEM). We compared RPEM with NONMEM's Importance Sampling Method (IMP), Monolix's Stochastic Approximation Expectation Maximization (SAEM), and Certara's Quasi‐Random Parametric Expectation Maximization (QRPEM) for a realistic two‐compartment voriconazole model with ordinary differential equations using simulated data. We show that RPEM is as fast and as accurate as the algorithms IMP, QRPEM, and SAEM for the voriconazole model in reconstructing the population parameters, for the normal and log‐normal cases.

In this paper we describe NPSA, the first parallel nonparametric global maximum likelihood optimization algorithm using simulated annealing (SA). Unlike the nonparametric adaptive grid search method NPAG, which is not guaranteed to find a global optimum solution, and may suffer from the curse of dimensionality, NPSA is a global optimizer and it is free from these grid related issues. We illustrate NPSA by a number of examples including a pharmacokinetics (PK) model for Voriconazole and show that NPSA may be taken as an upgrade to the current grid search based nonparametric methods.

Inspired from quantum Monte Carlo, by using unbiased estimators all the time and sampling discrete and continuous variables at the same time using Metropolis algorithm, we present a novel, fast, and accurate high performance Monte Carlo Parametric Expectation Maximization (MCPEM) algorithm. We named it Randomized Parametric Expectation Maximization (RPEM). In particular, we compared RPEM with Monolix's SAEM and Certara's QRPEM for a realistic two-compartment Voriconazole model with ordinary differential equations (ODEs) and using simulated data. We show that RPEM is 3 to 4 times faster than SAEM and QRPEM, and more accurate than them in reconstructing the population parameters.

We describe a path-integral ground-state quantum Monte Carlo method for light nuclei in continuous space. We show how to efficiently update and sample the paths with spin-isospin dependent and spin-orbit interactions. We apply the method to the triton and alpha particle using both local chiral interactions with next-to-next-to-leading-order %(N$^2$LO) and the Argonne interactions. For operators, like the total energy, that commute with the Hamiltonian, our results agree with Green's function Monte Carlo and auxiliary field diffusion Monte Carlo calculations. For operators that do not commute with the Hamiltonian and for Euclidean response functions, the path-integral formulation allows straightforward calculation without forward walking or the increased variance typical of diffusion methods. We demonstrate this by calculating density distributions, root mean square radii, and Euclidean response functions for single-nucleon couplings.