RCRobert Philip Chalmers

Robert Philip Chalmers

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Robert Philip Chalmers has a total of 30 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    A General Approach for Estimating Projective IRT Models

    by R. Philip Chalmers, Carl F. Falk, Steve P. Reise

    Projective Item Response Theory (PIRT) modeling is a measurement framework in which multidimensional item response models are expressed in terms of lower-dimensional (e.g., unidimensional) proxy IRT models. Existing PIRT methods and their associated estimators currently rely on logistic function approximations that are limited to a narrow class of ordered, monotonic response functions. Though unexplored to date, these methods also require computationally intensive procedures to obtain sampling variability estimates for their resulting PIRT estimates. These and other limitations restrict the practical utility and broad application of PIRT models, particularly in empirical settings where only moderate sample sizes are available. To address these challenges, we introduce a general maximum marginal likelihood PIRT (MML-PIRT) approach that leverages components of the expectation–maximization (EM) algorithm commonly used in MML estimation. The proposed method uses expected count information generated during the EM-MML to fit proxy response functions for any focal trait of interest, and for a much broader class of multidimensional IRT models. In addition, MML-PIRT provides accurate and efficient large-sample variability estimates of the resulting PIRT model, thereby enhancing both the flexibility and statistical efficacy of PIRT model applications.

  • Open Access

    SIMP2L: A template for SIMulation Preregistration, Piloting documentation and transparency over the entire simulation Life-cycle

    by Carolin Strobl, Robert Philip Chalmers, Mirka Henninger

    A growing body of methodological literature suggests that simulation studies are affected by issues similar to those underlying the replication crisis in empirical research, including inadequate reporting practices and the exploitation of researcher degrees of freedom (RDF). Open Science practices originally developed for empirical research have therefore been proposed as potential remedies. In this paper, we propose a template for transparently documenting the entire life-cycle of a simulation study. It is structured in three parts: (1) a Statement of Intent that can be used to specify and preregister hypotheses and core design choices early in the process; (2) a Piloting Documentation that records all explored simulation settings, including any choices not selected for the final study; and (3) a Final Documentation that supports honest reporting and reproducibility of the published results. Together, these components aim to improve the planning, documentation, and reporting of simulation studies. We motivate the template by discussing parallels and differences between empirical and simulation research. We argue that while exploitable RDF in empirical studies arise primarily during data analysis, simulation study RDF appear primarily during the design and piloting stages. Documenting all explored conditions therefore helps to make decisions in simulation research more transparent.

  • Spower: A general-purpose Monte Carlo simulation power analysis program

    by R. Philip Chalmers

    This article presents the software Spower, an R package designed as a general-purpose Monte Carlo simulation experiment tool to perform power analyses. The


  • Open Access

    Spower: A General-Purpose Monte Carlo Simulation Power Analysis Program

    by Robert Philip Chalmers

    This article present the software Spower, an R package designed as a general-purpose, Monte Carlo simulation experiment tool to perform power analyses with a unified API. The package includes complete customization capabilities with support for five distinct (expected) power analysis criteria (prospective/post-hoc, a priori, compromise, sensitivity, and criterion analysis), each of which reports the sampling uncertainty associated with the resulting estimates. Researchers may choose to define their own population generating and analysis function for their tailored simulation experiments, or may choose from a selection of the predefined simulation experiments defined within the package. To facilitate comparability and further extensibility, simulation counterparts of the subroutines from the popular stand-alone software G*Power 3.1 (Faul et al., 2009) are included within the package, along with other simulation experiment subroutines.

  • Open Access

    Including Empirical Prior Information in the Reliable Change Index

    by R. Philip Chalmers, Sarah Campbell

    The reliable change index (RCI; Jacobson & Truax, 1991) is commonly used to assess whether individuals have changed across two measurement occasions, and has seen many augmentations and improvements since its initial conception. In this study, we extend an item response theory version of the RCI presented by Jabrayilov et al. (2016) by including empirical priors in the associated RCI computations whenever group-level differences are quantifiable given post-test response information. Based on a reanalysis and extension of a previous simulation study, we demonstrate that although a small amount of bias is added to the estimates of the latent trait differences when no true change is present, including empirical prior information will generally improve the Type I behavior of the model-based RCI. Consequently, when non-zero changes in the latent trait are present the bias and sampling variability are show to be more favorable than competing estimators, subsequently leading to an increase in power to detect non-zero changes.


  • Spower: Power Analyses using Monte Carlo Simulations

    by Phil Chalmers

    Provides a general purpose simulation-based power analysis API for routine and customized simulation experimental designs. The package focuses exclusively on Monte Carlo simulation variants of (expected) prospective power analyses, criterion power analyses, compromise power analyses, sensitivity analyses, and prospective/post-hoc power analyses. The default simulation experiment functions found within the package provide stochastic variants of the power analyses subroutines found in the G*Power 3 software (Faul, Erdfelder, Buchner, and Lang, 2009) , along with various other power analysis examples (e.g., mediation analyses). Supporting functions are also included, such as for building empirical power curve estimates, which utilize a similar API structure.

  • Solving variables with Monte Carlo simulation experiments: A stochastic root-solving approach.

    by R. Philip Chalmers

    Despite their popularity and flexibility, questions remain regarding how to optimally solve particular unknown variables of interest using Monte Carlo simulation experiments. This article reviews two common approaches based on either performing deterministic iterative searches with noisy objective functions or by constructing interpolation estimates given fitted surrogate functions, highlighting the inefficiencies and inferential concerns of both methods. To address these limitations, and to fill a gap in existing Monte Carlo experimental methodology, a novel algorithm termed the probabilistic bisection algorithm with bolstering and interpolations (ProBABLI) is presented with the goal providing efficient, consistent, and unbiased estimates (with associated confidence intervals) for the stochastic root equations found in Monte Carlo simulation research. Properties of the ProBABLI approach are demonstrated using practical sample size planning applications for independent samples t tests and structural equation models given target power rates, precision criteria, and expected power functions that incorporate prior beliefs. (PsycInfo Database Record (c) 2024 APA, all rights reserved).


  • Multi-Group Generalizations of SIBTEST and Crossing-SIBTEST

    by R. Philip Chalmers, Guoguo Zheng

    This article presents generalizations of SIBTEST and crossing-SIBTEST statistics for differential item functioning (DIF) investigations involving more than two groups. After reviewing the original two-group setup for these statistics, a set of multigroup generalizations that support contrast matrices for joint tests of DIF are presented. To investigate the Type I error and power behavior of these generalizations, a Monte Carlo simulation study was then explored. Results indicated that the proposed generalizations are reasonably effective at recovering their respective population parameter definitions, maintain optimal Type I error control, have suitable power to detect uniform and non-uniform DIF, and in shorter tests are competitive with the generalized logistic regression and generalized Mantel–Haenszel tests for DIF.


  • Open Access

    The Impact of Measurement Model Misspecification on Coefficient Omega Estimates of Composite Reliability

    by Stephanie M. Bell, R. Philip Chalmers, David B. Flora

    Coefficient omega indices are model-based composite reliability estimates that have become increasingly popular. A coefficient omega index estimates how reliably an observed composite score measures a target construct as represented by a factor in a factor-analysis model; as such, the accuracy of omega estimates is likely to depend on correct model specification. The current paper presents a simulation study to investigate the performance of omega-unidimensional (based on the parameters of a one-factor model) and omega-hierarchical (based on a bifactor model) under correct and incorrect model misspecification for high and low reliability composites and different scale lengths. Our results show that coefficient omega estimates are unbiased when calculated from the parameter estimates of a properly specified model. However, omega-unidimensional produced positively biased estimates when the population model was characterized by unmodeled error correlations or multidimensionality, whereas omega-hierarchical was only slightly biased when the population model was either a one-factor model with correlated errors or a higher-order model. These biases were higher when population reliability was lower and increased with scale length. Researchers should carefully evaluate the feasibility of a one-factor model before estimating and reporting omega-unidimensional.


  • Open Access

    Using the Triad Task as a Measure of Thought Style: A Validation Study

    by Rui Zhang, Jolynn Pek, Jessica Kay Flake, Robert Philip Chalmers

    Initially developed to measure object categorization, the triad task has been adapted for use in different research fields and exhibits much variability in how it is implemented but limited validity evidence in our review of the literature. We report on a validation study of a common variant of the task as a measure of thought style in cultural psychology, following the three phases of validity (Loevinger, 1957). With the use of item response theory, we found support for a unidimensional construct in one American and one Chinese sample, but the measure showed other suboptimal psychometric properties. We subsequently improved the measure via item development, instruction modification, and invariance testing in three additional American samples. As a result, we found the expected within-culture differences. Buttressing the validity evidence of the triad measure reduces the threat of measurement-related alternative explanations and thus facilitates more valid group comparisons.