Bryan Keller
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Bryan Keller has a total of 23 co-authors. Here are the top co-authors by number of shared publications:
Zach Branson
Department of Statistics and Data Science, Carnegie Mellon University, Pittsburgh, USA
2 shared publications- Diego Luna-Bazaldua
The World Bank Group
1 shared publication - Elizabeth Tipton
Northwestern University
1 shared publication - James C. Borders
Department of Speech, Language & Hearing Sciences, Boston University, MA
1 shared publication - Jee-Seon Kim
Department of Educational Psychology, University of Wisconsin, Madison, Wisconsin, USA
1 shared publication Jessica E. Huber
Speech Physiology Lab, Department of Communicative Disorders and Sciences University at Buffalo Buffalo New York USA
1 shared publication
Latest Publications
- Open Access
- 17 September 2026
- Published by Center for Open Science in PsyArXiv
Estimating Effects of Time-Varying Special Education Regimes Using Targeted Maximum Likelihood
Longitudinal treatments are common in social science research but are often reduced to ever-versus-never exposure indicators that obscure their time-varying nature. This study shows how causal questions can be clarified by defining treatments through explicit intervention regimes rather than collapsed summaries. We introduce this framework by linking the longitudinal setting to single-time-point causal inference and extending it to time-varying exposures, sequential identification assumptions, and estimation using targeted maximum likelihood estimation (TMLE). We also address practical challenges, including missing data and positivity violations. Using data from the Early Childhood Longitudinal Study, Kindergarten Class of 1998-1999 (ECLS-K), we examine the effect of special education participation on 5th-grade mathematics achievement. Comparing static and dynamic regimes, we find meaningful differences: always-treated versus never-treated contrasts yield the largest negative estimates, while rule-based dynamic regimes produce substantially smaller effects. These results highlight the importance of well-defined interventions for interpreting longitudinal treatment effects.
- 17 September 2026
- Published by Center for Open Science in PsyArXiv
Open AccessEstimating Effects of Time-Varying Special Education Regimes Using Targeted Maximum Likelihood
Longitudinal treatments are common in social science research but are often reduced to ever-versus-never exposure indicators that obscure their time-varying nature. This study shows how causal questions can be clarified by defining treatments through explicit intervention regimes rather than collapsed summaries. We introduce this framework by linking the longitudinal setting to single-time-point causal inference and extending it to time-varying exposures, sequential identification assumptions, and estimation using targeted maximum likelihood estimation (TMLE). We also address practical challenges, including missing data and positivity violations. Using data from the Early Childhood Longitudinal Study, Kindergarten Class of 1998-1999 (ECLS-K), we examine the effect of special education participation on 5th-grade mathematics achievement. Comparing static and dynamic regimes, we find meaningful differences: always-treated versus never-treated contrasts yield the largest negative estimates, while rule-based dynamic regimes produce substantially smaller effects. These results highlight the importance of well-defined interventions for interpreting longitudinal treatment effects.
- Open Access
- 1 January 2025
- Published by Project MUSE in Observational Studies
- Vol. 11 (2) , 153-188
A new four-arm within-study comparison: Design, implementation, and data
Within-study comparisons (WSCs) use real, rather than simulated, data to compare estimates from observational studies against benchmarks from randomized controlled trials (RCTs). A primary goal of WSCs is to assess whether well-designed quasi-experimental designs (QEDs) can produce internally valid causal effect estimates comparable to those from RCTs. In this paper, we describe the design and implementation of a new type of WSC. Motivated by Shadish et al. (2008), we examine the impact of a mathematics training intervention and a vocabulary study session on posttest scores for mathematics and vocabulary, respectively. We extend the design in three ways. First, before random assignment, we ask participants to express a preference for either the mathematics or vocabulary training session, after which they are randomly assigned regardless of preferences. This allows us to experimentally identify and estimate the overall average treatment effect (ATE) and two conditional ATEs: the average treatment effect on the treated (ATT) and the average treatment effect on the untreated (ATU). Second, participant recruitment and sample size (N = 2200) were determined through power analyses for comparing RCT and QED estimates, ensuring sufficient power for methodological comparisons. Finally, the study's eligibility criteria, recruitment, treatment allocation, and analysis plan were preregistered on the Open Science Foundation platform, and the data are publicly accessible. We believe that this WSC design and the resulting data set will be valuable for researchers seeking to evaluate causal inference methods and test identification assumptions using real-world data.
- 1 January 2025
- Published by Project MUSE in Observational Studies
- Vol. 11 (2) , 153-188
Open AccessA new four-arm within-study comparison: Design, implementation, and data
Within-study comparisons (WSCs) use real, rather than simulated, data to compare estimates from observational studies against benchmarks from randomized controlled trials (RCTs). A primary goal of WSCs is to assess whether well-designed quasi-experimental designs (QEDs) can produce internally valid causal effect estimates comparable to those from RCTs. In this paper, we describe the design and implementation of a new type of WSC. Motivated by Shadish et al. (2008), we examine the impact of a mathematics training intervention and a vocabulary study session on posttest scores for mathematics and vocabulary, respectively. We extend the design in three ways. First, before random assignment, we ask participants to express a preference for either the mathematics or vocabulary training session, after which they are randomly assigned regardless of preferences. This allows us to experimentally identify and estimate the overall average treatment effect (ATE) and two conditional ATEs: the average treatment effect on the treated (ATT) and the average treatment effect on the untreated (ATU). Second, participant recruitment and sample size (N = 2200) were determined through power analyses for comparing RCT and QED estimates, ensuring sufficient power for methodological comparisons. Finally, the study's eligibility criteria, recruitment, treatment allocation, and analysis plan were preregistered on the Open Science Foundation platform, and the data are publicly accessible. We believe that this WSC design and the resulting data set will be valuable for researchers seeking to evaluate causal inference methods and test identification assumptions using real-world data.
- 8 October 2024
- Published by Wiley in Movement Disorders Clinical Practice
- Vol. 11 (12) , 1500-1515
Enhancing Cough Motor Learning in Parkinson's Disease Through Variable Practice During Skill Training
Background: When re‐learning a motor skill, practicing a variety of treatment targets promotes error processing and the exploration of motor control strategies, which initially disrupts accuracy during training (motor performance), but ultimately enhances generalization, retention, and transfer (motor learning). Cough skill training (CST) is feasible and efficacious to improve cough strength; however, previous studies have used the same practice target during training.Objectives: Our goal was to examine the impact of CST with variable practice on motor performance, motor learning, and respiratory system adaptations.Method: The study was a prospective three‐visit single group design. Twenty individuals with Parkinson's disease (PD) and concomitant dysphagia and dystussia completed two sessions of CST involving three randomized practice targets. Cough, lung volume, and airway clearance outcomes were assessed before and after treatment sessions with long‐term retention evaluated after 1 month.Results: Peak expiratory flow rate improved after CST with variable practice for voluntary single (β = 0.35 L/s) and sequential (β = 0.22 L/s) cough, which were maintained after 1 month without treatment. The ability to expel material from the upper airway demonstrated a small magnitude of improvement (β = −1.87%). During CST, participants altered lung volume based on the treatment target and lung volume decreased during reflex cough after completing CST.Conclusions: Individuals with PD demonstrated improvements in several aspects of motor learning after two sessions of CST with variable practice. Increasing lung volume may not be an implicit strategy to upregulate voluntary cough strength in this treatment paradigm. The findings support the need for larger investigations exploring the potential benefits of this CST approach.
- 8 October 2024
- Published by Wiley in Movement Disorders Clinical Practice
- Vol. 11 (12) , 1500-1515
Enhancing Cough Motor Learning in Parkinson's Disease Through Variable Practice During Skill Training
Background: When re‐learning a motor skill, practicing a variety of treatment targets promotes error processing and the exploration of motor control strategies, which initially disrupts accuracy during training (motor performance), but ultimately enhances generalization, retention, and transfer (motor learning). Cough skill training (CST) is feasible and efficacious to improve cough strength; however, previous studies have used the same practice target during training.Objectives: Our goal was to examine the impact of CST with variable practice on motor performance, motor learning, and respiratory system adaptations.Method: The study was a prospective three‐visit single group design. Twenty individuals with Parkinson's disease (PD) and concomitant dysphagia and dystussia completed two sessions of CST involving three randomized practice targets. Cough, lung volume, and airway clearance outcomes were assessed before and after treatment sessions with long‐term retention evaluated after 1 month.Results: Peak expiratory flow rate improved after CST with variable practice for voluntary single (β = 0.35 L/s) and sequential (β = 0.22 L/s) cough, which were maintained after 1 month without treatment. The ability to expel material from the upper airway demonstrated a small magnitude of improvement (β = −1.87%). During CST, participants altered lung volume based on the treatment target and lung volume decreased during reflex cough after completing CST.Conclusions: Individuals with PD demonstrated improvements in several aspects of motor learning after two sessions of CST with variable practice. Increasing lung volume may not be an implicit strategy to upregulate voluntary cough strength in this treatment paradigm. The findings support the need for larger investigations exploring the potential benefits of this CST approach.
- 10 May 2024
- Published by Springer Nature in Asia Pacific Education Review
- Vol. 25 (3) , 575-594
Defining, identifying, and estimating causal effects with the potential outcomes framework: a review for education research
- 10 May 2024
- Published by Springer Nature in Asia Pacific Education Review
- Vol. 25 (3) , 575-594
Defining, identifying, and estimating causal effects with the potential outcomes framework: a review for education research
- Open Access
- 12 October 2023
- Published by Center for Open Science in PsyArXiv
Defining, Identifying, and Estimating Causal Effects with the Potential Outcomes Framework: A Review for Education Research
Causal inference involves determining whether a treatment (e.g., an education program) causes a change in outcomes (e.g., academic achievement). It is well-known that causal effects are more challenging to estimate than associational effects. Over the past 50 years, the potential outcomes framework has become one of the most widely used approaches for defining, identifying, and estimating causal effects. In this paper we review the potential outcomes framework with a focus on potential outcomes notation to define individual and average causal effects. We then show how three canonical assumptions, Unconfoundedness, Positivity, and Consistency, may be used to identify average causal effects. The identification results motivate methods for estimating causal effects in practice, which include model-based estimators, such as regression, inverse probability weighting, and doubly robust estimation, and procedures that target covariate balance, such as matching and stratification. Examples and discussion are grounded in the context of a running example of a study aimed at assessing the causal effect of receipt of special education services on 5th grade mathematics achievement in school-aged children. Practical considerations for education research are discussed.
- 12 October 2023
- Published by Center for Open Science in PsyArXiv
Open AccessDefining, Identifying, and Estimating Causal Effects with the Potential Outcomes Framework: A Review for Education Research
Causal inference involves determining whether a treatment (e.g., an education program) causes a change in outcomes (e.g., academic achievement). It is well-known that causal effects are more challenging to estimate than associational effects. Over the past 50 years, the potential outcomes framework has become one of the most widely used approaches for defining, identifying, and estimating causal effects. In this paper we review the potential outcomes framework with a focus on potential outcomes notation to define individual and average causal effects. We then show how three canonical assumptions, Unconfoundedness, Positivity, and Consistency, may be used to identify average causal effects. The identification results motivate methods for estimating causal effects in practice, which include model-based estimators, such as regression, inverse probability weighting, and doubly robust estimation, and procedures that target covariate balance, such as matching and stratification. Examples and discussion are grounded in the context of a running example of a study aimed at assessing the causal effect of receipt of special education services on 5th grade mathematics achievement in school-aged children. Practical considerations for education research are discussed.
- 18 November 2017
- Published by Springer Nature in Asia Pacific Education Review
- Vol. 18 (4) , 585-598
Assessing the performance of classical test theory item discrimination estimators in Monte Carlo simulations
- 18 November 2017
- Published by Springer Nature in Asia Pacific Education Review
- Vol. 18 (4) , 585-598
Assessing the performance of classical test theory item discrimination estimators in Monte Carlo simulations
- 28 July 2017
- Published by Springer Nature in Journal of Youth and Adolescence
- Vol. 47 (3) , 490-500
An Online Drug Abuse Prevention Program for Adolescent Girls: Posttest and 1-Year Outcomes
Early adolescent girls’ rates of drug use have matched, and in some instances, surpassed boys’ rates. Though girls and boys share risk factors for drug use, girls also have gender-specific risks. Tailored interventions to prevent girls’ drug use are warranted. This study developed and tested a web-based, drug abuse prevention program for adolescent girls. The nationwide sample of 13- and 14-year-old girls (N = 788) was recruited via Facebook ads. Enrolled girls were randomly assigned to the intervention or control condition. All girls completed pretest measures online. Following pretest, intervention girls interacted with the 9-session, gender-specific prevention program online. The program aimed to reduce girls’ drug use and associated risk factors by improving their cognitive and behavioral skills around such areas as coping with stress, managing mood, maintaining a healthy body image, and refusing drug use offers. Girls in both conditions again completed measures at posttest and 1-year follow-up. At posttest, and compared to girls in the control condition, girls who received the intervention smoked fewer cigarettes and reported higher self-esteem, goal setting, media literacy, and self-efficacy. At 1-year follow-up, and compared to girls in the control condition, girls who received the intervention reported engaging in less binge drinking and cigarette smoking; girls assigned to the intervention condition also had higher alcohol, cigarette, and marijuana refusal skills, coping skills, and media literacy and lower rates of peer drug use. This study’s findings support the use of tailored, online drug abuse prevention programming for early adolescent girls.
- 28 July 2017
- Published by Springer Nature in Journal of Youth and Adolescence
- Vol. 47 (3) , 490-500
An Online Drug Abuse Prevention Program for Adolescent Girls: Posttest and 1-Year Outcomes
Early adolescent girls’ rates of drug use have matched, and in some instances, surpassed boys’ rates. Though girls and boys share risk factors for drug use, girls also have gender-specific risks. Tailored interventions to prevent girls’ drug use are warranted. This study developed and tested a web-based, drug abuse prevention program for adolescent girls. The nationwide sample of 13- and 14-year-old girls (N = 788) was recruited via Facebook ads. Enrolled girls were randomly assigned to the intervention or control condition. All girls completed pretest measures online. Following pretest, intervention girls interacted with the 9-session, gender-specific prevention program online. The program aimed to reduce girls’ drug use and associated risk factors by improving their cognitive and behavioral skills around such areas as coping with stress, managing mood, maintaining a healthy body image, and refusing drug use offers. Girls in both conditions again completed measures at posttest and 1-year follow-up. At posttest, and compared to girls in the control condition, girls who received the intervention smoked fewer cigarettes and reported higher self-esteem, goal setting, media literacy, and self-efficacy. At 1-year follow-up, and compared to girls in the control condition, girls who received the intervention reported engaging in less binge drinking and cigarette smoking; girls assigned to the intervention condition also had higher alcohol, cigarette, and marijuana refusal skills, coping skills, and media literacy and lower rates of peer drug use. This study’s findings support the use of tailored, online drug abuse prevention programming for early adolescent girls.
- 1 June 2016
- Published by American Educational Research Association (AERA) in Journal of Educational and Behavioral Statistics
- Vol. 41 (3) , 326-348
Propensity Score Analysis in R
In this article, we review four software packages for implementing propensity score analysis in R: Matching, MatchIt, PSAgraphics, and twang. After briefly discussing essential elements for propensity score analysis, we apply each package to a data set from the Early Childhood Longitudinal Study in order to estimate the average effect of elementary school special education services on math achievement in fifth grade. In the context of this real data example, we evaluate documentation and support resources, built-in quantitative and graphical diagnostic features, and methods available for estimating a causal effect. We conclude by making some recommendations aimed at helping researchers decide which package to turn to based upon their familiarity with propensity score methods, programming in R, and the type of analysis being conducted.
- 1 June 2016
- Published by American Educational Research Association (AERA) in Journal of Educational and Behavioral Statistics
- Vol. 41 (3) , 326-348
Propensity Score Analysis in R
In this article, we review four software packages for implementing propensity score analysis in R: Matching, MatchIt, PSAgraphics, and twang. After briefly discussing essential elements for propensity score analysis, we apply each package to a data set from the Early Childhood Longitudinal Study in order to estimate the average effect of elementary school special education services on math achievement in fifth grade. In the context of this real data example, we evaluate documentation and support resources, built-in quantitative and graphical diagnostic features, and methods available for estimating a causal effect. We conclude by making some recommendations aimed at helping researchers decide which package to turn to based upon their familiarity with propensity score methods, programming in R, and the type of analysis being conducted.
- 1 January 2015
- Published by Springer Nature
- p. 279-291
Neural Networks for Propensity Score Estimation: Simulation Results and Recommendations
- 1 January 2015
- Published by Springer Nature
- p. 279-291
Neural Networks for Propensity Score Estimation: Simulation Results and Recommendations