Christopher J. Urban
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Christopher J. Urban has a total of 15 co-authors. Here are the top co-authors by number of shared publications:
Daniel J. Bauer
Department of Psychology Laboratory Neuroscience University of North Carolina at Chapel Hill
2 shared publicationsKathleen M. Gates
Department of Psychology & Neuroscience University of North Carolina at Chapel Hill Chapel Hill North Carolina USA
2 shared publications- Abigail T. Panter
University of North Carolina at Chapel Hill
1 shared publication - Augustin Kelava
Eberhard Karls Universität Tübingen
1 shared publication - Chaewon Lee
Department of Psychology and Neuroscience, University of North Carolina at Chapel Hill, Chapel Hill, USA
1 shared publication - Daniel Leyhr
Eberhard Karls University of Tübingen
1 shared publication
Latest Publications
- 3 May 2024
- Published by Wiley in British Journal of Educational Technology
- Vol. 55 (5) , 1860-1883
Co‐designing enduring learning analytics prediction and support tools in undergraduate biology courses
Even highly motivated undergraduates drift off their STEM career pathways. In large introductory STEM classes, instructors struggle to identify and support these students. To address these issues, we developed co‐redesign methods in partnership with disciplinary experts to create high‐structure STEM courses that better support students and produce informative digital event data. To those data, we applied theory‐ and context‐relevant labels to reflect active and self‐regulated learning processes involving LMS‐hosted course materials, formative assessments, and help‐seeking tools. We illustrate the predictive benefits of this process across two cycles of model creation and reapplication. In cycle 1, we used theory‐relevant features from 3 weeks of data to inform a prediction model that accurately identified struggling students and sustained its accuracy when reapplied in future semesters. In cycle 2, we refit a model with temporally contextualized features that achieved superior accuracy using data from just two class meetings. This modelling approach can produce durable learning analytics solutions that afford scaled and sustained prediction and intervention opportunities that involve explainable artificial intelligence products. Those same products that inform prediction can also guide intervention approaches and inform future instructional design and delivery. Practitioner notes: What is already known about this topic: Learning analytics includes an evolving collection of methods for tracing and understanding student learning through their engagements with learning technologies. Prediction models based on demographic data can perpetuate systemic biases. Prediction models based on behavioural event data can produce accurate predictions of academic success, and validation efforts can enrich those data to reflect students' self‐regulated learning processes within learning tasks. What this paper adds: Learning analytics can be successfully applied to predict performance in an authentic postsecondary STEM context, and the use of context and theory as guides for feature engineering can ensure sustained predictive accuracy upon reapplication. The consistent types of learning resources and cyclical nature of their provisioning from lesson to lesson are hallmarks of high‐structure active learning designs that are known to benefit learners. These designs also provide opportunities for observing and modelling contextually grounded, theory‐aligned and temporally positioned learning events that informed prediction models that accurately classified students upon initial and later reapplications in subsequent semesters. Co‐design relationships where researchers and instructors work together toward pedagogical implementation and course instrumentation are essential to developing unique insights for feature engineering and producing explainable artificial intelligence approaches to predictive modelling. Implications for practice and/or policy: High‐structure course designs can scaffold student engagement with course materials to make learning more effective and products of feature engineering more explainable. Learning analytics initiatives can avoid perpetuation of systemic biases when methods prioritize theory‐informed behavioural data that reflect learning processes, sensitivity to instructional context and development of explainable predictors of success rather than relying on students' demographic characteristics as predictors. Prioritizing behaviours as predictors improves explainability in ways that can inform the redesign of courses and design of learning supports, which further informs the refinement of learning theories and their applications.
- 3 May 2024
- Published by Wiley in British Journal of Educational Technology
- Vol. 55 (5) , 1860-1883
Co‐designing enduring learning analytics prediction and support tools in undergraduate biology courses
Even highly motivated undergraduates drift off their STEM career pathways. In large introductory STEM classes, instructors struggle to identify and support these students. To address these issues, we developed co‐redesign methods in partnership with disciplinary experts to create high‐structure STEM courses that better support students and produce informative digital event data. To those data, we applied theory‐ and context‐relevant labels to reflect active and self‐regulated learning processes involving LMS‐hosted course materials, formative assessments, and help‐seeking tools. We illustrate the predictive benefits of this process across two cycles of model creation and reapplication. In cycle 1, we used theory‐relevant features from 3 weeks of data to inform a prediction model that accurately identified struggling students and sustained its accuracy when reapplied in future semesters. In cycle 2, we refit a model with temporally contextualized features that achieved superior accuracy using data from just two class meetings. This modelling approach can produce durable learning analytics solutions that afford scaled and sustained prediction and intervention opportunities that involve explainable artificial intelligence products. Those same products that inform prediction can also guide intervention approaches and inform future instructional design and delivery. Practitioner notes: What is already known about this topic: Learning analytics includes an evolving collection of methods for tracing and understanding student learning through their engagements with learning technologies. Prediction models based on demographic data can perpetuate systemic biases. Prediction models based on behavioural event data can produce accurate predictions of academic success, and validation efforts can enrich those data to reflect students' self‐regulated learning processes within learning tasks. What this paper adds: Learning analytics can be successfully applied to predict performance in an authentic postsecondary STEM context, and the use of context and theory as guides for feature engineering can ensure sustained predictive accuracy upon reapplication. The consistent types of learning resources and cyclical nature of their provisioning from lesson to lesson are hallmarks of high‐structure active learning designs that are known to benefit learners. These designs also provide opportunities for observing and modelling contextually grounded, theory‐aligned and temporally positioned learning events that informed prediction models that accurately classified students upon initial and later reapplications in subsequent semesters. Co‐design relationships where researchers and instructors work together toward pedagogical implementation and course instrumentation are essential to developing unique insights for feature engineering and producing explainable artificial intelligence approaches to predictive modelling. Implications for practice and/or policy: High‐structure course designs can scaffold student engagement with course materials to make learning more effective and products of feature engineering more explainable. Learning analytics initiatives can avoid perpetuation of systemic biases when methods prioritize theory‐informed behavioural data that reflect learning processes, sensitivity to instructional context and development of explainable predictors of success rather than relying on students' demographic characteristics as predictors. Prioritizing behaviours as predictors improves explainability in ways that can inform the redesign of courses and design of learning supports, which further informs the refinement of learning theories and their applications.
- Open Access
- 30 June 2023
- Published by Wiley in Statistical Analysis and Data Mining: An ASA Data Science Journal
- Vol. 16 (5) , 474-487
A deep learning factor analysis model based on importance‐weighted variational inference and normalizing flow priors: Evaluation within a set of multidimensional performance assessments in youth elite soccer players
- 30 June 2023
- Published by Wiley in Statistical Analysis and Data Mining: An ASA Data Science Journal
- Vol. 16 (5) , 474-487
Open AccessA deep learning factor analysis model based on importance‐weighted variational inference and normalizing flow priors: Evaluation within a set of multidimensional performance assessments in youth elite soccer players
- Open Access
Deep Learning-Based Estimation and Goodness-of-Fit for Large-Scale Confirmatory Item Factor Analysis
We investigate novel parameter estimation and goodness-of-fit (GOF) assessment methods for large-scale confirmatory item factor analysis (IFA) with many respondents, items, and latent factors. For parameter estimation, we extend Urban and Bauer's (2021) deep learning algorithm for exploratory IFA to the confirmatory setting by showing how to handle constraints on loadings and factor correlations. For GOF assessment, we explore simulation-based tests and indices that extend the classifier two-sample test (C2ST), a method that tests whether a deep neural network can distinguish between observed data and synthetic data sampled from a fitted IFA model. Proposed extensions include a test of approximate fit wherein the user specifies what percentage of observed and synthetic data should be distinguishable as well as a relative fit index (RFI) that is similar in spirit to the RFIs used in structural equation modeling. Via simulation studies, we show that: (1) the confirmatory extension of Urban and Bauer's (2021) algorithm obtains comparable estimates to a state-of-the-art estimation procedure in less time; (2) C2ST-based GOF tests control the empirical type I error rate and detect when the latent dimensionality is misspecified; and (3) the sampling distribution of the C2ST-based RFI depends on the sample size.
Open AccessDeep Learning-Based Estimation and Goodness-of-Fit for Large-Scale Confirmatory Item Factor Analysis
We investigate novel parameter estimation and goodness-of-fit (GOF) assessment methods for large-scale confirmatory item factor analysis (IFA) with many respondents, items, and latent factors. For parameter estimation, we extend Urban and Bauer's (2021) deep learning algorithm for exploratory IFA to the confirmatory setting by showing how to handle constraints on loadings and factor correlations. For GOF assessment, we explore simulation-based tests and indices that extend the classifier two-sample test (C2ST), a method that tests whether a deep neural network can distinguish between observed data and synthetic data sampled from a fitted IFA model. Proposed extensions include a test of approximate fit wherein the user specifies what percentage of observed and synthetic data should be distinguishable as well as a relative fit index (RFI) that is similar in spirit to the RFIs used in structural equation modeling. Via simulation studies, we show that: (1) the confirmatory extension of Urban and Bauer's (2021) algorithm obtains comparable estimates to a state-of-the-art estimation procedure in less time; (2) C2ST-based GOF tests control the empirical type I error rate and detect when the latent dimensionality is misspecified; and (3) the sampling distribution of the C2ST-based RFI depends on the sample size.
- Open Access
- 1 November 2022
- Published by Center for Open Science in PsyArXiv
An Evaluation of Deep Learning Approaches for Factor Analysis of Response and Response Time Data
An important problem in the application of psychometric models is the selection of suitable algorithms for parameter estimation. In a recent publication, Urban and Bauer (Psychometrika 86:1-29, 2021) proposed an estimation algorithm based on deep learning for item parameter estimation for large sample sizes. In this article, we investigate the accuracy of an application of this estimation method for a multidimensional variation of the response time model proposed by Klein Entink, Fox and van der Linden (Psychometrika 74:21-48, 2009) and a joint factor model for responses and response times. The second of these models can be seen as a variation of the joint model for responses and response times of Klein Entink, Fox and van der Linden. Our evaluation focuses on comparing the effects of different sets of hyperparameters on the accuracy of the estimation method. Our results indicate that the deep learning algorithm can be used for an accurate item parameter estimation for these models even in relatively small datasets and is computationally fast in large datasets, but that its accuracy depends on the selection of suitable hyperparameters, such as the learning rate. The evaluated algorithm is freely available in a Python package. We also describe an empirical application of this method to the Amsterdam Chess Test dataset.
- 1 November 2022
- Published by Center for Open Science in PsyArXiv
Open AccessAn Evaluation of Deep Learning Approaches for Factor Analysis of Response and Response Time Data
An important problem in the application of psychometric models is the selection of suitable algorithms for parameter estimation. In a recent publication, Urban and Bauer (Psychometrika 86:1-29, 2021) proposed an estimation algorithm based on deep learning for item parameter estimation for large sample sizes. In this article, we investigate the accuracy of an application of this estimation method for a multidimensional variation of the response time model proposed by Klein Entink, Fox and van der Linden (Psychometrika 74:21-48, 2009) and a joint factor model for responses and response times. The second of these models can be seen as a variation of the joint model for responses and response times of Klein Entink, Fox and van der Linden. Our evaluation focuses on comparing the effects of different sets of hyperparameters on the accuracy of the estimation method. Our results indicate that the deep learning algorithm can be used for an accurate item parameter estimation for these models even in relatively small datasets and is computationally fast in large datasets, but that its accuracy depends on the selection of suitable hyperparameters, such as the learning rate. The evaluated algorithm is freely available in a Python package. We also describe an empirical application of this method to the Amsterdam Chess Test dataset.
- 1 December 2021
- Published by American Psychological Association (APA) in Psychological Methods
- Vol. 26 (6) , 743-773
Deep learning: A primer for psychologists.
Deep learning has revolutionized predictive modeling in topics such as computer vision and natural language processing but is not commonly applied to psychological data. In an effort to bring the benefits of deep learning to psychologists, we provide an overview of deep learning for researchers who have a working knowledge of linear regression. We first discuss several benefits of the deep learning approach to predictive modeling. We then present three basic deep learning models that generalize linear regression: the feedforward neural network (FNN), the recurrent neural network (RNN), and the convolutional neural network (CNN). We include concrete toy examples with R code to demonstrate how each model may be applied to answer prediction-focused research questions using common data types collected by psychologists. (PsycInfo Database Record (c) 2022 APA, all rights reserved).
- 1 December 2021
- Published by American Psychological Association (APA) in Psychological Methods
- Vol. 26 (6) , 743-773
Deep learning: A primer for psychologists.
Deep learning has revolutionized predictive modeling in topics such as computer vision and natural language processing but is not commonly applied to psychological data. In an effort to bring the benefits of deep learning to psychologists, we provide an overview of deep learning for researchers who have a working knowledge of linear regression. We first discuss several benefits of the deep learning approach to predictive modeling. We then present three basic deep learning models that generalize linear regression: the feedforward neural network (FNN), the recurrent neural network (RNN), and the convolutional neural network (CNN). We include concrete toy examples with R code to demonstrate how each model may be applied to answer prediction-focused research questions using common data types collected by psychologists. (PsycInfo Database Record (c) 2022 APA, all rights reserved).
- 1 March 2021
- Published by Cambridge University Press (CUP) in Psychometrika
- Vol. 86 (1) , 1-29
A Deep Learning Algorithm for High-Dimensional Exploratory Item Factor Analysis
Marginal maximum likelihood (MML) estimation is the preferred approach to fitting item response theory models in psychometrics due to the MML estimator’s consistency, normality, and efficiency as the sample size tends to infinity. However, state-of-the-art MML estimation procedures such as the Metropolis–Hastings Robbins–Monro (MH-RM) algorithm as well as approximate MML estimation procedures such as variational inference (VI) are computationally time-consuming when the sample size and the number of latent factors are very large. In this work, we investigate a deep learning-based VI algorithm for exploratory item factor analysis (IFA) that is computationally fast even in large data sets with many latent factors. The proposed approach applies a deep artificial neural network model called an importance-weighted autoencoder (IWAE) for exploratory IFA. The IWAE approximates the MML estimator using an importance sampling technique wherein increasing the number of importance-weighted (IW) samples drawn during fitting improves the approximation, typically at the cost of decreased computational efficiency. We provide a real data application that recovers results aligning with psychological theory across random starts. Via simulation studies, we show that the IWAE yields more accurate estimates as either the sample size or the number of IW samples increases (although factor correlation and intercepts estimates exhibit some bias) and obtains similar results to MH-RM in less time. Our simulations also suggest that the proposed approach performs similarly to and is potentially faster than constrained joint maximum likelihood estimation, a fast procedure that is consistent when the sample size and the number of items simultaneously tend to infinity.
- 1 March 2021
- Published by Cambridge University Press (CUP) in Psychometrika
- Vol. 86 (1) , 1-29
A Deep Learning Algorithm for High-Dimensional Exploratory Item Factor Analysis
Marginal maximum likelihood (MML) estimation is the preferred approach to fitting item response theory models in psychometrics due to the MML estimator’s consistency, normality, and efficiency as the sample size tends to infinity. However, state-of-the-art MML estimation procedures such as the Metropolis–Hastings Robbins–Monro (MH-RM) algorithm as well as approximate MML estimation procedures such as variational inference (VI) are computationally time-consuming when the sample size and the number of latent factors are very large. In this work, we investigate a deep learning-based VI algorithm for exploratory item factor analysis (IFA) that is computationally fast even in large data sets with many latent factors. The proposed approach applies a deep artificial neural network model called an importance-weighted autoencoder (IWAE) for exploratory IFA. The IWAE approximates the MML estimator using an importance sampling technique wherein increasing the number of importance-weighted (IW) samples drawn during fitting improves the approximation, typically at the cost of decreased computational efficiency. We provide a real data application that recovers results aligning with psychological theory across random starts. Via simulation studies, we show that the IWAE yields more accurate estimates as either the sample size or the number of IW samples increases (although factor correlation and intercepts estimates exhibit some bias) and obtains similar results to MH-RM in less time. Our simulations also suggest that the proposed approach performs similarly to and is potentially faster than constrained joint maximum likelihood estimation, a fast procedure that is consistent when the sample size and the number of items simultaneously tend to infinity.