MSMeagan Sundstrom

Meagan Sundstrom

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Meagan Sundstrom has a total of 34 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Measuring fidelity of implementation of named active learning methods in physics

    by Ibukunoluwa Bukola, Meagan Sundstrom, Justin Gambrell, Colin Green, Adrienne L. Traxler, Eric Brewe

    Various active learning methods have been developed for introductory physics, and these methods are increasingly being adopted by instructors. However, instructors often do not implement these methods exactly as was originally intended by the developers, as they may face issues related to funding and institutional support for active learning and/or have different instructional contexts (e.g., student populations) and environments (e.g., physical classroom layouts) than the developers. Existing research does not sufficiently capture the range of variation in instructor implementation of established active learning methods, especially in comparison with high-fidelity implementations. In this study, we first identify the critical components (i.e., components without which the active learning method cannot be said to have been implemented) of three named active learning methods: Student-Centered Active Learning Environment with Upside-Down Pedagogies (SCALE-UP), Investigative Science Learning Environment (ISLE), and Tutorials. We then evaluate the fidelity with which 18 different introductory physics instructors implement these methods by analyzing classroom observations and comparing the extent to which these broader implementations use each critical component in their classroom to high-fidelity implementations. We find across all three active learning methods that broader implementations and high-fidelity implementations spend similar amounts of class time on critical components. At the same time, we observe substantial variation in the sequences of activities through which broader implementers operationalize these critical components (e.g., doing a few long activities versus many short activities). Finally, we find no clear relationship between fidelity of implementation (as measured by student and instructor behaviors during class) and student conceptual learning gains for our study’s sample of instructors.

  • Evaluating scientific impact

    by Ian F. Olivant, Eric T. Brewe, Meagan Sundstrom

    In this work, we investigate the effect of a large National Science Foundation (NSF) research grant on the impact of research output of physics faculty. Using 169 Faculty Early Career Development Program (CAREER) awardees in physics as a sample group, and a set of 351 associate professors at R1 institutions in physics as our comparison group, we calculated the mean Network-Normalized Citation Index (Ĉ) for all papers from each group published since 1995. Using a single-covariate propensity score matching technique, we matched mean Ĉ scores across awardee-comparison pairs of faculty in the five years prior to award reception to establish a baseline. We assessed the impact of research output post-award by comparing mean Ĉ in our pairs in the five years post-award. With 153 pairs in the full matched sample and 82 in a robust sample, we used bootstrap analysis to generate histograms of Ĉ that can be used to ascertain statistical significance. While our groups are likely not similar enough to make any broad claims about the effect of funding, our results show no statistically significant difference in Ĉ between the awardees and the comparison faculty (mean difference of 0.052±0.12 for the robust sample). We hope that an ongoing study alongside the Moore Foundation will help to remove group differences, increase our sample size, and confirm or reject this finding.

  • Open Access

    Predictive Modeling for High Impact Active Learning Classrooms

    by Olive Ross, Meagan Sundstrom, N. Holmes

    Over the past several decades, a large body of research has shown that undergraduate science students learn more and more equitably in active learning classrooms; however, the term “active learning” lacks definition and little research has examined which types and combinations of active learning strategies are most effective. In this study, we use a dataset representing over 10,000 students and 24 institutions to create a predictive model that maps classroom time spent on different activities to student conceptual learning. We find that four variables -- classroom time spent on lecture, group worksheets, clicker questions, and student questions -- are sufficient to reliably predict student learning, as measured by concept inventory scores. We identify one type of class that consistently demonstrates exceptional student learning gains (effect sizes greater than 2): those that spend 10-20\% of class time on group worksheets, 20-40\% of class time on group clicker questions, and average two or more student questions per hour of class time. We also find that classes which do not utilize group worksheets consistently have learning outcomes comparable to fully lecture classes. These results provide testable recommendations for future controlled studies to investigate effective active learning implementation in undergraduate physics courses.

  • Open Access

    Beyond named methods: A typology of active learning based on classroom observation networks

    by Meagan Sundstrom, Justin Gambrell, Colin Green, Adrienne L. Traxler, Eric Brewe

    A growing number of introductory physics instructors are implementing active learning methods in their classrooms, and they are modifying these methods to fit their local instructional contexts. However, we lack a detailed framework for describing the range of what these instructor adaptations of active learning methods look like in practice. Existing studies apply structured protocols to classroom observations and report descriptive statistics (e.g., the fraction of class time spent on each activity), but this approach overlooks the complex nature of instruction. In this study, we apply network analysis techniques to classroom observations to define a typology of active learning that considers the temporal and interactional nature of instructional practices. We analyze video data from 30 instructors at 27 institutions who implemented one of the following named active learning methods in their introductory physics or astronomy courses: Investigative Science Learning Environment (ISLE), Peer Instruction, Tutorials, and Student-Centered Active Learning Environment with Upside-down Pedagogies (SCALE-UP). We first create one classroom observation network per instructor that captures temporal sequences of activities measured using the Classroom Observation Protocol for Undergraduate STEM (COPUS). We then calculate the cosine similarity between all pairs of observation networks, create a similarity network where each instructor’s classroom observation network is a node and the edges represent cosine similarity, and apply the Infomap clustering algorithm to the similarity network to identify types of active learning instruction. We find five types of instruction: clicker lecture, dialogic clicker lecture, dialogic lecture with short groupwork activities, short groupwork activities, and long groupwork activities. We find no significant relationship between these instruction types and the named active learning methods; instead, implementations of each of the four methods are spread across different instruction types. This result prompts a shift in the way we discuss and study active learning: the names of developed active learning methods may not actually reflect classroom instruction. We also find that student conceptual learning does not vary across the identified instruction types, suggesting that instructors may be flexible in the style in which they implement these methods without sacrificing effectiveness. We discuss the implications of these results for both research and the professional development of college physics instructors.


  • Relative benefits of different active learning methods to conceptual physics learning

    by Meagan Sundstrom, Justin Gambrell, Colin Green, Adrienne L. Traxler, Eric Brewe


  • Open Access

    Multi-institutional assessment of Peer Instruction implementation and impacts using the Framework for Interactive Learning in Lectures

    by Ibukunoluwa Bukola, Meagan Sundstrom, Justin Gambrell, Olive Ross, Adrienne L. Traxler, Eric Brewe

    Substantial research indicates that active learning methods improve student learning more than traditional lecturing. Accordingly, current studies aim to characterize and evaluate different instructors' implementations of active learning methods. Peer Instruction is one of the most commonly used active learning methods in undergraduate physics instruction and typically involves the use of classroom response systems (e.g., clickers) where instructors pose conceptual questions that students answer individually and/or in collaboration with nearby peers. Several research studies have identified that different instructors vary in the ways they implement Peer Instruction (e.g., the time they give students to answer a question and the time they spend explaining the correct answer); however, these studies only take place at a single institution and do not relate the implementation of Peer Instruction to student learning. In this study, we analyze variation in both the implementation and impacts of Peer Instruction. We use classroom video observations and conceptual inventory data from seven introductory physics instructors across six U.S. institutions. We characterize implementation using the Framework for Interactive Learning in Lectures (FILL+), which classifies classroom activities as interactive (e.g., clicker questions), vicarious interactive (e.g., individual students asking a question), or non-interactive (e.g., instructor lecturing). Our preliminary results suggest that instructors who use both interactive and vicarious interactive strategies may exhibit larger student learning gains than instructors who predominantly use only one of the two strategies.


  • Open Access

    Evaluating recognition and recall formats of social network surveys in physics education research

    by Meagan Sundstrom, Justin Gambrell, Adrienne L. Traxler, Eric Brewe

    An increasing number of studies in physics education research use social network analysis to quantify interactions among students. These studies typically gather data through online surveys using one of two different survey formats: recognition, where students select peers' names from a provided course roster, and recall, where students type their peers' names from memory as an open response. These survey formats, however, may be subject to two possible systematic errors. First, students may report more peers' names on a recognition survey than a recall survey because the course roster facilitates their memory of their interactions, whereas they may only remember a subset of their interactions on the recall format. Second, recognition surveys may be subject to name order effects, where students are more likely to select peers' names that appear early on in the roster than those that appear later on (e.g., due to survey fatigue). Here we report the results of two methodological studies of these possible errors in the context of introductory physics courses: one directly comparing 65 student responses to recognition and recall versions of the same network survey prompt, and the other measuring name order effects on 54 recognition surveys from 27 different courses. We find that students may report more peer interactions on a recognition survey than a recall survey and that most recognition surveys are not subject to significant name order effects. These results help to inform survey design for future network studies in physics education research.

  • Open Access

    Advancing clustering methods in physics education research: A case for mixture models

    by Minghui Wang, Meagan Sundstrom, Karen Nylund-Gibson, Marsha Ing

    Clustering methods are often used in physics education research (PER) to identify subgroups of individuals within a population who share similar response patterns or characteristics. Among these, k -means (or k -modes, for categorical data) is one of the most commonly used clustering methods in PER. This algorithm, however, is distance-based rather than model-based: it relies on algorithmic partitioning and assigns each individual to one subgroup through hard assignment. Researchers must also conduct analyses to relate subgroup membership to other variables. Mixture models are a model-based alternative that offers several statistical tools for choosing the optimal number of subgroups, accounts for classification errors by assigning individuals probabilities of belonging to each subgroup rather than hard assignment, and allows researchers to directly integrate subgroup membership into a broader latent variable framework. In this paper, we outline the theoretical similarities and differences between k -modes clustering and latent class analysis (one type of mixture model for categorical data). We also present parallel analyses using each method to address the same research questions in order to demonstrate these similarities and differences. We provide the data and code to replicate the worked example presented in the paper for researchers interested in using mixture models.


  • Open Access

    Applying machine learning models in multi-institutional studies can generate bias

    by Rebeckah K. Fussell, Meagan Sundstrom, Sabrina McDowell, N. G. Holmes

    There is increasing interest in deploying machine learning models at scale for multi-institutional studies in physics education research. Here we investigate the efficacy of applying machine learning models to institutions outside of their training set, using natural language processing to code open-ended survey responses. We find that, in general, changing institutional contexts can affect machine learning estimates of code frequencies: either previously documented sources of uncertainty increase in magnitude, new unknown sources of uncertainty emerge, or both. We also find an example where uncertainties do not change between the institution used in the training data and an institution not in the training data. Results suggest that attention to uncertainty is critical, especially when making measurements of student writing across multi-institutional data sets.