JLJonathan Liu

Jonathan Liu

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Jonathan Liu has a total of 34 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Exploring the Impact of Gender and Affective Factors on Performance in Discrete Mathematics

    by Erica Goodwin, Jonathan Liu, Max Fowler, Juan Alvarez, Jennifer R. Amos, Benjamin Cosman, Yael Gertner, Diana Franklin

  • Open Access

    Evaluating LLM-Generated Contextualized Algorithm Design Problems

    by Erica Goodwin, Katherine Braught, Jonathan Liu, Dip Kiran Pradhan Newar, Yael Gertner, Seth Poulsen, Diana Franklin

    Background: Context personalization, the practice of adapting learning materials to students’ personal interests, has been shown to increase student learning and engagement. Within computer science education, research has found that LLMs can generate high-quality contextualized introductory programming exercises. Objective: In this paper, we evaluate the capability of LLMs to generate technically correct and thematically integrated contextualized algorithm design problems. Methods: In a series of three iterative studies, we use LLMs to generate contextualized algorithm design problems from a given base problem and theme, evaluating over 500 generated problems for technical and thematic alignment. Results: We find that LLM-generated algorithm design problems exhibit significantly more issues than prior work has found for introductory programming problems. We identify issues specific to the algorithm design context and then mitigate these issues with prompt engineering techniques and model choice. With these adjustments, we produce LLM-generated contextualized algorithm design problems that are technically strong, deeply themed, and largely realistic, though realism drops with more culturally and locally specific themes. Implications: We demonstrate a viable workflow for generating contextualized algorithm design problems using LLMs, including prompt design, model selection, and identification of specific issues to review for.

  • Open Access

    Corrigendum: Teaching Algorithm Design: A Literature Review

    by Jonathan Liu, Seth Poulsen, Erica Goodwin, Hongxuan Chen, Grace Williams, Yael Gertner, Diana Franklin

    This is a corrigendum for the article “Teaching Algorithm Design: A Literature Review” published in ACM Trans. Comput. Educ. 25, 2 Article 17 (May 2025), 20 pages.

  • Open Access

    Adapting an Ecological Belonging Intervention to an Introductory Computer Science Course

    by Janet Jiang, Jonathan Liu, Shao-Heng Ko, Diana Franklin, Kristin Stephens-Martinez

    Background and Context. Prior research has shown that students’ sense of belonging and self-efficacy are linked to their long-term performance and persistence in the computer science (CS) major. As a result, efforts specifically targeting these metrics may serve to broaden participation in CS. Brief psychological interventions within introductory classrooms have shown wide success in boosting student performance and sense of belonging in many other STEM fields, but the few implementations in CS courses have shown only mild success. More recently, STEM Education researchers developed an Ecological Belonging Intervention (EBI) that targets social contexts rather than individual psychology. It aims to create ecological change by introducing the classroom norm that adversity is surmountable and not a result of fixed or internal causes. The intervention was run in introductory biology, physics, and engineering classrooms, where it closed gender-based and race-based gaps in both performance and sense of belonging, and improved retention for students from historically underrepresented racial groups. Objectives. We sought to determine whether the EBI has similar effects in an introductory CS classroom. Additionally, we investigated the EBI’s impact on students’ self-efficacy. Method. We conducted a between-subjects study in an introductory CS course intended for students without prior programming experience, the majority of whom were women or non-binary. We evaluated the EBI’s influence on students’ sense of belonging, self-efficacy, performance, engagement, and persistence. Additionally, we examined whether the EBI is more effective for students from specific demographic groups. We measured students’ sense of belonging in CS using a recently-developed five-factor instrument that measures a student’s Acceptance, Affect, Membership, Desire to Participate, and Trust . We measured students’ self-efficacy using the corresponding subscale in the Motivated Strategies for Learning Questionnaire (MSLQ) instrument. Findings. We found that students who received the EBI experienced significantly larger increases in self-efficacy than students who did not. Moreover, first-year students who received the EBI had significantly larger increases in the Affect factor of their sense of belonging. There were no other significant effects between the EBI and the outcomes of either the entire population or specific demographic groups. Implications. The EBI’s ability to improve students’ self-efficacy and the Affect factor of their sense of belonging suggests that it has the potential to increase CS students’ persistence and performance over time . We encourage researchers to further explore the EBI’s benefits in other CS contexts.

  • Open Access

    Analogical Reasoning in Undergraduate Algorithms

    by Jonathan Liu, Erica Goodwin, Diana Franklin


  • Open Access

    Teaching Algorithm Design: A Literature Review

    by Jonathan Liu, Seth Poulsen, Erica Goodwin, Hongxuan Chen, Grace Williams, Yael Gertner, Diana Franklin

    Algorithm design is a vital skill developed in most undergraduate Computer Science (CS) programs, but few research studies focus on pedagogy related to algorithms coursework. To understand the work that has been done in the area, we present a systematic survey and literature review of CS Education studies. We search for research that is both related to algorithm design (as described by the ACM Curricular Guidelines) and evaluated on post-secondary-level students. Across all venues we searched prior to July 2024, we only find 102 such papers. We first classify these papers by topic, evaluation metric, evaluation methods, and intervention target. Through our classification, we find a broad sparsity of papers which indicates that many open questions remain about teaching algorithm design. We also note the need for papers using rigorous research methods, as only 43 out of 92 papers presenting quantitative data use statistical tests, and only 16 out of 47 papers presenting qualitative data follow a coding scheme. Only 18 papers report controlled trials. In addition, almost all authors only contribute to one publication, an indication that few groups are specializing on these topics. We then synthesize the results of the existing literature to give insights into what the corpus reveals about how we should teach algorithms. Broadly, we find that much of the literature explores implementing well-established practices, such as active learning or automated assessment, in the algorithms classroom. However, there are algorithms-specific results as well: A number of papers find that students may under-utilize certain algorithmic design techniques, and studies describe a variety of ways to select algorithms problems that increase student engagement and learning. The results we present, along with the publicly available set of papers collected, provide a detailed representation of the current corpus of CS Education work related to algorithm design and can orient further research in the area.


  • Open Access

    How Do Learners Use Scratch Paper When Working on Dynamic Programming Problems?

    by Zihan Wu, Jonathan Liu, Erica Goodwin, Diana Franklin

    Dynamic programming (DP) is one of the most challenging topics in algorithms courses. Although there exist animation tools that assist with the understanding of DP algorithms, few existing tools are aimed at scaffolding the process of solving DP algorithm design problems. To help create a learning tool able to provide the affordances learners need when attempting DP problems, we analyzed learners' scratch paper to understand how learners approach DP problems. Based on scratch paper from 18 learners solving DP problems during a think-aloud study, we created a codebook that characterized different elements and methods used by the learners on their scratch paper. We found that learners had distinct preferences when attempting DP problems. Some learners preferred using example input with specific values to simulate ideal program executions, while some used math representations of example inputs to help derive formulas. Learners interacted with their example input in multiple ways, including filling in hand-drawn tables to organize the calculation process and dynamically interacting with the inputs by crossing, circling, or using arrows to visualize the relationships between inputs. These findings suggest potential interactions that need to be taken into consideration when designing tools to support learners in solving DP problems.

  • Open Access

    Evaluating GPT for use in K-12 Block Based CS Instruction Using a Transpiler and Prompt Engineering

    by David Gonzalez-Maldonado, Jonathan Liu, Diana Franklin

    Though the increased availability of Large Language Models (LLMs) presents significant potential for change in the way students learn to program, the text-based nature of the available tools currently preclude block-based languages from much of that innovation. In an attempt to remedy this, we identify the strengths and weaknesses of using a transpiler to leverage the existing learning in commercially available LLMs and Scratch, a visual block-based programming language. Using only prompt engineering, we evaluate an LLM's performance on two common classroom tasks in a Scratch curriculum. We evaluate the LLM's ability to: 1) Create project solutions that compile and satisfy project requirements and 2) Analyze student projects' completion of project requirements using natural language. In both cases, we find results indicating that prompt-engineering alone is insufficient to reliably produce high-quality results. For projects of medium complexity, the LLM-generated solutions consistently failed to follow correct syntax or, in the few instances with correct syntax, produce correct solutions. When used for auto-grading, we found a correlation between scores assigned by the official Scratch Encore autograder and those generated by the LLM, nevertheless the discrepancies between the 'real' scores and the scores assigned by the LLM remained too great for the tool to be reliable in a classroom setting.


  • Open Access

    Teacher Decisions and Perspectives in Scratch TIPP&SEE Implementation

    by Jonathan Liu, Erica Goodwin, Dana Saito-Stehberger, Sharin Jacob, Mark Warschauer, Diana Franklin

    According to an ecological affordances perspective, any static curriculum has a set of affordances, and differences in teachers, students, and the teaching environment change how those affordances are viewed and used. Therefore, teaching is a relationship between the curriculum, the teacher, and the students. As such, it is not only possible but expected that a teacher will diverge from the details of a lesson plan to better accommodate the needs of themselves as a teacher and their students as learners. In this study, we report on a mixed-methods investigation that explores the different ways upper-elementary and middle-school (7-13 y.o. students) teachers implement the Scratch-based TIPP&SEE learning strategy and the reasoning for their approaches. As expected, we find that teachers across grade levels often deviate from lesson plan details to cater to their own classrooms. For example, teachers serving younger grades were far more likely to keep scaffolds that lesson plans suggest removing. The varied degree of deviation suggests that the repeated use of a learning strategy, alongside lesson plans that present a variety of scaffolded implementations, is beneficial in enabling teachers to adapt lesson content to serve the needs of their specific classroom.

  • Open Access

    Student Utilization of Metacognitive Strategies in Solving Dynamic Programming Problems

    by Jonathan Liu, Erica Goodwin, Diana Franklin

    Dynamic Programming (DP) is commonly regarded as one of the most difficult topics in the upper-level algorithms curriculum. The teaching of metacognitive strategies may prove effective in helping students learn to design DP algorithms. To explore both whether students learn and use these strategies on their own and the effect of guidance about using these strategies, we conducted think-aloud interviews with structured guidance at two points in a college algorithms course: once immediately after students learned the concept and once at the end of the course. We explore 1) what metacognitive strategies are commonly employed by students, 2) how effectively they help students solve problems, and 3) to what extent structured guidance about using metacognitive strategies is effective. We find that these strategies generally help students make progress in solving DP problems, but that they can mislead students as well. We also find that the adoption of these strategies is an individualized process and that structured strategy guidance is often insufficient in allowing students to solve individual DP problems, indicating the need for more extensive strategy instruction.