Andrew Tran
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Andrew Tran has a total of 21 co-authors. Here are the top co-authors by number of shared publications:
- Stephen MacNeil
Temple University, Philadelphia, PA, USA
7 shared publications - Arto Hellas
Aalto University, Espoo, Finland
5 shared publications - Joanne Kim
Temple University, Philadelphia, PA, USA
5 shared publications - Juho Leinonen
Aalto University, Espoo, Finland
5 shared publications - Paul Denny
University of Auckland, Auckland, New Zealand
5 shared publications - Sami Sarsa
University of Jyväskylä, Jyväskylä, Finland
5 shared publications
Latest Publications
- Open Access
- 18 February 2025
- Published by Association for Computing Machinery (ACM)
- p. 1575-1576
Hacking Student Leadership: Peer Mentorship and Leadership Skill Development Among Hackathon Organizers
While hackathons are often celebrated for their impact on participants, less attention has been given to the unique leadership development opportunities for the student organizers who create and run these events. Unlike traditional classroom settings, where leadership and collaboration skills are typically delayed until upper-level courses, hackathon organizers must tackle these challenges earlier on. In managing a large-scale, formative event, student organizers take on roles that require decision-making, teamwork, and project management. This poster explores the experiences of student organizers at Anonymous Hackathon, emphasizing how this informal learning opportunity complements gaps in the traditional computer science (CS) curriculum by fostering essential leadership and collaboration skills earlier in students' academic careers.
- 18 February 2025
- Published by Association for Computing Machinery (ACM)
- p. 1575-1576
Open AccessHacking Student Leadership: Peer Mentorship and Leadership Skill Development Among Hackathon Organizers
While hackathons are often celebrated for their impact on participants, less attention has been given to the unique leadership development opportunities for the student organizers who create and run these events. Unlike traditional classroom settings, where leadership and collaboration skills are typically delayed until upper-level courses, hackathon organizers must tackle these challenges earlier on. In managing a large-scale, formative event, student organizers take on roles that require decision-making, teamwork, and project management. This poster explores the experiences of student organizers at Anonymous Hackathon, emphasizing how this informal learning opportunity complements gaps in the traditional computer science (CS) curriculum by fostering essential leadership and collaboration skills earlier in students' academic careers.
- Open Access
- 29 January 2024
- Published by Association for Computing Machinery (ACM)
- p. 11-18
Decoding Logic Errors: A Comparative Study on Bug Detection by Students and Large Language Models
Identifying and resolving logic errors can be one of the most frustrating challenges for novices programmers. Unlike syntax errors, for which a compiler or interpreter can issue a message, logic errors can be subtle. In certain conditions, buggy code may even exhibit correct behavior – in other cases, the issue might be about how a problem statement has been interpreted. Such errors can be hard to spot when reading the code, and they can also at times be missed by automated tests. There is great educational potential in automatically detecting logic errors, especially when paired with suitable feedback for novices. Large language models (LLMs) have recently demonstrated surprising performance for a range of computing tasks, including generating and explaining code. These capabilities are closely linked to code syntax, which aligns with the next token prediction behavior of LLMs. On the other hand, logic errors relate to the runtime performance of code and thus may not be as well suited to analysis by LLMs. To explore this, we investigate the performance of two popular LLMs, GPT-3 and GPT-4, for detecting and providing a novice-friendly explanation of logic errors. We compare LLM performance with a large cohort of introductory computing students (n = 964) solving the same error detection task. Through a mixed-methods analysis of student and model responses, we observe significant improvement in logic error identification between the previous and current generation of LLMs, and find that both LLM generations significantly outperform students. We outline how such models could be integrated into computing education tools, and discuss their potential for supporting students when learning programming.
- 29 January 2024
- Published by Association for Computing Machinery (ACM)
- p. 11-18
Open AccessDecoding Logic Errors: A Comparative Study on Bug Detection by Students and Large Language Models
Identifying and resolving logic errors can be one of the most frustrating challenges for novices programmers. Unlike syntax errors, for which a compiler or interpreter can issue a message, logic errors can be subtle. In certain conditions, buggy code may even exhibit correct behavior – in other cases, the issue might be about how a problem statement has been interpreted. Such errors can be hard to spot when reading the code, and they can also at times be missed by automated tests. There is great educational potential in automatically detecting logic errors, especially when paired with suitable feedback for novices. Large language models (LLMs) have recently demonstrated surprising performance for a range of computing tasks, including generating and explaining code. These capabilities are closely linked to code syntax, which aligns with the next token prediction behavior of LLMs. On the other hand, logic errors relate to the runtime performance of code and thus may not be as well suited to analysis by LLMs. To explore this, we investigate the performance of two popular LLMs, GPT-3 and GPT-4, for detecting and providing a novice-friendly explanation of logic errors. We compare LLM performance with a large cohort of introductory computing students (n = 964) solving the same error detection task. Through a mixed-methods analysis of student and model responses, we observe significant improvement in logic error identification between the previous and current generation of LLMs, and find that both LLM generations significantly outperform students. We outline how such models could be integrated into computing education tools, and discuss their potential for supporting students when learning programming.
- Open Access
- 7 August 2023
- Published by Association for Computing Machinery (ACM)
- p. 22-23
Using Large Language Models to Automatically Identify Programming Concepts in Code Snippets
Curating course material that aligns with students’ learning goals is a challenging and time-consuming task that instructors undergo when preparing their curricula. For instance, it is a challenge to find multiple-choice questions or example codes that demonstrate recursion in an unlabeled question bank or repository. Recently, Large Language Models (LLMs) have demonstrated the capability to generate high-quality learning materials at scale. In this poster, we use LLMs to identify programming concepts found within code snippets, allowing instructors to quickly curate their course materials. We compare programming concepts generated by LLMs with concepts generated by experts to see the extent to which they agree. The agreement was calculated using Cohen’s Kappa.
- 7 August 2023
- Published by Association for Computing Machinery (ACM)
- p. 22-23
Open AccessUsing Large Language Models to Automatically Identify Programming Concepts in Code Snippets
Curating course material that aligns with students’ learning goals is a challenging and time-consuming task that instructors undergo when preparing their curricula. For instance, it is a challenge to find multiple-choice questions or example codes that demonstrate recursion in an unlabeled question bank or repository. Recently, Large Language Models (LLMs) have demonstrated the capability to generate high-quality learning materials at scale. In this poster, we use LLMs to identify programming concepts found within code snippets, allowing instructors to quickly curate their course materials. We compare programming concepts generated by LLMs with concepts generated by experts to see the extent to which they agree. The agreement was calculated using Cohen’s Kappa.
- Top Cited PapersOpen Access
- 29 June 2023
- Published by Association for Computing Machinery (ACM)
- p. 124-130
Comparing Code Explanations Created by Students and Large Language Models
Reasoning about code and explaining its purpose are fundamental skills for computer scientists. There has been extensive research in the field of computing education on the relationship between a student's ability to explain code and other skills such as writing and tracing code. In particular, the ability to describe at a high-level of abstraction how code will behave over all possible inputs correlates strongly with code writing skills. However, developing the expertise to comprehend and explain code accurately and succinctly is a challenge for many students. Existing pedagogical approaches that scaffold the ability to explain code, such as producing exemplar code explanations on demand, do not currently scale well to large classrooms. The recent emergence of powerful large language models (LLMs) may offer a solution. In this paper, we explore the potential of LLMs in generating explanations that can serve as examples to scaffold students' ability to understand and explain code. To evaluate LLM-created explanations, we compare them with explanations created by students in a large course (n ≈ 1000) with respect to accuracy, understandability and length. We find that LLM-created explanations, which can be produced automatically on demand, are rated as being significantly easier to understand and more accurate summaries of code than student-created explanations. We discuss the significance of this finding, and suggest how such models can be incorporated into introductory programming education.
- 29 June 2023
- Published by Association for Computing Machinery (ACM)
- p. 124-130
Top Cited PapersOpen AccessComparing Code Explanations Created by Students and Large Language Models
Reasoning about code and explaining its purpose are fundamental skills for computer scientists. There has been extensive research in the field of computing education on the relationship between a student's ability to explain code and other skills such as writing and tracing code. In particular, the ability to describe at a high-level of abstraction how code will behave over all possible inputs correlates strongly with code writing skills. However, developing the expertise to comprehend and explain code accurately and succinctly is a challenge for many students. Existing pedagogical approaches that scaffold the ability to explain code, such as producing exemplar code explanations on demand, do not currently scale well to large classrooms. The recent emergence of powerful large language models (LLMs) may offer a solution. In this paper, we explore the potential of LLMs in generating explanations that can serve as examples to scaffold students' ability to understand and explain code. To evaluate LLM-created explanations, we compare them with explanations created by students in a large course (n ≈ 1000) with respect to accuracy, understandability and length. We find that LLM-created explanations, which can be produced automatically on demand, are rated as being significantly easier to understand and more accurate summaries of code than student-created explanations. We discuss the significance of this finding, and suggest how such models can be incorporated into introductory programming education.
- Top Cited PapersOpen Access
- 2 March 2023
- Published by Association for Computing Machinery (ACM)
- p. 931-937
Experiences from Using Code Explanations Generated by Large Language Models in a Web Software Development E-Book
Advances in natural language processing have resulted in large language models (LLMs) that can generate code and code explanations. In this paper, we report on our experiences generating multiple code explanation types using LLMs and integrating them into an interactive e-book on web software development. Three different types of explanations -- a line-by-line explanation, a list of important concepts, and a high-level summary of the code -- were created. Students could view explanations by clicking a button next to code snippets, which showed the explanation and asked about its utility. Our results show that all explanation types were viewed by students and that the majority of students perceived the code explanations as helpful to them. However, student engagement varied by code snippet complexity, explanation type, and code snippet length. Drawing on our experiences, we discuss future directions for integrating explanations generated by LLMs into CS classrooms.
- 2 March 2023
- Published by Association for Computing Machinery (ACM)
- p. 931-937
Top Cited PapersOpen AccessExperiences from Using Code Explanations Generated by Large Language Models in a Web Software Development E-Book
Advances in natural language processing have resulted in large language models (LLMs) that can generate code and code explanations. In this paper, we report on our experiences generating multiple code explanation types using LLMs and integrating them into an interactive e-book on web software development. Three different types of explanations -- a line-by-line explanation, a list of important concepts, and a high-level summary of the code -- were created. Students could view explanations by clicking a button next to code snippets, which showed the explanation and asked about its utility. Our results show that all explanation types were viewed by students and that the majority of students perceived the code explanations as helpful to them. However, student engagement varied by code snippet complexity, explanation type, and code snippet length. Drawing on our experiences, we discuss future directions for integrating explanations generated by LLMs into CS classrooms.
- Top Cited PapersOpen Access
- 6 March 2022
- Published by Association for Computing Machinery (ACM)
- p. 1176
Automatically Generating CS Learning Materials with Large Language Models
Recent breakthroughs in Large Language Models (LLMs), such as GPT-3 and Codex, now enable software developers to generate code based on a natural language prompt. Within computer science education, researchers are exploring the potential for LLMs to generate code explanations and programming assignments using carefully crafted prompts. These advances may enable students to interact with code in new ways while helping instructors scale their learning materials. However, LLMs also introduce new implications for academic integrity, curriculum design, and software engineering careers. This workshop will demonstrate the capabilities of LLMs to help attendees evaluate whether and how LLMs might be integrated into their pedagogy and research. We will also engage attendees in brainstorming to consider how LLMs will impact our field.
- 6 March 2022
- Published by Association for Computing Machinery (ACM)
- p. 1176
Top Cited PapersOpen AccessAutomatically Generating CS Learning Materials with Large Language Models
Recent breakthroughs in Large Language Models (LLMs), such as GPT-3 and Codex, now enable software developers to generate code based on a natural language prompt. Within computer science education, researchers are exploring the potential for LLMs to generate code explanations and programming assignments using carefully crafted prompts. These advances may enable students to interact with code in new ways while helping instructors scale their learning materials. However, LLMs also introduce new implications for academic integrity, curriculum design, and software engineering careers. This workshop will demonstrate the capabilities of LLMs to help attendees evaluate whether and how LLMs might be integrated into their pedagogy and research. We will also engage attendees in brainstorming to consider how LLMs will impact our field.
- Open Access
- 6 March 2022
- Published by Association for Computing Machinery (ACM)
- p. 1255
The Implications of Large Language Models for CS Teachers and Students
The introduction of Large Language Models (LLMs) has generated a significant amount of excitement both in industry and among researchers. Recently, tools that leverage LLMs have made their way into the classroom where they help students generate code and help instructors generate learning materials. There are likely many more uses of these tools -- both beneficial to learning and possibly detrimental to learning. To help ensure that these tools are used to enhance learning, educators need to not only be familiar with these tools, but with their use and potential misuse. The goal of this BoF is to raise awareness about LLMs and to build a learning community around their use in computing education. Aligned with this goal of building an inclusive learning community, our BoF is led by globally distributed discussion leaders, including undergraduate researchers, to facilitate multiple coordinated discussions that can lead to a broader conversation about the role of LLMs in CS education.
- 6 March 2022
- Published by Association for Computing Machinery (ACM)
- p. 1255
Open AccessThe Implications of Large Language Models for CS Teachers and Students
The introduction of Large Language Models (LLMs) has generated a significant amount of excitement both in industry and among researchers. Recently, tools that leverage LLMs have made their way into the classroom where they help students generate code and help instructors generate learning materials. There are likely many more uses of these tools -- both beneficial to learning and possibly detrimental to learning. To help ensure that these tools are used to enhance learning, educators need to not only be familiar with these tools, but with their use and potential misuse. The goal of this BoF is to raise awareness about LLMs and to build a learning community around their use in computing education. Aligned with this goal of building an inclusive learning community, our BoF is led by globally distributed discussion leaders, including undergraduate researchers, to facilitate multiple coordinated discussions that can lead to a broader conversation about the role of LLMs in CS education.