APAshley Pang

Ashley Pang

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Ashley Pang has a total of 15 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Detecting AI-Generated Code in Introductory Programming Courses

    by Aryan Ramachandra, Suhani Chaudhary, Justin Tran, Riti Desai, Ashley Pang, Mariam Salloum


  • Open Access

    Creating a Second Pathway to the Computing Major

    by Ashley Pang, Paea LePendu, Mariam Salloum, Neftali Watkinson Medina, Carla E. Brodley


  • Open Access

    Midterm Exam Outliers Efficiently Highlight Potential Cheaters on Programming Assignments

    by Shirin Haji Amin Shirazi, Ashley Pang, Allan Knight, Mariam Salloum, Frank Vahid

    The ubiquitous use of online tools, contractors and homework sites, has made plagiarism a concerning topic in computer science education. With the introduction of ChatGPT, it poses a threat now more than ever. Many cheating detection tools, such as similarity checkers and style anomaly checkers, help instructors decide whether a student has plagiarized. However, these are not scalable to large classes. Similarity tools can produce high rates of suspected cheating and thus ineffectively use an instructor's time in weeding out the actual cheating cases, especially in the early weeks of CS courses where programs can be small and student solutions can be very similar. We developed a new approach using outlier detection to filter inconsistent performers based on their lab scores throughout the course and their midterm exam scores. Instructors can then manually analyze a manageable amount of students even with large class sizes. We performed our experiment on two large course offerings of CS1 (a total of 177 students) using our algorithm and compared it to a manual analysis performed by an experienced CS1 instructor. The detection approach identified 11 students in the first offering (Winter 2019) and 12 students in the second offering (Spring 2023). With an average precision of 83%, our tool produces a list of concerning students with high precision. This significantly helps teachers efficiently allocate their time and pursue cheating early in the term in order to address and prevent further issues.


  • Open Access

    Incentivizing Good Programming Practices: The Impact of Early Program Submission on Student Course and Exam Performance

    by Shirin Haji Amin Shirazi, Ashley Pang, Allan Knight, Mariam Salloum

    Motivating students to engage with a course, encouraging positive behavior, and inspiring them to take an active role in their educational process - particularly at the beginning of the course - are universal challenges in education. In this article, we share our experience implementing an early submission incentive policy in a Machine Organization and Assembly Language Programming course. This policy encourages students to complete and submit their weekly lab work early in exchange for bonus points. We examine the impact of this positive behavior reinforcement on overall student performance (final grade) and performance in specific components such as exams and programming assignments. Our results, based on data collected over four years and involving more than 1,400 students, indicate that students who participate in early submissions achieve a higher final grade and perform better on other assessments, such as programming assignments.


  • Open Access

    ChatGPT and Cheat Detection in CS1 Using a Program Autograding System

    by Ashley Pang, Frank Vahid

    We experimented with ChatGPT's ability to write programs in a CS1 class, and the ability of a popular tool to auto-detect ChatGPT-written programs. We found ChatGPT was proficient at generating correct programs from a mere copy-paste of the English programming assignment specifications. However, running ChatGPT for 10 programming assignments and acting as 20 different students, and using zyBook's APEX beta tool for academic integrity, we found: (1) ChatGPT-generated programs tend to use a programming style departing from the style taught in the textbook or by the instructor, and these "style anomalies" were automatically detected. (2) Although ChatGPT may for the same assignment generate a few different program solutions for different students, ChatGPT often generates highly-similar programs for different students, so if enough students in a class (e.g., 5 or more) use ChatGPT, their programs will likely be flagged by a similarity checker. (3) If students are required to do all programming in the autograder's IDE, then a student using ChatGPT ends up showing very little time relative to classmates, which is automatically flagged. (4) Manually, we observed that if a student consistently uses ChatGPT to submit programs, the programming style may vary across programs, something normal students don't do; automation of style inconsistency detection was recently added to APEX. In short, while there will no doubt be an arms race between AI-generated programs and automatic detection of AI-generated programs, currently students using ChatGPT for multiple CS1 programs can be detected by automated tools such as zyBooks' APEX.


  • Open Access

    Style Anomalies Can Suggest Cheating in CS1 Programs

    by Benjamin Denzler, Frank Vahid, Ashley Pang, Mariam Salloum

    Student cheating on at-home programming assignments is a well- known problem. A key contributor is externally-obtained solutions from websites, contractors, and recently generative AI. In our experience, such externally-obtained solutions often use coding styles that depart from a class' style, which we call "style anomalies," such as using untaught or advanced constructs like pointers or ternary operators, or having different indenting or brace usage from the class style. We developed a tool to auto-count style anomalies. For six labs across four terms in 2021-2022, and 50 sampled students per lab, we found 18% of submissions on average had unusually-high style anomaly counts. Importantly, 8% of submissions on average had a high style anomaly count but were not flagged by a similarity checker, meaning 8% of submissions are suspicious but might have been missed if using similarity checking alone. We repeated a similar analysis for Spring 2023 when generative AI (ChatGPT) was gaining popularity, and the numbers rose to 26% and 18%, respectively. Detailed investigations by instructors led to a majority (but not all) high style anomaly submissions being deemed cheating. Even for high-similarity submissions, counting style anomalies can help instructors focus investigations on the most-likely cheating cases, and can strengthen cases sent to student conduct offices. With the rise of externally-obtained solutions from websites, contractors, and generative AI, counting style anomalies may become an increasingly important complement to similarity checking; in fact, it is now the primary cheat-detection tool in our CS1 at a large state university, with similarity secondary.


  • Open Access

    Performance Analysis and Interviews of Non-CS-Major Students Sanctioned for Cheating in CS1

    by Ashley Pang, Frank Vahid

    College cheating is common, including in computer science (CS) classes like introductory programming (CS1). Much research surveys college students about cheating, but few survey students actually caught cheating, or analyze their performance. We analyzed performance of 24 students sanctioned for cheating on programs in our CS1 over three terms, out of 300+ students, mostly non-CS science and engineering majors. Sanctioned students participated less in lectures (scoring 77% vs. 91%, p = 0.00002), and were less earnest in their completion of the online book's readings (51% vs. 80%, p = 0.0000001) during weeks 1-5. Those findings suggest early disengagement which would correlate with a tendency to cheat, and might also suggest a lack of learning which might help cause cheating. Sanctioned students scored dramatically lower on the earlier midterm exam (61% vs. 83%, p = 0.0001). After being sanctioned with Fs in the course (typically post-midterm), most agreed to an optional later interview to help the professor learn how to prevent cheating, at which point most were quite forthright. Key themes included an inability or unwillingness to devote the time needed to learn programming, a belief that the required CS1 course was not important for non-CS majors, and a disbelief that cheating students would be caught or punished despite warnings. More studies of such experiences may help instructors reduce cheating; in our case, the findings suggest instructors should emphasize relevance, make detection/punishment efforts clear, and detect early disengagement and potentially intervene.


  • Open Access

    Style Anomalies Can Suggest Cheating in CS1 Programs

    by Benjamin Denzler, Frank Vahid, Ashley Pang

    Student cheating on at-home programming assignments is a well-known problem. A key contributor is externally obtained solutions from websites, contractors, and recently generative AI. In our experience, such externally obtained solutions often use coding styles that depart from a class's style, which we call "style anomalies". Examples of style anomalies include using untaught or advanced constructs like pointers or ternary operators or having different indenting or brace usage from the class style. We developed a tool to automatically count style anomalies in student code submissions. We used this tool to find suspected cheating in student submissions for lab assignments across five terms of CS1. This poster presents our findings: Some student submissions were suspected of cheating due to high style anomaly counts and were not flagged as suspicious by a code similarity checker. With the rise of externally obtained solutions from websites, contractors, and generative AI, style anomalies may become an important complement to similarity checking for detecting cheating.


  • Open Access

    Towards Comprehensive Metrics for Programming Cheat Detection

    by Frank Vahid, Ashley Pang, Benjamin Denzler

    Automated assistance for detecting cheating on programs has long been investigated by CS educators, especially with the rise of "homework help" websites over the past decade, and recently with AI tools like ChatGPT. The main detection approach has long been flagging similar submission pairs. Modern cheating, like hiring contractors or using ChatGPT, may not yield such similarity. And, cases based on similarity alone may be weak. Thus, over the past several years, building on logs from an online program auto-grader (zyBooks), we developed additional "cheating concern metrics": points rate, style anomalies, style inconsistencies, IP address anomalies, code replacements, and initial copying. Most are defined not only for one programming assignment but also across a set of assignments. The metrics can help catch more kinds of cheating, provide more compelling evidence of cheating, reduce false cheating accusations based on similarity alone, and help instructors focus their limited cheat-detection time on the most egregious cases. We describe the techniques, and our experiences (via our own Python scripts and a commercial tool) for several terms, showing benefits of having more metrics than just similarity. Of 30 cheating cases over 3 terms and 300 students, most were based on metrics beyond similarity, all students admitted, none later contested, and time per student was only 1-2 hours (far less than previously). Our goal is to prevent cheating in the first place, by reducing opportunity via strong detection tools, as part of a multi-faceted approach to having students truly learn and stay out of trouble.