Umair Z. Ahmed
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Umair Z. Ahmed has a total of 19 co-authors. Here are the top co-authors by number of shared publications:
Amey Karkare
2 shared publications- Amey Karkare
Indian Institute of Technology Kanpur
2 shared publications - Ben Leong
National University of Singapore
2 shared publications Abhik Roychoudhury
1 shared publication- Abhik Roychoudhury
National University of Singapore, Singapore, Singapore
1 shared publication Arpit Kumar
Microsoft Research India
1 shared publication
Latest Publications
- Open Access
- 12 February 2025
- Published by Association for Computing Machinery (ACM)
- p. 11-17
Feasibility Study of Augmenting Teaching Assistants with AI for CS1 Programming Feedback
With the increasing adoption of Large Language Models (LLMs), there are proposals to replace human Teaching Assistants (TAs) with LLM-based AI agents for providing feedback to students. In this paper, we explore a new hybrid model where human TAs receive AI-generated feedback for CS1 programming exercises, which they can then review and modify as needed. We conducted a large-scale randomized intervention with 185 CS1 undergraduate students, comparing the efficacy of this hybrid approach against manual feedback and direct AI-generated feedback. Our initial hypothesis predicted that AI-augmented feedback would improve TA efficiency and increase the accuracy of guidance to students. However, our findings revealed mixed results. Although students perceived improvements in feedback quality, the hybrid model did not consistently translate to better student performance. We also observed complacency among some TAs who over-relied on LLM generated feedback and failed to identify and correct inaccuracies. These results suggest that augmenting human tutors with AI may not always result in improved teaching outcomes, and further research is needed to ensure it is truly effective.
- 12 February 2025
- Published by Association for Computing Machinery (ACM)
- p. 11-17
Open AccessFeasibility Study of Augmenting Teaching Assistants with AI for CS1 Programming Feedback
With the increasing adoption of Large Language Models (LLMs), there are proposals to replace human Teaching Assistants (TAs) with LLM-based AI agents for providing feedback to students. In this paper, we explore a new hybrid model where human TAs receive AI-generated feedback for CS1 programming exercises, which they can then review and modify as needed. We conducted a large-scale randomized intervention with 185 CS1 undergraduate students, comparing the efficacy of this hybrid approach against manual feedback and direct AI-generated feedback. Our initial hypothesis predicted that AI-augmented feedback would improve TA efficiency and increase the accuracy of guidance to students. However, our findings revealed mixed results. Although students perceived improvements in feedback quality, the hybrid model did not consistently translate to better student performance. We also observed complacency among some TAs who over-relied on LLM generated feedback and failed to identify and correct inaccuracies. These results suggest that augmenting human tutors with AI may not always result in improved teaching outcomes, and further research is needed to ensure it is truly effective.
- 30 June 2020
- Published by Springer Nature
- p. 106-117
MACER: A Modular Framework for Accelerated Compilation Error Repair
- 30 June 2020
- Published by Springer Nature
- p. 106-117
MACER: A Modular Framework for Accelerated Compilation Error Repair
- 1 November 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 327-338
Targeted Example Generation for Compilation Errors
- 1 November 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 327-338
Targeted Example Generation for Compilation Errors
- Open Access
- 21 August 2017
- Published by Association for Computing Machinery (ACM)
- p. 740-751
A feasibility study of using automated program repair for introductory programming assignments
Despite the fact an intelligent tutoring system for programming (ITSP) education has long attracted interest, its widespread use has been hindered by the difficulty of generating personalized feedback automatically. Meanwhile, automated program repair (APR) is an emerging new technology that automatically fixes software bugs, and it has been shown that APR can fix the bugs of large real-world software. In this paper, we study the feasibility of marrying intelligent programming tutoring and APR. We perform our feasibility study with four state-of-the-art APR tools (GenProg, AE, Angelix, and Prophet), and 661 programs written by the students taking an introductory programming course. We found that when APR tools are used out of the box, only about 30% of the programs in our dataset are repaired. This low repair rate is largely due to the student programs often being significantly incorrect — in contrast, professional software for which APR was successfully applied typically fails only a small portion of tests. To bridge this gap, we adopt in APR a new repair policy akin to the hint generation policy employed in the existing ITSP. This new repair policy admits partial repairs that address part of failing tests, which results in 84% improvement of repair rate. We also performed a user study with 263 novice students and 37 graders, and identified an understudied problem; while novice students do not seem to know how to effectively make use of generated repairs as hints, the graders do seem to gain benefits from repairs.
- 21 August 2017
- Published by Association for Computing Machinery (ACM)
- p. 740-751
Open AccessA feasibility study of using automated program repair for introductory programming assignments
Despite the fact an intelligent tutoring system for programming (ITSP) education has long attracted interest, its widespread use has been hindered by the difficulty of generating personalized feedback automatically. Meanwhile, automated program repair (APR) is an emerging new technology that automatically fixes software bugs, and it has been shown that APR can fix the bugs of large real-world software. In this paper, we study the feasibility of marrying intelligent programming tutoring and APR. We perform our feasibility study with four state-of-the-art APR tools (GenProg, AE, Angelix, and Prophet), and 661 programs written by the students taking an introductory programming course. We found that when APR tools are used out of the box, only about 30% of the programs in our dataset are repaired. This low repair rate is largely due to the student programs often being significantly incorrect — in contrast, professional software for which APR was successfully applied typically fails only a small portion of tests. To bridge this gap, we adopt in APR a new repair policy akin to the hint generation policy employed in the existing ITSP. This new repair policy admits partial repairs that address part of failing tests, which results in 84% improvement of repair rate. We also performed a user study with 263 novice students and 37 graders, and identified an understudied problem; while novice students do not seem to know how to effectively make use of generated repairs as hints, the graders do seem to gain benefits from repairs.
- Open Access
Prutor: A System for Tutoring CS1 and Collecting Student Programs for Analysis
An introductory programming course (CS1) is an integral part of any undergraduate curriculum. Due to large number and diverse programming background of students, providing timely and personalised feedback to individual students is a challenging task for any CS1 instructor. The help provided by teaching assistants (typically senior students) is not sufficient as it suffers from unintentional bias and, most of the time, not quick enough. In this paper, we present Prutor, a tutoring system platform to conduct introductory programming courses. Prutor is a cloud-based web application that provides instant and useful feedback to students while solving programming problems. Prutor stores, at regular intervals, the snapshots of students' attempts to solve programming problems. These intermediate versions of the student programs provide the instructors (and data analysts) a view of the students' approach to solving programming problems. Since Prutor is accessible through any standard web browser, students do not need to worry about dependencies external to the programming course, viz. Operating Systems, Editors, Compilers, Compiler Options, etc.. This enables the students to focus on solving only the programming problems. Apart from the code snapshots at regular intervals, Prutor also collects other valuable data such as the time taken by the students to solve the problems, the number of compile and execution events, and the errors made. We have used this data in developing intelligent tools for giving feedback to students, some of which are described briefly in this paper. This system thus serves as a platform for tutoring as well as data collection for researchers.
Open AccessPrutor: A System for Tutoring CS1 and Collecting Student Programs for Analysis
An introductory programming course (CS1) is an integral part of any undergraduate curriculum. Due to large number and diverse programming background of students, providing timely and personalised feedback to individual students is a challenging task for any CS1 instructor. The help provided by teaching assistants (typically senior students) is not sufficient as it suffers from unintentional bias and, most of the time, not quick enough. In this paper, we present Prutor, a tutoring system platform to conduct introductory programming courses. Prutor is a cloud-based web application that provides instant and useful feedback to students while solving programming problems. Prutor stores, at regular intervals, the snapshots of students' attempts to solve programming problems. These intermediate versions of the student programs provide the instructors (and data analysts) a view of the students' approach to solving programming problems. Since Prutor is accessible through any standard web browser, students do not need to worry about dependencies external to the programming course, viz. Operating Systems, Editors, Compilers, Compiler Options, etc.. This enables the students to focus on solving only the programming problems. Apart from the code snapshots at regular intervals, Prutor also collects other valuable data such as the time taken by the students to solve the problems, the number of compile and execution events, and the errors made. We have used this data in developing intelligent tools for giving feedback to students, some of which are described briefly in this paper. This system thus serves as a platform for tutoring as well as data collection for researchers.
- 1 January 2012
- Published by Springer Nature in Lecture Notes in Computer Science
- p. 415-427
Can Modern Statistical Parsers Lead to Better Natural Language Understanding for Education?
- 1 January 2012
- Published by Springer Nature in Lecture Notes in Computer Science
- p. 415-427
Can Modern Statistical Parsers Lead to Better Natural Language Understanding for Education?