Sasmita Harini
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Research Interests
Sasmita Harini has a total of 3 co-authors. Here are the top co-authors by number of shared publications:
- Sannidhi V. Hebbar
Indian Institute of Science
2 shared publications - Viraj Kumar
Indian Institute of Science, Bengaluru, India
2 shared publications - Arun Raman
Birla Institute of Technology and Science, Goa, India
1 shared publication
Latest Publications
- Open Access
- 13 June 2025
- Published by Association for Computing Machinery (ACM)
- p. 30-36
Refuting LLM-generated Code with Reactive Task Comprehension
- 13 June 2025
- Published by Association for Computing Machinery (ACM)
- p. 30-36
Open AccessRefuting LLM-generated Code with Reactive Task Comprehension
- Open Access
- 12 August 2024
- Published by Association for Computing Machinery (ACM)
- p. 539-540
Refute Questions for Concrete, Cluttered Specifications
Learners must be able to critique AI-generated code, which is often plausible but not always appropriate. Refute Questions have been proposed as one way to develop this ability. Each such question has two components: a task specification and a purported solution (typically, a single function) for that task. Our prior work [1, 4, 5] has focused on Refute Questions where purported solutions are inappropriate because they are logically incorrect. Given such a question, learners must demonstrate their understanding of both the buggy function and the specification by providing an input on which the function’s behavior does not satisfy the specification. In this work, we modify our conception of Refute Questions in two ways. First, we critique not only the correctness of the purported function but also aspects of its design. Specifically, we focus on the appropriateness of: These design aspects are based on the first two steps of Felleisen et al.’s 6-step function design recipe [3]. Second, our prior work has only considered abstract task specifications. In our experience, AI-generated code is often well-designed for such specifications (e.g., see Figure 1). Thus, we consider task specifications that are concrete (i.e., they include task-specific details) and cluttered (i.e., they contain extraneous details). Abstracting away certain details from the real-world specifications and eliminating clutter are important and often challenging skills for learners [6, 7, 8, 9, 10]. Further, today’s code-generation tools often produce poorly-designed code given concrete, cluttered specifications (e.g., see Figure 2), partly because the performance of the underlying AI model drops substantially as the length of the specification grows [2]. We believe that our new style of Refute Questions may help learners develop the ability to critique such code. We illustrate our process for designing such Refute Questions with an example. We start with an abstract task specification such as shown in Figure 11. Next, we manually invent concretizations for the abstract terms ‘integers’ and ‘sign’ (respectively: ‘bank accounts’ and their ‘type’ – active, dormant, or overdrawn). Further, we clutter the specification by inventing extraneous facts about the percentage of permissible accounts of each type. For these questions, learners critique the AI-generated code by answering a series of True/False questions related to the design aspects noted previously. Further, if the design is appropriate, learners critique the logical correctness of the code, as with standard Refute Questions. This poster will present findings from an initial study of Refute Questions for Concrete, Cluttered Specifications in the context of an online course on programming with AI code-generation tools2, which runs in June 2024.
- 12 August 2024
- Published by Association for Computing Machinery (ACM)
- p. 539-540
Open AccessRefute Questions for Concrete, Cluttered Specifications
Learners must be able to critique AI-generated code, which is often plausible but not always appropriate. Refute Questions have been proposed as one way to develop this ability. Each such question has two components: a task specification and a purported solution (typically, a single function) for that task. Our prior work [1, 4, 5] has focused on Refute Questions where purported solutions are inappropriate because they are logically incorrect. Given such a question, learners must demonstrate their understanding of both the buggy function and the specification by providing an input on which the function’s behavior does not satisfy the specification. In this work, we modify our conception of Refute Questions in two ways. First, we critique not only the correctness of the purported function but also aspects of its design. Specifically, we focus on the appropriateness of: These design aspects are based on the first two steps of Felleisen et al.’s 6-step function design recipe [3]. Second, our prior work has only considered abstract task specifications. In our experience, AI-generated code is often well-designed for such specifications (e.g., see Figure 1). Thus, we consider task specifications that are concrete (i.e., they include task-specific details) and cluttered (i.e., they contain extraneous details). Abstracting away certain details from the real-world specifications and eliminating clutter are important and often challenging skills for learners [6, 7, 8, 9, 10]. Further, today’s code-generation tools often produce poorly-designed code given concrete, cluttered specifications (e.g., see Figure 2), partly because the performance of the underlying AI model drops substantially as the length of the specification grows [2]. We believe that our new style of Refute Questions may help learners develop the ability to critique such code. We illustrate our process for designing such Refute Questions with an example. We start with an abstract task specification such as shown in Figure 11. Next, we manually invent concretizations for the abstract terms ‘integers’ and ‘sign’ (respectively: ‘bank accounts’ and their ‘type’ – active, dormant, or overdrawn). Further, we clutter the specification by inventing extraneous facts about the percentage of permissible accounts of each type. For these questions, learners critique the AI-generated code by answering a series of True/False questions related to the design aspects noted previously. Further, if the design is appropriate, learners critique the logical correctness of the code, as with standard Refute Questions. This poster will present findings from an initial study of Refute Questions for Concrete, Cluttered Specifications in the context of an online course on programming with AI code-generation tools2, which runs in June 2024.