Sophia Mettille
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Sophia Mettille has a total of 10 co-authors. Here are the top co-authors by number of shared publications:
- Irene Hou
University of California, San Diego, San Diego, California, USA
3 shared publications - Owen Man
Temple University, Philadelphia, PA, USA
3 shared publications - Stephen MacNeil
Temple University, Philadelphia, PA, USA
3 shared publications - Kenneth Angelikas
Temple University, Philadelphia, PA, USA
2 shared publications - Sebastian Gutierrez
Computer and Information Sciences, Temple University, Philadelphia, Pennsylvania, USA
2 shared publications - Cynthia Zastudil
Temple University, Philadelphia, PA, USA
1 shared publication
Latest Publications
- Open Access
- 12 February 2025
- Published by Association for Computing Machinery (ACM)
- p. 124-133
Seeing the Forest and the Trees: Solving Visual Graph and Tree Based Data Structure Problems using Large Multimodal Models
- 12 February 2025
- Published by Association for Computing Machinery (ACM)
- p. 124-133
Open AccessSeeing the Forest and the Trees: Solving Visual Graph and Tree Based Data Structure Problems using Large Multimodal Models
- Top Cited PapersOpen Access
- 29 January 2024
- Published by Association for Computing Machinery (ACM)
- p. 39-48
The Effects of Generative AI on Computing Students’ Help-Seeking Preferences
Help-seeking is a critical way that students learn new concepts, acquire new skills, and get unstuck when problem-solving in their computing courses. The recent proliferation of generative AI tools, such as ChatGPT, offers students a new source of help that is always available on-demand. However, it is unclear how this new resource compares to existing help-seeking resources along dimensions of perceived quality, latency, and trustworthiness. In this paper, we investigate the help-seeking preferences and experiences of computing students now that generative AI tools are available to them. We collected survey data (n=47) and conducted interviews (n=8) with computing students. Our results suggest that although these models are being rapidly adopted, they have not yet fully eclipsed traditional help resources. The help-seeking resources that students rely on continue to vary depending on the task and other factors. Finally, we observed preliminary evidence about how help-seeking with generative AI is a skill that needs to be developed, with disproportionate benefits for those who are better able to harness the capabilities of LLMs. We discuss potential implications for integrating generative AI into computing classrooms and the future of help-seeking in the era of generative AI.
- 29 January 2024
- Published by Association for Computing Machinery (ACM)
- p. 39-48
Top Cited PapersOpen AccessThe Effects of Generative AI on Computing Students’ Help-Seeking Preferences
Help-seeking is a critical way that students learn new concepts, acquire new skills, and get unstuck when problem-solving in their computing courses. The recent proliferation of generative AI tools, such as ChatGPT, offers students a new source of help that is always available on-demand. However, it is unclear how this new resource compares to existing help-seeking resources along dimensions of perceived quality, latency, and trustworthiness. In this paper, we investigate the help-seeking preferences and experiences of computing students now that generative AI tools are available to them. We collected survey data (n=47) and conducted interviews (n=8) with computing students. Our results suggest that although these models are being rapidly adopted, they have not yet fully eclipsed traditional help resources. The help-seeking resources that students rely on continue to vary depending on the task and other factors. Finally, we observed preliminary evidence about how help-seeking with generative AI is a skill that needs to be developed, with disproportionate benefits for those who are better able to harness the capabilities of LLMs. We discuss potential implications for integrating generative AI into computing classrooms and the future of help-seeking in the era of generative AI.
- Open Access
- 29 January 2024
- Published by Association for Computing Machinery (ACM)
- p. 29-38
More Robots are Coming: Large Multimodal Models (ChatGPT) can Solve Visually Diverse Images of Parsons Problems
Large language models are reshaping computing education. Based on recent research, these models explain code better than students, answer multiple choice questions at or above the class average, and generate code that can pass automated tests in introductory courses. In response to these capabilities, instructors have quickly adjusted their courses and assessment methods to align with shifting learning goals and the increased risk of academic integrity issues. While some scholars have advocated for the integration of visual problems as a safeguard against the capabilities of language models, new multimodal models now have vision and language capabilities that may allow them to analyze and solve visual problems. In this paper, we compare the large multimodal model (LMMs) GPT-4V with Bard, an LLM that uses Google Lens for text recognition. We find that LMMs, which have learned both pixel features (from images) and text features (from prompts) in the same embedding space, performed substantially better than Bard which uses a piecemeal approach. With a specific focus on Parsons problems presented across diverse visual representations, our results show that GPT-4V solved 96.7% these visual problems, struggling minimally with a single Parsons problem. Conversely, Bard performed poorly by only solving 69.2% of problems, struggling with common issues like hallucinations and refusals. These findings suggest that merely transitioning to visual programming problems might not be a panacea to issues of academic integrity in the generative AI era.
- 29 January 2024
- Published by Association for Computing Machinery (ACM)
- p. 29-38
Open AccessMore Robots are Coming: Large Multimodal Models (ChatGPT) can Solve Visually Diverse Images of Parsons Problems
Large language models are reshaping computing education. Based on recent research, these models explain code better than students, answer multiple choice questions at or above the class average, and generate code that can pass automated tests in introductory courses. In response to these capabilities, instructors have quickly adjusted their courses and assessment methods to align with shifting learning goals and the increased risk of academic integrity issues. While some scholars have advocated for the integration of visual problems as a safeguard against the capabilities of language models, new multimodal models now have vision and language capabilities that may allow them to analyze and solve visual problems. In this paper, we compare the large multimodal model (LMMs) GPT-4V with Bard, an LLM that uses Google Lens for text recognition. We find that LMMs, which have learned both pixel features (from images) and text features (from prompts) in the same embedding space, performed substantially better than Bard which uses a piecemeal approach. With a specific focus on Parsons problems presented across diverse visual representations, our results show that GPT-4V solved 96.7% these visual problems, struggling minimally with a single Parsons problem. Conversely, Bard performed poorly by only solving 69.2% of problems, struggling with common issues like hallucinations and refusals. These findings suggest that merely transitioning to visual programming problems might not be a panacea to issues of academic integrity in the generative AI era.