Steve Oney
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Steve Oney has a total of 26 co-authors. Here are the top co-authors by number of shared publications:
- Ge Zhang
University of Michigan, Ann Arbor, Michigan, USA
7 shared publications - Yan Chen
Department of Computer Science, Virginia Tech, Blacksburg, Virginia, USA
5 shared publications - Maryam Arab
School of Information, University of Michigan, Ann Arbor, Michigan, USA
3 shared publications - Yinuo Yang
Department of Computer Science and Engineering, University of Notre Dame, Notre Dame, Indiana, USA
3 shared publications - Anhong Guo
University of Michigan, Ann Arbor, MI, USA
2 shared publications - April Yi Wang
ETH Zurich, Zurich, Switzerland
2 shared publications
Latest Publications
- Open Access
- 2 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 1-18
Editrail: Understanding AI Usage by Visualizing Student-AI Interaction in Code
- 2 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 1-18
Open AccessEditrail: Understanding AI Usage by Visualizing Student-AI Interaction in Code
- Open Access
- 13 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 1-5
Simulating Human Cursor Trajectories for Path-Sensitive GUI Evaluation
- 13 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 1-5
Open AccessSimulating Human Cursor Trajectories for Path-Sensitive GUI Evaluation
- Open Access
- 13 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 1-16
CodeStream: Augmenting Timelines with Code Annotation for Navigating Large Coding Histories
Code edit histories can offer instructors valuable insight into students’ problem-solving processes, revealing unproductive behaviors that final code alone cannot capture. For example, a correct solution may contain large copy-and-pasted segments (suggesting the code originated elsewhere) or unguided trial-and-error (suggesting a lack of clear strategy). Timelines are a common way to visualize code histories, but existing timeline visualizations of code or document histories show only when and where edits occurred, not what changed. Without this context, it is difficult to answer key questions about how students invested effort or to infer their intentions. We present CodeStream, a visualization system that augments timelines with situational code annotations, whose granularity and visibility dynamically adapt to scale and interaction state. A comparison study shows that CodeStream enables context-aware navigation of coding histories, supporting fast and accurate pattern identification, and helping instructors reason about students’ coding behaviors and identify who may need intervention.
- 13 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 1-16
Open AccessCodeStream: Augmenting Timelines with Code Annotation for Navigating Large Coding Histories
Code edit histories can offer instructors valuable insight into students’ problem-solving processes, revealing unproductive behaviors that final code alone cannot capture. For example, a correct solution may contain large copy-and-pasted segments (suggesting the code originated elsewhere) or unguided trial-and-error (suggesting a lack of clear strategy). Timelines are a common way to visualize code histories, but existing timeline visualizations of code or document histories show only when and where edits occurred, not what changed. Without this context, it is difficult to answer key questions about how students invested effort or to infer their intentions. We present CodeStream, a visualization system that augments timelines with situational code annotations, whose granularity and visibility dynamically adapt to scale and interaction state. A comparison study shows that CodeStream enables context-aware navigation of coding histories, supporting fast and accurate pattern identification, and helping instructors reason about students’ coding behaviors and identify who may need intervention.
- Open Access
- 27 September 2025
- Published by Association for Computing Machinery (ACM)
- p. 1-10
Multi-Click: Cross-Tab Web Automation via Action Generalization
- 27 September 2025
- Published by Association for Computing Machinery (ACM)
- p. 1-10
Open AccessMulti-Click: Cross-Tab Web Automation via Action Generalization
- Top Cited PapersOpen Access
- 11 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-13
VRCopilot: Authoring 3D Layouts with Generative AI Models in VR
Immersive authoring provides an intuitive medium for users to create 3D scenes via direct manipulation in Virtual Reality (VR). Recent advances in generative AI have enabled the automatic creation of realistic 3D layouts. However, it is unclear how capabilities of generative AI can be used in immersive authoring to support fluid interactions, user agency, and creativity. We introduce VRCopilot, a mixed-initiative system that integrates pre-trained generative AI models into immersive authoring to facilitate human-AI co-creation in VR. VRCopilot presents multimodal interactions to support rapid prototyping and iterations with AI, and intermediate representations such as wireframes to augment user controllability over the created content. Through a series of user studies, we evaluated the potential and challenges in manual, scaffolded, and automatic creation in immersive authoring. We found that scaffolded creation using wireframes enhanced the user agency compared to automatic creation. We also found that manual creation via multimodal specification offers the highest sense of creativity and agency.
- 11 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-13
Top Cited PapersOpen AccessVRCopilot: Authoring 3D Layouts with Generative AI Models in VR
Immersive authoring provides an intuitive medium for users to create 3D scenes via direct manipulation in Virtual Reality (VR). Recent advances in generative AI have enabled the automatic creation of realistic 3D layouts. However, it is unclear how capabilities of generative AI can be used in immersive authoring to support fluid interactions, user agency, and creativity. We introduce VRCopilot, a mixed-initiative system that integrates pre-trained generative AI models into immersive authoring to facilitate human-AI co-creation in VR. VRCopilot presents multimodal interactions to support rapid prototyping and iterations with AI, and intermediate representations such as wireframes to augment user controllability over the created content. Through a series of user studies, we evaluated the potential and challenges in manual, scaffolded, and automatic creation in immersive authoring. We found that scaffolded creation using wireframes enhanced the user agency compared to automatic creation. We also found that manual creation via multimodal specification offers the highest sense of creativity and agency.
- Open Access
- 9 July 2024
- Published by Association for Computing Machinery (ACM)
- p. 188-199
CFlow: Supporting Semantic Flow Analysis of Students' Code in Programming Problems at Scale
Introductory programming courses have been growing rapidly, now enrolling hundreds or thousands of students. In such large courses, it can be overwhelmingly difficult for instructors to understand class-wide problem-solving patterns or issues, which is crucial for improving instruction and addressing important pedagogical challenges. In this paper, we propose a technique and system, CFlow, for creating understandable and navigable representations of code at scale. CFlow is able to represent thousands of code samples in a visualization that resembles a single code sample. CFlow creates scalable code representations by (1) clustering individual statements with similar semantic purposes, (2) presenting clustered statements in a way that maintains semantic relationships between statements, (3) representing the correctness of different variations as a histogram, and (4) allowing users to navigate through solutions interactively using semantic filters. With a multi-level view design, users can navigate high-level patterns, and low-level implementations. This is in contrast to prior tools that either limit their focus on isolated statements (and thus discard the surrounding context of those statements) or cluster entire code samples (which can lead to large numbers of clusters---for example, if there are n code features and m implementations of each, there can be mn clusters). We evaluated the effectiveness of CFlow with a comparison study, found participants using CFlow spent only half the time identifying mistakes and recalled twice as many desired patterns from over 6,000 submissions.
- 9 July 2024
- Published by Association for Computing Machinery (ACM)
- p. 188-199
Open AccessCFlow: Supporting Semantic Flow Analysis of Students' Code in Programming Problems at Scale
Introductory programming courses have been growing rapidly, now enrolling hundreds or thousands of students. In such large courses, it can be overwhelmingly difficult for instructors to understand class-wide problem-solving patterns or issues, which is crucial for improving instruction and addressing important pedagogical challenges. In this paper, we propose a technique and system, CFlow, for creating understandable and navigable representations of code at scale. CFlow is able to represent thousands of code samples in a visualization that resembles a single code sample. CFlow creates scalable code representations by (1) clustering individual statements with similar semantic purposes, (2) presenting clustered statements in a way that maintains semantic relationships between statements, (3) representing the correctness of different variations as a histogram, and (4) allowing users to navigate through solutions interactively using semantic filters. With a multi-level view design, users can navigate high-level patterns, and low-level implementations. This is in contrast to prior tools that either limit their focus on isolated statements (and thus discard the surrounding context of those statements) or cluster entire code samples (which can lead to large numbers of clusters---for example, if there are n code features and m implementations of each, there can be mn clusters). We evaluated the effectiveness of CFlow with a comparison study, found participants using CFlow spent only half the time identifying mistakes and recalled twice as many desired patterns from over 6,000 submissions.
- Open Access
- 9 July 2024
- Published by Association for Computing Machinery (ACM)
- p. 544-546
VizCode: A Practical Real-time Tool for In-Class Computer Programming Tutoring
Prior research has shown the benefits and promise of allowing instructors in large programming classes to monitor students' coding activity in real-time. However, translating these findings into practical, user-friendly tools remains a challenge. This demonstration showcases VizCode, a tool that allows instructors to monitor students' code in real-time as they edit in the popular Visual Studio Code (VSCode) IDE. VizCode is designed to be practical (integrating with widely-used tools and requiring minimal server overhead), scalable (minimizing network latency by only communicating code changes), and easy to use (requiring minimal setup from students). By focusing on practicality and seamless integration with VSCode, VizCode bridges the gap between research and practice, making it easier for instructors to monitor students' code in real-time.
- 9 July 2024
- Published by Association for Computing Machinery (ACM)
- p. 544-546
Open AccessVizCode: A Practical Real-time Tool for In-Class Computer Programming Tutoring
Prior research has shown the benefits and promise of allowing instructors in large programming classes to monitor students' coding activity in real-time. However, translating these findings into practical, user-friendly tools remains a challenge. This demonstration showcases VizCode, a tool that allows instructors to monitor students' code in real-time as they edit in the popular Visual Studio Code (VSCode) IDE. VizCode is designed to be practical (integrating with widely-used tools and requiring minimal server overhead), scalable (minimizing network latency by only communicating code changes), and easy to use (requiring minimal setup from students). By focusing on practicality and seamless integration with VSCode, VizCode bridges the gap between research and practice, making it easier for instructors to monitor students' code in real-time.
- Open Access
- 9 July 2024
- Published by Association for Computing Machinery (ACM)
- p. 542-543
Demonstration of CFlow: Supporting Semantic Flow Analysis of Students' Code in Programming Problems at Scale
Introductory programming courses have been growing rapidly, now enrolling hundreds or thousands of students. In such large courses, it can be overwhelmingly difficult for instructors to understand class-wide problem-solving patterns or issues, which is crucial for improving instruction and addressing important pedagogical challenges. In this paper, we propose a technique and system, CFlow, for creating understandable and navigable representations of code at scale. CFlow is able to represent thousands of code samples in a visualization that resembles a single code sample. CFlow creates scalable code representations by (1) clustering individual statements with similar semantic purposes, (2) presenting clustered statements in a way that maintains semantic relationships between statements, (3) representing the correctness of different variations as a histogram, and (4) allowing users to navigate through solutions interactively using semantic filters. With a multi-level view design, users can navigate high-level patterns, and low-level implementations. This is in contrast to prior tools that either limit their focus on isolated statements (and thus discard the surrounding context of those statements) or cluster entire code samples (which can lead to large numbers of clusters---for example, if there are n code features and m implementations of each, there can be m^n clusters).
- 9 July 2024
- Published by Association for Computing Machinery (ACM)
- p. 542-543
Open AccessDemonstration of CFlow: Supporting Semantic Flow Analysis of Students' Code in Programming Problems at Scale
Introductory programming courses have been growing rapidly, now enrolling hundreds or thousands of students. In such large courses, it can be overwhelmingly difficult for instructors to understand class-wide problem-solving patterns or issues, which is crucial for improving instruction and addressing important pedagogical challenges. In this paper, we propose a technique and system, CFlow, for creating understandable and navigable representations of code at scale. CFlow is able to represent thousands of code samples in a visualization that resembles a single code sample. CFlow creates scalable code representations by (1) clustering individual statements with similar semantic purposes, (2) presenting clustered statements in a way that maintains semantic relationships between statements, (3) representing the correctness of different variations as a histogram, and (4) allowing users to navigate through solutions interactively using semantic filters. With a multi-level view design, users can navigate high-level patterns, and low-level implementations. This is in contrast to prior tools that either limit their focus on isolated statements (and thus discard the surrounding context of those statements) or cluster entire code samples (which can lead to large numbers of clusters---for example, if there are n code features and m implementations of each, there can be m^n clusters).
- Open Access
- 11 May 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-4
Improving Advising Relationships Between PhD Students and Faculty in Human-Computer Interaction
Advisor-advisee relationships between PhD students and faculty are vital to research, but advising dynamics can be challenging for both student and advisor. Though advising can involve egregious problems such as sexual harassment, we focus on what might be less serious but more common issues such as exploitation, unprofessional behavior, mishandling of credit, and inadequate communication. While problems can be caused by advisor or advisee, the power imbalance exacerbates problems for PhD students. In any case, open discussion about PhD advising is rare. In this panel, we hope to start a much-needed conversation about PhD advising to raise awareness within the SIGCHI community about common advising problems; and to begin brainstorming solutions that faculty, administrators, and PhD students can implement.
- 11 May 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-4
Open AccessImproving Advising Relationships Between PhD Students and Faculty in Human-Computer Interaction
Advisor-advisee relationships between PhD students and faculty are vital to research, but advising dynamics can be challenging for both student and advisor. Though advising can involve egregious problems such as sexual harassment, we focus on what might be less serious but more common issues such as exploitation, unprofessional behavior, mishandling of credit, and inadequate communication. While problems can be caused by advisor or advisee, the power imbalance exacerbates problems for PhD students. In any case, open discussion about PhD advising is rare. In this panel, we hope to start a much-needed conversation about PhD advising to raise awareness within the SIGCHI community about common advising problems; and to begin brainstorming solutions that faculty, administrators, and PhD students can implement.
- Open Access
- 11 May 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-18
Towards Inclusive Source Code Readability Based on the Preferences of Programmers with Visual Impairments
- 11 May 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-18
Open AccessTowards Inclusive Source Code Readability Based on the Preferences of Programmers with Visual Impairments