Ryan Yen
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Ryan Yen has a total of 24 co-authors. Here are the top co-authors by number of shared publications:
- Jian Zhao
School of Computer Science, University of Waterloo, Waterloo, Ontario, Canada
7 shared publications - Daniel Vogel
Cheriton School of Computer Science, University of Waterloo, Waterloo, Ontario, Canada
3 shared publications - Arvind Satyanarayan
CSAIL, MIT, Cambridge, Massachusetts, USA
2 shared publications - Li Feng
Computational Media and Arts Thrust, The Hong Kong University of Science and Technology (Guangzhou), China
2 shared publications - Nicole Sultanum
Tableau Research, Seattle, Washington, USA
2 shared publications - Sangho Suh
The Allen Institute for AI, Seattle, Washington, USA
2 shared publications
Latest Publications
- Open Access
- 2 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 1-3
Semipy: A Python Runtime to Program with Informal Spec
- 2 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 1-3
Open AccessSemipy: A Python Runtime to Program with Informal Spec
- Open Access
- 4 July 2025
- Published by Association for Computing Machinery (ACM)
- p. 1084-1106
To Search or To Gen? Design Dimensions Integrating Web Search and Generative AI in Programmers' Information-Seeking Process
- 4 July 2025
- Published by Association for Computing Machinery (ACM)
- p. 1084-1106
Open AccessTo Search or To Gen? Design Dimensions Integrating Web Search and Generative AI in Programmers' Information-Seeking Process
- Open Access
- 25 April 2025
- Published by Association for Computing Machinery (ACM)
- p. 1-17
Code Shaping: Iterative Code Editing with Free-form AI-Interpreted Sketching
- 25 April 2025
- Published by Association for Computing Machinery (ACM)
- p. 1-17
Open AccessCode Shaping: Iterative Code Editing with Free-form AI-Interpreted Sketching
- Open Access
- 13 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-3
Dynamic Abstractions: Building the Next Generation of Cognitive Tools and Interfaces | Adjunct Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology
This workshop provides a forum to discuss, brainstorm, and prototype the next generation of interfaces that leverage the dynamic experiences enabled by recent advances in AI and the generative capabilities of foundation models. These models simplify complex tasks by generating outputs in various representations (e.g., text, images, videos) through diverse input modalities like natural language, voice, and sketch. They interpret user intent to generate and transform representations, potentially changing how we interact with information and express ideas. Inspired by this potential, technologists, theorists, and researchers are exploring new forms of interaction by building demos and communities dedicated to concretizing and advancing the vision of working with dynamic abstractions. This UIST workshop provides a timely space to discuss AI’s impact on how we might design and use cognitive tools (e.g., languages, notations, diagrams). We will explore the challenges, critiques, and opportunities of this space by thinking through and prototyping use cases across various domains.
- 13 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-3
Open AccessDynamic Abstractions: Building the Next Generation of Cognitive Tools and Interfaces | Adjunct Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology
This workshop provides a forum to discuss, brainstorm, and prototype the next generation of interfaces that leverage the dynamic experiences enabled by recent advances in AI and the generative capabilities of foundation models. These models simplify complex tasks by generating outputs in various representations (e.g., text, images, videos) through diverse input modalities like natural language, voice, and sketch. They interpret user intent to generate and transform representations, potentially changing how we interact with information and express ideas. Inspired by this potential, technologists, theorists, and researchers are exploring new forms of interaction by building demos and communities dedicated to concretizing and advancing the vision of working with dynamic abstractions. This UIST workshop provides a timely space to discuss AI’s impact on how we might design and use cognitive tools (e.g., languages, notations, diagrams). We will explore the challenges, critiques, and opportunities of this space by thinking through and prototyping use cases across various domains.
- Open Access
- 13 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-3
Code Shaping: Iterative Code Editing with Free-form Sketching
We present an initial step towards building a system for programmers to edit code using free-form sketch annotations drawn directly onto editor and output windows. Using a working prototype system as a technical probe, an exploratory study (N = 6) examines how programmers sketch to annotate Python code to communicate edits for an AI model to perform. The results reveal personalized workflow strategies and how similar annotations vary in abstractness and intention across different scenarios and users.
- 13 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-3
Open AccessCode Shaping: Iterative Code Editing with Free-form Sketching
We present an initial step towards building a system for programmers to edit code using free-form sketch annotations drawn directly onto editor and output windows. Using a working prototype system as a technical probe, an exploratory study (N = 6) examines how programmers sketch to annotate Python code to communicate edits for an AI model to perform. The results reveal personalized workflow strategies and how similar annotations vary in abstractness and intention across different scenarios and users.
- Open Access
- 11 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-20
CoLadder: Manipulating Code Generation via Multi-Level Blocks
This paper adopted an iterative design process to gain insights into programmers’ strategies when using LLMs for programming. We proposed CoLadder, a novel system that supports programmers by facilitating hierarchical task decomposition, direct code segment manipulation, and result evaluation during prompt authoring. A user study with 12 experienced programmers showed that CoLadder is effective in helping programmers externalize their problem-solving intentions flexibly, improving their ability to evaluate and modify code across various abstraction levels, from their task’s goal to final code implementation.
- 11 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-20
Open AccessCoLadder: Manipulating Code Generation via Multi-Level Blocks
This paper adopted an iterative design process to gain insights into programmers’ strategies when using LLMs for programming. We proposed CoLadder, a novel system that supports programmers by facilitating hierarchical task decomposition, direct code segment manipulation, and result evaluation during prompt authoring. A user study with 12 experienced programmers showed that CoLadder is effective in helping programmers externalize their problem-solving intentions flexibly, improving their ability to evaluate and modify code across various abstraction levels, from their task’s goal to final code implementation.
- Open Access
- 11 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-16
Gait Gestures: Examining Stride and Foot Strike Variation as an Input Method While Walking
Walking is a cyclic pattern of alternating footstep strikes, with each pair of steps forming a stride, and a series of strides forming a gait. We conduct a systematic examination of different kinds of intentional variations from a normal gait that could be used as input actions without interrupting overall walking progress. A design space of 22 candidate Gait Gestures is generated by adapting previous standing foot input actions and identifying new actions possible in a walking context. A formative study (n=25) examines movement easiness, social acceptability, and walking compatibility with foot movement logging to calculate temporal and spatial characteristics. Using a categorization of these results, 7 gestures are selected for a wizard-of-oz prototype demonstrating an AR interface controlled by Gait Gestures for ordering food and audio playback while walking. As a technical proof-of-concept, a gait gesture recognizer is developed and tested using the formative study data.
- 11 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-16
Open AccessGait Gestures: Examining Stride and Foot Strike Variation as an Input Method While Walking
Walking is a cyclic pattern of alternating footstep strikes, with each pair of steps forming a stride, and a series of strides forming a gait. We conduct a systematic examination of different kinds of intentional variations from a normal gait that could be used as input actions without interrupting overall walking progress. A design space of 22 candidate Gait Gestures is generated by adapting previous standing foot input actions and identifying new actions possible in a walking context. A formative study (n=25) examines movement easiness, social acceptability, and walking compatibility with foot movement logging to calculate temporal and spatial characteristics. Using a categorization of these results, 7 gestures are selected for a wizard-of-oz prototype demonstrating an AR interface controlled by Gait Gestures for ordering food and audio playback while walking. As a technical proof-of-concept, a gait gesture recognizer is developed and tested using the formative study data.
- Open Access
- 11 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-22
Memolet: Reifying the Reuse of User-AI Conversational Memories
As users engage more frequently with AI conversational agents, conversations may exceed their “memory” capacity, leading to failures in correctly leveraging certain memories for tailored responses. However, in finding past memories that can be reused or referenced, users need to retrieve relevant information in various conversations and articulate to the AI their intention to reuse these memories. To support this process, we introduce Memolet, an interactive object that reifies memory reuse. Users can directly manipulate Memolet to specify which memories to reuse and how to use them. We developed a system demonstrating Memolet’s interaction across various memory reuse stages, including memory extraction, organization, prompt articulation, and generation refinement. We examine the system’s usefulness with an N=12 within-subject study and provide design implications for future systems that support user-AI conversational memory reusing.
- 11 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-22
Open AccessMemolet: Reifying the Reuse of User-AI Conversational Memories
As users engage more frequently with AI conversational agents, conversations may exceed their “memory” capacity, leading to failures in correctly leveraging certain memories for tailored responses. However, in finding past memories that can be reused or referenced, users need to retrieve relevant information in various conversations and articulate to the AI their intention to reuse these memories. To support this process, we introduce Memolet, an interactive object that reifies memory reuse. Users can directly manipulate Memolet to specify which memories to reuse and how to use them. We developed a system demonstrating Memolet’s interaction across various memory reuse stages, including memory extraction, organization, prompt articulation, and generation refinement. We examine the system’s usefulness with an N=12 within-subject study and provide design implications for future systems that support user-AI conversational memory reusing.
- Open Access
- 11 May 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-21
CoPrompt: Supporting Prompt Sharing and Referring in Collaborative Natural Language Programming
Natural language (NL) programming has become more approachable due to the powerful code-generation capability of large language models (LLMs). This shift to using NL to program enhances collaborative programming by reducing communication barriers and context-switching among programmers from varying backgrounds. However, programmers may face challenges during prompt engineering in a collaborative setting as they need to actively keep aware of their collaborators’ progress and intents. In this paper, we aim to investigate ways to assist programmers’ prompt engineering in a collaborative context. We first conducted a formative study to understand the workflows and challenges of programmers when using NL for collaborative programming. Based on our findings, we implemented a prototype, CoPrompt, to support collaborative prompt engineering by providing referring, requesting, sharing, and linking mechanisms. Our user study indicates that CoPrompt assists programmers in comprehending collaborators’ prompts and building on their collaborators’ work, reducing repetitive updates and communication costs.
- 11 May 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-21
Open AccessCoPrompt: Supporting Prompt Sharing and Referring in Collaborative Natural Language Programming
Natural language (NL) programming has become more approachable due to the powerful code-generation capability of large language models (LLMs). This shift to using NL to program enhances collaborative programming by reducing communication barriers and context-switching among programmers from varying backgrounds. However, programmers may face challenges during prompt engineering in a collaborative setting as they need to actively keep aware of their collaborators’ progress and intents. In this paper, we aim to investigate ways to assist programmers’ prompt engineering in a collaborative context. We first conducted a formative study to understand the workflows and challenges of programmers when using NL for collaborative programming. Based on our findings, we implemented a prototype, CoPrompt, to support collaborative prompt engineering by providing referring, requesting, sharing, and linking mechanisms. Our user study indicates that CoPrompt assists programmers in comprehending collaborators’ prompts and building on their collaborators’ work, reducing repetitive updates and communication costs.
- Open Access
- 2 May 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-8
To Search or To Gen? Exploring the Synergy between Generative AI and Web Search in Programming
The convergence of generative AI and web search is reshaping problem-solving for programmers. However, the lack of understanding regarding their interplay in the information-seeking process often leads programmers to perceive them as alternatives rather than complementary tools. To analyze this interaction and explore their synergy, we conducted an interview study with eight experienced programmers. Drawing from the results and literature, we have identified three major challenges and proposed three decision-making stages, each with its own relevant factors. Additionally, we present a comprehensive process model that captures programmers’ interaction patterns. This model encompasses decision-making stages, the information-foraging loop, and cognitive activities during system interaction, offering a holistic framework to comprehend and optimize the use of these convergent tools in programming.
- 2 May 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-8
Open AccessTo Search or To Gen? Exploring the Synergy between Generative AI and Web Search in Programming
The convergence of generative AI and web search is reshaping problem-solving for programmers. However, the lack of understanding regarding their interplay in the information-seeking process often leads programmers to perceive them as alternatives rather than complementary tools. To analyze this interaction and explore their synergy, we conducted an interview study with eight experienced programmers. Drawing from the results and literature, we have identified three major challenges and proposed three decision-making stages, each with its own relevant factors. Additionally, we present a comprehensive process model that captures programmers’ interaction patterns. This model encompasses decision-making stages, the information-foraging loop, and cognitive activities during system interaction, offering a holistic framework to comprehend and optimize the use of these convergent tools in programming.