Josh Pollock
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Research Interests
Josh Pollock has a total of 24 co-authors. Here are the top co-authors by number of shared publications:
- Arvind Satyanarayan
CSAIL, MIT, Cambridge, Massachusetts, USA
4 shared publications - Catherine Mei
CSAIL, MIT, Cambridge, Massachusetts, USA
2 shared publications - Daniel Hajas
Global Disability Innovation Hub University College London
1 shared publication - Daniel Jackson
Massachusetts Institute of Technology, United States
1 shared publication - Elliot Evans
Unaffiliated, Canada
1 shared publication - Grace Huang
CSAIL, MIT, United States
1 shared publication
Latest Publications
- Open Access
- 3 December 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Visualization and Computer Graphics
- Vol. 32 (1) , 549-559
GoFish: A Grammar of More Graphics!
Visualization grammars from ggplot2 to Vega-Lite are based on the Grammar of Graphics (GoG), our most comprehensive formal theory of visualization. The GoG helped expand the expressive gamut of visualization by moving beyond fixed chart types and towards a design space of composable operators. Yet, the resultant design space has surprising limitations, inconsistencies, and cliffs — even seemingly simple charts like mosaics, waffles, and ribbons fall out of scope of most GoG implementations. To author such charts, visualization designers must either rely on overburdened grammar developers to implement purpose-built mark types (thus reintroducing the issues of typologies) or drop to lower-level frameworks. In response, we present GoFish: a declarative visualization grammar that formalizes Gestalt principles (e.g., uniform spacing, containment, and connection) that have heretofore been complected in GoG constructs. These graphical operators achieve greater expressive power than their predecessors by enabling recursive composition: they can be nested and overlapped arbitrarily. Through a diverse example gallery, we demonstrate how graphical operators free users to arrange shapes in many different ways while retaining the benefits of high-level grammars like scale resolution and coordinate transform management. Recursive composition naturally yields an infinite design space that blurs the boundary between an expressive, low-level grammar and a concise, high-level one. In doing so, we point towards an updated theory of visualization, one that is open to an innumerable space of graphic representations instead of limited to a fixed set of “good” designs.
- 3 December 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Visualization and Computer Graphics
- Vol. 32 (1) , 549-559
Open AccessGoFish: A Grammar of More Graphics!
Visualization grammars from ggplot2 to Vega-Lite are based on the Grammar of Graphics (GoG), our most comprehensive formal theory of visualization. The GoG helped expand the expressive gamut of visualization by moving beyond fixed chart types and towards a design space of composable operators. Yet, the resultant design space has surprising limitations, inconsistencies, and cliffs — even seemingly simple charts like mosaics, waffles, and ribbons fall out of scope of most GoG implementations. To author such charts, visualization designers must either rely on overburdened grammar developers to implement purpose-built mark types (thus reintroducing the issues of typologies) or drop to lower-level frameworks. In response, we present GoFish: a declarative visualization grammar that formalizes Gestalt principles (e.g., uniform spacing, containment, and connection) that have heretofore been complected in GoG constructs. These graphical operators achieve greater expressive power than their predecessors by enabling recursive composition: they can be nested and overlapped arbitrarily. Through a diverse example gallery, we demonstrate how graphical operators free users to arrange shapes in many different ways while retaining the benefits of high-level grammars like scale resolution and coordinate transform management. Recursive composition naturally yields an infinite design space that blurs the boundary between an expressive, low-level grammar and a concise, high-level one. In doing so, we point towards an updated theory of visualization, one that is open to an innumerable space of graphic representations instead of limited to a fixed set of “good” designs.
- Open Access
- 22 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 1-17
Benthic: Perceptually Congruent Structures for Accessible Charts and Diagrams
- 22 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 1-17
Open AccessBenthic: Perceptually Congruent Structures for Accessible Charts and Diagrams
- Open Access
- 13 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-3
Dynamic Abstractions: Building the Next Generation of Cognitive Tools and Interfaces | Adjunct Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology
This workshop provides a forum to discuss, brainstorm, and prototype the next generation of interfaces that leverage the dynamic experiences enabled by recent advances in AI and the generative capabilities of foundation models. These models simplify complex tasks by generating outputs in various representations (e.g., text, images, videos) through diverse input modalities like natural language, voice, and sketch. They interpret user intent to generate and transform representations, potentially changing how we interact with information and express ideas. Inspired by this potential, technologists, theorists, and researchers are exploring new forms of interaction by building demos and communities dedicated to concretizing and advancing the vision of working with dynamic abstractions. This UIST workshop provides a timely space to discuss AI’s impact on how we might design and use cognitive tools (e.g., languages, notations, diagrams). We will explore the challenges, critiques, and opportunities of this space by thinking through and prototyping use cases across various domains.
- 13 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-3
Open AccessDynamic Abstractions: Building the Next Generation of Cognitive Tools and Interfaces | Adjunct Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology
This workshop provides a forum to discuss, brainstorm, and prototype the next generation of interfaces that leverage the dynamic experiences enabled by recent advances in AI and the generative capabilities of foundation models. These models simplify complex tasks by generating outputs in various representations (e.g., text, images, videos) through diverse input modalities like natural language, voice, and sketch. They interpret user intent to generate and transform representations, potentially changing how we interact with information and express ideas. Inspired by this potential, technologists, theorists, and researchers are exploring new forms of interaction by building demos and communities dedicated to concretizing and advancing the vision of working with dynamic abstractions. This UIST workshop provides a timely space to discuss AI’s impact on how we might design and use cognitive tools (e.g., languages, notations, diagrams). We will explore the challenges, critiques, and opportunities of this space by thinking through and prototyping use cases across various domains.
- Open Access
- 11 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-21
Bluefish: Composing Diagrams with Declarative Relations
Diagrams are essential tools for problem-solving and communication as they externalize conceptual structures using spatial relationships. But when picking a diagramming framework, users are faced with a dilemma. They can either use a highly expressive but low-level toolkit, whose API does not match their domain-specific concepts, or select a high-level typology, which offers a recognizable vocabulary but supports a limited range of diagrams. To address this gap, we introduce Bluefish: a diagramming framework inspired by component-based user interface (UI) libraries. Bluefish lets users create diagrams using relations: declarative, composable, and extensible diagram fragments that relax the concept of a UI component. Unlike a component, a relation does not have sole ownership over its children nor does it need to fully specify their layout. To render diagrams, Bluefish extends a traditional tree-based scenegraph to a compound graph that captures both hierarchical and adjacent relationships between nodes. To evaluate our system, we construct a diverse example gallery covering many domains including mathematics, physics, computer science, and even cooking. We show that Bluefish’s relations are effective declarative primitives for diagrams. Bluefish is open source, and we aim to shape it into both a usable tool and a research platform.
- 11 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-21
Open AccessBluefish: Composing Diagrams with Declarative Relations
Diagrams are essential tools for problem-solving and communication as they externalize conceptual structures using spatial relationships. But when picking a diagramming framework, users are faced with a dilemma. They can either use a highly expressive but low-level toolkit, whose API does not match their domain-specific concepts, or select a high-level typology, which offers a recognizable vocabulary but supports a limited range of diagrams. To address this gap, we introduce Bluefish: a diagramming framework inspired by component-based user interface (UI) libraries. Bluefish lets users create diagrams using relations: declarative, composable, and extensible diagram fragments that relax the concept of a UI component. Unlike a component, a relation does not have sole ownership over its children nor does it need to fully specify their layout. To render diagrams, Bluefish extends a traditional tree-based scenegraph to a compound graph that captures both hierarchical and adjacent relationships between nodes. To evaluate our system, we construct a diverse example gallery covering many domains including mathematics, physics, computer science, and even cooking. We show that Bluefish’s relations are effective declarative primitives for diagrams. Bluefish is open source, and we aim to shape it into both a usable tool and a research platform.
- Open Access
Relay: A High-Level Compiler for Deep Learning
Frameworks for writing, compiling, and optimizing deep learning (DL) models have recently enabled progress in areas like computer vision and natural language processing. Extending these frameworks to accommodate the rapidly diversifying landscape of DL models and hardware platforms presents challenging tradeoffs between expressivity, composability, and portability. We present Relay, a new compiler framework for DL. Relay's functional, statically typed intermediate representation (IR) unifies and generalizes existing DL IRs to express state-of-the-art models. The introduction of Relay's expressive IR requires careful design of domain-specific optimizations, addressed via Relay's extension mechanisms. Using these extension mechanisms, Relay supports a unified compiler that can target a variety of hardware platforms. Our evaluation demonstrates Relay's competitive performance for a broad class of models and devices (CPUs, GPUs, and emerging accelerators). Relay's design demonstrates how a unified IR can provide expressivity, composability, and portability without compromising performance.
Open AccessRelay: A High-Level Compiler for Deep Learning
Frameworks for writing, compiling, and optimizing deep learning (DL) models have recently enabled progress in areas like computer vision and natural language processing. Extending these frameworks to accommodate the rapidly diversifying landscape of DL models and hardware platforms presents challenging tradeoffs between expressivity, composability, and portability. We present Relay, a new compiler framework for DL. Relay's functional, statically typed intermediate representation (IR) unifies and generalizes existing DL IRs to express state-of-the-art models. The introduction of Relay's expressive IR requires careful design of domain-specific optimizations, addressed via Relay's extension mechanisms. Using these extension mechanisms, Relay supports a unified compiler that can target a variety of hardware platforms. Our evaluation demonstrates Relay's competitive performance for a broad class of models and devices (CPUs, GPUs, and emerging accelerators). Relay's design demonstrates how a unified IR can provide expressivity, composability, and portability without compromising performance.