Brushing and Interpretive Linking for Knowledge Work

Authors:
Open Access
Publisher website
Google Scholar
Abstract
This work builds on brushing and linking from data visualization and extends it to knowledge work. Knowledge workers frequently repurpose content across artifacts (e.g., creating a one‑page summary of a report and visuals for its presentation), yet the conceptual relationships between these representations are often implicit, placing a cognitive burden on knowledge workers to maintain alignment. We introduce interpretive linking, an approach that leverages generative AI to identify and highlight semantically related content across representations (brushing) and to interpret and propagate changes between them (linking). Rather than preserving strict correspondence, interpretive linking aims to maintain semantic coherence across formats and modalities. Using a research‑through‑design approach, we built a technology probe leveraging multiple models across textual and visual content and experimented with different strategies for managing semantic coherence. We then conducted a qualitative study with six knowledge workers, who used the probe to reflect on potential workflows. Participants generated 22 application concepts, ranging from responsive documents adapting to device form factor to collaborative tools supporting parallel work across modalities. Together, these findings surface design opportunities and open challenges for interpretive linking in knowledge work.