Yi Shen

Speculative decoding accelerates autoregressive large language model inference by producing low-cost draft work and committing accepted portions after parallel target-side verification. This draft-then-verify process defines the broad family, while recent methods differ in where draft work comes from, how candidate units are logically organized across drafting and verification, what fidelity claim licenses commitment, and how runtime systems realize committed state, control speculation, execute serving paths, and evaluate end-to-end performance. We therefore develop a systems co-design taxonomy with four interdependent aspects---Draft Source, Draft Geometry, Verification Fidelity, and Runtime Execution---and 19 categories. Applying it to 304 works yields Method Profiles that trace individual works through the design space and help researchers navigate the literature, compare methods consistently, and select category-matched baselines. We synthesize recurring combinations and interactions among the four aspects into Co-Design Gates---Representability, Verifiability, State Realizability, and Profitability---that organize open challenges and future directions. Together, the taxonomy, Method Profiles, and Gates frame speculative decoding as a systems co-design problem in which these choices must align to deliver the stated fidelity and end-to-end benefit.

Large language model agents are not merely long-prompt applications: they repeatedly prefill growing transcripts that mix instructions, tool schemas, tool observations, persistent memory, scratchpads, and sub-agent outputs, re-submitting the accumulated context on every turn. This creates a full-session optimization problem in which compression ratio alone is insufficient: practical designs must also preserve cacheable prefixes, recover omitted evidence when it is later needed, maintain provenance across privilege boundaries, and avoid delayed-relevance failures where prematurely discarded detail turns out to matter. This survey reframes context compression as lifecycle management for long-horizon agents rather than one-shot prompt shortening. We organize methods by where information is admitted, placed, compacted, recovered, reused, and governed across a session, then map four method families—natural-language, retrieval and offloading, soft-token, and KV-cache compression—together with learned policy-based methods into this lifecycle, rather than treating them as parallel silos. We analyze how production general-purpose agents instantiate these choices in practice, exposing the triggers, protected regions, and recovery sources that determine whether an agent survives a long run, and we synthesize the fast-moving 2025-2026 frontier in which compression becomes a learned action rather than an external scaffold. Finally, we propose an evaluation protocol that jointly reports task success, peak and total tokens, latency, cache behavior, recoverability, and security-relevant provenance, rather than compression ratio in isolation.