Chris Kennelly
Research Subjects
Chris Kennelly has a total of 44 co-authors. Here are the top co-authors by number of shared publications:
- Parthasarathy Ranganathan
Google, Mountain View, California, USA
5 shared publications - Martin Maas
Google LLC, Mountain View, California, United States
2 shared publications - Sotiris Apostolakis
Google
2 shared publications - Xinliang David Li
Google, Mountain View, USA
2 shared publications - Akanksha Jain
Google, Mountain View, CA, USA
1 shared publication - Alexander Potapenko
Google, Munich, Germany
1 shared publication
Latest Publications
- Open Access
- 25 September 2026
- Published by Association for Computing Machinery (ACM) in Communications of the ACM
- Vol. 69 (10) , 46-56
Instruction Set Migration at Warehouse Scale
Migrating codebases from one instruction set architecture (ISA) to another is a major engineering challenge. A recent example is the adoption of Arm (in addition to x86) across major cloud hyperscalers. Yet, this problem has seen limited attention from the academic community. Most work assumes migration is straightforward in the presence of source code, or that the main research challenges lie in binary translation used in its absence. We find that neither assumption holds today. Modern ISA migrations can build on a robust open-source ecosystem, making it possible to recompile all relevant software from scratch. However, while this avoids the need for binary translation, modern software ecosystems introduce a new, multifaceted set of challenges. By analyzing a large-scale migration from x86 to Arm at Google spanning nearly 40,000 code commits, we derive a taxonomy of tasks involved in modern ISA migration. We show how Google automated many of these steps and demonstrate how AI can play a major role in performing them. Finally, we identify tasks that remain difficult and highlight research challenges warranting further attention.
- 25 September 2026
- Published by Association for Computing Machinery (ACM) in Communications of the ACM
- Vol. 69 (10) , 46-56
Open AccessInstruction Set Migration at Warehouse Scale
Migrating codebases from one instruction set architecture (ISA) to another is a major engineering challenge. A recent example is the adoption of Arm (in addition to x86) across major cloud hyperscalers. Yet, this problem has seen limited attention from the academic community. Most work assumes migration is straightforward in the presence of source code, or that the main research challenges lie in binary translation used in its absence. We find that neither assumption holds today. Modern ISA migrations can build on a robust open-source ecosystem, making it possible to recompile all relevant software from scratch. However, while this avoids the need for binary translation, modern software ecosystems introduce a new, multifaceted set of challenges. By analyzing a large-scale migration from x86 to Arm at Google spanning nearly 40,000 code commits, we derive a taxonomy of tasks involved in modern ISA migration. We show how Google automated many of these steps and demonstrate how AI can play a major role in performing them. Finally, we identify tasks that remain difficult and highlight research challenges warranting further attention.
- Open Access
- 1 September 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Micro
- Vol. 46 (5) , 54-61
Necro-reaper: Pruning Away Dead Memory Traffic in Warehouse-Scale Computers
Memory bandwidth is emerging as a critical bottleneck in warehouse-scale computing (WSC). This work reveals that a significant portion of memory traffic in WSC is surprisingly unnecessary, consisting of needless writebacks of deallocated data and fetches of uninitialized data. This issue is particularly acute in WSC, where short-lived heap allocations bigger than a cache line are prevalent. To address this problem, this work proposes a pragmatic approach tailored to WSC. Leveraging the existing WSC ecosystem of vertical integration, profile-guided compilation flows, and customized memory allocators, this work presents NECRO-REAPER, a novel software/hardware co-design that avoids dead memory traffic without requiring the hardware tracking of prior work. Evaluations across diverse WSC workloads demonstrate that NECRO-REAPER reduces memory traffic by 26% and improves IPC by 6%. More broadly, this work drives a paradigm shift in memory efficiency, moving the design goal from simply optimizing raw bandwidth to maximizing semantic goodput.
- 1 September 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Micro
- Vol. 46 (5) , 54-61
Open AccessNecro-reaper: Pruning Away Dead Memory Traffic in Warehouse-Scale Computers
Memory bandwidth is emerging as a critical bottleneck in warehouse-scale computing (WSC). This work reveals that a significant portion of memory traffic in WSC is surprisingly unnecessary, consisting of needless writebacks of deallocated data and fetches of uninitialized data. This issue is particularly acute in WSC, where short-lived heap allocations bigger than a cache line are prevalent. To address this problem, this work proposes a pragmatic approach tailored to WSC. Leveraging the existing WSC ecosystem of vertical integration, profile-guided compilation flows, and customized memory allocators, this work presents NECRO-REAPER, a novel software/hardware co-design that avoids dead memory traffic without requiring the hardware tracking of prior work. Evaluations across diverse WSC workloads demonstrate that NECRO-REAPER reduces memory traffic by 26% and improves IPC by 6%. More broadly, this work drives a paradigm shift in memory efficiency, moving the design goal from simply optimizing raw bandwidth to maximizing semantic goodput.
- Open Access
- 30 March 2025
- Published by Association for Computing Machinery (ACM)
- p. 689-703
Necro-reaper: Pruning away Dead Memory Traffic in Warehouse-Scale Computers
- 30 March 2025
- Published by Association for Computing Machinery (ACM)
- p. 689-703
Open AccessNecro-reaper: Pruning away Dead Memory Traffic in Warehouse-Scale Computers
- Open Access
- 27 April 2024
- Published by Association for Computing Machinery (ACM)
- p. 577-590
Limoncello: Prefetchers for Scale
This paper presents Limoncello, a novel software system that dynamically configures data prefetchers for high-utilization systems. We demonstrate that in resource-constrained environments, such as large data centers, traditional methods of hardware prefetching can increase memory latency and decrease available memory bandwidth. To address this issue, Limoncello disables hardware prefetchers when memory bandwidth utilization is high, and it leverages targeted software prefetching to reduce cache misses when hardware prefetchers are disabled. Limoncello is software-centric and does not require any modifications to hardware. Our evaluation of the deployment on Google's fleet reveals that Limoncello unlocks significant performance gains for high-utilization systems: It improves application throughput by 10%, due to a 15% reduction in memory latency, while maintaining minimal change in cache miss rate for targeted library functions.
- 27 April 2024
- Published by Association for Computing Machinery (ACM)
- p. 577-590
Open AccessLimoncello: Prefetchers for Scale
This paper presents Limoncello, a novel software system that dynamically configures data prefetchers for high-utilization systems. We demonstrate that in resource-constrained environments, such as large data centers, traditional methods of hardware prefetching can increase memory latency and decrease available memory bandwidth. To address this issue, Limoncello disables hardware prefetchers when memory bandwidth utilization is high, and it leverages targeted software prefetching to reduce cache misses when hardware prefetchers are disabled. Limoncello is software-centric and does not require any modifications to hardware. Our evaluation of the deployment on Google's fleet reveals that Limoncello unlocks significant performance gains for high-utilization systems: It improves application throughput by 10%, due to a 15% reduction in memory latency, while maintaining minimal change in cache miss rate for targeted library functions.
- Open Access
- 27 April 2024
- Published by Association for Computing Machinery (ACM)
- p. 192-206
Characterizing a Memory Allocator at Warehouse Scale
Memory allocation constitutes a substantial component of warehouse-scale computation. Optimizing the memory allocator not only reduces the datacenter tax, but also improves application performance, leading to significant cost savings. We present the first comprehensive characterization study of TCMalloc, a memory allocator used by warehouse-scale applications in Google's production fleet. Our characterization reveals a profound diversity in the memory allocation patterns, allocated object sizes and lifetimes, for large-scale datacenter workloads, as well as in their performance on heterogeneous hardware platforms. Based on these insights, we optimize TCMalloc for warehouse-scale environments. Specifically, we propose optimizations for each level of its cache hierarchy that include usage-based dynamic sizing of allocator caches, leveraging hardware topology to mitigate inter-core communication overhead, and improving allocation packing algorithms based on statistical data. We evaluate these design choices using benchmarks and fleet-wide A/B experiments in our production fleet, resulting in a 1.4% improvement in throughput and a 3.4% reduction in RAM usage for the entire fleet. For the applications with the highest memory allocation usage, we observe up to 8.1% and 6.3% improvement in throughput and memory usage respectively. At our scale, even a single percent CPU or memory improvement translates to significant savings in server costs.
- 27 April 2024
- Published by Association for Computing Machinery (ACM)
- p. 192-206
Open AccessCharacterizing a Memory Allocator at Warehouse Scale
Memory allocation constitutes a substantial component of warehouse-scale computation. Optimizing the memory allocator not only reduces the datacenter tax, but also improves application performance, leading to significant cost savings. We present the first comprehensive characterization study of TCMalloc, a memory allocator used by warehouse-scale applications in Google's production fleet. Our characterization reveals a profound diversity in the memory allocation patterns, allocated object sizes and lifetimes, for large-scale datacenter workloads, as well as in their performance on heterogeneous hardware platforms. Based on these insights, we optimize TCMalloc for warehouse-scale environments. Specifically, we propose optimizations for each level of its cache hierarchy that include usage-based dynamic sizing of allocator caches, leveraging hardware topology to mitigate inter-core communication overhead, and improving allocation packing algorithms based on statistical data. We evaluate these design choices using benchmarks and fleet-wide A/B experiments in our production fleet, resulting in a 1.4% improvement in throughput and a 3.4% reduction in RAM usage for the entire fleet. For the applications with the highest memory allocation usage, we observe up to 8.1% and 6.3% improvement in throughput and memory usage respectively. At our scale, even a single percent CPU or memory improvement translates to significant savings in server costs.
- Open Access
- 14 April 2024
- Published by Association for Computing Machinery (ACM)
- p. 168-177
GWP-ASan: Sampling-Based Detection of Memory-Safety Bugs in Production
Despite the recent advances in pre-production bug detection, heap-use-after-free and heap-buffer-overflow bugs remain the primary problem for security, reliability, and developer productivity for applications written in C or C++, across all major software ecosystems. Memory-safe languages solve this problem when they are used, but the existing code bases consisting of billions of lines of C and C++ continue to grow, and we need additional bug detection mechanisms. This paper describes a family of tools that detect these two classes of memory-safety bugs, while running in production, at near-zero overhead. These tools combine page-granular guarded allocation and low-rate sampling. In other words, we added an "if" statement to a 36-year-old idea and made it work at scale. We describe the basic algorithm, several of its variants and implementations, and the results of multi-year deployments across mobile, desktop, and server applications.
- 14 April 2024
- Published by Association for Computing Machinery (ACM)
- p. 168-177
Open AccessGWP-ASan: Sampling-Based Detection of Memory-Safety Bugs in Production
Despite the recent advances in pre-production bug detection, heap-use-after-free and heap-buffer-overflow bugs remain the primary problem for security, reliability, and developer productivity for applications written in C or C++, across all major software ecosystems. Memory-safe languages solve this problem when they are used, but the existing code bases consisting of billions of lines of C and C++ continue to grow, and we need additional bug detection mechanisms. This paper describes a family of tools that detect these two classes of memory-safety bugs, while running in production, at near-zero overhead. These tools combine page-granular guarded allocation and low-rate sampling. In other words, we added an "if" statement to a 36-year-old idea and made it work at scale. We describe the basic algorithm, several of its variants and implementations, and the results of multi-year deployments across mobile, desktop, and server applications.
- Open Access
- 22 June 2021
- Published by Association for Computing Machinery (ACM)
- p. 39-51
automemcpy: a framework for automatic generation of fundamental memory operations
Memory manipulation primitives (memcpy, memset, memcmp) are used by virtually every application, from high performance computing to user interfaces. They often consume a significant portion of CPU cycles. Because they are so ubiquitous and critical, they are provided by language runtimes and in particular by libc, the C standard library. These implementations are heavily optimized, typically written in hand-tuned assembly for each target architecture. In this article, we propose a principled alternative to hand-tuning these functions: (1) we profile the calls to these functions in their production environment and use this data to drive the important high-level algorithmic decisions, (2) we use a high-level language for the implementation, delegate the job of tuning the generated code to the compiler, and (3) we use constraint programming and automatic benchmarks to select the optimal high-level structure of the functions. We compile our memfunctions implementations using the same compiler toolchain that we use for application code, which allows leveraging the compiler further by allowing whole-program optimization. We have evaluated our approach by applying it to the fleet of one of the largest computing enterprises in the world. This work increased the performance of the fleet by 1%.
- 22 June 2021
- Published by Association for Computing Machinery (ACM)
- p. 39-51
Open Accessautomemcpy: a framework for automatic generation of fundamental memory operations
Memory manipulation primitives (memcpy, memset, memcmp) are used by virtually every application, from high performance computing to user interfaces. They often consume a significant portion of CPU cycles. Because they are so ubiquitous and critical, they are provided by language runtimes and in particular by libc, the C standard library. These implementations are heavily optimized, typically written in hand-tuned assembly for each target architecture. In this article, we propose a principled alternative to hand-tuning these functions: (1) we profile the calls to these functions in their production environment and use this data to drive the important high-level algorithmic decisions, (2) we use a high-level language for the implementation, delegate the job of tuning the generated code to the compiler, and (3) we use constraint programming and automatic benchmarks to select the optimal high-level structure of the functions. We compile our memfunctions implementations using the same compiler toolchain that we use for application code, which allows leveraging the compiler further by allowing whole-program optimization. We have evaluated our approach by applying it to the fleet of one of the largest computing enterprises in the world. This work increased the performance of the fleet by 1%.
- Open Access
- 22 June 2021
- Published by Association for Computing Machinery (ACM)
- p. 28-38
Adaptive huge-page subrelease for non-moving memory allocators in warehouse-scale computers
Modern C++ server workloads rely on 2 MB huge pages to improve memory system performance via higher TLB hit rates. Huge pages have traditionally been supported at the kernel level, but recent work has shown that user-level, huge page-aware memory allocators can achieve higher huge page coverage and thus performance. These memory allocators deal with a trade-off: 1) allocate memory from the operating system (OS) at the granularity of a huge page, achieve high performance, but potentially waste memory due to fragmentation, or 2) limit fragmentation by breaking up huge pages into smaller 4 KB pages and returning them to the OS, but reduce performance due to lower huge page coverage. For example, the state-of-the-art TCMalloc allocator handles this trade-off by releasing memory to the OS at a configurable release rate, breaking up huge pages as necessary. This approach balances performance and fragmentation well for machines running one workload. For multiple applications on the same machine however, the reduction in memory usage is only useful to overall performance if another workload uses this memory. In warehouse-scale computers, when an application releases and then reacquires the same amount or more memory quickly, but no other application uses the memory in the meantime, the release causes poorer huge page coverage without any system-wide benefit. We introduce a metric, realized fragmentation, to capture this effect. We then present an adaptive release policy that dynamically determines when to break up huge pages and return them to the OS to optimize system-wide performance. We built this policy into TCMalloc and deployed it fleet-wide in our data centers, leading to an estimated 1% fleet-wide throughput improvement at negligible memory overhead.
- 22 June 2021
- Published by Association for Computing Machinery (ACM)
- p. 28-38
Open AccessAdaptive huge-page subrelease for non-moving memory allocators in warehouse-scale computers
Modern C++ server workloads rely on 2 MB huge pages to improve memory system performance via higher TLB hit rates. Huge pages have traditionally been supported at the kernel level, but recent work has shown that user-level, huge page-aware memory allocators can achieve higher huge page coverage and thus performance. These memory allocators deal with a trade-off: 1) allocate memory from the operating system (OS) at the granularity of a huge page, achieve high performance, but potentially waste memory due to fragmentation, or 2) limit fragmentation by breaking up huge pages into smaller 4 KB pages and returning them to the OS, but reduce performance due to lower huge page coverage. For example, the state-of-the-art TCMalloc allocator handles this trade-off by releasing memory to the OS at a configurable release rate, breaking up huge pages as necessary. This approach balances performance and fragmentation well for machines running one workload. For multiple applications on the same machine however, the reduction in memory usage is only useful to overall performance if another workload uses this memory. In warehouse-scale computers, when an application releases and then reacquires the same amount or more memory quickly, but no other application uses the memory in the meantime, the release causes poorer huge page coverage without any system-wide benefit. We introduce a metric, realized fragmentation, to capture this effect. We then present an adaptive release policy that dynamically determines when to break up huge pages and return them to the OS to optimize system-wide performance. We built this policy into TCMalloc and deployed it fleet-wide in our data centers, leading to an estimated 1% fleet-wide throughput improvement at negligible memory overhead.