Fan Lai
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Fan Lai has a total of 53 co-authors. Here are the top co-authors by number of shared publications:
- Mosharaf Chowdhury
University of Michigan, Ann Arbor, Michigan, USA
3 shared publications - Kai Chen
Hong Kong University of Science and Technology, Hong Kong, Hong Kong
2 shared publications - Aditya Akella
The University of Texas at Austin, Austin, TX, USA
1 shared publication - Ang Chen
University of Michigan, Ann Arbor, USA
1 shared publication - Arvind Krishnamurthy
University of Washington, Seattle, USA
1 shared publication - Chengsong Zhang
University of Illinois Urbana Champaign, Champaign, USA
1 shared publication
Latest Publications
- Open Access
- 11 August 2026
- Published by Association for Computing Machinery (ACM)
- p. 1617-1629
EMA: Efficient Model Adaptation for Learning-based Systems
- 11 August 2026
- Published by Association for Computing Machinery (ACM)
- p. 1617-1629
Open AccessEMA: Efficient Model Adaptation for Learning-based Systems
- 17 November 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Computers
- Vol. 75 (2) , 448-461
FIITED: Fine-Grained Embedding Dimension Optimization During Training for Recommender Systems
Huge embedding tables in modern deep learning recommender models (DLRM) require prohibitively large memory during training and inference. This paper proposes FIITED, a system to automatically reduce the memory footprint via FIne-grained In-Training Embedding Dimension pruning. By leveraging the key insight that embedding vectors are not equally important, FIITED adaptively adjusts the dimension of each individual embedding vector during model training, assigning larger dimensions to more important embeddings while adapting to dynamic changes in data. We prioritize embedding dimensions with higher frequencies and gradients as more important. To enable efficient pruning of embeddings and their dimensions during model training, we propose an embedding storage system based on virtually-hashed physically-indexed hash tables. Experiments on two industry models and months of realistic datasets show that FIITED can reduce DLRM embedding size by more than 65% while preserving model quality, outperforming state-of-the-art in-training embedding pruning methods and increasing the reduction ratio by 1.3$\boldsymbol{\times}$ to 1.67$\boldsymbol{\times}$ . On public datasets, FIITED can reduce the size of embedding tables by 2.2$\boldsymbol{\times}$ to 800$\boldsymbol{\times}$ with negligible accuracy drop, achieving 1.05$\boldsymbol{\times}$ to 12.5$\boldsymbol{\times}$ improvement in reduction ratio compared to baselines while improving model throughput.- 17 November 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Computers
- Vol. 75 (2) , 448-461
FIITED: Fine-Grained Embedding Dimension Optimization During Training for Recommender Systems
Huge embedding tables in modern deep learning recommender models (DLRM) require prohibitively large memory during training and inference. This paper proposes FIITED, a system to automatically reduce the memory footprint via FIne-grained In-Training Embedding Dimension pruning. By leveraging the key insight that embedding vectors are not equally important, FIITED adaptively adjusts the dimension of each individual embedding vector during model training, assigning larger dimensions to more important embeddings while adapting to dynamic changes in data. We prioritize embedding dimensions with higher frequencies and gradients as more important. To enable efficient pruning of embeddings and their dimensions during model training, we propose an embedding storage system based on virtually-hashed physically-indexed hash tables. Experiments on two industry models and months of realistic datasets show that FIITED can reduce DLRM embedding size by more than 65% while preserving model quality, outperforming state-of-the-art in-training embedding pruning methods and increasing the reduction ratio by 1.3$\boldsymbol{\times}$ to 1.67$\boldsymbol{\times}$ . On public datasets, FIITED can reduce the size of embedding tables by 2.2$\boldsymbol{\times}$ to 800$\boldsymbol{\times}$ with negligible accuracy drop, achieving 1.05$\boldsymbol{\times}$ to 12.5$\boldsymbol{\times}$ improvement in reduction ratio compared to baselines while improving model throughput.- Open Access
- 17 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 609-625
NetZIP: Algorithm/Hardware Co-design of In-network Lossless Compression for Distributed Large Model Training
- 17 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 609-625
Open AccessNetZIP: Algorithm/Hardware Co-design of In-network Lossless Compression for Distributed Large Model Training
- Open Access
- 12 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 375-398
IC-Cache: Efficient Large Language Model Serving via In-context Caching
- 12 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 375-398
Open AccessIC-Cache: Efficient Large Language Model Serving via In-context Caching
- Open Access
- 11 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 38-45
Towards Fully Disaggregated Recommendation Model Serving
- 11 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 38-45
Open AccessTowards Fully Disaggregated Recommendation Model Serving
- Open Access
- 10 February 2025
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Management of Data
- Vol. 3 (1) , 1-27
Sequoia: An Accessible and Extensible Framework for Privacy-Preserving Machine Learning over Distributed Data
Privacy-preserving machine learning (PPML) algorithms use secure computation protocols to allow multiple data parties to collaboratively train machine learning (ML) models while maintaining their data confidentiality. However, current PPML frameworks couple secure protocols with ML models in PPML algorithm implementations, making it challenging for non-experts to develop and optimize PPML applications, limiting their accessibility and performance. We propose Sequoia, a novel PPML framework that decouples ML models and secure protocols to optimize the development and execution of PPML applications across data parties. Sequoia offers JAX-compatible APIs for users to program their ML models, while using a compiler-executor architecture to automatically apply PPML algorithms and system optimizations for model execution over distributed data. The compiler in Sequoia incorporates cross-party PPML processes into user-defined ML models by transparently adding computation, encryption, and communication steps with extensible policies, and the executor efficiently schedules code execution across multiple data parties, considering data dependencies and device heterogeneity. Compared to existing PPML frameworks, Sequoia requires 64%-92% fewer lines of code for users to implement the same PPML algorithms, and achieves 88% speedup of training throughput in horizontal PPML.
- 10 February 2025
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Management of Data
- Vol. 3 (1) , 1-27
Open AccessSequoia: An Accessible and Extensible Framework for Privacy-Preserving Machine Learning over Distributed Data
Privacy-preserving machine learning (PPML) algorithms use secure computation protocols to allow multiple data parties to collaboratively train machine learning (ML) models while maintaining their data confidentiality. However, current PPML frameworks couple secure protocols with ML models in PPML algorithm implementations, making it challenging for non-experts to develop and optimize PPML applications, limiting their accessibility and performance. We propose Sequoia, a novel PPML framework that decouples ML models and secure protocols to optimize the development and execution of PPML applications across data parties. Sequoia offers JAX-compatible APIs for users to program their ML models, while using a compiler-executor architecture to automatically apply PPML algorithms and system optimizations for model execution over distributed data. The compiler in Sequoia incorporates cross-party PPML processes into user-defined ML models by transparently adding computation, encryption, and communication steps with extensible policies, and the executor efficiently schedules code execution across multiple data parties, considering data dependencies and device heterogeneity. Compared to existing PPML frameworks, Sequoia requires 64%-92% fewer lines of code for users to implement the same PPML algorithms, and achieves 88% speedup of training throughput in horizontal PPML.
- Open Access
- 5 December 2023
- Published by Association for Computing Machinery (ACM)
- p. 1-10
Flamingo: A User-Centric System for Fast and Energy-Efficient DNN Training on Smartphones
Training DNNs on a smartphone system-on-a-chip (SoC) without carefully considering its resource constraints leads to suboptimal training performance and significantly affects user experience. To this end, we present Flamingo, a system for smartphones that optimizes DNN training for time and energy under dynamic resource availability, by scaling parallelism and exploiting compute heterogeneity in real-time. As AI becomes a part of the mainstream smartphone experience, the need to train on-device becomes crucial to fine-tune predictive models while ensuring data privacy. Our experiments show that Flamingo achieves significant improvement in reducing time (12×) and energy (8×) for on-device training, while nearly eliminating detrimental user experience. Extensive large-scale evaluations show that Flamingo can improve end-to-end training performance by 1.2-23.3× and energy efficiency by 1.6--7× over the state-of-the-art.
- 5 December 2023
- Published by Association for Computing Machinery (ACM)
- p. 1-10
Open AccessFlamingo: A User-Centric System for Fast and Energy-Efficient DNN Training on Smartphones
Training DNNs on a smartphone system-on-a-chip (SoC) without carefully considering its resource constraints leads to suboptimal training performance and significantly affects user experience. To this end, we present Flamingo, a system for smartphones that optimizes DNN training for time and energy under dynamic resource availability, by scaling parallelism and exploiting compute heterogeneity in real-time. As AI becomes a part of the mainstream smartphone experience, the need to train on-device becomes crucial to fine-tune predictive models while ensuring data privacy. Our experiments show that Flamingo achieves significant improvement in reducing time (12×) and energy (8×) for on-device training, while nearly eliminating detrimental user experience. Extensive large-scale evaluations show that Flamingo can improve end-to-end training performance by 1.2-23.3× and energy efficiency by 1.6--7× over the state-of-the-art.
- Open Access
- 8 May 2023
- Published by Association for Computing Machinery (ACM)
- p. 851-866
Egeria: Efficient DNN Training with Knowledge-Guided Layer Freezing
- 8 May 2023
- Published by Association for Computing Machinery (ACM)
- p. 851-866
Open AccessEgeria: Efficient DNN Training with Knowledge-Guided Layer Freezing