Zhihan Guo
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Zhihan Guo has a total of 4 co-authors. Here are the top co-authors by number of shared publications:
- Irwin King
Department of Computer Science and Engineering, The Chinese University of Hong Kong, Hong Kong, Hong Kong
1 shared publication - Yifei Zhang
Northwestern Polytechnical University, Xi'an, China
1 shared publication - Zenglin Xu
Shanghai Academy of AI for Science Shanghai China
1 shared publication - Zhuo Zhang
Harbin Institute of Technology (Shenzhen), Shenzhen, China and Peng Cheng Laboratory, Shenzhen, China
1 shared publication
Latest Publications
- Open Access
- 13 May 2024
- Published by Association for Computing Machinery (ACM)
- p. 1558-1567
FedHLT: Efficient Federated Low-Rank Adaption with Hierarchical Language Tree for Multilingual Modeling
Federated Multilingual Modeling (FMM) has become an essential approach in natural language processing (NLP) due to increasing linguistic diversity and the heightened emphasis on data privacy. However, FMM faces two primary challenges: 1) the high communication costs inherent in network operations, and 2) the complexities arising from parameter interference, as languages exhibit both unique characteristics and shared features. To tackle these issues, we introduce a communication-efficient framework for Multilingual Modeling (MM) that combines low-rank adaptation with a hierarchical language tree structure. Our method maintains the base model's weights while focusing on updating only the Low-rank adaptation (LoRA) parameters, significantly reducing communication costs. Additionally, we mitigate parameter conflicts by organizing languages based on their familial ties rather than merging all LoRA parameters together. Our experimental findings reveal that this novel model surpasses established baseline models in performance and markedly decreases communication overhead.
- 13 May 2024
- Published by Association for Computing Machinery (ACM)
- p. 1558-1567
Open AccessFedHLT: Efficient Federated Low-Rank Adaption with Hierarchical Language Tree for Multilingual Modeling
Federated Multilingual Modeling (FMM) has become an essential approach in natural language processing (NLP) due to increasing linguistic diversity and the heightened emphasis on data privacy. However, FMM faces two primary challenges: 1) the high communication costs inherent in network operations, and 2) the complexities arising from parameter interference, as languages exhibit both unique characteristics and shared features. To tackle these issues, we introduce a communication-efficient framework for Multilingual Modeling (MM) that combines low-rank adaptation with a hierarchical language tree structure. Our method maintains the base model's weights while focusing on updating only the Low-rank adaptation (LoRA) parameters, significantly reducing communication costs. Additionally, we mitigate parameter conflicts by organizing languages based on their familial ties rather than merging all LoRA parameters together. Our experimental findings reveal that this novel model surpasses established baseline models in performance and markedly decreases communication overhead.