LSLei Shi

Lei Shi

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Lei Shi has a total of 15 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    OSP: Boosting Distributed Model Training with 2-stage Synchronization

    by Zixuan Chen, Lei Shi, Xuandong Liu, Jiahui Li, Sen Liu, Yang Xu

    Distributed deep learning (DDL) is a promising research area, which aims to increase the efficiency of training deep learning tasks with large size of datasets and models. As the computation capability of DDL nodes continues to increase, the network connection between nodes is becoming a major bottleneck. Various methods of gradient compression and improved model synchronization have been proposed to address this bottleneck in Parameter-Server-based DDL. However, these two types of methods can result in accuracy loss due to discarded gradients and have limited enhancement on the throughput of model synchronization, respectively. To address these challenges, we propose a new model synchronization method named Overlapped Synchronization Parallel (OSP), which achieves efficient communication with a 2-stage synchronization approach and uses Local-Gradient-based Parameter correction (LGP) to avoid accuracy loss caused by stale parameters. The prototype of OSP has been implemented using PyTorch and evaluated on commonly used deep learning models and datasets with a 9-node testbed. Evaluation results show that OSP can achieve up to 50% improvement in throughput without accuracy loss compared to popular synchronization models.


  • Open Access

    Augmenting Distributed AI Training with Loss-tolerant Transmission

    by Zixuan Chen, Lei Shi, Yongbo Gao, Xuandong Liu, Xin Ai, Sen Liu, Yang Xu

  • Advantages of virtual addressing for efficient and failure tolerant routing in sensor networks

    by Abdalkarim Awad, Lei Shi, Reinhard German, Falko Dressler

    We study the capabilities of virtual addressing schemes for efficient and failure tolerant routing in sensor networks. In particular, we present the virtual cord protocol (VCP) that uses techniques known from peer-to-peer networks, i.e. distributed hash tables (DHTs) are used to associate data items in sensor networks with particular node addresses. The addresses of nodes are dynamically maintained by the protocol to form a virtual cord. VCP uses two mechanisms for finding paths to nodes and associated data items: First, it relies on the virtual cord that always points towards the destination. Furthermore, locally available neighborhood information is exploited for greedy routing. Our simulation results show that VCP is able to find paths close to the possible shortest path with very low overhead. The routing performance of VCP, which clearly outperforms other ad hoc routing protocols such as Dynamic MANET On Demand (DYMO), is similar to other virtual addressing schemes, e.g. Virtual Ring Routing (VRR). However, we improved VCP to handle frequent node failures in an optimized way. The results presented in this paper outline the capabilities of VCP to handle such cases.


  • Static task scheduling based on ethernet virtual connections with varied granularities in ethernet over SDH networks

    by Lei Shi, Weiqiang Sun, Guowu Xie, Yaohui Jin, Wei Guo, Weisheng Hu

    This paper describes the Task Scheduling problem over Ethernet Virtual Connections which provides multiple-granularity and the problem of scheduling tasks over two user network interfaces is considered. Three approximate models, 2SP, 2LSP and BWS are proposed with algorithms respectively. Numerical simulation results and comparisons are also presented.