Changlong Li

Researchers use high-level synthesis (HLS) to develop large-scale accelerators, leveraging extensive resources to handle intricate computational tasks. Recent Multi-Die FPGAs feature several super logic regions (SLR) within one device to meet the high resource demands of advanced computation. However, implementing a large-scale accelerator on multi-die FPGAs presents three obstacles. The first is the considerable delay penalty caused by the interconnectivity of accelerator modules across multi-die boundaries. The second is the congestion caused by modules’ excessive use of local resources. The third is that the increasing number of accelerator modules expands the design space, making it difficult to find the optimal design point. All of these pose limitations on the maximum achievable frequency for developing large-scale accelerator designs on multi-die FPGAs. Therefore, this paper proposes a frequency-oriented design exploration with the congestion-aware module mapping algorithm, FrqBooster. It combines modules’ location and resource consumption to provide coarse-grained floorplanning optimization. Firstly, we analyze the connections between accelerator modules and build optimization objectives. Secondly, we analyze the resource cost of each module in accelerator design and propose a two-stage resource-balancing strategy, combined with resource consumption, to alleviate local congestion. Finally, we design a congestion-aware algorithm to complete the overall exploration for achieving higher potential performance. In the experiments, results show that FrqBooster achieves maximum frequency improvements of 73.15% and 81.22% on the U250 and U280 platforms, respectively, compared to the default implementation in Vivado. Compared to related works on accelerator DSE scenarios, FrqBooster achieved a maximum improvement of 25.84×, resulting in better overall performance.

As one of the most frequently occurring operations, I/Os significantly affect the application launching time and frame rate of mobile devices, hence influencing the user experience. However, the response speed of I/O requests is still the bottleneck in practice. This paper shows that high I/O latency is usually due to the congestion inside Flash instead of the system layer. Unfortunately, Flash is treated as a black box device and cannot be modified after delivery. In this paper, we propose a novel service to address this issue without an intra-Flash modification. Specifically, this paper proposes a lightweight I/O throttling framework in mobile systems named FlashDAM. This service throttles the I/O flow to make way for I/Os that may block the foreground application. FlashDAM is the first work that proves that proper I/O throttling positively affects the user experience, contrary to the common belief. Furthermore, this paper proposes FlashDAM$^+$, an enhanced version of FlashDAM. By coordinating I/O throttling and compression, FlashDAM's effect in the system layer is minimized. We have implemented FlashDAM on real mobile devices. Experimental results illustrate that the app launching speed and frame rate are enhanced by 72% and 45% separately compared to the state-of-the-art. When enabling the compression feature of FlashDAM, that is, FlashDAM$^+$, screen jank and application launch latency are further reduced by 9.5% and 11.4%, respectively, under heavy background I/O load.

Mobile devices with limited resources are prevalent, as they have a relatively low price. Providing a good user experience with limited resources has been a big challenge. This work finds that foreground applications are often unexpectedly interfered by background applications’ memory activities. Improving user experience on resource-limited mobile devices calls for a strong collaboration between memory and process management. This article proposes Ice , a framework to optimize the user experience on resource-limited mobile devices. With Ice, processes that will cause frequent refaults in the background are identified and frozen accordingly. The frozen application will be thawed when memory condition allows. Based on the proposed Ice, this work shows that the refault can be further reduced by revisiting the LRU lists in the original kernel with app-freezing awareness (called Ice ⁺ ). Evaluation of resource-limited mobile devices demonstrates that the user experience is effectively improved with Ice. Specifically, Ice boosts the frame rate by 1.57x on average over the state of the art. The frame rate is further enhanced by 5.14% on average with Ice ⁺ .

Federated learning (FL) is an advanced framework that enables collaborative training of machine learning models across edge devices. An effective strategy to enhance training efficiency is to allocate the optimal submodel based on each device’s resource capabilities. However, system heterogeneity significantly increases the difficulty of allocating submodel parameter budgets appropriately for each device, leading to the straggler problem. Meanwhile, data heterogeneity complicates the selection of the optimal submodel structure for specific devices, thereby impacting training performance. Furthermore, the dynamic nature of edge environments, such as fluctuations in network communication and computational resources, exacerbates these challenges, making it even more difficult to precisely extract appropriately sized and structured submodels from the global model. To address the challenges in heterogeneous training environments, we propose an efficient FL framework, namely, HaloFL. The framework dynamically adjusts the structure and parameter budget of submodels during training by evaluating three dimensions: 1) model-wise performance; 2) layer-wise performance; and 3) unit-wise performance. First, we design a data-aware model unit importance evaluation method to determine the optimal submodel structure for different data distributions. Next, using this evaluation method, we analyze the importance of model layers and reallocate parameters from noncritical layers to critical layers within a fixed parameter budget, further optimizing the submodel structure. Finally, we introduce a resource-aware dual-UCB multiarmed bandit agent, which dynamically adjusts the total parameter budget of submodels according to changes in the training environment, allowing the framework to better adapt to the performance differences of heterogeneous devices. Experimental results demonstrate that HaloFL exhibits outstanding efficiency in various dynamic and heterogeneous scenarios, achieving up to a 14.80% improvement in accuracy and a $3.06\times $ speedup compared to existing FL frameworks.

The development of deep learning has led to increasing demands for computation and memory, making multichiplet accelerators a powerful solution. Multichiplet accelerators require more precise consideration of hardware configurations and mapping schemes in terms of computation, memory, and communication patterns compared to monolithic designs, in order to avoid underutilization of performance. However, there is currently a lack of performance testing methods specifically tailored for multichiplet accelerators. Existing testing methods primarily focus on correctness testing and do not address potential performance issues from a hardware perspective. To address these issues, this article proposes Magnifier: a test case generation method for performance testing of multichiplet accelerators. First, we analyze typical multichiplet accelerator prototype from the perspectives of computation, memory, and communication patterns, and summarize a chiplet feature-aware operator task set. Next, we define the test evaluation metric interdevice percentile performance standard deviation and use a candidate operator set to construct a sampling space for model-level test cases. Finally, we build a generative adversarial network to learn the distribution of high-diversity test cases, enabling the rapid generation of high-quality test cases. We validate the proposed method on both simulated and real multichiplet accelerators. Experiments show that Magnifier can improve the metric of test cases by up to 3.42 times and significantly reduce generation time, providing valuable insights for optimizing the hardware and software of multichiplet accelerators.

This article presents LegoSwap, a cross-device memory swapping mechanism for mobile devices. It exploits the unbalanced utilization of memory resources across devices. With LegoSwap, remote memory is utilized in a seamless plug-and-play manner. It achieves comparable-to-local swapping performance based on existing network infrastructure. In addition, LegoSwap frees from the effect of remote I/O disconnection and minimizes the effect on remote devices. This is realized by three novel approaches: resource-dedicated swapping for fast swapping among devices, app-aware swapping for network connectivity considerations, and elastic swap area management for inter-device interference relieving. LegoSwap is implemented on real-life mobile devices. Experimental results show that LegoSwap can enhance app caching capability by 2x compared with no swapping, and improve performance by 2.3x compared with state-of-the-art remote swapping. More importantly, local swapping induced read-write conflicts are largely removed.

The cloud–edge collaborative inference requires efficient scheduling of artificial intelligence (AI) tasks to the appropriate edge intelligence devices. Gls DNN inference latency has become a vital basis for improving scheduling efficiency. However, edge devices exhibit highly heterogeneous due to the differences in hardware architectures, computing power, etc. Meanwhile, the diverse deep neural networks (DNNs) are continuing to iterate over time. The diversity of devices and DNNs introduces high computational costs for measurement methods, while invasive prediction methods face significant development efforts and application limitations. In this article, we propose and develop Ace-Sniper, a scheduling framework with DNN inference latency modeling on heterogeneous devices. First, to address the device heterogeneity, a unified hardware resource modeling (HRM) is designed by considering the platforms as black-box functions that output feature vectors. Second, neural network similarity (NNS) is introduced for feature extraction of diverse and frequently iterated DNNs. Finally, with the results of HRM and NNS as input, the performance characterization network is designed to predict the latencies of the given unseen DNNs on heterogeneous devices, which can be combined into most time-based scheduling algorithms. Experimental results show that the average relative error of DNN inference latency prediction is 11.11%, and the prediction accuracy reaches 93.2%. Compared with the nontime-aware scheduling methods, the average waiting time for tasks is reduced by 82.95%, and the platform throughput is improved by 63% on average.

Mobile systems and applications are becoming increasingly feature-rich and powerful, which constantly suffer from memory pressure, especially for devices equipped with limited DRAM. Swapping inactive DRAM pages to the storage device is a promising solution to extend the physical memory. However, existing mobile devices usually adopt flash memory as the storage device, where swapping DRAM pages to flash memory may introduce significant performance overhead. In this paper, we first conduct an in-depth analysis of the I/O characteristics of the flash-based memory swapping, including the I/O interference and swap I/O randomness in swap subsystem. Then an I/O efficiency optimization framework for memory swapping (IOSR) is proposed to enhance the performance of flash-based memory swapping for mobile devices. IOSR consists of two methods: swap I/O scheduling (SIOS) and swap I/O pattern reshaping (SIOR). SIOS is designed to schedule the swap I/O to reduce interference with other processes I/Os. SIOR is designed to reshape the swap I/O pattern with process-oriented swap slot allocation and adaptive granularity swap read-ahead. IOSR is implemented on Google Pixel 4. Experimental results show that IOSR reduces the application switching time by 31.7% and improves the swap-in bandwidth by 35.5% on average compared to the state-of-the-art.