Ziliang Zong

The significance of accurate power measurement in cloud computing can be attributed to its increasing usability as well as for attaining energy savings through optimization and advancing green computing in diverse cloud platforms like AWS, GCP, Chameleon Cloud, and Microsoft Azure. The challenge of providing a single yet accurate methodology, suitable for the cloud environment, is exacerbated by factors like numerous heterogeneity of the cloud, unavailability of administrative access required by precise methods like Intel's Running Average Power Limit (RAPL) and the inaccuracy of widely-accepted methods like Thermal Design Power (TDP) based estimation. In this paper, we propose Cloud Power Meter (CPM) in pursuit of sustainability in cloud operations, which integrates a decision tree with multi-variable polynomial regression. Being an ML-based method that is trained using diverse cloud instances from Amazon, Microsoft, and Google, CPM is capable of providing precise power estimation on the cloud platforms without administrative privileges. The decision tree guarantees performance capacity across heterogeneous cloud environments by locating the “CPM Instance” - the closest match of instances that are not familiar to CPM. In evaluation through the Standard Performance Evaluation Corporation (SPEC) benchmark's CPU power consumption, CPM outperforms 14 existing power models and its power estimation matches Intel's RAPL power model. To evaluate the effectiveness of CPM, we conduct an extensive analysis of approximately 2.6 million real-world Virtual Machines (VMs) running on Microsoft Azure. CPM successfully identifies opportunities to save over 80,000 kilowatt-hours (kWh) of energy in a single month through optimized resource allocation. Notably, about 25,000 kWh of this energy is being wasted by more than 17% of the VMs. Further, CPM's energy estimations reveal that 366 VMs (out of roughly 2.6 million) wasted nearly 15,000 kWh of energy despite never exceeding 2% CPU utilization.

The advent of large language models like ChatGPT and GPT-4 has undoubtedly revolutionized the AI landscape. However, training large language models is a sophisticated process, which demands substantial computational resources and leads to concerns regarding the environmental impact. This paper aims to reveal the energy usage and carbon emissions incurred in training a sizable Open Pretrained Transformers (OPT) model using the cutting-edge H100 GPU and Microsoft’s DeepSpeed-Chat training framework. The experimental results demonstrate that (1) the batch size has a large impact on accuracy and carbon emission, and (2) the carbon footprint varies greatly in each training step.

Relying on the premise that the performance of a binary neural network can be largely restored with eliminated quantization error between full-precision weight vectors and their corresponding binary vectors, existing works of network binarization frequently adopt the idea of model robustness to reach the aforementioned objective. However, robustness remains to be an ill-defined concept without solid theoretical support. In this work, we introduce the Lipschitz continuity, a well-defined functional property, as the rigorous criteria to define the model robustness for BNN. We then propose to retain the Lipschitz continuity as a regularization term to improve the model robustness. Particularly, while the popular Lipschitz-involved regularization methods often collapse in BNN due to its extreme sparsity, we design the Retention Matrices to approximate spectral norms of the targeted weight matrices, which can be deployed as the approximation for the Lipschitz constant of BNNs without the exact Lipschitz constant computation (NP-hard). Our experiments prove that our BNN-specific regularization method can effectively strengthen the robustness of BNN (testified on ImageNet-C), achieving state-of-the-art performance on CIFAR and ImageNet.

Neural network binarization accelerates deep models by quantizing their weights and activations into 1-bit. However, there is still a huge performance gap between Binary Neural Networks (BNNs) and their full-precision (FP) counterparts. As the quantization error caused by weights binarization has been reduced in earlier works, the activations binarization becomes the major obstacle for further improvement of the accuracy. BNN characterises a unique and interesting structure, where the binary and latent FP activations exist in the same forward pass (i.e., $\text{Binarize}(\mathbf{a}_F) = \mathbf{a}_B$). To mitigate the information degradation caused by the binarization operation from FP to binary activations, we establish a novel contrastive learning framework while training BNNs through the lens of Mutual Information (MI) maximization. MI is introduced as the metric to measure the information shared between binary and FP activations, which assists binarization with contrastive learning. Specifically, the representation ability of the BNNs is greatly strengthened via pulling the positive pairs with binary and FP activations from the same input samples, as well as pushing negative pairs from different samples (the number of negative pairs can be exponentially large). This benefits the downstream tasks, not only classification but also segmentation and depth estimation, etc. The experimental results show that our method can be implemented as a pile-up module on existing state-of-the-art binarization methods and can remarkably improve the performance over them on CIFAR-10/100 and ImageNet, in addition to the great generalization ability on NYUD-v2.