Yaniv Ben-Itzhak

Quantization reduces memory, computation, and communication costs by converting high-precision values into compact representations. Recent adaptive stochastic quantization (ASQ) methods optimize quantization values for a specific input vector, but they do not exploit shared randomness available to both the quantizer and the dequantizer. We introduce adaptive unbiased quantization (AUQ), which extends ASQ with shared randomness and can asymptotically improve achievable accuracy without increasing the encoded size. We then present Simba, a practical AUQ method that uses multiple sets of quantization values and selects among them using shared randomness while preserving unbiasedness. Simba starts by computing a fast approximate ASQ solution and refines it through a structured shared-randomness search. Across common synthetic distributions, Simba is up to about 24× more accurate than the state-of-the-art optimal ASQ method and can be up to two orders of magnitude faster. We also show that Simba can improve QUIC-FL by replacing its solver-generated lookup tables with tables computed by Simba.

Quantization is used to convert higher-precision data into a lower-precision representation, providing a fundamental technique to reduce memory usage, computational complexity, and communication costs in machine learning tasks. Recently, Adaptive Stochastic Quantization (ASQ) methods, which provide unbiased optimized quantized values for a specific input (i.e., when the input distribution is not known to the dequantizer), have been shown to be efficient for compressing large tensors on the fly. In this paper, we introduce the Adaptive Unbiased Quantization (AUQ) problem that generalizes ASQ to a setting where shared randomness known to both the quantizer and the dequantizer is allowed. We prove an asymptotic gap between the achievable accuracy of the problems, showing that AUQ can be quadratically more precise. We then introduce Simba, an AUQ technique that improves the accuracy of optimal ASQ methods. Intuitively, Simba uses multiple sets of quantization values, and the set used to quantize a specific value is determined using the shared randomness. Interestingly, we show that Simba improves both the speed and accuracy of optimal ASQ methods by leveraging an approximate ASQ solution as a starting point from which Simba improves. We measure Simba to be up to ≈ 24× more accurate for popular distributions than the state-of-the-art optimal ASQ method, QUIVER, while also being up to two orders of magnitude faster. Finally, we demonstrate that Simba can be used to improve the accuracy of QUIC-FL, a state-of-the-art Distributed Mean Estimation technique, by replacing its solver-based lookup tables with tables computed by Simba.

The soaring use of machine learning leads to increasing processing demands. As data volume keeps growing, providing classification services with good machine learning performance, high throughput, low latency, and minimal equipment overheads becomes a challenge. Offloading machine learning tasks to network switches can be a scalable solution to this problem, providing high throughput and low latency. However, network devices are resource constrained, and lack support for machine learning functionality. In this paper, we introduce IIsy -a novel mapping tool of machine learning classification models to off-the-shelf switches. Using an efficient encoding algorithm, enables fitting a range of classification models on switches, co-existing with standard switch functionality. To overcome resource constraints, adopts a hybrid approach for ensemble models, running a small model on a switch and a large model on the backend. The evaluation shows that achieves near-optimal classification results, within minimum resource overheads, and while reducing the load on the backend by 70% for data-intensive use cases.

Machine learning is widely used to solve networking challenges, ranging from traffic classification and anomaly detection to network configuration. However, machine learning also requires significant processing and often increases the load on both networks and servers. The introduction of in-network computing, enabled by programmable network devices, has allowed to run applications within the network, providing higher throughput and lower latency. Soon after, in-network machine learning solutions started to emerge, enabling machine learning functionality within the network itself. This survey introduces the concept of in-network machine learning and provides a comprehensive taxonomy. The survey provides an introduction to the technology and explains the different types of machine learning solutions built upon programmable network devices. It explores the different types of machine learning models implemented within the network, and discusses related challenges and solutions. In-network machine learning can significantly benefit cloud computing and next-generation networks, and this survey concludes with a discussion of future trends.

The rat race between user-generated data and data-processing systems is currently won by data. The increased use of machine learning leads to further increase in processing requirements, while data volume keeps growing. To win the race, machine learning needs to be applied to the data as it goes through the network. In-network classification of data can reduce the load on servers, reduce response time and increase scalability. In this paper, we introduce IIsy, implementing machine learning classification models in a hybrid fashion using off-the-shelf network devices. IIsy targets three main challenges of in-network classification: (i) mapping classification models to network devices (ii) extracting the required features and (iii) addressing resource and functionality constraints. IIsy supports a range of traditional and ensemble machine learning models, scaling independently of the number of stages in a switch pipeline. Moreover, we demonstrate the use of IIsy for hybrid classification, where a small model is implemented on a switch and a large model at the backend, achieving near optimal classification results, while significantly reducing latency and load on the servers.

Public clouds are one of the most thriving technologies of the past decade. Major applications over public clouds require world-wide distribution and large amounts of data exchange between their distributed servers. To that end, major cloud providers have invested tens of billions of dollars in building world-wide inter-region networking infrastructure that can support high performance communication into, out of, and across public cloud geographic regions. In this paper, we lay the foundation for a comprehensive study and real time monitoring of various characteristic of networking within and between public clouds. We start by presenting CloudCast, a world-wide and expandable measurements and analysis system, currently (January 2019)collecting data from three major public clouds (AWS, GCPand Azure), 59 regions, 1184 intra-cloud and 2238 cross-cloud links (each link represents a direct connection between a pair of regions), amounting to a total of 3422 continuously monitored links and providing active measurements every minute.CloudCast is composed of measurement agents automatically installed in each public cloud region, centralized control, measurement data base, analysis engine and visualization tools. Then we turn to analyze the latency measurement data collected over almost a year . Our analysis yields surprising results. First, each public cloud exhibits a unique set of link latency behaviors along time. Second, using a novel, fair evaluation methodology, termed similar links, we compare the three clouds. Third, we prove that more than 50% of all links do not provide the optimal RTT through the methodology of triangles. Triangles also provide a framework to get around bottlenecks, benefiting not only the majority (53%-70%) of the cross-cloud links by 30% to 70%, but also a significant portion (29%-45%) of intra-cloud links by 14%-33%.