Dan Lo

Quantum-based Machine Learning (QML) combines quantum computing (QC) with machine learning (ML), which can be applied in various sectors, and there is a high demand for QML professionals. However, QML is not yet in many schools' curricula. We design labware for the basic concepts of QC, ML, and QML and their applications in science and engineering fields in Google Colab, applying a three-stage learning strategy for efficient and effective student learning.

Residue Number Systems (RNS) demonstrate the fascinating potential to serve integer addition/ multiplication-intensive applications. The complexity of Artificial Intelligence (AI) models has grown enormously in recent years. From a computer system’s perspective, ensuring the training of these large-scale AI models within an adequate time and energy consumption has become a big concern. Matrix multiplication is a dominant subroutine in many prevailing AI models, with an addition/multiplication-intensive attribute. However, the data type of matrix multiplication within machine learning training typically requires real numbers, which indicates that RNS benefits for integer applications cannot be directly gained by AI training. The state-of-the-art RNS real-number encodings, including floating-point and fixed-point, have defects and can be further enhanced. To transform default RNS benefits to the efficiency of large-scale AI training, we propose a low-cost and high-accuracy RNS fixed-point representation: Single RNS Logical Partition (S-RNS-Logic-P) representation with Scaling-down Postprocessing Multiplication (SD-Post-Mul) . Moreover, we extend the implementation details of the other two RNS fixed-point methods: Double RNS Concatenation and S-RNS-Logic-P representation with Scaling-down Preprocessing Multiplication . We also design the architectures of these three fixed-point multipliers. In empirical experiments, our S-RNS-Logic-P representation with SD-Post-Mul method achieves less latency and energy overhead while maintaining good accuracy. Furthermore, this method can easily extend to the Redundant Residue Number System to raise the efficiency of error-tolerant domains, such as improving the error correction efficiency of quantum computing.

While existing work shows a great success, the accuracy, the response time, the interoperability, the extendibility, etc. remain to be improved. We proposed "ParlAIDialogTeacher" using the open-source ParlAI to improve the accuracy, the response time, the interoperability, and the extendibility. Training is based on Maximum Likelihood Estimation (MLE) approach for the generative model. This platform eases sharing, training, and evaluating dialog models with multiple datasets available. Our results indicate the proposed model outperforms x, y, z in terms of response time and accuracy. Future work include extending the model with other genitive models such as GAN.

The success of ChatGPT is reshaping the landscape of the entire IT industry. The large language model (LLM) powering ChatGPT is experiencing rapid development, marked by enhanced features, improved accuracy, and reduced latency. Due to the execution overhead of LLMs, prevailing commercial LLM products typically manage user queries on remote servers. However, the escalating volume of user queries and the growing complexity of LLMs have led to servers becoming bottlenecks, compromising the quality of service (QoS). To address this challenge, a potential solution is to shift LLM inference services to edge devices, a strategy currently being explored by industry leaders such as Apple, Google, Qualcomm, Samsung, and others. Beyond alleviating the computational strain on servers and enhancing system scalability, deploying LLMs at the edge offers additional advantages. These include real-time responses even in the absence of network connectivity and improved privacy protection for customized or personal LLMs. This article delves into the challenges and potential bottlenecks currently hindering the effective deployment of LLMs on edge devices. Through deploying the LLaMa-2 7B model with INT4 quantization on diverse edge devices and systematically analyzing experimental results, we identify insufficient memory and/or computing resources on traditional edge devices as the primary obstacles. Based on our observation and empirical analysis, we further provide insights and design guidance for the next generation of edge devices and systems from both hardware and software directions.

Training large-scale machine learning models typically requires support from High-performance Computers (HPCs) or cloud servers. One strategy to reduce the computational burden and alleviate the communication bottleneck of servers or HPCs is to move a subset of training computation to the edge or end-user devices. Federated learning is a distributed training approach that aims to protect the privacy of end-user data. However, most federated learning systems utilize centralized server architecture, which slows down the model training and limits the system's scalability. Many machine learning trainings require high performance while maintaining end-user data security. The state-of-the-art serverless approaches have performance limitations and can be further improved. This paper proposes Serverless-DFS, a serverless federated learning with a dynamic forest strategy, to improve the performance and scalability of federated learning while keeping its privacy protection capability. From the experimental results, our Serverless-DFS approach is observed to have a 63.9X speedup compared to the default server-client method in a large-scale system with 256 clients.