Kijung Jung

Apache Spark stands out as a well-known solution for big data processing because of its efficiency and rapid processing capabilities. One of its modules, Spark SQL, serves as a prominent big data query engine. However, executing Spark SQL applications with massive data can be time-intensive, and the execution time can vary significantly depending on its configurations. Recent studies try to reduce application execution times by searching optimal configurations for applications. While Bayesian optimization is recognized as a powerful method in recent studies for configuration optimization, it faces challenges such as computational costs and time-consuming computations, especially when dealing with large search spaces Due to these challenges, we propose QHB+, designed to rapidly search optimal configurations. QHB+ utilizes the Successive Halving Algorithm-based optimization methods, performing well in hyperparameter optimization of machine learning models, for configuration optimization of Spark SQL applications. Through empirical evaluations against established benchmarks, we show the efficiency of QHB+, highlighting them as swift alternatives to conventional optimization method for optimizing Spark SQL configurations.

Federated learning is a promising approach for training machine learning models using distributed data from multiple mobile devices. However, privacy concerns arise when sensitive data are used for training. In this paper, we discuss the challenges of applying local differential privacy to federated learning, which is compounded by limited resources of mobile clients and the asynchronicity of federated learning. To address these challenges, we propose a framework called LAFD, that stands for Local-differentially Private and Asynchronous Federated Learning with Direct Feedback Alignment. LAFD consists of two parts: (a) LFL-DFALS: Local differentially private Federated Learning with Direct Feedback Alignment and Layer Sampling, and (b) AFL-LMTGR: Asynchronous Federated Learning with Local Model Training and Gradient Rebalancing. LFL-DFALS effectively reduces the computation and communication costs via direct feedback alignment and layer sampling. AFL-LMTGR handles the problem of stragglers via local model training, and gradient rebalancing. We demonstrate the performance of LFL-DFALS and AFL-LMTGR through experiments.