Conghui Zhang

Summary: Hadoop is a typical framework for processing big data. Task scheduling algorithms have a significant impact on the processing performance of Hadoop clusters. Existing scheduling algorithms of Hadoop fail to consider the performance differences between nodes in heterogeneous Hadoop clusters, causing problems such as uneven task allocation and low resource utilization. Aiming to solve this problem, we propose a spider monkey optimization‐based scheduling algorithm (SMOSA) for heterogeneous Hadoop. First, the cluster heartbeat mechanism is used to obtain information such as memories and CPUs of nodes to comprehensively consider the actual load capacity of each node. Then, the spider monkey optimization algorithm is adopted to find the optimal mapping relationship between tasks and resources by taking the task completion time as the objective function and updating the position of the spider monkey. Finally, we calculate the remaining rate of node hardware resources, and according to the task type, the node with the higher remaining rate of resource will give priority to the task. Data are compressed for I/O type tasks to reduce disk operations and increase the speed of task execution. Experimental results show that, compared with existing scheduling algorithms, the SMOSA can effectively reduce task execution time and can significantly improve scheduling efficiency and task execution speed especially in heterogeneous Hadoop clusters. For different types of tasks, the execution time can be reduced by up to 19%.

The Controller Placement Problem (CPP) is a key technical challenge in a large-scale Software Defined Network (SDN). Low-complexity heuristic algorithm is widely used for solving the CPP. However, parameter settings of the heuristic algorithm greatly affect the result of the CPP. Therefore, we establish a Parameter Optimization Model (POM) for the heuristic algorithm applied to the CPP. The heuristic algorithm can effectively solve the CPP by using the optimized parameters obtained in POM. To verify the effectiveness of the POM, we first establish a synthetical-delay controller placement model to reduce the delay between the controllers and the switches and the delay between the controllers. Further, we select the bat algorithm, the firefly algorithm, and the varna-based optimization respectively to solve the model, and use the particle swarm optimization method to optimize the parameters of the three algorithms. Experimental results on real topologies show that compared with original algorithms and other similar algorithms, the algorithms with optimized parameters perform better.