Ozlem Imik Simsek

Architectures of neural networks affect the training performance of artificial neural networks. For more consistent performance evaluation of training algorithms, hard-to-train benchmarking architectures should be used. This study introduces a benchmark neural network architecture, which is called pipe-like architecture, and presents training performance analyses for popular Neural Network Backpropagation Algorithms (NNBA) and well-known Metaheuristic Search Algorithms (MSA). The pipe-like neural architectures essentially resemble an elongated fraction of a deep neural network and form a narrowed long bottleneck for the learning process. Therefore, they can significantly complicate the training process by causing the gradient vanishing problems and large training delays in backward propagation of parameter updates throughout the elongated pipe-like network. The training difficulties of pipe-like architectures are theoretically demonstrated in this study by considering the upper bound of weight updates according to an aggregated one-neuron learning channels conjecture. These analyses also contribute to Baldi et al.'s learning channel theorem of neural networks in a practical aspect. The training experiments for popular NNBA and MSA algorithms were conducted on the pipe-like benchmark architecture by using a biological dataset. Moreover, a Normalized Overall Performance Scoring (NOPS) was performed for the criterion-based assessment of overall performance of training algorithms.

The low-cost and small size solid-state sensor arrays are suitable to implement a wide-area electronic nose (e-nose) for real-time air quality monitoring. However, accuracy of these low-cost sensors is not adequate for precise measurements of pollutant concentrations. Artificial neural network (ANN) estimation models are used for the soft calibration of low-cost sensor array measurements and significantly improve the accuracy of low-cost multi-sensor measurements. However, optimality of neural architecture affects the performance of ANN estimation models, and optimization of the ANN architecture for a training data set is essential to improve data-driven modeling performance of ANNs to reach optimal neural complexity and improved generalization. In this study, an optimal architecture ANN estimator design scheme is suggested to improve the estimation performance of ANN models for e-nose applications. To this end, a gray wolf optimization (GWO) algorithm is modified, and an exploitative alpha gray wolf optimization (EA-GWO) algorithm is suggested. This modification enhances local exploitation skill of the best alpha gray wolf search agent, and thus allows the fine-tuning of ANN architectures by minimizing a multi-objective cost function that implements mean error search policy. Experimental study demonstrates the effectiveness of optimal architecture ANN models to estimate CO concentration from the low-cost multi-sensor data.

With the increase of e-commerce volumes in recent years, it is useful to estimate daily demand order numbers in order to improve the demand forecasts, production-distribution planning and sales services. In this manner, data-driven modeling and machine learning tools have been preferred to enhance demand order predictions, timely delivery, incomes and customer satisfaction in electronic trading because real-time data collection is possible in e-commerce platforms. Artificial Neural Networks (ANNs) are widely used for data-driven modeling and prediction problems. Since affecting the approximation performance of neural network function, the modeling performance of ANNs strongly depends on the architecture of neural networks, and architectural optimization of ANN models has become a main topic in the neuroevolution field. This study presents an architecture optimization method that implements Particle Swarm Optimization (PSO) and Differential Evolution (DE) algorithms to optimize ANN model architecture for the estimation of total demand order numbers from the sparse demand order data. In this approach, PSO and DE algorithm only optimizes neural model architecture according to an effective network search policy and the training of ANN models is carried out by using backpropagation algorithm. This neural architecture model optimization approach considers generalization of data, reducing neuron and training epoch numbers and it can yield an optimal architecture data-driven neural model for estimation of the daily total orders. In the experimental study, optimal architecture ANN models are obtained according to the daily order forecasting dataset.

Neuroevolutionary machine learning is an emerging topic in the evolutionary computation field and enables practical modeling solutions for data-driven engineering applications. Contributions of this study to the neuroevolutionary machine learning area are twofold: firstly, this study presents an evolutionary field theorem of search agents and suggests an algorithm for Evolutionary Field Optimization with Geometric Strategies (EFO-GS) on the basis of the evolutionary field theorem. The proposed EFO-GS algorithm benefits from a field-adapted differential crossover mechanism, a field-aware metamutation process to improve the evolutionary search quality. Secondly, the multiplicative neuron model is modified to develop Power-Weighted Multiplicative (PWM) neural models. The modified PWM neuron model involves the power-weighted multiplicative units similar to dendritic branches of biological neurons, and this neuron model can better represent polynomial nonlinearity and they can operate in the real-valued neuron mode, complex-valued neuron mode, and the mixed-mode. In this study, the EFO-GS algorithm is used for the training of the PWM neuron models to perform an efficient neuroevolutionary computation. Authors implement the proposed PWM neural processing with the EFO-GS in an electronic nose application to accurately estimate Nitrogen Oxides (NO_x) pollutant concentrations from low-cost multi-sensor array measurements and demonstrate improvements in estimation performance.

In recent years, with the increase in volume of buying orders, demand forecast based on the order data is important for improvement of production, distribution and selling services. For this reason, the predictability of orders will increase efficiency in many areas by timely delivering orders, increasing earnings, and customer satisfaction in trading. This article aims to estimate total amount of daily orders by using an optimal structured artificial neural network learning model. To optimize rectangular architecture of artificial neural network model, a metaheuristic optimization, which determines the number of hidden layers and number of neurons, is used. In the study, training of neural networks was carried out with the Levenberg-Marquardt backpropagation algorithm for daily orders collected for 60 days. During this training, the network's layer and neuron number were optimized with a gray wolf optimization algorithm. Results indicate that optimal architecture neural network can better estimate total daily demand orders.