Comparative study of optimization techniques in deep learning: Application in the ophthalmology field
Open Access
- 1 January 2021
- conference paper
- Published by IOP Publishing in Journal of Physics: Conference Series
- Vol. 1743 (1) , 012002
Abstract
The optimization is a discipline which is part of mathematics and which aims to model, analyse and solve analytically or numerically problems of minimization or maximization of a function on a specific dataset. Several optimization algorithms are used in systems based on deep learning (DL) such as gradient descent (GD) algorithm. Considering the importance and the efficiency of the GD algorithm, several research works made it possible to improve it and to produce several other variants which also knew great success in DL. This paper presents a comparative study of stochastic, momentum, Nesterov, AdaGrad, RMSProp, AdaDelta, Adam, AdaMax and Nadam gradient descent algorithms based on the speed of convergence of these different algorithms, as well as the mean absolute error of each algorithm in the generation of an optimization solution. The obtained results show that AdaGrad algorithm represents the best performances than the other algorithms with a mean absolute error (MAE) of 0.3858 in 53 iterations and AdaDelta one represents the lowest performances with a MAE of 0.6035 in 6000 iterations. The case study treated in this work is based on an extract of data from the keratoconus dataset of Harvard Dataverse and the results are obtained using Python.Keywords
This publication has 6 references indexed in Scilit:
- Keratoconus detection using deep learning of colour-coded maps with anterior segment optical coherence tomography: a diagnostic accuracy studyBMJ Open, 2019
- KeratoDetect: Keratoconus Detection Algorithm Using Convolutional Neural NetworksComputational Intelligence and Neuroscience, 2019
- Glaucoma diagnosis based on both hidden features and domain knowledge through deep learning modelsKnowledge-Based Systems, 2018
- Comparative analysis of stochastic optimization algorithms for image registration2018
- Automated Identification of Diabetic Retinopathy Using Deep LearningOphthalmology, 2017
- On the momentum term in gradient descent learning algorithmsNeural Networks, 1999