Over the past decade, volatility modeling has become increasingly important in quantitative finance, significantly influencing risk management, investment decisions and policy making. Traditionally, academic research has focused on linear models for forecasting realized volatility. However, advances in computational power have opened the way for more sophisticated methods, including machine learning (ML) and deep learning (DL) models. However, currently, no survey compares traditional linear approaches with these newer ML and DL models in the context of realized volatility forecasting. This survey addresses this gap by analyzing all models used to forecast realized volatility in academic literature from 2000 to H1 2024. It highlights key academic articles and examines the empirical characteristics of realized volatility. Our findings show that the best performing model-a combination of CNN and LSTM networks-outperforms both linear and other ML models in forecasting accuracy. This highlights that DL models can capture a wider range of factors driving realized volatility, highlighting their valuable contribution to quantitative modeling.