Vehicular ad hoc networks (VANETs) are one of the most essential parts of intelligent transportation system (ITS). VANETs fulfill a crucial role in continuous traffic monitoring, emergency message transmission, and in-vehicle infotainment services. Given the short communication distance, constraints of unpredictable traffic environments, and the dynamic topology caused by high mobility, effective VANET routing is vital for network performance. However, prior ad hoc routing schemes in the literature are unsuitable for VANETs dynamic environments. For this reason, recent works focusing on VANETs have proposed reinforcement learning (RL)-based approaches. In this paper, we survey the literature that tackles the VANET routing problem using RL, summarizing which RL algorithms are used and their optimization goals. In addition, we analyze and discuss the limitations of RL-based approaches to propose guidelines for promising VANET routing solutions for constructing the future of ITS.