The use of Deep Reinforcement Learning (DRL) in robotics has gained prominence in the last decade, but the impact of its intrinsic temporal components of DRL when applied to physical systems - generally summarized as the timestep duration -, remains underexplored. While some approaches to this issue include timestep duration as part of action space, they offer a limited analysis, overlooking key elements such as software architecture asynchrony or system-induced latencies. These factors cause deviations between nominal and actual timesteps, negatively impacting learning. This article proposes a set of navigation tasks for mobile robots specifically designed to analyze the effects of timestep duration, supported by over 2000 hours of simulation. To our knowledge, this constitutes the first multimetric evaluation of the influence of step time duration on DRL-based robotic tasks. The findings confirm the existence of optimal step times that maximize learning efficiency and task completion.