The growing importance of short-term electricity trading over independent subsequent markets in Europe presents market participants with intricate decision challenges. Established solutions based on stochastic programs are often used but suffer from shortcomings such as the curse of dimensionality in multi-stage decision processes. Reinforcement learning is a promising alternative. However, best practices for the comparison of the two approaches and the ex-post evaluation of reinforcement learning are not yet established. In this paper, we offer a comparison of stochastic programs and reinforcement learning and propose measures for a comparative performance evaluation between the two approaches. We demonstrate them on an empirical case study over subsequent market stages of the German market zone within the coupled European power market.