The earth observation satellite scheduling has always been critical for the maximum use of limited satellite resources, which basically includes scheduling ground target observation and observation data downloading. Due to the complexity of the mathematical model and its NP-hardness, current research usually splits it into two problems to simplify the model. In this paper, we propose a model of the integrated scheduling problem of satellite observation and data downloading (ISP-SODD) based on the decentralized partially observable Markov decision process (Dec-POMDP), treating each satellite as an agent with autonomous decision-making capability. A multi-agent deep reinforcement learning-based (MADRL-based) algorithm is proposed to solve it. A series of simulation experiments of different scales validate the effectiveness of the method and its advantages compared with the traditional heuristic algorithms. To the best of our knowledge, this study is the first attempt to apply multi-agent reinforcement learning to a multi-satellite scheduling scenario with downloading considerations.