With the rapid advancement of diffusion models, AI-generated content (AIGC), particularly in image and video generation, has achieved remarkable progress while simultaneously raising significant concerns about content safety within the community. Compared to image generation, the risks of generated videos remain largely overlooked and more challenging, as harmful content may be concealed at any temporal location or accumulate over time. Therefore, this paper proposes a novel Dual-Safety Approach (DSA) to remove temporal risks in generated videos while preserving their inherent quality at test time. The DSA solves the temporal risks from two aspects: the semantics in the text prompts and the temporal dynamics in the diffusion generation process. Specifically, we introduce a Semantic Adaptive Refinement module, which constructs a prompt-specific harmful subspace for targeted risk mitigation while preserving core semantics. Rather than treating video safety at a coarse, whole-video level, we introduce Temporal Dynamic Adaptive Optimization and Temporal Safety Guidance to dynamically constrain unsafe content over time while preserving temporal consistency. Moreover, to evaluate the effectiveness of our approach, we construct a novel Temporal Safety Benchmark for both Text-to-Video and Image-to-Video models. Extensive experiments demonstrate that our DSA can significantly reduce the risks of videos generated by open-source models without degrading video quality and even surpassing commercial models. Additional evaluations on multiple benchmarks, including adversarial attack datasets, further confirm that DSA surpasses existing methods in both effectiveness and robustness. Code and data will be publicly available.