Skyeyes: Ground Roaming using Aerial View Images
Preprint
Abstract
Integrating aerial imagery-based scene generation into applications like autonomous driving and gaming enhances realism in 3D environments, but challenges remain in creating detailed content for occluded areas and ensuring real-time, consistent rendering. In this paper, we introduce Skyeyes, a novel framework that can generate photorealistic sequences of ground view images using only aerial view inputs, thereby creating a ground roaming experience. More specifically, we combine a 3D representation with a view consistent generation model, which ensures coherence between generated images. This method allows for the creation of geometrically consistent ground view images, even with large view gaps. The images maintain improved spatial-temporal coherence and realism, enhancing scene comprehension and visualization from aerial perspectives. To the best of our knowledge, there are no publicly available datasets that contain pairwise geo-aligned aerial and ground view imagery. Therefore, we build a large, synthetic, and geo-aligned dataset using Unreal Engine. Both qualitative and quantitative analyses on this synthetic dataset display superior results compared to other leading synthesis approaches. See the project page for more results: https://chaoren2357.github.io/website-skyeyes/.Keywords
All Related Versions
This publication has 9 references indexed in Scilit:
- Geospecific View Generation Geometry-Context Aware High-Resolution Ground View Inference from Satellite Views2024
- SuGaR: Surface-Aligned Gaussian Splatting for Efficient 3D Mesh Reconstruction and High-Quality Mesh Rendering2024
- Adding Conditional Control to Text-to-Image Diffusion Models2023
- 3D Gaussian Splatting for Real-Time Radiance Field RenderingACM Transactions on Graphics, 2023
- Grid-guided Neural Radiance Fields for Large Urban Scenes2023
- SUDS: Scalable Urban Dynamic Scenes2023
- Instant neural graphics primitives with a multiresolution hash encodingACM Transactions on Graphics, 2022
- Conditional GAN with Discriminative Filter Generation for Text-to-Video Synthesis2019
- Image quality assessment: from error visibility to structural similarityIEEE Transactions on Image Processing, 2004