Street-View Image Generation from a Bird's-Eye View Layout
Preprint
Abstract
Bird's-Eye View (BEV) Perception has received increasing attention in recent years as it provides a concise and unified spatial representation across views and benefits a diverse set of downstream driving applications. While the focus has been placed on discriminative tasks such as BEV segmentation, the dual generative task of creating street-view images from a BEV layout has rarely been explored. The ability to generate realistic street-view images that align with a given HD map and traffic layout is critical for visualizing complex traffic scenarios and developing robust perception models for autonomous driving. In this paper, we propose BEVGen, a conditional generative model that synthesizes a set of realistic and spatially consistent surrounding images that match the BEV layout of a traffic scenario. BEVGen incorporates a novel cross-view transformation and spatial attention design which learn the relationship between cameras and map views to ensure their consistency. Our model can accurately render road and lane lines, as well as generate traffic scenes under different weather conditions and times of day. The code will be made publicly available.Keywords
All Related Versions
This publication has 12 references indexed in Scilit:
- ViT-BEVSeg: A Hierarchical Transformer Network for Monocular Birds-Eye-View Segmentation2022
- Look Outside the Room: Synthesizing A Consistent Long-Term 3D Scene Video from A Single Image2022
- On Aliased Resizing and Surprising Subtleties in GAN Evaluation2022
- Diverse Plausible 360-Degree Image Outpainting for Efficient 3DCG Background Creation2022
- Bird’s-Eye-View Panoptic Segmentation Using Monocular Frontal View ImagesIEEE Robotics and Automation Letters, 2022
- Geometry-Guided Street-View Panorama Synthesis From Satellite ImageryIEEE Transactions on Pattern Analysis and Machine Intelligence, 2022
- FIERY: Future Instance Prediction in Bird’s-Eye View from Surround Monocular Cameras2021
- PixelSynth: Generating a 3D-Consistent Experience from a Single Image2021
- Image Outpainting: Hallucinating Beyond the ImageIEEE Access, 2020
- Semantic Image Manipulation Using Scene Graphs2020