OmniZoomer: Learning to Move and Zoom in on Sphere at High-Resolution
Preprint
Abstract
Omnidirectional images (ODIs) have become increasingly popular, as their large field-of-view (FoV) can offer viewers the chance to freely choose the view directions in immersive environments such as virtual reality. The Möbius transformation is typically employed to further provide the opportunity for movement and zoom on ODIs, but applying it to the image level often results in blurry effect and aliasing problem. In this paper, we propose a novel deep learning-based approach, called \textbf{OmniZoomer}, to incorporate the Möbius transformation into the network for movement and zoom on ODIs. By learning various transformed feature maps under different conditions, the network is enhanced to handle the increasing edge curvatures, which alleviates the blurry effect. Moreover, to address the aliasing problem, we propose two key components. Firstly, to compensate for the lack of pixels for describing curves, we enhance the feature maps in the high-resolution (HR) space and calculate the transformed index map with a spatial index generation module. Secondly, considering that ODIs are inherently represented in the spherical space, we propose a spherical resampling module that combines the index map and HR feature maps to transform the feature maps for better spherical correlation. The transformed feature maps are decoded to output a zoomed ODI. Experiments show that our method can produce HR and high-quality ODIs with the flexibility to move and zoom in to the object of interest. Project page is available at http://vlislab22.github.io/OmniZoomer/.Keywords
All Related Versions
This publication has 8 references indexed in Scilit:
- VRT: A Video Restoration TransformerIEEE Transactions on Image Processing, 2024
- View-Aware Salient Object Detection for $360^{\circ }$ Omnidirectional ImageIEEE Transactions on Multimedia, 2022
- Omnidirectional Image Super-Resolution via Latitude Adaptive NetworkIEEE Transactions on Multimedia, 2022
- Applying inverse stereographic projection to manifold learning and clusteringApplied Intelligence, 2021
- Data augmentation with Mobius transformationsMachine Learning: Science and Technology, 2021
- Local Moebius transformations applied to omnidirectional imagesComputers & Graphics, 2017
- Weighted-to-Spherically-Uniform Quality Evaluation for Omnidirectional VideoIEEE Signal Processing Letters, 2017
- Joint Registration and Super-Resolution With Omnidirectional ImagesIEEE Transactions on Image Processing, 2011