From a Visual Scene to a Virtual Representation: A Cross-Domain Review
Open Access
- 7 June 2023
- journal article
- review article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 11, 57916-57933
Abstract
The widespread use of smartphones and other low-cost equipment as recording devices, the massive growth in bandwidth, and the ever-growing demand for new applications with enhanced capabilities, made visual data a must in several scenarios, including surveillance, sports, retail, entertainment, and intelligent vehicles. Despite significant advances in analyzing and extracting data from images and video, there is a lack of solutions able to analyze and semantically describe the information in the visual scene so that it can be efficiently used and repurposed. Scientific contributions have focused on individual aspects or addressing specific problems and application areas, and no cross-domain solution is available to implement a complete system that enables information passing between cross-cutting algorithms. This paper analyses the problem from an end-to-end perspective, i.e., from the visual scene analysis to the representation of information in a virtual environment, including how the extracted data can be described and stored. A simple processing pipeline is introduced to set up a structure for discussing challenges and opportunities in different steps of the entire process, allowing to identify current gaps in the literature. The work reviews various technologies specifically from the perspective of their applicability to an end-to-end pipeline for scene analysis and synthesis, along with an extensive analysis of datasets for relevant tasks.Keywords
Funding Information
- European Union’s Horizon Europe research and innovation program (101094831)
- Fundação para a Ciência e a Tecnologia (SFRH/BD/14600/2019)
This publication has 180 references indexed in Scilit:
- ImageNet Large Scale Visual Recognition ChallengeInternational Journal of Computer Vision, 2015
- Large-Scale Video Classification with Convolutional Neural Networks2014
- DeepPose: Human Pose Estimation via Deep Neural Networks2014
- Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural EnvironmentsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2013
- Towards Understanding Action Recognition2013
- Analysis of object description methods in a video object tracking environmentMachine Vision and Applications, 2013
- Learning discriminative spatial representation for image classification2011
- A semantic management model to enable the integrated management of media and devicesMultimedia Tools and Applications, 2009
- Rapid object detection using a boosted cascade of simple features2005
- Histograms of Oriented Gradients for Human Detection2005