SceneGrok
Authors:
Open Access
- 19 November 2014
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Graphics
- Vol. 33 (6) , 1-10
Abstract
With modern computer graphics, we can generate enormous amounts of 3D scene data. It is now possible to capture high-quality 3D representations of large real-world environments. Large shape and scene databases, such as the Trimble 3D Warehouse, are publicly accessible and constantly growing. Unfortunately, while a great amount of 3D content exists, most of it is detached from the semantics and functionality of the objects it represents. In this paper, we present a method to establish a correlation between the geometry and the functionality of 3D environments. Using RGB-D sensors, we capture dense 3D reconstructions of real-world scenes, and observe and track people as they interact with the environment. With these observations, we train a classifier which can transfer interaction knowledge to unobserved 3D scenes. We predict a likelihood of a given action taking place over all locations in a 3D environment and refer to this representation as an action map over the scene. We demonstrate prediction of action maps in both 3D scans and virtual scenes. We evaluate our predictions against ground truth annotations by people, and present an approach for characterizing 3D scenes by functional similarity using action maps.Keywords
This publication has 36 references indexed in Scilit:
- Real-time 3D reconstruction at scale using voxel hashingACM Transactions on Graphics, 2013
- Sketch2SceneACM Transactions on Graphics, 2013
- Symmetry in 3D Geometry: Extraction and ApplicationsComputer Graphics Forum, 2013
- Real-time human pose recognition in parts from single depth imagesCommunications of the ACM, 2013
- A search-classify approach for cluttered indoor scene understandingACM Transactions on Graphics, 2012
- Example-based synthesis of 3D object arrangementsACM Transactions on Graphics, 2012
- Data-driven graspingAutonomous Robots, 2011
- Learning Visual Object Categories for Robot Affordance PredictionThe International Journal of Robotics Research, 2009
- The WEKA data mining softwareACM SIGKDD Explorations Newsletter, 2009
- Efficient Graph-Based Image SegmentationInternational Journal of Computer Vision, 2004