SEAR: Scaling Experiences in Multi-user Augmented Reality
Authors:
- 15 February 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Visualization and Computer Graphics
- Vol. 28 (5) , 1982-1992
Abstract
In this paper, we present the design, implementation, and evaluation of SEAR, a collaborative framework for Scaling Experiences in multi-user Augmented Reality (AR). Most AR systems benefit from computer vision (CV) algorithms to detect, classify, or recognize physical objects for augmentation. A widely used acceleration method for mobile AR is to offload the compute-intensive tasks ( e.g. , CV algorithms) to the network edge. However, we show that the end-to-end latency, an important metric of mobile AR, may dramatically increase when offloading AR tasks from a large number of concurrent users to the edge. SEAR tackles this scalability issue through the innovation of a lightweight collaborative local caching scheme. Our key observation is that nearby AR users may share some common interests, and may even have overlapped views to augment ( e.g. , when playing a multi-user AR game). Thus, SEAR opportunistically exchanges the results of offloaded AR tasks among users when feasible and leverages compute resources on mobile devices to relieve, if necessary, the edge workload by intelligently reusing these results. We build a prototype of SEAR to demonstrate its efficacy in scaling AR experiences. We conduct extensive evaluations through both real-world experiments and trace-driven simulations. We observe that SEAR not only reduces the end-to-end latency, by up to 130×, compared to the state-of-the-art adaptive edge offloading scheme, but also achieves high object-recognition accuracy for mobile AR.Keywords
Funding Information
- Research Grants Council of Hong Kong (26211515, 16214817, 318927)
- Research Council of Finland (325570)
This publication has 56 references indexed in Scilit:
- Detecting, segmenting and tracking unknown objects using multi-label MRF inferenceComputer Vision and Image Understanding, 2014
- Military Applications of Augmented Reality2011
- Improving the Fisher Kernel for Large-Scale Image Classification2010
- SURF: Speeded Up Robust Features2006
- Towards Massively Multi-user Augmented Reality on Handheld Devices2005
- Distinctive Image Features from Scale-Invariant KeypointsInternational Journal of Computer Vision, 2004
- MagicMeeting: A Collaborative Tangible Augmented Reality SystemVirtual Reality, 2002
- Speaker Verification Using Adapted Gaussian Mixture ModelsDigital Signal Processing, 2000
- ?Studierstube?: An environment for collaboration in augmented realityVirtual Reality, 1998
- Determining optical flowArtificial Intelligence, 1981