Towards Energy-Efficiency by Navigating the Trilemma of Energy, Latency, and Accuracy
Preprint
Abstract
Extended Reality (XR) enables immersive experiences through untethered headsets but suffers from stringent battery and resource constraints. Energy-efficient design is crucial to ensure both longevity and high performance in XR devices. However, latency and accuracy are often prioritized over energy, leading to a gap in achieving energy efficiency. This paper examines scene reconstruction, a key building block for immersive XR experiences, and demonstrates how energy efficiency can be achieved by navigating the trilemma of energy, latency, and accuracy. We explore three classes of energy-oriented optimizations, covering the algorithm, execution, and data, that reveal a broad design space through configurable parameters. Our resulting 72 designs expose a wide range of latency and energy trade-offs, with a smaller range of accuracy loss. We identify a Pareto-optimal curve and show that the designs on the curve are achievable only through synergistic co-optimization of all three optimization classes and by considering the latency and accuracy needs of downstream scene reconstruction consumers. Our analysis covering various use cases and measurements on an embedded class system shows that, relative to the baseline, our designs offer energy benefits of up to 60X with potential latency range of 4X slowdown to 2X speedup. Detailed exploration of a use case across representative data sequences from ScanNet showed about 25X energy savings with 1.5X latency reduction and negligible reconstruction quality loss.Keywords
All Related Versions
This publication has 12 references indexed in Scilit:
- PockEngine: Sparse and Efficient Fine-tuning in a Pocket2023
- 3D Gaussian Splatting for Real-Time Radiance Field RenderingACM Transactions on Graphics, 2023
- Eye and head movements while encoding and recognizing panoramic scenes in virtual realityPLOS ONE, 2023
- Instant neural graphics primitives with a multiresolution hash encodingACM Transactions on Graphics, 2022
- Immersive audio-visual scene reproduction using semantic scene reconstruction from 360 camerasVirtual Reality, 2021
- NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis2020
- HAQ: Hardware-Aware Automated Quantization With Mixed Precision2019
- Acoustic Classification and Optimization for Multi-Modal Rendering of Real-World ScenesIEEE Transactions on Visualization and Computer Graphics, 2017
- Deep compression and EIE: Efficient inference engine on compressed deep neural network2016
- Efficient and robust radiance transfer for probeless photorealistic augmented reality2014