Abstract
Three-dimensional Gaussian splatting (3D GS) is capturing attention due to its widespread applications in cutting-edge fields. Compared with the traditional neural radiance field (NeRF)-based methods, 3D GS demonstrates superior rendering realism and speed on high-performance graphics processing units (GPUs). However, 3D GS still faces challenges on edge devices. First, rasterization in 3D GS requires a large number of operators and a high bitwidth. This resource-intensive dataflow leads to significant power consumption and area overhead. Moreover, the extensive input-dependent ineffective computation in 3D GS causes unnecessary computational delays. Finally, the irregular and repeated access pattern of off-chip Gaussian parameters presents challenges for external memory access (EMA). To address these challenges, this work presents an energy-efficient edge 3D GS processor featuring: 1) a shape-aware reconfigurable rasterization–interpolation computation architecture reducing area and power overhead; 2) an online shape-aware early skipping controller (SESC) which efficiently detects ineffective computation; and 3) a spatio-temporal Gaussian cache to alleviate EMA overhead. The fabricated silicon prototype achieves a maximum throughput of 373 fps and a minimum rendering energy of $0.58~\text {mJ}/\text {frame}$ . Compared with state-of-the-art (SOTA) rendering accelerators, it achieves $1.96\times $ higher performance and $6.72\times $ lower energy consumption.
Funding Information
  • National Science and Technology Major Project (2021ZD0114402)
  • National Natural Science Foundation of China (92267203)
  • Beijing Major Science and Technology Project (Z241100004224016)