Gaussian-plus-SDF SLAM: High-fidelity 3D Reconstruction at 150+ fps

Zhexi Peng1,2, Kun Zhou1,2, Tianjia Shao1,2,
1State Key Lab of CAD&CG, Zhejiang University
2Hangzhou Research Institute of AI and Holographic Technology
Paper Code & Data
MY ALT TEXT

In our Gaussian-plus-SDF representation, the SDF provides 3D structure and initial color, while the 3D Gaussians are optimized to correct residual color errors. This allows us to achieve high-fidelity reconstruction with a small number of Gaussians, enabling ultra-fast scene reconstruction.

Abstract

While recent Gaussian-based SLAM methods achieve photorealistic reconstruction from RGB-D data, their computational performance remains a critical bottleneck. State-of-the-art techniques operate at less than 20 fps, significantly lagging behind geometry-based approaches like KinectFusion (hundreds of fps). This limitation stems from the heavy computational burden: modeling scenes requires numerous Gaussians and complex iterative optimization to fit RGB-D data; insufficient Gaussian counts or optimization iterations cause severe quality degradation. To address this, we propose a Gaussian-SDF hybrid representation, combining a colorized signed distance field (SDF) for smooth geometry and appearance with 3D Gaussians to capture underrepresented details. The SDF is efficiently constructed via RGB-D fusion (as in geometry-based methods), while Gaussians undergo iterative optimization. Our representation enables significant Gaussian reduction (50% fewer) by avoiding full-scene Gaussian modeling, and efficient Gaussian optimization (75% fewer iterations) through targeted appearance refinement. Building upon this representation, we develop GPS-SLAM (Gaussian-plus-SDF SLAM), a real-time 3D reconstruction system achieving over 150 fps on real-world Azure Kinect sequences, faster by an order-of-magnitude than state-of-the-art techniques while maintaining comparable reconstruction quality.

Real-time Reconstruction in 15s

Overview

MY ALT TEXT

Overview of our GPS-SLAM system. Given an RGB-D image as input, standard SDF fusion is performed to update the SDF and color values in a global hash table. Then, we sample pixels within regions exhibiting significant color errors and add Gaussians to these locations. Online optimization for 3D Gaussians is executed based on our Gaussian-plus-SDF rendering.

Gaussian-Plus-SDF Rendering

MY ALT TEXT

Gaussian-plus-SDF representation and rendering process. Our representation consists of a SDF volume \( S \) and a set of 3D Gaussians \( G = \{ p_i, \sigma_i, r_i, s_i, SH_i \}_{i=1}^M \), with properties: position \( p_i \), maximum opacity \( \sigma_i \), scaling \( s_i \), rotation \( r_i \) and SH coefficients \( SH_i \). The rendering process consists of two passes. Firstly, a standard per-pixel ray-casting is performed on the SDF volume to obtain a color map \( C_t \) and depth map \( D_t \). Secondly, we splat the Gaussians to the screen, and their colors and weights are blended together independently of order with depth testing based on the SDF-rendered depth map \( D_t \). A small positive threshold \( \epsilon \) is used to prevent incorrect truncation. The accumulated Gaussian color and SDF-rendered color are combined as a weighted average to get the final image.

Rendering Quality Comparison

System Performance Comparison

MY ALT TEXT

More Results

BibTeX


      @article{peng2025gpsslam,
      title={Gaussian-Plus-SDF SLAM: High-fidelity 3D Reconstruction at 150+ fps},
      author={Zhexi Peng and Kun Zhou and Tianjia Shao},
      journal={Computational Visual Media},
      year={2025},
      publisher={TUP}
      }