What is 3D Gaussian Splatting?

In short, it's a method that can give robots a visual and spatial foundation for their 3D memory.

In detail, 3D Gaussian Splatting (3DGS) turns photos or video captured by a robot into a photorealistic 3D reconstruction of its surroundings that it can view from new angles.

This helps robots refer back to places and objects that are no longer in view when a task calls for it.

Here is how photos become a 3D scene for robots:

1. The system matches details across photos/videos to estimate where the cameras were and build a rough map of 3D points.

2. Those points become tiny, soft blobs called Gaussians. Each has a position, shape, color and opacity.

3. The system renders the room and checks it against the photos.
The Gaussians shift, stretch and change color until the views match more closely. Some split to capture finer details, and others are removed.

4. Once the Gaussians have been adjusted to match the photos, the scene can be viewed from new angles.
The GPU projects and blends them fast enough for you to look around in real time.

The result shows a scene from different viewpoints. However, it doesn’t explain its physics or predict motion.
And making it a robot’s memory requires another layer.

We've made a comprehensive guide about this hot topic and how Gaussian Splatting becomes Gaussian Memory.
A must-read about embodied memory: https://www.turingpost.com/p/3d-gaussian-splatting-explained