ROSplat is the first online ROS2-based visualizer that leverages Gaussian splatting to render complex 3D scenes. It is designed to efficiently visualize millions of gaussians by using custom ROS2 messages and GPU-accelerated sorting and rendering techniques. ROSplat also supports data loading from PLY files and integrates with ROS2 tools such as bag recording.

• Real-Time Visualization: Render millions of Gaussian “splats” in real time.
• ROS2 Integration: Built on ROS2 for online data exchange of Gaussians, Images, and IMU data.
• Custom Gaussian Messages: Uses custom message types (SingleGaussian and GaussianArray) to encapsulate properties such as position, rotation, scale, opacity, and spherical harmonics.
• GPU-Accelerated Sorting & Rendering: Offloads sorting and rendering tasks to the GPU.

This project was developed and tested on Ubuntu 24.04 LTS using ROS2 Jazzy. Please note: Performance degrades significantly without an NVIDIA graphics card.

• Mandatory: ROS2 (tested on ROS2 Jazzy)
• Optional (for GPU-based Sorting):
  • cupy (ensure compatibility with your CUDA version)
  • torch (if using PyTorch for GPU sorting)

To install the optional GPU-based libraries individually:

pip install cupy-cuda12x  # Install Cupy (replace 12x with your CUDA version)

pip install torch         # Install PyTorch

The program will automatically prioritize sorting methods in the following order: 1) Torch → 2) Cupy → 3) CPU

To install all dependencies at once:

pip install -r requirements.txt        # For GPU acceleration

pip install -r requirements-no-gpu.txt  # Without GPU acceleration

Alternatively, you can set up the project using Docker. A setup script is available under the docker directory.

Before running Docker, ensure you have installed:

sudo apt-get install -y nvidia-container-toolkit

This enables GPU communication between the host and the container. If you come accross any other issues, follow the instructions under the following link

Then, to build and run the Docker container:

cd docker
./run_docker.sh -h    # Display help and usage instructions
./run_docker.sh -bu   # Build the Docker image and launch the container with docker-compose

Important: Ensure the host machine's CUDA version matches the version specified in the Dockerfile. If you are using a CUDA version other than 12.6, update the Dockerfile accordingly.

ROSplat defines two custom ROS2 messages to handle Gaussian data, located in the gaussian_interface/msg folder.

Note: The Gaussian messages are based on the original Gaussian Splatting implementation by graphdeco-inria.

geometry_msgs/msg/Point xyz
geometry_msgs/msg/Quaternion rotation
float32 opacity
geometry_msgs/msg/Vector3 scale
float32[] spherical_harmonics

gaussian_interface/SingleGaussian[] gaussians

a) Build your workspace using colcon:

colcon build --packages-select gaussian_interface

b) Source your workspace:

Important: Depending on your shell, you might need to adjust these commands.

Once the Gaussian messages are built, you can launch the visualizer from the project's root directory:

Contributions and feedback are welcome!

I'm glad to have worked on such a challenging topic and grateful for the invaluable advice and support I received throughout this project.

Special thanks to Qihao Yuan and Kailai Li for their guidance and encouragement as well as the constructive feedback that helped shape this work.

This project was additionally influenced by limacv 's implementation of the GaussianSplattingViewer repository.

For questions or further information, please email: shady.gmira[at]gmail.com