Show HN:实时高斯 splatting
Show HN: Real-Time Gaussian Splatting

原始链接: https://github.com/axbycc/LiveSplat

LiveSplat是一个闭源的、alpha版本的实时高斯 splatting 算法,用于处理来自RGBD摄像头的流数据,由Mark Liu开发并公开发布。该算法设计用于Windows或Ubuntu系统(x86_64 CPU和Nvidia显卡),Python 3.12+版本,支持最多四个RGBD传感器。用户需要编写一个集成脚本将RGBD数据流传输到LiveSplat查看器,示例脚本针对Intel Realsense摄像头提供。虽然不是开源的,但LiveSplat已向公众提供。关于授权/集成事宜,请联系[email protected]。Ubuntu和Windows系统的安装说明均通过pip提供。

这个Hacker News帖子讨论了markisus创建的实时高斯散射系统“LiveSplat”,该系统使用神经网络将RGBD相机流转换为3D场景。虽然其保真度不如传统的高斯散射,但它通过利用深度数据实现了实时性能,从而允许交互式查看。 用户讨论了LiveSplat与其他高斯散射实现之间的技术差异,一些人指出它与点云渲染的相似性。Markisus承认为了速度而做出的妥协,强调了该系统能够根据神经网络对RGBD输入的解释动态调整散射大小和方向。 评论者表达了对潜在应用的兴奋,包括VR现场活动观看、风格转换和新型媒体创作。关于训练、视角相关效应以及该系统与简单的深度映射RGB可视化之间的差异的问题也随之出现。Markisus澄清说,使用了多摄像头的监督学习,神经网络学习根据相机位置插值颜色,从而产生比原始点云更具视觉吸引力的结果。

原文

LiveSplat is an algorithm for realtime Gaussian splatting using RGBD camera streams. Join our discord for discussion and help. Check out the demo video below to see the the type of output that LiveSplat produces.

LiveSplat_Alexa.mp4

LiveSplat was developed as a small part of a larger proprietary VR telerobotics system. I posted a video of the Gaussian splatting component of this system on Reddit and many people expressed an interest in experimenting with it themselves. So I spun it out and I'm making it publicly available as LiveSplat (see installation instructions below).

LiveSplat should be considered alpha quality. I do not have the resources to test the installation on many different machines, so let me know if the application does not run on yours (assuming your machine meets the requirements).

I've decided to keep LiveSplat closed source in order to explore potential business opportunities. If you are a business wanting to license/integrate this technology, please email [email protected].

I hope you have fun with LiveSplat!

— Mark Liu

  • Python 3.12+
  • Windows or Ubuntu (other Linux distributions may work, but are untested)
  • x86_64 CPU
  • Nvidia graphics card
  • One or more (up to four) RGBD sensors

Note that the application is not open source, and is covered by this license.

Ubuntu:
pip install https://livesplat.s3.us-east-2.amazonaws.com/livesplat-0.1.0-cp312-cp312-manylinux_x86_64.whl

Windows:
pip install https://livesplat.s3.us-east-2.amazonaws.com/livesplat-0.1.0-cp312-cp312-win_amd64.whl

To run LiveSplat, you will have to create an integration script that feeds your RGBD streams to the LiveSplat viewer. This repo provides an integration script for Intel Realsense devices called livesplat_realsense.py.

联系我们 contact @ memedata.com