shg8 / 3dgs.cpp Goto Github PK

A cross-platform, high performance renderer for Gaussian Splatting using Vulkan Compute. Supports ✅ Windows, Linux, macOS, iOS, and visionOS

License: GNU Lesser General Public License v2.1

C++ 85.52% CMake 3.93% GLSL 1.05% C 0.74% Swift 8.76%

3d-gaussian-splatting 3dgs gaussian-splatting nerf neural-rendering radiance-field ios visionos vulkan

3dgs.cpp's Introduction

3DGS.cpp

3DGS.cpp is a high performance, cross-platform implementation of Gaussian Splatting using the Vulkan API and compute pipelines.

Why Vulkan? We want to democratize the access to high-performance point-based radiance fields. Existing implementations of Gaussian Splatting are often limited to CUDA, which only runs on NVIDIA GPUs, and OpenGL is deprecated on Apple platforms. Additionally, Vulkan's compute capabilities are the closest to CUDA's with support for warp-level primitives (subgroups).

Downloads

Windows
macOS (arm64)
Linux
iOS, iPad OS, visionOS (as an iPad app) on TestFlight

Command Line Viewer

  ./3dgs_cpp_viewer {OPTIONS} [scene]

    Vulkan Splatting

  OPTIONS:

      -h, --help                        Display this help menu
      --validation                      Enable Vulkan validation layers
      -v, --verbose                     Enable verbose logging
      -d[physical-device],
      --device=[physical-device]        Select physical device by index
      -i, --immediate-swapchain         Set swapchain mode to immediate
                                        (VK_PRESENT_MODE_IMMEDIATE_KHR)
      -w[width], --width=[width]        Set window width
      -h[height], --height=[height]     Set window height
      --no-gui                          Disable GUI
      scene                             Path to scene fil

Building

Linux

3DGS.cpp requires the following dependencies:

Vulkan headers, Vulkan validation layers, glslangValidator, glfw, glm

The easiest way to install the first three is through the LunarG Vulkan SDK. Alternatively, you can install the corresponding packages from your distro. For Ubuntu, the packages to install are vulkan-headers, vulkan-validationlayers, glslang-dev, libglfw3-dev, libglm-dev.

git clone https://github.com/shg8/3DGS.cpp/
cd 3DGS.cpp
mkdir build
cmake -DCMAKE_BUILD_TYPE=Release -S ./ -B ./build
cmake --build ./build -j4

The command line viewer will be under build/apps/viewer/.

Windows

After installing Vulkan SDK, set the VULKAN_SDK environmental variable to the install path. Alternatively, pass -DVULKAN_SDK=\INSTALL\LOCATION\OF\YOUR\SDK to CMake when configuring.

macOS

After installing the Vulkan SDK, please proceed with CMake configuration and build steps.

iOS, iPadOS, visionOS

Make sure that the Vulkan SDK is installed
Run the xcode_setup CMake target
Download the prebuilt MoltenVK libraries or build it yourself. Unzip the downloaded artifact from a MoltenVK workflow. If you're not sure which one you want to use, pick the one from the latest tag
Place the MoltenVK folder that contains the dynamic, include, and static folders under apps/apple
Set your development team id in project.xcconfig
Profit

TODO

Better controls and GUI on GLFW
Apps for iOS and visionOS
Fully immersive app on visionOS using the Compositor Service framework
OpenXR support
App for Android and Qualcomm Spaces
Implement SOTA parallel radix sort
Use Vulkan subgroups to batch Gaussian retrievals at the warp level
Training

Please feel free to open an issue if you have any feature suggestions or are interested in contributing.

Benchmarking

There is room for performance improvements. See benchmarks on a 4090 at 3200x1400.

Contributing

If you are interested in integrating your Gaussian Splatting variant, please open an issue or a pull request. 3DGS.cpp's shaders follow the procedures outlined in the original paper, so it should be relatively easy to port your CUDA code. With cross-platform support, it's a great way to expand the reach and adoption of your research. If there are any questions, feel free to send me an email.

License

The main project is licensed under LGPL.

This project uses several third-party libraries. Here is a list of these libraries along with their respective licenses:

GLM: MIT License.
args.hxx: MIT License.
spdlog: MIT License.
ImGUI: MIT License.
Vulkan Memory Allocator: MIT License.
VkRadixSort: MIT License.
implot: MIT License.
glfw: zlib/libpng license.
libenvpp: Apache License 2.0.

3dgs.cpp's People

Contributors

Stargazers

Watchers

Forkers

cedric-chedaleux maksym-pasichnyk ciprianflorea2002 jerryzfc mizadri herculesyuan anonq115 joskid whztt07 xuhao1 chengwei920412 zhoushiwei chilieater yuhuoo xuyanging nobody-in-zhuposhan 6nevergiveup hujiajun16

3dgs.cpp's Issues

Question about sort.comp

Hello, this is a really great job for me! But I'm having trouble reading the source code, can you explain it to me? I think sort.comp needs to be sorted according to depth, but I didn’t find the part that uses depth sorting. I only saw the code that groups the tiles together in one tile. Can you help me answer it? Thank you very much.

Unable to find Vulkan SDK on mac

Hi there, I came across this project from reddit. I am trying to compile this project for macOS and also visionos to test on my vision pro.
I have install Vulkan SDK

(base) ➜  3DGS.cpp git:(main) ✗ ls /Users/xuhao/VulkanSDK/1.3.275.0
Applications        InstallationLog.txt MaintenanceTool.dat VERSIONS.txt        install_vulkan.py   macOS               tvOS                vulkan.7z
Helpers             Licenses            MaintenanceTool.ini components.xml      installerResources  network.xml         tvos.7z
INSTALL_BOM.json    MaintenanceTool.app README.txt          iOS                 ios.7z              setup-env.sh        uninstall.sh

However, when build this project with

cmake -DCMAKE_BUILD_TYPE=Release -DVULKAN_SDK="/Users/xuhao/VulkanSDK/1.3.275.0" -S ./ -B .\VulkanSplatting\build

cmake -DCMAKE_BUILD_TYPE=Release -DVULKAN_SDK="/Users/xuhao/VulkanSDK/1.3.275.0/macOS" -S ./ -B ./VulkanSplatting/build

It will come with

(base) ➜  3DGS.cpp git:(main) ✗ cmake -DCMAKE_BUILD_TYPE=Release -DVULKAN_SDK="/Users/xuhao/VulkanSDK/1.3.275.0/" -S ./ -B ./VulkanSplatting/build

-- Could NOT find Vulkan (missing: Vulkan_LIBRARY Vulkan_INCLUDE_DIR glslangValidator) (found version "")
-- Using Cocoa for window creation
-- GLM: Version 1.0.0
-- GLM: Disable -Wc++98-compat warnings
-- GLM: Build with C++20 features
GLM: Clang - AppleClang compiler
-- Module support is disabled.
-- Version: 9.1.0
-- Build type: Release
-- CXX_STANDARD: 20
-- Required features: cxx_variadic_templates
-- Build spdlog: 1.13.0
-- Build type: Release
-- Shader header file: /Users/xuhao/develop/SpatialComputing/3DGS.cpp/VulkanSplatting/build/shaders/shaders.h
-- Configuring done (1.1s)
CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
Please set them or make sure they are set and tested correctly in the CMake files:
/Users/xuhao/develop/SpatialComputing/3DGS.cpp/src/Vulkan_INCLUDE_DIR
   used as include directory in directory /Users/xuhao/develop/SpatialComputing/3DGS.cpp/src

CMake Error at src/CMakeLists.txt:45 (target_link_libraries):
  Target "vulkan_splatting" links to:

    Vulkan::Vulkan

  but the target was not found.  Possible reasons include:

    * There is a typo in the target name.
    * A find_package call is missing for an IMPORTED target.
    * An ALIAS target is missing.



-- Generating done (0.0s)
CMake Warning:
  Manually-specified variables were not used by the project:

    VULKAN_SDK


CMake Generate step failed.  Build files cannot be regenerated correctly.

Which path should I use?

missing head file shaders.h in GSScene.cpp

I run the program on apple m1 pro with next command line and run successfully:
mkdir build
cd build
cmake ..
make
cd build/apps/viewer
./3dgs_viewer ply_file_path

but when I want to check the code i found there is an error My CLion told me:

Need a trackbar control mode

Could you add a trackbar control mode to this great viewer, just like meshlab~
The default mouse and keyboard control modes are a bit uncomfortable to use now :(

Rendering red screen

Good afternoon! Thank you for the amazing 3DGS.cpp project. I have some background with WebGPU and basic TensorFlow but am generally still learning graphics programming and machine learning concepts.

I have an issue that may be related to my incorrect usage or settings, so my apologies ahead of time. However, the issue is pictured below:

I tried the nightly build as well 3 different builds at various stages in the commit history, but the latest seems most stable. Yet it is only giving a red screen.

I am able to open schoolbus.ply in another Python-based viewer. The file is binary (text header, binary body) as required.

I have a fresh installation of Vulkan and all the needed dependencies, and I am able to run Vulkan Cube, so it seems that Vulkan is working properly. If you'd like to examine schoolbus.ply, I could probably ask my client if that is okay!

Thank you for your time!

platform support

hi， thank you for your sharing code, i wonder can this project run in android platform such as Quest or mobilephone？

Gaussian instantiation out of memory

Hi, I have find some problem with Gaussian instantiation out of memory.

First time, I run a ply file ~2.1GB, and it worked well.
GPU memory usage went to ~8.5GB at first, then went to 9.4GB when the render result showed.

The next time, I run a ply file ~500MB, which is an edited version of the first file, by deleting some 'fog' points.
GPU memory usage also went to ~8.5GB at first, but then it came out Gaussian instantiation out of memory.

Why an edited small ply file causes out of memory?
The graphic card is Geforce RTX 2080 Ti 11GB.

Thanks.

Debug mode compile caused linking issue

Thanks for this good project! I have to say that I have successfully run the project with my data on Release Mode:

But on Windows platform with (MSVC 2022 buildtool), the Debug mode will cause the linking issue:

which is quite strange. Any suggestion?
On the other hands, the profiler sames not meet the diagram size (I guess?), because the diagrams crowed on the right side:

(This data is of a black background, don't worry about the result displayed)

Any suggestion about that?

Anyway, this is just a little "bug" I have encountered, so I would like to share and report it.
Thanks for the contribution of such a great project :-). Best!

Skip rendering loop when there's no view changes

I built the viewer and tried to use it to load output of original 3DGS and got this error message:

(base) linhanwang@MacBook-Air:~/softwares/3DGS.cpp ⚡ ./build/apps/viewer/3dgs_viewer -v ~/Documents/sep_road/
^[[O[13:06:26] [D] Initializing Vulkan
[13:06:26] [D] Vulkan instance created
[13:06:26] [I] Available physical devices:
[13:06:26] [I] [0] Apple M3
[13:06:26] [I] Selected physical device (automatically): Apple M3
[13:06:26] [D] Logical device created
[13:06:26] [D] Surface format: VK_FORMAT_B8G8R8A8_UNORM
[13:06:26] [D] Present mode: VK_PRESENT_MODE_FIFO_KHR
[13:06:26] [D] Swapchain extent range: 1x1 - 16384x16384
[13:06:26] [D] Swapchain extent: 2560x1440. Preferred extent: 2560x1440
[13:06:26] [D] Swapchain created
[13:06:26] [D] Creating GUI
[13:06:26] [D] Loading scene to GPU
[13:06:26] [C] Could not find end of header
Could not find end of header

Crash when selecting physical device

I use prebuilt binary.

../Downloads/3dgs_viewer-ubuntu-latest-amd64-nightly splat.ply --device=1