101001000 / elevenrender Goto Github PK

View Code? Open in Web Editor NEW

6.0 1.0 0.0 495 KB

A GPU-CPU SYCL 3D Rendering engine with it's own formally specified shading language "ASL"

C++ 91.49% C 2.48% CMake 3.87% Batchfile 0.40% Python 1.15% Shell 0.62%

cpp 3d-engine dpcpp gpu path-tracing raytracing-engine render rendering sycl

elevenrender's Introduction

Elever Render

A multi-platform vendor independent photorealistic rendering engine

Scene rendered with Eleven. 1000 samples and denoised

Eleven Render is my personal project. It's a unidirectional C++ path tracing rendering engine, where compatibility prevails over other aspects. Thanks to SYCL, it's possible to compile it for almost any GPU/CPU. It also seeks for broad compatibility with 3D Model software thanks to ASL, a shading language I'm working on, which allows other shading languages (like OSL), to be converted to it.

At the moment, Eleven is a WIP, and features like ASL are just proof of concepts.

Releases

The only way to use Eleven Render right now is through Blender, thanks to the ElevenBlender plug-in Download the last ElevenBlender release here: There's no releases yet.

Features

Compatibility with both GPU and CPU (vendor independent)
Normal recalculation with Vertex Weighted algorithm (so if your model has broken normals you won't have to worry about)
Osl compatibility through a custom shading language
MIS (BRDF, Environment, PointLights)
Denoising (Open Image Denoiser)

Build

Disclaimer At the current moment, there's not a builiding guide which will allow you to build Eleven flawlessly. I plan to release a Linux one, and to solve some compilation issues.

I'm also working on installing Boost libraries automatically, but there's some compatibility issues with Windows, Intel DPC++ and Boost

Windows (VS2022) - CUDA

Dependencies:

Python v3.11.0
CMake v3.25
Visual Studio 2022
Ninja v1.11.1
CUDA toolkit v11.8

The rest of the dependencies are downloaded by the building script.

Instructions:

Install Boost 1.80.00 on the system (C:\Program Files\boost\boost_1_80_0), and compile it following the step 5 from the linked instructions. Ideally you must compile it with clangw16 toolset.

Currently, the 16 version of Clang, is not available on Visual Studio, so if you want to avoid the mess to setup Visual Studio for clangw16, you can apply this quick dirty patch¹: You need to override the BOOST_LIB_TOOLSET variable, located in C:\Program Files\boost\boost_1_80_0\boost\config\auto_link.hpp to "clangw16", for example by typing #define BOOST_LIB_TOOLSET "clangw15" in the line 227.

Run build.bat. It will download automatically the rest of the dependencies, including the DPC++ Intel's toolchain.

Finally add "\build_deps\sycl_llvm-src\build\bin" to %PATH% and "\build_deps\sycl_llvm-src\build\lib" to %LIB% Environment variables

Docs

ASL

Eleven Render runs a custom formally specified shading language called Abstract Shading Language. ASL tries to unify shading languages, acting as middle step between other shading languages and Eleven.

Graph2OSL is a python tool I'm writting in the Eleven Blender's repo to convert Blender's shading node graph to OSL serialized shadergroup following the OSL specification serializing proposal.

OSL2ASL is a syntax converter I'm plannign to code, to convert OSL code to ASL code

ASL2LLVM is a formally correct compiler I'm writting as a part of my Master's thesis. The output is LLVM-IR code which can be compiled through some LLVM backend to almost any GPU/CPU architecture.

The compilation steps for a blender OSL material, are shown in the diagram below.

Compilation steps for blender materials

Third party libraries used

SYCL
Boost for TCP, logging, program args,
Intel Open Image Denoiser for final denoise
Flex for ASL parsing
Bison for ASL parsing
Mikktspace for generating the tangent space
Rapidobj for loading wavefront objects
stb_image & stb_image_write for image IO

Disclaimer

Eleven Render is the result of my Bachelor's Thesis and now my Master's Thesis. The way I worked on it was, to get the most features, in the smallest possible time, so most of the code is messy and unorganized. I'm working on modernizing the code and cleaning it up, as well as documenting it. Most of features are incomplete. The shading is giving me issues, so the implementation is probably not correct. The efficiency is also not the best and it's full of memory leaks. I hope to have all this sorted out in the following months, but for now, this is the render's state.

License

I was not able to compile Boost with the Intel's DPC++ toolchain, because it doesn't support Windows SEH exceptions, which are being used in the Boost::asio library. To solve it, I made a quick patch by compiling Boost manually and renaming some of the library files. I'm working on being able to compile Boost with CMAKE, meanwhile, that's the solution. ↩

elevenrender's People

Contributors

Stargazers

Watchers

elevenrender's Issues

A completely black HDRI will generate a completely black image.

Even with emission materials. But using an HDRI slightly brighter will make the final image normal again.

Adding more information to logs

Boost::log allows to add extra parameters to logging which can be really helpful.

https://stackoverflow.com/questions/24750218/boost-log-to-print-source-code-file-name-and-line-number

Blender image squished with new sensor size format

With the new sensor size format, the rendering for Blender is squished. Maybe I should check the sensor_fit = auto parameter

OpenImageDenoiser .zip not extracting

Adding kernels to the queue is extremely slow

There's a big overhead on adding samples to the queue. I need to figure out why SYCL is slowing down that part, or to make a sample scheduler for big sample targets.

How to handle the compilation step for end users.

Intel provides a oneAPI compiler installation which could reduce friction on the installation for render users. Codeplay also provides a plugin for that installation to work with CUDA.

https://www.intel.com/content/www/us/en/developer/articles/tool/oneapi-standalone-components.html#dpcpp-cpp

This compiler is the LLVM SCYL implementation I'm using right now, but with some proprietary Intel features, so maybe it's possible to use it instead distributing Clang directly.

That been said, after some preliminary tests, I wasn't able to compile Boost::log with icx neither with dpcpp by using only the -fsycl tag.

By using the flag -fsycl-targets=nvptx64-nvidia-cuda, I get an error of not being able to generate LLVM bytecode for that target because proprietary optimizations are enabled

I'm writing this issue for remembering the future me about this when I try to pack the first release.

RAII in everything

memory management is absolutely a nightmare right now.

Texture with different channel amount

Using 3 channel textures for everything is not ideal. The memory outprint can be x3 times higher for some cases, and texture loading requires specific constructions.

There's no way to handle engine stopping

--stop feature should be a thing, but it requires to rethink the dynamic loading architecture. I'm pretty sure that all those explicit memory allocations I did instead smart pointers are going to haunt me during sleep.

Windows building script needs to be run twice

The current CMakeLists.txt it's not detecting clang unix type arguments in the first pass, it's adding slash ones which are being interpreted as a directory.

Input abstraction

Right now I'm using just TCP inputs, but would be great if I could use also other inputs like stdin, a file input etc.

SEH from boost::asio win_mutex.ipp not compatible with clang SYCL

Windows uses SEH for some exception handling. \boost\asio\detail\impl\win_mutex.ipp and \boost\asio\detail\impl\win_static_mutex.ipp has a __try statement which is not compatible with the sycl clang build.

It's possible to replace those "__try" statements with "try" and it will work, even if I don't really know the consequences.

Device selector won't distinguish between the same device

There are occasions where there are two devices with the same name/platform and the current selector won't distinguish one from the other as it just compare name and platform. I didn't find any way to distinguish them with traits

Replace cmake FetchContent with git submodules

Install boost automatically with cmake Fetch Content

Currently, the CMakeLists.txt relies on a system boost installation, while it's possible to get the repo through cmake Fetch Content.

Running over CPU and a BLOCKSIZE=8 will create artifacts

I suspect that nd_range local^2 shouldn't exceed compute units. Maybe try to change nd_range for range and see if I'm correct.

InputManager and CommandManager do not have clear semantics

Would be good to actually rework the Command architecture to a more elegant one.

Visitor pattern for example.

Multi instance support

Blender requires to have multiple render instance support. This is conflicting with the current TCP architecture, and a must have for viewport rendering. A possible quick solution could be to trick blender to use a singleton and reset the render data with each instance.