Comments (7)
Hello,
Please use this script: https://github.com/NVlabs/nvdiffrecmc/blob/main/blender/blender.py
I just tested it in Blender, and it generates reasonable results. You need to look at the shading tab or render the mesh in Cycles to see the shading.
One example on the Bob model below (tested in Blender 3.3.1)
from nvdiffrec.
Thanks for your kind words!
Unfortunately, we cannot actively support the Blender workflow, as Blender keeps changing etc. I've attached a version of a script we use to load our exported scenes in Blender, which may serve as a starting point. Not entirely sure if roughness should be squared etc.
Note also that the NerFactor versions of the NerF scenes are easier (and what we used for the relighting examples in the paper).
Here is an example on the nerf_hotdog training
import os
import bpy
import numpy as np
# path to your mesh
MESH_PATH = ".../nerf_hotdog/mesh"
bpy.ops.file.pack_all()
scene = bpy.context.scene
scene.world.use_nodes = True
scene.render.engine = 'CYCLES'
################### Import obj mesh ###################
imported_object = bpy.ops.import_scene.obj(filepath=os.path.join(MESH_PATH, "mesh.obj"))
obj_object = bpy.context.selected_objects[0]
obj_object.rotation_euler = [0, 0, 0]
################### Fix material graph ###################
# Get material node tree, find BSDF and specular texture
material = obj_object.active_material
bsdf = material.node_tree.nodes["Principled BSDF"]
image_node_ks = bsdf.inputs["Specular"].links[0].from_node
# Split the specular texture into metalness and roughness
separate_node = material.node_tree.nodes.new(type="ShaderNodeSeparateRGB")
separate_node.name="SeparateKs"
material.node_tree.links.new(image_node_ks.outputs[0], separate_node.inputs[0])
material.node_tree.links.new(separate_node.outputs[2], bsdf.inputs["Metallic"])
material.node_tree.links.new(separate_node.outputs[1], bsdf.inputs["Roughness"])
material.node_tree.links.remove(bsdf.inputs["Specular"].links[0])
# Set default values
bsdf.inputs["Specular"].default_value = 0.5
bsdf.inputs["Specular Tint"].default_value = 0.0
bsdf.inputs["Sheen Tint"].default_value = 0.0
bsdf.inputs["Clearcoat Roughness"].default_value = 0.0
################### Load HDR probe ###################
envmap = scene.world.node_tree.nodes.new(type="ShaderNodeTexEnvironment")
envmap.image = bpy.data.images.load(os.path.join(MESH_PATH, "probe.hdr"))
scene.world.node_tree.links.new(envmap.outputs[0], scene.world.node_tree.nodes['Background'].inputs['Color'])
from nvdiffrec.
Here is a visual example of the shading network as well
from nvdiffrec.
Thanks for your kind words!
Unfortunately, we cannot actively support the Blender workflow, as Blender keeps changing etc. I've attached a version of a script we use to load our exported scenes in Blender, which may serve as a starting point. Not entirely sure if roughness should be squared etc.
Note also that the NerFactor versions of the NerF scenes are easier (and what we used for the relighting examples in the paper).
Here is an example on the nerf_hotdog training
import os import bpy import numpy as np # path to your mesh MESH_PATH = ".../nerf_hotdog/mesh" bpy.ops.file.pack_all() scene = bpy.context.scene scene.world.use_nodes = True scene.render.engine = 'CYCLES' ################### Import obj mesh ################### imported_object = bpy.ops.import_scene.obj(filepath=os.path.join(MESH_PATH, "mesh.obj")) obj_object = bpy.context.selected_objects[0] obj_object.rotation_euler = [0, 0, 0] ################### Fix material graph ################### # Get material node tree, find BSDF and specular texture material = obj_object.active_material bsdf = material.node_tree.nodes["Principled BSDF"] image_node_ks = bsdf.inputs["Specular"].links[0].from_node # Split the specular texture into metalness and roughness separate_node = material.node_tree.nodes.new(type="ShaderNodeSeparateRGB") separate_node.name="SeparateKs" material.node_tree.links.new(image_node_ks.outputs[0], separate_node.inputs[0]) material.node_tree.links.new(separate_node.outputs[2], bsdf.inputs["Metallic"]) material.node_tree.links.new(separate_node.outputs[1], bsdf.inputs["Roughness"]) material.node_tree.links.remove(bsdf.inputs["Specular"].links[0]) # Set default values bsdf.inputs["Specular"].default_value = 0.5 bsdf.inputs["Specular Tint"].default_value = 0.0 bsdf.inputs["Sheen Tint"].default_value = 0.0 bsdf.inputs["Clearcoat Roughness"].default_value = 0.0 ################### Load HDR probe ################### envmap = scene.world.node_tree.nodes.new(type="ShaderNodeTexEnvironment") envmap.image = bpy.data.images.load(os.path.join(MESH_PATH, "probe.hdr")) scene.world.node_tree.links.new(envmap.outputs[0], scene.world.node_tree.nodes['Background'].inputs['Color'])
thank you for your qucik reply, I think the script is very useful! Is there any render engine can repeat the results OR is it possible to use nvdiffrast to get the results?
from nvdiffrec.
nvdiffrec is rendering the scenes in the validation loop already, so that is the "reference" engine (rasterizer with direct illumination without shadows), and the renderer we use in the optimization process.
In Blender, we render with path tracing, so that is not a perfect match, but works reasonably well.
from nvdiffrec.
Got it, thanks for your kindly reply.
from nvdiffrec.
@jmunkberg, I am having trouble running the script you provided above.
When I run it, I am getting the error:
Info: No new files have been packed
Info: No new files have been packed
( 0.0000 sec | 0.0000 sec) Importing OBJ '/home/andrewg/Documents/nvdiffrec/out/nerf_hotdog/hotdog_mesh_nvdiffrec/mesh.obj'...
( 0.0001 sec | 0.0000 sec) Parsing OBJ file...
( 0.5009 sec | 0.5008 sec) Done, loading materials and images...
WARNING: '/home/andrewg/Documents/nvdiffrec/out/nerf_hotdog/hotdog_mesh_nvdiffrec/mesh.obj':b'bsdf pbr' (ignored)
( 0.5024 sec | 0.5023 sec) Done, building geometries (verts:27788 faces:55632 materials: 1 smoothgroups:1) ...
( 0.9443 sec | 0.9442 sec) Done.
( 0.9444 sec | 0.9444 sec) Finished importing: '/home/andrewg/Documents/nvdiffrec/out/nerf_hotdog/hotdog_mesh_nvdiffrec/mesh.obj'
Progress: 100.00%
1
Exception ignored in: <function CyclesRender.__del__ at 0x7f7cd9679a70>
Traceback (most recent call last):
File "/home/andrewg/miniconda3/envs/bpy/lib/python3.7/site-packages/2.91/scripts/addons/cycles/__init__.py", line 67, in __del__
TypeError: 'NoneType' object is not callable
Segmentation fault (core dumped)
I am new to using Blender and bpy
. What is the expected output of this script?
Update
I figured that I needed to run this script inside the Blender script editor. However, after I run the script, the loaded mesh does not have any colour.
from nvdiffrec.
Related Issues (20)
- Not able to learn specular and diffuse properties HOT 1
- What's the purpose of custom mipmap build function. HOT 2
- Add a how to cite! HOT 3
- Large gradient at mesh vertices HOT 1
- JSON parameter optimization for Mesh extraction HOT 2
- The results generated by the nerd dataset are poor HOT 4
- what's the poses_bounds.npy's data format HOT 3
- question about code details HOT 2
- is there any way to use multi lights? HOT 2
- How to train from a pre-defined mesh? HOT 2
- Creating a new bsdf_256_256.bin HOT 5
- util.rotate_x seems inverse from gl's right hand rule HOT 1
- Use DMTet on udf. HOT 2
- Why isn't the MLPTexture3D using the tcnn Network definition HOT 2
- OSError: CUDA_HOME environment variable is not set. Please set it to your CUDA install root. Building PyTorch extension for tiny-cuda-nn version 1.7 Obtained compute capabilities [86] from environment variable TCNN_CUDA_ARCHITECTURES Detected CUDA version 11.6 Targeting C++ standard 17 [end of output] HOT 1
- Getting stuck during installation HOT 2
- Normal map for training HOT 2
- `groupshared` is not supported in slangtorch. HOT 1
- Optimize Mesh of an Indoor Scene HOT 1
- Optimize mesh of a single building using custom images HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from nvdiffrec.