Git Product home page Git Product logo

Comments (13)

cmh1027 avatar cmh1027 commented on July 27, 2024 1

But I couldn't find ball.blend file in the original ref-nerf repository. Where did you find it?

from nmf.

half-potato avatar half-potato commented on July 27, 2024 1

https://drive.google.com/file/d/1Ii--GE9KwU5NJZ9WwbBxYLfKGH740YQx/view?usp=sharing

from nmf.

half-potato avatar half-potato commented on July 27, 2024

Checkout the relighting dataset

from nmf.

cmh1027 avatar cmh1027 commented on July 27, 2024

Can you share the script for relighting for the ball, or blend file? I want to do relighting with env maps used in NVDiffrecMC

from nmf.

half-potato avatar half-potato commented on July 27, 2024

Is this blend script what you are looking for? Just change out the envmap in blender.

# A simple script that uses blender to render views of a single object by rotation the camera around it.
# Also produces depth map at the same time.

import argparse, sys, os
import json
import bpy
import mathutils
import numpy as np
         
DEBUG = False
            
VIEWS = 100
RESOLUTION = 800
RESULTS_PATH = 'train'
DEPTH_SCALE = 1.4
COLOR_DEPTH = 8
FORMAT = 'PNG'
RANDOM_VIEWS = True


fp = bpy.path.abspath(f"//{RESULTS_PATH}")


def listify_matrix(matrix):
    matrix_list = []
    for row in matrix:
        matrix_list.append(list(row))
    return matrix_list

if not os.path.exists(fp):
    os.makedirs(fp)

# Data to store in JSON file
out_data = {
    'camera_angle_x': bpy.data.objects['Camera'].data.angle_x,
}

# Render Optimizations
bpy.context.scene.render.use_persistent_data = True


# Set up rendering of depth map.
bpy.context.scene.use_nodes = True
tree = bpy.context.scene.node_tree
links = tree.links

# Add passes for additionally dumping albedo and normals.
bpy.context.scene.view_layers["RenderLayer"].use_pass_normal = True
#bpy.context.scene.render.image_settings.file_format = str(FORMAT)
#bpy.context.scene.render.image_settings.color_depth = str(COLOR_DEPTH)


# Create input render layer node.
render_layers = tree.nodes.new('CompositorNodeRLayers')

depth_file_output = tree.nodes.new(type="CompositorNodeOutputFile")
depth_file_output.label = 'Depth Output'
if FORMAT == 'OPEN_EXR':
  links.new(render_layers.outputs['Depth'], depth_file_output.inputs[0])
else:
  # Remap as other types can not represent the full range of depth.
  map = tree.nodes.new(type="CompositorNodeMapValue")
  # Size is chosen kind of arbitrarily, try out until you're satisfied with resulting depth map.
  map.offset = [-0.7]
  map.size = [DEPTH_SCALE]
  map.use_min = True
  map.min = [0]
  links.new(render_layers.outputs['Depth'], map.inputs[0])

  links.new(map.outputs[0], depth_file_output.inputs[0])

normal_file_output = tree.nodes.new(type="CompositorNodeOutputFile")
normal_file_output.label = 'Normal Output'
links.new(render_layers.outputs['Normal'], normal_file_output.inputs[0])

# Background
bpy.context.scene.render.dither_intensity = 0.0
#bpy.context.scene.render.film_transparent = True

# Create collection for objects not to render with background

    
objs = [ob for ob in bpy.context.scene.objects if ob.type in ('EMPTY')]
bpy.ops.object.delete({"selected_objects": objs})

def parent_obj_to_camera(b_camera):
    origin = (0, 0, 0)
    b_empty = bpy.data.objects.new("Empty", None)
    b_empty.location = origin
    b_camera.parent = b_empty  # setup parenting

    scn = bpy.context.scene
    scn.collection.objects.link(b_empty)
    bpy.context.view_layer.objects.active = b_empty
    # scn.objects.active = b_empty
    return b_empty


scene = bpy.context.scene
scene.render.resolution_x = RESOLUTION
scene.render.resolution_y = RESOLUTION
scene.render.resolution_percentage = 100

cam = scene.objects['Camera']
cam.location = (0, 4.0, 0.5)
cam_constraint = cam.constraints.new(type='TRACK_TO')
cam_constraint.track_axis = 'TRACK_NEGATIVE_Z'
cam_constraint.up_axis = 'UP_Y'
b_empty = parent_obj_to_camera(cam)
cam_constraint.target = b_empty

#scene.render.image_settings.file_format = 'PNG'  # set output format to .png

from math import radians

stepsize = 360.0 / VIEWS
rotation_mode = 'XYZ'


for output_node in [depth_file_output, normal_file_output]:
    output_node.base_path = ''

out_data['frames'] = []

for i in range(0, VIEWS):
    if RANDOM_VIEWS:
        scene.render.filepath = fp + '/r_' + str(i)
        b_empty.rotation_euler = np.random.uniform(0, 2*np.pi, size=3)
    else:
        print("Rotation {}, {}".format((stepsize * i), radians(stepsize * i)))
        scene.render.filepath = fp + '/r_{0:03d}'.format(int(i * stepsize))

    # depth_file_output.file_slots[0].path = scene.render.filepath + "_depth_"
    # normal_file_output.file_slots[0].path = scene.render.filepath + "_normal_"

    if DEBUG:
        break
    else:
        bpy.ops.render.render(write_still=True)  # render still

    frame_data = {
        'file_path': scene.render.filepath,
        'rotation': radians(stepsize),
        'transform_matrix': listify_matrix(cam.matrix_world)
    }
    out_data['frames'].append(frame_data)

    if RANDOM_VIEWS:
        b_empty.rotation_euler = np.random.uniform(0, 2*np.pi, size=3)
    else:
        b_empty.rotation_euler[2] += radians(stepsize)

if not DEBUG:
    with open(fp + '/' + 'transforms.json', 'w') as out_file:
        json.dump(out_data, out_file, indent=4)

from nmf.

cmh1027 avatar cmh1027 commented on July 27, 2024

@half-potato
forest_r_003
forest_r_003_recon
These images are relighted images of coffee rendered on envmap forest.hdr (GT and prediction for each)
As you can see, the predicted one has very dark ambience. I'm not sure if I did something wrong while following the instruction in repo. How do you think?

from nmf.

half-potato avatar half-potato commented on July 27, 2024

There is an arbitrary multiplier on the brightness, a global scaling that must be tuned when transferring a lighting condition. It is fundamental to the problem of relighting. Finding this global scaling isn't done on prediction phase, so it must be done when either testing PSNR or displaying the image.

from nmf.

cmh1027 avatar cmh1027 commented on July 27, 2024

Is it done manually?

from nmf.

half-potato avatar half-potato commented on July 27, 2024

No. It's just the ratio between the average brightness of the target images and the predicted images.

from nmf.

gkouros avatar gkouros commented on July 27, 2024

https://drive.google.com/file/d/1Ii--GE9KwU5NJZ9WwbBxYLfKGH740YQx/view?usp=sharing

This doesn't seem to be the same blender model used for rendering the ball scene.

This is what I see in blender:
Screenshot from 2024-03-04 10-52-28

from nmf.

half-potato avatar half-potato commented on July 27, 2024

Just change the parameters for the shader.

from nmf.

cmh1027 avatar cmh1027 commented on July 27, 2024

The 'ball' scene was rendered in Python using a Phong BRDF, which is not available in Blender. The geometry is a unit sphere, and the two rough "bands" are at (abs(x) < 0.15) || (abs(y) < 0.15). The rough bands have a Phong exponent of 50, with the rest set to 500.

The original README file states like this. Is it possible to simulate the same Phong condition in Blender?

from nmf.

half-potato avatar half-potato commented on July 27, 2024

Oh Dor has a script for that. Not sure where it is, email him. The relighting dataset uses blender GGX though.

from nmf.

Related Issues (17)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.