compphoto / intrinsic Goto Github PK

Hi,
Thank you very much for your work. If you want to implement color editing and relighting in the paper, how to achieve it? I didn't find this function in the published code.

Hi, thanks for the great work.
In my task, I aim to decomposite images from a multi-view image set. First I take images one-by-one as input to your model. Results of two views are presented as follows:


It's obvious that the albedo images of the roof are not consistent.
Does this work provide multi-view consistency functionality? If not, could you please give me some advice to improve multi-view consistency when decompositing images into albedo and shading?
I sincerely look forward to your reply.

Thank you for this great work. I am trying to use both paper weights and rendered weights, but I get this issue:

File "/home/user/Intrinsic/intrinsic/model_util.py", line 40, in load_models
iid_model.load_state_dict(iid_state_dict)
File "/home/user/miniconda3/lib/python3.12/site-packages/torch/nn/modules/module.py", line 2215, in load_state_dict
raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for MidasNet_small:
size mismatch for scratch.layer2_rn.weight: copying a param with shape torch.Size([128, 48, 3, 3]) from checkpoint, the shape in current model is torch.Size([64, 48, 3, 3]).

I've understood that the paper is intended for use on single images. Hence, applying the work to a video results in flashes and artefacts over the course of the video. I'll attach an example tomorrow. Have you attempted any methods to make your model temporally consistent or any quick fixes that/ideas on how one might approach this problem?

I was trying to run the provided collab and running to the following error.

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-6-9b7c6ff62d88> in <cell line: 2>()
      1 # run the image through the pipeline (use R0 resizing dicussed in the paper)
----> 2 result = run_pipeline(
      3     intrinsic_model,
      4     img,
      5     resize_conf=0.0,

/usr/local/lib/python3.10/dist-packages/intrinsic/pipeline.py in run_pipeline(models, img_arr, output_ordinal, resize_conf, base_size, maintain_size, linear, device, lstsq_p, inputs)
    121     # put all the outputs into a dictionary to return
    122     inv_shd = inv_shd.squeeze(0).detach().cpu().numpy()
--> 123     alb = alb.permute(1, 2, 0).detach().cpu().numpy()
    124 
    125     if maintain_size:

RuntimeError: permute(sparse_coo): number of dimensions in the tensor input does not match the length of the desired ordering of dimensions i.e. input.dim() = 4 is not equal to len(dims) = 3

It appears that there might be an extra dimension in the tensor alb which is not accounted for. Applying .squeeze(0) before the permute operation might resolve this issue by removing any singleton dimension at position 0.

running the image decomposition pipeline script, it assumes a 'final_weights.pt' file is saved somewhere, but I did not find it

Thanks,
Jonathan

Hi, congratulations on achieving so much. Is it possible to provide training and testing datasets? I want to observe the performance effect of editing through a large number of images.