webtoon / matteformer Goto Github PK

Invalid issue

Thanks for your work. I have a picture, can you tell me how to test the matting in your code？

Can any friends share the training log so that we can use it as a reference?

Is there a way to convert this model to onnxruntime? I have been trying with no luck so far, so many errors with Wh, Ww = (H + 1) // 2, (W + 1) // 2 and lots of trace warnings TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect.

Thanks for your work！Could you please share a pre-trained model? Thank you very much.

Hello, author. First of all, thank you very much for providing such a good code. I would like to ask, is the number of background pictures you used in training 22W of all the pictures in coco data set? Or only part of it? Then, the initial learning rate used in the paper is 1E-4, and the initial learning rate set in the code is 1E-3. Would you like to confirm which one is more appropriate?

Hi, Thanks for your awesome work.
It makes me confused about trimap. If I want to generate a trimap but test image doesn't exist in compostion1k dataset, I can use other models for coarse human segmentation. However, it will need more computing resources and more time. Could I use this model to generate trimap directly?

I am having difficulty getting the training code to run on Windows.
I think I'm having issues due to the use of torch.distributed.
From the pytorch documentation, it looks like Windows does not support the NCCL backend.
Since I only have 1 GPU, is it possible to bypass the distributed code?

Hi Author @pgt4861,
Thanks for your work. I am a big fan of your paper and was really impressed by how well it uses the trimap information using prior tokens.
I was experimenting with the code but was unable to reproduce the results in terms of quality. Below are some of the examples that I ran.
In each row, from left to right, we have the original image, the trimap, the alpha mask predicted by matteformer and the final matted image.
As you can see, the output is not as good as the model can perform. Specifically in the unknown regions of the trimap, like around the fingers of the boy, the feet of the horse and the neck of the table lamp.
Can you please guide me how to improve these results?
P.S: I am generating these trimaps automatically from a segmentation mask using the Euclidean distance transform.

Hi,

A great work.
My question is , why shall we only perform rotation augmentation on large fg images ( width/height > 1024 pix) during training ?

https://github.com/webtoon/matteformer/blob/master/dataloader/data_generator.py

Hello author,
I want follow your experiments, but I my current single gpu is 8G, when in the test stage after some train iterations, my gpu memory usage is 7g+ when I just set the batch size = 1. (your paper batch size setting is 20)

I would like to know what are the gpu requirements for your experiments.

For setting the refine width, how did you land on self_refine_width1 = 30 and self_refine_width2 = 15 for the configuration? Was it a lot of trial and error with these values working well as an average, or is there another reason behind the 30 and 15? Did you observe any peculiar behavior with the network when these values were increased/decreased?

Does the image needs to be 512x512? Or can we input higher res images?

I used to be able to train model, but lately I can't. I made a mistake about the parameter local_rank. As follows:

I didn't edit anything in the main.py file, I tried to set os.environ['LOCAL_RANK']=0 or 1 but it wasn't works.
I run this model on Google Colab, so my GPU device is only one. But I have run it successfully before :(

Can you help me to solve this problems?

Hello author,
I want follow your experiments, but I my current single gpu is 8G, when in the test stage after some train iterations, my gpu memory usage is 7g+ when I just set the batch size = 1. (your paper batch size setting is 20)

I would like to know what are the gpu requirements for your experiments.

Speed of inference with V100 lower than GTX2080 for image size 128*128?
Ubuntu 18.04
cuda 11.1
cudnn 7
Consider for avoid CPU limitations, I split preprocess and upload part from model inference. But in GTX 2080 ti inference is done in 40 ms and in V100 time is 79 ms.
No CPU memory limit in the two systems.

Hi there! Is there any way to run matteformer on video? I suppose it can be done frame by frame, but do I have to generate a trimap per frame?

What happens if there are multiple objects?

Hi,

First thank you for this very interesting work!

I am trying to use it on my own images
The objective is to separate the object from the table on which it is laying.
I also have a way to identify most of the object from the geometry, so I make it the white area of the trimap
I have a way of identifying the table with some thickness from the geometry so I make it part of the gray area of the trimap
Finally I know that all the rest is background set to the black area of the trimap

When applying this approach to the sample image below, the results are disappointing:

Is my gray area strategy correct?
Are there any finetuning settings I could use?

Any advice from you would be greatly appreciated!

Regards,

Pierre