GFLOPs of PyNET about pynet-pytorch HOT 9 CLOSED

Hi @nieyan,

PyNET is a relatively big model, therefore it won't be possible to run it on embedded devices in real time. However, if the resolution of the images is quite small (less than 3MP), then it should be possible to optimize it (e.g., by disabling some layers / levels) for mobile devices.

@scoobym, do you have this OOM exception during the training or testing the model?

from pynet-pytorch.

Thanks for your great work! Could PyNET deploy in embedding device's cpu with real time FPS ? I am curious about PyNET's resource comsume.

I have the same problem, so i tried it on my gpu(rtx 2080ti), found that the model was too big to fit a single GPU. RuntimeError: CUDA out of memory. Tried to allocate 130.12 MiB (GPU 0; 10.76 GiB total capacity; 9.57 GiB already allocated; 7.12 MiB free; 190.92 MiB cached). So I'm afraid the model cannot run in real time on an embedded device.

from pynet-pytorch.

@scoobym I have a similar error.I tried to use 6 TITAN XP（each has 12G RAM）to train the model，and OOM appears when level = 1. But when level = 0, the RAM is enough.
Even I change the batchsize to 1 when level = 1, OOM appears still.

But I use the tensorflow version of PyNET, there's no OOM error, if you want to train by your self you can try the tensorflow version of PyNET too.

A another problem is that the whole training process took about 3-4days (I skip level 1 just training level 0 after level 2, and other settings like batchsize and epochs are same to the description at README.md). It is really slow.

from pynet-pytorch.

Hi @scoobym,

If you are training the model, then the size of the input patches should be 224x224x4:

device = torch.device("cuda")
generator = PyNET(level=0, instance_norm=True, instance_norm_level_1=True).to(device)

input = torch.rand([1, 4, 224, 224]).float().to(device)
output = generator(input)

If you are validating the model, then you should use torch.no_grad() to avoid the computation of the gradients:

device = torch.device("cuda")
generator = PyNET(level=0, instance_norm=True, instance_norm_level_1=True).to(device)

input = torch.rand([1, 4, 448, 448]).float().to(device)

generator.eval()
with torch.no_grad():
    output = generator(input)

Please note that I was able to run the above two examples on one GPU with only 3GB of RAM, therefore you shouldn't have any issues with Nvidia 2080 Ti.

@Jam-G, the mentioned OOM issue is happening during the validation process. To avoid it, just reduce the resolution of the validation images from 1280x1280 pixels to e.g., 960x960.

from pynet-pytorch.

@aiff22 I just run the code below and find that when level = 0 or 1, oom appears. Probably because the memory of 2080ti is too small, only 11G.
generator = PyNET(level=0, instance_norm=True, instance_norm_level_1=True).to(device) a = torch.rand([1,3,448,448]).float().to(device) b = generator(a)

from pynet-pytorch.

@aiff22 Thank you! I want to train the model on my own data, so I changed the channel to 3.

from pynet-pytorch.

@aiff22 thanks！I have a try.

from pynet-pytorch.

@aiff22
I have a similar problem.
When I run inference using level 0 and level 1, it will remind me that OOM.
I test it in 4*1080ti.

from pynet-pytorch.

@XGBoost, please look at the following answer.

from pynet-pytorch.