Git Product home page Git Product logo

deepfillv2-pytorch's Introduction

Hi there ๐Ÿ‘‹

I'm passionate about machine/deep learning, signal processing, languages, applied mathematics and topics of lower order.

Connect with me:

https://linkedin.com/in/lars-nippert

deepfillv2-pytorch's People

Contributors

nipponjo avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

deepfillv2-pytorch's Issues

Is the test.ipynb up-to-date ?

Hello,

In your test.ipynb, you create a Generator with cnum_in=5, but you pass as input an image with 3 channels only.

Is it up-to-date with the current train.py code ? Should you also concatenate the ones and ones*mask to the image channel to pass as input, like in the training ?

Thanks.

Error during Core ML conversion

Hello!

First I want you to thank you because you did a really great job migrating this awesome DeepFillV2 to Python.
Also than you for implementing the GUI. @nipponjo

But I cannot make this work when converting it to Core ML.
Is it nearly done but when is about to finish I get this error:

Converting PyTorch Frontend ==> MIL Ops: 84%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‰ | 518/619 [00:00<00:00, 3312.17 ops/s]
Error during Core ML conversion: ('Op "706" (op_type: conv_transpose) Input weight must be const at compile time', 'weight', 'wi_center')

Do you have any idea why this is happening?
Thanks in advance

An error

image
Thank for your work. And I'm getting an error that I can't solve. I would like to ask you how to solve it and do you have relevant experience.
plea

Why use a grid during inference but not during training ?

Hello,

I trained the model and images generated looked nice, but during inference, it's pretty bad. I even try to infer using training data (just to test), and even then, it's pretty bad, so I'm debugging it, making sure images at inference and training get the same preprocessing.

I noticed that you used a grid during the Generator inference, but not during training, why is that ?

Thank you.

็Žฏๅขƒ็‰ˆๆœฌ

ไฝ ๅฅฝ๏ผŒๆ„Ÿ่ฐขไฝ ็š„ๅˆ†ไบซ๏ผŒๆƒณๅ’จ่ฏขไธ€ไธ‹ๅฏไปฅ่ฏดไธ€ไธ‹ไฝ ๅฝ“ๅ‰็Žฏๅขƒ็š„็‰ˆๆœฌๅ— ็‰นๅˆซๆ˜ฏtorch็‰ˆๆœฌ๏ผŒๆˆ‘่ฟ™่พนไธ็Ÿฅ้“ๆ˜ฏ็‰ˆๆœฌ้ซ˜ไบ†่ฟ˜ๆ˜ฏไฝŽไบ†๏ผŒๆŠฅ้”™

Grayscale conversion

Is there a straightforward way to adapt the model to grayscales (one channel) images?

Weird results

Hi
Thanks for the repo!
I am running your provided training configuration on the celeba dataset, with 128x128 images. I get the following results in tensorboard:
image

Is this expected?

how to generate test masks?

Hello,Thanks for your code! I would like to know how do you generate your test masks? I find that when i try to use pretrained models to test on my masked images,the results seems to be not very good.
image
image

Question about train.yaml

What do you mean by the value below that?
For 1280x720 images, I would like to ask you how to set the values below.

mask options

height: 256
width: 256
max_delta_height: 32
max_delta_width: 32
vertical_margin: 0
horizontal_margin: 0

training time and convergence

Thanks for the nice code! Could you given an estimate about the training time of the model on CeleBA-HQ and Places2 ? And how is the model's performances with a smaller dataset with, for instance, thousands of data?

Export to onnx

Hello @nipponjo , I've exported your work to onnx but it didn't work
This is script

import torch

pretrained = "pretrained/states_pt_places2.pth"

generator_state_dict = torch.load(pretrained)['G']

if 'stage1.conv1.conv.weight' in generator_state_dict.keys():
    from model.networks import Generator
else:
    from model.networks_tf import Generator  

# set up network
generator = Generator(cnum_in=5, cnum=48, return_flow=False)

generator.load_state_dict(generator_state_dict, strict=True)

img = torch.rand([5, 512, 512]).unsqueeze(0).cpu()
mask = torch.empty(img.shape[0], 1, img.shape[2], img.shape[3]).cpu()
data = (img, mask)

generator.cpu().eval()

dynamic_axes = None
dynamic_export = True
if dynamic_export:
    dynamic_axes = {
        'input': {
            0: 'batch',
            2: 'height',
            3: 'width'
        },
        'output': {
            0: 'batch',
            2: 'height',
            3: 'width'
        }
    }
with torch.no_grad():
    torch.onnx.export(
        generator,
        data,
        "output.onnx",
        input_names=['input'],
        output_names=['output'],
        export_params=True,
        keep_initializers_as_inputs=False,
        verbose=False,
        opset_version=12,
        dynamic_axes=dynamic_axes)
print(f'Successfully exported ONNX model')

End the error was from ONNX

Traceback (most recent call last):
  File "deepfillv2-pytorch/export2onnx.py", line 65, in <module>
    torch.onnx.export(
  File "/miniconda3/envs/py39/lib/python3.9/site-packages/torch/onnx/__init__.py", line 316, in export
    return utils.export(model, args, f, export_params, verbose, training,
  File "/miniconda3/envs/py39/lib/python3.9/site-packages/torch/onnx/utils.py", line 107, in export
    _export(model, args, f, export_params, verbose, training, input_names, output_names,
  File "/miniconda3/envs/py39/lib/python3.9/site-packages/torch/onnx/utils.py", line 724, in _export
    _model_to_graph(model, args, verbose, input_names,
  File "/miniconda3/envs/py39/lib/python3.9/site-packages/torch/onnx/utils.py", line 497, in _model_to_graph
    graph = _optimize_graph(graph, operator_export_type,
  File "/miniconda3/envs/py39/lib/python3.9/site-packages/torch/onnx/utils.py", line 216, in _optimize_graph
    graph = torch._C._jit_pass_onnx(graph, operator_export_type)
  File "/miniconda3/envs/py39/lib/python3.9/site-packages/torch/onnx/__init__.py", line 373, in _run_symbolic_function
    return utils._run_symbolic_function(*args, **kwargs)
  File "/miniconda3/envs/py39/lib/python3.9/site-packages/torch/onnx/utils.py", line 1032, in _run_symbolic_function
    return symbolic_fn(g, *inputs, **attrs)
  File "/miniconda3/envs/py39/lib/python3.9/site-packages/torch/onnx/symbolic_helper.py", line 172, in wrapper
    return fn(g, *args, **kwargs)
  File "/miniconda3/envs/py39/lib/python3.9/site-packages/torch/onnx/symbolic_opset9.py", line 1281, in _convolution
    raise RuntimeError("Unsupported: ONNX export of convolution for kernel "
RuntimeError: Unsupported: ONNX export of convolution for kernel of unknown shape.

I've google it but they said that because there is an operation in pytorch that onnx doesn't support. But I could not investigate what operation.
Could you help me please? Thank you

Questions in the Learning Process

Hello.

I'm trying to learn an image with an image size (720, 1280, 3), and I've confirmed that the initial ether is 1,000,000.

Do we usually proceed with the learning until the iter reaches 1,000,000?
Currently (mine)
iter=180000,
d_loss=1.0000
g_loss=-0.2722
ae_loss=0.0576
ae_loss1=0.0150
ae_loss2=0.0425.

I wonder if it is right to converge to 1 for d_loss.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.