Git Product home page Git Product logo

draggan's Introduction

Drag Your GAN: Interactive Point-based Manipulation on the Generative Image Manifold

Xingang Pan · Ayush Tewari · Thomas Leimkühler · Lingjie Liu · Abhimitra Meka · Christian Theobalt

SIGGRAPH 2023 Conference Proceedings


PyTorch Twitter Paper PDF Project Page Open In Colab

Web Demos

Open in OpenXLab

Huggingface

Requirements

If you have CUDA graphic card, please follow the requirements of NVlabs/stylegan3.

The usual installation steps involve the following commands, they should set up the correct CUDA version and all the python packages

conda env create -f environment.yml
conda activate stylegan3

Then install the additional requirements

pip install -r requirements.txt

Otherwise (for GPU acceleration on MacOS with Silicon Mac M1/M2, or just CPU) try the following:

cat environment.yml | \
  grep -v -E 'nvidia|cuda' > environment-no-nvidia.yml && \
    conda env create -f environment-no-nvidia.yml
conda activate stylegan3

# On MacOS
export PYTORCH_ENABLE_MPS_FALLBACK=1

Run Gradio visualizer in Docker

Provided docker image is based on NGC PyTorch repository. To quickly try out visualizer in Docker, run the following:

# before you build the docker container, make sure you have cloned this repo, and downloaded the pretrained model by `python scripts/download_model.py`.
docker build . -t draggan:latest  
docker run -p 7860:7860 -v "$PWD":/workspace/src -it draggan:latest bash
# (Use GPU)if you want to utilize your Nvidia gpu to accelerate in docker, please add command tag `--gpus all`, like:
#   docker run --gpus all  -p 7860:7860 -v "$PWD":/workspace/src -it draggan:latest bash

cd src && python visualizer_drag_gradio.py --listen

Now you can open a shared link from Gradio (printed in the terminal console).
Beware the Docker image takes about 25GB of disk space!

Download pre-trained StyleGAN2 weights

To download pre-trained weights, simply run:

python scripts/download_model.py

If you want to try StyleGAN-Human and the Landscapes HQ (LHQ) dataset, please download weights from these links: StyleGAN-Human, LHQ, and put them under ./checkpoints.

Feel free to try other pretrained StyleGAN.

Run DragGAN GUI

To start the DragGAN GUI, simply run:

sh scripts/gui.sh

If you are using windows, you can run:

.\scripts\gui.bat

This GUI supports editing GAN-generated images. To edit a real image, you need to first perform GAN inversion using tools like PTI. Then load the new latent code and model weights to the GUI.

You can run DragGAN Gradio demo as well, this is universal for both windows and linux:

python visualizer_drag_gradio.py

Acknowledgement

This code is developed based on StyleGAN3. Part of the code is borrowed from StyleGAN-Human.

(cheers to the community as well)

License

The code related to the DragGAN algorithm is licensed under CC-BY-NC. However, most of this project are available under a separate license terms: all codes used or modified from StyleGAN3 is under the Nvidia Source Code License.

Any form of use and derivative of this code must preserve the watermarking functionality showing "AI Generated".

BibTeX

@inproceedings{pan2023draggan,
    title={Drag Your GAN: Interactive Point-based Manipulation on the Generative Image Manifold},
    author={Pan, Xingang and Tewari, Ayush, and Leimk{\"u}hler, Thomas and Liu, Lingjie and Meka, Abhimitra and Theobalt, Christian},
    booktitle = {ACM SIGGRAPH 2023 Conference Proceedings},
    year={2023}
}

draggan's People

Contributors

akashad98 avatar ariym avatar bulutthecat avatar cobanov avatar idadzie avatar kevinunger avatar koyuyesil avatar leoxing1996 avatar liusanchuan avatar merryhunter avatar ochafik avatar pdillis avatar silent790-2 avatar tzuifurther avatar xingangpan avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

draggan's Issues

RuntimeError: Tried to instantiate dummy base class Event

实在不太清楚是那有问题了,大会帮忙吧,困死了

(base) E:\DragGAN>pip list
Package                 Version
----------------------- ------------
boltons                 23.0.0
brotlipy                0.7.0
certifi                 2023.5.7
cffi                    1.15.1
charset-normalizer      2.0.4
colorama                0.4.6
conda                   23.5.0
conda-content-trust     0.1.3
conda-package-handling  2.1.0
conda_package_streaming 0.8.0
cryptography            39.0.1
filelock                3.12.2
idna                    3.4
Jinja2                  3.1.2
jsonpatch               1.32
jsonpointer             2.1
MarkupSafe              2.1.3
menuinst                1.4.19
mpmath                  1.3.0
networkx                3.1
numpy                   1.25.0
packaging               23.0
Pillow                  9.5.0
pip                     23.1.2
pluggy                  1.0.0
pycosat                 0.6.4
pycparser               2.21
pyOpenSSL               23.0.0
PySocks                 1.7.1
requests                2.29.0
ruamel.yaml             0.17.21
ruamel.yaml.clib        0.2.6
setuptools              65.6.3
six                     1.16.0
sympy                   1.12
toolz                   0.12.0
torch                   2.0.1+cu118
torchaudio              2.0.2+cu118
torchvision             0.15.2+cu118
tqdm                    4.65.0
typing_extensions       4.6.3
urllib3                 1.26.16
wheel                   0.37.1
win-inet-pton           1.1.0
wincertstore            0.2
zstandard               0.19.0
$ sh scripts/gui.sh
No CUDA runtime is found, using CUDA_HOME='C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.1'
Traceback (most recent call last):
  File "E:\DragGAN\visualizer_drag.py", line 402, in <module>
    main()
  File "C:\Users\***\AppData\Local\Programs\Python\Python310\lib\site-packages\click\core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "C:\Users\***\AppData\Local\Programs\Python\Python310\lib\site-packages\click\core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "C:\Users\***\AppData\Local\Programs\Python\Python310\lib\site-packages\click\core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "C:\Users\***\AppData\Local\Programs\Python\Python310\lib\site-packages\click\core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "E:\DragGAN\visualizer_drag.py", line 396, in main
    viz.draw_frame()
  File "E:\DragGAN\visualizer_drag.py", line 168, in draw_frame
    self._async_renderer.set_args(**self.args)
  File "E:\DragGAN\visualizer_drag.py", line 294, in set_args
    self._set_args_sync(**args)
  File "E:\DragGAN\visualizer_drag.py", line 311, in _set_args_sync
    self._renderer_obj = renderer.Renderer()
  File "E:\DragGAN\viz\renderer.py", line 78, in __init__
    self._start_event   = torch.cuda.Event(enable_timing=True)
  File "C:\Users\***\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\cuda\streams.py", line 164, in __new__
    return super(Event, cls).__new__(
  File "C:\Users\***\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\cuda\_utils.py", line 46, in err_fn
    raise RuntimeError(
RuntimeError: Tried to instantiate dummy base class Event

cool

Look really cool what software do you use

期待

看起来它适合做photoshop的插件

GLError

OpenGL.error.GLError: GLError(
err = 1282,
description = b'invalid operation',
baseOperation = glGetUniformLocation,
cArguments = (1, b'Texture\x00'),
result = -1
)

Run on WSL2 windows11

牛的

牛的大佬,等着看你们发布

AttributeError: module 'legacy' has no attribute 'load_network_pkl'

Running on Windows11 by Git Bash

Traceback (most recent call last):
File "C:\Users\adm\AppData\Local\Programs\Python\Python310\lib\site-packages\viz\renderer.py", line 131, in get_network
data = legacy.load_network_pkl(f)
AttributeError: module 'legacy' has no attribute 'load_network_pkl'

NameError: name 'inverse_image' is not defined

When trying to import a custom image file, I always receive the error message:

File "PathtoDragGAN\draggan\web.py", line 183, in on_image_change
    result = inverse_image(
NameError: name 'inverse_image' is not defined

Demo page!

Looking forward to publishing relevant operable display webpages, similar to SAM.
Again, it's a great job! 🤩

How to download and use Drag Gan

I am a newcomer. I learned about Drag Gan through news reports, so I registered and entered this community. I would like to ask everyone how to download and use this software?

Error: ModuleNotFoundError: No module named 'imgui'

I created a new coda env with Python 3.11 and installed the models, but when I execute the GUI, I encounter this error:

➜ conda create --name dragcan python=3.11

➜ pip install -r requirements.txt                          

➜ sh scripts/gui.sh  
Traceback (most recent call last):
  File "/home/dbugger/projects/ai/DragGAN/visualizer_drag.py", line 15, in <module>
    import imgui
ModuleNotFoundError: No module named 'imgui'
                

glfw error

/home/dell/anaconda3/envs/DragGAN/lib/python3.8/site-packages/glfw/init.py:834: GLFWError: (65544) b'X11: Failed to open display '
warnings.warn(message, GLFWError)
/home/dell/anaconda3/envs/DragGAN/lib/python3.8/site-packages/glfw/init.py:834: GLFWError: (65537) b'The GLFW library is not initialized'
warnings.warn(message, GLFWError)
python: /builds/florianrhiem/pyGLFW/glfw-3.3.4/src/input.c:832: glfwSetKeyCallback: Assertion `window != ((void *)0)' failed.
Aborted (core dumped)

AMD GPU

I only have an AMD GPU instead of a Nvidia GPU. Can I still use DragGAN?

GLFWError

(stylegan3) [root@aigc DragGAN]# ./gui.sh
/root/miniconda3/envs/stylegan3/lib/python3.9/site-packages/glfw/init.py:834: GLFWError: (65544) b'X11: The DISPLAY environment variable is missing'
warnings.warn(message, GLFWError)
/root/miniconda3/envs/stylegan3/lib/python3.9/site-packages/glfw/init.py:834: GLFWError: (65537) b'The GLFW library is not initialized'
warnings.warn(message, GLFWError)
python: /builds/florianrhiem/pyGLFW/glfw-3.3.4/src/input.c:832: glfwSetKeyCallback: Assertion `window != ((void *)0)' failed.
./gui.sh: line 11: 27555 Aborted python visualizer_drag.py checkpoints/stylegan2_lions_512_pytorch.pkl checkpoints/stylegan2-ffhq-512x512.pkl checkpoints/stylegan2-afhqcat-512x512.pkl checkpoints/stylegan2-car-config-f.pkl checkpoints/stylegan2_dogs_1024_pytorch.pkl checkpoints/stylegan2_horses_256_pytorch.pkl checkpoints/stylegan2-cat-config-f.pkl checkpoints/stylegan2_elephants_512_pytorch.pkl checkpoints/stylegan_human_v2_512.pkl checkpoints/stylegan2-lhq-256x256.pkl

Question: Is this standalone?

Wondering if this is standalone or if i have to use automatic1111. I have a tool I've built from scratch and would love to include this in it. I'll contribute back to the project as well. Amazing work~

Error 400/403 happened while running scripts/download_model.sh

drimoon@DESKTOP-1GO1HJO:/mnt/g/DragGAN$ sh scripts/download_model.sh
mkdir: cannot create directory ‘checkpoints\r’: File exists
: not foundnload_model.sh: 3:
--2023-06-26 12:54:13--  https://storage.googleapis.com/self-distilled-stylegan/lions_512_pytorch.pkl%0D
Resolving storage.googleapis.com (storage.googleapis.com)... 172.217.160.112, 142.251.43.16, 142.251.42.240, ...
Connecting to storage.googleapis.com (storage.googleapis.com)|172.217.160.112|:443... connected.
HTTP request sent, awaiting response... 400 Bad Request
2023-06-26 12:54:14 ERROR 400: Bad Request.

mv: cannot stat 'lions_512_pytorch.pkl': No such file or directory
: not foundnload_model.sh: 6:
--2023-06-26 12:54:14--  https://storage.googleapis.com/self-distilled-stylegan/dogs_1024_pytorch.pkl%0D
Resolving storage.googleapis.com (storage.googleapis.com)... 172.217.160.112, 142.251.43.16, 142.251.42.240, ...
Connecting to storage.googleapis.com (storage.googleapis.com)|172.217.160.112|:443... connected.
HTTP request sent, awaiting response... 400 Bad Request
2023-06-26 12:54:15 ERROR 400: Bad Request.

mv: cannot stat 'dogs_1024_pytorch.pkl': No such file or directory
: not foundnload_model.sh: 9:
--2023-06-26 12:54:15--  https://storage.googleapis.com/self-distilled-stylegan/horses_256_pytorch.pkl%0D
Resolving storage.googleapis.com (storage.googleapis.com)... 172.217.160.112, 142.251.43.16, 142.251.42.240, ...
Connecting to storage.googleapis.com (storage.googleapis.com)|172.217.160.112|:443... connected.
HTTP request sent, awaiting response... 400 Bad Request
2023-06-26 12:54:16 ERROR 400: Bad Request.

mv: cannot stat 'horses_256_pytorch.pkl': No such file or directory
: not foundnload_model.sh: 12:
--2023-06-26 12:54:16--  https://storage.googleapis.com/self-distilled-stylegan/elephants_512_pytorch.pkl%0D
Resolving storage.googleapis.com (storage.googleapis.com)... 172.217.160.112, 142.251.43.16, 142.251.42.240, ...
Connecting to storage.googleapis.com (storage.googleapis.com)|172.217.160.112|:443... connected.
HTTP request sent, awaiting response... 400 Bad Request
2023-06-26 12:54:16 ERROR 400: Bad Request.

mv: cannot stat 'elephants_512_pytorch.pkl': No such file or directory
: not foundnload_model.sh: 15:
--2023-06-26 12:54:16--  https://api.ngc.nvidia.com/v2/models/nvidia/research/stylegan2/versions/1/files/stylegan2-ffhq-512x512.pkl%0D
Resolving api.ngc.nvidia.com (api.ngc.nvidia.com)... 35.160.252.113, 52.12.123.121
Connecting to api.ngc.nvidia.com (api.ngc.nvidia.com)|35.160.252.113|:443... connected.
HTTP request sent, awaiting response... 400
2023-06-26 12:54:17 ERROR 400: (no description).

--2023-06-26 12:54:17--  https://api.ngc.nvidia.com/v2/models/nvidia/research/stylegan2/versions/1/files/stylegan2-afhqcat-512x512.pkl%0D
Resolving api.ngc.nvidia.com (api.ngc.nvidia.com)... 35.160.252.113, 52.12.123.121
Connecting to api.ngc.nvidia.com (api.ngc.nvidia.com)|35.160.252.113|:443... connected.
HTTP request sent, awaiting response... 400
2023-06-26 12:54:18 ERROR 400: (no description).

--2023-06-26 12:54:18--  http://d36zk2xti64re0.cloudfront.net/stylegan2/networks/stylegan2-car-config-f.pkl%0D
Resolving d36zk2xti64re0.cloudfront.net (d36zk2xti64re0.cloudfront.net)... 99.84.224.57, 99.84.224.91, 99.84.224.202, ...
Connecting to d36zk2xti64re0.cloudfront.net (d36zk2xti64re0.cloudfront.net)|99.84.224.57|:80... connected.
HTTP request sent, awaiting response... 403 Forbidden
2023-06-26 12:54:19 ERROR 403: Forbidden.

--2023-06-26 12:54:19--  http://d36zk2xti64re0.cloudfront.net/stylegan2/networks/stylegan2-cat-config-f.pkl%0D
Resolving d36zk2xti64re0.cloudfront.net (d36zk2xti64re0.cloudfront.net)... 99.84.224.57, 99.84.224.91, 99.84.224.202, ...
Connecting to d36zk2xti64re0.cloudfront.net (d36zk2xti64re0.cloudfront.net)|99.84.224.57|:80... connected.
HTTP request sent, awaiting response... 403 Forbidden
2023-06-26 12:54:20 ERROR 403: Forbidden.

[BUG] OpenGL error ,try pip install PyOpenGL

I try it: export MESA_GL_VERSION_OVERRIDE=3.3. But the following error still occurred, My computer is configured with MAC 2.4 GHz 8-core Intel Core i9

raise self._errorClass(

OpenGL.error.GLError: GLError(
err = 1282,
description = b'invalid operation',
baseOperation = glGetUniformLocation,
cArguments = (1, b'Texture\x00'),
result = -1
)

Attribute Error: image

Traceback (most recent call last):
File "C:\Users\alime\Downloads\DragGAN-main1\DragGAN-main\dnnlib\util.py", line 45, in getattr
return self[name]
KeyError: 'image'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Users\alime\Downloads\DragGAN-main1\DragGAN-main\visualizer_drag.py", line 402, in
main()
File "C:\Users\alime\AppData\Local\Programs\Python\Python310\lib\site-packages\click\core.py", line 1130, in call
return self.main(*args, **kwargs)
File "C:\Users\alime\AppData\Local\Programs\Python\Python310\lib\site-packages\click\core.py", line 1055, in main
rv = self.invoke(ctx)
File "C:\Users\alime\AppData\Local\Programs\Python\Python310\lib\site-packages\click\core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "C:\Users\alime\AppData\Local\Programs\Python\Python310\lib\site-packages\click\core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "C:\Users\alime\Downloads\DragGAN-main1\DragGAN-main\visualizer_drag.py", line 396, in main
viz.draw_frame()
File "C:\Users\alime\Downloads\DragGAN-main1\DragGAN-main\visualizer_drag.py", line 181, in draw_frame
h, w, _ = self.result.image.shape
File "C:\Users\alime\Downloads\DragGAN-main1\DragGAN-main\dnnlib\util.py", line 47, in getattr
raise AttributeError(name)
AttributeError: image

Trying to run on Mac m1

My Mac configs:
os: Mac OS 13.4 (22F66)
cpu: Mac m1
ram: 8GB

after running through
sh scripts/gui.sh

I am getting segment fault:
scripts/gui.sh: line 2: 77912 Segmentation fault: 11 python visualizer_drag.py checkpoints/stylegan2_lions_512_pytorch.pkl

need any more information would be happy to share.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.