Git Product home page Git Product logo

oneshotclip's Introduction

OneshotCLIP

Official Source code of "One-Shot Adaptation of GAN in Just One CLIP" accepted to Transactions on Pattern Analysis and Machine Intelligence (TPAMI)

Environment

Pytorch 1.7.1, Python 3.6

$ conda create -n oneshotCLIP python=3.6
$ conda install --yes -c pytorch pytorch=1.7.1 torchvision cudatoolkit=11.0
$ pip install ftfy regex tqdm
$ conda install -c anaconda git
$ conda install -c conda-forge packaging
$ pip install git+https://github.com/openai/CLIP.git

Before training, please download the pre-trained models on large datasets:

LINK: FFHQ

Training

To train the model, run

python train_oneshot.py --exp exp1 --data_path $DATA_PATH$ --ckpt $SRC_MODEL_PATH$

$DATA_PATH$ is a directory for single-shot target image

$SRC_MODEL_PATH$ is a path for source domain pre-trained model.

Default: ./pretrained_model/stylegan2-ffhq-config-f.pt

--exp is for checkpoint directory name

For human face dataset training, download portrait dataset in LINK

Testing

To test the model with adapted generator,

python test_oneshot.py --exp exp1 --ckpt $TARGET_MODEL_PATH$ --ckpt_source $SOURCE_MODEL_PATH$

$TARGET_MODEL_PATH$ is path for adapted target domain model.

$SOURCE_MODEL_PATH$ is path for source domain model. Default: ./pretrained_model/stylegan2-ffhq-config-f.pt

For testing, we provide several adapted models

LINK

Testing for real images

For testing on real images, we provide demo on Google Colab Open In Colab.

oneshotclip's People

Contributors

cyclomon avatar submission6378 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

Forkers

haizhu12

oneshotclip's Issues

add web demo/model to Huggingface

Hi, would you be interested in adding OneshotCLIP to Hugging Face? The Hub offers free hosting, and it would make your work more accessible and visible to the rest of the ML community.

Example from other organizations:
Keras: https://huggingface.co/keras-io
Microsoft: https://huggingface.co/microsoft
Facebook: https://huggingface.co/facebook

Example spaces with repos:
github: https://github.com/salesforce/BLIP
Spaces: https://huggingface.co/spaces/akhaliq/BLIP

github: https://github.com/facebookresearch/omnivore
Spaces: https://huggingface.co/spaces/akhaliq/omnivore

and here are guides for adding spaces/models/datasets to your org

How to add a Space: https://huggingface.co/blog/gradio-spaces
how to add models: https://huggingface.co/docs/hub/adding-a-model
uploading a dataset: https://huggingface.co/docs/datasets/upload_dataset.html

Please let us know if you would be interested and if you have any questions, we can also help with the technical implementation.

Took norm twice while calculating Patch loss.

Hi mate,
Thanks for publicizing this exellent work. I skimmed through the code and found that you took norm twice while calculating the patch level contrastive loss here https://github.com/cyclomon/OneshotCLIP/blob/main/train_oneshot.py#L385 . Once right after calculating clip features here https://github.com/cyclomon/OneshotCLIP/blob/main/train_oneshot.py#L380 and then here https://github.com/cyclomon/OneshotCLIP/blob/main/losses.py#L59 . I am lil confused wether its experimental or any logic behind it? i also checked the MoCo loss which only use L_2 norm. I will appreciate your response. Thanks!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.