Git Product home page Git Product logo

volux-gan's Introduction

VoLux-GAN: A Generative Model for 3D Face Synthesis with HDRI Relighting

Tensorflow implementation of the paper "VoLux-GAN: A Generative Model for 3D Face Synthesis with HDRI Relighting", SIGGRAPH 2022.

Setup

  • Python 3.6
  • TensorFlow 2.0
  • Tensorflow-Addon
  • gin-config
  • OpenCV
  • ImageIO
  • gdown
pip install -r requirements.txt  --user

1. Download pretrained model and example HDRI.

bash download.sh

2. Inference on generating 3D face for visualization.

Check out ./inference_demo.ipynb for toy examples.

Citation

If you find this code useful in your research, please cite:

@article{tan2022volux,
  title={VoLux-GAN: A Generative Model for 3D Face Synthesis with HDRI Relighting},
  author={Tan, Feitong and Fanello, Sean and Meka, Abhimitra and Orts-Escolano, Sergio and Tang, Danhang and Pandey, Rohit and Taylor, Jonathan and Tan, Ping and Zhang, Yinda},
  journal={ACM SIGGRAPH},
  year={2022}
}

volux-gan's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

volux-gan's Issues

The predicted front normals are zeros using the pretrained model

Thanks for your wonderful work. I'm trying to use your method and pretrained model to generate relighting images. The albedo image seems correct, but I find the predicted values of the front normals['full'] are all 0 with different input latent codes. Did I use the model in a wrong way, hope for your respones, Thank you very much.

Can you release your augmented dataset?

Hi!

Thanks for your awesome work of enabling HDRI lighting manipulation in the facial NeRF! The design is really inspiring.

I observe that the lighting manipulation relies on the augmented dataset, which is generated by the Total Relighting according to the paper.

Since the Total Relighting is not open-source, I wonder whether it is possible to release your augmented dataset for boosting relevant studies?

Thanks in advance!

How to rotate the environment map?

Hi, awesome work!

I have tried the notebook, but I wonder how to fix the pose while rotating the environment map?

I have tested:

  • Horizontally or Vertically shifting the HDRI map as arrays.
  • Rotating the HDRI map as arrays.
  • Explicitly passing the camera_orientation to the infer_relit.

However, none of them give me the results shown in the project page which features the obvious moving shadow.

Thus, I wonder what's the correct way to rotate the environment map? Thanks!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.