Git Product home page Git Product logo

ffhq-uv's Introduction

FFHQ-UV

FFHQ-UV: Normalized Facial UV-Texture Dataset for 3D Face Reconstruction

By Haoran Bai, Di Kang, Haoxian Zhang, Jinshan Pan, and Linchao Bao

teaser

FFHQ-UV is a large-scale facial UV-texture dataset that contains over 50,000 high-quality texture UV-maps with even illuminations, neutral expressions, and cleaned facial regions, which are desired characteristics for rendering realistic 3D face models under different lighting conditions.

The dataset is derived from a large-scale face image dataset namely FFHQ, with the help of our fully automatic and robust UV-texture production pipeline. Our pipeline utilizes the recent advances in StyleGAN-based facial image editing approaches to generate multi-view normalized face images from single-image inputs. An elaborated UV-texture extraction, correction, and completion procedure is then applied to produce high-quality UV-maps from the normalized face images. Compared with existing UV-texture datasets, our dataset has more diverse and higher-quality texture maps.

Updates

[2022-01-19] The source codes are available.
[2022-12-16] The OneDrive download link is available.
[2022-12-16] The AWS CloudFront download link is offline.
[2022-12-06] The script for generating face images from latent codes is available.
[2022-12-02] The latent codes and attributes of the multi-view normalized face images are available.
[2022-12-02] The FFHQ-UV-Interpolate dataset is available.
[2022-12-01] The FFHQ-UV dataset is available.
[2022-11-28] The paper is available here.

Dependencies

  • Linux + Anaconda
  • CUDA 10.0 + CUDNN 7.6.0
  • Python 3.7
  • dlib: pip install dlib
  • PyTorch 1.7.1: pip install torch==1.7.1 torchvision==0.8.2 torchaudio==0.7.2
  • TensorBoard: pip install tensorboard
  • TensorFlow 1.15.0: pip install tensorflow-gpu==1.15.0
  • MS Face API: pip install --upgrade azure-cognitiveservices-vision-face
  • Other packages: pip install tqdm scikit-image opencv-python pillow imageio matplotlib mxnet Ninja google-auth google-auth-oauthlib click requests pyspng imageio-ffmpeg==0.4.3 scikit-learn torchdiffeq==0.0.1 flask kornia==0.2.0 lmdb psutil dominate rtree
  • PyTorch3D and Nvdiffrast:
mkdir thirdparty
cd thirdparty
git clone https://github.com/facebookresearch/iopath
git clone https://github.com/facebookresearch/fvcore
git clone https://github.com/facebookresearch/pytorch3d
git clone https://github.com/NVlabs/nvdiffrast
conda install -c bottler nvidiacub
pip install -e iopath
pip install -e fvcore
pip install -e pytorch3d
pip install -e nvdiffrast

Dataset

Download

Dataset file structure

|--FFHQ-UV  
    |--ffhq-uv  # FFHQ-UV dataset
    |--ffhq-uv-face-latents  # The normalized face images' latent codes of FFHQ-UV dataset
    |--ffhq-uv-face-attributes  # The normalized face images' attributes of FFHQ-UV dataset
    |--ffhq-uv-interpolate    # FFHQ-UV-Interpolate dataset
    |--ffhq-uv-interpolate-face-latents  # The normalized face images' latent codes of FFHQ-UV-Interpolate dataset
    |--ffhq-uv-interpolate-face-attributes  # The normalized face images' attributes of FFHQ-UV-Interpolate dataset

FFHQ-UV-Interpolate dataset

  • FFHQ-UV-Interpolate is a variant of FFHQ-UV. Please refer to this readme for details.

The latent codes and attributes of the normalized face images

  • We provide the latent codes of the multi-view normalized face images which are used for extracting texture UV-maps. Along with the latent codes, we also provide the attributes (gender, age, beard) of each face, which are detected by Microsoft Face API.
  • One can generate face images from download latent codes by using the following script.
sh run_gen_face_from_latent.sh  # Please refer to this script for detailed configuration

Run source codes

Download checkpoints and topology assets

  • Please refer to this readme for details of checkpoints.
  • Please refer to this readme for details of topology assets.

Create facial UV-texture dataset

  • Prepare a directory of dataset project, which contains a "images" subfolder.
  • Put the original images into the "images" subfolder.
  • Modify the configuration and then run the following script to create the facial UV-texture dataset.
sh run_ffhq_uv_dataset.sh  # Please refer to this script for detailed configuration

RGB fitting

  • Put the input images into a folder.
  • Modify the configuration and then run the following script for fitting.
run_rgb_fitting.sh  # Please refer to this script for detailed configuration

Citation

@article{FFHQ-UV,
  title={FFHQ-UV: Normalized Facial UV-Texture Dataset for 3D Face Reconstruction},
  author={Bai, Haoran and Kang, Di and Zhang, Haoxian and Pan, Jinshan and Bao, Linchao},
  journal={arXiv preprint arXiv:2211.13874},
  year={2022}
}

ffhq-uv's People

Contributors

csbhr avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.