COFW sample images segmented using our method.

Yuval Nirkin, Iacopo Masi, Anh Tuan Tran, Tal Hassner, and Gerard Medioni.

• New FCN model released for lower resolution images (300X300), trained without augmentations. Useful if you have limited GPU memory.
• A better performing and more efficient U-Net model will be released soon, including training and inference scripts using PyTorch.

This project provides an interface for face segmentation using Caffe with a fully convolutional neural network. The network was trained on IARPA Janus CS2 dataset (excluding subjects that are also in LFW) using a novel process for collecting ground truth face segmentations, involving our tool for semi-supervised Face video segmentation. Additional synthetic images were generated by augmenting hands from the EgoHands dataset, and augmenting 3D models of glasses and microphones.

If you find this code useful, please make sure to cite our paper in your work:

Yuval Nirkin, Iacopo Masi, Anh Tuan Tran, Tal Hassner, Gerard Medioni, "On Face Segmentation, Face Swapping, and Face Perception", IEEE Conference on Automatic Face and Gesture Recognition (FG), Xi'an, China on May 2018

Download the face_seg_fcn8s.zip or face_seg_fcn8s_300_no_aug.zip and extract to "data" in the installation directory.

build docker, then run it for inference.

--input argument can be a single image path or a directory path

Build the docker image

docker build -t face_segmentation:gpu .

Run the docker image

nvidia-docker run -it -v /path/to/face_segmentation:/workspace/face_segmentation face_segmentation:gpu --input ./input

If you want build cpu version of docker, first go to ./Dockerfile, comment first line and uncomment second line.

Build the docker image

docker build -t face_segmentation:cpu .

Run the docker image

docker run -it -v /path/to/face_segmentation:/workspace/face_segmentation face_segmentation:cpu --input ./input

Note: The segmentation model was trained by cropping the training images using find_face_landmarks. For best results crop the input images the same way, with crop resolution below 350 X 350. A Matlab function is available here.

Please cite our paper with the following bibtex if you use our face segmentation network:

@inproceedings{nirkin2018_faceswap,
      title={On Face Segmentation, Face Swapping, and Face Perception},
      booktitle = {IEEE Conference on Automatic Face and Gesture Recognition},
      author={Nirkin, Yuval and Masi, Iacopo and Tran, Anh Tuan and Hassner, Tal and Medioni, and G\'{e}rard Medioni},
      year={2018},
    }

• End-to-end, automatic face swapping pipeline, example application using out face segmentation method.
• Interactive system for fast face segmentation ground truth labeling, used to produce the training set for our deep face segmentation.
• CNN3DMM, estimation of 3D face shapes from single images.
• ResFace101, deep face recognition used in the paper to test face swapping capabilities.

Copyright 2017, Yuval Nirkin, Iacopo Masi, Anh Tuan Tran, Tal Hassner, and Gerard Medioni

The SOFTWARE provided in this page is provided "as is", without any guarantee made as to its suitability or fitness for any particular use. It may contain bugs, so use of this tool is at your own risk. We take no responsibility for any damage of any sort that may unintentionally be caused through its use.