This repository is a fork of the original face_de_mask with changes to make it compatible with running on Google Colab.

1. Upload the necessary files to Google Drive

   1. Download the folder from this link
   2. Upload the folder to your Google Drive (recommended to put it in MyDrive/)

2. Run Colab

   

• Create a fully integrated notebook, including the processing in Deep3DFaceReconstruction-Pytorch and 3DFace repositories necessary to perform face_de_mask
• Fork and modify face_de_mask to work in the Colab environment
• Fork and modify Deep3DFaceReconstruction-Pytorch to work in the Colab environment

The following is the original README

Evaluation code for : Non-Deterministic Face Mask Removal Based On 3D Priors

This paper presents a novel image inpainting framework for face mask removal. Although current methods have demonstrated their impressive ability in recovering damaged face images, they suffer from two main problems: the dependence on manually labeled missing regions and the deterministic result corresponding to each input. The proposed approach tackles these problems by integrating a multi-task 3D face reconstruction module with a face inpainting module. Given a masked face image, the former predicts a 3DMM-based reconstructed face together with a binary occlusion map, providing dense geometrical and textural priors that greatly facilitate the inpainting task of the latter. By gradually controlling the 3D shape parameters, our method generates high-quality dynamic inpainting results with different expressions and mouth movements. Qualitative and quantitative experiments verify the effectiveness of the proposed method.

• Python >= 3.7
• Pytorch >= 1.6
• cv2
• numpy
• PyTorch3D : https://pytorch3d.org/

https://drive.google.com/drive/folders/1-th-qJQGGgzWQF2qrAGr8njJ9EBWHGIR?usp=sharing

• Create BFM related files following the instructions of this project
• Download the pre-trained models and put them in the ckpts folder.
• Run test.py