This repository is the official implementation of Joint Physical-Digital Facial Attack Detection Via Simulating Spoofing Clues

The overview pipeline of our method. We propose Simulated Physical Spoofing Clues augmentation (SPSC), which augments live samples into simulated physical attack samples for training within protocols 1 and 2.1. Concurrently, we present Simulated Digital Spoofing Clues augmentation (SDSC), converting live samples into simulated digital attack samples, tailored for training under protocols 1 and 2.2.

1. employ ColorJitter to simulate the spoofing clues of print attacks
2. use moire pattern augmentation to simulate the spoofing clues of replay attacks
3. SPSC consists of ColorJitter and moire pattern augmentation
4. introduce SPSC to simulate the spoofing clues of digital forgery
5. attempt to use GaussNoise or gradient-based noise to simulate the spoofing clues of adversarial attacks, but do not work

Please make a copy of the original data and place it in the following format：

1. copy data and get txt files

cp -r xxx/phase1/* cvpr2024/data/
cp -r xxx/phase2/* cvpr2024/data/
cd cvpr2024/data
cat p1/dev.txt p1/test.txt > p1/dev_test.txt
cat p2.1/dev.txt p2.1/test.txt > p2.1/dev_test.txt
cat p2.2/dev.txt p2.2/test.txt > p2.2/dev_test.txt

2. get train_dev_label.txt

python data_preprocess/merge_dev_train_data.py # please modify the base path
# p1: merge all dev data to train
# p2.1: only merge dev live data to train
# p2.2: only merge dev live data to train

3. final data format:

base_path = "xxx/cvpr2024"

|----cvpr2024/data
  |----p1
    |----train
    |----dev
    |----test
    |----train_live_mask
    |----train_label.txt
    |----dev_label.txt
    |----train_dev_label.txt
    |----dev_test.txt
  |----p2.1
    |----train
    |----dev
    |----test
    |----train_live_mask
    |----train_label.txt
    |----dev_label.txt
    |----train_dev_label.txt
    |----dev_test.txt
  |----p2.2
    |----train
    |----dev
    |----test
    |----train_live_mask
    |----train_label.txt
    |----dev_label.txt
    |----train_dev_label.txt
    |----dev_test.txt

pip install -r requirements.txt

if image width >= 700 and image height >= 700, we will detect face from the image and expand 20 pixel crop face from the image. If no face is detected in the image, we will center crop a 500*500 bbox from the image which implements this logic in dataset.py where the training data is loaded. we need pip install insightface and use insightface to detect face.It will download the model by default to detect faces.
Please modify the base path and run detect_face.py.The crop_face will overwrite the original image.

cd data_preprocess
python detect_face.py

face-parsing github
download face-parsing model from 79999_iter.pth
Please modify root_dir and model_path.

cd data_preprocess/face_parsing
bash generate_mask.sh

download pretrain model: resnet50

• Please use 1*A100(80G) for training, only modify the dataset base path, do not modify other parameters in training.
• We fixed the random seed to ensure reproducible results. Modifying other training parameters will cause fluctuations in the final results.
• Training only takes 1 hour for each protocol.
• Inference only takes 1 minute for each protocol.

Train p1 protocol:

bash scripts/train_p1.sh

Test: select the 200th epoch model weight

bash scripts/test_p1.sh

Train p2.1 protocol:

bash scripts/train_p21.sh

Test: select the 200th epoch model weight

bash scripts/test_p21.sh

Train p2.2 protocol:

bash scripts/train_p22.sh

Test: select the 200th epoch model weight

bash scripts/test_p22.sh

If you find this repository useful for your research, please use the following entry.

@article{he2024joint,
  title={Joint Physical-Digital Facial Attack Detection Via Simulating Spoofing Clues},
  author={He, Xianhua and Liang, Dashuang and Yang, Song and Hao, Zhanlong and Ma, Hui and Mao, Binjie and Li, Xi and Wang, Yao and Yan, Pengfei and Liu, Ajian},
  journal={arXiv preprint arXiv:2404.08450},
  year={2024}
}