This repository contains the code for the paper Detecting Adversarial Samples from Artifacts (Feinman et al. 2017).

This code repository requires Keras > 2.0 and TensorFlow. Keras must be configured to use TensorFlow backend. A full list of requirements can be found in requirements.txt. On UNIX machines, after cloning this repository, it is recommended that you add the path to the repository to your PYTHONPATH environment variable to enable imports from any folder:

export PYTHONPATH="/path/to/detecting-adversarial-samples:$PYTHONPATH"

The source code is located in the src/ subfolder, and scripts that users will run to perform various steps are located in the scripts/ subfolder. An empty subfolder, data/, is included for storing trained models and adversarial sample arrays. Instructions for running the code are below.

All of the scripts for running the various parts of the code are located in the scripts/ subfolder.

To train a new model for a particular data set, simply run

python train_model.py -d=<dataset> -e=<nb_epochs>

where <dataset> is one of either 'mnist,' 'cifar' or 'svhn,' and <nb_epochs> is an integer indicating the number of epochs to train for. We recommend using 10 epochs for MNIST, and 60 for each of CIFAR and SVHN. For example, to train the MNIST model for 10 epochs, we would run

python train_model.py -d=mnist -e=10

The model will be trained and saved into the data/ subfolder and named model_<dataset>.h5. An optional batch size parameter is also available, specified with -b=<batch_size>. The default training batch size is 128.

To craft adversarial samples for a particular data set, you must first train the model for that data set (details above). Then, simply run

python craft_adv_samples.py -d=<dataset> -a=<attack>

where <dataset> is the same as above and <attack> is one of either 'fgsm,' 'jsma,' 'bim-a,' 'bim-b' or 'all,' indicating which method to use to craft adversarial samples. For example, to craft adversarial samples for the MNIST model using FGSM, we would run

python craft_adv_samples.py -d=mnist -a=fgsm

If 'all' is chosen (the default), all types of adversarial samples will be generated. Arrays holding the adversarial samples are stored in the data/ subfolder and named Adv_<dataset>_<attack>.npy. An optional batch size parameter for evaluating adversarial samples is again provided (-b=<batch_size>). The default is 256.

To run the detection script, you must first train the model and craft adversarial samples for each data set you would like to use (details above). Then, simply run

python detect_adv_samples.py -d=<dataset> -a=<attack>

where <dataset> and <attack> are the same as described above. An optional batch size parameter is again provided (-b=<batch_size>). For all of the adversarial samples provided, an equal number of noisy samples will be generated and included alongside the original samples as part of the 'negative' class for the detector. The perturbation size of these noisy samples is determined based on the average L2 perturbation size of the adversarial samples. Then, the Bayesian uncertainty and kernel density features will be computed for each of the normal, noisy and adversarial samples. A logistic regression model is trained on these features and the detector is built.

Here, a simple demonstration is provided of the commands issued to run the full experiment with MNIST, using the FGSM attack. The following commands are used to run all 3 steps:

1. python train_model.py -d=mnist -e=10
2. python craft_adv_samples.py -d=mnist -a=fgsm
3. python detect_adv_samples.py -d=mnist -a=fgsm

This code depends on the cleverhans repository; however, some of the attack code is re-implemented here with minor modifications (e.g. the "fgsm" function). When cleverhans 2.0 is released, these modifications will no longer be necessary, and this repository will be updated.