laurencebont / fact-full-grad-uva Goto Github PK

We will use this Github repo for the project

License: Other

Python 14.62% Shell 0.09% Jupyter Notebook 85.28%

fact-full-grad-uva's Introduction

Fairness, Accountability, Confidentiality and Transparency in AI

People who worked on this project:

Daan Le - 11329866 - [email protected]
Mathieu Bartels - 11329521 - [email protected]
David Wessels - 11323272 - [email protected]
Laurence Bont - 11198788 - [email protected]

Dependencies?

To get all dependencies for this paper use pipenv or conda

pipenv:

pip install pipenv
pipenv shell
pipenv sync

Now al packages needed for this project are installed!

Data

For these experiments to run efficiently it is advised to download the pre-trained models and pre adjusted datasets.

unpack the datasets in the 'dataset' folder
pixel_perbutation roar extra-test

unpack the models in the saved-models folder models

Please note that once these links have expired the data cannot be fetched anymore! Please reach out to [email protected] if you want to gain acces to the data.

How to run?

Look at some examples in the .ipynb or run

python main.py --experiment [roar|extra|pixel_perturbation]

Research

This project is based on a paper. Make sure to check out this paper, and the original github!

Full-Gradient Saliency Maps

@inproceedings{srinivas2019fullgrad,
    title={Full-Gradient Representation for Neural Network Visualization},
    author={Srinivas, Suraj and Fleuret, François},
    booktitle={Advances in Neural Information Processing Systems (NeurIPS)},
    year={2019}
}

fact-full-grad-uva's People

Contributors

Stargazers

Watchers

fact-full-grad-uva's Issues

Answers to questions in the TA sessions

Hi,

I was asked if the Python Notebook should in the end generate all the results and if that's maybe infeasible, for example since dataset generation would take too long or since it would be way too much image data to be sent if you provide all the perturbed data to the user.

The answer is:

All code of how to generate the results in your paper should be somewhere in the repository
The notebook can also demonstrate similar results on an easier dataset, for example MNIST, such that it's faster to show the functionality of your code.

In any way, I just trust you to do the right thing there: Think about what would be useful for someone who wants to get to know your code, and then provide a notebook that's most useful in that regard.

I also asked Ana: "Students asked me whether there is research applying transparency methods like FullGrad to fairness problems, i.e. in order to see whether the network made judgments based on "protected" characteristics, for example. Do you have any suggestions?"

She answered: "i’m not aware of any but this would be an EXCELLENT extension of the paper if that’s what they want to do."

See you on Tuesday!

Best,
Leon

Food for thought

Just as food for thought: Instead of measuring something like an "absolute fractional output change", one may also try to measure how much the output distribution of the network as a whole changes. I.e.: Regard the output of the network as a distribution over classes and measure how much that distribution changes if you remove pixels.

Here is a list of statistical distances; the Kullback-Leibler divergence is probably the most well-known:

https://en.wikipedia.org/wiki/Statistical_distance

DataLoader uses normalize transformation, but the values are hardcoded

We should find a method to calculate these values, or load them from a trusted source or something. These values have an influence on the way the data is processed

We have no way of checking the accuracy on the validation set

For one of the experiments we need a module that calculates the accuracy of the model given a dataset.

repo name is not comfort any naming standard

I propose using snake-case, a proposed name is: $paper-name$-$fairness/transparency$-project

TA session cancelled next Wednesday

Hello,

since all other TA groups have a session next Friday, which is replaced by the presentations, Ana decided that it's fairer that we have only one session next week instead of two (additional to the presentations).

Therefore, TUESDAY IS THE LAST SESSION. There is no session anymore next Wednesday. Please tell your group members so that everyone knows.

Best,
Leon

Need for reproduction image removal experiment

You win the contest

Hi,

congrats, you win the contest about best implementation!

Grades will follow on Canvas (probably this Friday).

If you want, you can already do some last clean-ups to your code before it goes public on the joint codebase with all other winning teams. More details on how this works will also follow (but I don't know when).