Do Explanations Improve the Quality of AI-assisted Human Decisions? An Algorithm-in-the-Loop Analysis of the Effects of Factual & Counterfactual Explanations
This repository is the official implementation of Do Explanations Improve the Quality of AI-assisted Human Decisions? An Algorithm-in-the-Loop Analysis of the Effects of Factual & Counterfactual Explanations published in The 22nd International Conference on Autonomous Agents and Multiagent Systems in London, 29 May - 2 June 2023.
analysis
: contains all the analysis files used to generate data and figures used in publicationdatasets
: contains three csv files.defendants.csv
is the file with the 300 defendants presented to participants in the experiment,results.csv
is the file with all the prediction data collected from the experiments,participants.csv
is the file with information on each of the respondents from the experiment.accuracy.ipynb
: analyses under the first desideratum -> accuracyreliability.ipynb
: analyses under the second desideratum -> reliabilityfairness.ipynb
: analyses under the third desideratum -> fairnesseffective-explanations.ipynb
: analyses under the fourth desideratum -> effective explanationssurvey-summary
: summary of some intro and exit survey responses
model-and-exp
: contains files to train and test risk assessment model using COMPAS data, and generate explanations (via SHAP and DiCE)datasets
: contains COMPAS data (compas-scores-two-years.csv
), crime categorization file (crime-categories.csv
), explanation files (shap_exp.csv
,diff_sel.csv
,diff_div.csv
), the model test set (narratives.csv
), and the defendant sample used in the experiment (sample.csv
).
requirements.txt
: required python libraries for model training, explanation generation, and analysis
-
To install Python 3, follow these instructions.
-
To install Pip, follow these instructions.
-
To install Jupyter Lab/Notebook, follow these instructions. To run Jupyter Lab/Notebook, follow these instructions.
-
To set up a virtual environment and use it in Jupyter Lab/Notebook, follow these instructions.
-
To install requirements:
- Clone this github repository
git clone <url-to-this-repo>
cd <cloned-repo>
cd public-repo
- Get Python requirements needed
pip3 install -r requirements.txt
There are three datasets used in the analysis:
- defendants.csv: file with information on the 300 defendants sampled and presented to participants in the experiment
id
: unique defendant identifierage
: defendant agesex
: defendant sex (male/female)race
: defendant race (Caucasian/African-American)priors_count
: defendant number of prior convictionsjuv_fel_count
: defendant number of juvenile felony chargesjuv_misd_count
: defendant number of juvenile misdemeanor chargesc_charge_degree
: defendant criminal charge degree (felony/misdemeanor)offense_type
: defendant offense type (one of 8 categories)real_outcome
: whether or not the defendant reoffended (recidivism = 1, no recidivism = 0)alg_outcome
: whether or not the model predicted the defendant will reoffend (recidivism = 1, no recidivism = 0)alg_risk_score
: probability of the defendant reoffending predicted by the modelalg_risk_score_decile
:alg_risk_score
as a decile scoreinfluence_all
: influence of the risk assessment model on participants making predictions about the defendant across all treatmentsinfluence_1
: influence of the risk assessment model on participants making predictions about the defendant in treatment 1 (unexplained risk assessment model)influence_2
: influence of the risk assessment model on participants making predictions about the defendant in treatment 2 (diverse counterfactual)influence_3
: influence of the risk assessment model on participants making predictions about the defendant in treatment 3 (selective counterfactual)influence_4
: influence of the risk assessment model on participants making predictions about the defendant in treatment 4 (complete feature attribution)influence_5
: influence of the risk assessment model on participants making predictions about the defendant in treatment 5 (selective feature attribution)
- results.csv: file with information on all the predictions made in the experiment
session_id
: unique identifier of an experiment session (30 predictions by a unique participant)response_id
: unique identifier of each prediction in the resultstreatment
: the treatment each session belongs todefendant_id
: unique defendant identifierdefendant_race
: defendant race (Caucasian/African-American)defendant_age
: defendant agedefendant_sex
: defendant sex (male/female)defendant_priors
: defendant number of prior convictionsdefendant_juv_fel_count
: defendant number of juvenile felony chargesdefendant_juv_misd_count
: defendant number of juvenile misdemeanor chargesdefendant_charge_degree
: defendant criminal charge degree (felony/misdemeanor)defendant_offense_type
: defendant offense type (one of 8 categories)task_order
: the order of the prediction task within the 30 tasksra_score
: decile risk score predicted by the risk assessment modelparticipant_score
: decile risk score predicted by the participantparticipant_gender
: participant genderparticipant_age
: participant ageparticipant_degree
: participant education degreeparticipant_ethnicity
: participant ethnicityparticipant_politics
: participant political party affiliationactual_outcome
: actual recidivism outcome of defendantinfluence
: average influence of the risk assessment model on the participant over all the 30 predictionsdeviation
: amount of deviation of participant score from risk assessment model scoretask_sub_time
: timestamp of task submission
- participants.csv: file with information on participants and their survey responses. All multiple choice questions (MCQ) are on a 5-point Likert scale: (1) Not at all, (2) Slightly, (3) Moderately, (4) Very, (5) Extremely, except for the question on
accountability
session_id
: unique identifier of an experiment session (30 predictions by a unique participant)treatment
: the treatment each session belongs toparticipant_gender
: participant genderparticipant_age
: participant ageparticipant_degree
: participant education degreeparticipant_ethnicity
: participant ethnicityparticipant_politics
: participant political party affiliationml_fam
: MCQ answer to this survey question, "How familiar are you with machine learning?"cj_fam
: MCQ answer to this survey question, "How familiar are you with the U.S. Criminal Justice System?"confidence
: MCQ answer to this survey question, "How confident were you in your decisions?"relative_confidence
: MCQ answer to this survey question, "How well do you think you did compared to other experiment participants?"self_reported_influence
: MCQ answer to this survey question, "How much did the algorithm's risk score influence your decision?"self_reported_exp_usefulness
: MCQ answer to this survey question, "For each defendant, you were presented with an explanation shedding light on why the algorithm predicted a specific score for the defendant. How useful was that explanation?"self_reported_ra_accuracy
: MCQ answer to this survey question, "How accurate do you think the risk score algorithm is?"self_reported_ra_fairness
: MCQ answer to this survey question, "How fair (i.e. neutral and unbiased) do you think the risk score algorithm is?"self_reported_exp_ability
: MCQ answer to this survey question, "If one of the decisions you made goes wrong or is questioned, how well can you explain how you arrived at that decision?"accountability
: MCQ answer to this survey question, "If one of the decisions you made goes wrong or is questioned, how much accountability do you think you should face?" Options: (1) None, (2) Less than the developers of the algorithm, (3) Equal to the developers of the algorithm, (4) More than the developers of the algorithm, (5) I should face accountability, but the developers of the algorithm should notopen_response_1
: open-response answer to this survey question, "Could you tell us how you incorporated the algorithm's risk scores in your decisions (if at all)?"open_response_2
open-response answer to this survey question, "Could you tell us how you incorporated those explanations in your decisions (if at all)?"influence
: influence of risk assessment model on participant predictions over 30 predictionsparticipant_brier_score
: (1 - brier loss) of participant over 30 predictionsfalse_positive_participant
: overall participant false positive ratera_brier_score
: (1 - brier loss) of the risk assessment model over 30 predictionsfalse_positive_ra_black
: risk assessment model false positve rates for black defendantsfalse_positive_ra_white
: risk assessment model false positve rates for white defendantsfalse_positive_ra_diff
: difference in risk assessment model false positve rates for black vs white defendantssession_submit_time
: timestamp of session submission
All the analysis notebooks used to generate the figures and results used in the publication can be found in this folder:
- datasets
- accuracy.ipynb
- reliability.ipynb
- fairness.ipynb
- effective-explanations.ipynb
- survey-summary.ipynb
To train the gradient boosted model used and to generate explanations, follow the instructions in this notebook.