Git Product home page Git Product logo

ml-fairness-gym's Introduction

What is ML-fairness-gym?

ML-fairness-gym is a set of components for building simple simulations that explore the potential long-run impacts of deploying machine learning-based decision systems in social environments. As the importance of machine learning fairness has become increasingly apparent, recent research has focused on potentially surprising long term behaviors of enforcing measures of fairness that were originally defined in a static setting. Key findings have shown that under specific assumptions in simplified dynamic simulations, long term effects may in fact counteract the desired goals. Achieving a deeper understanding of such long term effects is thus a critical direction for ML fairness research. ML-fairness-gym implements a generalized framework for studying and probing long term fairness effects in carefully constructed simulation scenarios where a learning agent interacts with an environment over time. This work fits into a larger push in the fair machine learning literature to design decision systems that induce fair outcomes in the long run, and to understand how these systems might differ from those designed to enforce fairness on a one-shot basis.

This initial version of the ML-fairness-gym (v 0.1.0) focuses on reproducing and generalizing environments that have previously been discussed in research papers.

ML-fairness-gym environments implement the environment API from OpenAI Gym.

This is not an officially supported Google product.

Contents

Contact us

The ML fairness gym project discussion group is: [email protected].

Versions

v0.1.0: Initial release. v0.1.1: Update to use gym 0.19.0.

ml-fairness-gym's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

ml-fairness-gym's Issues

Update folders or imports in quick start

Hello. Thanks for ML for fairness!

(venv) โžœ experiments git:(master) โœ— python ./lending_demo.py --num_steps=500
Traceback (most recent call last):
File "./lending_demo.py", line 26, in
from agents import threshold_policies

I already use the PATH code
export PYTHONPATH="${PYTHONPATH}:/path/to/ml-fairness-gym"
After ./tests.sh
File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/traceback.py", line 285, in line
self._line = linecache.getline(self.filename, self.lineno).strip()
File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/linecache.py", line 16, in getline
lines = getlines(filename, module_globals)
File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/linecache.py", line 48, in getlines
for mod in sys.modules.values():
RuntimeError: dictionary changed size during iteration

And there I can not proceed anymore from here.

Errors when running tests and lending_demo.py

I installed ml-fairness-gym using the instructions and Python version 3.7, but when I ran tests.sh a number of the tests failed and eventually I had to cancel the script because it froze my computer.

When I try running lending_demo.py I get the following error. I'm not sure what the issue is and haven't been able to debug it thus far.

fairness_error

Assertion Error when running quickstart example

I get the following error when trying to run the example:
AssertionError: Observation {'bank_cash': array(10001.), 'applicant_features': array([0, 1, 0, 0, 0, 0, 0], dtype=int32), 'group': array([1., 0.])} is not contained in self.observation_space
In call to configurable 'run_simulation' (<function run_simulation at 0x7fa07162a440>)

all the requirements should be installed, however I get a few failed tests when running ./tests.sh

  • some undefined metric warnings: UndefinedMetricWarning: No positive samples in y_true, true positive value should be meaningless
  • AssertionErrors as in the example.
    image

Any help appreciated

PDF of the "Fairness is not Static" paper?

Hi,

Thanks for this really nice environment. I have been testing it out. However, I was looking to read the paper: "Fairness is not Static: Deeper understanding of long term fairness via agents and environments. KDD workshop on Explainable AI (XAI) 2019 " that I assume this work is based on.

Can you provide link to the provide if it is available somewhere?

Thanks!

Can't Reproduce Results: Tensorflow==1.15 not available on pip any more

Do you have a github version of this ml-fairness-code with tensorflow 1.8 or later? Currently requirements.txt uses tensorflow==1.15 which is not available for pip installation anymore. I have tensorflow 1.8 but that (obviously) creates a lot of dependancies issues. I am struggling to reproduce the results.

Please let me know.

Many thanks,
Ali

Update folders or imports in quick start example

hello, thanks for sharing! ๐Ÿ‘

The quick start doesn't work for me:

Traceback (most recent call last):
  File "examples/lending_demo.py", line 26, in <module>
    from agents import threshold_policies
ModuleNotFoundError: No module named 'agents'

I think this is trying to import a file, but agents/threshold_policies is in the root and not the examples folder.

This works though:

$ cp examples/lending_demo.py .
$ python ./lending_demo.py --num_steps=500

...and opens a bunch of windows with charts! :)

So maybe these paths or imports need to be updated, or the quick start instructions? I'm using conda so if this is just because of that vs. virtualenv then sorry if I don't know enough to tell the difference, just trying to be helpful :)

Error While running attention_allocation_experiment in acm_fat_2020

While the other two experiments run, i keep getting this error concerned with Multi processing pool.
multiprocessing.pool.MaybeEncodingError: Error sending result: '<multiprocessing.pool.ExceptionWithTraceback object at 0x114b00f10>'. Reason: 'PicklingError("Can't pickle <class 'TypeError'>: it's not the same object as builtins.TypeError")'

Add .gitignore

This is what I see trying to run the commands in the quick start, after git status:

	__pycache__/
	agents/__pycache__/
	bin/
	environments/__pycache__/
	examples/__pycache__/
	include/
	lib/
	metrics/__pycache__/
	spaces/__pycache__/

Should this project have a .gitignore file?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.