Comments (2)
amad-person
commented on June 12, 2024
Hi @tr0p1x, thanks for opening the issue.
From what I can see, in fact, the error comes from the way SKLearn sets the thresholds for computing the false positive rate and the true positive rate when passing an array with float values.
Yes, you're right. sklearn is computing the threshold after seeing the data instead of fixing it at 0.5. However, this is desired behaviour for our library - because in membership inference attacks we want to find the best threshold that separates out members and non-members.
Hope this clears things up!
from ml_privacy_meter.
luigitropiano
commented on June 12, 2024
One more doubt then: isn't that conflicting with how the attack accuracy is computed (with a fixed threshold at 0.5) ?
I think there may be situations where the tool reports a very low attack accuracy but an almost perfect ROC curve.
from ml_privacy_meter.
Related Issues (20)
- Attack S and Attack P cant be reproduced HOT 3
- OOM problem in attack alexnet HOT 1
- feberated learning HOT 1
- Can't achieve a better accuracy than 0.5121 with the blackbox tutorial: Running the Alexnet CIFAR-100 Attack HOT 5
- Pytorch implementation HOT 2
- pip install -r requirements.txt throws: ERROR: No matching distribution found for tensorflow-gpu==2.5.0 HOT 1
- Can't exploit gradients of ResNet-20 HOT 4
- can i attack linear regression、logistic、XGBoost
- can i attack linear regression、logistic、XGBoost models? HOT 1
- attacking convolutional layer's gradient - shape mismatch HOT 5
- MIA blackbox attack accuracy repeats same value HOT 3
- Code of "MIA via Distillation" HOT 1
- Blackbox attack of a basic binary TensorFlow classifier with tabular data HOT 1
- Request for FL and Unsupervised Learning Version HOT 1
- A question for attack_alexnet.py. HOT 1
- Old tutorials with restructured code HOT 1
- Add conda recipe HOT 8
- FileNotFoundError: [Errno 2] No such file or directory: '../privacy_meter/report_files/explanations.json' HOT 2
- Bug in notebook examples that use PyTorch models HOT 1
- Enhanced MIA HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ml_privacy_meter.