airbnb / artificial-adversary Goto Github PK

Currently this assumes words are separated by spaces. If this is not the case, or if an earlier attack removes a space, several words may be treated as a single word (such as send.me.money).

One way around this is explicitly to keep track of word indices in the original string (assume here that they are separated by whitespace), and then modify these as attacks modify words/text.

Right now we assume no feedback between adversary and classifier.

What if the adversary has access to the labels? What if the adversary has access to the raw probabilities? What is the adversary has access to some observation that can be linked back to the label or probability?

These are very broad, and while some have been addressed in machine learning literature, there are many possible takes on this as it specifically applies to text classification.

Potential ideas (this list will grow):

• Use Lime to identify words that are important to classification results and apply targeted attacks
• Simulate a sequence of back-and-forths between classifier and adversary

There are many possible attacks that have not yet been implemented.

Some of these include:

• Phrase-level attacks
  • Invert part-of-speech order
  • Change tense
• Replacing words with homonyms, or symbols that are pronounced as a homonym (ate -> eight, for -> 4)
• Surrounding characters (or other alternating patterns) (bank -> (b)(a)(n)(k))

I think it is necessary to add an experiment that compare the test accuracy of the original text and the adversarial text examples in the target model to judge whether the adversarial text examples really reduce the accuracy.

👨‍🍳 🍆 🍽 👍

UGC may feature emojis

import Adversary

/anaconda3/lib/python3.6/site-packages/Adversary/init.py in ()
----> 1 from Adversary.adversary import Adversary
2 from Adversary.attacks import *

ModuleNotFoundError: No module named 'Adversary.adversary'

Because /anaconda3/lib/python3.6/site-packages/Adversary/Adversary.py
mv Adversary.py adversary.py