binliang-nlp / arts_testset Goto Github PK

This is the repository for our 2020 paper "Tasty Burgers, Soggy Fries: Probing Aspect Robustness in Aspect-Based Sentiment Analysis".

We provide a Aspect Robustness Probing test set for SemEval 2014 Aspect-Based Sentiment Analysis (ABSA).

• Our new enriched test sets are at data/arts_testset
• Our AspectSet mentioned in the paper Section 2.3 (Table 4) is provided in data/aspectset

We generate our new probing test set by three automatic strategies:

• RevTgt (sentence with a red background): Reverse the sentiment of the Target aspect.
• RevNon (sentence with a green background): Reverse the sentiment of the Non-target aspect.
• AddDiff (sentence with a blue background): Add new aspects with Different sentiment.

We probed nine ABSA models (as mentioned in our paper).

• Their outputs on SemEval 2014 are in the output folder.

If you have a new ABSA dataset, you can run our code to generate you own aspect robustness probing test set.

python code/main.py -dataset_name laptop

If you have more questions, please feel free to submit a GitHub issue.