tailor's Introduction

Tailor

This repository contains code for compositional perturbations as described in the following paper:

Tailor: Generating and Perturbing Text with Semantic Controls
Alexis Ross*, Tongshuang Wu*, Hao Peng, Matthew E. Peters, Matt Gardner Association for Computational Linguistics (ACL), 2022

Bibtex for citations:

@inproceedings{ross-etal-2022-tailor,
    title = "Tailor: Generating and Perturbing Text with Semantic Controls",
    author = "Ross, Alexis and
        Wu, Tongshuang and
        Peng, Hao and
        Peters, Matthew E and
            Gardner, Matt",
    booktitle = "Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics",
    month = aug,
    year = "2022",
    address = "Online",
    publisher = "Association for Computational Linguistics",
}

Installation

From Pypi:

pip install tailor_nlp

From source:

git clone https://github.com/allenai/tailor.git
cd tailor
pip install -e .

Recreating Tailor

See link for information on how to format Ontonotes 5.0 and train the Tailor generator.

Tailor-Generated Contrast Sets

See link for the data. (More information in Section 5 of the paper.)

Tailor-Generated Augmented Examples for NLI

See link for the perturbations used in our NLI data augmentation experiments. (More information in Section 6 of the paper.)

Using Tailor: walkthrough cases

See the tutorial notebook for a detailed walkthrough of the API.
See the documents in the main Python file for more explanations.
See Tutorial 02 to learn how to use the default perturbation function on NLI data.
See Tutorial 03 to learn how to define a customized perturbation function for MATRES data.

Basic Perturbation demo

# initiate a wrapper.
from tailor import Tailor
tl = Tailor()

text = "In the operation room, the doctor comforted the athlete."

# perturb the sentence with one line:
# When running it for the first time, the wrapper will automatically
# load related models, e.g. the generator and the perplexity filter.
perturbations = tl.perturb(text)

# return: [
# 'the athlete was comforted by the doctor .',
# 'In which case , the doctor comforted the athlete.',]

More advanced APIs

To perturb with more controls,

perturbations = tl.perturb(
    sentence=text,
    selected_span = "In the operation room",
    # can filter perturbations by their change type, as printed above.
    allowed_perturbs=["change_content"],
    # can reuse the detected strategies
    candidate_inputs = perturb_strategies,
    # filter out degeneration with gpt-2 perplexity score. If None, then this step is skiped.
    perplex_thred=50,
    # max number of perturbations to return.
    num_perturbs=10
)

# return: ["In case of an injury , the doctor 's comforted the athlete.",
# "In case of a fatal accident , the doctor 's comforted the athlete.",
# "In case of a bruised hand , the doctor 's comforted the athlete."]

To attach additional context,

tl.perturb_with_context(
    "In the operation room, the doctor comforted the athlete.",
    "In the operation room",
    to_content="bridge",
    verbalize=True
)
# return: ["Under the bridge , the doctor 's comforted the athlete.",
# "Under a bridge , the doctor 's comforted the athlete."]

tl.perturb_with_context(
    "In the operation room, the doctor comforted the athlete.",
    "In the operation room",
    to_semantic_role="TEMPORAL",
    verbalize=True
)

# return: ['When the doctor came into the operation room , the physician comforted the athlete.',
# "While the doctor was in the operation room , the physician 's comforted the athlete."]


tl.perturb_with_context(
    "In the operation room, the doctor comforted the athlete.",
    "comforted",
    to_tense="future",
    verbalize=True
)

# return: ['In the operation room , the doctor will comfort the athlete.',
# "In the operation room , the doctor 's will comfort the athlete."]

tailor's People

Contributors

Stargazers

Watchers

tailor's Issues

Augment step

Lack of `init.py` file in ./tailor/common/filters

Is your feature request related to a problem? Please describe.
When installing from Pypi (tailor-nlp 0.1.0), the following bug will appear:

ModuleNotFoundError: No module named 'tailor.common.filters'

It turns out that there is no __init__.py file under path ./tailor/common/filters, so this directory would be ignored and totally lost when installing the package.

Describe the solution you'd like
Add a blank ./tailor/common/filters/__init__.py

Describe alternatives you've considered
N/A

Additional context
N/A

AttributeError: No huggingface_hub attribute hf_api error

Note: this issue seems to be resolved with updating the transformers version in requirements.txt, but a new error pops up when trying to perturb using Tailor. See EDIT below.

I have installed Tailor using pip: pip install tailor_nlp. I used a new Conda environment for installation with Python 3.7.

However, I get the following error when trying to import Tailor and initiate a wrapper: AttributeError: No huggingface_hub attribute hf_api. I'm not sure what to change. I am using transformers version 4.4.2, huggingface-hub version 0.8.1, and datasets version 1.15.1.

# initiate a wrapper.
from tailor import Tailor
tl = Tailor()

EDIT: I was able to fix this issue by changing the requirements to be transformers<=4.20.1. However, I now get another error TypeError: Can't convert {'output_ids': [[[0, 86, 8, 2986, 562, 6, 8, 2472, 3, 31, 7, 784, 5365, 308, 4090, 10, 56, 908, 784, 16174, 279, 10, 2115, 908, 8, 17893, 5, 1]]]} to Sequence when trying to run perturbations = tl.perturb(text) where text="In the operation room, the doctor comforted the athlete.".

allenai / tailor Goto Github PK