This repo contains the source code in the paper On the Robustness of ChatGPT: An Adversarial and Out-of-distribution Perspective.

This project is to evaluate the robustness of ChatGPT as well as some foundation language models. You can find the results here.

First, clone and get into the repo:

git clone https://github.com/microsoft/robustlearn.git
cd robustlearn/chatgpt-robust

Then, install the following important depencencies by:

• pip install transformers pandas nltk jieba
• Note that this code repo does NOT provide usage of ChatGPT. Optionally, you can use it from its official website (https://chat.openai.com/chat) or use the third-party library to imititate the brower activity (https://github.com/mmabrouk/chatgpt-wrapper).

You can also create an conda virtual environment by running conda env create -f environment.yml.

All things can be used by running main.py:

For classification tasks:

• Use Huggingface: python main.py --dataset advglue --task sst2 --service hug --model xxx
• Use GPT API: python main.py --dataset advglue --task sst2 --service gpt --model text-davinci-003

For translation tasks:

• Use Huggingface: python main.py --dataset advglue-t --task translation_en_to_zh --service hug --model xx

Note that you will not get the final results by simply running the codes, since the outputs of generative models are not stable. We need some manual processing. Bad cases of AdvGLUE and Flipkart are pvovided in this folder.

Here is the summary of the results.

The metric is attack success rate (ASR).

The metrics are BLEU, GLEU, and METEOR.

The metric is F1 score.

@article{wang2013robustness,
  title={On the Robustness of ChatGPT: An Adversarial and Out-of-distribution Perspective},
  author={Wang, Jindong and Hu, Xixu and Hou, Wenxin and Chen, Hao and Zheng, Runkai and Wang, Yidong and Yang, Linyi and Huang, Haojun and Ye, Wei and Geng, Xiubo and Jiao, Binxin and Zhang, Yue and Xie, Xing},
  journal={arXiv preprint arXiv:2302.12095},
  year={2023}
}

Note that the results of some generative models might change due to the nature of generation. Thus, the results of this code repo could also change. Additionally, the results output by generative foundation models are not very clean, so you need to manually process some dirty outputs to get what you want. Our best suggestion to use this code is for demo and practice, since it makes it easy to get the outputs of multiple models. But remember, human evaluation and processing are also important.