This repository mainly contains the Colorful Extended Cleanup World (CECW) dataset, a benchmark for compositional generalization investigation such as the work in Revisit Systematic Generalization via Meaningful Learning.

As the name suggests, the CECW dataset is a color-extended version of the Cleanup World (CW) borrowed from the mobile-manipulation robot domain (MacGlashan et al., 2015). CW refers to a world equipped with a movable object as well as four rooms in four colors, including "blue," "green," "red," and "yellow," which is designed as a simulation environment where the agent can act based on the instructions received (Gopalan et al., 2018). CW obeys a particular Geometric Linear Temporal Logic (GLTL) to parse commands by grammatical syntax, resulting in a total of 3382 commands reflecting 39 GLTL expressions. In addition, commands can be represented in textual expressions as shown in the table.

The task in CW can be formatted as a supervised semantic parsing problem to translate commands (e.g., "go to the red room") to their textual expressions (e.g., "F R"). In this repository, we show how CECW is generated on the basis of CW in detail. Besides, we illustrate how to generate subsets from CECW to study synonymous generalization. I hope the publication of the CECW dataset can promote the research in related fields, as well as yours.

• CW - the folder contains the original CW dataset
• train_test_split.ipynb - the main process to generate CECW from CW
• pp_data.ipynb - the data preprocessing to generate 6 subsets from CECW so as to investigate synonymous generalization
• train_src.txt - CECW source data for training
• train_tar.txt - CECW target data for training
• test_src.txt - CECW source data for testing
• test_tar.txt - CECW target data for testing

CECW
├── ALL
├── B
├── BC
├── BCR
├── BCRY
├── CW
├── Colorless
├── README.md
├── helper_functions.py
├── pp_data.ipynb
├── reference
├── test_src.txt
├── test_tar.txt
├── train_src.txt
├── train_tar.txt
└── train_test_split.ipynb

• python >= 3.7.6
• jupyter >= 1.0.0
• numpy>=1.18.1

Please ensure the following packages are already installed. A virtual environment is recommended.

• Jupyter Notebook (for .ipynb)

$ cd CECW/
$ pip install pip --upgrade
$ pip install notebook
$ pip install -r requirements.txt

Simply view train_test_split.ipynb and pp_data.ipynb via jupyter notebook. You can also use train_src.txt, train_tar.txt, test_src.txt, and test_tar.txt directly.

• Ning Shi - [email protected]

1. Squire, S., Tellex, S., Arumugam, D., & Yang, L. Grounding English Commands to Reward Functions.
2. Gopalan, N., Arumugam, D., Wong, L. L., & Tellex, S. (2018). Sequence-to-Sequence Language Grounding of Non-Markovian Task Specifications. In Robotics: Science and Systems.