This archive contains the code for reproducing our experiments. Before running the code, install the dependencies in requirements.txt
. The next three section will provide an overview of how to reproduce the results in each of the domains: music, CLEVR, and maps.
We have uploaded the large files that are necessary (datasets, checkpoints) to Google Drive
- Dataset: we used the standard Slakh2100 dataset for training. This instruction describes how to download and preprocess the dataset.
- Training: run the script
music/train.py
to train the base model. You will need to specify in the code the path to the train dataset. To train any coordinator model, use the scriptmusic/train_coordinator.py
and specify the path to the base model, path to the dataset and the type of coordinator architecture you want to train. - Evaluation: run the script
music/sample_tracks.py
first to sample tracks from the model. It supports multiple options, for example you can sample from MultiDiffusion(--multi
) or Concatenation (--concat
), or from the coordinator model, specified by--rnn
option. After the sampling has finished, now you need to runmusic/calc_fad.py
script, where you need to specify paths to the generated samples and the real dataset to get the FAD value.
- Dataset: we include the dataset in the Google Drive as
clevr_pos_data_128_30000.npz
. You need to specify path to this dataset in all the training scripts. - Training: run the script
clevr/train.py
to train the base model. You will need to fill indataset_path
parameter. To train the classifier, run the scriptclevr/classifier/train.py
. You will need to fill indata_path
parameter. Finally, to train the coordinator model, run the scriptcontrol/train.py
. You will need to fill incube_model_path
(= base model path),classifier_path
anddataset_path
parameters. - Evaluation: run the script
clevr/evaluation/eval_script.py
to calculate accuracy. You will need to fill incube_model_path
(= base model or coordinator model) andclassifier_path
. You can change the sampler you use and whether you want to do sum of scores model or not.
-
Dataset: to generate the dataset, you will have to use Google Maps API. Run the
parser/main.py
script to generate a folderdataset
, containing satellite-map pairs. Then, run theparser/folder_to_npz.py
script to downsample the maps and convert the dataset to a single.npz
file. -
Training: run the script
maps/cond/train.py
to train the map model. You will need to fill indataset_path
,image_size
arguments. Settingditi_devil
to False will train the base model, setting it to True would requirebase_model_path
, and it will train a coordinator model. -
Evaluation run the script
maps/cond/eval.py
to evaluate the FID metric. You will need to fill indataset_path
,image_size
,model_path
, andsample_cnt
. If you want to evaluate multidiffusion, setmulti_diffusion
to True, choose base size and stride, and use the checkpoint of the base model.
The Google Drive contains the base model used for evaluations, and two trained coordinator models.