fewview_train subset JSON contain frames that belong in both of train and test sets about co3d HOT 2 CLOSED

Hello, this is by design.

Tl;dr: Indeed, using the train setlist of set_lists_fewview_train is the best way to train your few-view model.

In more detail, all frames within a category are separated to 6 sets <sequence_set>_<seen|unknown>, i.e.:

train_unseen
train_known
dev_unseen
dev_known
test_unseen
test_known

The set_lists_fewview_*.json set lists are defined as follows:

set_lists_fewview_train: {
    "train": train_known,
    "val": train_known + train_unseen,
    "test": train_known + train_unseen,
}
set_lists_fewview_dev: {
    "train": train_known,
    "val": dev_known + dev_unseen,
    "test": dev_known + dev_unseen,
}
set_lists_fewview_test: {
    "train": train_known,
    "val": dev_known + dev_unseen,
    "test": test_known + test_unseen,
}

For your case specifically, the train setlist of set_lists_fewview_train contains only the train_known frames which should be used for training. However, the val setlist of set_lists_fewview_train contains train_known but ALSO train_unseen. This is why you see that all frames from train are also in val.

The "val" set contains also the "train" views because, when validating/testing, one needs to have access to the "known" source views (from the train set) in order to be able to generate the unseen views. This requires both known and unseen views to live in the same set of loaded images.

Indeed, if you inspect the eval_batches files, you will discover that the first (target) frame in an eval batch is always drawn from the unseen set of frames, while the rest of the frames comes from the known frames.

In order to find out which frames are known/unseen, feel free to inspect the meta.frame_type fields in frame_annotations.jgz.

I hope this helps, let me know if further clarification is needed.

from co3d.

Thank you so much for the reply! This is super helpful!

from co3d.