Camera focal length for the test set different from dev set about co3d HOT 2 CLOSED

Hi David, thanks for the response! I really appreciate it!

It took a while for me to look at the example_co3d_challenge_submission.py and see how the query image is rendered and pasted onto the original image.

Some notes in case it may be helpful for future readers:

1. When we are loading pytorch3d cameras and cropped images with the training data loader, the cameras intrinsic parameters are adjusted according to the object bounding box detected from the foreground mask.
2. When we load the fewview_dev/fewvew_test sets, the query image is not given; thus, we are given unadjusted intrinsic parameters. This may result in different focal length than we are used to seeing for views with box crop.

from co3d.

Hi, the focal lengths within a scene are different because we let COLMAP to extract an image-specific (as opposed to a camera-specific) focal length. The main reason is that the consumer smartphones used to capture the videos often automatically change focus during the capture, which effectively leads to a varying focal length of the video frames.

In terms of example_co3d_challenge_submission.py, we did several unit tests that ensure that the focal length is correctly adjusted when cropping the image around a segmentation mask. The algorithm does not handle samples from different sets (fewview_dev/fewvew_test) differently so there should not be an issue.

Please note that, for the test subset (i.e. fewvew_test/manyview_test), all depth maps and all test images are redacted (i.e. these are blank black images). Hence, example_co3d_challenge_submission.py produces non-sensical results for fewvew_test/manyview_test since the DBIR renderer generates an invalid point cloud from the redacted depth maps of the source views.

from co3d.