gap-lab-cuhk-sz / mvimgnet Goto Github PK

CVPR2023 | MVImgNet: A Large-scale Dataset of Multi-view Images

Home Page: https://gaplab.cuhk.edu.cn/projects/MVImgNet/

License: Other

Python 100.00%

cvpr2023 multi-view multi-view-stereo nerf real-world-data dataset deep-learning pointcloud 3d-reconstruction 3d-understanding

mvimgnet's Introduction

MVImgNet: A Large-scale Dataset of Multi-view Images

by Xianggang Yu*, Mutian Xu*†, Yidan Zhang*, Haolin Liu*, Chongjie Ye*, Yushuang Wu, Zizheng Yan, Chenming Zhu, Zhangyang Xiong, Tianyou Liang, Guanying Chen, Shuguang Cui, Xiaoguang Han‡ from GAP-Lab.

Introduction

This repository is built for:

MVImgNet: A Large-scale Dataset of Multi-view Images (CVPR2023) [arXiv]

If you find our work useful in your research, please consider citing:

@inproceedings{yu2023mvimgnet,
    title     = {MVImgNet: A Large-scale Dataset of Multi-view Images},
    author    = {Yu, Xianggang and Xu, Mutian and Zhang, Yidan and Liu, Haolin and Ye, Chongjie and Wu, Yushuang and Yan, Zizheng and Liang, Tianyou and Chen, Guanying and Cui, Shuguang and Han, Xiaoguang},
    booktitle = {CVPR},
    year      = {2023}
}
}

MVImgNet

MVImgNet contains 6.5 million frames from 219,188 videos crossing objects from 238 classes. We provide an OneDrive link to download the full data. Please fill out this form to get the download link and password.

We split the full data into 42 zip files, the total size is about 3.4 TB.

Usage

cd path/to/mvimgnet_zip_file
unzip './*.zip'

Folder structure

|-- ROOT
    |-- class_label
        |-- instance_id
            |-- images
            |-- sparse/0
                |-- cameras.bin   # COLMAP reconstructed cameras
                |-- images.bin    # binary data of input images
                |-- points3D.bin  # COLMAP reconstructed sparse point cloud (not dense)

The mapping between class_label and class name can be found in mvimgnet_category.txt.

The images folder contains the multi-view images, and the sparse folder contains the reconstructed camera parameters using COLMAP. It is recommended to use the functions provided by COLMAP to read the binary files under sparse folder. Moreover, the gen_poses function from this repo is recommended to convert the poses for NeRF training.

Script for downloading MVImgNet

We also provide the script, at download_tool.py, for downloading all the content of our dataset. Before using it, please make sure you have filled out our form and get the password.

MVPNet

MVPNet now contains 87,825 point clouds from 180 categories. Please fill out the following form to download MVPNet.

Demo

MVImgNet is also shown by Voxel51 at here, which is publicly demo-able!

License

The data is released under the MVImgNet Terms of Use, and the code is released under the Attribution-NonCommercial 4.0 International License.

Acknowledgement

Thanks to Wei Cheng for the new downloading solution for our dataset.

Thanks to Gege Gao for providing tips on downloading our dataset.

Thanks to Voxel51 for providing the dataset demo.

mvimgnet's People

Contributors

Stargazers

Watchers

Forkers

alexanderinum goodyenough asdlei99

mvimgnet's Issues

How to download all files in Linux

Hi,

Thanks for sharing this dataset. I've seen the downloading tips in readme and that is very helpful. I do not quite get how to download all files together. When I select all files and click download button, I didnot see any links starting from "download.asp". Any suggestions here?

Best,

MVPNet release time?

When will the MVPNet dataset be released? Thanks for sharing the dataset with the community!

About download MVPNET dataset

@hugoycj
hi, i cant download the raw MVPNET for the internet due to network stability and speed, do you have other ways for downloading the dataset like google drive or baidu pan?

Besides, would you kindly to explane the difference between the raw dataset and the preprocessed dataset from https://github.com/hugoycj/MVImgNet_3D_Pretraining?

And would you plan to release the training code of PointNet++\PointMLP\PCT ,(as in Tab.12) ，one of them will be enough.

Best wishes,

Aspect Ratio - 1080 x 1920

Hello Authors,

Thank you for the great work!
I downloaded some zip files (MVImgNet) and found out that most of the scenes (more than 90%) have a portrait aspect ratio (1920 x 1080). I wonder approximately what percentage of scenes have a landscape ratio (1080 x 1920). Could you please share some statistics?

Cheers,
Wonbong

Camera poses

Could you provide the straightforward code processing .bin files into text format.

Do you have any plan to add textual annotations?

Hi, congratulations on your fantastic work! I believe providing textual descriptions for MVImgNet and MVPNet would facilitate a lot of applications, such as multi-modal pretraining. Do you have any plan to do this?

Camera poses

Hi, thank you for the fantastic dataset!
I saw that sometimes there are multiple cameras for a sample (e.g. for mvi_30.zip 175/280142be/sparse/0/cameras.bin and 175/280142be/sparse/1/cameras.bin). They sometimes seem to correspond to the same images - which ones are the correct ones to use and what do the different cameras correspond to?

How to read points3d.bin?

Thank you for your hard work in making this data set open.
I have downloaded part of the MVImgNet, and I am curious about how to read point3d.bin to get the pointclouds.
Can you give me an example?

Is it possible to share the source code of Data Processing?

Is it possible to share the source code of Data Processing?
thanks!

About camera extrinsics

Can you open source a subset that only contains point clouds?

MVPNet download link is down.

MVPNet for point cloud classification

Thank you for your excellent work. Could you please provide the specific file partition and corresponding labels of the training and test of the MVPNet dataset for point cloud classification challenge in the paper? By the way, I noticed in section 6.1 of the paper that MVPNet has 150 classes, however, there are 180 classes in the provided files. Thank you very much.

How to download in linux terminal

Thanks for the great dataset! I'm working on a remote server and wondering how to download it directly to the remote machine.

Thanks in advance!

when I click the download link, it saids Access to this document has been removed.

how to figure it out?

Read point cloud data from the MVPNet dataset.

Excellent work!
I have a question.
What does the MVPNet data directory look like, and how do I read the data from MVPNet?
If I want to use the PointMAE baseline for pretraining on MVPNet, how do I read the data from MVPNet? The original code is designed to read ShapeNet data for pretraining.
I hope to receive a detailed response from you, preferably with example code for data reading.

Wrong citation format

Hi, there is one extra comma in the provided bibtex, maybe you can fix it to:

@inproceedings{yu2023mvimgnet,
    title     = {MVImgNet: A Large-scale Dataset of Multi-view Images},
    author    = {Yu, Xianggang and Xu, Mutian and Zhang, Yidan and Liu, Haolin and Ye, Chongjie and Wu, Yushuang and Yan, Zizheng and Liang, Tianyou and Chen, Guanying and Cui, Shuguang and Han, Xiaoguang},
    booktitle = {CVPR},
    year      = {2023}
}

Cannot open the download link

After I filled the form, the website shows: 此链接已删除。
很抱歉，已删除对此文档的访问权限。请联系与你共享它的人员。

Object 3D bounding box

Hi,

How can we get the object 3D bounding box or the object diameter? Thanks!

Download Auto Canceled By Brower

I used chrome to download the dataset, but the download tasks were alwayed terminated by the browser (or maybe onedrive server). Does anyone meet the same question as me?

Question about data structure

Dear there,

Thanks for sharing this dataset. I'm wondering if the image frames under images folder is temporally continuous, or it is just random organized views?

Best,

MVImgNet for multi-view classification

Hi, thanks for the fantastic work for multi-view dataset. I'm working on multi-view classification now, so I'm interested in the raw image and image label only. The full dataset is so huge to download. Could you please release the dataset with raw image and label only later?

(23) Failed writing body

Hi,

After using the curl command to download the files, I met a new problem indicating "failed writing body" as follows:

Could anyone meets this problem?

How to get the segmentation masks?

Hi,

Thanks for sharing the dataset!

I downloaded a subset of the dataset and was trying to get the dense foreground mask, but I have no clue where and how to get the object mask. Is there any instruction for this? Thanks!

No camera for some videos

Hi,

Thank you for your great work ! I downloaded the MVImgNet dataset and I noticed that sometimes there is nothing in the sparse folder. Is it normal ?

Masks seem not very clear

Hi developers,
First of all, thank you for creating such a large-scale dataset.
I am trying to reconstruct point clouds using your multi-view data set with masks, but I find some masks not very clear. Like this:

I want to know:

How did you create these masks?
How were the clear pointclouds in MVPNet made?
Is there anyway to optimize masks？

The error in downloading the dataset

Hi, when I try to download a file in the data set, the following error message appears after a period of downloading. How can I fix it?

Correspondence between the zip file and object category

Thanks for the great dataset! However, the dataset is extremely huge, and I'm only interested in some specific categories. Therefore, could you provide the correspondence between the zip file and the object category?