- Python (tested on 3.8)
- Pytorch (tested on 1.7)
git clone https://github.com/nicolasugrinovic/size_depth_disambiguation.git
cd size_depth_disambiguation
conda create -n sizedepth python=3.8
conda activate sizedepth
pip install -r requirements.txt
Install Detectron2 so it can be accesible from the conda environment. The fastest way to install is via pip.
Go to "external" folder:
cd ./external
and install in this folder the following:
We use Midas-v3 as the depth detector. There is no need
for installation as the code used is provided here. However,
you need to download the model weights and place them on ./weights folder. The
weights to be donwloaded are dpt_large-midas-2f21e586.pt
mkdir ./weights
wget https://github.com/intel-isl/DPT/releases/download/1_0/dpt_large-midas-2f21e586.pt
Install pytorch3d
. Though the only function used is load_obj. You may use
any other library to load the meshes, just be aware of the
vertex order at reading.
To run the demo, you first need to generate (or precompute) data which is then used by the optimization method. You need to generate initial pose/shape estimations and give them the correct format. Alternatevily, you can download and use this data.
To precompute the data run the following command:
python precompute_estimation_data.py --input_path=./input/coco_demo --output_path=./precomputed_data/coco_demo/ --mode=smpl_reproj --dataset=demo --model_type=dpt_large --config-file ./external/panoptic_deeplab/tools_d2/configs/COCO-PanopticSegmentation/panoptic_deeplab_H_48_os16_mg124_poly_200k_bs64_crop_640_640_coco_dsconv.yaml --opts MODEL.WEIGHTS ./external/panoptic_deeplab/tools_d2/checkpoints/panoptic_deeplab_H_48_os16_mg124_poly_200k_bs64_crop_640_640_coco_dsconv.pth
Change input_path accordingly to where you have the initial input data.
This can have any location as long as it has the following structure:
input
|-- data_name
`-- img_name1.jpg
`-- img_name1_TRANS_person0.obj
.
.
.
`-- img_name1_TRANS_personN.obj
`-- img_name1_3djoints_0.json
.
.
.
`-- img_name1_3djoints_N.json
`-- img_name2.jpg
.
.
.
This will generate the data inside ./precomputed_data/coco_demo/ folder.
Finally, to run the optimization from our method execute the following command:
python run_optim_demo.py --model_type=dpt_large --input_path=./input/coco_demo --output_path=./output/coco_demo --input=input/coco_demo/*.jpg --mode=smpl_reproj --plane_scale=1.0 --n_iters=2000 --w_ordinal_loss=0 --w_reg_size=0.0 --config-file ./external/panoptic_deeplab/tools_d2/configs/COCO-PanopticSegmentation/panoptic_deeplab_H_48_os16_mg124_poly_200k_bs64_crop_640_640_coco_dsconv.yaml --opts MODEL.WEIGHTS ./external/panoptic_deeplab/tools_d2/checkpoints/panoptic_deeplab_H_48_os16_mg124_poly_200k_bs64_crop_640_640_coco_dsconv.pth
Alternatively, you can run the scripts precompute_data.sh and
run_demo.sh found in the scripts folder
Run eval.py
If you use this code or ideas from the paper for your research, please cite our paper:
@inproceedings{ugrinovic2021body,
title={Body Size and Depth Disambiguation in Multi-Person Reconstruction from Single Images},
author={Ugrinovic, Nicolas and Ruiz, Adria and Agudo, Antonio and Sanfeliu, Alberto and Moreno-Noguer, Francesc},
booktitle={2021 International Conference on 3D Vision (3DV)},
pages={53--63},
year={2021},
organization={IEEE}
}
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent