vevenom / roomlayout3d_randc Goto Github PK

Hi, I wonder how to generate the mseg prediction confidence for my own images?

To my knowledge and experience, MSEG network only generates gray scale segmentation result, I can't find confidence part. Could you tell me how to get the confidence?

Thank you very much.

I use planercnn and mseg network to produce the inputs on scannet_layout dataset. There are about 15 examples to fail. And we only calculate the metric on successful cases using [(https://github.com/vevenom/ScanNet-Layout)]. The result is
[2D IoU: 0.645 PE:0.157 EE:25.364 RMSE: 0.455], there is a gap on EE metric, is there any question? Please.

1. What should be the preprocessing step for the depth images ? The values within the depth images are really low how do we get the images in the same format.
2. How can we run the model with out passing the depth image as an input

Hello, @vevenom
Thanks for your impressive work of this project .
I`m working on my idea based on ScanNet-layout, and i need to annotate some custom pictures manually.
So I wonder if you can send me the tagging tools or just tell me the name of these tools ?
Thanks a lot .

HuiYao

Excuse me, I wonder why do I produce wrong point cloud projection during visualization?
Is it cause by the dependence packages?
Or the process computing the depth and camera parameters for projecting point cloud?

Great work. Thanks for sharing the code.
I need to run it for a custom dataset. But confused about mseg_confidence. How to get it? I am a bit puzzled with mseg code. Can you please leave a hint?

Is it possible to run the model on a RGB custom image without having the depth, or it is necessary to have the depth and thus we must use a predicted one?

• In the first case, can you explain how to run the model without the depth.
• In the second case, I used DPT model to predict the depth from the image, but it seems that my predicted depth does not have the right scale/format (the command cv2.imread with the arguments "cv2.IMREAD_ANYCOLOR | cv2.IMREAD_ANYDEPTH" does not work and does not remove the 3rd dimension and convert the image into 16bits). Can you explain the preprocess steps to perform on the predicted depth so that it can be used as input? Also the ground truth depth from scanNet has low values, between 0 and 18 for some images. The predicted depth from DPT has values between 0 and 1, can be transformed between 0 and 255, but what are the constraints on the values?

Also, globally, can you explain the preprocess that should be performed on each input before running the model, or things to check on the input to be sure they are valid?

If needed I can send the inputs I'm currently trying to use so you can see why it does not work.

Hello, I have a question regarding the visualization. Let's say that I want to visualize only the 3D room layout(without point clouds of the objects) just by using the planes in the npy file and the depth map. Without running the optimization and using the segmentation. Is that possible using Open3D?