thu-da-6d-pose-group / self6dpp Goto Github PK

Thanks for your excellent work and open it ！
I had trained yolox for a customized obj，but how get .json for gdrnet/self6dpp like this:

And, if this .json file needs to convert something via Convert_det_results_to_bop.py, how to ensure that the input format meets the input requirements of Convert_det_results_to_bop.py

I met an error ’no module named 'kaolin.graphics'' when training the self6dpp. Did I install a wrong 'kaolin' package?

Thanks for the great work, I was wondering how the enhancement here is done? What part of the code is it used in the project?

I run core/self6dpp/train_self6dpp.sh /home/zhumengjie/self6dpp/configs/self6dpp/ssLMO/ss_mlBCE_MaskFull_PredFull_lr1e_5_lower_woCenter_refinePM10_lmoNoBopTest/ss_mlBCE_MaskFull_PredFull_lr1e_5_lower_woCenter_refinePM10_01_ape.py 0 and it produces an error.

When I run det/yolov4/train_yolov4.sh configs/yolov4/bop_pbr/yolov4x_640_augCosyAAEGray_ranger_lm_pbr_lm_test.py 0.After it has finished training and the model is saved, an error is generated.I set the TOTAL_EPOCHS=4.

As shown in the image:

I downloaded the LM dataset that was using the diagram command:

Thank you for your work, I would like to know if there is a file under lm_renders_blender in the datasets directory. If so, where would I like to download it?

Hi author,

Thank you for your great work!
I'd like to know whether you train a network for each object, or all objects together?

Best,
Leroy

Hi!

Thanks for the amazing project and for releasing the code!

I have a question regarding a specific step of DeepIM during testing. If I understand correctly, during training, the DeepIM is trained on synthetic data, with perturbed ground truth pose as supervision. And during testing on real test data, the init_poses will be loaded to provide both estimated pose and bbox.

So I wonder, how do you acquire the init_poses that are used for test time. From the files in this, is it correct that the init_pose files are generated by the prediction of PoseCNN from here?

Thanks!

Hello, I found a missing model while training Self6D++after pre training_ Final_ Wo_ The optim-e8c99c96.pth file. This file can also be found in the trained models you provided, but I would like to ask if this file is a model trained through the GDR-Net network? I did not generate this file after pre training. Can I get your answer

Hi,

Can you share a link to the dataset and model files that one can download using wget? It is much more convenient to download large files using wget than via a browser.

Thanks!
Rajat

  File "/cluster/personal/self6dpp/self6dpp/core/gdrn_modeling/../../core/gdrn_modeling/datasets/lm_pbr.py", line 185, in __call__
    assert osp.exists(xyz_path), xyz_path
           │   │      │          └ '/cluster/personal/self6dpp/self6dpp/datasets/BOP_DATASETS/lmo/train_pbr/xyz_crop/000000/000000_000006-xyz.pkl'
           │   │      └ '/cluster/personal/self6dpp/self6dpp/datasets/BOP_DATASETS/lmo/train_pbr/xyz_crop/000000/000000_000006-xyz.pkl'
           │   └ <function exists at 0x7f08c2b6e0e0>
           └ <module 'posixpath' from '/opt/conda/lib/python3.7/posixpath.py'>

AssertionError: /cluster/personal/self6dpp/self6dpp/datasets/BOP_DATASETS/lmo/train_pbr/xyz_crop/000000/000000_000006-xyz.pkl

when I run training on LM pbr datasets it shows this bug. Thank you!

Hi,
Thank you for this interesting paper!
I'd be interested to know whether you've ran any performance tests on video footage to investigate whether the model would be applicable in a real-time scenario?

I run core/gdrn_modeling/train_gdrn.sh resnest50d_a6_AugCosyAAEGary_BG05_mlBCE_DoubleMask_lm_pbr_100e_ape.py 0 and it produces an error.

I met an error when training the selff6dpp.
No such file or directory: '/home/self6dpp/datasets/BOP_DATASETS/lm/models_scaled_f5k/ape/textured.obj'

First of all, thank you very much for sharing, but when will you release a code?

Hi, Thanks for sharing the codes.
I am not sure whether my setting is right. When I run :
core/self6dpp/train_self6dpp.sh configs/self6dpp/ssLM/ss_v1_dibr_mlBCE_FreezeBN_woCenter_woDepth_refinePM10/ss_v1_dibr_mlBCE_FreezeBN_woCenter_woDepth_refinePM10_ape.py 0
It shows:

So how to set the config file correctly?

Hi there,

First of all, congratulations on your great work and making it available on github!

I have a question regarding the ablation study of your paper on Occluded Linemod and the results of Table 3.
If I understand the results correctly (please correct me if I'm wrong), the row OURS(LB) + Dref is the performance of the baseline algorithm only with the additional refiner in the teacher-student training paradigm. These results show that this addition already achieves an average recall of 62.1% and the other branches "only" add 2.6% to reach the top performance of 64.7%.

So could it be that the main performance gain simply comes from the refiner? That is, the refiner's capabilities are transferred to the GDR net in this way, and therefore the additional self-supervised learning from the other branches from the unannotated real data is actually minimal?

In other words, if we were to compare the performance of your best version with just the GDR-net and a downstream refiner (both trained on synthetic data only), would we get similar results?

If I understand correctly, this is what happened in the results in Table 7. The results suggest that the difference between GDR-Net with a downstream refiner and your self-supervised method is not really significant.

Would be nice to hear your opinion on that, as such an interpretation could influence further research :). Thx in advance!

Best regards,

Philipp

Hi~ thanks for publishing your code. The self-training mechanism is a good inspiration in pose estimation without real annotations.
I ran your code to reproduce the results on Linemod dataset. However, the final result is slightly worse than the results on paper(just a little bit). I trace the results during the whole training procedure and record the best results. The best results seem to match the results on paper. I wonder if you simply report the final result on the paper or do some other chosen like me.
Appreciate it if you can give me some advice. Maybe I omitted something.

Hi!

Thanks for clarifying the init_poses problem. I would like to know whether you use the differentiable renderer based on kaolin v0.1 as in your Self6D paper (since I find the mode is set to be "VertexColorBatch", and the VCRenderBatch renderer uses linear_rasterizer that uses the old kaolin.graphics library here ).

And I find that there would be conflicting dependencies between the detectron2 and the kaolin v0.1, so I would love to know how you solved this issue. (also maybe missing files for DIBR if you actually used a newer version kaolin)

Thanks!

你好 有个问题想请教您一下：
当我在读论文时了解到6DOF 姿态估计有三种主流的方法，分别是基于关键点correspond 、模板匹配的方法‘投票的方法。但在读2021年新的论文时有许多涉及到直接回归的方法，所以想请教你一下这种直接回归的方法是属于他们其中的一个吗？还是另外的一种方法。
并且在训练时要知道数据集的哪些信息，比如采用correspond的方法我们可以要知道特征点，选择模板匹配的方法我们要知道每个模板图的6D位姿和摄像机参数。那么直接回归的方法对于训练集通常有哪些要求吗？

Hi, I encountered this issue while running the command 'core/self6dpp/train_self6dpp. sh'. Can I seek your help

Hi, Thanks for releasing the code :)
I am wondering if you could upload the pre-computed 2d bbox from yolov4 (datasets/BOP_DATASETS/lmo/test/test_bboxes/yolov4x_640_test672_augCosyAAEGray_ranger_lmo_pbr_lmo_bop_test_16e.json).

I was using your old pre-computed box from GDRN (datasets/BOP_DATASETS/lmo/test/test_bboxes/faster_R50_FPN_AugCosyAAE_HalfAnchor_lmo_pbr_lmo_fuse_real_all_8e_test_480x640.json) and the results were slightly worse than the paper. Thought it could be the problem of the detector :)

Thanks!

When I run det/yolov4/train_yolov4.sh configs/yolov4/bop_pbr/yolov4x_640_augCosyAAEGray_ranger_lm_pbr_lm_test.py 0, It produces an error. I found that there are no “sphere_synt.py” files under the “def” directory.How can I fix it?

Hi, appreciate for opening your code! It really helps me a lot. You have released the gdrn and self6dpp model, can you also release the trained deepim model?

So I want to know time to train a model. For example , trained 50 epoch may spend how much time.

Hi,

Could you please specify this value? I cannot find it in the config file.

Best regards,
Rui Cao

May I ask if your dataset was generated during the pre training process，Do I need to tailor the real LM dataset myself