jmoonr / latr Goto Github PK

Why does the training code throw exceptions about 'CUDA out of memory'.

I'd tried runing the code after setting 2-layer decoder in LATR without any other code changes, but the problem is still not solved!

I trained models on 4 x NVIDIA RTX A4000 with 16G memory per GPU.

What type of GPU is required？ And how many GPUs are required？

hello, thank for your excellent work!
I met a n error when i run CUDA_VISIBLE_DEVICES='0,1,2,3' python -m torch.distributed.launch --nproc_per_node 4 main.py --config config/release_iccv/latr_1000_baseline.py,
the error message is:

Traceback (most recent call last):
  File "main.py", line 48, in <module>
    main()
  File "main.py", line 42, in main
    runner.train()
  File "/data/usr/LATR/experiments/runner.py", line 184, in train
    optimizer.step()
  File "/home/usr/miniconda3/envs/latr_/lib/python3.8/site-packages/torch/optim/lr_scheduler.py", line 65, in wrapper
    return wrapped(*args, **kwargs)
  File "/home/usr/miniconda3/envs/latr_/lib/python3.8/site-packages/torch/optim/optimizer.py", line 89, in wrapper
    return func(*args, **kwargs)
  File "/home/usr/miniconda3/envs/latr_/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "/home/usr/miniconda3/envs/latr_/lib/python3.8/site-packages/torch/optim/adamw.py", line 117, in step
    beta1,
UnboundLocalError: local variable 'beta1' referenced before assignment

And then I change the config file as:

optimizer_cfg = dict(
    type='AdamW',
    lr=2e-4,
    # paramwise_cfg=dict(
    #     custom_keys={
    #         'sampling_offsets': dict(lr_mult=0.1),
    #     }),
    weight_decay=0.01)

After changing, the error disappeared.
Why does the above error occur?
Will these changes I make affect performance?
Expect the above method, what should I do to avoid the above error?

Thank you for your great contribution.

I have noticed that differently from the PersFormer study, you seems to no longer use the P_g2gflat in the training pipeline. Have you tried with and without it? Is there any difference between different implementations?

https://github.com/JMoonr/LATR/blob/7bb195e05d4f8f6bb48aa7a30c1222bfd1ea683b/data/Load_Data.py#L210C9-L210C18

Thanks in advance.

Hi,

I've been exploring your project and am really impressed with your work. However, I've been facing challenges with package installations due to a restrictive proxy on my network. I often encounter errors like the 'import nms' error.

Providing a conda environment .yml file could significantly streamline the setup process and enhance the project's accessibility for myself and others in similar situations. Your assistance with this would be greatly appreciated.

Thank you for your consideration and the excellent work on the project.

When attempting to evaluate your pretrained model on the OpenLane dataset, I encounter a RuntimeError indicating a mismatch in the expected dimensions of the input.
RuntimeError: Expected 4-dimensional input for 4-dimensional weight [64, 3, 7, 7], but got 3-dimensional input of size [3, 720, 960] instead
this is the command i used "CUDA_VISIBLE_DEVICES='0' python -m torch.distributed.launch --nproc_per_node 1 main.py --config config/release_iccv/latr_1000_baseline.py --cfg-options evaluate=true eval_ckpt=pretrained_models/openlane.pth"

Hi, guys !
I appreciate the work of you share, When is the DV-3DLane code released？

May I ask if the code for predicting visualization results will be open? If so, thank you

So interesting project！在官方指引下成功跑通了验证流程，请问一下，能否分享一下可视化代码？

Great work!

Do you have some hints on how to implement the visualization you have. I am using gt_lanes and pred_lanes in bench function, but apparently using the transformation matrix (P_g2im) does not give reasonable results.
Thank you!!

mmdet3d==1.0.0rc3 requires mmcv>=2.0.0rc4. But mmcv>=2.0.0rc4 will cause running errors. How to deal with this?

Thank you for the author's great engineering work and the open-source code.
I have a quick question: could you clarify the definition of the camera coordinate system in openlane or latr (before transformation in data/Load_Data.py)? Is it based on the Apollo camera coordinate system, the WOD camera coordinate system, or something else?

Is it a real time work?

感谢大佬的分享，models/latr_head.py代码里面每次都把query设置为0=》query = torch.zeros_like(query_embeds)。这步操作确实看不懂，大佬能否解答一下。

Hi, thanks for your nice work!
When I run your code, there met a problem.
Because my device is unable to connect to the internet, I download the resnet50 pth pretrained on ImageNet instead of using the mm-pretrain model. This means I change the encoder config init_cfg=dict(type='Pretrained', checkpoint='torchvision://resnet50') to init_cfg=dict(type='Pretrained', checkpoint='/data1/pbw/LATR-main/resnet50-19c8e357.pth')
By doing so, I successfully run the code. But it stops at operation "Convert model with Sync BatchNorm", and cannot go on. I have taken a screenshot as follows.

Could you give me some advice? Thanks for your time.

Hi,

Are you planning to share the inference (testing) script for a custom dataset (single image)?

What data is needed to run inference on single image in current version?

Thanks.

Could I ask whether I only need to install mmdet and mmdet3d with pip, or whether I should clone their repositories to perform comprehensive installation?

Extremely grateful!

Hi,

Can you tell me what python version you use in this project?

Thank you very much for your project.

how to solve this problem:
assert (mmcv_min_version <= mmcv_version < mmcv_max_version),
AssertionError: MMCV==1.5.0 is used but incompatible. Please install mmcv>=2.0.0rc4.

I would like to run your model on NuScenes, have you implemented already something? Do you think it could be possible to run it over NuScenes? Especially regarding missing annotation on nuSceens, for example I have seen you are using some extra info in the inference:

output = self.head(
            dict(
                x=neck_out,
                lane_idx=extra_dict['seg_idx_label'],
                seg=extra_dict['seg_label'],
                lidar2img=extra_dict['lidar2img'],
                pad_shape=extra_dict['pad_shape'],
                ground_lanes=extra_dict['ground_lanes'] if is_training else None,
                ground_lanes_dense=extra_dict['ground_lanes_dense'] if is_training else None,
                image=image,
            ),
            is_training=is_training,
        )

I am not quite sure where I can get this extra information from nuscenes, I thought having camera parameters and image would have been enough.
Thank you!! :)

Hello. I am very impressed by your work and I would like to use your method. Could you please share the code or update it on GitHub? Thank you very much.

I wanted to inquire if there are any Docker images available that could be used, as I found some trouble in installing the environment, I would greatly appreciate it if you could provide me with some guidance or pointers.