er-muyue / bemapnet Goto Github PK
View Code? Open in Web Editor NEWLicense: Other
License: Other
Thanks for your interesting work!
I wonder how you split the dataset by different conditions.
Which part of information do you use in the nuScenes dataset?
Hi, @er-muyue, thanks for your great work!
I've noticed that there is a huge performance gap between results of different evaluation protocol, but details of your own evaluation protocol are missing. Could you please provide more information? Thanks a lot!
i can access the npy file after load it, it give me shape = ()
and i can t visualize it
Which nuscene data set I should download?
I have tried to reproduce the Pivot Dynamic Matching algorithm of PivotNet. But I find that it is very slow during training. Could you please give some optimization ideas or have I made some mistakes in my reproduction? Thanks in advance!
When will you plan to open your codes?
When I run the bezier_converter, it said "No such file or directory: './data/nuscenes/maps/expansion/boston-seaport.json". How can I fix it?
For segmentation head, why you set the output channel = 2, which means [b, 2, 400, 200], but then when calculating the loss function you only use dt_masks[:, 1, :, :] ? Why don't you predict only one channel, like [b, 1, 400, 200] ? Thanks
Hi,@er-muyue
Thanks for your greak work! I notice that you push a release version, but it's empty. Would you please open source the code again?
How can I visualize the inference result and generate the video as you showed?
Hi, thanks for the code. Could you help me understand more why you do cross attention first in the instance decoder, instead of self-attention? As some architecture like DETR, they run with self-attention first. Thanks a lot.
@er-muyue Hi, thanks for your great work. PDM elaborates on the point-level matching for PivotNet, but I did not find the details about how to do instance-level matching in the paper.
Specifically, PDM assumes that "We consider the point matching problem between a predicted sequence Sˆ = {vˆn} and a ground truth sequence S^p = {v_n}", but what is your approach to assigning a ground truth sequence to a prediction?
Could you please explain more? Thanks.
Hi, thanks for your greate work, when will you release the codes.
Looking forward to your reply, thanks! @er-muyue
Hello,
Im trying to run training process with the command mentioned in readme.
bash run.sh train bemapnet_nuscenes_swint 30
I completed all of the setup steps and my project directory same as mentioned in readme.
But i have this error
assets/weights/upernet_swin_tiny_patch4_window7_512x512.pth
Backbone missing_keys: []
Backbone unexpected_keys: ['norm0.weight', 'norm0.bias']
Traceback (most recent call last):
File "configs/bemapnet_nuscenes_swint.py", line 314, in
BeMapNetCli(Exp).run()
File "/home/adastec/catkin_bemapnet/BeMapNet/bemapnet/engine/core.py", line 172, in run
self.dispatch(self.executor)
File "/home/adastec/catkin_bemapnet/BeMapNet/bemapnet/engine/core.py", line 169, in dispatch
executor_func()
File "/home/adastec/catkin_bemapnet/BeMapNet/bemapnet/engine/core.py", line 153, in executor
self.get_trainer().train()
File "/home/adastec/catkin_bemapnet/BeMapNet/bemapnet/engine/core.py", line 143, in get_trainer
trainer = Trainer(exp=exp, callbacks=callbacks, logger=logger, evaluator=evaluator)
File "/home/adastec/catkin_bemapnet/BeMapNet/bemapnet/engine/executor.py", line 104, in init
super(Trainer, self).init(exp, callbacks, logger)
File "/home/adastec/catkin_bemapnet/BeMapNet/bemapnet/engine/executor.py", line 62, in init
self._invoke_callback("after_init")
File "/home/adastec/catkin_bemapnet/BeMapNet/bemapnet/engine/executor.py", line 93, in _invoke_callback
func(self, *args, **kwargs)
File "/home/adastec/catkin_bemapnet/BeMapNet/bemapnet/engine/environ.py", line 91, in after_init
ranks = np.arange(self.world_size()).reshape(-1, self.sync_bn)
ValueError: cannot reshape array of size 1 into shape (8)
[Epoch]: 0%| | 0/30 [00:00<?, ?it/s
Python version: Python 3.8.10
CUDA Version: 11.8
Torch version: 2.0.1+cu117 (also tried with 1.10.1+cu111)
Hi, thanks for your great work!
May I please ask when would the source code of pivotnet be released?
Looking forward to your reply! @er-muyue
Hi~ It seems that the model does not converge at 4 gpus? Do you have any ideas about this problem? And can you tell me how the thickness of semantic lane affects the performance?
Thanks for your works!
If I want to set batch size to 2 in run.sh
, do I need to set sync_bn to 16 (8*2) ?
Looking forward to your kindly reply!
When will you plan to open your codes?
We use your BeMapNet-SwinT checkpoint to evaluate nuScenes val, but get results which is different from your github results.
Ours: mAP@EASY 70, mAP@HARD 55.3
Yours: mAP@EASY 67, mAP@HARD 49.1
Our test setting and environments
python==3.8, torch==1.9.1, CUDA==11.4, mmcv-full==1.4.0, pillow==8.1.2, numpy==1.21.6, detectron2(build from source)
We use 1*A6000 to test
@er-muyue Hi, thanks for your great work. About function get_embedding in position_encoding.py, why just do xy_world_coords = xy_world_coords * depth.unsqueeze(-1), without add trans_xy
how to visualize the npy file ? can you provide a script for visualization
Hi, your job is really impressive, here is one question I wanna figure out.
From the eval results given in BeMapNet & Pivot Net, it seems that BeMapNet actually have better results than PivotNet, is that true? Why the results from BeMapNet were not compared with pivot net? If there is any different, could you please point out? Thank you
Thank you for opening this to the public. Greate work!
I tried testing the published weight but couldn't get it to work. The error is as follows:
2023-08-23 15:46:08.332 | INFO | ...engine.callbacks:load_checkpoint:110 - Loading parameters from checkpoint assets/weights/bemapnet_nuscenes_swint_ep110_model.pth
.../site-packages/torch/functional.py:445: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at ../aten/src/ATen/native/TensorShape.cpp:2157.)
return _VF.meshgrid(tensors, **kwargs) # type: ignore[attr-defined]
assets/weights/upernet_swin_tiny_patch4_window7_512x512.pth
Backbone missing_keys: []
Backbone unexpected_keys: ['norm0.weight', 'norm0.bias']
Traceback (most recent call last):
File "configs/bemapnet_nuscenes_swint.py", line 314, in <module>
BeMapNetCli(Exp).run()
File ".../engine/core.py", line 172, in run
self.dispatch(self.executor)
File ".../engine/core.py", line 169, in dispatch
executor_func()
File ".../engine/core.py", line 148, in executor
self.get_evaluator().eval()
File ".../engine/core.py", line 190, in get_evaluator
evaluator = BeMapNetEvaluator(exp=exp, callbacks=callbacks, logger=logger)
File ".../engine/executor.py", line 190, in __init__
super(BeMapNetEvaluator, self).__init__(exp, callbacks, logger)
File ".../engine/executor.py", line 61, in __init__
self._invoke_callback("load_checkpoint")
File ".../engine/executor.py", line 93, in _invoke_callback
func(self, *args, **kwargs)
File ".../engine/callbacks.py", line 134, in load_checkpoint
trainer.optimizer.load_state_dict(checkpoint["optimizer_state"])
KeyError: 'optimizer_state'
When I change the training batch size to 2
python3 configs/"${CONFIG_NAME}".py -d 0-15 -b 2 -e ${NUM_EPOCHS} --sync_bn 16 --no-clearml
I got the following error
RuntimeError: Boolean value of Tensor with more than one value is ambiguous
Can I train with batch size 2?
It worked fine when I use batch size 1 (-b 1)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.