afeng-x / smt Goto Github PK

Why did the training time for 10 batches increase several times when replacing the backbone network from CrossFormer-S to SMT-T, despite SMT-T having much fewer parameters and computations than CrossFormer-S?

Now the biggest model is smt_large, Can you design a bigger model like smt_huge?
Thank you very much!

Hello, author, how to solve the following problem:

ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 0 (pid: 2694) of binary: /usr/bin/python
Traceback (most recent call last):
File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/usr/local/lib/python3.6/dist-packages/torch/distributed/launch.py", line 193, in
main()
File "/usr/local/lib/python3.6/dist-packages/torch/distributed/launch.py", line 189, in main
launch(args)
File "/usr/local/lib/python3.6/dist-packages/torch/distributed/launch.py", line 174, in launch
run(args)
File "/usr/local/lib/python3.6/dist-packages/torch/distributed/run.py", line 713, in run
)(*cmd_args)
File "/usr/local/lib/python3.6/dist-packages/torch/distributed/launcher/api.py", line 131, in call
return launch_agent(self._config, self._entrypoint, list(args))
File "/usr/local/lib/python3.6/dist-packages/torch/distributed/launcher/api.py", line 261, in launch_agent
failures=result.failures,
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:

Thank you very much for sharing your great work, in the paper I noticed that DW_Conv were used in the SAM module. If I use vanilla convolution instead of DW_Conv, will it have a performance improvement in addition to growing parameters?

Hi author, I didn't find the SAM block in the code, please tell me where it is, thanks!

I am not sure I train rightly.
some setting:

• opt: adamw
• baselr: 1e-3 warmup first 5 epoch
• batch_size : 128
• total epoch: 150 (not 300 because of too long gpu time with 10 days, 150 epoches ,4 T4 gpus)
• lr_scheduler : cos from 5th epoch
  now trained 40 epoches ,65% top1 ,and increase rate slowly per epoch.
  and except total epoch(150) less than default 300;others are almost same as git .
  I want to know 2 information:
  1、Will total epoch that set 150 be big influence on final result ?
  2、Is it normal that 40 epoches ,65% top1 ? bacause training need long time ,I want to get some information before. If it is wrong,I will stop it early.

Hi @AFeng-x, thanks for sharing the great SMT work!

I'd like to bring up another highly related hierarchical vision transformer that also deals with the scale problems: MaxViT: Multi-Axis Vision Transformer [ECCV 2022]. I'm wondering if you could also add our work to your comparison figure and tables? Thanks a lot!

作者你好，非常感谢你们的工作，但是我没有找到如何可以进行语义分割训练的文件

这个ckpt.pth找不到 麻烦可以提供一下吗 谢谢

Hi there! This is a nice work. But I have a little query about the motivation of the architecture design. In the paper "Based on the research conducted in [11, 4], which performed a quantitative analysis of different depths of self-attention blocks and discovered that shallow blocks tend to capture short-range dependencies while deeper ones capture long-range dependencies".

From my knowledge, the transformer can always model globally and can capture high effective receptive field from initial stages. Why did you refer that the shallow blocks capture short-range dependencies and deeper ones capture long-range dependencies? Why did they both capture long-range? Why did shallow capture short range but deep capture long-range?

Thank you for your great work. I'm quite interested in your visualization works. It will contribute much to the research community.
It would be better if the authors could provide the code to visualize feature maps across heads and how to compute and draw relative receptive fields.
Thank you so much in advance.

Thanks for your team's efforts. Could you please share this weight file？

Hi,thank you for your work. Can you share the pretrain weights for segmentation task？

Thank you for your sharing, may I ask how the figure 4 in the paper is drawn? Can you provide the code?

Hi author, your code is great, but when I introduce your SMT module for training I always have a case of not enough memory in the attn = (q @ k.transpose(-2, -1)) * self.scale statement when calculating the Attention, and it's not enough for me to set the Batchsize to 1. Can the author give some ideas how to modify it, please. I'm only using stage3's structure

Your work is so great, thank you for your contribution to science, but I am having problems reproducing it：TypeError: RetinaNet: init_weights() got an unexpected keyword argument 'pretrained' ， may I ask what is your version of mmdet, I am having the above problem with both MaskRCNN and RetinaNet.

您好，我使用您代码中给的vis.ipynb代码，但只画出了如下图一样的图。请问如何画出您论文中Figure 4(a) Different Heads一样的图？