尊敬的作者您好，您的这篇文章非常棒，我在复现的时候发现您分享的X-101特征文件实在太大了，您可否分享一下您处理后的特征文件呢，我看您在其它issue里说您处理后的特征文件大概30-50G，您可否通过百度网盘或者谷歌网盘分享一下呢，您如果空间不够的话我可以给您提供足够大的云盘账号，非常感谢！

This is too many bugs, I gave up

作者您好，代码中交叉熵的batchsize设置为50，但强化学习时batchsize除以5，这样缩小batchsize的原因是什么呢？这样的操作会使读取速度降低。请问这样有什么依据吗？期待您的回复。

I use conda on Google Colab, not my local.

When I was creating the environment, It couldn't found these packages:

I try to delete these dependencies from environment.yml, and start to create the environment again, it was successfully installed, but when I activate the m2release env, an error happened:

I can't figure out what's wrong, hope someone can help me to solve it.

Much thanks.

您好，很荣幸能够拜读到您的论文并且欣赏您的代码，想请问一下，由于特征文件300多个g过于庞大，我们如果只想进行captioning验证推理的话一定需要该特征文件吗？然后就是，是否可以通过coco2014数据集直接生成特征文件，避免下载过程。

非常感谢

您好！请问您是怎么提取的在线特征啊，我在使用grid-feats-vqa .提取时会报错：

[06/16 14:17:39 fvcore.common.checkpoint]: [Checkpointer] Loading from others/X-101.pth ...
Traceback (most recent call last):
  File "/home/bwh/anaconda3/envs/m2release/lib/python3.6/site-packages/detectron2/data/catalog.py", line 55, in get
    f = DatasetCatalog._REGISTERED[name]
KeyError: 'coco_2014_test'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/bwh/python/d_3_1/others/extract_region_feature.py", line 139, in <module>
    data_loader = build_detection_test_loader_with_attributes(cfg, dataset_name)
  File "/home/bwh/python/d_3_1/others/grid_feats/build_loader.py", line 88, in build_detection_test_loader_with_attributes
    else None,
  File "/home/bwh/anaconda3/envs/m2release/lib/python3.6/site-packages/detectron2/data/build.py", line 224, in get_detection_dataset_dicts
    dataset_dicts = [DatasetCatalog.get(dataset_name) for dataset_name in dataset_names]
  File "/home/bwh/anaconda3/envs/m2release/lib/python3.6/site-packages/detectron2/data/build.py", line 224, in <listcomp>
    dataset_dicts = [DatasetCatalog.get(dataset_name) for dataset_name in dataset_names]
  File "/home/bwh/anaconda3/envs/m2release/lib/python3.6/site-packages/detectron2/data/catalog.py", line 59, in get
    name, ", ".join(DatasetCatalog._REGISTERED.keys())
KeyError: "Dataset 'coco_2014_test' is not registered! Available datasets are: coco_2014_train, coco_2014_val, coco_2014_minival, coco_2014_minival_100, coco_2014_valminusminival, coco_2017_train, coco_2017_val, coco_2017_test, coco_2017_test-dev, coco_2017_val_100, keypoints_coco_2014_train, keypoints_coco_2014_val, keypoints_coco_2014_minival, keypoints_coco_2014_valminusminival, keypoints_coco_2014_minival_100, keypoints_coco_2017_train, keypoints_coco_2017_val, keypoints_coco_2017_val_100, coco_2017_train_panoptic_separated, coco_2017_train_panoptic_stuffonly, coco_2017_val_panoptic_separated, coco_2017_val_panoptic_stuffonly, coco_2017_val_100_panoptic_separated, coco_2017_val_100_panoptic_stuffonly, lvis_v0.5_train, lvis_v0.5_val, lvis_v0.5_val_rand_100, lvis_v0.5_test, lvis_v0.5_train_cocofied, lvis_v0.5_val_cocofied, cityscapes_fine_instance_seg_train, cityscapes_fine_sem_seg_train, cityscapes_fine_instance_seg_val, cityscapes_fine_sem_seg_val, cityscapes_fine_instance_seg_test, cityscapes_fine_sem_seg_test, voc_2007_trainval, voc_2007_train, voc_2007_val, voc_2007_test, voc_2012_trainval, voc_2012_train, voc_2012_val, visual_genome_train, visual_genome_val, visual_genome_test"

hello,author.I have two question about your code.
1.In the field.py, you import default_collate,but there is no default_collate in the dataloader.

2.Could you tell me how to deal the User warning,thank you.

您好，非常感谢您能开源论文的代码。您提供的X-101-features特征有326G, 我下载时总是因为网络中断而下载失败，请问您有百度网盘版的特征链接吗？

您好请问可以您提取grid特征的代码是基于https://github.com/rentainhe/TRAR-Feature-Extraction 这个项目做的么，如果不是的话可以分享一下您提取grid特征的代码么

您好，请问models/rstnet/language_model.py第4行，BertModel模块如何引入？显示未找到transformers模块

Thank you for opening sources.
Could you provide the paper for this project?

您好，我在您所开源的代码中注意到有‘’TransformerEnsemble‘’这个类，请问应该如何使用它训练及测试指定数量的ensemble模型呢（比如指定模型是 ensemble 2 或者ensemble 4）？

非常感谢作者能够将这么好的工作开源出来，并且还提供了 online test 的代码。但是我在 online test 代码中发现模型有4个权重，

请问你论文中 online test 的结果是多个模型集成后的结果吗，我在你的论文中没有找到相关的描述。非常期待你的回复！

请问应该如何解决，非常感谢

Hi, that's very interesting work that reuses the grid feature again.

I want to test and reuse the pre-trained model for some tasks,
However, according to the instruction in README, the file is currently inaccessible.
Would u mind uploading the pre-trained model file again?

Thx alot~

Hello, thanks for sharing your code!
But I wonder how to get a image's caption like figure 1(b) or 6 in your artical
Could you share me the code ?
Thanks a lot!

When I run train_transformer.py, I'm having difficulty, showing RuntimeError: gather(): Expected dtype int64 for index, can anyone tell me how to solve it?
thank you very much

师兄您好，非常有幸能看到您团队的这篇工作，我对该工作非常有兴趣，但是在制作hdf5特征时遇到了一些问题，特此想请教一下您：
我在解压了X-101-features.tgz后，运行feats_process.py时遇到了一点意外。程序报错截图如下，请问如何解决呢？
非常期待您的回复，感谢！

请问这个问题怎么解决

非常高兴看到您出色的工作，给我带来了很大帮助。在测试阶段我遇到了一个问题，在训练阶段，预训练语言模型通过融合所有前文来预测下一个单词，但是在测试中模型似乎只能通过上一步的一个单词来预测，不知道是不是我的理解有误，如果确实是这样，这种测试方法是否会降低影最终的效果？

Thank you for your work and open source code. I use your code to train the rstnet, and I get the language model by myself. However, at the end of the cross-entropy training stage, I get the CIDEr score about 103, is this normal？
How much the CIDEr score should be expected at the end of the cross-entropy training stage? And how many epochs should be trained during reinforcement learning？

Hello, thanks for sharing the code. This error occurs when I am running offline tests. What happens? I haven't found a solution yet. Thank you very much for your help.

Hello author, thank you very much for sharing your source code with us. I have two questions I would like to ask you about the position encoding in your model.

1. Are the two parts of the image pointed by the red arrow equivalent?
   Position encoding is mentioned only once in the picture，but in the section 3.2 of the paper, there is another position encoding mentioned in Eq. (8).
2. How position encoding is performed?
   In Figure 3 of the paper, Does the position encoding occur between the bert module and the Masked Multi-Head Attention module?
   
   

作者您好！很庆幸能看到你的作品！我在自己的电脑调试项目的时候，出现了很多意想不到的问题，我刚接触cv+nlp领域，有些方面还不太了解，想请教一下您

1. readme中提到代码是在python3.6环境下运行的，我使用的是Windows系统的python3.9 是否会因为版本不一样导致运行时报错？

2. 在train_transformer中，dataloader_train函数中shuffle如果为true，会产生ValueError: DataLoader with IterableDataset: expected unspecified shuffle option, but got shuffle=True
   把shuffle改成false之后，会出现下面的报错（ File "E:\Anaconda\lib\multiprocessing\reduction.py", line 60, in dump
   ForkingPickler(file, protocol).dump(obj)
   TypeError: 'generator' object is not callable），想问问是否是上面的版本系统不兼容的问题引起的呢？
   

3. X-101_features中使用的是关于2014和2015的pth文件，在这个条件下用自己的数据集（例如说COCO2017版本的数据集）会不会有影响，如果要用自己的数据集需要进行什么修改吗？

期待作者您的回复，非常感谢！

Hello，thank you for sharing you code. I have checked the size of vocab.pkl and found that it is 10199 not 10201. I am strange to this field so could you please tell me what the other two token are?

您好，当我运行train_transformer.py文件时, "from middle import TensorRecorder"这句出现了问题. ImportError: cannot import name 'TensorRecorder'. 我已经安装了middle包, 但是不知道错误出在哪里。

Hello, I am interested in your visualness word. How will deal with the token that haven't occured for one image?

您好，我在使用spice评估时，调用spice-1.0.jar包时总会发生CalledProcessError的错误，很疑惑

The link of rstnet.pth is invalid.

非常感谢你能开源论文的代码。但是在 evaluation 中，并没有计算 spice 分数的相关代码，我想知道你是如何得到 spice 分数的？

Thanks for this amazing work.
Could you provide the inference captions of COCO test set by using the method proposed in this paper.
It is hard to download the features used in this paper due to its large size.
Thanks again.

$ wget https://drive.google.com/file/d/1sayx7qwOd79XE4RFdvSXG3zyQH4FpyYJ/view?usp=sharing
--2022-03-23 18:09:05-- https://drive.google.com/file/d/1sayx7qwOd79XE4RFdvSXG3zyQH4FpyYJ/view?usp=sharing
Resolving drive.google.com (drive.google.com)... 31.13.95.37, 2001::68f4:2b68
Connecting to drive.google.com (drive.google.com)|31.13.95.37|:443... failed: Connection timed out.
Connecting to drive.google.com (drive.google.com)|2001::68f4:2b68|:443... failed: Network is unreachable.
，网盘账号是15603379965

您好，非常感谢您的开源代码！
您的预训练模型链接失效了，能麻烦您重新上传一下吗？

您好！请问用于生成在线评测的代码是否包含在项目内呢？还是需要我们自己去编写呢

Dear author @zhangxuying1004 ,
It is a wonderful work. If my own dataset have not Object Detection Annotation, how can i use your model?

@zhangxuying1004 where is the supplementary material that you talk about in section 3.1 about specific calculations of 2d relative position of each grid ?

你好，我运行 test.py 文件时，模型只会生成 <'unk'> tokens，没有其他单词，请问是什么原因呢

您好，非常感谢开源您的代码。
您所分享的X-101-features特征，我在解压该文件时遇到很大困扰，它首先解压出800多GB的tar压缩包，而后提取其中的pth文件，2TB内存都不能成功解压，请问有处理后的特征吗？可以分享我一下吗？
另外论文中的语言模型特征是要自己训练吗？

Thanks for this amazing work.
Could you provide the x152 pretrained grid feature h5 file used in this paper?

你好，你所的工作很不错，对此很感兴趣。但是我按照你所给的提示运行了代码，在在RL训练前是没有问题的，但是进行RL训练后模型的性能急剧下降，最终的cider为0.009，我不清楚是那里出现问题了，能否给一些意见

The connection seems to be down. could u please give me another url to download the image_info_test2014.json, thanks a lot

Hello,author,Thanks for your sharing,What is the reason for this error？

您好，我看到您在原论文3.1章节中提到具体计算过程在补充材料中，但是我在CVPR官网中没有找到补充材料对应的下载链接，能麻烦您提供一下补充材料吗

请问，为什么强化学习阶段训练损失是负数啊？

非常抱歉再次打扰你，我使用你开源的 test_online 代码生成两个 json 文件并提交后，并没有顺利给出分数，反而报错了：

请问你有遇到过这个问题吗。我是在 win11 系统下把两个 json 文件放到 result 文件夹下，然后打包成 zip 文件：

是我文件结构有问题吗？非常期待你的回复

你好，可以发一下X152的特征文件吗？

Hello, thank you for sharing your code and pretrained model. But when I use your language pretrained model 'language_context.pth', I received the following error. Could you please tell me how to solve it? Thank you very much!