mcg-nju / ddm Goto Github PK

请问你们会开源CSN+DDM-Net模型吗？

I can’t understand the" k400_train_raw_annotation.pkl and k400_val_raw_annotation.pkl",for I don't understand why it has included f1_consis.
If I want to use my own data. How can I prepare the data for train.

I used the video __NrybzYzUg_000415_000425.mp4 and followed guide.md to prepare data
Ran test.py with the Namespace(batch_size=128, data_dir='', dataset='kinetics_multiframes', model='multiframes_resnet', no_resume_opt=False, num_classes=2, pred_output='./multif-pred_outputs', rank=0, resume='../checkpoint.pth.tar', train_split='train', val_split='val')
Got the error and didn't know the reason
Traceback (most recent call last):
File "D:\DFL_BASE\DDM-main\DDM-Net\test.py", line 162, in
main()
File "D:\DFL_BASE\DDM-main\DDM-Net\test.py", line 115, in main
outps, _, _ = model(inps.cuda(non_blocking=True))
File "C:\ProgramData\Anaconda3\envs\DDM\lib\site-packages\torch\nn\modules\module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "C:\ProgramData\Anaconda3\envs\DDM\lib\site-packages\torch\nn\parallel\data_parallel.py", line 159, in forward
return self.module(*inputs[0], **kwargs[0])
File "C:\ProgramData\Anaconda3\envs\DDM\lib\site-packages\torch\nn\modules\module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "D:\DFL_BASE\DDM-main\DDM-Net\modeling\resnetGEBD.py", line 670, in forward
intra_rgb_feat = self.intra_transformer1(x4, pos)[-1].permute(0, 2, 1)
File "C:\ProgramData\Anaconda3\envs\DDM\lib\site-packages\torch\nn\modules\module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "D:\DFL_BASE\DDM-main\DDM-Net\modeling\transformer.py", line 67, in forward
tgt, src, memory_key_padding_mask=None, pos=pos_embed, query_pos=query_embed
File "C:\ProgramData\Anaconda3\envs\DDM\lib\site-packages\torch\nn\modules\module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "D:\DFL_BASE\DDM-main\DDM-Net\modeling\transformer.py", line 123, in forward
query_pos=query_pos,
File "C:\ProgramData\Anaconda3\envs\DDM\lib\site-packages\torch\nn\modules\module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "D:\DFL_BASE\DDM-main\DDM-Net\modeling\transformer.py", line 300, in forward
query_pos,
File "D:\DFL_BASE\DDM-main\DDM-Net\modeling\transformer.py", line 222, in forward_post
key=self.with_pos_embed(memory, pos),
File "D:\DFL_BASE\DDM-main\DDM-Net\modeling\transformer.py", line 185, in with_pos_embed
return tensor if pos is None else tensor + pos
RuntimeError: The size of tensor a (10) must match the size of tensor b (11) at non-singleton dimension 0

Thanks

The code here should be written like this. I am looking forward to you can proofread it.

    def forward(self, locations):
        result = (
            # self.position_table[: locations.shape[1]]
            self.position_table[:, :locations.shape[1], :]
            .clone()
            .detach()
            .repeat(locations.shape[0], 1, 1)
        )
        return result

您好，请教一个问题！

这里（https://github.com/MCG-NJU/DDM/blob/main/GUIDE.md）的 ”1-c“ 步，分割出10s的视频片段，是用 ffmpeg么

命令比如： ffmpeg -ss 00:01:38 -i input.mp4 -t 00:00:10 -vcodec copy -acodec copy output.mp4

Hi, this project is great and thanks for releasing the code!
I've re-trained DMM and the evaluation result on GEBD val set is as follows, which is around 2% lower than the reported result.

+GEBD Performance on Kinetics-GEBD----+--------+--------+--------+--------+--------+--------+--------+--------+
| Rel.Dis. | 0.05 | 0.10 | 0.15 | 0.20 | 0.25 | 0.30 | 0.35 | 0.40 | 0.45 | 0.50 | Avg |
+----------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
| F1 | 0.7447 | 0.8252 | 0.8496 | 0.8615 | 0.8679 | 0.8722 | 0.8750 | 0.8774 | 0.8796 | 0.8817 | 0.8535 |
+----------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+

I've also tried loading the trained weights you've released and run the evaluation again, the result is still around 2% lower, which is,

+GEBD Performance on Kinetics-GEBD----+--------+--------+--------+--------+--------+--------+--------+--------+
| Rel.Dis. | 0.05 | 0.10 | 0.15 | 0.20 | 0.25 | 0.30 | 0.35 | 0.40 | 0.45 | 0.50 | Avg |
+----------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
| F1 | 0.7462 | 0.8234 | 0.8462 | 0.8578 | 0.8642 | 0.8684 | 0.8715 | 0.8739 | 0.8758 | 0.8776 | 0.8505 |
+----------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+.

I would really appreciate it if you could provide any insights on possible reasons of this. Thanks a lot!

The steps involving for creating the dataset that has been mentioned in the args default="kinetics_multiframes"

In the validation set, IjFrO11sQng.mp4 and SaJWnqViSLo.mp4 are corrupted videos. Can you provide these two videos? Thanks

大佬您好，想请教一下，如果想要测试一个单独的视频应该要用哪个代码？谢谢啦

When I try to use --balance-batch option, there is a problem.

In MultiFDataset
def _get_training_samples(self, index): indices = [] for class_ in self.labels_set: real_index = self.label_to_indices[class_][int(index * self.ratios[class_])] indices.append(real_index) return indices

at the part of (index * self.ratios[class_]), their range is over the self.label_to_indices itself.