I want to use spatial TR as an augmentation to improve my code, and i have seen the paper(Sec. V(E)), but i cant find the precise location. Could you help me to find where it is? Thanks a lot.

Hi Chiara,
After reading your paper, I think this is a very meaningful study. I was trying to reproduce the results but I’m can’t find many files.
1.When I try the nut_gendata.py, the error is that No such file or directory:’resource/NTU-RGB-D/samples_with_missing_skeletons.txt’. For this problem, I copied the resource file in st-gcn.
2.When I try the nut_gen_bones.py, the error is that No such file or directory:’./xview/train_data_joint.npy’.
3.When I try the preprocess.py, the error is that No such file or directory:’../data/nturgbd_raw/samples_with_missing_skeletons.txt’.
Can you give advice on how to solve these problems？
Thanks a lot，
best

Hi:
In spatial_transformer.py line 131
if (self.drop_connect and self.training):
mask = torch.bernoulli((0.5) * torch.ones(B * self.Nh * V, device))
mask = mask.reshape(B, self.Nh, V).unsqueeze(2).expand(B, self.Nh, V, V)
weights = weights * mask
Why dose weight multiply mask will drop connect and avoid overfitting?

The data does exist, but why do the following problems occur? We have searched for some reasons, but they have not been solved. We hope to receive your reply.

FileNotFoundError: [Errno 2] No such file or directory: './data/.train_data.npy'

I change the data set into my own data set. After the training, how can I test it

Traceback (most recent call last):
File "/media/max/b/2021/FG_2021/ST-TR-master_FIRST/src/main.py", line 973, in
processor.start()
File "/media/max/b/2021/FG_2021/ST-TR-master_FIRST/src/main.py", line 885, in start
self.train(epoch, save_model=save_model)
File "/media/max/b/2021/FG_2021/ST-TR-master_FIRST/src/main.py", line 529, in train
for batch_idx, (data, label, name) in enumerate(loader):
File "/home/max/anaconda3/envs/GaitGraph/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 517, in next
data = self._next_data()
File "/home/max/anaconda3/envs/GaitGraph/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1199, in _next_data
return self._process_data(data)
File "/home/max/anaconda3/envs/GaitGraph/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1225, in _process_data
data.reraise()
File "/home/max/anaconda3/envs/GaitGraph/lib/python3.7/site-packages/torch/_utils.py", line 429, in reraise
raise self.exc_type(msg)
IndexError: Caught IndexError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "/home/max/anaconda3/envs/GaitGraph/lib/python3.7/site-packages/torch/utils/data/_utils/worker.py", line 202, in _worker_loop
data = fetcher.fetch(index)
File "/home/max/anaconda3/envs/GaitGraph/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/max/anaconda3/envs/GaitGraph/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/media/max/b/2021/FG_2021/ST-TR-master_FIRST/src/st_gcn/feeder/feeder.py", line 127, in getitem
data_numpy = np.array(self.data[index])
File "/home/max/anaconda3/envs/GaitGraph/lib/python3.7/site-packages/numpy/core/memmap.py", line 331, in getitem
res = super(memmap, self).getitem(index)
IndexError: index 23667 is out of bounds for axis 0 with size 16560

For the NTU RGB+D datasets, there are XXX_gendata.py and preprocess_XXX.py which read the same inputs and produce the same outputs. What are their differences and how the results differ if training on the corresponding .npy and .label files generated?

Traceback (most recent call last):
File "code/main.py", line 973, in
processor.start()
File "code/main.py", line 885, in start
self.train(epoch, save_model=save_model)
File "code/main.py", line 529, in train
for batch_idx, (data, label, name) in enumerate(loader):
File "E:\anaconda3\envs\pytorch\lib\site-packages\torch\utils\data\dataloader.py", line 345, in next
data = self._next_data()
File "E:\anaconda3\envs\pytorch\lib\site-packages\torch\utils\data\dataloader.py", line 385, in _next_data
data = self._dataset_fetcher.fetch(index) # may raise StopIteration
File "E:\anaconda3\envs\pytorch\lib\site-packages\torch\utils\data_utils\fetch.py", line 44, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "E:\anaconda3\envs\pytorch\lib\site-packages\torch\utils\data_utils\fetch.py", line 44, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "E:\ST-TR-master\code\st_gcn\feeder\feeder.py", line 142, in getitem
if self.mirroring and bool(torch.bernoulli((0.5) * torch.ones(1))):
AttributeError: 'Feeder' object has no attribute 'mirroring'

Then I tried to output the training dataset:<torch.utils.data.dataloader.DataLoader object at 0x0000025C9B030CD0>
Train size: 10023

I hope you can help me solve this problem. Thanks.

when I run without bones on NTU-RGB-D, this problem has occurred as the follow picture.
Looking forward to your early reply.

torch.Size([32, 3, 150, 18, 2])
torch.Size([32, 3, 150, 18, 2])
Traceback (most recent call last):
File "code/main.py", line 973, in
processor.start()
File "code/main.py", line 885, in start
self.train(epoch, save_model=save_model)
File "code/main.py", line 540, in train
output = self.model(data, label, name)
File "/home/user/anaconda3/envs/st-tr/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/user/anaconda3/envs/st-tr/lib/python3.7/site-packages/torch/nn/parallel/data_parallel.py", line 161, in forward
outputs = self.parallel_apply(replicas, inputs, kwargs)
File "/home/user/anaconda3/envs/st-tr/lib/python3.7/site-packages/torch/nn/parallel/data_parallel.py", line 171, in parallel_apply
return parallel_apply(replicas, inputs, kwargs, self.device_ids[:len(replicas)])
File "/home/user/anaconda3/envs/st-tr/lib/python3.7/site-packages/torch/nn/parallel/parallel_apply.py", line 86, in parallel_apply
output.reraise()
File "/home/user/anaconda3/envs/st-tr/lib/python3.7/site-packages/torch/_utils.py", line 428, in reraise
raise self.exc_type(msg)
RuntimeError: Caught RuntimeError in replica 0 on device 0.
Original Traceback (most recent call last):
File "/home/user/anaconda3/envs/st-tr/lib/python3.7/site-packages/torch/nn/parallel/parallel_apply.py", line 61, in _worker
output = module(*input, **kwargs)
File "/home/user/anaconda3/envs/st-tr/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/data3/2019/liuyuan/ST-TR/code/st_gcn/net/st_gcn.py", line 267, in forward
x = self.data_bn(x)
File "/home/user/anaconda3/envs/st-tr/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/user/anaconda3/envs/st-tr/lib/python3.7/site-packages/torch/nn/modules/batchnorm.py", line 136, in forward
self.weight, self.bias, bn_training, exponential_average_factor, self.eps)
File "/home/user/anaconda3/envs/st-tr/lib/python3.7/site-packages/torch/nn/functional.py", line 2058, in batch_norm
training, momentum, eps, torch.backends.cudnn.enabled
RuntimeError: running_mean should contain 108 elements not 216

I hope you can help me sovle this problem and thank you so much!

Hi Chiara,

I am trying to load the file "checkpoint_ST-TR/ntu120_xsub_spatial.pt" using the following command:

model.load_state_dict(weights)

and I get an error telling me that I have missing keys and unexpected keys in state_dict. From the trace, I see that all the expected keys start with "module.key_name", but when I load and print the keys in the file I don't have this "module." in front of each keys. All the keys look correct but I don't know where this "module." come from. Have you encounter this problem before and do you know how to load this model ?

Thanks in advance
Benjamin

Here is the full trace:

RuntimeError: Error(s) in loading state_dict for DataParallel: Missing key(s) in state_dict: "module.data_bn.weight", "module.data_bn.bias", "module.data_bn.running_mean", "module.data_bn.running_var", "module.backbone.0.gcn1.conv_list.0.weight", "module.backbone.0.gcn1.conv_list.0.bias", "module.backbone.0.gcn1.conv_list.1.weight", "module.backbone.0.gcn1.conv_list.1.bias", "module.backbone.0.gcn1.conv_list.2.weight", "module.backbone.0.gcn1.conv_list.2.bias", "module.backbone.0.gcn1.bn.weight", "module.backbone.0.gcn1.bn.bias", "module.backbone.0.gcn1.bn.running_mean", "module.backbone.0.gcn1.bn.running_var", "module.backbone.0.tcn1.conv.weight", "module.backbone.0.tcn1.conv.bias", "module.backbone.0.tcn1.bn.weight", "module.backbone.0.tcn1.bn.bias", "module.backbone.0.tcn1.bn.running_mean", "module.backbone.0.tcn1.bn.running_var", "module.backbone.1.gcn1.conv_list.0.weight", "module.backbone.1.gcn1.conv_list.0.bias", "module.backbone.1.gcn1.conv_list.1.weight", "module.backbone.1.gcn1.conv_list.1.bias", "module.backbone.1.gcn1.conv_list.2.weight", "module.backbone.1.gcn1.conv_list.2.bias", "module.backbone.1.gcn1.bn.weight", "module.backbone.1.gcn1.bn.bias", "module.backbone.1.gcn1.bn.running_mean", "module.backbone.1.gcn1.bn.running_var", "module.backbone.1.tcn1.conv.weight", "module.backbone.1.tcn1.conv.bias", "module.backbone.1.tcn1.bn.weight", "module.backbone.1.tcn1.bn.bias", "module.backbone.1.tcn1.bn.running_mean", "module.backbone.1.tcn1.bn.running_var", "module.backbone.2.gcn1.conv_list.0.weight", "module.backbone.2.gcn1.conv_list.0.bias", "module.backbone.2.gcn1.conv_list.1.weight", "module.backbone.2.gcn1.conv_list.1.bias", "module.backbone.2.gcn1.conv_list.2.weight", "module.backbone.2.gcn1.conv_list.2.bias", "module.backbone.2.gcn1.bn.weight", "module.backbone.2.gcn1.bn.bias", "module.backbone.2.gcn1.bn.running_mean", "module.backbone.2.gcn1.bn.running_var", "module.backbone.2.tcn1.conv.weight", "module.backbone.2.tcn1.conv.bias", "module.backbone.2.tcn1.bn.weight", "module.backbone.2.tcn1.bn.bias", "module.backbone.2.tcn1.bn.running_mean", "module.backbone.2.tcn1.bn.running_var", "module.backbone.3.gcn1.data_bn.weight", "module.backbone.3.gcn1.data_bn.bias", "module.backbone.3.gcn1.data_bn.running_mean", "module.backbone.3.gcn1.data_bn.running_var", "module.backbone.3.gcn1.bn.weight", "module.backbone.3.gcn1.bn.bias", "module.backbone.3.gcn1.bn.running_mean", "module.backbone.3.gcn1.bn.running_var", "module.backbone.3.gcn1.attention_conv.qkv_conv.weight", "module.backbone.3.gcn1.attention_conv.qkv_conv.bias", "module.backbone.3.gcn1.attention_conv.attn_out.weight", "module.backbone.3.gcn1.attention_conv.attn_out.bias", "module.backbone.3.tcn1.conv.weight", "module.backbone.3.tcn1.conv.bias", "module.backbone.3.tcn1.bn.weight", "module.backbone.3.tcn1.bn.bias", "module.backbone.3.tcn1.bn.running_mean", "module.backbone.3.tcn1.bn.running_var", "module.backbone.3.down1.conv.weight", "module.backbone.3.down1.conv.bias", "module.backbone.3.down1.bn.weight", "module.backbone.3.down1.bn.bias", "module.backbone.3.down1.bn.running_mean", "module.backbone.3.down1.bn.running_var", "module.backbone.4.gcn1.data_bn.weight", "module.backbone.4.gcn1.data_bn.bias", "module.backbone.4.gcn1.data_bn.running_mean", "module.backbone.4.gcn1.data_bn.running_var", "module.backbone.4.gcn1.bn.weight", "module.backbone.4.gcn1.bn.bias", "module.backbone.4.gcn1.bn.running_mean", "module.backbone.4.gcn1.bn.running_var", "module.backbone.4.gcn1.attention_conv.qkv_conv.weight", "module.backbone.4.gcn1.attention_conv.qkv_conv.bias", "module.backbone.4.gcn1.attention_conv.attn_out.weight", "module.backbone.4.gcn1.attention_conv.attn_out.bias", "module.backbone.4.tcn1.conv.weight", "module.backbone.4.tcn1.conv.bias", "module.backbone.4.tcn1.bn.weight", "module.backbone.4.tcn1.bn.bias", "module.backbone.4.tcn1.bn.running_mean", "module.backbone.4.tcn1.bn.running_var", "module.backbone.5.gcn1.data_bn.weight", "module.backbone.5.gcn1.data_bn.bias", "module.backbone.5.gcn1.data_bn.running_mean", "module.backbone.5.gcn1.data_bn.running_var", "module.backbone.5.gcn1.bn.weight", "module.backbone.5.gcn1.bn.bias", "module.backbone.5.gcn1.bn.running_mean", "module.backbone.5.gcn1.bn.running_var", "module.backbone.5.gcn1.attention_conv.qkv_conv.weight", "module.backbone.5.gcn1.attention_conv.qkv_conv.bias", "module.backbone.5.gcn1.attention_conv.attn_out.weight", "module.backbone.5.gcn1.attention_conv.attn_out.bias", "module.backbone.5.tcn1.conv.weight", "module.backbone.5.tcn1.conv.bias", "module.backbone.5.tcn1.bn.weight", "module.backbone.5.tcn1.bn.bias", "module.backbone.5.tcn1.bn.running_mean", "module.backbone.5.tcn1.bn.running_var", "module.backbone.6.gcn1.data_bn.weight", "module.backbone.6.gcn1.data_bn.bias", "module.backbone.6.gcn1.data_bn.running_mean", "module.backbone.6.gcn1.data_bn.running_var", "module.backbone.6.gcn1.bn.weight", "module.backbone.6.gcn1.bn.bias", "module.backbone.6.gcn1.bn.running_mean", "module.backbone.6.gcn1.bn.running_var", "module.backbone.6.gcn1.attention_conv.qkv_conv.weight", "module.backbone.6.gcn1.attention_conv.qkv_conv.bias", "module.backbone.6.gcn1.attention_conv.attn_out.weight", "module.backbone.6.gcn1.attention_conv.attn_out.bias", "module.backbone.6.tcn1.conv.weight", "module.backbone.6.tcn1.conv.bias", "module.backbone.6.tcn1.bn.weight", "module.backbone.6.tcn1.bn.bias", "module.backbone.6.tcn1.bn.running_mean", "module.backbone.6.tcn1.bn.running_var", "module.backbone.6.down1.conv.weight", "module.backbone.6.down1.conv.bias", "module.backbone.6.down1.bn.weight", "module.backbone.6.down1.bn.bias", "module.backbone.6.down1.bn.running_mean", "module.backbone.6.down1.bn.running_var", "module.backbone.7.gcn1.data_bn.weight", "module.backbone.7.gcn1.data_bn.bias", "module.backbone.7.gcn1.data_bn.running_mean", "module.backbone.7.gcn1.data_bn.running_var", "module.backbone.7.gcn1.bn.weight", "module.backbone.7.gcn1.bn.bias", "module.backbone.7.gcn1.bn.running_mean", "module.backbone.7.gcn1.bn.running_var", "module.backbone.7.gcn1.attention_conv.qkv_conv.weight", "module.backbone.7.gcn1.attention_conv.qkv_conv.bias", "module.backbone.7.gcn1.attention_conv.attn_out.weight", "module.backbone.7.gcn1.attention_conv.attn_out.bias", "module.backbone.7.tcn1.conv.weight", "module.backbone.7.tcn1.conv.bias", "module.backbone.7.tcn1.bn.weight", "module.backbone.7.tcn1.bn.bias", "module.backbone.7.tcn1.bn.running_mean", "module.backbone.7.tcn1.bn.running_var", "module.backbone.8.gcn1.data_bn.weight", "module.backbone.8.gcn1.data_bn.bias", "module.backbone.8.gcn1.data_bn.running_mean", "module.backbone.8.gcn1.data_bn.running_var", "module.backbone.8.gcn1.bn.weight", "module.backbone.8.gcn1.bn.bias", "module.backbone.8.gcn1.bn.running_mean", "module.backbone.8.gcn1.bn.running_var", "module.backbone.8.gcn1.attention_conv.qkv_conv.weight", "module.backbone.8.gcn1.attention_conv.qkv_conv.bias", "module.backbone.8.gcn1.attention_conv.attn_out.weight", "module.backbone.8.gcn1.attention_conv.attn_out.bias", "module.backbone.8.tcn1.conv.weight", "module.backbone.8.tcn1.conv.bias", "module.backbone.8.tcn1.bn.weight", "module.backbone.8.tcn1.bn.bias", "module.backbone.8.tcn1.bn.running_mean", "module.backbone.8.tcn1.bn.running_var", "module.gcn0.conv_list.0.weight", "module.gcn0.conv_list.0.bias", "module.gcn0.conv_list.1.weight", "module.gcn0.conv_list.1.bias", "module.gcn0.conv_list.2.weight", "module.gcn0.conv_list.2.bias", "module.gcn0.bn.weight", "module.gcn0.bn.bias", "module.gcn0.bn.running_mean", "module.gcn0.bn.running_var", "module.tcn0.conv.weight", "module.tcn0.conv.bias", "module.tcn0.bn.weight", "module.tcn0.bn.bias", "module.tcn0.bn.running_mean", "module.tcn0.bn.running_var", "module.person_bn.weight", "module.person_bn.bias", "module.person_bn.running_mean", "module.person_bn.running_var", "module.fcn.weight", "module.fcn.bias". 

Unexpected key(s) in state_dict: "data_bn.weight", "data_bn.bias", "data_bn.running_mean", "data_bn.running_var", "data_bn.num_batches_tracked", "backbone.0.gcn1.mask", "backbone.0.gcn1.conv_list.0.weight", "backbone.0.gcn1.conv_list.0.bias", "backbone.0.gcn1.conv_list.1.weight", "backbone.0.gcn1.conv_list.1.bias", "backbone.0.gcn1.conv_list.2.weight", "backbone.0.gcn1.conv_list.2.bias", "backbone.0.gcn1.bn.weight", "backbone.0.gcn1.bn.bias", "backbone.0.gcn1.bn.running_mean", "backbone.0.gcn1.bn.running_var", "backbone.0.gcn1.bn.num_batches_tracked", "backbone.0.tcn1.conv.weight", "backbone.0.tcn1.conv.bias", "backbone.0.tcn1.bn.weight", "backbone.0.tcn1.bn.bias", "backbone.0.tcn1.bn.running_mean", "backbone.0.tcn1.bn.running_var", "backbone.0.tcn1.bn.num_batches_tracked", "backbone.1.gcn1.mask", "backbone.1.gcn1.conv_list.0.weight", "backbone.1.gcn1.conv_list.0.bias", "backbone.1.gcn1.conv_list.1.weight", "backbone.1.gcn1.conv_list.1.bias", "backbone.1.gcn1.conv_list.2.weight", "backbone.1.gcn1.conv_list.2.bias", "backbone.1.gcn1.bn.weight", "backbone.1.gcn1.bn.bias", "backbone.1.gcn1.bn.running_mean", "backbone.1.gcn1.bn.running_var", "backbone.1.gcn1.bn.num_batches_tracked", "backbone.1.tcn1.conv.weight", "backbone.1.tcn1.conv.bias", "backbone.1.tcn1.bn.weight", "backbone.1.tcn1.bn.bias", "backbone.1.tcn1.bn.running_mean", "backbone.1.tcn1.bn.running_var", "backbone.1.tcn1.bn.num_batches_tracked", "backbone.2.gcn1.mask", "backbone.2.gcn1.conv_list.0.weight", "backbone.2.gcn1.conv_list.0.bias", "backbone.2.gcn1.conv_list.1.weight", "backbone.2.gcn1.conv_list.1.bias", "backbone.2.gcn1.conv_list.2.weight", "backbone.2.gcn1.conv_list.2.bias", "backbone.2.gcn1.bn.weight", "backbone.2.gcn1.bn.bias", "backbone.2.gcn1.bn.running_mean", "backbone.2.gcn1.bn.running_var", "backbone.2.gcn1.bn.num_batches_tracked", "backbone.2.tcn1.conv.weight", "backbone.2.tcn1.conv.bias", "backbone.2.tcn1.bn.weight", "backbone.2.tcn1.bn.bias", "backbone.2.tcn1.bn.running_mean", "backbone.2.tcn1.bn.running_var", "backbone.2.tcn1.bn.num_batches_tracked", "backbone.3.gcn1.data_bn.weight", "backbone.3.gcn1.data_bn.bias", "backbone.3.gcn1.data_bn.running_mean", "backbone.3.gcn1.data_bn.running_var", "backbone.3.gcn1.data_bn.num_batches_tracked", "backbone.3.gcn1.bn.weight", "backbone.3.gcn1.bn.bias", "backbone.3.gcn1.bn.running_mean", "backbone.3.gcn1.bn.running_var", "backbone.3.gcn1.bn.num_batches_tracked", "backbone.3.gcn1.attention_conv.qkv_conv.weight", "backbone.3.gcn1.attention_conv.qkv_conv.bias", "backbone.3.gcn1.attention_conv.attn_out.weight", "backbone.3.gcn1.attention_conv.attn_out.bias", "backbone.3.tcn1.conv.weight", "backbone.3.tcn1.conv.bias", "backbone.3.tcn1.bn.weight", "backbone.3.tcn1.bn.bias", "backbone.3.tcn1.bn.running_mean", "backbone.3.tcn1.bn.running_var", "backbone.3.tcn1.bn.num_batches_tracked", "backbone.3.down1.conv.weight", "backbone.3.down1.conv.bias", "backbone.3.down1.bn.weight", "backbone.3.down1.bn.bias", "backbone.3.down1.bn.running_mean", "backbone.3.down1.bn.running_var", "backbone.3.down1.bn.num_batches_tracked", "backbone.4.gcn1.data_bn.weight", "backbone.4.gcn1.data_bn.bias", "backbone.4.gcn1.data_bn.running_mean", "backbone.4.gcn1.data_bn.running_var", "backbone.4.gcn1.data_bn.num_batches_tracked", "backbone.4.gcn1.bn.weight", "backbone.4.gcn1.bn.bias", "backbone.4.gcn1.bn.running_mean", "backbone.4.gcn1.bn.running_var", "backbone.4.gcn1.bn.num_batches_tracked", "backbone.4.gcn1.attention_conv.qkv_conv.weight", "backbone.4.gcn1.attention_conv.qkv_conv.bias", "backbone.4.gcn1.attention_conv.attn_out.weight", "backbone.4.gcn1.attention_conv.attn_out.bias", "backbone.4.tcn1.conv.weight", "backbone.4.tcn1.conv.bias", "backbone.4.tcn1.bn.weight", "backbone.4.tcn1.bn.bias", "backbone.4.tcn1.bn.running_mean", "backbone.4.tcn1.bn.running_var", "backbone.4.tcn1.bn.num_batches_tracked", "backbone.5.gcn1.data_bn.weight", "backbone.5.gcn1.data_bn.bias", "backbone.5.gcn1.data_bn.running_mean", "backbone.5.gcn1.data_bn.running_var", "backbone.5.gcn1.data_bn.num_batches_tracked", "backbone.5.gcn1.bn.weight", "backbone.5.gcn1.bn.bias", "backbone.5.gcn1.bn.running_mean", "backbone.5.gcn1.bn.running_var", "backbone.5.gcn1.bn.num_batches_tracked", "backbone.5.gcn1.attention_conv.qkv_conv.weight", "backbone.5.gcn1.attention_conv.qkv_conv.bias", "backbone.5.gcn1.attention_conv.attn_out.weight", "backbone.5.gcn1.attention_conv.attn_out.bias", "backbone.5.tcn1.conv.weight", "backbone.5.tcn1.conv.bias", "backbone.5.tcn1.bn.weight", "backbone.5.tcn1.bn.bias", "backbone.5.tcn1.bn.running_mean", "backbone.5.tcn1.bn.running_var", "backbone.5.tcn1.bn.num_batches_tracked", "backbone.6.gcn1.data_bn.weight", "backbone.6.gcn1.data_bn.bias", "backbone.6.gcn1.data_bn.running_mean", "backbone.6.gcn1.data_bn.running_var", "backbone.6.gcn1.data_bn.num_batches_tracked", "backbone.6.gcn1.bn.weight", "backbone.6.gcn1.bn.bias", "backbone.6.gcn1.bn.running_mean", "backbone.6.gcn1.bn.running_var", "backbone.6.gcn1.bn.num_batches_tracked", "backbone.6.gcn1.attention_conv.qkv_conv.weight", "backbone.6.gcn1.attention_conv.qkv_conv.bias", "backbone.6.gcn1.attention_conv.attn_out.weight", "backbone.6.gcn1.attention_conv.attn_out.bias", "backbone.6.tcn1.conv.weight", "backbone.6.tcn1.conv.bias", "backbone.6.tcn1.bn.weight", "backbone.6.tcn1.bn.bias", "backbone.6.tcn1.bn.running_mean", "backbone.6.tcn1.bn.running_var", "backbone.6.tcn1.bn.num_batches_tracked", "backbone.6.down1.conv.weight", "backbone.6.down1.conv.bias", "backbone.6.down1.bn.weight", "backbone.6.down1.bn.bias", "backbone.6.down1.bn.running_mean", "backbone.6.down1.bn.running_var", "backbone.6.down1.bn.num_batches_tracked", "backbone.7.gcn1.data_bn.weight", "backbone.7.gcn1.data_bn.bias", "backbone.7.gcn1.data_bn.running_mean", "backbone.7.gcn1.data_bn.running_var", "backbone.7.gcn1.data_bn.num_batches_tracked", "backbone.7.gcn1.bn.weight", "backbone.7.gcn1.bn.bias", "backbone.7.gcn1.bn.running_mean", "backbone.7.gcn1.bn.running_var", "backbone.7.gcn1.bn.num_batches_tracked", "backbone.7.gcn1.attention_conv.qkv_conv.weight", "backbone.7.gcn1.attention_conv.qkv_conv.bias", "backbone.7.gcn1.attention_conv.attn_out.weight", "backbone.7.gcn1.attention_conv.attn_out.bias", "backbone.7.tcn1.conv.weight", "backbone.7.tcn1.conv.bias", "backbone.7.tcn1.bn.weight", "backbone.7.tcn1.bn.bias", "backbone.7.tcn1.bn.running_mean", "backbone.7.tcn1.bn.running_var", "backbone.7.tcn1.bn.num_batches_tracked", "backbone.8.gcn1.data_bn.weight", "backbone.8.gcn1.data_bn.bias", "backbone.8.gcn1.data_bn.running_mean", "backbone.8.gcn1.data_bn.running_var", "backbone.8.gcn1.data_bn.num_batches_tracked", "backbone.8.gcn1.bn.weight", "backbone.8.gcn1.bn.bias", "backbone.8.gcn1.bn.running_mean", "backbone.8.gcn1.bn.running_var", "backbone.8.gcn1.bn.num_batches_tracked", "backbone.8.gcn1.attention_conv.qkv_conv.weight", "backbone.8.gcn1.attention_conv.qkv_conv.bias", "backbone.8.gcn1.attention_conv.attn_out.weight", "backbone.8.gcn1.attention_conv.attn_out.bias", "backbone.8.tcn1.conv.weight", "backbone.8.tcn1.conv.bias", "backbone.8.tcn1.bn.weight", "backbone.8.tcn1.bn.bias", "backbone.8.tcn1.bn.running_mean", "backbone.8.tcn1.bn.running_var", "backbone.8.tcn1.bn.num_batches_tracked", "gcn0.mask", "gcn0.conv_list.0.weight", "gcn0.conv_list.0.bias", "gcn0.conv_list.1.weight", "gcn0.conv_list.1.bias", "gcn0.conv_list.2.weight", "gcn0.conv_list.2.bias", "gcn0.bn.weight", "gcn0.bn.bias", "gcn0.bn.running_mean", "gcn0.bn.running_var", "gcn0.bn.num_batches_tracked", "tcn0.conv.weight", "tcn0.conv.bias", "tcn0.bn.weight", "tcn0.bn.bias", "tcn0.bn.running_mean", "tcn0.bn.running_var", "tcn0.bn.num_batches_tracked", "person_bn.weight", "person_bn.bias", "person_bn.running_mean", "person_bn.running_var", "person_bn.num_batches_tracked", "fcn.weight", "fcn.bias". 

Hi,

Thank you for sharing the code. As a university student, this project helped me understand STGCN better. Could you please help me understand how you generated the heatmaps to visualize the classes and the attention? As I couldn't see it in the project, kindly share the code to visualise the NTURGBD 60 dataset.

Thank you
Muskan
[email protected]

I have followed your instructions on Github and used the following configuration for S-TR and T-TR
respectively, but I only got 83% top-1 acc for S-TR and 58% top-1 acc for T-TR (much lower than the number listed in your paper, especially for T-TR).

I don't know if the dim_block 1-3 should be set to 64, 128, 256. However, even with larger block dim, T-TR still seems to perform much worse than S-TR.
I am looking forward to hearing from you.

For S-TR:

feeder

feeder: st_gcn.feeder.Feeder
feeder_augmented: st_gcn.feeder.FeederAugmented
train_feeder_args:
data_path: /ssd2/zhouyuxuan/Dataset/NTU-RGB-D/xsub/train_data.npy
label_path: /ssd2/zhouyuxuan/Dataset/NTU-RGB-D/xsub/train_label.pkl

random_choose: False
random_shift: False
random_move: False
window_size: -1
normalization: False
mirroring: False

test_feeder_args:
data_path: /ssd2/zhouyuxuan/Dataset/NTU-RGB-D/xsub/val_data.npy
label_path: /ssd2/zhouyuxuan/Dataset/NTU-RGB-D/xsub/val_label.pkl

model

model: st_gcn.net.ST_GCN
training: True

model_args:
num_class: 60
channel: 3
window_size: 300
num_point: 25
num_person: 2
mask_learning: True
use_data_bn: True
attention: True
only_attention: True
tcn_attention: False
data_normalization: True
skip_conn: True
weight_matrix: 2
only_temporal_attention: True
bn_flag: True
attention_3: False
kernel_temporal: 9
more_channels: False
double_channel: False
drop_connect: True
concat_original: True
all_layers: False
adjacency: False
agcn: False
dv: 0.25
dk: 0.25
Nh: 8
n: 4
dim_block1: 10
dim_block2: 30
dim_block3: 75
relative: False
graph: st_gcn.graph.NTU_RGB_D
visualization: False
graph_args:
labeling_mode: 'spatial'
#optical_flow: True

optim

#0: old one, 1: new one
scheduler: 1
weight_decay: 0.0001
base_lr: 0.1
step: [60,90]

training

device: [0,1,2,3]
batch_size: 32
test_batch_size: 64
num_epoch: 120
nesterov: True

For T-TR:

feeder

feeder: st_gcn.feeder.Feeder
feeder_augmented: st_gcn.feeder.FeederAugmented
train_feeder_args:
data_path: /ssd2/zhouyuxuan/Dataset/NTU-RGB-D/xsub/train_data.npy
label_path: /ssd2/zhouyuxuan/Dataset/NTU-RGB-D/xsub/train_label.pkl

random_choose: False
random_shift: False
random_move: False
window_size: -1
normalization: False
mirroring: False

test_feeder_args:
data_path: /ssd2/zhouyuxuan/Dataset/NTU-RGB-D/xsub/val_data.npy
label_path: /ssd2/zhouyuxuan/Dataset/NTU-RGB-D/xsub/val_label.pkl

model

model: st_gcn.net.ST_GCN
training: True

model_args:
num_class: 60
channel: 3
window_size: 300
num_point: 25
num_person: 2
mask_learning: True
use_data_bn: True
attention: False
only_attention: True
tcn_attention: True
data_normalization: True
skip_conn: True
weight_matrix: 2
only_temporal_attention: True
bn_flag: True
attention_3: False
kernel_temporal: 9
more_channels: False
double_channel: False
drop_connect: True
concat_original: True
all_layers: False
adjacency: False
agcn: False
dv: 0.25
dk: 0.25
Nh: 8
n: 4
dim_block1: 10
dim_block2: 30
dim_block3: 75
relative: False
graph: st_gcn.graph.NTU_RGB_D
visualization: False
graph_args:
labeling_mode: 'spatial'
#optical_flow: True

optim

#0: old one, 1: new one
scheduler: 1
weight_decay: 0.0001
base_lr: 0.1
step: [60,90]

training

device: [0,1,2,3]
batch_size: 32
test_batch_size: 64
num_epoch: 120
nesterov: True

Hello, may I ask if the parameters are shared between the multi-layer SSA? Thank you very much for your reply

Hello,

I was trying to use relative argument to add relative positional encoding in tcn_unit_attention but there is a dimension mismatch error in the function rel_to_abs.

File "/home/dusko/Documents/projects/ST-TR/code/st_gcn/net/temporal_transformer.py", line 283, in rel_to_abs
flat_x = torch.reshape(x, (B, Nh, L * 2 * L))
RuntimeError: shape '[400, 8, 45000]' is invalid for input of size 72000000

Did anyone train the model with the positional encoding?
I would appreicate any reply.

What will happen and what kind of results do we get if we have a window size of more than one and switch the model to spatial architecture ?

hi Chiaraplizz. I have got new bugs LOL
i was running the main.py in my datasets. and i just run 5 epochs in order to have a try.

Validation: Epoch [4/5], Samples [25.0/465], Loss: 2.3637566566467285, Validation Accuracy: 5.376344086021505
[ Sun Nov 29 14:19:36 2020 ] Load weights from /home/lijianwei/st_tr_wsq/check_points/epoch4_model.pt.
Traceback (most recent call last):
  File "main.py", line 959, in <module>
    processor.start()
  File "main.py", line 890, in start
    weights = torch.load(
  File "/home/lijianwei/.pyenv/versions/TTNet/lib/python3.8/site-packages/torch/serialization.py", line 584, in load
    with _open_file_like(f, 'rb') as opened_file:
  File "/home/lijianwei/.pyenv/versions/TTNet/lib/python3.8/site-packages/torch/serialization.py", line 234, in _open_file_like
    return _open_file(name_or_buffer, mode)
  File "/home/lijianwei/.pyenv/versions/TTNet/lib/python3.8/site-packages/torch/serialization.py", line 215, in __init__
    super(_open_file, self).__init__(open(name, mode))
FileNotFoundError: [Errno 2] No such file or directory: '/home/lijianwei/st_tr_wsq/check_points/epoch4_model.pt'

here is my check_points dir:

[sse-server st_tr_wsq]$ ls
conf_val_0.npy  conf_val_1.npy  conf_val_2.npy  conf_val_3.npy  conf_val_4.npy

 save_model = ((epoch + 1) % self.arg.save_interval == 0) or (
                        epoch + 1 == self.arg.num_epoch)
                print(save_model)
                eval_model = ((epoch + 1) % self.arg.eval_interval == 0) or (
                        epoch + 1 == self.arg.num_epoch)

                self.train(epoch, save_model=save_model)

i'm going to set the SAVE_MODE = True.
Hoping that would solve the questions.

Hi Chiara,
After reading your paper, I think this is a very meaningful study. I was trying to reproduce the results and only got 0.24% for the top-1. 1.22% for the top-5.
I used your pre-trained model(kinetics_spatial.pt) to test the results. Modify channel=3 and double_channel: false in kinetics-skeleton/train.yaml.There is no error in the code running.
When i run the temporal transformer stream,the output of the network will become smaller and smaller as the batch_idx increases.When batch_idx>=76, the output will all become 'nan', so I finally got an unsatisfactory result.
The attachment is my configuration file and main.py
main+train.zip
Can you give advice on how to solve this problem？
Thanks a lot，
best

xset key is missing in the paris dictionary.

Becasue I did not load the model's weight when I test the code, the loss is too large. I am wondering how to load the trained model to test process. Whould you please solve it?

I have solved this problem. Thank you.

Hi, Chiaraplizz,
I'd like to consult a problem encountered in running the code.
When I start training process, following error has occured:

RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [16, 512, 75, 25]], which is output 0 of ReluBackward0, is at version 1; expected version 0 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).

My device config is set as [0, 1, 2, 3] and use dataParallel for multi-processor calculating, which is the same as your configuration in the responsitory. Did youmeet the same issue? I can't appreciate more if you can help me solve this problem.

I only have train_label.pkl, then how to get train_label_filtered.pkl?

Where is the code of Fig. 7. Layer visualization. Each column represent the k-th layer, along with the corresponding spatial self-attention heatmap on top, and the skeleton of the subjects performing the action on the bottom.

Hello,

I am having some issues in the middle of training.
Until I was running around 20 epochs there was no problem. The loss and accuracy functions look normal until I got this warning

WARNING:root:NaN or Inf found in input tensor.
WARNING:root:NaN or Inf found in input tensor.
WARNING:root:NaN or Inf found in input tensor.

I don't think it is the dataset issue since it was running fine for the first 20 epochs...

If anyone had this issue, would you please explain what could potentially cause this?

Traceback (most recent call last):
File "", line 1, in
File "C:\ProgramData\anaconda3\lib\multiprocessing\spawn.py", line 116, in spawn_main
exitcode = _main(fd, parent_sentinel)
File "C:\ProgramData\anaconda3\lib\multiprocessing\spawn.py", line 126, in _main
self = reduction.pickle.load(from_parent)
_pickle.UnpicklingError: pickle data was truncated

Why am I getting this error? @Chiaraplizz

Hi Chiara,

Really enjoy your paper. Thank you for sharing the code as well. We (@sneheshs and I) were trying to reproduce the results and only got 0.25% for the top-1 instead of 37.0% as reported in the paper. The only thing we had to change was the number of channels from 6 to 3 and double channel to False as the Kinetics dataset does not have bones which perhaps is causing this discrepancy.

  channel: 3 # 6
  double_channel: False # True

Based on the train.yaml file, however, we noticed that you are using 6 channels with bones data which the currently published script kinetics_gendata.py does not output. Could you please share the updated file so we can reproduce the results accurately?

Training config file: https://github.com/Chiaraplizz/ST-TR/blob/master/code/config/st_gcn/kinetics-skeleton/train.yaml

test_feeder_args:
  data_path: ./kinetics_data/val_data_joint_bones.npy

Thanks for your help.

Best,
Snehesh and Andy

Traceback (most recent call last):
File "/content/drive/MyDrive/ST-TR-master/code/main.py", line 974, in
processor.start()
File "/content/drive/MyDrive/ST-TR-master/code/main.py", line 886, in start
self.train(epoch, save_model=save_model)
File "/content/drive/MyDrive/ST-TR-master/code/main.py", line 442, in train
for batch_idx, (data, label, name) in enumerate(loader):
File "/usr/local/lib/python3.7/dist-packages/torch/utils/data/dataloader.py", line 521, in next
data = self._next_data()
File "/usr/local/lib/python3.7/dist-packages/torch/utils/data/dataloader.py", line 1203, in _next_data
return self._process_data(data)
File "/usr/local/lib/python3.7/dist-packages/torch/utils/data/dataloader.py", line 1229, in _process_data
data.reraise()
File "/usr/local/lib/python3.7/dist-packages/torch/_utils.py", line 434, in reraise
raise exception
AttributeError: Caught AttributeError in DataLoader worker process 0.

I hope you can help me to solve this problem.

Hi:
When I run python main.py, there is a problem,
Traceback (most recent call last):
File "/data/wangzeyu/code/ST-TR-master/code/main.py", line 956, in
processor.start()
File "/data/wangzeyu/code/ST-TR-master/code/main.py", line 867, in start
self.train(epoch, save_model=save_model)
File "/data/wangzeyu/code/ST-TR-master/code/main.py", line 538, in train
output = self.model(data, label, name)
File "/data/wangzeyu/wangzeyu_torch_clone/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/data/wangzeyu/wangzeyu_torch_clone/lib/python3.7/site-packages/torch/nn/parallel/data_parallel.py", line 159, in forward
return self.module(*inputs[0], **kwargs[0])
File "/data/wangzeyu/wangzeyu_torch_clone/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/data/wangzeyu/code/ST-TR-master/code/st_gcn/net/st_gcn.py", line 267, in forward
x = self.data_bn(x)
File "/data/wangzeyu/wangzeyu_torch_clone/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/data/wangzeyu/wangzeyu_torch_clone/lib/python3.7/site-packages/torch/nn/modules/batchnorm.py", line 136, in forward
self.weight, self.bias, bn_training, exponential_average_factor, self.eps)
File "/data/wangzeyu/wangzeyu_torch_clone/lib/python3.7/site-packages/torch/nn/functional.py", line 2058, in batch_norm
training, momentum, eps, torch.backends.cudnn.enabled
RuntimeError: running_mean should contain 108 elements not 216

How to solve it?
Thanks

Hello, we are ready to quote your literature and compare our method with ST-TR in my paper. But there are many problems in the setup, can you help me run the program on our data set?
thanks!!!!!!!!!

Hi,

I am trying to make the code working on my pc and to do so, I have generated the data from the NTURGB dataset using the preprocess.py script. When I try to run the main.py script, I get the following error:

Traceback (most recent call last):
  File "main.py", line 1015, in <module>
    processor.start()
  File "main.py", line 926, in start
    self.train(epoch, save_model=save_model)
  File "main.py", line 581, in train
    for batch_idx, (data, label, name) in enumerate(loader):
  File "C:\Users\Benjamin\Anaconda3\envs\py38\lib\site-packages\torch\utils\data\dataloader.py", line 355, in __iter__
    return self._get_iterator()
  File "C:\Users\Benjamin\Anaconda3\envs\py38\lib\site-packages\torch\utils\data\dataloader.py", line 301, in _get_iterator
    return _MultiProcessingDataLoaderIter(self)
  File "C:\Users\Benjamin\Anaconda3\envs\py38\lib\site-packages\torch\utils\data\dataloader.py", line 914, in __init__
    w.start()
  File "C:\Users\Benjamin\Anaconda3\envs\py38\lib\multiprocessing\process.py", line 121, in start
    self._popen = self._Popen(self)
  File "C:\Users\Benjamin\Anaconda3\envs\py38\lib\multiprocessing\context.py", line 224, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "C:\Users\Benjamin\Anaconda3\envs\py38\lib\multiprocessing\context.py", line 327, in _Popen
    return Popen(process_obj)
  File "C:\Users\Benjamin\Anaconda3\envs\py38\lib\multiprocessing\popen_spawn_win32.py", line 93, in __init__
    reduction.dump(process_obj, to_child)
  File "C:\Users\Benjamin\Anaconda3\envs\py38\lib\multiprocessing\reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
OSError: [Errno 22] Invalid argument
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Users\Benjamin\Anaconda3\envs\py38\lib\multiprocessing\spawn.py", line 116, in spawn_main
    exitcode = _main(fd, parent_sentinel)
  File "C:\Users\Benjamin\Anaconda3\envs\py38\lib\multiprocessing\spawn.py", line 126, in _main
    self = reduction.pickle.load(from_parent)
_pickle.UnpicklingError: pickle data was truncated

From what I understand, the preprocess.py script only generates the joints data. So my question is do I also need to generate the bones data and merge them with the joints or can I use only the joints data to train the network ?
Also, the error I get make me think that not all the data are loaded correctly and I am thinking of testing the code on another pc with more memory. What are the specs of the machine on which you train the network or have you encounter this error before ?

Thanks in advance.

Best regards.
Benjamin

When I run the code, no matter which of the three data sets I use, I get to epoch with the following problems. Using only the node and using the joint data of the node and the skeleton results in an error at 10 epoch. So I wonder if the code is storing other unimportant data during execution, please kindly comment.

RuntimeError:CUDA out of memory.Tried to allocate 29.38 MiB (GPU 0; 7.79 GiB total capacity; 6.61 GiB already allocated;21.38 MiB free; 91.98 MiB cached)

Hi Chiara.

I have an error when I run main.py with pre-trained models:

python main.py --phase test --weights /home/lucas/Desktop/ST-TR/checkpoint_ST-TR/kinetics_temporal.pt

ERROR:

Traceback (most recent call last):
File "main.py", line 972, in
processor = Processor(arg)
File "main.py", line 234, in init
self.load_model()
File "main.py", line 354, in load_model
self.model.load_state_dict(state)
File "/home/lucas/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1406, in load_state_dict
raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for Model:
size mismatch for data_bn.weight: copying a param with shape torch.Size([108]) from checkpoint, the shape in current model is torch.Size([216]).
size mismatch for data_bn.bias: copying a param with shape torch.Size([108]) from checkpoint, the shape in current model is torch.Size([216]).
size mismatch for data_bn.running_mean: copying a param with shape torch.Size([108]) from checkpoint, the shape in current model is torch.Size([216]).
size mismatch for data_bn.running_var: copying a param with shape torch.Size([108]) from checkpoint, the shape in current model is torch.Size([216]).
size mismatch for backbone.0.gcn1.conv_list.0.weight: copying a param with shape torch.Size([64, 64, 1, 1]) from checkpoint, the shape in current model is torch.Size([128, 128, 1, 1]).
size mismatch for backbone.0.gcn1.conv_list.0.bias: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for backbone.0.gcn1.conv_list.1.weight: copying a param with shape torch.Size([64, 64, 1, 1]) from checkpoint, the shape in current model is torch.Size([128, 128, 1, 1]).
size mismatch for backbone.0.gcn1.conv_list.1.bias: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for backbone.0.gcn1.conv_list.2.weight: copying a param with shape torch.Size([64, 64, 1, 1]) from checkpoint, the shape in current model is torch.Size([128, 128, 1, 1]).
size mismatch for backbone.0.gcn1.conv_list.2.bias: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for backbone.0.gcn1.bn.weight: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for backbone.0.gcn1.bn.bias: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for backbone.0.gcn1.bn.running_mean: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for backbone.0.gcn1.bn.running_var: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for backbone.0.tcn1.conv.weight: copying a param with shape torch.Size([64, 64, 9, 1]) from checkpoint, the shape in current model is torch.Size([128, 128, 9, 1]).
size mismatch for backbone.0.tcn1.conv.bias: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for backbone.0.tcn1.bn.weight: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for backbone.0.tcn1.bn.bias: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for backbone.0.tcn1.bn.running_mean: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for backbone.0.tcn1.bn.running_var: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for backbone.1.gcn1.conv_list.0.weight: copying a param with shape torch.Size([64, 64, 1, 1]) from checkpoint, the shape in current model is torch.Size([128, 128, 1, 1]).
size mismatch for backbone.1.gcn1.conv_list.0.bias: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for backbone.1.gcn1.conv_list.1.weight: copying a param with shape torch.Size([64, 64, 1, 1]) from checkpoint, the shape in current model is torch.Size([128, 128, 1, 1]).
size mismatch for backbone.1.gcn1.conv_list.1.bias: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for backbone.1.gcn1.conv_list.2.weight: copying a param with shape torch.Size([64, 64, 1, 1]) from checkpoint, the shape in current model is torch.Size([128, 128, 1, 1]).
size mismatch for backbone.1.gcn1.conv_list.2.bias: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for backbone.1.gcn1.bn.weight: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for backbone.1.gcn1.bn.bias: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for backbone.1.gcn1.bn.running_mean: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for backbone.1.gcn1.bn.running_var: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for backbone.1.tcn1.conv.weight: copying a param with shape torch.Size([64, 64, 9, 1]) from checkpoint, the shape in current model is torch.Size([128, 128, 9, 1]).
size mismatch for backbone.1.tcn1.conv.bias: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for backbone.1.tcn1.bn.weight: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for backbone.1.tcn1.bn.bias: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for backbone.1.tcn1.bn.running_mean: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for backbone.1.tcn1.bn.running_var: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for backbone.2.gcn1.conv_list.0.weight: copying a param with shape torch.Size([64, 64, 1, 1]) from checkpoint, the shape in current model is torch.Size([128, 128, 1, 1]).
size mismatch for backbone.2.gcn1.conv_list.0.bias: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for backbone.2.gcn1.conv_list.1.weight: copying a param with shape torch.Size([64, 64, 1, 1]) from checkpoint, the shape in current model is torch.Size([128, 128, 1, 1]).
size mismatch for backbone.2.gcn1.conv_list.1.bias: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for backbone.2.gcn1.conv_list.2.weight: copying a param with shape torch.Size([64, 64, 1, 1]) from checkpoint, the shape in current model is torch.Size([128, 128, 1, 1]).
size mismatch for backbone.2.gcn1.conv_list.2.bias: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for backbone.2.gcn1.bn.weight: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for backbone.2.gcn1.bn.bias: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for backbone.2.gcn1.bn.running_mean: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for backbone.2.gcn1.bn.running_var: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for backbone.2.tcn1.conv.weight: copying a param with shape torch.Size([64, 64, 9, 1]) from checkpoint, the shape in current model is torch.Size([128, 128, 9, 1]).
size mismatch for backbone.2.tcn1.conv.bias: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for backbone.2.tcn1.bn.weight: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for backbone.2.tcn1.bn.bias: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for backbone.2.tcn1.bn.running_mean: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for backbone.2.tcn1.bn.running_var: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for backbone.3.gcn1.conv_list.0.weight: copying a param with shape torch.Size([128, 67, 1, 1]) from checkpoint, the shape in current model is torch.Size([256, 134, 1, 1]).
size mismatch for backbone.3.gcn1.conv_list.0.bias: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone.3.gcn1.conv_list.1.weight: copying a param with shape torch.Size([128, 67, 1, 1]) from checkpoint, the shape in current model is torch.Size([256, 134, 1, 1]).
size mismatch for backbone.3.gcn1.conv_list.1.bias: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone.3.gcn1.conv_list.2.weight: copying a param with shape torch.Size([128, 67, 1, 1]) from checkpoint, the shape in current model is torch.Size([256, 134, 1, 1]).
size mismatch for backbone.3.gcn1.conv_list.2.bias: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone.3.gcn1.bn.weight: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone.3.gcn1.bn.bias: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone.3.gcn1.bn.running_mean: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone.3.gcn1.bn.running_var: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone.3.tcn1.bn.weight: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone.3.tcn1.bn.bias: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone.3.tcn1.bn.running_mean: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone.3.tcn1.bn.running_var: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone.3.tcn1.down.conv.weight: copying a param with shape torch.Size([128, 128, 1, 1]) from checkpoint, the shape in current model is torch.Size([256, 256, 1, 1]).
size mismatch for backbone.3.tcn1.down.conv.bias: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone.3.tcn1.down.bn.weight: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone.3.tcn1.down.bn.bias: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone.3.tcn1.down.bn.running_mean: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone.3.tcn1.down.bn.running_var: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone.3.tcn1.data_bn.weight: copying a param with shape torch.Size([2304]) from checkpoint, the shape in current model is torch.Size([4608]).
size mismatch for backbone.3.tcn1.data_bn.bias: copying a param with shape torch.Size([2304]) from checkpoint, the shape in current model is torch.Size([4608]).
size mismatch for backbone.3.tcn1.data_bn.running_mean: copying a param with shape torch.Size([2304]) from checkpoint, the shape in current model is torch.Size([4608]).
size mismatch for backbone.3.tcn1.data_bn.running_var: copying a param with shape torch.Size([2304]) from checkpoint, the shape in current model is torch.Size([4608]).
size mismatch for backbone.3.tcn1.qkv_conv.weight: copying a param with shape torch.Size([192, 128, 1, 1]) from checkpoint, the shape in current model is torch.Size([384, 256, 1, 1]).
size mismatch for backbone.3.tcn1.qkv_conv.bias: copying a param with shape torch.Size([192]) from checkpoint, the shape in current model is torch.Size([384]).
size mismatch for backbone.3.tcn1.attn_out.weight: copying a param with shape torch.Size([128, 128, 1, 1]) from checkpoint, the shape in current model is torch.Size([256, 256, 1, 1]).
size mismatch for backbone.3.tcn1.attn_out.bias: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone.3.down1.conv.weight: copying a param with shape torch.Size([128, 67, 1, 1]) from checkpoint, the shape in current model is torch.Size([256, 134, 1, 1]).
size mismatch for backbone.3.down1.conv.bias: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone.3.down1.bn.weight: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone.3.down1.bn.bias: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone.3.down1.bn.running_mean: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone.3.down1.bn.running_var: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone.4.gcn1.conv_list.0.weight: copying a param with shape torch.Size([128, 128, 1, 1]) from checkpoint, the shape in current model is torch.Size([256, 256, 1, 1]).
size mismatch for backbone.4.gcn1.conv_list.0.bias: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone.4.gcn1.conv_list.1.weight: copying a param with shape torch.Size([128, 128, 1, 1]) from checkpoint, the shape in current model is torch.Size([256, 256, 1, 1]).
size mismatch for backbone.4.gcn1.conv_list.1.bias: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone.4.gcn1.conv_list.2.weight: copying a param with shape torch.Size([128, 128, 1, 1]) from checkpoint, the shape in current model is torch.Size([256, 256, 1, 1]).
size mismatch for backbone.4.gcn1.conv_list.2.bias: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone.4.gcn1.bn.weight: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone.4.gcn1.bn.bias: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone.4.gcn1.bn.running_mean: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone.4.gcn1.bn.running_var: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone.4.tcn1.bn.weight: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone.4.tcn1.bn.bias: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone.4.tcn1.bn.running_mean: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone.4.tcn1.bn.running_var: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone.4.tcn1.data_bn.weight: copying a param with shape torch.Size([2304]) from checkpoint, the shape in current model is torch.Size([4608]).
size mismatch for backbone.4.tcn1.data_bn.bias: copying a param with shape torch.Size([2304]) from checkpoint, the shape in current model is torch.Size([4608]).
size mismatch for backbone.4.tcn1.data_bn.running_mean: copying a param with shape torch.Size([2304]) from checkpoint, the shape in current model is torch.Size([4608]).
size mismatch for backbone.4.tcn1.data_bn.running_var: copying a param with shape torch.Size([2304]) from checkpoint, the shape in current model is torch.Size([4608]).
size mismatch for backbone.4.tcn1.qkv_conv.weight: copying a param with shape torch.Size([192, 128, 1, 1]) from checkpoint, the shape in current model is torch.Size([384, 256, 1, 1]).
size mismatch for backbone.4.tcn1.qkv_conv.bias: copying a param with shape torch.Size([192]) from checkpoint, the shape in current model is torch.Size([384]).
size mismatch for backbone.4.tcn1.attn_out.weight: copying a param with shape torch.Size([128, 128, 1, 1]) from checkpoint, the shape in current model is torch.Size([256, 256, 1, 1]).
size mismatch for backbone.4.tcn1.attn_out.bias: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone.5.gcn1.conv_list.0.weight: copying a param with shape torch.Size([128, 128, 1, 1]) from checkpoint, the shape in current model is torch.Size([256, 256, 1, 1]).
size mismatch for backbone.5.gcn1.conv_list.0.bias: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone.5.gcn1.conv_list.1.weight: copying a param with shape torch.Size([128, 128, 1, 1]) from checkpoint, the shape in current model is torch.Size([256, 256, 1, 1]).
size mismatch for backbone.5.gcn1.conv_list.1.bias: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone.5.gcn1.conv_list.2.weight: copying a param with shape torch.Size([128, 128, 1, 1]) from checkpoint, the shape in current model is torch.Size([256, 256, 1, 1]).
size mismatch for backbone.5.gcn1.conv_list.2.bias: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone.5.gcn1.bn.weight: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone.5.gcn1.bn.bias: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone.5.gcn1.bn.running_mean: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone.5.gcn1.bn.running_var: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone.5.tcn1.bn.weight: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone.5.tcn1.bn.bias: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone.5.tcn1.bn.running_mean: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone.5.tcn1.bn.running_var: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone.5.tcn1.data_bn.weight: copying a param with shape torch.Size([2304]) from checkpoint, the shape in current model is torch.Size([4608]).
size mismatch for backbone.5.tcn1.data_bn.bias: copying a param with shape torch.Size([2304]) from checkpoint, the shape in current model is torch.Size([4608]).
size mismatch for backbone.5.tcn1.data_bn.running_mean: copying a param with shape torch.Size([2304]) from checkpoint, the shape in current model is torch.Size([4608]).
size mismatch for backbone.5.tcn1.data_bn.running_var: copying a param with shape torch.Size([2304]) from checkpoint, the shape in current model is torch.Size([4608]).
size mismatch for backbone.5.tcn1.qkv_conv.weight: copying a param with shape torch.Size([192, 128, 1, 1]) from checkpoint, the shape in current model is torch.Size([384, 256, 1, 1]).
size mismatch for backbone.5.tcn1.qkv_conv.bias: copying a param with shape torch.Size([192]) from checkpoint, the shape in current model is torch.Size([384]).
size mismatch for backbone.5.tcn1.attn_out.weight: copying a param with shape torch.Size([128, 128, 1, 1]) from checkpoint, the shape in current model is torch.Size([256, 256, 1, 1]).
size mismatch for backbone.5.tcn1.attn_out.bias: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone.6.gcn1.conv_list.0.weight: copying a param with shape torch.Size([256, 128, 1, 1]) from checkpoint, the shape in current model is torch.Size([512, 256, 1, 1]).
size mismatch for backbone.6.gcn1.conv_list.0.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([512]).
size mismatch for backbone.6.gcn1.conv_list.1.weight: copying a param with shape torch.Size([256, 128, 1, 1]) from checkpoint, the shape in current model is torch.Size([512, 256, 1, 1]).
size mismatch for backbone.6.gcn1.conv_list.1.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([512]).
size mismatch for backbone.6.gcn1.conv_list.2.weight: copying a param with shape torch.Size([256, 128, 1, 1]) from checkpoint, the shape in current model is torch.Size([512, 256, 1, 1]).
size mismatch for backbone.6.gcn1.conv_list.2.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([512]).
size mismatch for backbone.6.gcn1.bn.weight: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([512]).
size mismatch for backbone.6.gcn1.bn.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([512]).
size mismatch for backbone.6.gcn1.bn.running_mean: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([512]).
size mismatch for backbone.6.gcn1.bn.running_var: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([512]).
size mismatch for backbone.6.tcn1.bn.weight: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([512]).
size mismatch for backbone.6.tcn1.bn.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([512]).
size mismatch for backbone.6.tcn1.bn.running_mean: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([512]).
size mismatch for backbone.6.tcn1.bn.running_var: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([512]).
size mismatch for backbone.6.tcn1.down.conv.weight: copying a param with shape torch.Size([256, 256, 1, 1]) from checkpoint, the shape in current model is torch.Size([512, 512, 1, 1]).
size mismatch for backbone.6.tcn1.down.conv.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([512]).
size mismatch for backbone.6.tcn1.down.bn.weight: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([512]).
size mismatch for backbone.6.tcn1.down.bn.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([512]).
size mismatch for backbone.6.tcn1.down.bn.running_mean: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([512]).
size mismatch for backbone.6.tcn1.down.bn.running_var: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([512]).
size mismatch for backbone.6.tcn1.data_bn.weight: copying a param with shape torch.Size([4608]) from checkpoint, the shape in current model is torch.Size([9216]).
size mismatch for backbone.6.tcn1.data_bn.bias: copying a param with shape torch.Size([4608]) from checkpoint, the shape in current model is torch.Size([9216]).
size mismatch for backbone.6.tcn1.data_bn.running_mean: copying a param with shape torch.Size([4608]) from checkpoint, the shape in current model is torch.Size([9216]).
size mismatch for backbone.6.tcn1.data_bn.running_var: copying a param with shape torch.Size([4608]) from checkpoint, the shape in current model is torch.Size([9216]).
size mismatch for backbone.6.tcn1.qkv_conv.weight: copying a param with shape torch.Size([384, 256, 1, 1]) from checkpoint, the shape in current model is torch.Size([768, 512, 1, 1]).
size mismatch for backbone.6.tcn1.qkv_conv.bias: copying a param with shape torch.Size([384]) from checkpoint, the shape in current model is torch.Size([768]).
size mismatch for backbone.6.tcn1.attn_out.weight: copying a param with shape torch.Size([256, 256, 1, 1]) from checkpoint, the shape in current model is torch.Size([512, 512, 1, 1]).
size mismatch for backbone.6.tcn1.attn_out.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([512]).
size mismatch for backbone.6.down1.conv.weight: copying a param with shape torch.Size([256, 128, 1, 1]) from checkpoint, the shape in current model is torch.Size([512, 256, 1, 1]).
size mismatch for backbone.6.down1.conv.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([512]).
size mismatch for backbone.6.down1.bn.weight: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([512]).
size mismatch for backbone.6.down1.bn.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([512]).
size mismatch for backbone.6.down1.bn.running_mean: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([512]).
size mismatch for backbone.6.down1.bn.running_var: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([512]).
size mismatch for backbone.7.gcn1.conv_list.0.weight: copying a param with shape torch.Size([256, 256, 1, 1]) from checkpoint, the shape in current model is torch.Size([512, 512, 1, 1]).
size mismatch for backbone.7.gcn1.conv_list.0.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([512]).
size mismatch for backbone.7.gcn1.conv_list.1.weight: copying a param with shape torch.Size([256, 256, 1, 1]) from checkpoint, the shape in current model is torch.Size([512, 512, 1, 1]).
size mismatch for backbone.7.gcn1.conv_list.1.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([512]).
size mismatch for backbone.7.gcn1.conv_list.2.weight: copying a param with shape torch.Size([256, 256, 1, 1]) from checkpoint, the shape in current model is torch.Size([512, 512, 1, 1]).
size mismatch for backbone.7.gcn1.conv_list.2.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([512]).
size mismatch for backbone.7.gcn1.bn.weight: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([512]).
size mismatch for backbone.7.gcn1.bn.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([512]).
size mismatch for backbone.7.gcn1.bn.running_mean: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([512]).
size mismatch for backbone.7.gcn1.bn.running_var: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([512]).
size mismatch for backbone.7.tcn1.bn.weight: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([512]).
size mismatch for backbone.7.tcn1.bn.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([512]).
size mismatch for backbone.7.tcn1.bn.running_mean: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([512]).
size mismatch for backbone.7.tcn1.bn.running_var: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([512]).
size mismatch for backbone.7.tcn1.data_bn.weight: copying a param with shape torch.Size([4608]) from checkpoint, the shape in current model is torch.Size([9216]).
size mismatch for backbone.7.tcn1.data_bn.bias: copying a param with shape torch.Size([4608]) from checkpoint, the shape in current model is torch.Size([9216]).
size mismatch for backbone.7.tcn1.data_bn.running_mean: copying a param with shape torch.Size([4608]) from checkpoint, the shape in current model is torch.Size([9216]).
size mismatch for backbone.7.tcn1.data_bn.running_var: copying a param with shape torch.Size([4608]) from checkpoint, the shape in current model is torch.Size([9216]).
size mismatch for backbone.7.tcn1.qkv_conv.weight: copying a param with shape torch.Size([384, 256, 1, 1]) from checkpoint, the shape in current model is torch.Size([768, 512, 1, 1]).
size mismatch for backbone.7.tcn1.qkv_conv.bias: copying a param with shape torch.Size([384]) from checkpoint, the shape in current model is torch.Size([768]).
size mismatch for backbone.7.tcn1.attn_out.weight: copying a param with shape torch.Size([256, 256, 1, 1]) from checkpoint, the shape in current model is torch.Size([512, 512, 1, 1]).
size mismatch for backbone.7.tcn1.attn_out.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([512]).
size mismatch for backbone.8.gcn1.conv_list.0.weight: copying a param with shape torch.Size([256, 256, 1, 1]) from checkpoint, the shape in current model is torch.Size([512, 512, 1, 1]).
size mismatch for backbone.8.gcn1.conv_list.0.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([512]).
size mismatch for backbone.8.gcn1.conv_list.1.weight: copying a param with shape torch.Size([256, 256, 1, 1]) from checkpoint, the shape in current model is torch.Size([512, 512, 1, 1]).
size mismatch for backbone.8.gcn1.conv_list.1.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([512]).
size mismatch for backbone.8.gcn1.conv_list.2.weight: copying a param with shape torch.Size([256, 256, 1, 1]) from checkpoint, the shape in current model is torch.Size([512, 512, 1, 1]).
size mismatch for backbone.8.gcn1.conv_list.2.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([512]).
size mismatch for backbone.8.gcn1.bn.weight: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([512]).
size mismatch for backbone.8.gcn1.bn.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([512]).
size mismatch for backbone.8.gcn1.bn.running_mean: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([512]).
size mismatch for backbone.8.gcn1.bn.running_var: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([512]).
size mismatch for backbone.8.tcn1.bn.weight: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([512]).
size mismatch for backbone.8.tcn1.bn.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([512]).
size mismatch for backbone.8.tcn1.bn.running_mean: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([512]).
size mismatch for backbone.8.tcn1.bn.running_var: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([512]).
size mismatch for backbone.8.tcn1.data_bn.weight: copying a param with shape torch.Size([4608]) from checkpoint, the shape in current model is torch.Size([9216]).
size mismatch for backbone.8.tcn1.data_bn.bias: copying a param with shape torch.Size([4608]) from checkpoint, the shape in current model is torch.Size([9216]).
size mismatch for backbone.8.tcn1.data_bn.running_mean: copying a param with shape torch.Size([4608]) from checkpoint, the shape in current model is torch.Size([9216]).
size mismatch for backbone.8.tcn1.data_bn.running_var: copying a param with shape torch.Size([4608]) from checkpoint, the shape in current model is torch.Size([9216]).
size mismatch for backbone.8.tcn1.qkv_conv.weight: copying a param with shape torch.Size([384, 256, 1, 1]) from checkpoint, the shape in current model is torch.Size([768, 512, 1, 1]).
size mismatch for backbone.8.tcn1.qkv_conv.bias: copying a param with shape torch.Size([384]) from checkpoint, the shape in current model is torch.Size([768]).
size mismatch for backbone.8.tcn1.attn_out.weight: copying a param with shape torch.Size([256, 256, 1, 1]) from checkpoint, the shape in current model is torch.Size([512, 512, 1, 1]).
size mismatch for backbone.8.tcn1.attn_out.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([512]).
size mismatch for gcn0.conv_list.0.weight: copying a param with shape torch.Size([64, 3, 1, 1]) from checkpoint, the shape in current model is torch.Size([128, 6, 1, 1]).
size mismatch for gcn0.conv_list.0.bias: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for gcn0.conv_list.1.weight: copying a param with shape torch.Size([64, 3, 1, 1]) from checkpoint, the shape in current model is torch.Size([128, 6, 1, 1]).
size mismatch for gcn0.conv_list.1.bias: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for gcn0.conv_list.2.weight: copying a param with shape torch.Size([64, 3, 1, 1]) from checkpoint, the shape in current model is torch.Size([128, 6, 1, 1]).
size mismatch for gcn0.conv_list.2.bias: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for gcn0.bn.weight: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for gcn0.bn.bias: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for gcn0.bn.running_mean: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for gcn0.bn.running_var: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for tcn0.conv.weight: copying a param with shape torch.Size([64, 64, 9, 1]) from checkpoint, the shape in current model is torch.Size([128, 128, 9, 1]).
size mismatch for tcn0.conv.bias: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for tcn0.bn.weight: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for tcn0.bn.bias: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for tcn0.bn.running_mean: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for tcn0.bn.running_var: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for person_bn.weight: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([512]).
size mismatch for person_bn.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([512]).
size mismatch for person_bn.running_mean: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([512]).
size mismatch for person_bn.running_var: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([512]).
size mismatch for fcn.weight: copying a param with shape torch.Size([400, 256, 1]) from checkpoint, the shape in current model is torch.Size([400, 512, 1]).

Could you help me? tks

hi i set the config/stgcn/kinetics-skeleton/train.yaml like this:
windows_size :150
num_class 400
channel:6
num_point:18
num_person:2

and run main.py
here is the bug

File "main.py", line 959, in <module> processor.start() File "main.py", line 870, in start self.train(epoch, save_model=save_model) File "main.py", line 453, in train output = self.model(data, label, name) File "/home/lijianwei/.pyenv/versions/TTNet/lib/python3.8/site-packages/torch/nn/modules/module.py", line 550, in __call__ result = self.forward(*input, **kwargs) File "/home/lijianwei/.pyenv/versions/TTNet/lib/python3.8/site-packages/torch/nn/parallel/data_parallel.py", line 155, in forward outputs = self.parallel_apply(replicas, inputs, kwargs) File "/home/lijianwei/.pyenv/versions/TTNet/lib/python3.8/site-packages/torch/nn/parallel/data_parallel.py", line 165, in parallel_apply return parallel_apply(replicas, inputs, kwargs, self.device_ids[:len(replicas)]) File "/home/lijianwei/.pyenv/versions/TTNet/lib/python3.8/site-packages/torch/nn/parallel/parallel_apply.py", line 85, in parallel_apply output.reraise() File "/home/lijianwei/.pyenv/versions/TTNet/lib/python3.8/site-packages/torch/_utils.py", line 395, in reraise raise self.exc_type(msg) RuntimeError: Caught RuntimeError in replica 0 on device 0. Original Traceback (most recent call last): File "/home/lijianwei/.pyenv/versions/TTNet/lib/python3.8/site-packages/torch/nn/parallel/parallel_apply.py", line 60, in _worker output = module(*input, **kwargs) File "/home/lijianwei/.pyenv/versions/TTNet/lib/python3.8/site-packages/torch/nn/modules/module.py", line 550, in __call__ result = self.forward(*input, **kwargs) File "/home/lijianwei/st_tr_wsq/ST-TR-master/code/st_gcn/net/st_gcn.py", line 255, in forward x = self.data_bn(x) File "/home/lijianwei/.pyenv/versions/TTNet/lib/python3.8/site-packages/torch/nn/modules/module.py", line 550, in __call__ result = self.forward(*input, **kwargs) File "/home/lijianwei/.pyenv/versions/TTNet/lib/python3.8/site-packages/torch/nn/modules/batchnorm.py", line 103, in forward return F.batch_norm( File "/home/lijianwei/.pyenv/versions/TTNet/lib/python3.8/site-packages/torch/nn/functional.py", line 1921, in batch_norm return torch.batch_norm( RuntimeError: running_mean should contain 108 elements not 216

then i check the kinetics_train_joint.npy. The shape of that is (240436,3,300,18,2)

next i change the train.yaml like this:
windows_size :300
num_class 600
channel:3
num_point:18
num_person:2

and i run the main.py. I could run the train step with no bugs but still error in val step. it might be the num_class error . and i set num_class 400 . it comes the cuda error
A1 = self.soft(torch.matmul(A1, A2) / A1.size(-1)) # N V V RuntimeError: CUDA error: CUBLAS_STATUS_ALLOC_FAILED when callingcublasCreate(handle)``
can you help me?

There are 5 pre-trained models for each split (xub vs. xview) on NTU60: ntu60_xsub_spatial.pt, ntu60_xsub_temporal.pt, ntu60_xsub_bones_spatial.pt, ntu60_xsub_bones_temporal.pt and ntu60_xsub_bones_temporal-agcn.pt. I was able to successfully test the first 4 by editing the configs in train.yaml, but cannot figure out the last one with AGCN. My configs are listed as follows (compared to ntu60_xsub_bones_temporal.pt the only thing I changed was agcn: True):

# feeder
feeder: st_gcn.feeder.Feeder
feeder_augmented: st_gcn.feeder.FeederAugmented
train_feeder_args:
  data_path: ../data/NTU-RGB-D_Processed/xsub/train_data_joint_bones.npy
  label_path: ../data/NTU-RGB-D_Processed/xsub/train_label.pkl
  random_choose: False
  random_shift: False
  random_move: False
  window_size: -1
  normalization: False
  mirroring: False
test_feeder_args:
  data_path: ../data/NTU-RGB-D_Processed/xsub/val_data_joint_bones.npy
  label_path: ../data/NTU-RGB-D_Processed/xsub/val_label.pkl
# model
model: st_gcn.net.ST_GCN
training: True
model_args:
  num_class: 60
  channel: 6
  window_size: 300
  num_point: 25
  num_person: 2
  mask_learning: True
  use_data_bn: True
  attention: False
  only_attention: True
  tcn_attention: True
  data_normalization: True
  skip_conn: True
  weight_matrix: 2
  only_temporal_attention: True
  bn_flag: True
  attention_3: False
  kernel_temporal: 9
  more_channels: False
  double_channel: True
  drop_connect: True
  concat_original: True
  all_layers: False
  adjacency: False
  agcn: True
  dv: 0.25
  dk: 0.25
  Nh: 8
  n: 4
  dim_block1: 10
  dim_block2: 30
  dim_block3: 75
  relative: False
  graph: st_gcn.graph.NTU_RGB_D
  visualization: False
  graph_args:
    labeling_mode: 'spatial'
  #optical_flow: True
# optim
#0: old one, 1: new one
scheduler: 1
weight_decay: 0.0001
base_lr: 0.1
step: [60,90]
# training
device: [0,1,2,3]
batch_size: 2
test_batch_size: 1
num_epoch: 120
nesterov: True

Here is the command I ran: python main.py --config config/st_gcn/nturgbd/train.yaml --phase test --weights ../checkpoint_ST-TR/ntu60_xsub_bones_temporal-agcn.pt

And here is the traceback of error messages:

Traceback (most recent call last):
  File "main.py", line 966, in <module>
    processor = Processor(arg)
  File "main.py", line 234, in __init__
    self.load_model()
  File "main.py", line 354, in load_model
    self.model.load_state_dict(state)
  File "/localhome/local-thtang/anaconda3/envs/st-tr/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1406, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for Model:
	size mismatch for fcn.weight: copying a param with shape torch.Size([120, 512, 1]) from checkpoint, the shape in current model is torch.Size([60, 512, 1]).
	size mismatch for fcn.bias: copying a param with shape torch.Size([120]) from checkpoint, the shape in current model is torch.Size([60]).

There are 18 key points used in openpose in your paper. I want to pick out the key points only for the arms, plus the key points for the hands. How do you make them? Thank you

Hi:
I found there is a problem exist in you code.
In st_gcn.py line 110, a tensor named self.A , stands for three adjacency matrix of Kinetics.
In unit_agcn.py, line 47 and 48
self.PA = torch.nn.Parameter(A)
torch.nn.init.constant_(self.PA, 1e-6)
The problem is when A is tensor type, after the initialization, the formal parameter A and the actual parameter self.A will be
changed to the initialized result 1e-6 . That means the next agcn(gcn) adjacency matrix will all be a tensor of (3, 18, 18) shape
and 1e-6 value, so this caused adjacency matrix information is useless in gcn state.
I found when line47 variable A is array type, it will be consistent after torch.nn.paramter.init operation.