hlt-hitsz / mgf Goto Github PK

由于显卡缘故，我在cuda11。1下运行，torch版本为1.8.0+cu111，dgl版本为dgl-cu111==0.6.1，并且在python控制台可以检测到gpu

请问这是什么原因捏？

[80,0,0] Assertion srcIndex < srcSelectDimSize failed.
/opt/conda/conda-bld/pytorch_1659484809535/work/aten/src/ATen/native/cuda/Indexing.cu:975: indexSelectLargeIndex: block: [424,0,0], thread: [81,0,0] Assertion srcIndex < srcSelectDimSize failed.
/opt/conda/conda-bld/pytorch_1659484809535/work/aten/src/ATen/native/cuda/Indexing.cu:975: indexSelectLargeIndex: block: [424,0,0], thread: [82,0,0] Assertion srcIndex < srcSelectDimSize failed.
/opt/conda/conda-bld/pytorch_1659484809535/work/aten/src/ATen/native/cuda/Indexing.cu:975: indexSelectLargeIndex: block: [424,0,0], thread: [83,0,0] Assertion srcIndex < srcSelectDimSize failed.
/opt/conda/conda-bld/pytorch_1659484809535/work/aten/src/ATen/native/cuda/Indexing.cu:975: indexSelectLargeIndex: block: [424,0,0], thread: [84,0,0] Assertion srcIndex < srcSelectDimSize failed.
/opt/conda/conda-bld/pytorch_1659484809535/work/aten/src/ATen/native/cuda/Indexing.cu:975: indexSelectLargeIndex: block: [424,0,0], thread: [85,0,0] Assertion srcIndex < srcSelectDimSize failed.
/opt/conda/conda-bld/pytorch_1659484809535/work/aten/src/ATen/native/cuda/Indexing.cu:975: indexSelectLargeIndex: block: [424,0,0], thread: [86,0,0] Assertion srcIndex < srcSelectDimSize failed.
/opt/conda/conda-bld/pytorch_1659484809535/work/aten/src/ATen/native/cuda/Indexing.cu:975: indexSelectLargeIndex: block: [424,0,0], thread: [87,0,0] Assertion srcIndex < srcSelectDimSize failed.
/opt/conda/conda-bld/pytorch_1659484809535/work/aten/src/ATen/native/cuda/Indexing.cu:975: indexSelectLargeIndex: block: [424,0,0], thread: [88,0,0] Assertion srcIndex < srcSelectDimSize failed.
/opt/conda/conda-bld/pytorch_1659484809535/work/aten/src/ATen/native/cuda/Indexing.cu:975: indexSelectLargeIndex: block: [424,0,0], thread: [89,0,0] Assertion srcIndex < srcSelectDimSize failed.
/opt/conda/conda-bld/pytorch_1659484809535/work/aten/src/ATen/native/cuda/Indexing.cu:975: indexSelectLargeIndex: block: [424,0,0], thread: [90,0,0] Assertion srcIndex < srcSelectDimSize failed.
/opt/conda/conda-bld/pytorch_1659484809535/work/aten/src/ATen/native/cuda/Indexing.cu:975: indexSelectLargeIndex: block: [424,0,0], thread: [91,0,0] Assertion srcIndex < srcSelectDimSize failed.
/opt/conda/conda-bld/pytorch_1659484809535/work/aten/src/ATen/native/cuda/Indexing.cu:975: indexSelectLargeIndex: block: [424,0,0], thread: [92,0,0] Assertion srcIndex < srcSelectDimSize failed.
/opt/conda/conda-bld/pytorch_1659484809535/work/aten/src/ATen/native/cuda/Indexing.cu:975: indexSelectLargeIndex: block: [424,0,0], thread: [93,0,0] Assertion srcIndex < srcSelectDimSize failed.
/opt/conda/conda-bld/pytorch_1659484809535/work/aten/src/ATen/native/cuda/Indexing.cu:975: indexSelectLargeIndex: block: [424,0,0], thread: [94,0,0] Assertion srcIndex < srcSelectDimSize failed.
/opt/conda/conda-bld/pytorch_1659484809535/work/aten/src/ATen/native/cuda/Indexing.cu:975: indexSelectLargeIndex: block: [424,0,0], thread: [95,0,0] Assertion srcIndex < srcSelectDimSize failed.
0%| | 0/1816 [08:18<?, ?it/s]
Traceback (most recent call last):
File "/home/bfs/.pycharm_helpers/pydev/pydevd.py", line 1438, in _exec
pydev_imports.execfile(file, globals, locals) # execute the script
File "/home/bfs/.pycharm_helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "/home/bfs/MGF/run.py", line 331, in
reply_para_tokens_list, reply_tags_list, train_graph_batch)
File "/home/bfs/miniconda3/envs/MGF/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "./utils/models.py", line 115, in forward
= self.am_tagging(review_para_tokens_list, review_tags_list, mode='train')
File "./utils/models.py", line 80, in am_tagging
sent_emb = self.bert_emb(para_tokens_list)
File "./utils/models.py", line 66, in bert_emb
bert_outputs = self.bert(ids_padding_tensor, attention_mask = mask_tensor)
File "/home/bfs/miniconda3/envs/MGF/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/home/bfs/miniconda3/envs/MGF/lib/python3.7/site-packages/transformers/models/bert/modeling_bert.py", line 1016, in forward
past_key_values_length=past_key_values_length,
File "/home/bfs/miniconda3/envs/MGF/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/home/bfs/miniconda3/envs/MGF/lib/python3.7/site-packages/transformers/models/bert/modeling_bert.py", line 236, in forward
token_type_embeddings = self.token_type_embeddings(token_type_ids)
File "/home/bfs/miniconda3/envs/MGF/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/home/bfs/miniconda3/envs/MGF/lib/python3.7/site-packages/torch/nn/modules/sparse.py", line 160, in forward
self.norm_type, self.scale_grad_by_freq, self.sparse)
File "/home/bfs/miniconda3/envs/MGF/lib/python3.7/site-packages/torch/nn/functional.py", line 2199, in embedding
return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
RuntimeError: CUDA error: device-side assert triggered
CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.

Process finished with exit code 1

config.py里的这几个文件都没有诶，下图是我的项目运行了prepare_data.py和to_bioes.py后的项目结构

config.py里缺的文件能发一下吗