renli1024 / se-gnn Goto Github PK

Semantic Evidence aware Graph Neural Network (SE-GNN) for Knowledge Graph Embedding task (AAAI'22).

Python 100.00%

se-gnn's Introduction

SE-GNN

This is the PyTorch implementation of the Semantic Evidence aware Graph Neural Network (SE-GNN) for Knowledge Graph Embedding task, as described in our paper:

Ren Li, Yanan Cao, Qiannan Zhu, Guanqun Bi, Fang Fang, Yi Liu, Qian Li, How Does Knowledge Graph Embedding Extrapolate to Unseen Data: a Semantic Evidence View (AAAI'22)

Dependencies

PyTorch >= 1.9.1
DGL >= 0.7.2 (for graph neural network implementation)
Hydra >= 1.1.1 (for project configuration)

Usage

code/: includes code scripts.
data/:
- dataset/FB15k_237/: FB15k-237 dataset resources
- dataset/WN18RR/: WN18RR dataset resources
- output/FB15k_237/: model outputs for FB15k-237 dataset
- output/WN18RR/: model outputs for WN18RR dataset
config/: We use Hydra toolkit to manage the model hyper-parameters, which can be either stored in YAML file or passed by command-line. More details can be seen in official docs.
- config.yaml: project configurations and general parameters
- dataset/FB15k_237.yaml: best hyper-parameters for FB15k-237 dataset
- dataset/WN18RR.yaml: best hyper-parameters for WN18RR dataset
drawing/: includes materials to re-draw the paper figs.
- data_analyse.py: drawing code
- FB15k_237/ and WN18RR/: includes computed Semantic Evidence metrics data2metrics and reproduced baseline prediction xx_rank. More details can be found in our paper.

Model Training

# enter the project directory
cd SE-GNN-main

# set the config/config.yaml `project_dir` field to your own project path

# FB15k-237
python code/run.py

# WN18RR
python code/run.py dataset=WN18RR

# We store the best hyper-parameters in dataset's corresponding .yaml file, 
# and there are two ways to re-set the parameter:
# 1. Directly modify the .yaml file;
# 2. Pass the value by command-line, like:
# python code/run.py dataset=WN18RR rel_drop=0.2 ...

# draw the FB15k-237 pictures (paper Figure 2)
python drawing/data_analyse.py

# draw the WN18RR pictures (paper Figure 6)
python drawing/data_analyse.py dataset=WN18RR

The model takes about 10h for training on a single GPU, and the GPU memory cost is about 11GB for FB15k-237 and 3GB for WN18RR dataset.

Citation

Please cite the following paper if you use this code in your work:

@inproceedings{li2022segnn,
    title={How Does Knowledge Graph Embedding Extrapolate to Unseen Data: a Semantic Evidence View},
    author={Ren Li and Yanan Cao and Qiannan Zhu and Guanqun Bi and Fang Fang and Yi Liu and Qian Li},
    booktitle={Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2022},
    year={2022}
}

If you have any questions about the paper or the code, please feel free to create an issue or contact Ren Li <[email protected]>. Sometimes I may not reply very quickly because of the engaged matters, but I will do it asap when I am free :)

se-gnn's People

Contributors

Stargazers

Watchers

Forkers

foboshi moguizhizi hell-to-heaven modberge fengjixuchui0001028 fantastic-413 hussien tjudingling

se-gnn's Issues

The GNN Aggregation Neighborhood Problem

Hello! In the model diagram of this article, the aggregation of neighborhoods is bidirectional. In Eq. (1), the neighbouring entities only include the entities pointing to the target entity, without considering the entities pointed to by the target entity. Does this result in inconsistent or incomplete aggregations? Or is the definition of $Ni$ incomplete?

Looking forward to your reply

有关图4的实验

我想咨询一下对于图四中对比convE和se-gnn对语义嵌入进行比较的结果。
在这个结果中convE的实体嵌入是怎么来的，对于关系的嵌入可能是随机的，但是实体也是随机的吗。那如果都是随机的话，何来的隐式语义，求回复感谢您

请问怎么做消融实验呢？

非常荣幸能看到您的工作。
我的导师希望我能阅读您的论文并调通您的代码，但是因为我的经验不足，不知道您如何完成消融实验，请问在消融实验时是直接将对应的layer部分全部注释掉吗？

Which gnn does the model use? Is it gat?

运行环境报错

您好！我在运行的过程中遇到问题，报错如下：
------Config------
{'dataset': 'FB15k_237', 'epoch': 600, 'batch_size': 1024, 'learning_rate': 0.00035, 'h_dim': 450, 'pred_rel_w': False, 'label_smooth': 0.1, 'kg_layer': 2, 'rm_rate': 0.5, 'ent_drop': 0.3, 'rel_drop': 0.1, 'comp_op': 'mul', 'bn': False, 'k_h': 15, 'k_w': 30, 'conv_drop': 0.3, 'ent_drop_pred': 0.3, 'fc_drop': 0.5, 'ker_sz': 8, 'out_channel': 200, 'device': 0, 'max_no_improve': 50, 'cpu_worker_num': 10, 'warmup_epoch': 5}
[2023-02-28 11:28:06,052][root][INFO] - Code dir path: code
[2023-02-28 11:28:06,052][root][INFO] - Config dir path: config
[2023-02-28 11:28:06,052][root][INFO] - Model save path: /home/yuxc/SE-GNN-main/data/output/FB15k_237/2023-02-28/11-28-05
cp: cannot stat 'code': No such file or directory
cp: cannot stat 'config': No such file or directory
/home/yuxc/anaconda3/envs/hypernode/lib/python3.6/site-packages/torch/nn/modules/container.py:434: UserWarning: Setting attributes on ParameterList is not supported.
warnings.warn("Setting attributes on ParameterList is not supported.")
[2023-02-28 11:28:13,029][root][INFO] - kg # node: 14505
[2023-02-28 11:28:13,030][root][INFO] - kg # edge: 544230
[2023-02-28 11:28:13,030][root][INFO] - kg # zero deg node: 0
[2023-02-28 11:28:13,030][root][INFO] - ---Load Train Data---
[2023-02-28 11:28:15,710][root][INFO] - -----Model Parameter Configuration-----
[2023-02-28 11:28:15,710][root][INFO] - Parameter ent_emb: torch.Size([14541, 450]), require_grad = True
[2023-02-28 11:28:15,710][root][INFO] - Parameter pred_rel_emb: torch.Size([474, 450]), require_grad = True
[2023-02-28 11:28:15,710][root][INFO] - Parameter edge_layers.0.neigh_w: torch.Size([450, 450]), require_grad = True
[2023-02-28 11:28:15,710][root][INFO] - Parameter edge_layers.1.neigh_w: torch.Size([450, 450]), require_grad = True
[2023-02-28 11:28:15,710][root][INFO] - Parameter node_layers.0.neigh_w: torch.Size([450, 450]), require_grad = True
[2023-02-28 11:28:15,710][root][INFO] - Parameter node_layers.1.neigh_w: torch.Size([450, 450]), require_grad = True
[2023-02-28 11:28:15,710][root][INFO] - Parameter comp_layers.0.neigh_w: torch.Size([450, 450]), require_grad = True
[2023-02-28 11:28:15,710][root][INFO] - Parameter comp_layers.1.neigh_w: torch.Size([450, 450]), require_grad = True
[2023-02-28 11:28:15,710][root][INFO] - Parameter rel_embs.0: torch.Size([474, 450]), require_grad = True
[2023-02-28 11:28:15,710][root][INFO] - Parameter rel_embs.1: torch.Size([474, 450]), require_grad = True
[2023-02-28 11:28:15,711][root][INFO] - Parameter predictor.bn0.weight: torch.Size([1]), require_grad = True
[2023-02-28 11:28:15,711][root][INFO] - Parameter predictor.bn0.bias: torch.Size([1]), require_grad = True
[2023-02-28 11:28:15,711][root][INFO] - Parameter predictor.bn1.weight: torch.Size([200]), require_grad = True
[2023-02-28 11:28:15,711][root][INFO] - Parameter predictor.bn1.bias: torch.Size([200]), require_grad = True
[2023-02-28 11:28:15,711][root][INFO] - Parameter predictor.bn2.weight: torch.Size([450]), require_grad = True
[2023-02-28 11:28:15,711][root][INFO] - Parameter predictor.bn2.bias: torch.Size([450]), require_grad = True
[2023-02-28 11:28:15,711][root][INFO] - Parameter predictor.conv.weight: torch.Size([200, 1, 8, 8]), require_grad = True
[2023-02-28 11:28:15,711][root][INFO] - Parameter predictor.fc.weight: torch.Size([450, 105800]), require_grad = True
[2023-02-28 11:28:15,712][root][INFO] - Training... total epoch: 600, step: 88200
Error executing job with overrides: []
Traceback (most recent call last):
File "code/run.py", line 146, in main
train_log = train_step(model, batch_data, kg, optimizer, scheduler)
File "/home/yuxc/SE-GNN-main/code/model_helper.py", line 27, in train_step
kg.remove_edges(rm_edges)
AttributeError: 'DGLHeteroGraph' object has no attribute 'remove_edges'

Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.

第一处为project_dir_path路径获取问题，第二处是dgl环境问题，请问您知道如何解决吗

您好，有关代码运行内存溢出的问题

D:\anaconda\envs\py38\python.exe C:/Users/wsco28/Desktop/SE-GNN-main/code/run.py
C:/Users/wsco28/Desktop/SE-GNN-main/code/run.py:78: UserWarning:
The version_base parameter is not specified.
Please specify a compatability version level, or None.
Will assume defaults for version 1.1
@hydra.main(config_path=join('..', 'config'), config_name="config")
D:\anaconda\envs\py38\lib\site-packages\hydra_internal\hydra.py:119: UserWarning: Future Hydra versions will no longer change working directory at job runtime by default.
See https://hydra.cc/docs/next/upgrades/1.1_to_1.2/changes_to_job_working_dir/ for more information.
ret = run_job(
[2022-05-19 10:21:29,312][root][INFO] -
------Config------
{'dataset': 'FB15k_237', 'epoch': 600, 'batch_size': 128, 'learning_rate': 0.00035, 'h_dim': 450, 'pred_rel_w': False, 'label_smooth': 0.1, 'kg_layer': 2, 'rm_rate': 0.5, 'ent_drop': 0.3, 'rel_drop': 0.1, 'comp_op': 'mul', 'bn': False, 'k_h': 15, 'k_w': 30, 'conv_drop': 0.3, 'ent_drop_pred': 0.3, 'fc_drop': 0.5, 'ker_sz': 8, 'out_channel': 200, 'device': 0, 'max_no_improve': 50, 'cpu_worker_num': 10, 'warmup_epoch': 5}
[2022-05-19 10:21:29,312][root][INFO] - Code dir path: C:/Users/wsco28/Desktop/SE-GNN-main/code
[2022-05-19 10:21:29,312][root][INFO] - Config dir path: C:/Users/wsco28/Desktop/SE-GNN-main\config
[2022-05-19 10:21:29,312][root][INFO] - Model save path: C:\Users\wsco28\Desktop\SE-GNN-main\data\output\FB15k_237\2022-05-19\10-21-29
'cp' ��ڲ��ⲿ��Ҳ��ǿ��еĳ��
��ļ��
'cp' ��ڲ��ⲿ��Ҳ��ǿ��еĳ��
��ļ��
[2022-05-19 10:21:33,070][root][INFO] - kg # node: 14541
[2022-05-19 10:21:33,070][root][INFO] - kg # edge: 544230
[2022-05-19 10:21:33,071][root][INFO] - kg # zero deg node: 36
[2022-05-19 10:21:33,071][root][INFO] - ---Load Train Data---
[2022-05-19 10:21:35,245][root][INFO] - -----Model Parameter Configuration-----
[2022-05-19 10:21:35,245][root][INFO] - Parameter ent_emb: torch.Size([14541, 450]), require_grad = True
[2022-05-19 10:21:35,245][root][INFO] - Parameter pred_rel_emb: torch.Size([474, 450]), require_grad = True
[2022-05-19 10:21:35,245][root][INFO] - Parameter edge_layers.0.neigh_w: torch.Size([450, 450]), require_grad = True
[2022-05-19 10:21:35,245][root][INFO] - Parameter edge_layers.1.neigh_w: torch.Size([450, 450]), require_grad = True
[2022-05-19 10:21:35,246][root][INFO] - Parameter node_layers.0.neigh_w: torch.Size([450, 450]), require_grad = True
[2022-05-19 10:21:35,246][root][INFO] - Parameter node_layers.1.neigh_w: torch.Size([450, 450]), require_grad = True
[2022-05-19 10:21:35,246][root][INFO] - Parameter comp_layers.0.neigh_w: torch.Size([450, 450]), require_grad = True
[2022-05-19 10:21:35,246][root][INFO] - Parameter comp_layers.1.neigh_w: torch.Size([450, 450]), require_grad = True
[2022-05-19 10:21:35,246][root][INFO] - Parameter rel_embs.0: torch.Size([474, 450]), require_grad = True
[2022-05-19 10:21:35,246][root][INFO] - Parameter rel_embs.1: torch.Size([474, 450]), require_grad = True
[2022-05-19 10:21:35,246][root][INFO] - Parameter predictor.bn0.weight: torch.Size([1]), require_grad = True
[2022-05-19 10:21:35,246][root][INFO] - Parameter predictor.bn0.bias: torch.Size([1]), require_grad = True
[2022-05-19 10:21:35,246][root][INFO] - Parameter predictor.bn1.weight: torch.Size([200]), require_grad = True
[2022-05-19 10:21:35,246][root][INFO] - Parameter predictor.bn1.bias: torch.Size([200]), require_grad = True
[2022-05-19 10:21:35,246][root][INFO] - Parameter predictor.bn2.weight: torch.Size([450]), require_grad = True
[2022-05-19 10:21:35,246][root][INFO] - Parameter predictor.bn2.bias: torch.Size([450]), require_grad = True
[2022-05-19 10:21:35,246][root][INFO] - Parameter predictor.conv.weight: torch.Size([200, 1, 8, 8]), require_grad = True
[2022-05-19 10:21:35,246][root][INFO] - Parameter predictor.fc.weight: torch.Size([450, 105800]), require_grad = True
[2022-05-19 10:21:35,247][root][INFO] - Training... total epoch: 600, step: 702000
C:\Users\wsco28\Desktop\SE-GNN-main\code\run.py:78: UserWarning:
The version_base parameter is not specified.
Please specify a compatability version level, or None.
Will assume defaults for version 1.1
@hydra.main(config_path=join('..', 'config'), config_name="config")
C:\Users\wsco28\Desktop\SE-GNN-main\code\run.py:78: UserWarning:
The version_base parameter is not specified.
Please specify a compatability version level, or None.
Will assume defaults for version 1.1
@hydra.main(config_path=join('..', 'config'), config_name="config")
C:\Users\wsco28\Desktop\SE-GNN-main\code\run.py:78: UserWarning:
The version_base parameter is not specified.
Please specify a compatability version level, or None.
Will assume defaults for version 1.1
@hydra.main(config_path=join('..', 'config'), config_name="config")
C:\Users\wsco28\Desktop\SE-GNN-main\code\run.py:78: UserWarning:
The version_base parameter is not specified.
Please specify a compatability version level, or None.
Will assume defaults for version 1.1
@hydra.main(config_path=join('..', 'config'), config_name="config")
C:\Users\wsco28\Desktop\SE-GNN-main\code\run.py:78: UserWarning:
The version_base parameter is not specified.
Please specify a compatability version level, or None.
Will assume defaults for version 1.1
@hydra.main(config_path=join('..', 'config'), config_name="config")
C:\Users\wsco28\Desktop\SE-GNN-main\code\run.py:78: UserWarning:
The version_base parameter is not specified.
Please specify a compatability version level, or None.
Will assume defaults for version 1.1
@hydra.main(config_path=join('..', 'config'), config_name="config")
C:\Users\wsco28\Desktop\SE-GNN-main\code\run.py:78: UserWarning:
The version_base parameter is not specified.
Please specify a compatability version level, or None.
Will assume defaults for version 1.1
@hydra.main(config_path=join('..', 'config'), config_name="config")
C:\Users\wsco28\Desktop\SE-GNN-main\code\run.py:78: UserWarning:
The version_base parameter is not specified.
Please specify a compatability version level, or None.
Will assume defaults for version 1.1
@hydra.main(config_path=join('..', 'config'), config_name="config")
C:\Users\wsco28\Desktop\SE-GNN-main\code\run.py:78: UserWarning:
The version_base parameter is not specified.
Please specify a compatability version level, or None.
Will assume defaults for version 1.1
@hydra.main(config_path=join('..', 'config'), config_name="config")
C:\Users\wsco28\Desktop\SE-GNN-main\code\run.py:78: UserWarning:
The version_base parameter is not specified.
Please specify a compatability version level, or None.
Will assume defaults for version 1.1
@hydra.main(config_path=join('..', 'config'), config_name="config")
Error executing job with overrides: []
Traceback (most recent call last):
File "C:/Users/wsco28/Desktop/SE-GNN-main/code/run.py", line 146, in main
train_log = train_step(model, batch_data, kg, optimizer, scheduler)
File "C:\Users\wsco28\Desktop\SE-GNN-main\code\model_helper.py", line 30, in train_step
loss.backward()
File "D:\anaconda\envs\py38\lib\site-packages\torch\tensor.py", line 245, in backward
torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs)
File "D:\anaconda\envs\py38\lib\site-packages\torch\autograd_init_.py", line 145, in backward
Variable._execution_engine.run_backward(
RuntimeError: CUDA out of memory. Tried to allocate 934.00 MiB (GPU 0; 12.00 GiB total capacity; 8.53 GiB already allocated; 88.81 MiB free; 9.52 GiB reserved in total by PyTorch)

Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.

Process finished with exit code 1

您好，请问一下这个内存溢出应该如何解决，我调了237.yaml的bacth_size不起效果~

实验环境

你好，

我在运行代码时，一直因为实验环境的问题报错，您能分享一下实验环境的配置吗？

Miao