svjan5 / kg-reeval Goto Github PK

so in the end ,i couldn`t get the test result. hit@1,hit@3,hit@10 are all 0!

I used the following command to run the code:
python conve.py -name reprod_fb15k_237 -data FB15k-237 -gpu 0 -eval_type random
Error after 1 epoch:

Traceback (most recent call last):
  File "conve.py", line 318, in <module>
    model.fit()
  File "conve.py", line 250, in fit
    val_results = self.evaluate('valid', epoch)
  File "conve.py", line 139, in evaluate
    left_results, left_scores   = self.predict(split=split, mode='tail_batch')
  File "conve.py", line 158, in predict
    pred, zero_cnt              = self.model.forward(sub, rel, None, zero_cnt=True)
TypeError: forward() got multiple values for argument 'zero_cnt'

Hi, I'm really interested in your paper A Re-evaluation of Knowledge Graph Completion Methods in 2020ACL which is excellent work. But I find it hard to understand your meaning about figure 3. May I ask your explanations about figure 3? In other words,：

1. How do we understand the x-coordinate and the y-coordinate at the same time？
2. How do you save the data to make Figure 3 look like this?
3. Since the problem is the relu function, can we use prelu instead to improve the performance?

请问您完善的代码在哪个文件，暂时没有找到，谢谢

Hi, Have you studied whether RGHAT has the same data leakage problem as KBGAT?

when I run the code of KBAT, it raises an error that can not find the file entity2vec.txt, where can I find that file , or just revise the pre_embed to be false default?

Hi Zhiqing, Shikhar,
I was trying to run RotatE baselines.
I got this error,

UnboundLocalError: local variable 'positive_arg' referenced before assignment

I could see in the model.py file 'positive_arg' is defined before its usage. Could you please let me know if I am missing something obvious?

Thanks for your help!

File "conve.py", line 312, in
model = Main(args)
File "conve.py", line 82, in init
self.logger = get_logger(self.p.name, self.p.log_dir, self.p.config_dir)
File "C:\Users\Kano_Hayashi\Desktop\kg-reeval-master\ConvE\helper.py", line 52, in get_logger
logging.config.dictConfig(config_dict)
File "C:\Users\Kano_Hayashi.conda\envs\rota\lib\logging\config.py", line 800, in dictConfig
dictConfigClass(config).configure()
File "C:\Users\Kano_Hayashi.conda\envs\rota\lib\logging\config.py", line 571, in configure
'%r' % name) from e
ValueError: Unable to configure handler 'file_handler'

In RotatE new evaluation, you change the order of scores before sorting. So the key problem becomes the sort algorithm used in torch.argsort should be stable. I am not sure about this. But I google it. Seems like it's not. So I think it would be a problem. Maybe we should change to np.argsort(xxx, kind='stable')?
Try to this:

import torch

a = torch.tensor(
[[ 0., 3.],
[ 2., 3.],
[ 2., 2.],
[10., 2.],
[ 0., 2.],
[ 6., 2.],
[10., 1.],
[ 2., 1.],
[ 0., 1.],
[ 6., 1.],
[10., 0.],
[12., 0.]]
)
print(a[torch.argsort(a[:, 0])])

Output:

tensor([[ 0., 3.],
[ 0., 2.],
[ 0., 1.],
[ 2., 1.],
[ 2., 2.],
[ 2., 3.],
[ 6., 1.],
[ 6., 2.],
[10., 1.],
[10., 2.],
[10., 0.],
[12., 0.]])

I am very interested in KBAT. Can you provide the best hyper-parameter for each dataset after correcting the test leakage problem? Thanks a lot. Moreover, have you done any work at SpGAT+ConvE? I'm trying to do this. But the results are not good enough on UMLS and Kinship.

I want to know whether the result reported in your paper about KBAT is the result after fix. I only saw the content of the evaluation protocol modified in the code, and did not see the modification of the valid_triples_dict you mentioned, thanks.