Why I get the 0.94 in hop2, 0.972 in hop1, 0.48 in hop3 (metaQA full KG) about embedkgqa HOT 12 CLOSED

Can you please tell the exact commands you used?

from embedkgqa.

parser = argparse.ArgumentParser()
parser.add_argument('--dataname', type=str, default='metaQA')
parser.add_argument('--hops', type=str, default='3')
parser.add_argument('--ls', type=float, default=0.0)
parser.add_argument('--validate_every', type=int, default=5)
parser.add_argument('--model', type=str, default='ComplEx')
parser.add_argument('--kg_type', type=str, default='full')

parser.add_argument('--mode', type=str, default='train')
parser.add_argument('--batch_size', type=int, default=128)
parser.add_argument('--dropout', type=float, default=0.1)
parser.add_argument('--entdrop', type=float, default=0.1)
parser.add_argument('--reldrop', type=float, default=0.2)
parser.add_argument('--scoredrop', type=float, default=0.2)
parser.add_argument('--l3_reg', type=float, default=0.0)
parser.add_argument('--decay', type=float, default=1.0)
parser.add_argument('--shuffle_data', type=bool, default=True)
parser.add_argument('--num_workers', type=int, default=15)
parser.add_argument('--lr', type=float, default=0.0005)
parser.add_argument('--nb_epochs', type=int, default=90)
parser.add_argument('--gpu', type=int, default=4)
parser.add_argument('--neg_batch_size', type=int, default=128)
parser.add_argument('--hidden_dim', type=int, default=256)
parser.add_argument('--embedding_dim', type=int, default=256)
parser.add_argument('--relation_dim', type=int, default=200)
parser.add_argument('--use_cuda', type=bool, default=True)
parser.add_argument('--patience', type=int, default=5)
parser.add_argument('--freeze', type=str2bool, default=True)

from embedkgqa.

Are you using the pretrained KG embeddings or did you retrain them?

from embedkgqa.

Yes, I retrained it by your command in #11.

from embedkgqa.

Those aren't the best hyperparameters in #11, I just wrote that command to clarify that it does work even when batch_norm is set to 0. Please set batch_norm=1, and if possible, please let me know the MRR you get when training full MetaQA KG embedding. It should be very close to 1.0, if not 1.0 .

from embedkgqa.

Also, as mentioned in #15, we need relation scoring module for 3-hop SOTA performance. Without it we get ~0.75 test accuracy

from embedkgqa.

if set batch_norm=1, #11 is the best hyperparameters ?

from embedkgqa.

Let me confirm, please wait 15-20 minutes

from embedkgqa.

@ToneLi Yes, please try same command with batch_norm = 1. As explained in #11, this is needed because of the implementation of KG embedding done by https://github.com/ibalazevic/TuckER from which this code has been taken

from embedkgqa.

Closing this for now, please reopen if issue remains @ToneLi

from embedkgqa.

Hi, Sorry to disturb you, I used the hyperparamaters, why I get the HIT value is 1. It's the right result? (FULL KG metaQA)

from embedkgqa.

I train KG embedding again, but I still cannot get 0.49 in metaQA-full KG-3hops, do you have any advice to get 0.75 you mentioned?

from embedkgqa.