zju-daily / metakg Goto Github PK

I tried to reproduce the experimental result, and fortunately I got the slightly better performance results than what was reported in original paper when I used ALL datasets in a UIC scenario.

However, when I tried to reproduce the results in a UC scenario, the Amazon and LFM datasets reported results that were far inferior to those reported in the original paper.

I want to know if there is anything I have overlooked. Can you publish the optimal hyperparameter settings of each data set in the UC scenario? Or are there other things that need to be paid attention to to ensure that the experimental results in the article can be reproduced?

Thank you so much

HERE is the logs.

Amazon:

python main.py --dataset amazon-book --batch_size 3 --num_inner_update 1 --node_dropout 0.4

+-------+-------------------+-------------------+--------------------+----------------------------------------------------------+----------------------------------------------------------+
| Epoch | training time | tesing time | Loss | recall | ndcg |
+-------+-------------------+-------------------+--------------------+----------------------------------------------------------+----------------------------------------------------------+
| 100 | 7.052807092666626 | 8.986499309539795 | 0.1047884076833725 | [0.18275862 0.25221675 0.3 0.33743842 0.36600985] | [0.18255526 0.21384595 0.23247322 0.24591087 0.25561279] |
+-------+-------------------+-------------------+--------------------+----------------------------------------------------------+----------------------------------------------------------+
Early stopping is trigger at step: 20 log:0.1827586206896552
early stopping at 100, recall@20:0.1936

LFM

python main.py --dataset last-fm --batch_size 4 --num_inner_update 2 --node_dropout 0.5

+-------+--------------------+------------------+---------------------+----------------------------------------------------------+----------------------------------------------------------+
| Epoch | training time | tesing time | Loss | recall | ndcg |
+-------+--------------------+------------------+---------------------+----------------------------------------------------------+----------------------------------------------------------+
| 735 | 61.490965127944946 | 56.5982711315155 | 0.16019470134051517 | [0.26468268 0.34291861 0.39268555 0.42871997 0.45697383] | [0.26681281 0.30199533 0.32134821 0.33424693 0.34380739] |
+-------+--------------------+------------------+---------------------+----------------------------------------------------------+----------------------------------------------------------+
Early stopping is trigger at step: 20 log:0.26468268196486616
early stopping at 735, recall@20:0.2662

when I was running the construct_data.py, it Error Display “[Errno 2] No such file or directory: './datasets/amazon-book/user_list.txt'”，
where can I get the file "user_list.txt"？

How long does it take to train? It seems I cannot get the results given in the paper?

It remind me about this when I run python main.py --dataset last-fm --use_meta_model True
Is there anyone can help?

Environment Requirements
Windows 11
Python = 3.8&3.10
PyTorch 1.12.1
A Nvidia GPU with cuda 11.8