Comments (3)
Hi 👋!
Apologies for the delayed response, and thank you for your interest in SSLRec! Currently, we only test and saved the optimal hyperparameters based on the Gowalla dataset for all methods, for reference.
To conduct a grid search on a new dataset (e.g., the Yelp and Amazon), you can change the value of tune-enable
in the YAML file to true
and then run the main.py
file. The results will be saved in a log file, where you can find the best performance.
Here are some results we obtained before that may be helpful to you :)
yelp
LightGCN:
- Default hyperparameter
- Test set [recall@10: 0.0450 recall@20: 0.0713 recall@40: 0.1085 ] Test set [ndcg@10: 0.0273 ndcg@20: 0.0351 ndcg@40: 0.0441 ]
SGL:
- hyperparameter: {'layer_num': 2, 'reg_weight': 1e-05, 'cl_weight': 1.0, 'temperature': 0.5}
- Test set [recall@10: 0.0556 recall@20: 0.0850 recall@40: 0.1293 ] Test set [ndcg@10: 0.0341 ndcg@20: 0.0428 ndcg@40: 0.0536 ]
NCL:
- hyperparameter: {'temperature': 1.0, 'proto_weight': 0.01, 'struct_weight': 0.001}
- Test set [recall@10: 0.0471 recall@20: 0.0753 recall@40: 0.1142 ] Test set [ndcg@10: 0.0291 ndcg@20: 0.0374 ndcg@40: 0.0469 ]
DCCF:
- hyperparameter: {'layer_num': 2, 'reg_weight': 0.001, 'cl_weight': 0.1, 'temperature': 0.5}
- Test set [recall@10: 0.0546 recall@20: 0.0854 recall@40: 0.1302 ] Test set [ndcg@10: 0.0342 ndcg@20: 0.0433 ndcg@40: 0.0542 ]
amazon
LightGCN:
- Default hyperparameter
- Test set [recall@10: 0.0522 recall@20: 0.0821 recall@40: 0.1245 ] Test set [ndcg@10: 0.0431 ndcg@20: 0.0532 ndcg@40: 0.0662 ]
SGL:
- hyperparameter: {'layer_num': 2, 'reg_weight': 1e-06, 'cl_weight': 1.0, 'temperature': 0.2}
- Test set [recall@10: 0.0654 recall@20: 0.0978 recall@40: 0.1416 ] Test set [ndcg@10: 0.0554 ndcg@20: 0.0662 ndcg@40: 0.0797 ]
NCL:
- hyperparameter: {'temperature': 0.1, 'proto_weight': 0.0001, 'struct_weight': 0.001}
- Test set [recall@10: 0.0607 recall@20: 0.0951 recall@40: 0.1426 ] Test set [ndcg@10: 0.0502 ndcg@20: 0.0617 ndcg@40: 0.0764 ]
DCCF:
- hyperparameter: {'layer_num': 2, 'reg_weight': 0.0001, 'cl_weight': 0.1, 'temperature': 0.2}
- Test set [recall@10: 0.0814 recall@20: 0.1155 recall@40: 0.1583 ] Test set [ndcg@10: 0.0719 ndcg@20: 0.0829 ndcg@40: 0.0961 ]
You can also try it on your own device to see the results, which may have some fluctuations, but that's fine :)
Best regards,
Xubin
from sslrec.
Thank you for your reply.I have another question.I find that the tune cl_weight of SGL and Simgcl is [1.0, 1.0e-1, 1.0e-2].So should i set the same one for my model to tune in order to keep fair?Could i set the different one such as [0.1,0.2,0.5,1]?
from sslrec.
Hi!
I think that's okay because the purpose of grid search is to find the best hyper-parameters for the model. We just set the search range based on our empirical experience, and you can also change the search range by yourself :)
Best regards,
Xubin
from sslrec.
Related Issues (20)
- 关于Epinions数据集缺少category.pkl文件的问题 HOT 1
- 多行为推荐中的 MBSSL(SIGIR2023) 是否在计划中? 🌱 HOT 1
- Gumbel噪声的计算会出现-inf HOT 1
- SimGCL: InfoNCE Loss Calculation Batch vs. Entire Embeddings HOT 2
- 关于评价指标 HOT 2
- 关于多行为推荐中数据集问题 HOT 6
- 如何构建自己的数据集 HOT 2
- 关于其他数据集 HOT 2
- pickle加载文件时报错ModuleNotFoundError: No module named 'scipy.sparse._csr',scipy库版本为1.7.3,希望得到作者的解答,万分感谢 HOT 1
- 关于retail_rocket数据集下生成的kg.txt的含义 HOT 2
- 关于lightgcn层数问题 HOT 4
- 训练集测试集的划分
- Can evaluation metrics support AUC-ROC calculations? HOT 1
- cuda版本问题 HOT 1
- 关于SSLRec运行时出现“Cuda Kernel Error”? HOT 4
- 关于构建自己的kg数据集 HOT 2
- 关于模型评测指标数值的问题 HOT 1
- KGIN在SSLRec仓库和KGRec仓库在我的测试下差距过大
- 关于dgl的cuda版本适配问题
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from sslrec.