The SGL results in SSLRec seem overly distinguished. about sslrec HOT 6 CLOSED

I have same questions with you, and have you tried using other SGL code to run these datasets?

from sslrec.

Hi!

Thank you for your interest in SSLRec! MMSSL is indeed an impressive work that was completed prior to SSLRec, and currently, SSLRec does not include the implementation of multi-modal recommendation scenarios. The performance of the implemented SGL in SSLRec is suitable for general collaborative filtering task.

It's important to note that there are variations in the details across different tasks, especially those not currently included in SSLRec. These details encompass aspects such as training strategy, data processing, and test protocols. To ensure a fair comparison between different models, it is crucial to use the same settings. Different settings may yield different performance even for the same model.

Nevertheless, it's great to hear that you have achieved better performance with SGL using SSLRec. This demonstrates the effectiveness of our SSLRec framework. However, if you aim to reproduce the results in the field of multi-modal recommendation, it would be advisable to consider code frameworks like LATTICE and MMSSL. Only under the same settings can the performance outputs be compared fairly.

I hope the above answers can help with your confusion :)

Best regards,
Xubin

from sslrec.

from sslrec.

Thanks for your attention, and hope you enjoy SSLRec :)

from sslrec.

I have same questions with you, and have you tried using other SGL code to run these datasets?

Could you share the code for converting data from MMSSL format to SSLRec format with me? I will help you check for any potential issues.

from sslrec.

Thank you very much for your outstanding work.

When I use SGL to run the Tiktok, Amazon_sports dataset, the final result is very good, and has surpassed most of the baselines. here are my parameters and training results, is there something I have set up incorrectly? (The source of the dataset is "MMSSL: Multi-Modal Self-Supervised Learning for Recommendation" from your team).

Amazon_sports {'optimizer': {'name': 'adam', 'lr': 0.001, 'weight_decay': 0}, 'train': {'epoch': 300, 'batch_size': 4096, 'save_model': False, 'loss': 'pairwise', 'log_loss': False, 'test_step': 10, 'patience': 5, 'reproducible': True, 'seed': 2023, 'early_stop': True}, 'test': {'metrics': ['recall', 'precision', 'ndcg'], 'k': [20], 'batch_size': 1024}, 'data': {'type': 'general_cf', 'name': 'sports', 'user_num': 35598, 'item_num': 18357}, 'model': {'name': 'sgl', 'keep_rate': 0.5, 'layer_num': 2, 'reg_weight': 1e-05, 'cl_weight': 0.01, 'temperature': 0.1, 'embedding_size': 64, 'augmentation': 'node_drop'}

Validation set [recall@20: 0.0980 ] Validation set [precision@20: 0.0055 ] Validation set [ndcg@20: 0.0444 ] Test set [recall@20: 0.0963 ] Test set [precision@20: 0.0051 ] Test set [ndcg@20: 0.0437 ]

Tiktok {'optimizer': {'name': 'adam', 'lr': 0.001, 'weight_decay': 0}, 'train': {'epoch': 600, 'batch_size': 4096, 'save_model': False, 'loss': 'pairwise', 'log_loss': False, 'test_step': 10, 'patience': 5, 'reproducible': True, 'seed': 2023, 'early_stop': True}, 'test': {'metrics': ['recall', 'ndcg'], 'k': [10, 20], 'batch_size': 1024}, 'data': {'type': 'general_cf', 'name': 'tiktok', 'user_num': 9308, 'item_num': 6710}, 'model': {'name': 'sgl', 'keep_rate': 0.5, 'layer_num': 3, 'reg_weight': 1e-05, 'cl_weight': 1.0, 'temperature': 0.8, 'embedding_size': 64, 'augmentation': 'node_drop'}

Validation set [recall@20: 0.0993 ] Validation set [ndcg@20: 0.0429 ] Test set [recall@20: 0.0905 ] Test set [ndcg@20: 0.0379 ]

Results of SGL in MMSSL. Amazon_sports recall@20: 0.0779 ndcg@20: 0.0361 Tiktok recall@20: 0.0603 ndcg@20: 0.0238

Sincerely.

Could you please share the code for converting data from MMSSL format to SSLRec format to facilitate the assistance in checking for issues? Thank you immensely for your gracious consideration～🌹

from sslrec.