Discrepancy between results reported in CQL and D4rl papers about d4rl HOT 5 OPEN

The numbers in the NeurIPS version of the CQL paper: https://proceedings.neurips.cc/paper/2020/file/0d2b2061826a5df3221116a5085a6052-Paper.pdf are supposed to be used as reference, which refers to the table you mentioned. The original CQL paper (old version) matches the D4RL paper. We are in the process of fixing github issues in D4RL and will report the updated numbers in the next update.

from d4rl.

Hi, CQL reported numbers from the first arxiv version of the D4RL paper, which (for BEAR) have then improved in the newer version of D4RL. We will update the numbers for baselines in CQL, and so the results in D4RL should be used as reference. I think the difference is mainly in BEAR numbers, which changed since we moved to a better BEAR implementation.

from d4rl.

Hi, I also cross check the CQL scores reported in D4RL (arXiv-v4).
1. The mismatches has not been fixed @IcarusWizard @justinjfu @aviralkumar2907
2. In Table2 and Table3 of D4RL, there are also few mismatches for the same env.
For BC, 923 / 3234 = 29
For CQL, 2557 / 3234 = 79 not 58

from d4rl.

Thanks for your response.
One more question, were hyperparameters tuned per environment and data setting? or just one set of hyperparameter is used for all environments?

from d4rl.

Hi, I just cross check the CQL scores reported in D4RL (arXiv-v4) and CQL (NeurIPS) papers, there are few mismatches.

I hope you can clarify which one can be correctly used as a reference. Especially for hopper-medium, since the difference is huge.

from d4rl.