Hi, may I know if you have evaluated the performance (e.g., Alpaca) of the resulting i

We evaluated the RLHF checkpoint: <a href="https://huggingface.co/OpenRLHF/Llama-3-8b-

Performance of Iterative DPO? about openrlhf HOT 1 CLOSED

yesiam-png commented on July 30, 2024

Performance of Iterative DPO?

from openrlhf.

Comments (1)

hijkzzz commented on July 30, 2024

We evaluated the RLHF checkpoint: https://huggingface.co/OpenRLHF/Llama-3-8b-rlhf-100k

Chat-Arena-Hard
-------------------------------------------
llama-3-8b-sft                 | score: 5.6   
llama-3-8b-rlhf-100k           | score: 20.5

And, the Iterative DPO checkpoint is here: https://huggingface.co/OpenRLHF/Llama-3-8b-iter-dpo-179k

update:

########## First turn ##########
                      score
model           turn
Llama3-iter-dpo 1      8.55
########## Second turn ##########
                        score
model           turn
Llama3-iter-dpo 2     7.95625
########## Average ##########
                    score
model
Llama3-iter-dpo  8.253125
Llama3-sft-baseline 7.69

from openrlhf.

Related Issues (20)

DPO Finetuning constantly gives preference loss as 0.6931 HOT 8
Difference between `DeepSpeedEngine.save_checkpoint()` and `DeepSpeedStrategy.save_model()` HOT 2
DPO后的模型推理出的结果都是无序符号 HOT 1
Support training from breakpoint HOT 3
llama3 70B DPO example script
where is gradient_accumulation HOT 1
Support RLOO HOT 1
现在Train_PPO_llama_ray 过程中会把Actor Model切分到不同卡上吗 HOT 4
ConnectionRefusedError: [Errno 111] Connection refused HOT 5
packing的问题 HOT 2
"right" padding hardcoded HOT 3
Error while saving the model under 4bit lora HOT 2
multinode ppo training extremely slow HOT 15
使用ray的时候Request Entity Too Large HOT 3
dpo 训练显存 OOM HOT 1
Online DPO 支持 HOT 4
Feature: add DPO-P
Zero stage 3 error HOT 1
Why multiplying rstd instead of dividing by rstd? HOT 1

Performance of Iterative DPO? about openrlhf HOT 1 CLOSED

Comments (1)

Related Issues (20)

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent