cannot load int8/4 model with deepspeed zero3 about peft HOT 5 CLOSED

Please use the main branch of transformers and let us know if that solves the issue.

from peft.

I try to use the main branch of transformers, error:
Traceback (most recent call last):
File "/workspace/xxx/code/train.py", line 7, in
from trl import SFTTrainer
File "/usr/local/lib/python3.10/dist-packages/trl/init.py", line 5, in
from .core import set_seed
File "/usr/local/lib/python3.10/dist-packages/trl/core.py", line 25, in
from transformers import top_k_top_p_filtering
ImportError: cannot import name 'top_k_top_p_filtering' from 'transformers' (/usr/local/lib/python3.10/dist-packages/transformers/init.py)
how to solve this error?

from peft.

@mxjyst please use also the main branch of TRL

from peft.

You can also refer to these scripts from @pacman100 : https://github.com/huggingface/peft/blob/main/examples/sft/

from peft.

I can run successfully with the main branch of transformers and trl. Thanks!

from peft.