中文乱码 about chinese-vicuna HOT 5 CLOSED

又用Facico提供的测试脚本测试，脚本内容如下：
import sys
import torch
from peft import PeftModel
import transformers
from transformers import LlamaTokenizer, LlamaForCausalLM

#tokenizer = LlamaTokenizer.from_pretrained("../LLM_pretrained_model/decapoda-research/llama-7b-hf")
#BASE_MODEL = "../LLM_pretrained_model/decapoda-research/llama-7b-hf"
tokenizer = LlamaTokenizer.from_pretrained("../FastChat_lm-sys_20230630/conv_models/7B")
BASE_MODEL = "../FastChat_lm-sys_20230630/conv_models/7B"

model = LlamaForCausalLM.from_pretrained(
BASE_MODEL,
load_in_8bit=True,
torch_dtype=torch.float16,
device_map="auto",
)
model.eval()
inputs = "Hello, Where is the capital of the United States?" #"你好,美国的首都在哪里？"
input_ids = tokenizer(inputs, return_tensors="pt")['input_ids']
print(input_ids)
input_ids = input_ids.to('cuda')
generation_output = model.generate(
input_ids=input_ids,
max_new_tokens=256,
)
print(generation_output)
print(tokenizer.decode(generation_output[0]))

model = PeftModel.from_pretrained(
model,
"../LLM_pretrained_model/Facico/Chinese-Vicuna-lora-7b-chatv1",
torch_dtype=torch.float16,
device_map={'': 0}
)

inputs = "你好,**的首都在哪里？" #"你好,美国的首都在哪里？"
inputs = "告诉我肚子疼吃什么药？"
input_ids = tokenizer(inputs, return_tensors="pt")['input_ids']
print(input_ids)
input_ids = input_ids.to('cuda')
generation_output = model.generate(
input_ids=input_ids,
max_new_tokens=256,
)
print(generation_output)
print(tokenizer.decode(generation_output[0]))

输出如下：
tensor([[ 1, 15043, 29892, 6804, 338, 278, 7483, 310, 278, 3303,
3900, 29973]])
tensor([[ 1, 15043, 29892, 6804, 338, 278, 7483, 310, 278, 3303,
3900, 29973, 13, 1576, 7483, 310, 278, 3303, 3900, 338,
7660, 29892, 360, 29889, 29907, 29889, 13, 1576, 7483, 310,
278, 3303, 3900, 338, 7660, 29892, 360, 29889, 29907, 29889,
13, 1576, 7483, 310, 278, 3303, 3900, 338, 7660, 29892,
360, 29889, 29907, 29889, 13, 1576, 7483, 310, 278, 3303,
3900, 338, 7660, 29892, 360, 29889, 29907, 29889, 13, 1576,
7483, 310, 278, 3303, 3900, 338, 7660, 29892, 360, 29889,
29907, 29889, 13, 1576, 7483, 310, 278, 3303, 3900, 338,
7660, 29892, 360, 29889, 29907, 29889, 13, 1576, 7483, 310,
278, 3303, 3900, 338, 7660, 29892, 360, 29889, 29907, 29889,
13, 1576, 7483, 310, 278, 3303, 3900, 338, 7660, 29892,
360, 29889, 29907, 29889, 13, 1576, 7483, 310, 278, 3303,
3900, 338, 7660, 29892, 360, 29889, 29907, 29889, 13, 1576,
7483, 310, 278, 3303, 3900, 338, 7660, 29892, 360, 29889,
29907, 29889, 13, 1576, 7483, 310, 278, 3303, 3900, 338,
7660, 29892, 360, 29889, 29907, 29889, 13, 1576, 7483, 310,
278, 3303, 3900, 338, 7660, 29892, 360, 29889, 29907, 29889,
13, 1576, 7483, 310, 278, 3303, 3900, 338, 7660, 29892,
360, 29889, 29907, 29889, 13, 1576, 7483, 310, 278, 3303,
3900, 338, 7660, 29892, 360, 29889, 29907, 29889, 13, 1576,
7483, 310, 278, 3303, 3900, 338, 7660, 29892, 360, 29889,
29907, 29889, 13, 1576, 7483, 310, 278, 3303, 3900, 338,
7660, 29892, 360, 29889, 29907, 29889, 13, 1576, 7483, 310,
278, 3303, 3900, 338, 7660, 29892, 360, 29889, 29907, 29889,
13, 1576, 7483, 310, 278, 3303, 3900, 338, 7660, 29892,
360, 29889, 29907, 29889, 13, 1576, 7483, 310]],
device='cuda:0')
Hello, Where is the capital of the United States?
The capital of the United States is Washington, D.C.
The capital of the United States is Washington, D.C.
The capital of the United States is Washington, D.C.
The capital of the United States is Washington, D.C.
The capital of the United States is Washington, D.C.
The capital of the United States is Washington, D.C.
The capital of the United States is Washington, D.C.
The capital of the United States is Washington, D.C.
The capital of the United States is Washington, D.C.
The capital of the United States is Washington, D.C.
The capital of the United States is Washington, D.C.
The capital of the United States is Washington, D.C.
The capital of the United States is Washington, D.C.
The capital of the United States is Washington, D.C.
The capital of the United States is Washington, D.C.
The capital of the United States is Washington, D.C.
The capital of the United States is Washington, D.C.
The capital of the United States is Washington, D.C.
The capital of
tensor([[ 1, 29871, 31785, 235, 178, 140, 30672, 235, 133, 157,
30319, 234, 153, 191, 232, 147, 134, 231, 190, 131,
31882, 235, 144, 178, 30882]])
tensor([[ 1, 29871, 31785, 235, 178, 140, 30672, 235, 133, 157,
30319, 234, 153, 191, 232, 147, 134, 231, 190, 131,
31882, 235, 144, 178, 30882, 13, 235, 133, 157, 30319,
234, 153, 191, 30392, 30287, 31893, 31190, 235, 170, 132,
30210, 234, 153, 191, 234, 154, 158, 30214, 30682, 30815,
30392, 31272, 30909, 232, 147, 187, 232, 179, 232, 146,
154, 231, 191, 167, 30330, 232, 145, 142, 31074, 31391,
31149, 31221, 30667, 31570, 31674, 31558, 30210, 30267, 30651, 30557,
30392, 30287, 31959, 30682, 30815, 30417, 31931, 30909, 234, 191,
150, 31201, 235, 133, 157, 30319, 234, 153, 191, 30210,
235, 144, 178, 30383, 13, 29896, 29889, 29871, 233, 141,
154, 233, 179, 170, 30705, 235, 144, 178, 13, 233,
141, 154, 233, 179, 170, 30705, 235, 144, 178, 30682,
30651, 232, 187, 177, 31931, 234, 191, 150, 31201, 235,
133, 157, 30319, 234, 153, 191, 30214, 31570, 30573, 232,
177, 134, 30682, 30651, 232, 138, 233, 184, 233, 141,
233, 179, 170, 30705, 30834, 30210, 233, 178, 158, 232,
158, 30214, 232, 138, 233, 184, 233, 141, 233, 179,
170, 30705, 30834, 30210, 233, 178, 158, 232, 158, 30214,
232, 138, 233, 184, 233, 141, 233, 179, 170, 30705,
30834, 30210, 233, 178, 158, 232, 158, 30214, 232, 138,
233, 184, 233, 141, 233, 179, 170, 30705, 30834, 30210,
233, 178, 158, 232, 158, 30214, 232, 138, 233, 184,
233, 141, 233, 179, 170, 30705, 30834, 30210, 233, 178,
158, 232, 158, 30214, 232, 138, 233, 184, 233, 141,
233, 179, 170, 30705, 30834, 30210, 233, 178, 158, 232,
158, 30214, 232, 138, 233, 184, 233, 141, 233, 179,
170, 30705, 30834, 30210, 233, 178, 158, 232, 158, 30214,
232]], device='cuda:0')
告诉我肚子疼吃什么药？
肚子疼是一种常见的疼痛，可能是由于吸��受伤、压力或其他原因引起的。以下是一些可能有助于缓解肚子疼的药：

from chinese-vicuna.

我这个跑英文没有问题，不会出现乱码。跑中文，经常会出现乱码。
我用的库版本：transformers==4.30.2, tokenizers==0.13.3, sentencepiece==0.1.99

大佬们知道是什么原因吗？给指点一下，多谢了。

from chinese-vicuna.

很可能是python第3方包的版本问题，可以参考 https://github.com/Facico/Chinese-Vicuna/blob/master/docs/problems.md @NewEricWang

from chinese-vicuna.

Thanks a lot! @Gzj369

from chinese-vicuna.

from chinese-vicuna.