[BUG] glm-10b-ch 模型不能正确inference about flagai HOT 5 CLOSED

先检查一下你的模型有没有下载下来
在checkpoints/glm-10-ch里看有没有一个18.4G的pytorch_model.bin文件

from flagai.

模型下载完了，这个没问题

from flagai.

还是有问题吗，可以在根目录下python setup.py install安装最新版
然后我测试的代码如下：
import torch
from flagai.model.glm_model import GLMModel
from flagai.data.tokenizer import Tokenizer
from flagai.model.predictor.predictor import Predictor
import bminf

if name == "main":
"""Main training program."""
print('Generate Samples')
# Random seeds for reproducability.
# Model,
model_name = 'GLM-10b-ch'
model = GLMModel.from_pretrain(model_name=model_name,
download_path="./checkpoints")
tokenizer = Tokenizer.from_pretrained(model_name)
with torch.cuda.device(0):
model = bminf.wrapper(model, quantization=False, memory_limit=30 << 39)
predictor = Predictor(model, tokenizer)

text = ["今天天气不错[gMASK]"]
for t in text:
    output = predictor.predict_generate_randomsample(
        t, top_k=50, repetition_penalty=4.0, top_p=1.0)
    print(t, '\n', output)

测完没问题

from flagai.

还是有问题吗，可以在根目录下python setup.py install安装最新版 然后我测试的代码如下： import torch from flagai.model.glm_model import GLMModel from flagai.data.tokenizer import Tokenizer from flagai.model.predictor.predictor import Predictor import bminf

if name == "main": """Main training program.""" print('Generate Samples') # Random seeds for reproducability. # Model, model_name = 'GLM-10b-ch' model = GLMModel.from_pretrain(model_name=model_name, download_path="./checkpoints") tokenizer = Tokenizer.from_pretrained(model_name) with torch.cuda.device(0): model = bminf.wrapper(model, quantization=False, memory_limit=30 << 39) predictor = Predictor(model, tokenizer)

text = ["今天天气不错[gMASK]"]
for t in text:
    output = predictor.predict_generate_randomsample(
        t, top_k=50, repetition_penalty=4.0, top_p=1.0)
    print(t, '\n', output)

测完没问题

我因为memory不够加了bminf, 你要是够的话不需要加那步

from flagai.

测试了这样结果是正常的：

from flagai.model.glm_model import GLMModel
from flagai.data.tokenizer import Tokenizer
from flagai.model.predictor.predictor import Predictor

model_name = 'GLM-10b-ch'
model = GLMModel.from_pretrain(model_name=model_name, download_path="./checkpoints")
tokenizer = Tokenizer.from_pretrained(model_name)

predictor = Predictor(model, tokenizer)

text = "今天天气不错[gMASK]"
output = predictor.predict_generate_randomsample(text, out_max_length=10)
print(text, '\n', output)

所以这是AutoLoader相关的bug？

我这pytorch装的是没集成cuda的版本，跟这个有关么？

from flagai.