giogiobond / nbceonchatglm6b Goto Github PK
View Code? Open in Web Editor NEW(NBCE)Naive Bayes-based Context Extension on ChatGLM-6b
(NBCE)Naive Bayes-based Context Extension on ChatGLM-6b
目前来看,受显存大小限制很大,不知是否支持多卡推理
vocab_size:150528 这个版本的chatglm,出现了这个bug
RuntimeError: The following operation failed in the TorchScript interpreter.
Traceback of TorchScript (most recent call last):
RuntimeError: The following operation failed in the TorchScript interpreter.
Traceback of TorchScript (most recent call last):
File
"/root/.cache/huggingface/modules/transformers_modules/chatglm-6b/modeling_chatglm.py",
line 198, in <fused code>
cos, sin = F.embedding(position_id, cos.squeeze(1)).unsqueeze(2), \
F.embedding(position_id, sin.squeeze(1)).unsqueeze(2)
q, k = (q * cos) + (rotate_half(q) * sin), (k * cos) + (rotate_half(k) * sin)
~~~~~~~ <--- HERE
return q, k
RuntimeError: The size of tensor a (103) must match the size of tensor b (102) at
non-singleton dimension 0
速度太慢了
# 拼接context和question
contexts = [''] + contexts # 添加空Context(无Context预测)
batch = [f'''
上下文:
{context}
问题:
{question}
回答:
''' for context in contexts]
用这个模版能有比较稳定的效果。
提问也可以稍微修改一下:
question = """请根据上诉材料,一句话回答下面问题:
- 菲律宾国家电网公司,**占股多少?
- 领英计划裁员多少人?
- 吉利德收购Pharmasset的价格是多少?
- 丙肝神药Sovaldi在哪一年上市?
- 中亚峰会将在哪里举行?由谁主持?
- 哪个演员由于侮辱人民军队而被立案调查?
- 哪个项目宣称“能过坦克”的水上道路?
"""
40G显存,context长度好像没有什么限制。
Traceback (most recent call last):
File "/home/xierunfeng/NBCE-main/test.py", line 111, in
generate(1000)
File "/home/xierunfeng/miniconda3/envs/test/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/home/xierunfeng/NBCE-main/test.py", line 67, in generate
_attention_mask = get_attention_mask(attention_mask)
File "/home/xierunfeng/NBCE-main/test.py", line 45, in get_attention_mask
if attention_mask[i][j] == 0:
RuntimeError: Boolean value of Tensor with more than one value is ambiguous
ssh://[email protected]:12301/root/anaconda3/envs/chatglm/bin/python -u /data/cxj_workplace/learning/NBCE/test.py
Loading checkpoint shards: 100%|██████████████████| 7/7 [00:09<00:00, 1.42s/it]
Context长度分布: [230, 780, 1308, 968, 677, 1126, 697, 706, 768, 894, 631, 737]
Context总长度: 9522
input_ids torch.Size([12, 790])
<0x0A>1.菲律宾国家电网公司,**占股多少?<0x0A><0x0A>根据材料,菲律宾国家电网公司是**在2019年收购菲律宾国家电网公司时,**占股25%,**占领菲律宾国家电网公司总股的40%。<0x0A><0x0A><0x0A>2.领英计划裁员多少人裁员多少人?<0x0A><0x0A><0x0A>领英媒,微软公司计划裁员71.微软公司计划裁员71.微软公司全球裁员71.微软公司全球裁员71.微软公司71微软公司71微软公司71微软公司71微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软1微软
Process finished with exit code 0
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.