internlm / internlm Goto Github PK
View Code? Open in Web Editor NEWOfficial release of InternLM2 7B and 20B base and chat models. 200K context support
Home Page: https://internlm.intern-ai.org.cn/
License: Apache License 2.0
Official release of InternLM2 7B and 20B base and chat models. 200K context support
Home Page: https://internlm.intern-ai.org.cn/
License: Apache License 2.0
The same mistakes to other language doc
conda env
No response
I run the code, but only got 90+ tflops.
INFO train.py:317 in record_current_batch_training_metrics -- tflops=93.48098385143103,step=9,loss=7.502509117126465,tgs (tokens/gpu/second)=2104.89,lr=2.2e-06,loss_scale=65536.0,grad_norm=20.60409540743281,micro_num=4,num_consumed_tokens=2621440,inf_nan_skip_batches=0,num_samples_in_batch=13,largest_length=2048,largest_batch=4,smallest_batch=3,adam_beta2=0.95,fwd_bwd_time=6.15
click the link: https://internlm.readthedocs.io/en/latest/
the result is 404
any browser
No response
预料 -> 语料
不涉及
No response
请问有官方微信群或者QQ群吗?
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x90 in position 25401: invalid start byte
The above exception was the direct cause of the following exception:
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /data/llm/anaconda3/envs/llm/lib/python3.9/site-packages/transformers/modeling_utils.py:457 in │
│ load_state_dict │
│ │
│ 454 │ │ │ │ │ │ "you cloned." │
│ 455 │ │ │ │ │ ) │
│ 456 │ │ │ │ else: │
│ ❱ 457 │ │ │ │ │ raise ValueError( │
│ 458 │ │ │ │ │ │ f"Unable to locate the file {checkpoint_file} which is necessary │
│ 459 │ │ │ │ │ │ "model. Make sure you have saved the model properly." │
│ 460 │ │ │ │ │ ) from e │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
ValueError: Unable to locate the file /data/llm/internlm-chat-7b/pytorch_model-00001-of-00002.bin which is
necessary to load this pretrained model. Make sure you have saved the model properly.
python3.9
No response
Hi InternLM team, thank you for this open source contribution! InternLM looks like a really strong 7B model.
I think the research community would greatly benefit from learning about the training details of InternLM. Are you open to sharing the token budget and global batch size used for this model?
In the README I see this comment which suggests a token budget over 1T tokens:
It leverages trillions of high-quality tokens for training to establish a powerful knowledge base.
And in the training performance README I see that the max performance was achieved at 16k tokens per GPU. If this was used across 1024 GPUs for pretraining it would imply a global batch size of 16M tokens which is larger than I've seen before (especially for 7B models).
Thank you again!
The example does not work
response, history = model.chat(tokenizer, "hello", history=[])
AttributeError: 'InternLMModel' object has no attribute 'chat'
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("internlm/internlm-7b", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("internlm/internlm-7b", trust_remote_code=True).cuda()
model = model.eval()
inputs = tokenizer(["A beautiful flower"], return_tensors="pt")
for k,v in inputs.items():
gen_kwargs = {"max_length": 128, "top_p": 0.8, "temperature": 0.8, "do_sample": True, "repetition_penalty": 1.1}
output = model.generate(**inputs, **gen_kwargs)
print(output)
按照这个代码运行只输出编码
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("internlm/internlm-7b", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("internlm/internlm-7b", trust_remote_code=True).cuda()
model = model.eval()
inputs = tokenizer(["A beautiful flower"], return_tensors="pt")
for k,v in inputs.items():
gen_kwargs = {"max_length": 128, "top_p": 0.8, "temperature": 0.8, "do_sample": True, "repetition_penalty": 1.1}
output = model.generate(**inputs, **gen_kwargs)
print(output)
No response
感谢开源!
我想做一组Retrieval QA的测试,请问用什么prompt比较合适,我试了几个prompt都不太好,比如下面这个internLM-7b-chat会用英文来回答,而直接使用chat接口回答都很简短。
prompt_template = f"""User: 已知信息:
{context}
根据上述已知信息,专业的回答用户的问题。如果无法从中得到答案,请说 “抱歉,根据已知信息无法回答该问题”,不允许在答案中添加编造成分,答案请使用中文。
问题:{question}
Assistant:"""
It would be extremely helpful if you could share the list of languages that are included in the dataset used to train the model. If the dataset is available on Hugging Face, it would be even better if you could provide a direct link to it.
Example provided in https://github.com/InternLM/InternLM#import-from-transformers toke more than 40s to finish at V100. Why?
Can you provide a docker so we can deploy it out-of-box?
torch.cuda.empty_cache()对其他模型生效,都能释放显存,这个模型无效
希望增加释放显存缓存的功能,在用户手动清除缓存的情况下,希望显存能恢复到刚加载模型的状态
Dear InternLM team,
Thanks for your Acknowledgements to Colossal-AI.
It would be appreciated if you could adhere to the open source agreement and cite Colossal-AI license in your license file.
Thank you very much.
Colossal-AI team
目前的仓库有几个复杂的依赖库apex、flash-attention,需要可以在没有这些依赖的条件下,可以fallback执行。
目前pre-commit hook的规则跟 lmdeploy 和 opencompass不完全一致,需要进行对齐和统一验证
The hugging face modeling file: https://huggingface.co/internlm/internlm-7b/blob/main/modeling_internlm.py
if I train SFT from base model, which modeling file should I use?
thank you!
enter huggingface link: "https://huggingface.co/spaces/internlm/InternLM-Chat-7B"
submit the first question then get the answer, but if you continue submiting the question, then response nothing
any browser
No response
是参数不对,还是我调用程序(https://github.com/wenda-LLM/wenda/blob/main/llms/llm_internlm.py )有问题?
在ReadMe中提到了 模型的特点
“3. 提供了一个多功能工具集,让用户灵活构建自己的工作流程。”
请问 这里的工作流程是什么意思,是指的LLM调用外部工具吗
之前的stable diffusion部署在了google云端硬盘上,学习和使用都比较方便,请问这个项目能这样做吗?谢谢
The promt log when exit the dialogue:
Loading checkpoint shards: 100%|██████████| 2/2 [00:18<00:00, 9.30s/it]
^Csrun: interrupt (one more within 1 sec to abort)
srun: StepId=7154040.0 task 0: running
^Csrun: sending Ctrl-C to StepId=7154040.0
load model end.
load model begin.
load model end.
cur real input:
<|User|>:介绍下你自己~
<|Bot|>:
cur total response:
你好!我是一个人工智能助手,我的名字是书生·浦语,我致力于帮助用户解决问题和提供信息服务。我能够回答问题、提供定义和解释、将文本从一种语言翻译成另一种语言、总结文本、生成文本、编写故事、分析情感、提供推荐、开发算法、编写代码以及其他任何基于语言的任务。我的设计理念是有用、诚实并且无害,我使用深度学习技术进行构建并且不断从与用户的对话中学习。
load model begin.
load model end.
cur real input:
<|User|>:介绍下你自己~
<|Bot|>:你好!我是一个人工智能助手,我的名字是书生·浦语,我致力于帮助用户解决问题和提供信息服务。我能够回答问题、提供定义和解释、将文本从一种语言翻译成另一种语言、总结文本、生成文本、编写故��、分析情感、提供推荐、开发算法、编写代码以及其他任何基于语言的任务。我的设计理念是有用、诚实并且无害,我使用深度学习技术进行构建并且不断从与用户的对话中学习。
<|User|>:我们来对联吧,上联:生意如春意
<|Bot|>:
cur total response:
下联:婚姻若水纹
load model begin.
load model end.
cur real input:
<|User|>:介绍下你自己~
<|Bot|>:你好!我是一个人工智能助手,我的名字是书生·浦语,我致力于帮助用户解决问题和提供信息服务。我能够回答问题、提供定义和解释、将文本从一种语言翻译成另一种语言、总结文本、生成文本、编写故事、分析情感、提供推荐、开发算法、编写代码以及其他任何基于语言的任务。我的设计理念是有用、诚实并且无害,我使用深度学习技术进行构建并且不断从与用户的对话中学习。
<|User|>:我们来对联吧,上联:生意如春意
<|Bot|>:下联:婚姻若水纹
<|User|>:GOOD
<|Bot|>:
cur total response:
非常感谢!
load model begin.
load model end.
cur real input:
<|User|>:介绍下你自己~
<|Bot|>:你好!我是一个人工智能助手,我的名字是书生·浦语,我致力于帮助用户解决问题和提供信息服务。我能够回答问题、提供定义和解释、将文本从一种语言翻译成另一种语言、总结文本、生成文本、编写故事、分析情感、提供推荐、开发算法、编写代码以及其他任何基于语言的任务。我的设计理念是有用、诚实并且无害,我使用深度学习技术进行构建并且不断从与用户的对话中学习。
<|User|>:我们来对联吧,上联:生意如春意
<|Bot|>:下联:婚姻若水纹
<|User|>:GOOD
<|Bot|>:非常感谢!
<|User|>:展开说明为什么这样对下联
<|Bot|>:
slurm platform
No response
感谢您分享的成果。
想确认一下,目前是否计划支持windows,有的话大概什么时候,谢谢!
在已安装好环境(包括transformers库)时报错内容为找不到对应的类,请问如何解决:
File "/HOME/scz3924/.conda/envs/InternLM/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py", line 719, in from_pretrained
raise ValueError(
ValueError: Unrecognized configuration class <class 'transformers_modules.internlm.internlm-chat-7b.dd2fa16d14c8b21fea4b4c168b9fef839154e305.configuration_internlm.InternLMConfig'> to build an AutoTokenizer.
Model type should be one of AlbertConfig, AlignConfig, BartConfig, BertConfig, BertGenerationConfig, BigBirdConfig, BigBirdPegasusConfig, BioGptConfig, BlenderbotConfig, BlenderbotSmallConfig, BlipConfig, Blip2Config, BloomConfig, BridgeTowerConfig, CamembertConfig, CanineConfig, ChineseCLIPConfig, ClapConfig, CLIPConfig, CLIPSegConfig, CodeGenConfig, ConvBertConfig, CpmAntConfig, CTRLConfig, Data2VecTextConfig, DebertaConfig, DebertaV2Config, DistilBertConfig, DPRConfig, ElectraConfig, ErnieConfig, ErnieMConfig, EsmConfig, FlaubertConfig, FNetConfig, FSMTConfig, FunnelConfig, GitConfig, GPT2Config, GPT2Config, GPTBigCodeConfig, GPTNeoConfig, GPTNeoXConfig, GPTNeoXJapaneseConfig, GPTJConfig, GPTSanJapaneseConfig, GroupViTConfig, HubertConfig, IBertConfig, JukeboxConfig, LayoutLMConfig, LayoutLMv2Config, LayoutLMv3Config, LEDConfig, LiltConfig, LlamaConfig, LongformerConfig, LongT5Config, LukeConfig, LxmertConfig, M2M100Config, MarianConfig, MBartConfig, MegaConfig, MegatronBertConfig, MgpstrConfig, MobileBertConfig, MPNetConfig, MT5Config, MvpConfig, NezhaConfig, NllbMoeConfig, NystromformerConfig, OneFormerConfig, OpenAIGPTConfig, OPTConfig, OwlViTConfig, PegasusConfig, PegasusXConfig, PerceiverConfig, Pix2StructConfig, PLBartConfig, ProphetNetConfig, QDQBertConfig, RagConfig, RealmConfig, ReformerConfig, RemBertConfig, RetriBertConfig, RobertaConfig, RobertaPreLayerNormConfig, RoCBertConfig, RoFormerConfig, RwkvConfig, Speech2TextConfig, Speech2Text2Config, SpeechT5Config, SplinterConfig, SqueezeBertConfig, SwitchTransformersConfig, T5Config, TapasConfig, TransfoXLConfig, ViltConfig, VisualBertConfig, Wav2Vec2Config, Wav2Vec2ConformerConfig, WhisperConfig, XCLIPConfig, XGLMConfig, XLMConfig, XLMProphetNetConfig, XLMRobertaConfig, XLMRobertaXLConfig, XLNetConfig, XmodConfig, YosoConfig.
满足requirments
No response
none
none
No response
Is the IntermLM 7B the only one open to the community, larger models are commercially licensed?
已按制式用公司信息填写申请表
I want to finetune the chat model using an instruction dataset.
can you give me the prompt used in your chat model?
USER: <instruction>\nASSISTANT: <output>
is this right?
It is a small problem. When I follow REAMDE Import from Transformers
to initialize model, a error (NameError: name 'torch' is not defined) occurs. So it is need to import torch
.
Also, I have a problem if internlm-chat-7b
can be load on multi gpus since my machine is 11G x 8.
>>> from transformers import AutoTokenizer, AutoModelForCausalLM
>>> tokenizer = AutoTokenizer.from_pretrained("internlm/internlm-chat-7b", trust_remote_code=True)
>>> model = AutoModelForCausalLM.from_pretrained("internlm/internlm-chat-7b", torch_dtype=torch.float16, trust_remote_code=True).cuda()
torch 1.13.1+cu117
torch-scatter 2.1.1+pt113cu117
torchaudio 0.13.1+cu117
torchvision 0.14.1+cu117
No response
你好呀,我想微调咱们的大模型,但是在集群上直接出现了这个错误。这有可能是什么原因导致的呢?
python tools/alpaca_tokenizer.py /mnt/petrelfs/wangxiaochen/workspace/alpaca_data.json /mnt/petrelfs/wangxiaochen/workspace/dataset /mnt/petrelfs/wangxiaochen/workspace/pretrained_models/internlm-chat-7b --split_ratio 0.1
Aborted
所有的环境按照说明配好,但是mpfr、mpc还没配置。在集群share区有此文件,却没法编译引用。
No response
有没有运行的方法?
I found that a single RTX4090 (24G) can not load InternLM-Chat-7B model. But when I use device_map="auto"
, an exception occurs:
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 172.00 MiB (GPU 0; 23.65 GiB total capacity; 22.99 GiB already allocated; 154.06 MiB free; 22.99 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
I don't know why it didn't evenly utilize multiple GPUs. Therefore, the first GPU is out of Memory.
Here is the code I use:
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("internlm/internlm-chat-7b", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("internlm/internlm-chat-7b", device_map="auto", trust_remote_code=True).cuda()
model = model.eval()
response, history = model.chat(tokenizer, "hello", history=[])
print(response)
The error happens when executing
model = AutoModelForCausalLM.from_pretrained("internlm/internlm-chat-7b", device_map="auto", trust_remote_code=True).cuda()
What's the reason for this error?
python 3.8
cuda 11.7
No response
Hi, I tried to install per requirements/torch.txt and got following errors. Anything did I miss?
➜ InternLM git:(main) pip install -r requirements/torch.txt
Looking in indexes: https://pypi.org/simple, https://download.pytorch.org/whl/cu117
ERROR: Could not find a version that satisfies the requirement torch==1.13.1+cu117 (from versions: 2.0.0, 2.0.1)
ERROR: No matching distribution found for torch==1.13.1+cu117
My environment is:
when I install apex, some error occurred:
ModuleNotFoundError: No module named 'packaging'
error: subprocess-exited-with-error
CUDA: 11.7
python: 3.10
No response
你们的开源许可证
本仓库的代码依照 Apache-2.0 协议开源。模型权重对学术研究完全开放,也可申请免费的商业使用授权(申请表)。其他问题与合作请联系 [email protected]。
但 Apache-2.0 这个协议本身就支持商业化授权,是不需要再找你们申请的!
申请限制
而且你的申请表只支持公司,不支持个人,我没太懂:彻底开源,免费商用,上海AI实验室把大模型门槛打下来 是什么意思。
能解释一下,你们是怎么做出这个决定?
您好,我们在调用internlm-chat-7b这个模型的时候,发现模型针对一些prompt,会自行进行多轮对话。我看到官方提供的对话实例是model.chat,这个在内部实现是用response.split("")[0]来保证了只返回第一个回复。但是如果要使用这个模型进行后续的实验如RLHF等,在调用model.generate时会输出后面的多轮内容,相关token也会加入到计算中,这个是不可接受的。请问有什么方式来避免输出多轮对话的内容吗?
略
No response
packages/torch/include/torch/csrc/api/include/torch/serialize/archive.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data/samplers/serialize.h:4,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data/samplers.h:8,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data/datasets/chunk.h:7,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data/datasets.h:4,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data.h:4,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/all.h:8,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/extension.h:4,
from /data2/InternLM/apex/csrc/fused_dense.cpp:1:
/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/jit/ir/attributes.h:35:27: note: ‘torch::jit::toString’
35 | static inline const char* toString(AttributeKind kind) {
| ^~~~~~~~
/data2/InternLM/apex/csrc/fused_dense.cpp: In function ‘std::vectorat::Tensor linear_gelu_linear_backward(at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor)’:
/data2/InternLM/apex/csrc/fused_dense.cpp:149:73: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
149 | auto d_weight1 = at::empty({hidden_features, in_features}, input.type());
| ^
In file included from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/ATen/core/Tensor.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/ATen/DeviceGuard.h:4,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/ATen/ATen.h:11,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/all.h:8,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/extension.h:4,
from /data2/InternLM/apex/csrc/fused_dense.cpp:1:
/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:213:30: note: declared here
213 | DeprecatedTypeProperties & type() const {
| ^~~~
/data2/InternLM/apex/csrc/fused_dense.cpp:150:74: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
150 | auto d_weight2 = at::empty({out_features, hidden_features}, input.type());
| ^
In file included from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/ATen/core/Tensor.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/ATen/DeviceGuard.h:4,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/ATen/ATen.h:11,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/all.h:8,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/extension.h:4,
from /data2/InternLM/apex/csrc/fused_dense.cpp:1:
/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:213:30: note: declared here
213 | DeprecatedTypeProperties & type() const {
| ^~~~
/data2/InternLM/apex/csrc/fused_dense.cpp:151:58: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
151 | auto d_bias1 = at::empty({hidden_features}, input.type());
| ^
In file included from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/ATen/core/Tensor.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/ATen/DeviceGuard.h:4,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/ATen/ATen.h:11,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/all.h:8,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/extension.h:4,
from /data2/InternLM/apex/csrc/fused_dense.cpp:1:
/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:213:30: note: declared here
213 | DeprecatedTypeProperties & type() const {
| ^~~~
/data2/InternLM/apex/csrc/fused_dense.cpp:152:55: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
152 | auto d_bias2 = at::empty({out_features}, input.type());
| ^
In file included from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/ATen/core/Tensor.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/ATen/DeviceGuard.h:4,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/ATen/ATen.h:11,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/all.h:8,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/extension.h:4,
from /data2/InternLM/apex/csrc/fused_dense.cpp:1:
/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:213:30: note: declared here
213 | DeprecatedTypeProperties & type() const {
| ^~~~
/data2/InternLM/apex/csrc/fused_dense.cpp:153:66: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
153 | auto d_input = at::empty({batch_size, in_features}, input.type());
| ^
In file included from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/ATen/core/Tensor.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/ATen/DeviceGuard.h:4,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/ATen/ATen.h:11,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/all.h:8,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/extension.h:4,
from /data2/InternLM/apex/csrc/fused_dense.cpp:1:
/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:213:30: note: declared here
213 | DeprecatedTypeProperties & type() const {
| ^~~~
/data2/InternLM/apex/csrc/fused_dense.cpp:154:72: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
154 | auto d_output1 = at::empty({batch_size, hidden_features}, input.type());
| ^
In file included from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/ATen/core/Tensor.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/ATen/DeviceGuard.h:4,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/ATen/ATen.h:11,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/all.h:8,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/extension.h:4,
from /data2/InternLM/apex/csrc/fused_dense.cpp:1:
/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:213:30: note: declared here
213 | DeprecatedTypeProperties & type() const {
| ^~~~
/data2/InternLM/apex/csrc/fused_dense.cpp:157:55: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
157 | auto lt_workspace = at::empty({1 << 22}, input.type());
| ^
In file included from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/ATen/core/Tensor.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/ATen/DeviceGuard.h:4,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/ATen/ATen.h:11,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/all.h:8,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/extension.h:4,
from /data2/InternLM/apex/csrc/fused_dense.cpp:1:
/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:213:30: note: declared here
213 | DeprecatedTypeProperties & type() const {
| ^~~~
In file included from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/ATen/ATen.h:13,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/all.h:8,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/extension.h:4,
from /data2/InternLM/apex/csrc/fused_dense.cpp:1:
/data2/InternLM/apex/csrc/fused_dense.cpp: In lambda function:
/data2/InternLM/apex/csrc/fused_dense.cpp:159:94: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
159 | AT_DISPATCH_FLOATING_TYPES_AND2(at::ScalarType::Half, at::ScalarType::BFloat16, input.type(), "linear_bias_backward", [&] {
| ^
/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/ATen/Dispatch.h:312:28: note: in definition of macro ‘AT_DISPATCH_FLOATING_TYPES_AND2’
312 | const auto& the_type = TYPE;
| ^~~~
In file included from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/ATen/core/Tensor.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/ATen/DeviceGuard.h:4,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/ATen/ATen.h:11,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/all.h:8,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/extension.h:4,
from /data2/InternLM/apex/csrc/fused_dense.cpp:1:
/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:213:30: note: declared here
213 | DeprecatedTypeProperties & type() const {
| ^~~~
In file included from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/ATen/ATen.h:13,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/all.h:8,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/extension.h:4,
from /data2/InternLM/apex/csrc/fused_dense.cpp:1:
/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/ATen/Dispatch.h:314:56: warning: ‘c10::ScalarType detail::scalar_type(const at::DeprecatedTypeProperties&)’ is deprecated: passing at::DeprecatedTypeProperties to an AT_DISPATCH macro is deprecated, pass an at::ScalarType instead [-Wdeprecated-declarations]
314 | at::ScalarType st = ::detail::scalar_type(the_type);
| ^
/data2/InternLM/apex/csrc/fused_dense.cpp:159:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND2’
159 | AT_DISPATCH_FLOATING_TYPES_AND2(at::ScalarType::Half, at::ScalarType::BFloat16, input.type(), "linear_bias_backward", [&] {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/ATen/Dispatch.h:178:23: note: declared here
178 | inline at::ScalarType scalar_type(const at::DeprecatedTypeProperties& t) {
| ^~~~~~~~~~~
/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/ATen/Dispatch.h:314:56: warning: ‘c10::ScalarType detail::scalar_type(const at::DeprecatedTypeProperties&)’ is deprecated: passing at::DeprecatedTypeProperties to an AT_DISPATCH macro is deprecated, pass an at::ScalarType instead [-Wdeprecated-declarations]
314 | at::ScalarType st = ::detail::scalar_type(the_type);
| ^
/data2/InternLM/apex/csrc/fused_dense.cpp:159:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND2’
159 | AT_DISPATCH_FLOATING_TYPES_AND2(at::ScalarType::Half, at::ScalarType::BFloat16, input.type(), "linear_bias_backward", [&] {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/ATen/Dispatch.h:178:23: note: declared here
178 | inline at::ScalarType scalar_type(const at::DeprecatedTypeProperties& t) {
| ^~~~~~~~~~~
/data2/InternLM/apex/csrc/fused_dense.cpp: In lambda function:
/data2/InternLM/apex/csrc/fused_dense.cpp:163:10: warning: unused variable ‘result’ [-Wunused-variable]
163 | auto result = linear_gelu_linear_backward_cuda<scalar_t>(
| ^~~~~~
/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/ATen/Dispatch.h:68:12: note: in definition of macro ‘AT_PRIVATE_CASE_TYPE_USING_HINT’
68 | return VA_ARGS();
| ^~~~~~~~~~~
/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/ATen/Dispatch.h:317:7: note: in expansion of macro ‘AT_PRIVATE_CASE_TYPE’
317 | AT_PRIVATE_CASE_TYPE(NAME, at::ScalarType::Double, double, VA_ARGS)
| ^~~~~~~~~~~~~~~~~~~~
/data2/InternLM/apex/csrc/fused_dense.cpp:159:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND2’
159 | AT_DISPATCH_FLOATING_TYPES_AND2(at::ScalarType::Half, at::ScalarType::BFloat16, input.type(), "linear_bias_backward", [&] {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/data2/InternLM/apex/csrc/fused_dense.cpp: In lambda function:
/data2/InternLM/apex/csrc/fused_dense.cpp:163:10: warning: unused variable ‘result’ [-Wunused-variable]
163 | auto result = linear_gelu_linear_backward_cuda<scalar_t>(
| ^~~~~~
/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/ATen/Dispatch.h:68:12: note: in definition of macro ‘AT_PRIVATE_CASE_TYPE_USING_HINT’
68 | return VA_ARGS();
| ^~~~~~~~~~~
/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/ATen/Dispatch.h:318:7: note: in expansion of macro ‘AT_PRIVATE_CASE_TYPE’
318 | AT_PRIVATE_CASE_TYPE(NAME, at::ScalarType::Float, float, VA_ARGS)
| ^~~~~~~~~~~~~~~~~~~~
/data2/InternLM/apex/csrc/fused_dense.cpp:159:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND2’
159 | AT_DISPATCH_FLOATING_TYPES_AND2(at::ScalarType::Half, at::ScalarType::BFloat16, input.type(), "linear_bias_backward", [&] {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/data2/InternLM/apex/csrc/fused_dense.cpp: In lambda function:
/data2/InternLM/apex/csrc/fused_dense.cpp:163:10: warning: unused variable ‘result’ [-Wunused-variable]
163 | auto result = linear_gelu_linear_backward_cuda<scalar_t>(
| ^~~~~~
/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/ATen/Dispatch.h:68:12: note: in definition of macro ‘AT_PRIVATE_CASE_TYPE_USING_HINT’
68 | return VA_ARGS();
| ^~~~~~~~~~~
/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/ATen/Dispatch.h:319:7: note: in expansion of macro ‘AT_PRIVATE_CASE_TYPE’
319 | AT_PRIVATE_CASE_TYPE(
| ^~~~~~~~~~~~~~~~~~~~
/data2/InternLM/apex/csrc/fused_dense.cpp:159:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND2’
159 | AT_DISPATCH_FLOATING_TYPES_AND2(at::ScalarType::Half, at::ScalarType::BFloat16, input.type(), "linear_bias_backward", [&] {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/data2/InternLM/apex/csrc/fused_dense.cpp: In lambda function:
/data2/InternLM/apex/csrc/fused_dense.cpp:163:10: warning: unused variable ‘result’ [-Wunused-variable]
163 | auto result = linear_gelu_linear_backward_cuda<scalar_t>(
| ^~~~~~
/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/ATen/Dispatch.h:68:12: note: in definition of macro ‘AT_PRIVATE_CASE_TYPE_USING_HINT’
68 | return VA_ARGS();
| ^~~~~~~~~~~
/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/ATen/Dispatch.h:324:7: note: in expansion of macro ‘AT_PRIVATE_CASE_TYPE’
324 | AT_PRIVATE_CASE_TYPE(
| ^~~~~~~~~~~~~~~~~~~~
/data2/InternLM/apex/csrc/fused_dense.cpp:159:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND2’
159 | AT_DISPATCH_FLOATING_TYPES_AND2(at::ScalarType::Half, at::ScalarType::BFloat16, input.type(), "linear_bias_backward", [&] {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/c10/core/Device.h:5,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/c10/core/Allocator.h:6,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/ATen/ATen.h:7,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/all.h:8,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/extension.h:4,
from /data2/InternLM/apex/csrc/fused_dense.cpp:1:
/data2/InternLM/apex/csrc/fused_dense.cpp: In lambda function:
/data2/InternLM/apex/csrc/fused_dense.cpp:159:94: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
159 | AT_DISPATCH_FLOATING_TYPES_AND2(at::ScalarType::Half, at::ScalarType::BFloat16, input.type(), "linear_bias_backward", [&] {
| ^
/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/c10/util/Exception.h:260:39: note: in definition of macro ‘C10_EXPAND_MSVC_WORKAROUND’
260 | #define C10_EXPAND_MSVC_WORKAROUND(x) x
| ^
/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/c10/util/Exception.h:465:9: note: in expansion of macro ‘TORCH_CHECK_MSG’
465 | TORCH_CHECK_MSG(cond, "", ##VA_ARGS));
| ^~~~~~~~~~~~~~~
/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/c10/util/Exception.h:637:32: note: in expansion of macro ‘TORCH_CHECK’
637 | C10_EXPAND_MSVC_WORKAROUND(TORCH_CHECK(false, ::c10::str(VA_ARGS)));
| ^~~~~~~~~~~
/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/ATen/Dispatch.h:330:9: note: in expansion of macro ‘AT_ERROR’
330 | AT_ERROR(#NAME, " not implemented for '", toString(TYPE), "'");
| ^~~~~~~~
/data2/InternLM/apex/csrc/fused_dense.cpp:159:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND2’
159 | AT_DISPATCH_FLOATING_TYPES_AND2(at::ScalarType::Half, at::ScalarType::BFloat16, input.type(), "linear_bias_backward", [&] {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/ATen/core/Tensor.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/ATen/DeviceGuard.h:4,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/ATen/ATen.h:11,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/all.h:8,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/extension.h:4,
from /data2/InternLM/apex/csrc/fused_dense.cpp:1:
/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:213:30: note: declared here
213 | DeprecatedTypeProperties & type() const {
| ^~~~
In file included from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/c10/core/Device.h:5,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/c10/core/Allocator.h:6,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/ATen/ATen.h:7,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/all.h:8,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/extension.h:4,
from /data2/InternLM/apex/csrc/fused_dense.cpp:1:
/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/ATen/Dispatch.h:330:51: error: ‘toString’ was not declared in this scope
330 | AT_ERROR(#NAME, " not implemented for '", toString(TYPE), "'");
| ^~~~~~~~
/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/c10/util/Exception.h:260:39: note: in definition of macro ‘C10_EXPAND_MSVC_WORKAROUND’
260 | #define C10_EXPAND_MSVC_WORKAROUND(x) x
| ^
/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/c10/util/Exception.h:465:9: note: in expansion of macro ‘TORCH_CHECK_MSG’
465 | TORCH_CHECK_MSG(cond, "", ##VA_ARGS));
| ^~~~~~~~~~~~~~~
/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/c10/util/Exception.h:637:32: note: in expansion of macro ‘TORCH_CHECK’
637 | C10_EXPAND_MSVC_WORKAROUND(TORCH_CHECK(false, ::c10::str(VA_ARGS)));
| ^~~~~~~~~~~
/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/ATen/Dispatch.h:330:9: note: in expansion of macro ‘AT_ERROR’
330 | AT_ERROR(#NAME, " not implemented for '", toString(TYPE), "'");
| ^~~~~~~~
/data2/InternLM/apex/csrc/fused_dense.cpp:159:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND2’
159 | AT_DISPATCH_FLOATING_TYPES_AND2(at::ScalarType::Half, at::ScalarType::BFloat16, input.type(), "linear_bias_backward", [&] {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/ATen/Dispatch.h:330:51: note: suggested alternatives:
330 | AT_ERROR(#NAME, " not implemented for '", toString(TYPE), "'");
| ^~~~~~~~
/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/c10/util/Exception.h:260:39: note: in definition of macro ‘C10_EXPAND_MSVC_WORKAROUND’
260 | #define C10_EXPAND_MSVC_WORKAROUND(x) x
| ^
/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/c10/util/Exception.h:465:9: note: in expansion of macro ‘TORCH_CHECK_MSG’
465 | TORCH_CHECK_MSG(cond, "", ##VA_ARGS));
| ^~~~~~~~~~~~~~~
/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/c10/util/Exception.h:637:32: note: in expansion of macro ‘TORCH_CHECK’
637 | C10_EXPAND_MSVC_WORKAROUND(TORCH_CHECK(false, ::c10::str(VA_ARGS)));
| ^~~~~~~~~~~
/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/ATen/Dispatch.h:330:9: note: in expansion of macro ‘AT_ERROR’
330 | AT_ERROR(#NAME, " not implemented for '", toString(TYPE), "'");
| ^~~~~~~~
/data2/InternLM/apex/csrc/fused_dense.cpp:159:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND2’
159 | AT_DISPATCH_FLOATING_TYPES_AND2(at::ScalarType::Half, at::ScalarType::BFloat16, input.type(), "linear_bias_backward", [&] {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/ATen/core/op_registration/infer_schema.h:8,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/library.h:61,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/types.h:12,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/all.h:8,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/extension.h:4,
from /data2/InternLM/apex/csrc/fused_dense.cpp:1:
/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/ATen/core/function_schema.h:522:20: note: ‘c10::toString’
522 | inline std::string toString(const FunctionSchema& schema) {
| ^~~~~~~~
In file included from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/jit/ir/ir.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/jit/api/function_impl.h:4,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/jit/api/method.h:7,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/jit/api/object.h:6,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/jit/api/module.h:4,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/serialize/input-archive.h:7,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/serialize/archive.h:3,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data/samplers/serialize.h:4,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data/samplers.h:8,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data/datasets/chunk.h:7,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data/datasets.h:4,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/data.h:4,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/all.h:8,
from /data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/extension.h:4,
from /data2/InternLM/apex/csrc/fused_dense.cpp:1:
/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/jit/ir/attributes.h:35:27: note: ‘torch::jit::toString’
35 | static inline const char* toString(AttributeKind kind) {
| ^~~~~~~~
[2/2] /usr/local/cuda-11.3/bin/nvcc -I/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include -I/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/TH -I/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/THC -I/usr/local/cuda-11.3/include -I/data1/lym/anaconda3/envs/lab2/include/python3.10 -c -c /data2/InternLM/apex/csrc/fused_dense_cuda.cu -o /data2/InternLM/apex/build/temp.linux-x86_64-cpython-310/csrc/fused_dense_cuda.o -D__CUDA_NO_HALF_OPERATORS -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -O3 -DVERSION_GE_1_1 -DVERSION_GE_1_3 -DVERSION_GE_1_5 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=fused_dense_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 -std=c++14
/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/c10/core/SymInt.h(84): warning: integer conversion resulted in a change of sign
/data2/InternLM/apex/csrc/fused_dense_cuda.cu(1631): warning: variable "beta_zero" was declared but never referenced
/data2/InternLM/apex/csrc/fused_dense_cuda.cu(1755): warning: variable "alpha" was declared but never referenced
/data2/InternLM/apex/csrc/fused_dense_cuda.cu(1756): warning: variable "beta_zero" was declared but never referenced
/data2/InternLM/apex/csrc/fused_dense_cuda.cu(1757): warning: variable "status" was declared but never referenced
/data2/InternLM/apex/csrc/fused_dense_cuda.cu(1812): warning: variable "alpha" was declared but never referenced
/data2/InternLM/apex/csrc/fused_dense_cuda.cu(1813): warning: variable "beta_zero" was declared but never referenced
/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/include/c10/core/SymInt.h(84): warning: integer conversion resulted in a change of sign
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
File "/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1808, in _run_ninja_build
subprocess.run(
File "/data1/lym/anaconda3/envs/lab2/lib/python3.10/subprocess.py", line 524, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "", line 2, in
File "", line 34, in
File "/data2/InternLM/apex/setup.py", line 795, in
setup(
File "/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/setuptools/init.py", line 107, in setup
return distutils.core.setup(**attrs)
File "/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 185, in setup
return run_commands(dist)
File "/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
dist.run_commands()
File "/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
self.run_command(cmd)
File "/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/setuptools/dist.py", line 1234, in run_command
super().run_command(command)
File "/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
cmd_obj.run()
File "/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/setuptools/command/install.py", line 74, in run
return orig.install.run(self)
File "/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/setuptools/_distutils/command/install.py", line 697, in run
self.run_command('build')
File "/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
self.distribution.run_command(command)
File "/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/setuptools/dist.py", line 1234, in run_command
super().run_command(command)
File "/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
cmd_obj.run()
File "/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/setuptools/_distutils/command/build.py", line 131, in run
self.run_command(cmd_name)
File "/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
self.distribution.run_command(command)
File "/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/setuptools/dist.py", line 1234, in run_command
super().run_command(command)
File "/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
cmd_obj.run()
File "/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/setuptools/command/build_ext.py", line 84, in run
_build_ext.run(self)
File "/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 345, in run
self.build_extensions()
File "/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 765, in build_extensions
build_ext.build_extensions(self)
File "/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 467, in build_extensions
self._build_extensions_serial()
File "/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 493, in _build_extensions_serial
self.build_extension(ext)
File "/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/setuptools/command/build_ext.py", line 246, in build_extension
_build_ext.build_extension(self, ext)
File "/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 548, in build_extension
objects = self.compiler.compile(
File "/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 586, in unix_wrap_ninja_compile
_write_ninja_file_and_compile_objects(
File "/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1487, in _write_ninja_file_and_compile_objects
_run_ninja_build(
File "/data1/lym/anaconda3/envs/lab2/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1824, in _run_ninja_build
raise RuntimeError(message) from e
RuntimeError: Error compiling objects for extension
error: subprocess-exited-with-error
× Running setup.py install for apex did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
full command: /data1/lym/anaconda3/envs/lab2/bin/python -u -c '
exec(compile('"'"''"'"''"'"'
distutils.core
to work with newer packaging standards.sys.argv[0]
to the underlying setup.py
, when invoking setup.py
so-c
. This avoids the following warning:import os, sys, tokenize
try:
import setuptools
except ImportError as error:
print(
"ERROR: Can not execute setup.py
since setuptools is not available in "
"the build environment.",
file=sys.stderr,
)
sys.exit(1)
file = %r
sys.argv[0] = file
if os.path.exists(file):
filename = file
with tokenize.open(file) as f:
setup_py_code = f.read()
else:
filename = ""
setup_py_code = "from setuptools import setup; setup()"
exec(compile(setup_py_code, filename, "exec"))
'"'"''"'"''"'"' % ('"'"'/data2/InternLM/apex/setup.py'"'"',), "", "exec"))' --cpp_ext --cuda_ext install --record /tmp/pip-record-_mp6xe2r/install-record.txt --single-version-externally-managed --compile --install-headers /data1/lym/anaconda3/envs/lab2/include/python3.10/apex
cwd: /data2/InternLM/apex/
Running setup.py install for apex ... error
error: legacy-install-failure
× Encountered error while trying to install package.
╰─> apex
note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.
No response
In demo code: python tools/tokenizer.py --raw_data_name your_raw_data_file_name(without suffix) --input_file_type 'text' or 'json' or 'jsonl' --bin your_output_bin_path
'text' should be 'txt'
same to the demo code: python tools/tokenizer.py --raw_data_name raw_data --input_file_type 'text' --bin cn/output.bin
Both in chinese and engish,and other language if existed
any browser
No response
model = AutoModelForCausalLM.from_pretrained("/home/hope/work/models/internlm-chat-7b-8k", trust_remote_code=True).to(torch.bfloat16).cuda()
Loading checkpoint shards: 100%|███████████████████████████████| 2/2 [00:11<00:00, 5.90s/it]
Traceback (most recent call last):
File "", line 1, in
NameError: name 'torch' is not defined
import torch
tokenizer = AutoTokenizer.from_pretrained("/home/hope/work/models/internlm-chat-7b-8k", trust_remote_code=True)
model = model.eval()Traceback (most recent call last):
File "", line 1, in
NameError: name 'model' is not defined
I can load the model into VRAM. However, when I call the chat methods, an exception occurs:
CUDA out of memory. Tried to allocate 1.57 GiB (GPU 0; 22.05 GiB total capacity; 19.76 GiB already allocated; 1.32 GiB free; 19.77 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
Is it an expected behavior? How much VRAM do I need to run the model?
Python 3.8
Cuda 12.1
CPU: 8 cores, RAM: 32GB
GPU: A10, VRAM: 24GB
I set the device_map to auto since I need to load the model into GPU. Otherwise, it says
RuntimeError: [enforce fail at alloc_cpu.cpp:75] err == 0. DefaultCPUAllocator: can't allocate memory: you tried to allocate 845152256 bytes. Error code 12 (Cannot allocate memory)
Why is this required? It is not secure. Can you change it so this requirement goes away? Thanks.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.