Comments (3)
Your M1 Mac does not have Cuda, which is a feature of Nvidia GPUs. From what I found out, theoretically you should use .to("mps")
instead of .cuda()
to replace Cuda with Metal, but this causes a different problem for me. The script crashes and I get the error described here: pytorch/pytorch#99564.
from stablelm.
The GGML project, for running LLMs on CPUs (including specifically mac support!) has an initial example project that can run StableLM: https://github.com/ggerganov/ggml/tree/master/examples/stablelm
There's also https://huggingface.co/cakewalk/ggml-q4_0-stablelm-tuned-alpha-7b/tree/main which supposedly works in llama.cpp
from stablelm.
Hey all,
same for me. If I try to execute the "python app.py" which i got from: https://huggingface.co/spaces/stabilityai/stablelm-tuned-alpha-chat/tree/main I will get:
stablelm-tuned-alpha-chat git:(main) ✗ python app.py
Starting to load the model to memory
Loading checkpoint shards: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:34<00:00, 8.60s/it]
Traceback (most recent call last):
File "/Users/rspies/work/projects/stablelm-tuned-alpha-chat/app.py", line 12, in <module>
"stabilityai/stablelm-tuned-alpha-7b", torch_dtype=torch.float16).cuda()
^^^^^^
File "/Users/rspies/work/projects/stablelm-tuned-alpha-chat/my_project_env/lib/python3.11/site-packages/torch/nn/modules/module.py", line 905, in cuda
return self._apply(lambda t: t.cuda(device))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/rspies/work/projects/stablelm-tuned-alpha-chat/my_project_env/lib/python3.11/site-packages/torch/nn/modules/module.py", line 797, in _apply
module._apply(fn)
File "/Users/rspies/work/projects/stablelm-tuned-alpha-chat/my_project_env/lib/python3.11/site-packages/torch/nn/modules/module.py", line 797, in _apply
module._apply(fn)
File "/Users/rspies/work/projects/stablelm-tuned-alpha-chat/my_project_env/lib/python3.11/site-packages/torch/nn/modules/module.py", line 820, in _apply
param_applied = fn(param)
^^^^^^^^^
File "/Users/rspies/work/projects/stablelm-tuned-alpha-chat/my_project_env/lib/python3.11/site-packages/torch/nn/modules/module.py", line 905, in <lambda>
return self._apply(lambda t: t.cuda(device))
^^^^^^^^^^^^^^
File "/Users/rspies/work/projects/stablelm-tuned-alpha-chat/my_project_env/lib/python3.11/site-packages/torch/cuda/__init__.py", line 239, in _lazy_init
raise AssertionError("Torch not compiled with CUDA enabled")
AssertionError: Torch not compiled with CUDA enabled
Is there any way to use cpu instead cuda (or similar)?
I tried it on a Mac with M1.
Best regards,
Reinhard
from stablelm.
Related Issues (20)
- loss not decreasing with deepspeed HOT 1
- Training Script stablity 3B and 7B HOT 6
- Unclear tokenizer class HOT 2
- Cannot run demo HOT 2
- fairyfloss HOT 2
- process killed HOT 4
- License unclear HOT 8
- Is it normal to take a long time ( about 15min )to generate an answer? HOT 1
- How to expand the sequence length of llama? HOT 1
- Consider using OpenAI Evals
- The output is the same as the input. HOT 1
- Is this project abandoned? HOT 4
- Stability AI
- Hello, how to convert the statityai/tablelm-base-alpha-3b to ggml format HOT 1
- Target modules ['query_key_value', 'dense', 'dense_h_to_4h', 'dense_4h_to_h'] not found in the base model. Please check the target modules and try again. HOT 2
- OSError: stabilityai/stablelm-base-alpha-3b-v2 does not appear to have a file named pytorch_model.bin, tf_model.h5, model.ckpt or flax_model.msgpack. HOT 3
- Windows fatal exception: access violation
- Chatting and prompt
- Big difference between the before-cooldown-ckpt and the final checkpoint in the results of downstream tasks?
- Can you share code/resources for Self Knowledge learning? HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from stablelm.