Comments (2)
mrwyattii
commented on May 20, 2024
Hi @C00reNUT, sorry for the late reply and thanks for checking out MII. Do you have DeepSpeed installed in your environment?
Could you show the output of ds_report?
from deepspeed-mii.
C00reNUT
commented on May 20, 2024
I tried it once more and when I run the code as a .py scripts it works, but when I run it from jupyter lab cells (which is what I did in the first place) it throws the FileNotFoundError error. The jupyter notebook is in the same location as the .py script and it uses the same conda env.
Anyway it works now. And I guess that the problem is probably some mismatch with PATHs on my machine...
ds_report
--------------------------------------------------
DeepSpeed C++/CUDA extension op report
--------------------------------------------------
NOTE: Ops not installed will be just-in-time (JIT) compiled at
runtime if needed. Op compatibility means that your system
meet the required dependencies to JIT install the op.
--------------------------------------------------
JIT compiled ops requires ninja
ninja .................. [OKAY]
--------------------------------------------------
op name ................ installed .. compatible
--------------------------------------------------
cpu_adam ............... [NO] ....... [OKAY]
cpu_adagrad ............ [NO] ....... [OKAY]
fused_adam ............. [NO] ....... [OKAY]
fused_lamb ............. [NO] ....... [OKAY]
[WARNING] please install triton==1.0.0 if you want to use sparse attention
sparse_attn ............ [NO] ....... [NO]
transformer ............ [NO] ....... [OKAY]
stochastic_transformer . [NO] ....... [OKAY]
[WARNING] async_io requires the dev libaio .so object and headers but these were not found.
[WARNING] async_io: please install the libaio-dev package with apt
[WARNING] If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.
async_io ............... [NO] ....... [NO]
utils .................. [NO] ....... [OKAY]
quantizer .............. [NO] ....... [OKAY]
transformer_inference .. [NO] ....... [OKAY]
--------------------------------------------------
DeepSpeed general environment info:
torch install path ............... ['/mnt/2287294e-32c7-437b-84bd-452a29548b1a/conda_env/DeepSpeedInterface/lib/python3.9/site-packages/torch']
torch version .................... 1.12.1
torch cuda version ............... 11.6
torch hip version ................ None
nvcc version ..................... 11.6
deepspeed install path ........... ['/mnt/2287294e-32c7-437b-84bd-452a29548b1a/conda_env/DeepSpeedInterface/lib/python3.9/site-packages/deepspeed']
deepspeed info ................... 0.7.2, unknown, unknown
deepspeed wheel compiled w. ...... torch 1.12, cuda 11.6
from deepspeed-mii.
Related Issues (20)
- Base model support HOT 1
- Support for Codellama Model in deepspeed-fastgen HOT 2
- Invalid parameter bricks service HOT 1
- Deadlock detected HOT 5
- AttributeError: 'NoneType' object has no attribute 'value' HOT 2
- RuntimeError: There is no current event loop in thread 'Thread-1'. HOT 2
- support chatglm3 HOT 1
- Loading Bloom-3b in persisten mode fails HOT 1
- restful api host need configuration HOT 1
- Can deepspeed-MII run on AMD GPU? HOT 2
- The choice of the split size for splitAndFuse HOT 4
- How to stream tokens? HOT 1
- Error messages spilled from persistent deployment for every request HOT 6
- Can you support DeepSeek's inference acceleration? Thank you very much. HOT 2
- mixtral support HOT 2
- Deployment in kubernetes
- Reproduced readme results HOT 8
- Can MII support quanted Llama2 of AWQ?
- Problem while running facebook/opt-125m with MII HOT 2
- When running mii.serv, it keeps print waiting for server to start. HOT 6
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from deepspeed-mii.