Comments (3)
I think this a great idea. We have something similar right now via the DeploymentType.AML
pathway, however the Docker image we build there is intended for AzureML. We may be able to adapt the work we have done there to create a Docker image for easier deployment.
In the meantime, could you please share what errors you are running into with installing deepspeed-mii
? is the error related to protobuf
versioning? We may have a bug in our requirements or packaging that should be resolved.
from deepspeed-mii.
I think this a great idea. We have something similar right now via the DeploymentType.AML pathway, however the Docker image we build there is intended for AzureML. We may be able to adapt the work we have done there to create a Docker image for easier deployment.
Nice! If I can help in anyway I would be happy to! This would be useful for me to have a proper image to deploy π
In the meantime, could you please share what errors you are running into with installing deepspeed-mii? is the error related to protobuf versioning? We may have a bug in our requirements or packaging that should be resolved
Yes, here are the steps followed with there respective outputs:
- conda create -n mii
- conda activate mii
- conda install pip
- pip install deepspeed-mii
- python main.py:
File "/home/vdmatos/miniconda3/envs/mii-2/lib/python3.10/site-packages/mii/models/providers/diffusers.py", line 6, in diffusers_provider
from diffusers import DiffusionPipeline
ModuleNotFoundError: No module named 'diffusers'
- pip install diffusers
- python main.py:
File "/home/vdmatos/miniconda3/envs/mii-2/lib/python3.10/site-packages/deepspeed/ops/transformer/inference/attention.py", line 22, in load_triton_flash_attn
raise ImportError("Please install triton 2.0+ or `pip install deepspeed[sd]`")
- pip install deepspeed[sd]
- python main.py:
File "/home/vdmatos/miniconda3/envs/mii-2/lib/python3.10/site-packages/deepspeed/inference/engine.py", line 238, in _validate_args
raise ValueError(f"model must be a torch.nn.Module, got {type(self.module)}")
ValueError: model must be a torch.nn.Module, got <class 'diffusers.pipelines.stable_diffusion.pipeline_stable_diffusion.StableDiffusionPipeline'>
(still by running this example)
Feel free to let me know if :
- I did any mistake in my installation step
- You want me to open a proper issue relating this error (rather than following it on this feature request)
from deepspeed-mii.
@Thytu there were some recent updates to MII and DeepSpeed with regards to Stable Diffusion. Your installation methods look fine, could you try running again with latest of each?
from deepspeed-mii.
Related Issues (20)
- RuntimeError: The server socket has failed to listen on any local network address HOT 1
- Only running one replica even though setting many replicas
- [Problem]errno: 98 - Address already in use
- Performance with vllm
- error when using Qwen1.5-32B
- ValueError: Unsupported model type phi3
- BUG in run_batch_processing
- Cannot run Yi-34B-Chat => ValueError: Unsupported q_ratio: 7 HOT 2
- [REQUEST] Mixtral-8x22B support
- [REQUEST] LLAMA-3 support
- Does deepspeed-mii support prefix_allowed_tokens_fn?
- DeepSpeed-MII θ½ε θ½½ιεηint4ζθ int8η樑εεοΌ
- Tf32 support
- How can I use the same prompt to produce the same text output as vllm
- Support LLava next stronger
- support Qwen
- support Qwen1.5
- support stream
- [BUG] MII Backend Hangs After 9999 Exceptions in `MIIAsyncPipeline.put_request` HOT 2
- few questions regarding the implementation of streaming and batching
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. πππ
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google β€οΈ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from deepspeed-mii.