Comments (6)
if you want to run the demo on cpu, you need to use float32 and init all param on cpu.
1. change demo.py
# model = model_cls.from_config(model_config).to('cuda:{}'.format(args.gpu_id)).
model = model_cls.from_config(model_config).to('cpu')
# chat = Chat(model, vis_processor, device='cuda:{}'.format(args.gpu_id))
chat = Chat(model, vis_processor, device='cpu')
2. change minigpt4.yaml
# vit_precision: "fp16"
vit_precision: "fp32"
3. change minigpt4_eval.yaml
# low_resource: True
low_resource: False
4. change mini_gpt4.py, about line 90:
# torch_dtype=torch.float16,
torch_dtype=torch.float32
that is all you need to do to use cpu run the demo.py, but the speed is very slow
from minigpt-4.
yep,I need help to run on CPU
I am downloading and merging models
from minigpt-4.
Hi! Has there been any progress on running it on a CPU? I'm really interested in this as well, since I don't have a powerful GPU. Any updates or workarounds you've discovered would be greatly appreciated. Thanks!
from minigpt-4.
yep,I need help to run on CPU
I am downloading and merging models
@kenneth104
Any progress of using CPU was made? Can you share?
from minigpt-4.
yep,I need help to run on CPU
I am downloading and merging models@kenneth104 Any progress of using CPU was made? Can you share?
noI can’t run on CPU, something need cuda and output error
from minigpt-4.
@liyaozong1991 thanks, I'll try this.
from minigpt-4.
Related Issues (20)
- Vicuna 13b model, could not parse ModelProto from /home/ken/git-repos/vicuna/config.json HOT 3
- MiniGPTV2 Text to Text
- what does the checkpoint2(stage2) and checkpoint(stage3) mean in the Minigpt_v2?
- lego
- [Detection] Is there any way to output detection confidence?
- /media/dell/data1/ljw/code/test3/New/MiniGPT-4/eval_scripts/eval_vqa.py: 行 24: 未预期的符号 `(' 附近有语法错误
- The order of connection between word vectors and image vectors in prompt
- The Hugging Space app is not available.
- I want to use minigpt-v2 to achieve image search. What are the implementation ideas
- What does the parameter "data" printed during training mean HOT 1
- Where is the region_descriptions.json?
- loss becomes nan during training
- valid_step in training epochs
- Could not create share link. Please check your internet connection or our status page: https://status.gradio.app
- MiniGPT-4 Colab notebook
- demo.py error when starting chat HOT 2
- Training Data Download
- Method to accelerate the inference
- Where I can add special token?
- Inconsistent outputs of MiniGPT-v2 (peft version is 0.2.0)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from minigpt-4.