Successfully run on windows 11! It took me 6 hours to complete, and I encountered many problems, which were finally solved.

My pc is rtx4090, and I use vicuna 13B. When uploading pictures and asking questions, it takes up 20G of video memory, and it takes about 5-30 seconds to wait for a reply.

from minigpt-4.

成功了，比较耗显存。

from minigpt-4.

Hi @Kizai , from the error info I see the placeholder of vicuna weight '/path/to/vicuna/weights', that means the vicuna weight path is not set. Please check the Readme to know how to set it. Thanks!

from minigpt-4.

怎么17G显存就能跑呀，你这卡也太多了吧

from minigpt-4.

卡在准备模型这一步了

from minigpt-4.

@huangzhongzhong It looks like the name of your tokenizer is incorrect. It should be 'LlamaTokenizer' instead of 'LLaMATokenizer'. You can update it in the tokenizer config file ' tokenizer_config.json' in your llama weight folder

from minigpt-4.

怎么17G显存就能跑呀，你这卡也太多了吧

int4量化了的应该是

from minigpt-4.

我运行报这个错误

from minigpt-4.

@TsuTikgiau

1. I have set vicuna weights in minigpt4/configs/models/minigpt4.yaml
2. I have set minigpt4 weights in eval_configs/minigpt4_eval.yaml
3. How should the llama weights be configured to load?

from minigpt-4.

@WangRongsheng Hello! How do you prepare the vicuna weights? The vicuna weight from huggingface is a delta version and cannot be directly used. We provide a guide PrepareVicuna.md to show you how to prepare the final working vicuna weight. LLAMA weight is used only in this preparation. Thanks!

from minigpt-4.

I encountered the same problem as the first floor，I'm not worthy

from minigpt-4.

I encountered the same problem as the first floor，I'm not worthy

yes, we are

from minigpt-4.

@WangRongsheng Hello! How do you prepare the vicuna weights? The vicuna weight from huggingface is a delta version and cannot be directly used. We provide a guide PrepareVicuna.md to show you how to prepare the final working vicuna weight. LLAMA weight is used only in this preparation. Thanks!

good! I have solved it!

from minigpt-4.

very vague，numb

from minigpt-4.

我在huggingface上建立了一个空间，MiniGPT-4的docker部署，有没有成功的大佬来指点一下
https://huggingface.co/spaces/zylj/MiniGPT-4

from minigpt-4.

楼主你好，请问LLaMA-13B的原始模型参数可以分享一下吗？十分感谢

from minigpt-4.

楼主你好，请问LLaMA-13B的原始模型参数可以分享一下吗？十分感谢

我也没有申请到模型，但是我直接在hugging face上找了别人的，也不知道是不是原始模型参数
https://huggingface.co/decapoda-research

from minigpt-4.

I encountered the same problem as the first floor，I'm not worthy

大佬你是如何解决repo_name问题的呀。我无论是使用相对路径还是绝对路径都会报错。附图：

from minigpt-4.

I use rtx4090 on windows 11 as well, but I got error message
AssertionError: Torch not compiled with CUDA enabled

Loading VIT
Loading VIT Done
Loading Q-Former
Loading Q-Former Done
Loading LLAMA

============BUG REPORT===================================
Welcome to bitsandbytes. For bug reports, please submit your error trace to: https://github.com/TimDettmers/bitsandbytes/

Loading checkpoint shards: 0%| | 0/3 [00:08<?, ?it/s]
╭────── Traceback (most recent call last) ────────────────────────────────╮
│ C:\MiniGPT-4\demo.py:60 in
│ ****
│ 57 model_config = cfg.model_cfg
│ 58 model_config.device_8bit = args.gpu_id
│ 59 model_cls = registry.get_model_class(model_config.arch)
│ ❱ 60 model = model_cls.from_config(model_config).to('cuda:{}'.format(args.gpu_id))
│ 61
│ 62 vis_processor_cfg = cfg.datasets_cfg.cc_sbu_align.vis_processor.train
│ 63 vis_processor = registry.get_processor_class(vis_processor_cfg.name).from_config(vis_pro
│
│ C:\MiniGPT-4\minigpt4\models\mini_gpt4.py:243 in from_config
│
│ 240 max_txt_len = cfg.get("max_txt_len", 32)
│ 241 end_sym = cfg.get("end_sym", '\n')
│ 242
│ ❱ 243 model = cls(
│ 244 vit_model=vit_model,
│ 245 q_former_model=q_former_model,
│ 246 img_size=img_size,
│
│ C:\MiniGPT-4\minigpt4\models\mini_gpt4.py:90 in init
│
│ 87 self.llama_tokenizer.pad_token = self.llama_tokenizer.eos_token
│ 88
│ 89 if self.low_resource:
│ ❱ 90 self.llama_model = LlamaForCausalLM.from_pretrained(
│ 91 llama_model,
│ 92 torch_dtype=torch.float16,
│ 93 load_in_8bit=True,
│
│ C:\Users\User\anaconda3\envs\minigpt4\lib\site-packages\transformers\modeling_utils.py:2795 in │
│ from_pretrained
│
│ 2792 mismatched_keys,
│ 2793 offload_index, │
│ 2794 error_msgs, │
│ ❱ 2795 ) = cls._load_pretrained_model(
│ 2796 model,
│ 2797 state_dict,
│ 2798 loaded_state_dict_keys, # XXX: rename?
│
│ C:\Users\User\anaconda3\envs\minigpt4\lib\site-packages\transformers\modeling_utils.py:3124 in
│ _load_pretrained_model
│
│ 3121 )
│ 3122
│ 3123 if low_cpu_mem_usage:
│ ❱ 3124 new_error_msgs, offload_index, state_dict_index = _load_state_dict_i
│ 3125 model_to_load,
│ 3126 state_dict,
│ 3127 loaded_keys,
│
│ C:\Users\User\anaconda3\envs\minigpt4\lib\site-packages\transformers\modeling_utils.py:706 in
│ load_state_dict_into_meta_model
│
│ 703 fp16_statistics = None
│ 704
│ 705 if "SCB" not in param_name:
│ ❱ 706 set_module_8bit_tensor_to_device(
│ 707 model, param_name, param_device, value=param, fp16_statistics=fp16_s
│ 708 )
│ 709
│
│ C:\Users\User\anaconda3\envs\minigpt4\lib\site-packages\transformers\utils\bitsandbytes.py:87 in
│ set_module_8bit_tensor_to_device
│
│ 84 if value is None:
│ 85 new_value = old_value.to(device)
│ 86 elif isinstance(value, torch.Tensor):
│ ❱ 87 new_value = value.to(device)
│ 88 else:
│ 89 new_value = torch.tensor(value, device=device)
│ 90
│ C:\Users\User\anaconda3\envs\minigpt4\lib\site-packages\torch\cuda_init.py:239 in _lazy_init
│
│ 236 "Cannot re-initialize CUDA in forked subprocess. To use CUDA with "
│ 237 "multiprocessing, you must use the 'spawn' start method")
│ 238 if not hasattr(torch._C, '_cuda_getDeviceCount'):
│ ❱ 239 raise AssertionError("Torch not compiled with CUDA enabled")
│ 240 if _cudart is None:
│ 241 raise AssertionError(
│ 242 "libcudart functions unavailable. It looks like you have a broken build?
╰───────────────────────────────────────────
AssertionError: Torch not compiled with CUDA enabled

Can someone know how to fix it? Thanks

from minigpt-4.

#81 maybe you can see it!☺️

from minigpt-4.

I use rtx4090 on windows 11 as well, but I got error message AssertionError: Torch not compiled with CUDA enabled

Loading VIT Loading VIT Done Loading Q-Former Loading Q-Former Done Loading LLAMA

============BUG REPORT===================================

Welcome to bitsandbytes. For bug reports, please submit your error trace to: https://github.com/TimDettmers/bitsandbytes/
Loading checkpoint shards: 0%| | 0/3 [00:08<?, ?it/s] ╭────── Traceback (most recent call last) ────────────────────────────────╮ │ C:\MiniGPT-4\demo.py:60 in │ **** │ 57 model_config = cfg.model_cfg │ 58 model_config.device_8bit = args.gpu_id │ 59 model_cls = registry.get_model_class(model_config.arch) │ ❱ 60 model = model_cls.from_config(model_config).to('cuda:{}'.format(args.gpu_id)) │ 61 │ 62 vis_processor_cfg = cfg.datasets_cfg.cc_sbu_align.vis_processor.train │ 63 vis_processor = registry.get_processor_class(vis_processor_cfg.name).from_config(vis_pro │ │ C:\MiniGPT-4\minigpt4\models\mini_gpt4.py:243 in from_config │ │ 240 max_txt_len = cfg.get("max_txt_len", 32) │ 241 end_sym = cfg.get("end_sym", '\n') │ 242 │ ❱ 243 model = cls( │ 244 vit_model=vit_model, │ 245 q_former_model=q_former_model, │ 246 img_size=img_size, │ │ C:\MiniGPT-4\minigpt4\models\mini_gpt4.py:90 in init │ │ 87 self.llama_tokenizer.pad_token = self.llama_tokenizer.eos_token │ 88 │ 89 if self.low_resource: │ ❱ 90 self.llama_model = LlamaForCausalLM.from_pretrained( │ 91 llama_model, │ 92 torch_dtype=torch.float16, │ 93 load_in_8bit=True, │ │ C:\Users\User\anaconda3\envs\minigpt4\lib\site-packages\transformers\modeling_utils.py:2795 in │ │ from_pretrained │ │ 2792 mismatched_keys, │ 2793 offload_index, │ │ 2794 error_msgs, │ │ ❱ 2795 ) = cls._load_pretrained_model( │ 2796 model, │ 2797 state_dict, │ 2798 loaded_state_dict_keys, # XXX: rename? │ │ C:\Users\User\anaconda3\envs\minigpt4\lib\site-packages\transformers\modeling_utils.py:3124 in │ _load_pretrained_model │ │ 3121 ) │ 3122 │ 3123 if low_cpu_mem_usage: │ ❱ 3124 new_error_msgs, offload_index, state_dict_index = load_state_dict_i │ 3125 model_to_load, │ 3126 state_dict, │ 3127 loaded_keys, │ │ C:\Users\User\anaconda3\envs\minigpt4\lib\site-packages\transformers\modeling_utils.py:706 in │ load_state_dict_into_meta_model │ │ 703 fp16_statistics = None │ 704 │ 705 if "SCB" not in param_name: │ ❱ 706 set_module_8bit_tensor_to_device( │ 707 model, param_name, param_device, value=param, fp16_statistics=fp16_s │ 708 ) │ 709 │ │ C:\Users\User\anaconda3\envs\minigpt4\lib\site-packages\transformers\utils\bitsandbytes.py:87 in │ set_module_8bit_tensor_to_device │ │ 84 if value is None: │ 85 new_value = old_value.to(device) │ 86 elif isinstance(value, torch.Tensor): │ ❱ 87 new_value = value.to(device) │ 88 else: │ 89 new_value = torch.tensor(value, device=device) │ 90 │ C:\Users\User\anaconda3\envs\minigpt4\lib\site-packages\torch\cuda__init.py:239 in _lazy_init │ │ 236 "Cannot re-initialize CUDA in forked subprocess. To use CUDA with " │ 237 "multiprocessing, you must use the 'spawn' start method") │ 238 if not hasattr(torch._C, '_cuda_getDeviceCount'): │ ❱ 239 raise AssertionError("Torch not compiled with CUDA enabled") │ 240 if _cudart is None: │ 241 raise AssertionError( │ 242 "libcudart functions unavailable. It looks like you have a broken build? ╰─────────────────────────────────────────── AssertionError: Torch not compiled with CUDA enabled

Can someone know how to fix it? Thanks

It seems like pytorch version is cpu version. Maybe change torch version and try again can work.

from minigpt-4.

乱码问题，这个要改一下Chrome编码方式？

from minigpt-4.

乱码问题，这个要改一下Chrome编码方式？

我也都是乱码，想问下应该怎么解决呀

from minigpt-4.

楼主你好，请问LLaMA-13B的原始模型参数可以分享一下吗？十分感谢

llamA weights

magnet:?xt=urn:btih:ZXXDAUWYLRUXXBHUYEMS6Q5CE5WA3LVA&dn=LLaMA

vicuna weight magnet

https://huggingface.co/lmsys/vicuna-13b-delta-v0 add to the header

magnet:?xt=urn:btih:a7fac57094561a63d53eed943f904abf24c6969d&dn=Vicuna-13B-HF-fp16-delta-merged_2023-04-03&tr=udp%3a%2f%2ftracker.opentrackr.o
rg%3a1337%2fannounce&tr=udp%2ftracker%2 -udp.gbitt.info%3a80%2fannounce&tr=udp%3a%2f%2ftracker1.bt.moack.co.kr%3a80%2fannounce&tr=udp%3a%2f%2f
tracker.tiny-vps.com%3a6969%2fannounce&tr=udp %3a%2f%2ftracker2.dler.org%3a80%2fannounce&tr=udp%3a%2f%2fopentracker.i2p.rocks%3a6969%2fannounc
e&tr=udp%3a%2f%2ftracker.altrosky.nl%3a6969%2fannounce&tr=udp%3a %2f%2ftracker.theoks.net%3a6969%2fannounce&tr=udp%3a%2f%2ftracker.dler.org%3a
6969%2fannounce&tr=udp%3a%2f%2ftracker.torrent.eu.org%3a451%2fannounce&tr=udp%3a %2f%2ftracker.openbittorrent.com%3a6969%2fannounce&tr=https%3
a%2f%2fopentracker.i2p.rocks%3a443%2fannounce&tr=http%3a%2f%2ftracker.openbittorrent.com%3a80%2fannounce&tr=udp%3a%2f%2ftracker.moeking.me%3a6
969%2fannounce&tr=udp%3a%2f%2ftracker.monitorit4.me%3a6969%2fannounce&tr=udp%3a%2f%2f9.rarbg.com% 3a2810%2fannounce

magnet:?xt=urn:btih:1e0c3dbeefe82483f81bd4e7ea959e4953c8081f&dn=Vicuna-13B-ggml-4bit-delta-merged_2023-04-03&tr=udp%3a%2f%2ftracker.opentrackr
.org%3a1337%2fannounce&tr=udp%2fa%2 .rarbg.com%3a2810%2fannounce&tr=udp%3a%2f%2ftracker.monitorit4.me%3a6969%2fannounce&tr=udp%3a%2f%2ftracker
2.dler.org%3a80%2fannounce&tr=udp%3a%2f%2fopentracker.i2p .rocks%3a6969%2fannounce&tr=udp%3a%2f%2ftracker.torrent.eu.org%3a451%2fannounce&tr=u
dp%3a%2f%2ftracker.openbittorrent.com%3a6969%2fannounce&tr=udp%3a%2f%2ftracker.dler .org%3a6969%2fannounce&tr=udp%3a%2f%2ftracker-udp.gbitt.in
fo%3a80%2fannounce&tr=udp%3a%2f%2ftracker1.bt.moack.co.kr%3a80%2fannounce&tr=https%3a%2f %2fopentracker.i2p.rocks%3a443%2fannounce&tr=udp%3a%2
f%2ftracker.altrosky.nl%3a6969%2fannounce&tr=udp%3a%2f%2ftracker.tiny-vps.com%3a6969%2fannounce&tr=udp%3a%2f%2ftracker.moeking.me%3a6969%2fann
ounce&tr=http%3a%2f%2ftracker.openbittorrent.com%3a80%2fannounce&tr=udp%3a%2f%2ftracker.theoks.net% 3a6969%2fannounce

from minigpt-4.

乱码问题，这个要改一下Chrome编码方式？

可能你的权重加载错误，你可以参考 #81

from minigpt-4.

使用的Windows 上面的docker跑的吗？我的会报错

from minigpt-4.

乱码问题，这个要改一下Chrome编码方式？

同样乱码，我配置的是7B的模型，请问如何解决？

from minigpt-4.

window10下的经验：
git clone https://github.com/Vision-CAIR/MiniGPT-4.git
cd MiniGPT-4
conda env create -f environment.yml
conda activate minigpt4
pip install torch==1.13.1+cu117 torchvision==0.14.1+cu117 torchaudio==0.13.1 --extra-index-url https://download.pytorch.org/whl/cu117
pip uninstall bitsandbytes
pip install https://github.com/jllllll/bitsandbytes-windows-webui/releases/download/wheels/bitsandbytes-0.39.1-py3-none-win_amd64.whl
其中有2个模型配置修改，和模型权重下载，可以参考：https://github.com/rbbrdckybk/MiniGPT-4
run python demo.py --cfg-path eval_configs/minigpt4_eval.yaml --gpu-id 0

from minigpt-4.