can't not load model.. about alpaca-turbo HOT 11 OPEN

can you try the alpaca-native-enhanced model
and can you tell me how many cores are there in your system

from alpaca-turbo.

I've had the same problem with the "ggml-model-q4_0.bin" but had no issue with "ggml-model-q4_0_unfiltered.bin".
So it's probably about the model.

from alpaca-turbo.

I'm having a similar problem. Win10, ggml Alpaca 7B downloaded from huggingface:
https://huggingface.co/Pi3141

from alpaca-turbo.

Same issue here with: gpt4all-lora-quantized.bin and ggml-alpaca-7b-q4.bin

from alpaca-turbo.

Using 2 files from Pi3141, gpt4-x-alpaca-native-13B-ggml worked, alpaca-native-7B-ggml didn't load

gpt4-x-alpaca-native-13B-ggml:
'models\ggml-model-q4_1.bin',
Working... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:00

alpaca-native-7B-ggml:
'models\ggml-model-q4_0.bin',
ERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoR

from alpaca-turbo.

Similar, won't load any of my 3 quantized 7B and 13B alpaca variants that worked in dalai or alpaca.cpp.

...
'-m',
'models\ggml-model-q4_0.bin',
'--interactive-first'
]
ERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoR

from alpaca-turbo.

Same here ggml-alpaca-7b-q4.bin not loading

from alpaca-turbo.

Same here, llama7B, llama13B, alpaca,... - all working locally with llama.cpp on the commandline. All hanging on load.

Parameters for invoking llama.cpp commandline seem right and commandline status shows apparent completion:
Working... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:00

But web-ui dialogue hangs with loading.

EDIT: running on macOS/Apple Silicon via current git-clone + copying templates folder from ZIP.

from alpaca-turbo.

Same here, endless -loading model "ggml-model-q4_0.bin"

from alpaca-turbo.

Updated / totally edited for better clarification.

• I'm on macOS/Apple silicon. Running the current/latest llama.cpp with several models from terminal. It all works fine in terminal, even when testing in alpaca-turbo's environment with its parameters from the terminal
• Because I want the latest llama.cpp+models, I can't just run the docker or other images. Needed to git-clone (+ copy templates folder from ZIP). Linked my working llama.cpp "main" to ./bin/mac, and its models' *.bin into ./models with nice names for the UI.

Cause of model-hang for me:
Alpaca-Turbo expects a certain string "RSTsr" to be displayed by it's invocation of llama.cpp once its finished loading (set in alpaca_turbo.py line 46). The current llama.cpp + llama 7B e.g. does not show this string in its invocation. So the load hangs. For me it seems, it's not how the model is, it's more how Alpaca-Turbo was constructed. The current llama.cpp+models produces very different responses - I'm trying to understand alpaca_turbo.py, so that maybe I can fix it (at least for me).

Hope this clarifies/helps.

from alpaca-turbo.

Update - I got it to work (most of the time) on my Mac by changing alpaca_turbo.py quite a bit. But I don't think it is mergeable into a pull-request, because my solution seems to be one-off just for my situation.

from alpaca-turbo.