Will LLMFarm support llama2? about llmfarm HOT 5 CLOSED

FYI: q2_k is not working in my tests, I got bad_alloc crash or unexpectedly reached end of file (enabling Extended Virtual Addressing). q4_0 works fine.

I also trying integrate llama.cpp to my iOS apps (latest commit), the problem is looks same with here.

You may have compiled llmfarm_core with -DGGML_QKK_64 in Package.swift. You need to rebuild the project without this flag, or requantize the models for QKK_64.

from llmfarm.

if someone makes a ggml version

from llmfarm.

This models works fine.

from llmfarm.

FYI: q2_k is not working in my tests, I got bad_alloc crash or unexpectedly reached end of file (enabling Extended Virtual Addressing). q4_0 works fine.

I also trying integrate llama.cpp to my iOS apps (latest commit), the problem is looks same with here.

from llmfarm.

FYI: q2_k is not working in my tests, I got bad_alloc crash or unexpectedly reached end of file (enabling Extended Virtual Addressing). q4_0 works fine.

I also trying integrate llama.cpp to my iOS apps (latest commit), the problem is looks same with here.

In the current version of llmfarm, I use this commit. q2_k works fine on iphone(without metal) and Mac.

from llmfarm.