Problems on linux about gpt-llama.cpp HOT 10 OPEN

did you properly set up your path to your model in your auth? how are you sending your request?

from gpt-llama.cpp.

Yes, I believe so. I am testing with curl.

curl --location --request POST 'http://localhost:8080/v1/chat/completions' --header 'Authorization: Bearer /sol/ll
m/LLaMA/7B/ggml-model-q4_0.bin' --header 'Content-Type: application/json' --data-raw '{
 "model": "gpt-3.5-turbo",
   "messages": [
      {
         "role": "system",
         "content": "You are ChatGPT, a helpful assistant developed by OpenAI."
      },
      {
         "role": "user",
         "content": "How are you doing today?"
      }
   ]
}'
curl: (52) Empty reply from server

and

$ ls /sol/llm/LLaMA/7B/ggml-model-q4_0.bin
/sol/llm/LLaMA/7B/ggml-model-q4_0.bin

from gpt-llama.cpp.

Regarding the port 443 being reserved for system processes, you could do the following:

sudo setcap cap_net_bind_service=+ep `readlink -f \`which node\``

and then
npm start

from gpt-llama.cpp.

@atisharma can you try the new ./test-installation.sh script to help validate if you're giving it the right path?

from gpt-llama.cpp.

#16
Should fix the whole Port issue. I have similar changes in a dev branch and it works fine

from gpt-llama.cpp.

Still curl: (52) Empty reply from server.
Your test script also hardcodes port 443 by the way.

from gpt-llama.cpp.

good point @atisharma , let me update the script to be more flexible. and @th-neu , i'll take a quick look at your PR.

for now, @atisharma , what do u get if you change the script to your port? does the curl command return the proper value?

from gpt-llama.cpp.

No, the empty reply was with using 8080 in both index.js and the test script.

from gpt-llama.cpp.

i just updated the test script and added support for port – PORT=8080 npm run start should do the trick. are you still running into the same issue?

from gpt-llama.cpp.

@atisharma are you still running into the same issue?

from gpt-llama.cpp.