Requests use 7B model when only 65B model is installed about dalai HOT 8 OPEN

As a workaround just change the default here
https://github.com/cocktailpeanut/dalai/blob/main/bin/web/views/index.ejs#L115

from dalai.

I am garbage at node.js but if it looks like the query is incorrect. Do you know how to change the query?

Server running on http://localhost:3000/

query: { method: 'installed' }
modelsPath /home/xdoestech/llama.cpp/models
{ modelFolders: [ '13B', '30B', '65B', '7B' ] }
exists 65B
query: {
seed: -1,
threads: 4,
n_predict: 1000,
model: '7B', THIS SHOULD BE 65B
top_k: 40,
top_p: 0.9,
temp: 0.8,
repeat_last_n: 64,
repeat_penalty: 1.3,
models: [ '65B' ],
prompt: 'the fuck'
}

from dalai.

ok so this runs in the terminal but does not run in the webUI... I know am missing something basic here

const Dalai = require('dalai');

new Dalai().request({
model: "65B",
prompt: "The following is a conversation between a boy and a girl:",
}, (token) => {
process.stdout.write(token)
})

new Dalai().serve(3000)

from dalai.

I am garbage at node.js but if it looks like the query is incorrect. Do you know how to change the query?

Server running on http://localhost:3000/

query: { method: 'installed' }
modelsPath /home/xdoestech/llama.cpp/models
{ modelFolders: [ '13B', '30B', '65B', '7B' ] }
exists 65B
query: {
seed: -1,
threads: 4,
n_predict: 1000,
model: '7B', THIS SHOULD BE 65B
top_k: 40,
top_p: 0.9,
temp: 0.8,
repeat_last_n: 64,
repeat_penalty: 1.3,
models: [ '65B' ],
prompt: 'the fuck'
}

I can confirm that this is what my queries look like as well.
A quick fix is to simply also install the 7B model. I can confirm that after I installed the 7B model, and select 65B from the web ui the queries then come through successfully.

Seems like there is some form of bug where if there is only one model installed it assumes that the model is 7B.

Unfortunately I haven't had time to dig through the code and figure out why this is happening or how to fix the query

from dalai.

i wonder if making a symlink to the 30B/65B/etc folder would work instead? haven't tried it myself but as long as it's not expecting an exact match of the model's files it might be worth a shot as a temporary workaround at least

from dalai.

The issue is the website defaults to 7B and only updates the value when you have a change event on the drop down. Since there's only one item in the drop down a change event will never fire and never update the model.

from dalai.

No the code I posted above works just the webui does not work with it. There are instructions to make the webui but I do not have time to learn socket.io until this weekend. The api works try the code above.

from dalai.

The issue is the website defaults to 7B and only updates the value when you have a change event on the drop down. Since there's only one item in the drop down a change event will never fire and never update the model.

Do you know how to access the code to change this? Ive looked throught the github and dont see anything (I have no idea what I am doing however). I have no storage for another model at this point

from dalai.