Eval error often appear when using llama 2 about llmfarm HOT 10 CLOSED

try increasing the context size to 2048

from llmfarm.

try increasing the context size to 2048

Thanks! Now that error disappeared, but the new problem is that the llama 2 doesn't finished the answer and apparently aborted at some point. Here are examples:

from llmfarm.

That's weird. The dialog is terminated without an error only if llama generates an end-of-session token. If you continue the conversation, will it continue normally or with an error?

from llmfarm.

It just continue without showing obvious error message, other than the conversation ends early.

from llmfarm.

Tell me, did the update solve the problem?

from llmfarm.

Just finished the update. The problem apparently still exists as llama2 still ends its reply early.

from llmfarm.

Can you send me the text of the request? I'll try it myself.

from llmfarm.

Yes. Here is my prompt:

Give me a Seoul travel plan

from llmfarm.

I think I found what the problem was. Tell me, did the update solve the problem?

from llmfarm.

I think I found what the problem was. Tell me, did the update solve the problem?

It fixed!

from llmfarm.