Error: SyntaxError: Unexpected non-whitespace character after JSON at position 230 about obsidian-local-llm HOT 2 CLOSED

Hey @TehNomad,

Thanks for giving me a heads-up about this problem. Could you provide some additional info?

1. What LLM model are you using?
2. What version of langchain are you on?
3. Are specific prompts causing the issue, or are they all?

The more details, the better :)
Thanks!

from obsidian-local-llm.

@zatevakhin

I know this has been closed since there weren't any other details but I'm seeing the same thing today. I'm using mixtral 8x7b, and it was every prompt tested. I wasn't using llama-cpp server but instead i'm using oobabooga's text-generation-webui and their openai compatible api. It starts decoding and working fine as the response streams in but the response is built, but then fails at last token being generated for some reason.

If I disable "typewriter" mode so that it doesn't use the streaming api then it works completely fine.

I've captured the entire api call made when it fails:

{"prompt":"\n\n### Instructions: Let's write a story about a cat named Buzz \"Light-year\" Aldrin\nCome up with 5 story topics\n\n### Response:\n","stop":["###"],"max_tokens":256,"temperature":0.1,"repeat_penalty":1.1,"top_p":0.95,"top_k":40,"stream":true}

and the response from the server (as it was streamed, captured via wireshark), attached as a file because it's gigantic.
data_stream.txt

from obsidian-local-llm.