Template format for local models about wingman HOT 6 OPEN

I totally agree. This is another one of the things handled by the lib I use for request/response formatting. Specifically, this happens here:

https://github.com/transitive-bullshit/chatgpt-api/blob/main/src/chatgpt-api.ts#L395

Maybe it's time to roll my own solution for this, so that a prompt formatting can be explicitly defined. I'd like to implement this, but need some time to think about how best to do so.

from wingman.

I studied this library recently when trying to implement OpenAi support in my server project. It is easy to replace: example code:

• With the library
• Without the library: using just the eventsource-parser library (used internaly in the chatgpt api lib)

from wingman.

yes it looks good. I would define a template in a very simple maner. An Alpaca one:

const template = `{system}

### Instruction: fix this invalid json:

'''json
  {prompt}
'''

### Response: (answer in valid json)
`

const system = "You are a javascript code specialist. Below is an instruction that describes a task. Write a response that appropriately completes the request";

const _prompt = '{"a": 1,}';

const finalPrompt = template.replace("{system}", system).replace("{prompt}", _prompt)

where {system} is actually covered by systemMessageTemplate in Wingman

from wingman.

Nice, seems easy enough. I should have time to get this done this evening.

How would you expect to define a template? I think I'd probably surface this as an item in the wingman settings panel as e.g. wingman.openai.template, as well as allow it to be overridden by any given command. Does that seem okay to you?

from wingman.

Note about handling the sse request: the eventsource-parser lib is not really necessary: we can handle this manually using a regex, this is what I did in my frontend: example

from wingman.

About templates I just started a little library to manage them more efficiently an use prebuilt template formats: https://github.com/synw/modprompt

I got tired to copy and change text to different template formats depending on the models, so I wrote this to ease the process of using templates in Typescript. It might be useful for Wingman as well

from wingman.