categorize internal jobs and expose a mechanism for the user to specify/override what model/api gets used for different categories about autopr HOT 3 CLOSED

Currently the model config option applies to all LLM calls. All agents should have their own model, as well as other config options. I'll open a PR implementing the refactor soon.

from autopr.

Sorry for the long pause, I've implemented a refactor that allows agents to have their own config options, but haven't yet implemented picking different models for different actions. This will come with a refactor into an "action choice engine".

I did some testing, and I experienced a pretty bad performance drop with gpt-3.5-turbo, even for parsing steps. Although, earlier this week we changed how we were specifying instructions in guardrails, and it seems to greatly improve gpt-3.5-turbo performance; I'll try again to see if we can offload parsing steps to it.

from autopr.

yeah forget gpt-3.5-turbo. I think it's just too unreliable for code generation. may as well focus on using whatever tools are most likely to work rather than wrestling with 3.5

from autopr.