Comments (4)
I think switching the system prompt half-way through a conversation is an edge case. You already addressed it through your last system prompt (which could also be an injected user message btw.).
Given that the recommend default is to have the system prompt first, and your situation is an edge case with a workaround, I'll close this issue.
from ai.
Thanks for the quick response @lgrammel
Yeah I have checked the ai/rsc package and apparently, you are injecting the system prompt as a user
message:
case 'prompt': {
languageModelMessages.push({
role: 'user',
content: [{ type: 'text', text: prompt.prompt }],
});
break;
}
Do you know why it is recommended to have the system prompt first? As written it seems to me to lose attention if the message array gets too long. It could even get cropped if we go above the max_tokens for an API request.
Closing is fine for me.
from ai.
a) no we are not, this is the prompt and not the system message
b) it's recommend to have the system prompt first because that's how the language models are trained. they get lots of examples where the system prompt (inside special tokens) comes at the beginning.
that being said, i understand your issue and ran into it myself. adding a system or user message as the final message can help (i've done this as well), but it is up to you to add it, and it can lead to issues with e.g. tool calls and tool results.
from ai.
Okay, thanks for the response and the feedback.
I will try my best to bend it towards my needs then :) I haven't tried with tools calling yet.
from ai.
Related Issues (20)
- Implicit `Buffer` usage fails in Cloudflare Workers HOT 1
- Invalid JSON payload received (Google Provider). HOT 12
- Unable to get 8192 tokens back from claude using streamText() HOT 4
- Composing "streamableUI" elements throws "TypeError: Cannot read properties of null (reading 'children')" HOT 1
- When using >0 maxToolRoundTrips onFinish gets called prematurely
- Unknown name minItems - Google AI SDK and OpenAI SDK HOT 1
- Token Usage with LiteLLM HOT 2
- More Gemini safety categories HOT 1
- Error is thrown while using `useChat` in client component in 3.3 HOT 3
- Using a schema with useChat HOT 7
- On bedrock can't set tokens for claude-sonnet-3-5 > 4096 even though the model supports it HOT 1
- Module not found: Can't resolve '@google-cloud/vertexai' HOT 1
- convertToCoreMessages does not convert Message[] to CoreMessage[] HOT 1
- Bedrock client does not work natively with profiles, instance roles, container roles, etc. HOT 1
- Add Ai21Lab Provider @ai-sdk/ai21 for Jamba 1.5 Models HOT 1
- streamObject breaks with undefined (Mistral provider) HOT 4
- OpenAI Model tries to call unavailable tool `multi_tool_use.parallel` HOT 2
- Add JigsawStack SDK as part of the AI SDK HOT 3
- Type error when using model map HOT 2
- Adding .optional() or .partial() to the parent zod schema object with OpenAIs Structured output leads to error HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ai.