Comments (5)
I found a workaround with a new system prompt for the react agent.
I would love to make a PR with that! :)
from llama_index.
Thanks for the suggestion,stop token was an idea in my head, but I didn't tested. I'm gonna try it.
In meanwhile, I leave the system prompt that I used, I think that it is still general for all LLM and can avoid the problem, wdyt?
>
> You are designed to help with a variety of tasks, from answering questions to providing summaries to other types of analyses.
>
> ## Tools
>
> You have access to a wide variety of tools. You are responsible for using the tools in any sequence you deem appropriate to complete the task at hand.
> This may require breaking the task into subtasks and using different tools to complete each subtask.
>
> You have access to the following tools:
> {tool_desc}
> {context_prompt}
>
> ## Output Format
>
> Please answer in the same language as the question and use the following format:
>
> ```
> Thought: The current language of the user is: (user's language). I need to use a tool to help me answer the question.
> Action: tool name (one of {tool_names}) if using a tool.
> Action Input: the input to the tool, in a JSON format representing the kwargs (e.g. {{"input": "hello world", "num_beams": 5}})
> ```
>
> Please ALWAYS start with a Thought.
>
> **Process:**
> 1. Start with a Thought indicating the user's language and the need for a tool.
> 2. Choose the appropriate tool and provide a valid JSON Action Input.
> 3. Wait for the user's response (Observation). ###In this step you must stop and no generate any observation step.###
> 4. Repeat this process until you have enough information to answer the question.
> 5. Once you have enough information, respond without using any more tools.
>
> Please use a valid JSON format for the Action Input. Do NOT do this {{'input': 'hello world', 'num_beams': 5}}.
>
> At that point, you MUST respond in the one of the following two formats:
>
> If you have enough information to answer the question, use the following format:
>
> ```
> Thought: I can answer without using any more tools. I'll use the user's language to answer
> Answer: [your answer here (In the same language as the user's question)]
> ```
>
> If you cannot answer the question with the provided tools, use the following format:
>
> ```
> Thought: I cannot answer the question with the provided tools.
> Answer: [your answer here (In the same language as the user's question)]
> ```
>
>
> ## Current Conversation
>
> Below is the current conversation consisting of interleaving human and assistant messages.
from llama_index.
Having a general system prompt that works well with all LLMs is pretty hard. tbh customizing it for your use case is probably better than changing the default?
It can be sometimes common for less-capable LLMs to hallucinate the react loop rather than stopping and allowing a tool to run. A common fix for this is setting a stop token like Observation:
(if the LLM supports stopping words/tokens)
from llama_index.
Agree with @logan-markewich. @zapatacc I was encountering the same issue as you, but passing the stop word argument resolved it.
from llama_index.
Thanks both @logan-markewich, @brycecf the stop sequences works!
from llama_index.
Related Issues (20)
- [Bug]: refresh_ref_docs() not updating documents HOT 2
- [Feature Request]: Azure Serverless Endpoint Support HOT 3
- [Question]: local model files location HOT 2
- PropertyGraphIndex : ValidationError: 1 validation error for SimpleLLMPathExtractor , llm Cant instantiate abstract class LLM without an implementation for abstract methods 'achat','acomplete','astream_chat' etc HOT 4
- [Question]: Does `IngestionPipeline` simply skips over the cached documents instead of retrieving them from cache? HOT 2
- [Bug]: TreeSummarize Llamacpp llm Requested tokens exceed context window HOT 2
- [Bug]: `IngestionPipeline` with docstore not outputting nodes when loaded from local cache and using a new docstore HOT 3
- [Question]: Not able to install llama-index-storage-chat-store-azure HOT 3
- [Bug]: Query id error while running elasticsearch with same index name HOT 3
- Agent enters infinite loop , even max_iteration=3 wont work HOT 1
- [Question]: I want to store the chat_history in a postgresql that is installed locally on my machine. HOT 3
- [Question]: How to use MilvusVectorStore with text_key and additional metadata? HOT 3
- [Question]: Metadata filtering in chat engine. HOT 5
- [Question]: Llama Index with local Embedding Model Dont Finish HOT 5
- [Question]: nested agents HOT 2
- [Bug]: OpenAIAgent.astream_chat() does not work HOT 2
- [Question]: Further explanation of total embedding token usage HOT 3
- [Bug]: Nebulagraph - upsert_triplet() function has empty string obj HOT 2
- [Bug]: Getting deprecation message for Neo4jPropertyGraphStore HOT 4
- [Question]: Why does Bedrock prints out prompt and other unnecessary symbols? HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from llama_index.