[Question]: multiple-PDF with local LLM llama3 about llama_index HOT 3 OPEN

open-source llms are pretty bad in general at being agents right now. Also, Ollama is slow in general compared to other APIs.

• I want the query engine to perform only one iteration without fetching data again for LLM integration -- I don't know what this means tbh

from llama_index.

@logan-markewich thanks for helping answer the question.
I mean that, because my query engine performs somethings multiple observations to provide responses to the LLM, I want to control it so that it responds after the first observation. How can I achieve this?

Thanks.

from llama_index.

From what I can see, the implementation of the ollama wrapper instantiates an httpx instance each time it runs a query. This is where I'm seeing all of the time cost for responses.

from llama_index.