pdf_bot on Mac (with Ollama) outputting gibberish about genai-stack HOT 7 CLOSED

Screenshot of the issue (incomprehensible, gibberish output)

from genai-stack.

Can you share the PDF you used?

from genai-stack.

Sure, the PDF files I tested with were:
https://delcode.delaware.gov/title8/Title8.pdf
https://constitutioncenter.org/media/files/constitution.pdf

from genai-stack.

What kind of machine do you have and do you use a local LLM (which one did you use) or OpenAI?
It could be that your GPU is too small for the local models?
Can you try with OpenAI?

from genai-stack.

The machine I have is Macbook Air M1 8GB, and I'm using the llama2 model with Ollama 0.1.23.
The local model does work properly when conversing with it directly using ollama run llama2, but are you suggesting that my system resources are insufficient to use the local model with embeddings?
Using the same stack, I tested with OpenAI on a separate Windows machine, which does work.

from genai-stack.

Tested it on Macbook pro M3 18GB and it worked with the same doc

from genai-stack.

Closing the issue as I can confirm that it works on an Ubuntu VM with 17GB of RAM and 4 cores / 8 threads (Ryzen 7 5000 series). A little bit of a shame it won't work with the base M1 configuration as I was hoping to see how it'd perform on ARM, but understandable.

from genai-stack.