CPU t/s regression from GPT4All 2.6.1 -> 2.6.2 about gpt4all HOT 6 OPEN

I need some more information:

• What operating system do you have?
• What model of GPU are you currently using?
• What device is being used in GPT4All 2.6.1 and GPT4All 2.7.3? It will tell you this next to the token count. If one is using the GPU and the other is not (possibly due to OOM), that would explain the speed difference.

There were significant changes to the GPU backend in v2.6.2.

from gpt4all.

Windows 10 in a VM.
I'm using NVIDIA GT1030.
I never noticed the device beside the token count before but in both cases it only says CPU. Once again, I got 3.8 t/s on 2.6.1 and 1.6 on 2.7.3. Exact same setup.

So my GPU was never utilized, interesting.

from gpt4all.

Thanks for the reply. To help narrow down the issue further, could you try a few versions between 2.6.1 and 2.7.3 to find which specific version caused the slowdown? Here are links to the versions in between:

• https://github.com/nomic-ai/gpt4all/releases/tag/v2.6.2
• https://github.com/nomic-ai/gpt4all/releases/tag/v2.7.0
• https://github.com/nomic-ai/gpt4all/releases/tag/v2.7.1
• https://github.com/nomic-ai/gpt4all/releases/tag/v2.7.2

from gpt4all.

Good catch! 2.6.2 appears to be the culprit. 3.2 t/s on 2.6.1, 1.1 t/s on 2.6.2. The output was worse too but that's just my subjective opinion.

from gpt4all.

Windows 10 in a VM.

By the way - GPT4All will likely not be able to see your GPU inside of a virtual machine, unless you are a real power user doing PCIe passthrough. It most likely does not appear as an option under Settings > Application > Device.

from gpt4all.

Windows 10 in a VM.

By the way - GPT4All will likely not be able to see your GPU inside of a virtual machine, unless you are a real power user doing PCIe passthrough. It most likely does not appear as an option under Settings > Application > Device.

Hey, have you figured out the cause of the regression yet?

My Windows 10 box is not ready, but when I tested it on Windows 10 on the same hardware (not a VM) I got 4.9 t/s, not a huge improvement. However, this was while I was using the onboard GPU so no idea what a GT1030 would do.

from gpt4all.