imaurer / awesome-llm-json Goto Github PK

In my search for easy LLM inference on PC, I stumbled on TextSynth Server, distributed as shareware for CPU inference. It provides easy inference with a webgui on Linux and Windows, supporting a bunch of models, some of them quantized.

So far, it's the easiest way I found to run an LLM locally, but since it's not open source, I don't know if it can be added.

If you have something interesting to share, just go ahead and add your link following the pattern in the README and submit a PR.

I will look it over and merge it in if it's on-topic and interesting.

If in a few weeks, there isn't any new activity or traction I will remove it.

Thanks...
Ian

https://github.com/hyperonym/basaran "Basaran is an open-source alternative to the OpenAI text completion API. It provides a compatible streaming API for your Hugging Face Transformers-based text generation models."

In short, Basaran is a drop-in replacement for api.openai.com, for local models.

It lets you use any local LLM in virtually any service or project made for GPT-3 or GPT-4.

hi @imaurer , thanks for sharing resource with us. I wrote a clo voice chat bot https://github.com/vra/talkGPT4All
, is it okay to add it to this repo? thanks