flatlinerdoa / perceptivepyro Goto Github PK

Add Github label: Torchsharp

Proposal

Incorporate the ability to load GPT2 models (and others in future) from the GGUF format as defined here:
https://github.com/ggerganov/ggml/blob/master/docs/gguf.md

There is a fairly approachable library written in TypeScript that could be a good basis:
https://github.com/ahoylabs/gguf.js

Benefits

• Allows loading of models from a single file.
• Easy extensibility for new model support such as Llama2 models in the future.

Proposal

• Split the tests and the usable models into separate projects so that a Nuget Package can be published (making this more accessible)
• Add automated build and publish.

Benefits

• Allow others to adopt the package once it becomes more than an experimental playground.