Is there any way to make this work with a local model? about starcoderex HOT 8 CLOSED

Hi. Yep, i think it possible right now, if you will use local API, but i don't know how the local model works and does the local model has a API for work with them. Can you give more information about installing and running local model for debug and develop?

from starcoderex.

I mean, if the local model have web API like reference API from site, you can use them changing URL of API in extension settings to localhost or something like that.

from starcoderex.

I am running it locally with auto gptq and 10gig of GPU ram. Bits quite fast. But I am now looking for how to set up the api so that the plugin can prompt it correctly

from starcoderex.

I readed docs for StarCoder and i think, for work with plugin necessary write driver. I can write it but i need to install StarCoder on computer. So, wait changes in that issue branch.

from starcoderex.

I just found the server side code for the plugin. https://github.com/LucienShui/huggingface-vscode-endpoint-server. Would this work?

from starcoderex.

Yes, i think it must work. Please, after test tell me results ;)

from starcoderex.

I think it works. It was a bit buggy but it did return some responses. I think it's an issue with the version of remote vscode I am using.

from starcoderex.

Thanks for feedback!

from starcoderex.