Comments (13)
@zwilch Please try using
"host_permissions":[
"http://127.0.0.1/"
]
like this and see if that allows you to avoid CORS entirely.
Port numbers are not allowed in host_permissions. I'm suspecting the extension is not picking up the setting correctly because the pattern is malformed.
see
https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/Match_patterns#sect2
and https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/manifest.json/host_permissions#format
from gpt4all.
Alright, some initial notes on this:
- One thing I've found strange is that CORS should not require any preflight with the following:
GET
,POST
,HEAD
(a so-called "simple request", as explained in [1][2][4]) - You're using
fetch()
whereas my example uses a plain oldXMLHttpRequest
. I don't really know aboutfetch()
but it should do the right thing with regards to CORS, according to the docs [3].- Although seeing as that doesn't work in your case, you could probably work the problem by falling back to the latter; as long as GPT4All doesn't implement OPTIONS, I mean.
- Seeing as this should, in fact, be a simple request, something else must be wrong.
- => I think it's because
Content-Type
isn't any of the three "safe" ones, as defined by the new Fetch Standard spec (application/x-www-form-urlencoded
,multipart/form-data
, ortext/plain
), as opposed to the older CORS spec. - Therefore,
fetch()
wants to do a preflight regardless of the method beingGET
,POST
, orHEAD
.
- => I think it's because
"mode": "no-cors"
in your example should never work in afetch()
. That is as intended, because CORS is definitely required when using a local file, or I guess in your case, a browser extension.
So as mentioned, you could probably get around it through XmlHttpRequest
s, if there aren't any restrictions on that in an extension. But seeing as the server's output is application/json
, it should support that OPTIONS request as per the spec. The proper response for that should probably look something like in [1].
I'll further edit this comment after investigating some more
[1]: https://developer.mozilla.org/en-US/docs/Glossary/Preflight_request
[2]: https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS#simple_requests
[3]: https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API/Using_Fetch#supplying_request_options
[4]: https://developer.mozilla.org/en-US/docs/Web/API/fetch#exceptions
from gpt4all.
Thank you
[2]: https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS#simple_requests
helped me a lot to do a simple_request.
Seems you are right, with a "simple_request" its possible to access the json answhere.
Possible Solution with simple_request
manifest.json
{
"manifest_version": 3,
"name": "GPT4ALL Localhost Fetch Extension",
"version": "1.0",
"description": "Using fetch to get data from gpt4all API on http://localhost:4891/v1",
"background": {
"scripts": ["background.js"]
},
"host_permissions":[
"http://127.0.0.1/",
"http://localhost/"
]
}
background.js
// background.js
// This event listener will be triggered when the extension is installed or updated
chrome.runtime.onInstalled.addListener(() => {
console.log('Extension installed or updated.');
main(); // start main()
});
async function gpt4allAPIXMLHttpRequest(){
const json_completion = JSON.stringify(
{stream:false,
temperature:0.6,
max_tokens:100,
messages:[{role:"user",content:"Hello."}],
model: 'Nous Hermes 2 Mistral DPO'
}
);
const http = new XMLHttpRequest()
http.open('POST', 'http://127.0.0.1:4891/v1/chat/completions')
http.setRequestHeader('Content-type', 'text/plain')
http.send(json_completion) // Make sure to stringify
http.onload = function() {
// Do whatever with response
console.log(http.responseText)
}
}
async function gpt4allAPIfetch(CORSmode){
console.log("call gpt4allAPI("+CORSmode+")");
const json_completion = JSON.stringify(
{stream:false,
temperatur:0.6,
max_tokens:100,
messages:[{role:"user",content:"Hello."}],
model: 'Nous Hermes 2 Mistral DPO'
}
);
var options={
keepalive: true,
method: "POST",
headers: {
Accept: 'application/json',
'Content-Type': 'text/plain',
},
body:json_completion
};
if ( CORSmode) options.mode = CORSmode;
const completions = await fetch("http://127.0.0.1:4891/v1/chat/completions",options);
console.log("API.js completions",completions);
var completionjson=null; //var declaration
try{
completionjson = await completions.json();
console.log("API.js completionjson=",completionjson);
}catch(err){console.log("switch null err=",err);}
}//end gpt4allAPIfetch()
async function main(){
const models = await fetch("http://127.0.0.1:4891/v1/models");
const modelsData = await models.json();
console.log("modelsData",modelsData);
console.log("main gpt4allAPIXMLHttpRequest()");
try{await gpt4allAPIXMLHttpRequest();}catch(e){console.log(e);}
console.log("main null");
try{await gpt4allAPIfetch(null);}catch(e){console.log(e);}
console.log("main cors");
try{await gpt4allAPIfetch("cors");}catch(e){console.log(e);}
}//end main()
CORS Simple Request Fetch/XMLHttpRequest
from gpt4all.
with your Ressource [2} specifying a "CORS Simple Request" I had success with Fetch and with XMLHttpRequest.
I have editing it above within a working Firefox Extension Example.
You see no OPTION is send by Browser Firefox more.
You see the POST get "OK 200"
In Console you can see the fetch and XMLHttpRequest answhere from server.
With this Example to implementing CORS Simple Request we should be able to program Firefox/Thunderbird Extension to use GPT4ALL locally together with this Applications (FF/Thunderbird).
Thank you so much, may it implements in future also the CORS Preflight, so no developer does strugling with the unsuccessfully CORS Preflight.
from gpt4all.
I have not tested the following thoroughly, but it might be enough to make a browser's preflight request happy. I don't have the time to do more tests right now, though, so I'll leave this here for the moment:
// server.cpp: Server::start()
m_server->route("<arg>", QHttpServerRequest::Method::Options, // <arg> with QUrl is basically a wildcard match
[](const QUrl &anyPath, const QHttpServerRequest &request, QHttpServerResponder &&responder) {
auto headers = request.headers();
// reuse and allow request's "Origin" and "Access-Control-Request-Headers" headers:
QByteArray allowOrigin = "";
QByteArray allowRequestHeaders = "";
for (auto const& header: headers) {
if (header.first == "Origin") {
allowOrigin = header.second;
} else if (header.first == "Access-Control-Request-Headers") {
allowRequestHeaders = header.second;
}
}
// send OPTIONS preflight response for CORS:
responder.write(
"", // empty body, only headers matter
QHttpServerResponder::HeaderList {
std::make_pair("Connection", "Keep-Alive"),
std::make_pair("Access-Control-Allow-Origin", allowOrigin),
std::make_pair("Access-Control-Allow-Methods", "GET, POST, OPTIONS"),
std::make_pair("Access-Control-Allow-Headers", allowRequestHeaders),
std::make_pair("Access-Control-Max-Age", "86400"),
},
QHttpServerResponder::StatusCode::NoContent
);
}
);
from gpt4all.
You've already commented in #1008, which enables CORS (or at least part of it). Said pull request includes a simple HTML file with a bit of JavaScript. I've just tested that again, and it still works.
So my questions are:
- Did you try that HTML example? Does it not work for you for some reason?
- If you need something else, can you describe it in detail and especially how it differs from what's already there?
from gpt4all.
Hello cosmic-snow,
I have tried to make a simplified version for a firefox extension.
make a folder an put two file inside "manifest.json" and "background.js"
manifest.json
{
"manifest_version": 3,
"name": "GPT4ALL Localhost Fetch Extension",
"version": "1.0",
"description": "Using fetch to get data from gpt4all API on http://localhost:4891/v1",
"background": {
"scripts": ["background.js"]
},
"host_permissions":[
"http://127.0.0.1:4891/*",
"http://localhost:4891/*"
]
}
background.js
// background.js
// This event listener will be triggered when the extension is installed or updated
chrome.runtime.onInstalled.addListener(() => {
console.log('Extension installed or updated.');
main(); // start main()
});
// This event listener will be triggered when the extension is first installed
chrome.runtime.onStartup.addListener(() => {
console.log('Extension started.');
});
async function gpt4allAPI(CORSmode){
console.log("call gpt4allAPI("+CORSmode+")");
const json_completion = JSON.stringify(
{stream:false,
temperature:0.6,
max_tokens:100,
messages:[{role:"user",content:"Hello."}],
model: 'Nous Hermes 2 Mistral DPO'
}
);
const completions = await fetch("http://127.0.0.1:4891/v1/chat/completions",{
keepalive: true,
method: "POST",
mode: CORSmode,
headers: {
Accept: 'application/json',
'Content-Type': 'application/json',
'Access-Control-Allow-Origin': "*",
'Access-Control-Allow-Headers': "*",
"access-control-allow-methods":"GET, POST, OPTIONS",
"access-control-expose-headers":"*"
},
body:json_completion
});
console.log("API.js completions",completions);
var completionjson=null; //var declaration
switch(CORSmode){
case "no-cors":
console.log("switch() no-cors");
//completionjson = await completions.json();
//console.log("API.js completionjson=",completionjson);
break;
case "cors":
console.log("switch() cors");
completionjson = await completions.json(); //trows a error on no-cors to access server response
console.log("API.js completionjson=",completionjson);
break;
}//switch
}//end gpt4allAPI()
async function main(){
await gpt4allAPI("no-cors");
await gpt4allAPI("cors");
}//end main()
Load this Extension in Firefox
- open main menu and find "Add Ons and Themes"
- open AddOn Debugging tab (context menu behinde the wheel)
- in new Tab Debuging (or with about:debugging#/runtime/this-firefox)
- load a Temporary AddOn load (lgo to your folder and oad this manifest.json)
- with Inspector button you can inspect console and network from AddOn
- with reload it reloads and start again
What this AddOn do
- it connect to localhost:4891/v1/chat/completions
- and send as user "Hello" to API Endpoint of gpt4all (need to be started and enabled)
- first it use "no-corse"mode of fetch so it can connect without "CORS"
- the server will respons to "Hello" with specified model (can be changed in script)
- then the server finish the connection
- with "no-cors" the JavaScript can not access the response (security reason)
- second fetch start with "cors" mode
- fetch send a OPTION (CORS Preflight) to server, the server hast to respons to this
- the server do not response so CORS is failed
Networking Console
First is "no-corse" it can access the gpt4all API, but Javascript can not access the content.
the response from server is correct you can watch in wireshark, but in javascript the type is "opaque" see here => https://stackoverflow.com/questions/54896998/how-to-process-fetch-response-from-an-opaque-type
With CORS (Cross-Origin Resource Sharing), the browser sends a preflight OPTION request before making a POST request to another domain. However, it seems that you're not receiving the correct response for this preflight request, leading to a failed POST API request due to a CORS error.
JavaScript Console
Conclusion
Without proper correspondence regarding CORS (Cross-Origin Resource Sharing) preflight OPTIONS, no one can implement a browser extension or Thunderbird extension for using gpt4all.
from gpt4all.
Alright, I'll look at that example of yours later. But before I invest any time in that:
- Did you try that HTML example? Does it not work for you for some reason?
It's in the demo section of PR #1008.
from gpt4all.
insert in main() in background.js script
const models = await fetch("http://127.0.0.1:4891/v1/models");
const modelsData = await models.json();
console.log("modelsData",modelsData)
It is similary to #1008
it uses GET and not POST and the browser does not send OPTIONS CORS Prefligt to gpt4all server.cpp
GET Models is working.
GET Models Network Console
here is no OPTIONS send first from Browser and it get all models
before POST chat/completion is send, there is a OPTION send first from Browser, wich failes and does not get the right header back
from gpt4all.
Here the trace in wireshark the Browser get status 404 for asking first with OPTION the server
May it would help the server answheres with "OK" Status Code 200 to the OPTION question from Browser.
Wireshark Browser to gpt4all server.cpp
OPTIONS /v1/chat/completions HTTP/1.1
Host: 127.0.0.1:4891
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:124.0) Gecko/20100101 Firefox/124.0
Accept: /
Accept-Language: de,en-US;q=0.7,en;q=0.3
Accept-Encoding: gzip, deflate, br
Access-Control-Request-Method: POST
Access-Control-Request-Headers: access-control-allow-headers,access-control-allow-methods,access-control-allow-origin,access-control-expose-headers,content-type
Origin: moz-extension://9bbd4855-8087-4878-913c-dbd5499e292a
Connection: keep-alive
Sec-Fetch-Dest: empty
Sec-Fetch-Mode: cors
Sec-Fetch-Site: cross-site
Wrieshark Answhere gpt4all server.cpp Error 404
HTTP/1.1 404 Not Found
Status Code: 404
Access-Control-Allow-Origin: *
Content-Type: application/x-empty
Content-Length: 0
from gpt4all.
Yes, the OPTIONS method of HTTP is not implemented in the chat server. That's something which needs to be done separately. The Qt framework's HTTP server (which is used by this application) doesn't have a full-fledged web API framework so that would have to be done explicitely (see the code from here).
I just wanted to make sure that there isn't some other problem on your end and the basic CORS example works.
I haven't looked at/tried your example yet, but will do so sometime today.
from gpt4all.
-
I am looking into what it would take to respond with OPTIONS here, but it's a bit tricky. The link in your OP (nikhilm/qhttpserver#63) is not the same as what's in use here. You can look at the Qt implementation here, and I think it's just a mirror on GitHub, can't interact there.
-
What I meant was that it needs to be one of the "safe" accepted
Content-Type
headers -- but the server responds with an "unsafe"application/json
anyway. So implementing that OPTIONS is what should be done on a proper server. -
Looks like you got something working at least? I'll still take some time to see if something can be done with the server.
from gpt4all.
@iimez I have changed manifest.json to this:
@zwilch Please try using
"host_permissions":[ "http://127.0.0.1/" ]
see
https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/Match_patterns#sect2
and
https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/manifest.json/host_permissions#format
You are correct, this works perfectly! All POST requests from the browser now go through without the OPTION CORS preflight request before. This was a misconfiguration in "host_permissions" with no error related to manifest.json
.
Thank you very much for this hint.
from gpt4all.
Related Issues (20)
- Crashes when indexing RAG
- GPT4All version inconsistency in winget upgrade on Windows HOT 8
- [Feature] Support for Linux ARM64 HOT 1
- [Feature] Custom stopping word list HOT 1
- [Feature] GPU support for GPT-J HOT 2
- [Feature] Search via CTRL+F in references (context) for LocalDocs
- GPT-J models fail to load unless device is set to "CPU"
- [Feature] User-configurable stop sequences
- Documentation for build the project from source code HOT 1
- How come GLM-4-9b is not supported yet?
- Multithreaded app/concurrent generation requests - Thread 4 "python3" received signal SIGSEGV, Segmentation fault. HOT 4
- Error during installation process (gpt4all) ... missing d3dcompiler_47.dll HOT 1
- [Feature] Saving default model settings
- Not accepting ChatGTP-4 API key
- Answers containing code are unreadable with light theme
- Calling generate before opening a chat session (with a system prompt specified) consistently malfunctions
- [Feature] Add a callback for generation complete in streaming mode or some other signal like is_final_token
- The API server is not working
- [Feature] Windows ARM64 Support
- flatpak crashes on launch HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from gpt4all.