zotero / stream-server Goto Github PK

Zotero Streaming API Server

License: GNU Affero General Public License v3.0

JavaScript 100.00%

stream-server's Issues

Check for response from keepalive

We're sending pings to the client, but we're not checking for the response. It's possible that this is causing us to not properly detect broken connections, which would presumably cause a memory leak. We should check for a pong as explained in the ws docs.

Faster shutdown

Right now a restart or shutdown triggers lots of activity as all connections are deregistered. This can take a while when there are lots of connections, and the delay means that health checks can fail, causing the instance to be taken out of service.

We should do the bare minimum to stop accepting new connections and close all connections cleanly before shutting down.

Handle errors from API properly

request() doesn't throw on a non-200 status codes, so our handling code in zotero_api.js is wrong.

Error deleting public/global topics

A deleteSubscriptions request for a public or global topic without an API key results in an 'apiKey' not provided error. This was an existing bug before global topics, but it prevents global topics from being deleted.

Limit simultaneous connections to API

So that we don't DoS ourselves with key requests when a server restarts

Switch from `uws` to `ws`

See WIP code in the https://github.com/zotero/stream-server/tree/uws-to-ws branch

Debug segfaults in streamserver

Thanks for the great work to build Zotero. I'm trying to set up an air-gapped version and while all other services seem to run, the streamserver dies occasionally and sync with a rebuilt Zotero client does not work (endless syncing).

The log says

streamserver-1   | [03/Apr/2024:09:06:30 +0000] [1 connection, 5 subscriptions]
streamserver-1   | [03/Apr/2024:09:06:40 +0000] [1 connection, 5 subscriptions]
streamserver-1   | [03/Apr/2024:09:06:50 +0000] [1 connection, 5 subscriptions]
streamserver-1   | [03/Apr/2024:09:07:00 +0000] [1 connection, 5 subscriptions]
streamserver-1   | [03/Apr/2024:09:07:04 +0000] [10.5.5.1] WebSocket connection was closed
streamserver-1   | [03/Apr/2024:09:07:04 +0000] [10.5.5.1] Removing subscription for /users/1
streamserver-1   | [03/Apr/2024:09:07:04 +0000] [10.5.5.1] Removing subscription for /users/1/publications
streamserver-1   | [03/Apr/2024:09:07:04 +0000] [10.5.5.1] Removing subscription for /groups/1
streamserver-1   | [03/Apr/2024:09:07:04 +0000] [10.5.5.1] Removing subscription for styles
streamserver-1   | [03/Apr/2024:09:07:04 +0000] [10.5.5.1] Removing subscription for translators
streamserver-1   | [03/Apr/2024:09:07:04 +0000] [10.5.5.1] Closing connection
streamserver-1   | Segmentation fault
streamserver-1   | npm ERR! code ELIFECYCLE
streamserver-1   | npm ERR! errno 139
streamserver-1   | npm ERR! [email protected] start: `node index.js`
streamserver-1   | npm ERR! Exit status 139
streamserver-1   | npm ERR!
streamserver-1   | npm ERR! Failed at the [email protected] start script.
streamserver-1   | npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
streamserver-1   |
streamserver-1   | npm ERR! A complete log of this run can be found in:
streamserver-1   | npm ERR!     /root/.npm/_logs/2024-04-03T09_07_04_144Z-debug.log

And the mentioned debug log says

0 info it worked if it ends with ok
1 verbose cli [ '/usr/local/bin/node', '/usr/local/bin/npm', 'start' ]
2 info using [email protected]
3 info using [email protected]
4 verbose run-script [ 'prestart', 'start', 'poststart' ]
5 info lifecycle [email protected]~prestart: [email protected]
6 info lifecycle [email protected]~start: [email protected]
7 verbose lifecycle [email protected]~start: unsafe-perm in lifecycle true
8 verbose lifecycle [email protected]~start: PATH: /usr/local/lib/node_modules/npm/node_modules/npm-lifecycle/node-gyp-bin:/usr/src/app/node_modules/.bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
9 verbose lifecycle [email protected]~start: CWD: /usr/src/app
10 silly lifecycle [email protected]~start: Args: [ '-c', 'node index.js' ]
11 silly lifecycle [email protected]~start: Returned: code: 139  signal: null
12 info lifecycle [email protected]~start: Failed to exec start script
13 verbose stack Error: [email protected] start: `node index.js`
13 verbose stack Exit status 139
13 verbose stack     at EventEmitter.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/npm-lifecycle/index.js:285:16)
13 verbose stack     at emitTwo (events.js:126:13)
13 verbose stack     at EventEmitter.emit (events.js:214:7)
13 verbose stack     at ChildProcess.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/npm-lifecycle/lib/spawn.js:55:14)
13 verbose stack     at emitTwo (events.js:126:13)
13 verbose stack     at ChildProcess.emit (events.js:214:7)
13 verbose stack     at maybeClose (internal/child_process.js:925:16)
13 verbose stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:209:5)
14 verbose pkgid [email protected]
15 verbose cwd /usr/src/app
16 verbose Linux 5.15.133.1-microsoft-standard-WSL2
17 verbose argv "/usr/local/bin/node" "/usr/local/bin/npm" "start"
18 verbose node v8.9.4
19 verbose npm  v5.6.0
20 error code ELIFECYCLE
21 error errno 139
22 error [email protected] start: `node index.js`
22 error Exit status 139
23 error Failed at the [email protected] start script.
23 error This is probably not a problem with npm. There is likely additional logging output above.
24 verbose exit [ 139, true ]

I'm running it on a local Docker instance in WSL2.

What means error status 139, can I debug that a little bit more?

Thanks!

Add StatsD gauge with current connections

On connect and disconnect

We'll need a JS-based StatsD library (and we might need to upgrade our server StatsD version to actually receive these, though that doesn't matter for this).

I assume we'll want to base this on #2.

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.