plaid / npmserve-server Goto Github PK

the raw node_modules directories take up a lot of disk space, we can delete these after building the archive to prevent servers from filling up so quickly.

the default express timeout is 120000 (2 minutes). This means that any install running over 2 minutes will fail. Short-term let's dial up the timeout.

cc @soroushjp

When we have a dependency like "foo": "0.0.x", npm might first install [email protected]. Then when a new version 0.0.2 is a released, it'd be nice if npmserve could detect a potential update and trigger a rebuild when the package is requested to make sure that all calls to /npm/install return the latest dependencies.

approach 1: npm shrinkwrap

1. first POST /npm/install builds with the latest dependencies and stores the node_modules.tar.gz. It could also generate an npm shrinkwrap file indicating what dependency versions were used.
2. second POST /npm/install generates a new shrinkwrap. If no changes, return cached node_modules.tar.gz. If there are changes, do a full rebuild and return the result.

approach 2: npm update

Whenever we receive a POST /npm/install for a cached package, run an npm update on the server and rebuild the archive.

approach 3: cron-based updates

Configure a recurring job to rebuild and re-archive all packages on the server. This is the least intelligent approach but the easiest to implement.

These are ideas, not something we should necessarily jump into building as it has a high cost and will slow down installs in the average case. let's gather data on whether this would be worth the complexity before implementing. Right now users can do this manually by calling npmserve remove and I suspect that might be good enough for most use cases.

s/conent-type/content-type

The screenshot shows some buttons which we removed. Let's update it with a new version.

investigate and enable end-to-end server tests in circle CI.
see endtoend/server.js.

Old builds on the server will eventually stop being downloaded. We should build a feature to expire and delete old build data to keep disk usage low. We should expose a web interface to configure expiration settings based on build date or last access date. Would be nice to have a way to configure the server to automatically run this job nightly.

The server errors out when a package with no dependencies is uploaded. This is because node_modules does not get created in the data directory, and then we try to tar an archive that isn't there.

repro:

1. create a package.json with no dependencies or devDependencies
2. submit the package via an npmserve install

expected: return a tarball which decompresses, yielding an empty node_modules
actual: server 500 error, server responds with an error.

Let's incorporate a DB to store build metadata, this is currently just reading files written to disk.

We should add authentication to the server. Could use to investigate what existing auth services might work well here (maybe NPM or github) so that npmserve can continue to be a natural extension of npm and wouldn't require new credentials (assuming that provides us with the right security guarantees)