local-npm / local-npm Goto Github PK
View Code? Open in Web Editor NEWLocal and offline-first npm mirror
License: Apache License 2.0
Local and offline-first npm mirror
License: Apache License 2.0
Especially for syncing local-npm between different computers ("protip: You can replicate from your friend's local-npm..."), it would be nice to increase the batch size, unless there's some other limitation that forbids this.
Or, batch size could be automatically increased if you detect that the remote address is on a local subnet (192.168.x.x / 10.x.x.x / that other local subnet that nobody uses)
Hey, I have a problem that after using local-npm some packages are always trying to get installed from http://127.0.0.1:5080 even though I have npm config get registry
= https://registry.npmjs.org/
. I've tried running npm cache clean
but no luck. What can I do?
Hi, I'm very interested in using local-npm but currently I'm facing the problem that our "master" npm registry is based on Sinopia which hosts bunch of private NPM packages and it's currently difficult to change that. Is it somehow possible to bind local-npm against Sinopia (or similar solutions that provide only a fullfatdb)? I'd love to use local-npm so that I can work also offline.
Thanks a lot in advance!
This is a bug in npm itself (apparently), but maybe we can use it to test some odd error cases?
To repro:
npm install RemoteTestService
Looks like the tarball referenced from http://registry.npmjs.org/RemoteTestService gives a 404: http://registry.npmjs.org/RemoteTestService/-/RemoteTestService-0.0.1.tgz
@remy For the record, I am still seeing this sometimes. Happened a moment ago when my laptop went into hibernate mode and then restarted; might be related. Still, since the checkpoint is stored on disk, there's no reason it should start re-replicating from scratch.
...
local-npm-0 Replicating skimdb, last_seq is: 35869 (18.74%)
local-npm-0 Replicating skimdb, last_seq is: 36071 (18.84%)
local-npm-0 Replicating skimdb, last_seq is: 36271 (18.95%)
local-npm-0 Replicating skimdb, last_seq is: 36472 (19.05%)
local-npm-0 Replicating skimdb, last_seq is: 36674 (19.16%)
local-npm-0 Replicating skimdb, last_seq is: 36875 (19.26%)
local-npm-0 Replicating skimdb, last_seq is: 37076 (19.37%)
local-npm-0 Replicating skimdb, last_seq is: 37276 (19.47%)
local-npm-0 Replicating skimdb, last_seq is: 37476 (19.58%)
local-npm-0 Replicating skimdb, last_seq is: 37677 (19.68%)
local-npm-0 Replicating skimdb, last_seq is: 37877 (19.78%)
local-npm-0 Replicating skimdb, last_seq is: 38077 (19.89%)
local-npm-0 Replicating skimdb, last_seq is: 38277 (19.99%)
local-npm-0 Replicating skimdb, last_seq is: 38478 (20.1%)
When you install lodash@^4.0.0
, then local-npm
will cache and store 4.0.0
for future npm install
s.
However, if 4.0.1
is published after you npm install
, then local-npm
will find out about it via the changes feed, but it will not download the actual tarballs.
So if you then go offline and try to npm install
again, it will fail because local-npm
knows that there is a newer version that matches your version range, but it hasn't been downloaded yet.
Question: should local-npm
optimistically cache every version of Lodash that gets published, simply because you once installed lodash@^4.0.0
?
Pros: more likely to work offline, less likely to cause unexpected errors, fits the "standard" pattern of caret- and tilde-versioning.
Cons: Potentially a lot of network traffic and a lot of wasted disk spaces for module versions that you might never use.
I tried to disconnect the internet connection and leave local-npm
server
running. Then, I do npm install
it gives me:
stream.js:74
throw er; // Unhandled stream error in pipe.
^
Error: getaddrinfo ENOTFOUND registry.npmjs.org registry.npmjs.org:443
at errnoException (dns.js:26:10)
at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:77:26)
And then the process exited.
The listed packages on package.json
is already stored locally. Because, before I run npm install
offline,
I installed everything first and local-npm
say this:
downloaded koa at version 2.0.0-alpha.3 and saved it locally
and some other things
It supposed to work offline, right?
Since the whole skimdb is replicated, you could have most of the functionality of the npm website itself served locally.
Would be great to be able to pass a directory in which the db and other files will be made, so that we do not have to invoke local-npm
in the directory where we want the files. This would help a lot with tools like forever.
Steps to reproduce:
npm cache clean
Saw this:
npm WARN deprecated [email protected]: v4.0.0 no longer shims Array#splice with ES5 behavior, makes shims non-enumerable when possible, and fixes a bug in ES3 browsers referencing String#indexOf/lastIndexOf before polyfilling it
npm ERR! fetch failed http://localhost:5080/tarballs/plist-with-patches/0.5.1.tgz
npm ERR! fetch failed http://localhost:5080/tarballs/plist-with-patches/0.5.1.tgz
npm WARN deprecated [email protected]: Please update to the latest object-keys
> [email protected] install /Users/nolan/workspace/pouchdb/node_modules/leveldown
> node-gyp rebuild
CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/db/builder.o
CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/db/db_impl.o
CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/db/db_iter.o
CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/db/filename.o
CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/db/dbformat.o
CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/db/log_reader.o
CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/db/log_writer.o
CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/db/memtable.o
CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/db/repair.o
CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/db/table_cache.o
CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/db/version_edit.o
CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/db/version_set.o
CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/db/write_batch.o
CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/helpers/memenv/memenv.o
CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/table/block.o
CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/table/block_builder.o
CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/table/filter_block.o
CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/table/format.o
CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/table/iterator.o
CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/table/merger.o
CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/table/table.o
CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/table/table_builder.o
CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/table/two_level_iterator.o
CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/util/arena.o
CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/util/bloom.o
CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/util/cache.o
CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/util/coding.o
CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/util/comparator.o
CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/util/crc32c.o
CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/util/env.o
CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/util/filter_policy.o
CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/util/hash.o
CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/util/logging.o
CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/util/options.o
CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/util/status.o
CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/port/port_posix.o
CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/util/env_posix.o
LIBTOOL-STATIC Release/leveldb.a
CXX(target) Release/obj.target/snappy/deps/snappy/snappy-1.1.1/snappy-sinksource.o
CXX(target) Release/obj.target/snappy/deps/snappy/snappy-1.1.1/snappy-stubs-internal.o
CXX(target) Release/obj.target/snappy/deps/snappy/snappy-1.1.1/snappy.o
LIBTOOL-STATIC Release/snappy.a
CXX(target) Release/obj.target/leveldown/src/batch.o
CXX(target) Release/obj.target/leveldown/src/batch_async.o
CXX(target) Release/obj.target/leveldown/src/database.o
CXX(target) Release/obj.target/leveldown/src/database_async.o
CXX(target) Release/obj.target/leveldown/src/iterator.o
CXX(target) Release/obj.target/leveldown/src/iterator_async.o
CXX(target) Release/obj.target/leveldown/src/leveldown.o
CXX(target) Release/obj.target/leveldown/src/leveldown_async.o
SOLINK_MODULE(target) Release/leveldown.node
SOLINK_MODULE(target) Release/leveldown.node: Finished
> [email protected] install /Users/nolan/workspace/pouchdb/node_modules/watchify/node_modules/chokidar/node_modules/fsevents
> node-gyp rebuild
CXX(target) Release/obj.target/fse/fsevents.o
SOLINK_MODULE(target) Release/fse.node
SOLINK_MODULE(target) Release/fse.node: Finished
> [email protected] install /Users/nolan/workspace/pouchdb/node_modules/exec-sync/node_modules/ffi/node_modules/ref
> node-gyp rebuild
CXX(target) Release/obj.target/binding/src/binding.o
SOLINK_MODULE(target) Release/binding.node
SOLINK_MODULE(target) Release/binding.node: Finished
> [email protected] install /Users/nolan/workspace/pouchdb/node_modules/phantomjs
> node install.js
Looks like an `npm install -g`; unable to check for already installed version.
Download already available at /var/folders/3x/d09_d8b15tn8yvg8_18pm8200000gn/T/phantomjs/phantomjs-1.9.7-macosx.zip
Extracting zip contents
Copying extracted folder /var/folders/3x/d09_d8b15tn8yvg8_18pm8200000gn/T/phantomjs/phantomjs-1.9.7-macosx.zip-extract-1407707491777/phantomjs-1.9.7-macosx -> /Users/nolan/workspace/pouchdb/node_modules/phantomjs/lib/phantom
Writing location.js file
Done. Phantomjs binary available at /Users/nolan/workspace/pouchdb/node_modules/phantomjs/lib/phantom/bin/phantomjs
> [email protected] install /Users/nolan/workspace/pouchdb/node_modules/exec-sync/node_modules/ffi
> node-gyp rebuild
CC(target) Release/obj.target/ffi/deps/libffi/src/prep_cif.o
CC(target) Release/obj.target/ffi/deps/libffi/src/types.o
CC(target) Release/obj.target/ffi/deps/libffi/src/raw_api.o
CC(target) Release/obj.target/ffi/deps/libffi/src/java_raw_api.o
CC(target) Release/obj.target/ffi/deps/libffi/src/closures.o
CC(target) Release/obj.target/ffi/deps/libffi/src/x86/ffi.o
CC(target) Release/obj.target/ffi/deps/libffi/src/x86/ffi64.o
CC(target) Release/obj.target/ffi/deps/libffi/src/x86/darwin.o
CC(target) Release/obj.target/ffi/deps/libffi/src/x86/darwin64.o
LIBTOOL-STATIC Release/libffi.a
CXX(target) Release/obj.target/ffi_bindings/src/ffi.o
CXX(target) Release/obj.target/ffi_bindings/src/callback_info.o
CXX(target) Release/obj.target/ffi_bindings/src/threaded_callback_invokation.o
SOLINK_MODULE(target) Release/ffi_bindings.node
ld: warning: could not create compact unwind for _ffi_call_unix64: does not use RBP or RSP based frame
SOLINK_MODULE(target) Release/ffi_bindings.node: Finished
npm ERR! fetch failed http://localhost:5080/tarballs/plist-with-patches/0.5.1.tgz
npm ERR! Error: Parse Error
npm ERR! at Socket.socketOnData (http.js:1583:20)
npm ERR! at TCP.onread (net.js:527:27)
npm ERR! If you need help, you may report this *entire* log,
npm ERR! including the npm and node versions, at:
npm ERR! <http://github.com/npm/npm/issues>
npm ERR! System Darwin 13.3.0
npm ERR! command "node" "/usr/local/bin/npm" "install"
npm ERR! cwd /Users/nolan/workspace/pouchdb
npm ERR! node -v v0.10.28
npm ERR! npm -v 1.4.13
npm ERR! code HPE_INVALID_CONSTANT
npm ERR!
npm ERR! Additional logging details can be found in:
npm ERR! /Users/nolan/workspace/pouchdb/npm-debug.log
npm ERR! not ok code 0
nolan@cinnabar pouchdb master (>'.')> rm -fr node_modules/
nolan@cinnabar pouchdb master <('.'<) npm cache clean
nolan@cinnabar pouchdb master <('.'<) npm install
npm ERR! fetch failed http://localhost:5080/tarballs/argsarray/0.0.1.tgz
npm ERR! fetch failed http://localhost:5080/tarballs/spark-md5/0.0.5.tgz
npm WARN deprecated [email protected]: v4.0.0 no longer shims Array#splice with ES5 behavior, makes shims non-enumerable when possible, and fixes a bug in ES3 browsers referencing String#indexOf/lastIndexOf before polyfilling it
npm ERR! fetch failed http://localhost:5080/tarballs/sauce-connect-launcher/0.2.2.tgz
npm ERR! fetch failed http://localhost:5080/tarballs/argsarray/0.0.1.tgz
npm ERR! fetch failed http://localhost:5080/tarballs/spark-md5/0.0.5.tgz
npm ERR! fetch failed http://localhost:5080/tarballs/sauce-connect-launcher/0.2.2.tgz
npm ERR! fetch failed http://localhost:5080/tarballs/argsarray/0.0.1.tgz
npm ERR! Error: Parse Error
npm ERR! at Socket.socketOnData (http.js:1583:20)
npm ERR! at TCP.onread (net.js:527:27)
npm ERR! If you need help, you may report this *entire* log,
npm ERR! including the npm and node versions, at:
npm ERR! <http://github.com/npm/npm/issues>
npm ERR! System Darwin 13.3.0
npm ERR! command "node" "/usr/local/bin/npm" "install"
npm ERR! cwd /Users/nolan/workspace/pouchdb
npm ERR! node -v v0.10.28
npm ERR! npm -v 1.4.13
npm ERR! code HPE_INVALID_CONSTANT
npm ERR! fetch failed http://localhost:5080/tarballs/spark-md5/0.0.5.tgz
npm ERR! fetch failed http://localhost:5080/tarballs/sauce-connect-launcher/0.2.2.tgz
npm ERR!
npm ERR! Additional logging details can be found in:
npm ERR! /Users/nolan/workspace/pouchdb/npm-debug.log
npm ERR! not ok code 0
first install npmrc
npm install npmrc -g
npmrc
npmrc -r cn
The first line installs the app
then initialize it to setup default
npmrc -r [registry] to use an npm mirror - in this case the china mirror
use
http://npm.taobao.org to get npm packages
to setup
npmrc -c taobao
To use:
npmrc default
and to switch to china server
npmrc taobao
If you want to use a local-npm repositry then use the remote option
Remote registry at china server
local-npm -R http://registry.npm.taobao.org/
Switch between different .npmrc files with ease and grace.
usage:
npmrc list all profiles
npmrc [name] change npmrc profile (uses fuzzy matching)
npmrc -c [name] create a new npmrc profile called name
npmrc -r [registry] to use an npm mirror
available mirrors for npmrc -r:
au - Australian registry mirror
eu - European registry mirror
cn - Chinese registry mirror
default - Default registry
example:
$ npmrc -c work
$ npmrc work
$ npmrc default
Follow-up to #84: we need a better way to detect whether the --remote
repo is another local-npm or not, otherwise e.g. if someone has an HTTP proxy to the main npm, then we will assume it's local-npm
and be unable to fetch tarballs. The current system for detecting this is hacky and is based on looking for a URL that looks like https://registry.npmjs.com
.
What we should do is first do a GET
request to the fat registry and check if it sends back some piece of information to identify it as local-npm
(I think local-npm already does this, just for convenience). This could be done in the setup, along with the info()
operation on the skimdb.
This may be fixed in the latest version of express-pouchdb, but I was on [email protected] and constantly getting "too many open files" on my MacBook Pro running Mavericks.
I'm not sure how to prove that it was local-npm causing it, but it seemed to come up only when I was running it and go away when I shut it down.
I'm reinstalling local-npm. I'll close this if it goes away after an upgrade.
I have some packages in an NPM organization that are private. When I try to install them, I get a 404 error from NPM. I assume this is because they are kept outside of the standard, replicated database so local-npm cannot even know about them.
Knowing the answer is probably no, I figured I'd ask anyway: can private package support be added to local-npm?
npm ERR! Darwin 15.0.0
npm ERR! argv "/usr/local/bin/node" "/usr/local/bin/npm" "update"
npm ERR! node v5.4.0
npm ERR! npm v3.3.12
npm ERR! shasum check failed for /var/folders/kz/l3s2fpd51ln9_p9ymvqbyshm0000gr/T/npm-85850-aee4b87e/127.0.0.1_5080/tarballs/karma-phantomjs-launcher/0.2.3.tgz
npm ERR! Expected: 77f68243fad7852c5b321bcf9d691f1223cc0809
npm ERR! Actual: 971e32af3eeb22d6165a10d8bc214d981701024f
npm ERR! From: http://127.0.0.1:5080/tarballs/karma-phantomjs-launcher/0.2.3.tgz
npm ERR!
npm ERR! If you need help, you may report this error at:
npm ERR! <https://github.com/npm/npm/issues>
npm ERR! Please include the following file with any support request:
npm ERR! /Users/Ingwie/Work/BIRD3/npm-debug.log
How do I fix that? :/ It keeps happening, and clearing my cache did not do a thing.
i get an error when i try to install on a windows 7 computer
C:\temp\node_modules\local-npm\node_modules\pouchdb\node_modules\leveldown>if not defined npm_config_node_gyp (n
ode "C:\Program Files (x86)\nodejs\node_modules\npm\bin\node-gyp-bin....\node_modules\node-gyp\bin\node-gyp.j
s" rebuild ) else (node rebuild )
C:\Program Files (x86)\nodejs\node_modules\npm\node_modules\node-gyp\gyp\pylib\gyp\input.py:883: Warning: 'as' w
ill become a reserved keyword in Python 2.6
Traceback (most recent call last):
File "C:\Program Files (x86)\nodejs\node_modules\npm\node_modules\node-gyp\gyp\gyp_main.py", line 11, in
import gyp
File "C:\Program Files (x86)\nodejs\node_modules\npm\node_modules\node-gyp\gyp\pylib\gyp__init__.py", line 8,
in
import gyp.input
File "C:\Program Files (x86)\nodejs\node_modules\npm\node_modules\node-gyp\gyp\pylib\gyp\input.py", line 883
except ImportError as e:
^
SyntaxError: invalid syntax
gyp ERR! configure error
gyp ERR! stack Error: gyp
failed with exit code: 1
gyp ERR! stack at ChildProcess.onCpExit (C:\Program Files (x86)\nodejs\node_modules\npm\node_modules\node-gy
p\lib\configure.js:355:16)
gyp ERR! stack at ChildProcess.emit (events.js:110:17)
gyp ERR! stack at Process.ChildProcess._handle.onexit (child_process.js:1074:12)
gyp ERR! System Windows_NT 6.1.7601
gyp ERR! command "node" "C:\Program Files (x86)\nodejs\node_modules\npm\node_modules\node-gyp\bin\node-g
yp.js" "rebuild"
gyp ERR! cwd C:\temp\node_modules\local-npm\node_modules\pouchdb\node_modules\leveldown
gyp ERR! node -v v0.12.7
gyp ERR! node-gyp -v v2.0.1
gyp ERR! not ok
npm WARN optional dep failed, continuing [email protected]
npm ERR! Windows_NT 6.1.7601
npm ERR! argv "C:\Program Files (x86)\nodejs\node.exe" "C:\Program Files (x86)\nodejs\node_modules\npm
\bin\npm-cli.js" "install" "local-npm"
npm ERR! node v0.12.7
npm ERR! npm v2.11.3
npm ERR! code ELIFECYCLE
npm ERR! [email protected] install: node-gyp rebuild
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] install script 'node-gyp rebuild'.
npm ERR! This is most likely a problem with the leveldown package,
npm ERR! not with npm itself.
After updating to node 4, local-npm ceased to work at some point.
Uninstall was ok, but when npm install -g local-npm
, I got this error.
Here is npm-debug.log file.
As you can see I'm getting between 1 and 3 seconds for some of my packages even though I had just run npm install
.
Shouldn't this be faster since it's already downloaded those packages the first time I ran npm install
?
local-npm-0 GET /os-name 304 17.730 ms - -
local-npm-0 GET /object-assign 200 21.770 ms - -
local-npm-0 GET /chalk 304 1152.665 ms - -
local-npm-0 GET /configstore 304 1621.630 ms - -
local-npm-0 GET /inquirer 304 2061.907 ms - -
local-npm-0 GET /xdg-basedir 304 6.666 ms - -
local-npm-0 GET /user-home 304 770.410 ms - -
local-npm-0 GET /uuid 304 1101.557 ms - -
local-npm-0 GET /graceful-fs 304 1361.589 ms - -
local-npm-0 GET /js-yaml 304 1941.291 ms - -
local-npm-0 GET /esprima 304 1963.567 ms - -
local-npm-0 GET /argparse 304 2221.692 ms - -
local-npm-0 GET /inherit 304 2244.285 ms - -
local-npm-0 GET /sprintf-js 304 1918.798 ms - -
local-npm-0 GET /lodash 304 7534.282 ms - -
local-npm-0 GET /lodash.isfunction 304 5.733 ms - -
local-npm-0 GET /lodash.isobject 304 10.476 ms - -
local-npm-0 GET /lodash.now 304 13.117 ms - -
local-npm-0 GET /lodash._objecttypes 304 2.931 ms - -
local-npm-0 GET /lodash._isnative 304 1.427 ms - -
local-npm-0 GET /win-release 304 3.705 ms - -
local-npm-0 GET /osx-release 304 5.265 ms - -
local-npm-0 GET /minimist 304 1084.798 ms - -
local-npm-0 GET /semver 304 1959.615 ms - -
See pouchdb/pouchdb-security#2 for details.
Replicating skimdb, last_seq is: 11803 (6.92%)
Replicating skimdb, last_seq is: 12003 (7.04%)
Replicating skimdb, last_seq is: 12203 (7.16%)
Replicating skimdb, last_seq is: 12403 (7.27%)
Replicating skimdb, last_seq is: 12603 (7.39%)
Replicating skimdb, last_seq is: 12803 (7.51%)
Replicating skimdb, last_seq is: 13003 (7.63%)
Replicating skimdb, last_seq is: 13203 (7.74%)
error during replication with skimdb
{"status":503,"name":"pound_service_unavailable","message":"Database encountered an unknown error"}
Replicating skimdb, last_seq is: 1 (0%)
Replicating skimdb, last_seq is: 2 (0%)
Replicating skimdb, last_seq is: 200 (0.11%)
Replicating skimdb, last_seq is: 401 (0.23%)
Replicating skimdb, last_seq is: 601 (0.35%)
error during replication with skimdb
{"status":503,"name":"pound_service_unavailable","message":"Database encountered an unknown error"}
Replicating skimdb, last_seq is: 1 (0%)
Replicating skimdb, last_seq is: 2 (0%)
Replicating skimdb, last_seq is: 200 (0.11%)
Replicating skimdb, last_seq is: 401 (0.23%)
Replicating skimdb, last_seq is: 601 (0.35%)
Replicating skimdb, last_seq is: 801 (0.47%)
error during replication with skimdb
{"status":503,"name":"pound_service_unavailable","message":"Database encountered an unknown error"}
(node) warning: possible EventEmitter memory leak detected. 11 listeners added. Use emitter.setMaxListeners() to increase limit.
Trace
at PouchDB.addListener (events.js:160:15)
at PouchDB.once (events.js:185:8)
at new Changes (/Users/remy/.nvm/v0.10.38/lib/node_modules/local-npm/node_modules/pouchdb/lib/changes.js:41:6)
at PouchDB.AbstractPouchDB.changes (/Users/remy/.nvm/v0.10.38/lib/node_modules/local-npm/node_modules/pouchdb/lib/adapter.js:680:10)
at getChanges (/Users/remy/.nvm/v0.10.38/lib/node_modules/local-npm/node_modules/pouchdb/lib/replicate/replicate.js:332:23)
at /Users/remy/.nvm/v0.10.38/lib/node_modules/local-npm/node_modules/pouchdb/lib/replicate/replicate.js:376:9
It seems to have reset after around 7%...
Currently for updates it just goes straight to remote and if it fails, it fails, Should log the modules that need updating somewhere and ensure they get updated eventually.
Hi,
I'm trying to use local-npm behind proxy. Is it possible or does that require code modifications?
Publishing a module - authorized previously through regular NPM - results in a 400 error message and NPM refusing to publish.
[email protected]_W723_V_Typ_A_1_01_011 ~/W/t/kute-git $ npm publish
npm http request PUT http://127.0.0.1:5080/kute.js
npm http 400 http://127.0.0.1:5080/kute.js
npm ERR! registry error parsing json
npm ERR! publish Failed PUT 400
npm ERR! Darwin 15.3.0
npm ERR! argv "/usr/local/Cellar/node/5.5.0/bin/node" "/usr/local/bin/npm" "publish"
npm ERR! node v5.5.0
npm ERR! npm v3.6.0
npm ERR! Unexpected token <
npm ERR! <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
npm ERR! <html><head>
npm ERR! <title>400 Bad Request</title>
npm ERR! </head><body>
npm ERR! <h1>Bad Request</h1>
npm ERR! <p>Your browser sent a request that this server could not understand.<br />
npm ERR! </p>
npm ERR! </body></html>
npm ERR!
npm ERR!
npm ERR! If you need help, you may report this error at:
npm ERR! <https://github.com/npm/npm/issues>
npm ERR! Please include the following file with any support request:
npm ERR! /Users/Ingwie/Work/thednp/kute-git/npm-debug.log
Currently URLs like GET /-/by-field/?key=%22leveldown%22&field=repository
do not work, because NPM has a bunch of redirects defined in _design/app
that handle this, but we don't.
To test it out, you can run something like changed-log
and then run e.g. changed-log leveldown 0.10.0 1.0.0
.
$ npm login # registry set to http://localhost:5080 and remote http://registry.npmjs.org
Username: ...
Password: ...
Email-thing: ...
npm http request PUT http://127.0.0.1:5080/-/user/org.couchdb.user:ingwiephoenix
npm http 400 http://127.0.0.1:5080/-/user/org.couchdb.user:ingwiephoenix
npm ERR! registry error parsing json
npm ERR! Darwin 15.3.0
npm ERR! argv "/usr/local/bin/node" "/usr/local/bin/npm" "login"
npm ERR! node v5.7.0
npm ERR! npm v3.7.3
npm ERR! Unexpected token <
npm ERR! <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
npm ERR! <html><head>
npm ERR! <title>400 Bad Request</title>
npm ERR! </head><body>
npm ERR! <h1>Bad Request</h1>
npm ERR! <p>Your browser sent a request that this server could not understand.<br />
npm ERR! </p>
npm ERR! </body></html>
npm ERR! (...)
Installed version is: 1.5.7
So...how do I log in?
For a certain package, local-npm isn't showing the latest available version from npm. Is falling out of async an issue? Is there any easy way to rescan to get it back?
--preload
a list of modules or --preload-top 100
or something like that.
seeing this occasionally:
fullfat hit an error
{ [Error: socket hang up] code: 'ECONNRESET' }
Never depend on a branch in Github, somebody might remove it... >_<
On the real npm registry, you can do any one of:
But local-npm only works if you spell out the full version number (the last one).
$ local-npm
Welcome!
To start using local-npm, just run:
$ npm set registry http://127.0.0.1:5080
To switch back, you can run:
$ npm set registry https://registry.npmjs.org
A simple npm-like UI is available here: http://127.0.0.1:5080/_browse
PouchDB Server listening on port 16984.
Navigate to http://localhost:16984/_utils for the Fauxton UI.
error doing logger.info() on https://skimdb.npmjs.com/registry
Error: getaddrinfo ENOTFOUND skimdb.npmjs.com
error doing logger.info() on https://skimdb.npmjs.com/registry
Error: getaddrinfo ENOTFOUND skimdb.npmjs.com
error doing logger.info() on https://skimdb.npmjs.com/registry
Error: getaddrinfo ENOTFOUND skimdb.npmjs.com
error doing logger.info() on https://skimdb.npmjs.com/registry
Error: getaddrinfo ENOTFOUND skimdb.npmjs.com
...
error doing logger.info() on https://skimdb.npmjs.com/registry
{"status":503,"name":"pound_service_unavailable","message":"Database encountered an unknown error"}
GET /lodash 200 573.090 ms - -
error doing logger.info() on https://skimdb.npmjs.com/registry
{"status":503,"name":"pound_service_unavailable","message":"Database encountered an unknown error"}
...
go offline and reinstalling lodash
Error: getaddrinfo ENOTFOUND skimdb.npmjs.com
error doing logger.info() on https://skimdb.npmjs.com/registry
Error: getaddrinfo ENOTFOUND skimdb.npmjs.com
error doing logger.info() on https://skimdb.npmjs.com/registry
Error: getaddrinfo ENOTFOUND skimdb.npmjs.com
stream.js:94
throw er; // Unhandled stream error in pipe.
^
Error: getaddrinfo ENOTFOUND registry.npmjs.org
at errnoException (dns.js:44:10)
at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:94:26)
$
I guess that it is not intended to work offline neither with intermitent network connection, right?
I wonder if anyone has given any thought to having an entirely user-local version that uses ~/.npm
to store data. With something like that, npm could safely use --cache-min=Infinity
and avoid even talking to the proxy unless it needed something not cached.
It looks like this will just involve hooking the pouchdb
changes feed to update metadata and making it possible to swap out leveldb
for something that talks to the npm cache.
Thoughts? Would you all be interested in this as a PR?
first time running publish i get ENEEDAUTH
run npm adduser
fill out stuff
get the following
npm ERR! Unexpected token C
npm ERR! Cannot PUT /-/user/org.couchdb.user:mattkrick
currently switching back whenever i need to publish, which isn't great, but sure beats the heck out of waiting 5 mins to run install thanks to npm3 ๐
that should be pretty easy to impliment
It seems I haven't read much about how to use this tools. So, I were following Addy Osmani article https://addyosmani.com/blog/using-npm-offline. I run with local-npm -d ~/.local-npm
then I follow this guide:
When I turn off my wifi, I got this..
stream.js:74
throw er; // Unhandled stream error in pipe.
^
Error: getaddrinfo ENOTFOUND registry.npmjs.org registry.npmjs.org:443
at errnoException (dns.js:26:10)
at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:77:26)
If I start local-npm
when I'm offline, I got this message..
Error: getaddrinfo ENOTFOUND skimdb.npmjs.com skimdb.npmjs.com:443
Error fetching info() from https://skimdb.npmjs.com/registry, retrying after 1100 ms...
And when I try to install a package..
stream.js:74
throw er; // Unhandled stream error in pipe.
^
Error: getaddrinfo ENOTFOUND registry.npmjs.org registry.npmjs.org:443
at errnoException (dns.js:26:10)
at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:77:26)
Then local-npm
just stopped.
Is this the correct behaviour?
I thought it should be working when I'm offline.
Thanks.
cdieringer@Snapper-osx:~/node/coinstac$ npmo # alias to "(cd ~/.local-npm && local-npm)"
Welcome!
To start using local-npm, just run:
$ npm set registry http://127.0.0.1:5080
To switch back, you can run:
$ npm set registry https://registry.npmjs.org
A simple npm-like UI is available here: http://127.0.0.1:5080/_browse
PouchDB Server listening on port 16984.
Navigate to http://localhost:16984/_utils for the Fauxton UI.
Replicating skimdb, last_seq is: 646278 (100%)
GET /_utils 301 18.618 ms - 72
GET /_utils/ 200 10.739 ms - -
GET /_utils/css/index-6688cd4426ead40b085270f8cb007530.css 200 6.072 ms - -
GET /_utils/js/require-34a7e370ea98389d118cc328682c0939.js 200 4.591 ms - -
GET /_session 200 4.187 ms - 140
GET /_session 304 4.046 ms - -
GET / 200 6.428 ms - 149
GET /_utils/img/pouchdb-site.png 200 16.347 ms - 4155
GET /_utils/fonts/fontawesome-webfont.woff?v=3.2.1 200 5.596 ms - 43572
GET /_utils/fonts/fauxtonicon.woff 200 5.308 ms - 10368
GET /_all_dbs?_=1452717931519 200 7.709 ms - 34
GET /_utils/js/zeroclipboard/ZeroClipboard.swf?noCache=1452717931887 200 4.012 ms - 2157
GET /_replicator 200 17.624 ms - 184
GET /_users 200 18.950 ms - 179
GET /skimdb 200 16.862 ms - 194
GET /skimdb/_all_docs?limit=21 200 18.113 ms - -
GET /_all_dbs 200 8.482 ms - 34
GET /skimdb/_all_docs?startkey=%22_design%2F%22&endkey=%22_design0%22&include_docs=true&limit=501 200 19.206 ms - -
GET /skimdb/_all_docs?startkey=%22_design%2F%22&endkey=%22_design0%22&include_docs=true&limit=501 304 10.355 ms - -
GET /skimdb/_all_docs?limit=101 200 21.521 ms - -
GET /skimdb/_index? 200 8.383 ms - 97
GET /_utils/img/couch-watermark.png 200 5.570 ms - 7053
GET /_utils/js/ace/mode-javascript.js 200 4.806 ms - -
GET /_utils/js/ace/theme-idle_fingers.js 200 3.044 ms - -
GET /_utils/js/ace/worker-javascript.js 200 4.611 ms - -
GET /_all_dbs 304 4.169 ms - -
GET /_active_tasks 200 7.776 ms - 3
GET /_active_tasks 304 3.136 ms - -
GET /_active_tasks 304 4.686 ms - -
GET /_all_dbs?_=1452717931520 200 399.420 ms - 34
GET /_replicator 200 4155.618 ms - 184
<--- Last few GCs --->
113602 ms: Scavenge 1393.9 (1458.0) -> 1393.9 (1458.0) MB, 0.2 / 0 ms (+ 2.1 ms in 1 steps since last GC) [allocation failure] [incremental marking delaying mark-sweep].
114852 ms: Mark-sweep 1393.9 (1458.0) -> 1391.4 (1458.0) MB, 1249.4 / 0 ms (+ 3.1 ms in 2 steps since start of marking, biggest step 2.1 ms) [last resort gc].
115911 ms: Mark-sweep 1391.4 (1458.0) -> 1365.3 (1458.0) MB, 1059.8 / 0 ms [last resort gc].
<--- JS stacktrace --->
==== JS stack trace =========================================
2: arguments adaptor frame: 2->3
Security context: 0x14d6ffeb4629 <JS Object>
3: /* anonymous */ [/usr/local/lib/node_modules/local-npm/node_modules/compression/index.js:87] [pc=0x177d4c80e1d2] (this=0x17b3bbb7f19 <a ServerResponse with map 0x26f8641338b1>,chunk=0x17b3bbc0769 <an Uint8Array with map 0x26f8641337a9>,encoding=0x14d6ffe041b9 <undefined>)
4: write [/usr/local/lib/node_modules/local-npm/node_modules/compression/ind...
FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - process out of memory
I think we could do a get
with revisions=true
and essentially replicate on-demand (inserting with new_edits=false
, as the replicator does). This would mean you wouldn't have to wait for a slow ramp-up time for all the initial replication of the skimdb metadata to complete. See #96 for more info.
node-gyp v3.2.1
node.js v5.3.0
npm v3.5.3
prebuild v2.8.1
When I run local-npm I get the following error:
C:\Users\david.brower\AppData\Roaming\npm\node_modules\local-npm\node_modules\bi
ndings\bindings.js:91
throw err
^
Error: Could not locate the bindings file. Tried:
โ C:\Users\david.brower\AppData\Roaming\npm\node_modules\local-npm\node_modules
\leveldown\build\leveldown.node
โ C:\Users\david.brower\AppData\Roaming\npm\node_modules\local-npm\node_modules
\leveldown\build\Debug\leveldown.node
โ C:\Users\david.brower\AppData\Roaming\npm\node_modules\local-npm\node_modules
\leveldown\build\Release\leveldown.node
โ C:\Users\david.brower\AppData\Roaming\npm\node_modules\local-npm\node_modules
\leveldown\out\Debug\leveldown.node
โ C:\Users\david.brower\AppData\Roaming\npm\node_modules\local-npm\node_modules
\leveldown\Debug\leveldown.node
โ C:\Users\david.brower\AppData\Roaming\npm\node_modules\local-npm\node_modules
\leveldown\out\Release\leveldown.node
โ C:\Users\david.brower\AppData\Roaming\npm\node_modules\local-npm\node_modules
\leveldown\Release\leveldown.node
โ C:\Users\david.brower\AppData\Roaming\npm\node_modules\local-npm\node_modules
\leveldown\build\default\leveldown.node
โ C:\Users\david.brower\AppData\Roaming\npm\node_modules\local-npm\node_modules
\leveldown\compiled\5.3.0\win32\x64\leveldown.node
at bindings (C:\Users\david.brower\AppData\Roaming\npm\node_modules\local-np
m\node_modules\bindings\bindings.js:88:9)
at Object. (C:\Users\david.brower\AppData\Roaming\npm\node_module
s\local-npm\node_modules\leveldown\leveldown.js:4:46)
at Module._compile (module.js:398:26)
at Object.Module._extensions..js (module.js:405:10)
at Module.load (module.js:344:32)
at Function.Module._load (module.js:301:12)
at Module.require (module.js:354:17)
at require (internal/module.js:12:17)
at Object. (C:\Users\david.brower\AppData\Roaming\npm\node_module
s\local-npm\node_modules\level\level.js:1:106)
at Module._compile (module.js:398:26)
at Object.Module._extensions..js (module.js:405:10)
at Module.load (module.js:344:32)
at Function.Module._load (module.js:301:12)
at Module.require (module.js:354:17)
at require (internal/module.js:12:17)
at Object. (C:\Users\david.brower\AppData\Roaming\npm\node_module
s\local-npm\lib\index.js:10:13)
It's downright embarrassing not to have any. This tool has grown enough (in codebase size and popularity) that it deserves tests.
Probably we can use a combination of pouchdb-server and pouchdb-load to avoid having to replicate all of npm just to run the tests. Running the tests on a subset should be fine.
Hey!
Thanks for local-npm, love it!
However, I've had some problems that when I've went offline for a while I'm behind the main registry and it doesn't seem to reconnect to get up to speed again...
I've tried restarting the local-npm process without any success. This is, using the OS X launch daemon (https://gist.github.com/nolanlawson/83ba5862bd719925d9cd).
Any tips on what I should do to get this working?
This is awesome! Will you add a small note to the README on how to restore the original npm registry? I'm sure it's trivial, but that was the first thing that came to mind after reading the README.
Hey,
The package is great and helps a lot especially on very slow connections, but on big installs local-npm will crash a few times as a result of Timeouts with the following error -
events.js:141
throw er; // Unhandled 'error' event
^
Error: connect ETIMEDOUT 199.27.79.162:443
at Object.exports._errnoException (util.js:856:11)
at exports._exceptionWithHostPort (util.js:879:20)
at TCPConnectWrap.afterConnect as oncomplete
In cases like this 'npm install' will keep on running and restarting local-npm will allow for it to continue installation. But unless I am watching local-npm I sometimes miss the crash and the time for install gets longer.
Changing the way local-npm works to issue a warning but continue retrying will be a fix for this.
Thanks
I get the same error for both, npm install node-sass
and npm install node-sass@latest
$ npm i node-sass@latest
npm ERR! Darwin 15.3.0
npm ERR! argv "/Users/gregor/.nvm/versions/node/v4.2.2/bin/node" "/Users/gregor/.nvm/versions/node/v4.2.2/bin/npm" "i" "node-sass@latest"
npm ERR! node v4.2.2
npm ERR! npm v3.5.2
npm ERR! No compatible version found: node-sass@latest
npm ERR! Valid install targets:
npm ERR! 3.5.0-beta.1, 3.4.2, 3.4.1, 3.4.0, 3.4.0-beta1, 3.4.0-beta.2, 3.3.3, 3.3.2, 3.3.1, 3.3.0, 3.2.0, 3.1.2, 3.1.1, 3.1.0, 3.0.0, 3.0.0-beta.7, 3.0.0-beta.5, 3.0.0-beta.4, 3.0.0-beta.3, 3.0.0-beta.2, 3.0.0-alpha.0, 2.1.1, 2.1.0, 2.0.1, 2.0.0, 2.0.0-beta, 1.2.3, 1.2.2, 1.2.1, 1.2.0, 1.1.4, 1.1.3, 1.1.2, 1.1.1, 1.1.0, 1.0.3, 1.0.2, 1.0.2-alpha, 1.0.1, 1.0.0, 0.9.6, 0.9.5, 0.9.5-rc1, 0.9.4, 0.9.4-rc1, 0.9.3, 0.9.2, 0.9.1, 0.9.0, 0.8.6, 0.8.5, 0.8.4, 0.8.3, 0.8.2, 0.8.1, 0.8.0, 0.7.0, 0.7.0-alpha, 0.6.7, 0.6.6, 0.6.5, 0.6.4, 0.6.3, 0.6.2, 0.6.1, 0.6.0, 0.5.4, 0.5.3, 0.5.2, 0.5.1, 0.5.0, 0.4.4, 0.4.3, 0.4.2, 0.4.1, 0.4.0, 0.3.0, 0.2.6, 0.2.5, 0.2.4, 0.2.3, 0.2.2, 0.2.1, 0.2.0
npm ERR!
npm ERR!
npm ERR! If you need help, you may report this error at:
npm ERR! <https://github.com/npm/npm/issues>
npm ERR! Please include the following file with any support request:
npm ERR! /private/tmp/npm-debug.log
Outputh with loglevel=silly
: https://gist.github.com/f11dfb3b2c3e2b04d480
The issue does not occur when using the standard registry (npm set registry https://registry.npmjs.org
).
This works: npm install [email protected]
I've updated local-npm to the latest version
$ local-npm --version
1.5.7
All packages are synced (Replicating skimdb, last_seq is: 768417 (100%)
)
Here's an example of the error from npm install
:
npm ERR! 404 Not found : @jimpick/spectacle
npm ERR! 404
npm ERR! 404 '@jimpick/spectacle' is not in the npm registry.
Information on scoped (and private) modules:
It would be great if it was possible to publish a private repository to local-npm. I.e. when private: true
is set in package.json and one runs npm publish
it will no be published to npm but it still can be installed locally via local-npm.
It's more a question:
Is it transparent using ~/.npmrc for proxy settings when doing the replication?
Cuz I see those
error doing logger.info() on https://skimdb.npmjs.com/registry
Error: ETIMEDOUT
error doing logger.info() on https://skimdb.npmjs.com/registry
Error: ETIMEDOUT
error doing logger.info() on https://skimdb.npmjs.com/registry
Error: ETIMEDOUT
error doing logger.info() on https://skimdb.npmjs.com/registry
Error: ETIMEDOUT
error doing logger.info() on https://skimdb.npmjs.com/registry
While my cmd line proxy and .npmrc are both setup properly.
This is what pnpm
does, so we'll need to do it to support it.
I was looking at http://127.0.0.1:5080/_browse/#/local-npm and the "How it works" section tells of the http://127.0.0.1:16984/_utils entry point - but because the markdown parser used seems a little bit flaky (I haven't investigated any further yet) the url http://127.0.0.1:16984/utils is shown instead with a bit in italic (because of the two underline characters. Compare the output with https://github.com/nolanlawson/local-npm#how-it-works. This is just one example.
What markdown library is used in the deps? Perhaps a bit of config can fix its output.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.