Git Product home page Git Product logo

local-npm's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

local-npm's Issues

Allow specifying replication batch size in command line

Especially for syncing local-npm between different computers ("protip: You can replicate from your friend's local-npm..."), it would be nice to increase the batch size, unless there's some other limitation that forbids this.

Or, batch size could be automatically increased if you detect that the remote address is on a local subnet (192.168.x.x / 10.x.x.x / that other local subnet that nobody uses)

Bind local-npm against remote Sinopia instance

Hi, I'm very interested in using local-npm but currently I'm facing the problem that our "master" npm registry is based on Sinopia which hosts bunch of private NPM packages and it's currently difficult to change that. Is it somehow possible to bind local-npm against Sinopia (or similar solutions that provide only a fullfatdb)? I'd love to use local-npm so that I can work also offline.

Thanks a lot in advance!

Starts re-replicating from scratch sometimes

@remy For the record, I am still seeing this sometimes. Happened a moment ago when my laptop went into hibernate mode and then restarted; might be related. Still, since the checkpoint is stored on disk, there's no reason it should start re-replicating from scratch.

...
local-npm-0 Replicating skimdb, last_seq is: 35869 (18.74%)
local-npm-0 Replicating skimdb, last_seq is: 36071 (18.84%)
local-npm-0 Replicating skimdb, last_seq is: 36271 (18.95%)
local-npm-0 Replicating skimdb, last_seq is: 36472 (19.05%)
local-npm-0 Replicating skimdb, last_seq is: 36674 (19.16%)
local-npm-0 Replicating skimdb, last_seq is: 36875 (19.26%)
local-npm-0 Replicating skimdb, last_seq is: 37076 (19.37%)
local-npm-0 Replicating skimdb, last_seq is: 37276 (19.47%)
local-npm-0 Replicating skimdb, last_seq is: 37476 (19.58%)
local-npm-0 Replicating skimdb, last_seq is: 37677 (19.68%)
local-npm-0 Replicating skimdb, last_seq is: 37877 (19.78%)
local-npm-0 Replicating skimdb, last_seq is: 38077 (19.89%)
local-npm-0 Replicating skimdb, last_seq is: 38277 (19.99%)
local-npm-0 Replicating skimdb, last_seq is: 38478 (20.1%)

Optimistically cache tarballs when packages change?

When you install lodash@^4.0.0, then local-npm will cache and store 4.0.0 for future npm installs.

However, if 4.0.1 is published after you npm install, then local-npm will find out about it via the changes feed, but it will not download the actual tarballs.

So if you then go offline and try to npm install again, it will fail because local-npm knows that there is a newer version that matches your version range, but it hasn't been downloaded yet.

Question: should local-npm optimistically cache every version of Lodash that gets published, simply because you once installed lodash@^4.0.0?

Pros: more likely to work offline, less likely to cause unexpected errors, fits the "standard" pattern of caret- and tilde-versioning.

Cons: Potentially a lot of network traffic and a lot of wasted disk spaces for module versions that you might never use.

Throws error unhandled stream error in pipe when do npm install offline

I tried to disconnect the internet connection and leave local-npm server
running. Then, I do npm install it gives me:

stream.js:74
      throw er; // Unhandled stream error in pipe.
      ^

Error: getaddrinfo ENOTFOUND registry.npmjs.org registry.npmjs.org:443
    at errnoException (dns.js:26:10)
    at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:77:26)

And then the process exited.

The listed packages on package.json is already stored locally. Because, before I run npm install offline,
I installed everything first and local-npm say this:

downloaded koa at version 2.0.0-alpha.3 and saved it locally

and some other things

It supposed to work offline, right?

Add a sweet UI

Since the whole skimdb is replicated, you could have most of the functionality of the npm website itself served locally.

Ability to specify a working directory

Would be great to be able to pass a directory in which the db and other files will be made, so that we do not have to invoke local-npm in the directory where we want the files. This would help a lot with tools like forever.

Error: Parse Error (HPE_INVALID_CONSTANT)

Steps to reproduce:

  1. Check out pouchdb
  2. npm cache clean
  3. npm install

Saw this:

npm WARN deprecated [email protected]: v4.0.0 no longer shims Array#splice with ES5 behavior, makes shims non-enumerable when possible, and fixes a bug in ES3 browsers referencing String#indexOf/lastIndexOf before polyfilling it
npm ERR! fetch failed http://localhost:5080/tarballs/plist-with-patches/0.5.1.tgz
npm ERR! fetch failed http://localhost:5080/tarballs/plist-with-patches/0.5.1.tgz
npm WARN deprecated [email protected]: Please update to the latest object-keys

> [email protected] install /Users/nolan/workspace/pouchdb/node_modules/leveldown
> node-gyp rebuild

  CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/db/builder.o
  CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/db/db_impl.o
  CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/db/db_iter.o
  CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/db/filename.o
  CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/db/dbformat.o
  CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/db/log_reader.o
  CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/db/log_writer.o
  CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/db/memtable.o
  CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/db/repair.o
  CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/db/table_cache.o
  CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/db/version_edit.o
  CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/db/version_set.o
  CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/db/write_batch.o
  CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/helpers/memenv/memenv.o
  CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/table/block.o
  CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/table/block_builder.o
  CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/table/filter_block.o
  CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/table/format.o
  CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/table/iterator.o
  CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/table/merger.o
  CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/table/table.o
  CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/table/table_builder.o
  CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/table/two_level_iterator.o
  CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/util/arena.o
  CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/util/bloom.o
  CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/util/cache.o
  CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/util/coding.o
  CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/util/comparator.o
  CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/util/crc32c.o
  CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/util/env.o
  CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/util/filter_policy.o
  CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/util/hash.o
  CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/util/logging.o
  CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/util/options.o
  CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/util/status.o
  CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/port/port_posix.o
  CXX(target) Release/obj.target/leveldb/deps/leveldb/leveldb-1.14.0/util/env_posix.o
  LIBTOOL-STATIC Release/leveldb.a
  CXX(target) Release/obj.target/snappy/deps/snappy/snappy-1.1.1/snappy-sinksource.o
  CXX(target) Release/obj.target/snappy/deps/snappy/snappy-1.1.1/snappy-stubs-internal.o
  CXX(target) Release/obj.target/snappy/deps/snappy/snappy-1.1.1/snappy.o
  LIBTOOL-STATIC Release/snappy.a
  CXX(target) Release/obj.target/leveldown/src/batch.o
  CXX(target) Release/obj.target/leveldown/src/batch_async.o
  CXX(target) Release/obj.target/leveldown/src/database.o
  CXX(target) Release/obj.target/leveldown/src/database_async.o
  CXX(target) Release/obj.target/leveldown/src/iterator.o
  CXX(target) Release/obj.target/leveldown/src/iterator_async.o
  CXX(target) Release/obj.target/leveldown/src/leveldown.o
  CXX(target) Release/obj.target/leveldown/src/leveldown_async.o
  SOLINK_MODULE(target) Release/leveldown.node
  SOLINK_MODULE(target) Release/leveldown.node: Finished

> [email protected] install /Users/nolan/workspace/pouchdb/node_modules/watchify/node_modules/chokidar/node_modules/fsevents
> node-gyp rebuild

  CXX(target) Release/obj.target/fse/fsevents.o
  SOLINK_MODULE(target) Release/fse.node
  SOLINK_MODULE(target) Release/fse.node: Finished

> [email protected] install /Users/nolan/workspace/pouchdb/node_modules/exec-sync/node_modules/ffi/node_modules/ref
> node-gyp rebuild

  CXX(target) Release/obj.target/binding/src/binding.o
  SOLINK_MODULE(target) Release/binding.node
  SOLINK_MODULE(target) Release/binding.node: Finished

> [email protected] install /Users/nolan/workspace/pouchdb/node_modules/phantomjs
> node install.js

Looks like an `npm install -g`; unable to check for already installed version.
Download already available at /var/folders/3x/d09_d8b15tn8yvg8_18pm8200000gn/T/phantomjs/phantomjs-1.9.7-macosx.zip
Extracting zip contents
Copying extracted folder /var/folders/3x/d09_d8b15tn8yvg8_18pm8200000gn/T/phantomjs/phantomjs-1.9.7-macosx.zip-extract-1407707491777/phantomjs-1.9.7-macosx -> /Users/nolan/workspace/pouchdb/node_modules/phantomjs/lib/phantom
Writing location.js file
Done. Phantomjs binary available at /Users/nolan/workspace/pouchdb/node_modules/phantomjs/lib/phantom/bin/phantomjs

> [email protected] install /Users/nolan/workspace/pouchdb/node_modules/exec-sync/node_modules/ffi
> node-gyp rebuild

  CC(target) Release/obj.target/ffi/deps/libffi/src/prep_cif.o
  CC(target) Release/obj.target/ffi/deps/libffi/src/types.o
  CC(target) Release/obj.target/ffi/deps/libffi/src/raw_api.o
  CC(target) Release/obj.target/ffi/deps/libffi/src/java_raw_api.o
  CC(target) Release/obj.target/ffi/deps/libffi/src/closures.o
  CC(target) Release/obj.target/ffi/deps/libffi/src/x86/ffi.o
  CC(target) Release/obj.target/ffi/deps/libffi/src/x86/ffi64.o
  CC(target) Release/obj.target/ffi/deps/libffi/src/x86/darwin.o
  CC(target) Release/obj.target/ffi/deps/libffi/src/x86/darwin64.o
  LIBTOOL-STATIC Release/libffi.a
  CXX(target) Release/obj.target/ffi_bindings/src/ffi.o
  CXX(target) Release/obj.target/ffi_bindings/src/callback_info.o
  CXX(target) Release/obj.target/ffi_bindings/src/threaded_callback_invokation.o
  SOLINK_MODULE(target) Release/ffi_bindings.node
ld: warning: could not create compact unwind for _ffi_call_unix64: does not use RBP or RSP based frame
  SOLINK_MODULE(target) Release/ffi_bindings.node: Finished
npm ERR! fetch failed http://localhost:5080/tarballs/plist-with-patches/0.5.1.tgz
npm ERR! Error: Parse Error
npm ERR!     at Socket.socketOnData (http.js:1583:20)
npm ERR!     at TCP.onread (net.js:527:27)
npm ERR! If you need help, you may report this *entire* log,
npm ERR! including the npm and node versions, at:
npm ERR!     <http://github.com/npm/npm/issues>

npm ERR! System Darwin 13.3.0
npm ERR! command "node" "/usr/local/bin/npm" "install"
npm ERR! cwd /Users/nolan/workspace/pouchdb
npm ERR! node -v v0.10.28
npm ERR! npm -v 1.4.13
npm ERR! code HPE_INVALID_CONSTANT
npm ERR! 
npm ERR! Additional logging details can be found in:
npm ERR!     /Users/nolan/workspace/pouchdb/npm-debug.log
npm ERR! not ok code 0
nolan@cinnabar pouchdb master (>'.')> rm -fr node_modules/
nolan@cinnabar pouchdb master <('.'<) npm cache clean
nolan@cinnabar pouchdb master <('.'<) npm install
npm ERR! fetch failed http://localhost:5080/tarballs/argsarray/0.0.1.tgz
npm ERR! fetch failed http://localhost:5080/tarballs/spark-md5/0.0.5.tgz
npm WARN deprecated [email protected]: v4.0.0 no longer shims Array#splice with ES5 behavior, makes shims non-enumerable when possible, and fixes a bug in ES3 browsers referencing String#indexOf/lastIndexOf before polyfilling it
npm ERR! fetch failed http://localhost:5080/tarballs/sauce-connect-launcher/0.2.2.tgz
npm ERR! fetch failed http://localhost:5080/tarballs/argsarray/0.0.1.tgz
npm ERR! fetch failed http://localhost:5080/tarballs/spark-md5/0.0.5.tgz
npm ERR! fetch failed http://localhost:5080/tarballs/sauce-connect-launcher/0.2.2.tgz
npm ERR! fetch failed http://localhost:5080/tarballs/argsarray/0.0.1.tgz
npm ERR! Error: Parse Error
npm ERR!     at Socket.socketOnData (http.js:1583:20)
npm ERR!     at TCP.onread (net.js:527:27)
npm ERR! If you need help, you may report this *entire* log,
npm ERR! including the npm and node versions, at:
npm ERR!     <http://github.com/npm/npm/issues>

npm ERR! System Darwin 13.3.0
npm ERR! command "node" "/usr/local/bin/npm" "install"
npm ERR! cwd /Users/nolan/workspace/pouchdb
npm ERR! node -v v0.10.28
npm ERR! npm -v 1.4.13
npm ERR! code HPE_INVALID_CONSTANT
npm ERR! fetch failed http://localhost:5080/tarballs/spark-md5/0.0.5.tgz
npm ERR! fetch failed http://localhost:5080/tarballs/sauce-connect-launcher/0.2.2.tgz
npm ERR! 
npm ERR! Additional logging details can be found in:
npm ERR!     /Users/nolan/workspace/pouchdb/npm-debug.log
npm ERR! not ok code 0

Tip: For those in China

For those that are in China to use the Chinese registry

first install npmrc
npm install npmrc -g
npmrc
npmrc -r cn

The first line installs the app
then initialize it to setup default
npmrc -r [registry] to use an npm mirror - in this case the china mirror

Alternative

use
http://npm.taobao.org to get npm packages
to setup
npmrc -c taobao

To use:
npmrc default
and to switch to china server
npmrc taobao

local-npm

If you want to use a local-npm repositry then use the remote option

Remote registry at china server
local-npm -R http://registry.npm.taobao.org/

help section from npmrc

Switch between different .npmrc files with ease and grace.

usage:
npmrc list all profiles
npmrc [name] change npmrc profile (uses fuzzy matching)
npmrc -c [name] create a new npmrc profile called name
npmrc -r [registry] to use an npm mirror

available mirrors for npmrc -r:
au - Australian registry mirror
eu - European registry mirror
cn - Chinese registry mirror
default - Default registry

example:

Creating and activating a new .npmrc called "work":

$ npmrc -c work

Switch betwen "work" and "default"

$ npmrc work
$ npmrc default

Use the European npm mirror $ npmrc -r eu

Incorrectly assumes that the --remote repo is another local-npm

Follow-up to #84: we need a better way to detect whether the --remote repo is another local-npm or not, otherwise e.g. if someone has an HTTP proxy to the main npm, then we will assume it's local-npm and be unable to fetch tarballs. The current system for detecting this is hacky and is based on looking for a URL that looks like https://registry.npmjs.com.

What we should do is first do a GET request to the fat registry and check if it sends back some piece of information to identify it as local-npm (I think local-npm already does this, just for convenience). This could be done in the setup, along with the info() operation on the skimdb.

Too many open files

This may be fixed in the latest version of express-pouchdb, but I was on [email protected] and constantly getting "too many open files" on my MacBook Pro running Mavericks.

I'm not sure how to prove that it was local-npm causing it, but it seemed to come up only when I was running it and go away when I shut it down.

I'm reinstalling local-npm. I'll close this if it goes away after an upgrade.

Install private NPM packages?

I have some packages in an NPM organization that are private. When I try to install them, I get a 404 error from NPM. I assume this is because they are kept outside of the standard, replicated database so local-npm cannot even know about them.

Knowing the answer is probably no, I figured I'd ask anyway: can private package support be added to local-npm?

SHA missmatches

npm ERR! Darwin 15.0.0
npm ERR! argv "/usr/local/bin/node" "/usr/local/bin/npm" "update"
npm ERR! node v5.4.0
npm ERR! npm  v3.3.12

npm ERR! shasum check failed for /var/folders/kz/l3s2fpd51ln9_p9ymvqbyshm0000gr/T/npm-85850-aee4b87e/127.0.0.1_5080/tarballs/karma-phantomjs-launcher/0.2.3.tgz
npm ERR! Expected: 77f68243fad7852c5b321bcf9d691f1223cc0809
npm ERR! Actual:   971e32af3eeb22d6165a10d8bc214d981701024f
npm ERR! From:     http://127.0.0.1:5080/tarballs/karma-phantomjs-launcher/0.2.3.tgz
npm ERR! 
npm ERR! If you need help, you may report this error at:
npm ERR!     <https://github.com/npm/npm/issues>

npm ERR! Please include the following file with any support request:
npm ERR!     /Users/Ingwie/Work/BIRD3/npm-debug.log

How do I fix that? :/ It keeps happening, and clearing my cache did not do a thing.

leveldown dependancy error

i get an error when i try to install on a windows 7 computer

C:\temp\node_modules\local-npm\node_modules\pouchdb\node_modules\leveldown>if not defined npm_config_node_gyp (n
ode "C:\Program Files (x86)\nodejs\node_modules\npm\bin\node-gyp-bin....\node_modules\node-gyp\bin\node-gyp.j
s" rebuild ) else (node rebuild )
C:\Program Files (x86)\nodejs\node_modules\npm\node_modules\node-gyp\gyp\pylib\gyp\input.py:883: Warning: 'as' w
ill become a reserved keyword in Python 2.6
Traceback (most recent call last):
File "C:\Program Files (x86)\nodejs\node_modules\npm\node_modules\node-gyp\gyp\gyp_main.py", line 11, in
import gyp
File "C:\Program Files (x86)\nodejs\node_modules\npm\node_modules\node-gyp\gyp\pylib\gyp__init__.py", line 8,
in
import gyp.input
File "C:\Program Files (x86)\nodejs\node_modules\npm\node_modules\node-gyp\gyp\pylib\gyp\input.py", line 883
except ImportError as e:
^
SyntaxError: invalid syntax
gyp ERR! configure error
gyp ERR! stack Error: gyp failed with exit code: 1
gyp ERR! stack at ChildProcess.onCpExit (C:\Program Files (x86)\nodejs\node_modules\npm\node_modules\node-gy
p\lib\configure.js:355:16)
gyp ERR! stack at ChildProcess.emit (events.js:110:17)
gyp ERR! stack at Process.ChildProcess._handle.onexit (child_process.js:1074:12)
gyp ERR! System Windows_NT 6.1.7601
gyp ERR! command "node" "C:\Program Files (x86)\nodejs\node_modules\npm\node_modules\node-gyp\bin\node-g
yp.js" "rebuild"
gyp ERR! cwd C:\temp\node_modules\local-npm\node_modules\pouchdb\node_modules\leveldown
gyp ERR! node -v v0.12.7
gyp ERR! node-gyp -v v2.0.1
gyp ERR! not ok
npm WARN optional dep failed, continuing [email protected]
npm ERR! Windows_NT 6.1.7601
npm ERR! argv "C:\Program Files (x86)\nodejs\node.exe" "C:\Program Files (x86)\nodejs\node_modules\npm
\bin\npm-cli.js" "install" "local-npm"
npm ERR! node v0.12.7
npm ERR! npm v2.11.3
npm ERR! code ELIFECYCLE

npm ERR! [email protected] install: node-gyp rebuild
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] install script 'node-gyp rebuild'.
npm ERR! This is most likely a problem with the leveldown package,
npm ERR! not with npm itself.

Slow loading times, 1 - 3 seconds per package

As you can see I'm getting between 1 and 3 seconds for some of my packages even though I had just run npm install.
Shouldn't this be faster since it's already downloaded those packages the first time I ran npm install?

local-npm-0 GET /os-name 304 17.730 ms - -
local-npm-0 GET /object-assign 200 21.770 ms - -
local-npm-0 GET /chalk 304 1152.665 ms - -
local-npm-0 GET /configstore 304 1621.630 ms - -
local-npm-0 GET /inquirer 304 2061.907 ms - -
local-npm-0 GET /xdg-basedir 304 6.666 ms - -
local-npm-0 GET /user-home 304 770.410 ms - -
local-npm-0 GET /uuid 304 1101.557 ms - -
local-npm-0 GET /graceful-fs 304 1361.589 ms - -
local-npm-0 GET /js-yaml 304 1941.291 ms - -
local-npm-0 GET /esprima 304 1963.567 ms - -
local-npm-0 GET /argparse 304 2221.692 ms - -
local-npm-0 GET /inherit 304 2244.285 ms - -
local-npm-0 GET /sprintf-js 304 1918.798 ms - -
local-npm-0 GET /lodash 304 7534.282 ms - -
local-npm-0 GET /lodash.isfunction 304 5.733 ms - -
local-npm-0 GET /lodash.isobject 304 10.476 ms - -
local-npm-0 GET /lodash.now 304 13.117 ms - -
local-npm-0 GET /lodash._objecttypes 304 2.931 ms - -
local-npm-0 GET /lodash._isnative 304 1.427 ms - -
local-npm-0 GET /win-release 304 3.705 ms - -
local-npm-0 GET /osx-release 304 5.265 ms - -
local-npm-0 GET /minimist 304 1084.798 ms - -
local-npm-0 GET /semver 304 1959.615 ms - -

Replication completely resets on timeout

Replicating skimdb, last_seq is: 11803 (6.92%)
Replicating skimdb, last_seq is: 12003 (7.04%)
Replicating skimdb, last_seq is: 12203 (7.16%)
Replicating skimdb, last_seq is: 12403 (7.27%)
Replicating skimdb, last_seq is: 12603 (7.39%)
Replicating skimdb, last_seq is: 12803 (7.51%)
Replicating skimdb, last_seq is: 13003 (7.63%)
Replicating skimdb, last_seq is: 13203 (7.74%)
error during replication with skimdb
{"status":503,"name":"pound_service_unavailable","message":"Database encountered an unknown error"}
Replicating skimdb, last_seq is: 1 (0%)
Replicating skimdb, last_seq is: 2 (0%)
Replicating skimdb, last_seq is: 200 (0.11%)
Replicating skimdb, last_seq is: 401 (0.23%)
Replicating skimdb, last_seq is: 601 (0.35%)
error during replication with skimdb
{"status":503,"name":"pound_service_unavailable","message":"Database encountered an unknown error"}
Replicating skimdb, last_seq is: 1 (0%)
Replicating skimdb, last_seq is: 2 (0%)
Replicating skimdb, last_seq is: 200 (0.11%)
Replicating skimdb, last_seq is: 401 (0.23%)
Replicating skimdb, last_seq is: 601 (0.35%)
Replicating skimdb, last_seq is: 801 (0.47%)
error during replication with skimdb
{"status":503,"name":"pound_service_unavailable","message":"Database encountered an unknown error"}
(node) warning: possible EventEmitter memory leak detected. 11 listeners added. Use emitter.setMaxListeners() to increase limit.
Trace
    at PouchDB.addListener (events.js:160:15)
    at PouchDB.once (events.js:185:8)
    at new Changes (/Users/remy/.nvm/v0.10.38/lib/node_modules/local-npm/node_modules/pouchdb/lib/changes.js:41:6)
    at PouchDB.AbstractPouchDB.changes (/Users/remy/.nvm/v0.10.38/lib/node_modules/local-npm/node_modules/pouchdb/lib/adapter.js:680:10)
    at getChanges (/Users/remy/.nvm/v0.10.38/lib/node_modules/local-npm/node_modules/pouchdb/lib/replicate/replicate.js:332:23)
    at /Users/remy/.nvm/v0.10.38/lib/node_modules/local-npm/node_modules/pouchdb/lib/replicate/replicate.js:376:9

It seems to have reset after around 7%...

Retry failed updates

Currently for updates it just goes straight to remote and if it fails, it fails, Should log the modules that need updating somewhere and ensure they get updated eventually.

Proxy support

Hi,
I'm trying to use local-npm behind proxy. Is it possible or does that require code modifications?

Pushing results in 400.

Publishing a module - authorized previously through regular NPM - results in a 400 error message and NPM refusing to publish.

[email protected]_W723_V_Typ_A_1_01_011 ~/W/t/kute-git $ npm publish
npm http request PUT http://127.0.0.1:5080/kute.js
npm http 400 http://127.0.0.1:5080/kute.js
npm ERR! registry error parsing json
npm ERR! publish Failed PUT 400
npm ERR! Darwin 15.3.0
npm ERR! argv "/usr/local/Cellar/node/5.5.0/bin/node" "/usr/local/bin/npm" "publish"
npm ERR! node v5.5.0
npm ERR! npm  v3.6.0

npm ERR! Unexpected token <
npm ERR! <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
npm ERR! <html><head>
npm ERR! <title>400 Bad Request</title>
npm ERR! </head><body>
npm ERR! <h1>Bad Request</h1>
npm ERR! <p>Your browser sent a request that this server could not understand.<br />
npm ERR! </p>
npm ERR! </body></html>
npm ERR! 
npm ERR! 
npm ERR! If you need help, you may report this error at:
npm ERR!     <https://github.com/npm/npm/issues>

npm ERR! Please include the following file with any support request:
npm ERR!     /Users/Ingwie/Work/thednp/kute-git/npm-debug.log

Support _design/app redirects

Currently URLs like GET /-/by-field/?key=%22leveldown%22&field=repository do not work, because NPM has a bunch of redirects defined in _design/app that handle this, but we don't.

To test it out, you can run something like changed-log and then run e.g. changed-log leveldown 0.10.0 1.0.0.

Can not log in.

$ npm login # registry set to http://localhost:5080 and remote http://registry.npmjs.org
Username: ...
Password: ...
Email-thing: ...
npm http request PUT http://127.0.0.1:5080/-/user/org.couchdb.user:ingwiephoenix
npm http 400 http://127.0.0.1:5080/-/user/org.couchdb.user:ingwiephoenix
npm ERR! registry error parsing json
npm ERR! Darwin 15.3.0
npm ERR! argv "/usr/local/bin/node" "/usr/local/bin/npm" "login"
npm ERR! node v5.7.0
npm ERR! npm  v3.7.3

npm ERR! Unexpected token <
npm ERR! <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
npm ERR! <html><head>
npm ERR! <title>400 Bad Request</title>
npm ERR! </head><body>
npm ERR! <h1>Bad Request</h1>
npm ERR! <p>Your browser sent a request that this server could not understand.<br />
npm ERR! </p>
npm ERR! </body></html>
npm ERR! (...)

Installed version is: 1.5.7

So...how do I log in?

ECONNRESET, ETIMEDOUT

seeing this occasionally:

fullfat hit an error
{ [Error: socket hang up] code: 'ECONNRESET' }

Configuration issues?

$ local-npm

Welcome!
To start using local-npm, just run: 

  $ npm set registry http://127.0.0.1:5080

To switch back, you can run: 

  $ npm set registry https://registry.npmjs.org

A simple npm-like UI is available here: http://127.0.0.1:5080/_browse

PouchDB Server listening on port 16984.
Navigate to http://localhost:16984/_utils for the Fauxton UI.

error doing logger.info() on https://skimdb.npmjs.com/registry
Error: getaddrinfo ENOTFOUND skimdb.npmjs.com
error doing logger.info() on https://skimdb.npmjs.com/registry
Error: getaddrinfo ENOTFOUND skimdb.npmjs.com
error doing logger.info() on https://skimdb.npmjs.com/registry
Error: getaddrinfo ENOTFOUND skimdb.npmjs.com
error doing logger.info() on https://skimdb.npmjs.com/registry
Error: getaddrinfo ENOTFOUND skimdb.npmjs.com
...
error doing logger.info() on https://skimdb.npmjs.com/registry
{"status":503,"name":"pound_service_unavailable","message":"Database encountered an unknown error"}
GET /lodash 200 573.090 ms - -
error doing logger.info() on https://skimdb.npmjs.com/registry
{"status":503,"name":"pound_service_unavailable","message":"Database encountered an unknown error"}
...

go offline and reinstalling lodash

Error: getaddrinfo ENOTFOUND skimdb.npmjs.com
error doing logger.info() on https://skimdb.npmjs.com/registry
Error: getaddrinfo ENOTFOUND skimdb.npmjs.com
error doing logger.info() on https://skimdb.npmjs.com/registry
Error: getaddrinfo ENOTFOUND skimdb.npmjs.com
stream.js:94
      throw er; // Unhandled stream error in pipe.
            ^
Error: getaddrinfo ENOTFOUND registry.npmjs.org
    at errnoException (dns.js:44:10)
    at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:94:26)
$

I guess that it is not intended to work offline neither with intermitent network connection, right?

A user-local version that uses the npm cache as its db?

I wonder if anyone has given any thought to having an entirely user-local version that uses ~/.npm to store data. With something like that, npm could safely use --cache-min=Infinity and avoid even talking to the proxy unless it needed something not cached.

It looks like this will just involve hooking the pouchdb changes feed to update metadata and making it possible to swap out leveldb for something that talks to the npm cache.

Thoughts? Would you all be interested in this as a PR?

publish not working unless you're already logged in

[email protected]

first time running publish i get ENEEDAUTH
run npm adduser
fill out stuff
get the following

npm ERR! Unexpected token C
npm ERR! Cannot PUT /-/user/org.couchdb.user:mattkrick

currently switching back whenever i need to publish, which isn't great, but sure beats the heck out of waiting 5 mins to run install thanks to npm3 ๐Ÿ‘

Can't use when offline

It seems I haven't read much about how to use this tools. So, I were following Addy Osmani article https://addyosmani.com/blog/using-npm-offline. I run with local-npm -d ~/.local-npm then I follow this guide:

  • Check your network connection is up
  • Run an npm install to grab a module or collection of dependencies from a package.json file. For example, lodash.
  • At this point, local-npm will have a cached version of these modules stored
  • Clear the npm cache using npm cache clean or delete the modules from your node_modules directory
  • Turn off your wifi/kill your network connection
  • Try running that npm install again. Everything should install correctly through the local-npm cache without hitting the network at all

When I turn off my wifi, I got this..

stream.js:74
      throw er; // Unhandled stream error in pipe.
      ^

Error: getaddrinfo ENOTFOUND registry.npmjs.org registry.npmjs.org:443
    at errnoException (dns.js:26:10)
    at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:77:26)

If I start local-npm when I'm offline, I got this message..

Error: getaddrinfo ENOTFOUND skimdb.npmjs.com skimdb.npmjs.com:443
Error fetching info() from https://skimdb.npmjs.com/registry, retrying after 1100 ms...

And when I try to install a package..

stream.js:74
      throw er; // Unhandled stream error in pipe.
      ^

Error: getaddrinfo ENOTFOUND registry.npmjs.org registry.npmjs.org:443
    at errnoException (dns.js:26:10)
    at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:77:26)

Then local-npm just stopped.

Is this the correct behaviour?
I thought it should be working when I'm offline.

Thanks.

crashes when browsing in fauxton

cdieringer@Snapper-osx:~/node/coinstac$ npmo # alias to "(cd ~/.local-npm && local-npm)"

Welcome!
To start using local-npm, just run:

  $ npm set registry http://127.0.0.1:5080

To switch back, you can run:

  $ npm set registry https://registry.npmjs.org

A simple npm-like UI is available here: http://127.0.0.1:5080/_browse

PouchDB Server listening on port 16984.
Navigate to http://localhost:16984/_utils for the Fauxton UI.


Replicating skimdb, last_seq is: 646278 (100%)
GET /_utils 301 18.618 ms - 72
GET /_utils/ 200 10.739 ms - -
GET /_utils/css/index-6688cd4426ead40b085270f8cb007530.css 200 6.072 ms - -
GET /_utils/js/require-34a7e370ea98389d118cc328682c0939.js 200 4.591 ms - -
GET /_session 200 4.187 ms - 140
GET /_session 304 4.046 ms - -
GET / 200 6.428 ms - 149
GET /_utils/img/pouchdb-site.png 200 16.347 ms - 4155
GET /_utils/fonts/fontawesome-webfont.woff?v=3.2.1 200 5.596 ms - 43572
GET /_utils/fonts/fauxtonicon.woff 200 5.308 ms - 10368
GET /_all_dbs?_=1452717931519 200 7.709 ms - 34
GET /_utils/js/zeroclipboard/ZeroClipboard.swf?noCache=1452717931887 200 4.012 ms - 2157
GET /_replicator 200 17.624 ms - 184
GET /_users 200 18.950 ms - 179
GET /skimdb 200 16.862 ms - 194
GET /skimdb/_all_docs?limit=21 200 18.113 ms - -
GET /_all_dbs 200 8.482 ms - 34
GET /skimdb/_all_docs?startkey=%22_design%2F%22&endkey=%22_design0%22&include_docs=true&limit=501 200 19.206 ms - -
GET /skimdb/_all_docs?startkey=%22_design%2F%22&endkey=%22_design0%22&include_docs=true&limit=501 304 10.355 ms - -
GET /skimdb/_all_docs?limit=101 200 21.521 ms - -
GET /skimdb/_index? 200 8.383 ms - 97
GET /_utils/img/couch-watermark.png 200 5.570 ms - 7053
GET /_utils/js/ace/mode-javascript.js 200 4.806 ms - -
GET /_utils/js/ace/theme-idle_fingers.js 200 3.044 ms - -
GET /_utils/js/ace/worker-javascript.js 200 4.611 ms - -
GET /_all_dbs 304 4.169 ms - -
GET /_active_tasks 200 7.776 ms - 3
GET /_active_tasks 304 3.136 ms - -
GET /_active_tasks 304 4.686 ms - -
GET /_all_dbs?_=1452717931520 200 399.420 ms - 34
GET /_replicator 200 4155.618 ms - 184

<--- Last few GCs --->

  113602 ms: Scavenge 1393.9 (1458.0) -> 1393.9 (1458.0) MB, 0.2 / 0 ms (+ 2.1 ms in 1 steps since last GC) [allocation failure] [incremental marking delaying mark-sweep].
  114852 ms: Mark-sweep 1393.9 (1458.0) -> 1391.4 (1458.0) MB, 1249.4 / 0 ms (+ 3.1 ms in 2 steps since start of marking, biggest step 2.1 ms) [last resort gc].
  115911 ms: Mark-sweep 1391.4 (1458.0) -> 1365.3 (1458.0) MB, 1059.8 / 0 ms [last resort gc].


<--- JS stacktrace --->

==== JS stack trace =========================================

    2: arguments adaptor frame: 2->3
Security context: 0x14d6ffeb4629 <JS Object>
    3: /* anonymous */ [/usr/local/lib/node_modules/local-npm/node_modules/compression/index.js:87] [pc=0x177d4c80e1d2] (this=0x17b3bbb7f19 <a ServerResponse with map 0x26f8641338b1>,chunk=0x17b3bbc0769 <an Uint8Array with map 0x26f8641337a9>,encoding=0x14d6ffe041b9 <undefined>)
    4: write [/usr/local/lib/node_modules/local-npm/node_modules/compression/ind...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - process out of memory

Cache metadata optimistically

I think we could do a get with revisions=true and essentially replicate on-demand (inserting with new_edits=false, as the replicator does). This would mean you wouldn't have to wait for a slow ramp-up time for all the initial replication of the skimdb metadata to complete. See #96 for more info.

Error "Could not locate the bindings file"

node-gyp v3.2.1
node.js v5.3.0
npm v3.5.3
prebuild v2.8.1

When I run local-npm I get the following error:

C:\Users\david.brower\AppData\Roaming\npm\node_modules\local-npm\node_modules\bi
ndings\bindings.js:91
throw err
^

Error: Could not locate the bindings file. Tried:
โ†’ C:\Users\david.brower\AppData\Roaming\npm\node_modules\local-npm\node_modules
\leveldown\build\leveldown.node
โ†’ C:\Users\david.brower\AppData\Roaming\npm\node_modules\local-npm\node_modules
\leveldown\build\Debug\leveldown.node
โ†’ C:\Users\david.brower\AppData\Roaming\npm\node_modules\local-npm\node_modules
\leveldown\build\Release\leveldown.node
โ†’ C:\Users\david.brower\AppData\Roaming\npm\node_modules\local-npm\node_modules
\leveldown\out\Debug\leveldown.node
โ†’ C:\Users\david.brower\AppData\Roaming\npm\node_modules\local-npm\node_modules
\leveldown\Debug\leveldown.node
โ†’ C:\Users\david.brower\AppData\Roaming\npm\node_modules\local-npm\node_modules
\leveldown\out\Release\leveldown.node
โ†’ C:\Users\david.brower\AppData\Roaming\npm\node_modules\local-npm\node_modules
\leveldown\Release\leveldown.node
โ†’ C:\Users\david.brower\AppData\Roaming\npm\node_modules\local-npm\node_modules
\leveldown\build\default\leveldown.node
โ†’ C:\Users\david.brower\AppData\Roaming\npm\node_modules\local-npm\node_modules
\leveldown\compiled\5.3.0\win32\x64\leveldown.node
at bindings (C:\Users\david.brower\AppData\Roaming\npm\node_modules\local-np
m\node_modules\bindings\bindings.js:88:9)
at Object. (C:\Users\david.brower\AppData\Roaming\npm\node_module
s\local-npm\node_modules\leveldown\leveldown.js:4:46)
at Module._compile (module.js:398:26)
at Object.Module._extensions..js (module.js:405:10)
at Module.load (module.js:344:32)
at Function.Module._load (module.js:301:12)
at Module.require (module.js:354:17)
at require (internal/module.js:12:17)
at Object. (C:\Users\david.brower\AppData\Roaming\npm\node_module
s\local-npm\node_modules\level\level.js:1:106)
at Module._compile (module.js:398:26)
at Object.Module._extensions..js (module.js:405:10)
at Module.load (module.js:344:32)
at Function.Module._load (module.js:301:12)
at Module.require (module.js:354:17)
at require (internal/module.js:12:17)
at Object. (C:\Users\david.brower\AppData\Roaming\npm\node_module
s\local-npm\lib\index.js:10:13)

Add tests

It's downright embarrassing not to have any. This tool has grown enough (in codebase size and popularity) that it deserves tests.

Probably we can use a combination of pouchdb-server and pouchdb-load to avoid having to replicate all of npm just to run the tests. Running the tests on a subset should be fine.

Not reconnecting after offline

Hey!

Thanks for local-npm, love it!
However, I've had some problems that when I've went offline for a while I'm behind the main registry and it doesn't seem to reconnect to get up to speed again...
I've tried restarting the local-npm process without any success. This is, using the OS X launch daemon (https://gist.github.com/nolanlawson/83ba5862bd719925d9cd).

Any tips on what I should do to get this working?

Restore Original npm Registry in README?

This is awesome! Will you add a small note to the README on how to restore the original npm registry? I'm sure it's trivial, but that was the first thing that came to mind after reading the README.

Crash on timeout errors when on slow connection

Hey,
The package is great and helps a lot especially on very slow connections, but on big installs local-npm will crash a few times as a result of Timeouts with the following error -
events.js:141
throw er; // Unhandled 'error' event
^

Error: connect ETIMEDOUT 199.27.79.162:443
at Object.exports._errnoException (util.js:856:11)
at exports._exceptionWithHostPort (util.js:879:20)
at TCPConnectWrap.afterConnect as oncomplete

In cases like this 'npm install' will keep on running and restarting local-npm will allow for it to continue installation. But unless I am watching local-npm I sometimes miss the crash and the time for install gets longer.

Changing the way local-npm works to issue a warning but continue retrying will be a fix for this.

Thanks

npm install node-sass fails with "No compatible version found"

I get the same error for both, npm install node-sass and npm install node-sass@latest

$ npm i node-sass@latest
npm ERR! Darwin 15.3.0
npm ERR! argv "/Users/gregor/.nvm/versions/node/v4.2.2/bin/node" "/Users/gregor/.nvm/versions/node/v4.2.2/bin/npm" "i" "node-sass@latest"
npm ERR! node v4.2.2
npm ERR! npm  v3.5.2

npm ERR! No compatible version found: node-sass@latest
npm ERR! Valid install targets:
npm ERR! 3.5.0-beta.1, 3.4.2, 3.4.1, 3.4.0, 3.4.0-beta1, 3.4.0-beta.2, 3.3.3, 3.3.2, 3.3.1, 3.3.0, 3.2.0, 3.1.2, 3.1.1, 3.1.0, 3.0.0, 3.0.0-beta.7, 3.0.0-beta.5, 3.0.0-beta.4, 3.0.0-beta.3, 3.0.0-beta.2, 3.0.0-alpha.0, 2.1.1, 2.1.0, 2.0.1, 2.0.0, 2.0.0-beta, 1.2.3, 1.2.2, 1.2.1, 1.2.0, 1.1.4, 1.1.3, 1.1.2, 1.1.1, 1.1.0, 1.0.3, 1.0.2, 1.0.2-alpha, 1.0.1, 1.0.0, 0.9.6, 0.9.5, 0.9.5-rc1, 0.9.4, 0.9.4-rc1, 0.9.3, 0.9.2, 0.9.1, 0.9.0, 0.8.6, 0.8.5, 0.8.4, 0.8.3, 0.8.2, 0.8.1, 0.8.0, 0.7.0, 0.7.0-alpha, 0.6.7, 0.6.6, 0.6.5, 0.6.4, 0.6.3, 0.6.2, 0.6.1, 0.6.0, 0.5.4, 0.5.3, 0.5.2, 0.5.1, 0.5.0, 0.4.4, 0.4.3, 0.4.2, 0.4.1, 0.4.0, 0.3.0, 0.2.6, 0.2.5, 0.2.4, 0.2.3, 0.2.2, 0.2.1, 0.2.0
npm ERR! 
npm ERR! 
npm ERR! If you need help, you may report this error at:
npm ERR!     <https://github.com/npm/npm/issues>

npm ERR! Please include the following file with any support request:
npm ERR!     /private/tmp/npm-debug.log

Outputh with loglevel=silly: https://gist.github.com/f11dfb3b2c3e2b04d480

The issue does not occur when using the standard registry (npm set registry https://registry.npmjs.org).

This works: npm install [email protected]

I've updated local-npm to the latest version

$ local-npm --version
1.5.7

All packages are synced (Replicating skimdb, last_seq is: 768417 (100%))

Publish private modules locally

It would be great if it was possible to publish a private repository to local-npm. I.e. when private: true is set in package.json and one runs npm publish it will no be published to npm but it still can be installed locally via local-npm.

Doesn't read proxy settings from .npmrc

It's more a question:

Is it transparent using ~/.npmrc for proxy settings when doing the replication?

Cuz I see those

error doing logger.info() on https://skimdb.npmjs.com/registry
Error: ETIMEDOUT
error doing logger.info() on https://skimdb.npmjs.com/registry
Error: ETIMEDOUT
error doing logger.info() on https://skimdb.npmjs.com/registry
Error: ETIMEDOUT
error doing logger.info() on https://skimdb.npmjs.com/registry
Error: ETIMEDOUT
error doing logger.info() on https://skimdb.npmjs.com/registry

While my cmd line proxy and .npmrc are both setup properly.

markdown compatibility

I was looking at http://127.0.0.1:5080/_browse/#/local-npm and the "How it works" section tells of the http://127.0.0.1:16984/_utils entry point - but because the markdown parser used seems a little bit flaky (I haven't investigated any further yet) the url http://127.0.0.1:16984/utils is shown instead with a bit in italic (because of the two underline characters. Compare the output with https://github.com/nolanlawson/local-npm#how-it-works. This is just one example.

What markdown library is used in the deps? Perhaps a bit of config can fix its output.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.