Git Product home page Git Product logo

Comments (9)

remy avatar remy commented on May 19, 2024

Doesn't seem to be linked to the event listeners, but the 503 causes the replication to reset. I'm not sure if that's normal or not...

More logs:

Navigate to http://localhost:16984/_utils for the Fauxton UI.

Replicating skimdb, last_seq is: 1 (0%)
Replicating skimdb, last_seq is: 2 (0%)
Replicating skimdb, last_seq is: 200 (0.11%)
Replicating skimdb, last_seq is: 401 (0.23%)
Replicating skimdb, last_seq is: 601 (0.35%)
Replicating skimdb, last_seq is: 801 (0.47%)
Replicating skimdb, last_seq is: 1001 (0.58%)
Replicating skimdb, last_seq is: 1201 (0.7%)
Replicating skimdb, last_seq is: 1401 (0.82%)
Replicating skimdb, last_seq is: 1601 (0.93%)
Replicating skimdb, last_seq is: 1801 (1.05%)
Replicating skimdb, last_seq is: 2001 (1.17%)
Replicating skimdb, last_seq is: 2201 (1.29%)
Replicating skimdb, last_seq is: 2401 (1.4%)
Replicating skimdb, last_seq is: 2601 (1.52%)
Replicating skimdb, last_seq is: 2801 (1.64%)
error during replication with skimdb
{"status":503,"name":"pound_service_unavailable","message":"Database encountered an unknown error"}
Replicating skimdb, last_seq is: 1 (0%)
Replicating skimdb, last_seq is: 2 (0%)
Replicating skimdb, last_seq is: 200 (0.11%)
Replicating skimdb, last_seq is: 401 (0.23%)
Replicating skimdb, last_seq is: 601 (0.35%)
Replicating skimdb, last_seq is: 801 (0.47%)
Replicating skimdb, last_seq is: 1001 (0.58%)
Replicating skimdb, last_seq is: 1201 (0.7%)
error during replication with skimdb
{"status":503,"name":"pound_service_unavailable","message":"Database encountered an unknown error"}
Replicating skimdb, last_seq is: 1 (0%)
Replicating skimdb, last_seq is: 2 (0%)
Replicating skimdb, last_seq is: 200 (0.11%)
Replicating skimdb, last_seq is: 401 (0.23%)
Replicating skimdb, last_seq is: 601 (0.35%)

from local-npm.

nolanlawson avatar nolanlawson commented on May 19, 2024

I think the reason the replication is resetting is because the target database is read-only. We have a longstanding issue where we need to write _local checkpoints to the remote database in order to keep track of our progress in replicating from that database.

As for the EventEmitter warning, that's definitely a bug that I thought we had fixed awhile ago. Filed: pouchdb/pouchdb#4093

from local-npm.

daleharvey avatar daleharvey commented on May 19, 2024

If the database is read only then we should fallback to just relying on the local checkpoint (https://github.com/pouchdb/pouchdb/blob/master/lib/replicate/checkpointer.js#L101) as long as the error we get back is as expected, is that changing somehow?

from local-npm.

remy avatar remy commented on May 19, 2024

There's definitely something super wrong.

I left local-npm running and it go to 100%, then when restarting and catching up with changes it randomly drops back down to zero:

...
Error: ETIMEDOUT
error during replication with skimdb
Error: ETIMEDOUT
error doing logger.info() on https://skimdb.npmjs.com/registry
Error: ETIMEDOUT
Replicating skimdb, last_seq is: 175769 (98.82%)
Replicating skimdb, last_seq is: 175771 (98.83%)
error during replication with skimdb
Error: ETIMEDOUT
error doing logger.info() on https://skimdb.npmjs.com/registry
Error: ETIMEDOUT
Replicating skimdb, last_seq is: 1 (0%)
Replicating skimdb, last_seq is: 2 (0%)
Replicating skimdb, last_seq is: 200 (0.11%)
Replicating skimdb, last_seq is: 401 (0.22%)
Replicating skimdb, last_seq is: 402 (0.22%)
Replicating skimdb, last_seq is: 403 (0.22%)
Replicating skimdb, last_seq is: 601 (0.33%)
Replicating skimdb, last_seq is: 602 (0.33%)
Replicating skimdb, last_seq is: 603 (0.33%)
...

from local-npm.

nolanlawson avatar nolanlawson commented on May 19, 2024

Based on the output it seems like you are not using the latest version. Could you update to 1.4.2 and try again?

from local-npm.

nolanlawson avatar nolanlawson commented on May 19, 2024

Oh shoot my bad, excuse me. I opened a PR and then never merged it. I'll publish 1.4.3 right now.

from local-npm.

nolanlawson avatar nolanlawson commented on May 19, 2024

Should be fixed by #41, please reopen if it's not.

from local-npm.

remy avatar remy commented on May 19, 2024

Sorted.

I've seen all of the view docs in pouch, is it possible to query this local-npm using those desig docs? i.e. find all the modules that depend on semver for instance?

from local-npm.

nolanlawson avatar nolanlawson commented on May 19, 2024

Yeah, that's actually a really neat idea. In the Fauxton UI (http://localhost:16984/_utils) I can see a ton of secondary indexes that are already in that _design/app ddoc, but since it's a replicated CouchDB, you can also add whatever design docs you want, and then query them.

As a test, I did this:

curl http://localhost:16984/skimdb/_design/by_maintainer -X PUT -H 'Content-Type: application/json' --data-binary '{"_id":"_design/by_maintainer","views":{"by_maintainer":{"map":"function (doc) {\n  if (doc.maintainers) {\n    for (var i = 0; i < doc.maintainers.length; i++) {\n      var email = doc.maintainers[i].email;\n      if (email) {\n        emit(email);  \n      }\n    }\n  }\n}"}},"language":"javascript"}'
curl 'http://localhost:16984/skimdb/_design/by_maintainer/_view/by_maintainer?key=%[email protected]%22&limit=5'

The secondary index is slow to build, but it works! Prints out:

{"total_rows":203396,"offset":0,"rows":[{"key":"[email protected]","id":"add-cors-to-couchdb","value":null},{"key":"[email protected]","id":"blob-util","value":null},{"key":"[email protected]","id":"chord-magic","value":null},{"key":"[email protected]","id":"couchdb-harness","value":null},{"key":"[email protected]","id":"debug-websql","value":null}]}

For querying modules that depend on semver, I guess you would need to emit all dependency versions or maybe just emit if any version numbers start with a ^ or a ~. The Fauxton UI is actually pretty nice for editing these kinds of ddocs; highly recommended.

from local-npm.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.