@skoni @tlvince Could you please confirm that this is open sourcable?

include https://github.com/jo/couchdb-bulk

Include https://github.com/jo/couchdb-create-index

We have a few projects where we make use of PouchDB’s different adapters (level and http in our case) to run software in different environments. We also use couchdb-bootstrap, which doesn’t work with PouchDB instances itself, but only a CouchDB server.

In theory, would this project (and the sub-projects it is built from) accept a set of PRs that add support for PouchDB instances instead of just URLs that are passed to nano?

A first implementation could do if/else switches at the various places where they are needed, but one could argue that once you use PouchDB and since it can use a CouchDB URL using the http adapter, wouldn’t it make sense to just stop using nano here. This question doesn’t ask to do the full replacement, just additional support for bootstrapping a PouchDB instance, but a wholesale transition might be less effort.

It’d be totally fine to say “no, out of scope” :)

@jo

Hi,

I just went from 1.14.0 to 14.2.0 and noticed that when I have an empty folder in couchdb there used to be a DB created for it.

This should already work, needs tests/examples.

How do I use configuration object for _config.json? I want to use couchdb-bootstrap to initialise a new installation of couchdb so configuration are always the same.

And also a shorter executable or a short alias.

I need to create a partitioned database, with the structure being defined using the 'CouchDB Filesystem Mapping'. I can't find any specific documentation showing an example of how this would be represented, but my guess has been to create a file in the folder representing the database called partitioned containing the value true (following the example in https://github.com/jo/couchdb-compile/tree/main/test/fixtures/full for specifying the language property).

However, if I then run couchdb-bootstrap via the CLI, I get:

C:\Dvt\devops\node_modules\couchdb-bootstrap\index.js:130
              if (!('_id' in doc)) doc._id = id
                          ^

TypeError: Cannot use 'in' operator to search for '_id' in true
    at C:\Dvt\devops\node_modules\couchdb-bootstrap\index.js:130:27
    at Array.reduce (<anonymous>)
    at C:\Dvt\devops\node_modules\couchdb-bootstrap\index.js:103:14
    at C:\Dvt\devops\node_modules\async\dist\async.js:243:13
    at eachOfArrayLike (C:\Dvt\devops\node_modules\async\dist\async.js:500:13)
    at eachOf (C:\Dvt\devops\node_modules\async\dist\async.js:551:16)
    at awaitable (C:\Dvt\devops\node_modules\async\dist\async.js:208:32)
    at _asyncMap (C:\Dvt\devops\node_modules\async\dist\async.js:241:16)
    at Object.map (C:\Dvt\devops\node_modules\async\dist\async.js:594:16)
    at Object.awaitable [as map] (C:\Dvt\devops\node_modules\async\dist\async.js:208:32)

So it looks like it's trying to interpret the file contents as a JSON document, rather than a property value.

Does couchdb-bootstrap (15.0.0) support creating partitioned databases and if so, what is the required file/folder structure to represent that?

Hi,

I installed the program locally using npm install couchdb-bootstrap and then ran it using npx couchdb-bootstrap http://localhost:5984 (the database folders were in the root of the project). The program descended into the node_modules and executed the files in
the test directory. I guess that is the expected behavior, however it took me a good while to figure out what the hell was going on (I got an error Cannot set property 'alice' of undefined, there is no alice in any of my databases). I then fixed the problem by moving the database folders to a separate directory. Can you maybe mention something in the documentation?

jo/couchdb-configure#5

Let's say I want to define slightly different configs for my dev and prod couches. From the docs it looks like this should work:

// _config.js OR _config/index.js
module.exports = {
  "vhosts": (function (env) {
    return env === 'production'
       ? { "mydb.production-couch.com": "/mydb/_design/app/_rewrite" }
       : { "mydb.localhost:5984": "/mydb/_design/app/_rewrite" };
  })(process.env.NODE_ENV)
}

However, only _config.json seems to be recognized. Am I doing it wrong?

Hi there,

First, thanks for the tools you made, that helped me a lot :)

I am having trouble to use couchdb-bootstrap with CommonJS modules though. I would like to be able to write any view, filter or validation function in a module, so that i can unit test it. I am sure this is possible, as you mention that in the README.

My problem is that the generated _design documents are not as i would expect them to be.

It might be related to #49

I have to mention that i am pretty new with CouchDB, so i probably might be missing something obvious...

Here is the structure i am trying to use (only a small relevant part):

├── package.json
├── bin
│   └── bootstrap.js
├── tests
│   └── // Here i will test my CJS modules with Jest
└── src
    ├── _users
    │   ├── alice.json
    │   └── bob.json
    ├── official-db
    │   ├── _design
    │   │   └── validation
    │   │       ├── index.js
    │   │       └── validate_doc_update.js
    │   ├── _security.json
    │   └── adoc.json
    ├── alice-db
    │   └── _security.json
    └── bob-db
        └── _security.json

I run the bootstrap process with a custom script (bin/bootsrap.js) which basically only executes your module with my CouchDB server admin credentials, passes the index = true option, and generate a report of what have been done. (I don't think the problem comes from that script, so i'm not including it in this issue)

Following the README, here is the content of my official-db/_design/validation folder :

// index.js
module.exports = {
  validate_doc_update: require("./validate_doc_update")
};

// validate_doc_update.js
module.exports = function(newDoc, oldDoc, userCtx, secObj) {
  throw { forbidden: "Nobody can update anything here !!" }; // this is only pseudo code to test :)
};

In the official-db database, i would have expected the validation design document to be like :

// offical-db > _design/validation
{
  "_id": "_design/validation",
  "validate_doc_update": "function(newDoc, oldDoc, userCtx, secObj) {\n  throw { forbidden: \"Nobody can update anything here !!\" };\n};"
}

But instead, i end up with that :

// offical-db > _design/validation
{
  "_id": "_design/validation",
  "index": "module.exports = {\n  validate_doc_update: require(\"./validate_doc_update\")\n};",
  "validate_doc_update": "module.exports = function(newDoc, oldDoc, userCtx, secObj) {\n  throw { forbidden: \"Nobody can update anything here !!\" };\n};"
}

My main concern is that the module.export is not stripped out, so the database is "broken" with the error Save failed: Expression does not eval to a function.

If i run the command couchdb-compile src --index, the problem remains, but if i couchdb-compile src/official-db/_design/validation --index, everything works :

couchdb-compile src/official-db/_design/validation --index

# outputs
{
  "_id": "_design/validation",
  "validate_doc_update": "function(newDoc, oldDoc, userCtx, secObj) {\n  throw { forbidden: \"Nobody can update anything here !!\" };\n};"
}

As far as i understand the code from couchdb-compile, the option index is only used for the root folder, and is not recursive. Shouldn't it be ?

I am sorry for this long issue, but i have been stuck on it for the whole day, and i really need some help... I hope i have provided enough details to get some :) Let me know if you need more infos !

Thanks a lot, regards

include https://github.com/jo/couchdb-create-index

Running a simple cdbb http://localhost:5984 ./couchdb works fine in version 1.14.0, but is completely broken in 14.2.0.

Basically, despite what the docs say, CommonJS index.js modules at best make the cdbb explode, and at worst - create an unexpected document with a single field called index containing its source code, which is totally not what I want.

Project structure:

project
  couchdb
    mydb
      index.js

Contents of project/couchdb/mydb/index.js:

module.exports = { _id: "test", foo: "bar" };

[email protected] runs as expected, creating a mydb/test document.

[email protected] throws an error at index.js:130:

TypeError: Cannot use 'in' operator to search for '_id' in module.exports = {
    "_id": "test",
    "foo": "bar"
};

In my bootstrap folder I'm creating a document in _replicator to replicate from one database setup via bootstrap to another database setup via bootstrap. It seems because _replicator comes first alphabetically it's parsed by couchdb-bootstrap first before the other databases are setup. This is causing the replication document created to fail.

The replicator document shows _replication_state is error and _replication_state_reason is

Couldn't open document _design/myddocname from source database models: {error,<<"not_found">>}

I either have to run setup twice or re-save the replicator document to fix the error.

I'd like to be able to define a bootstrap layout like this:

module.exports = {
  // set custom config option for CouchDB OS Daemons
  // http://docs.couchdb.org/en/2.1.1/config/externals.html
  _config: {
    migrator: {
      max_jobs: 100
    }
  },
  // setup users Alice and Bob
  _users: {
    alice: {
      _id: 'org.couchdb.user:[email protected]',
      type: 'user',
      roles: [],
      name: '[email protected]',
      password: 'secure'
    },
    bob: {
      _id: 'org.couchdb.user:[email protected]',
      type: 'user',
      roles: [],
      name: '[email protected]',
      password: 'secure'
    }
  },
  // replication jobs from template database to user database
  // http://docs.couchdb.org/en/2.1.1/config/couch-peruser.html
  _replicator: {
    // Alice database
    'todo-app-replication@1:todos-template:userdb-616c696365406578616d706c652e636f6d': {
      source: 'https://couchdb.example.com/todos-template',
      target: 'https://couchdb.example.com/userdb-616c696365406578616d706c652e636f6d',
      continuous: true
    },
    // Bobs database
    'todo-app-replication@1:todos-template:userdb-626f62406578616d706c652e636f6d': {
      source: 'https://couchdb.example.com/todos-template',
      target: 'https://couchdb.example.com/userdb-626f62406578616d706c652e636f6d',
      continuous: true
    }
  },
  // template database, holding indices, views and initial document.
  // continuously replicated to user databases
  'todos-template': {
    // define indices for Mango queries
    // http://docs.couchdb.org/en/2.1.1/api/database/find.html
    _index: {
      'todo-app-index-by-group-id' {
        index: {
          partial_filter_selector: {
            schema: 'todo-item',
            version: 3
           },
          fields: ['schema', 'version', 'groupId']
         },
        ddoc: 'todo-app-index@1',
        name: 'index-by-group-id'
      }
    },
    // install design documents
    _design: {
      'todo-app-view@1': {
        views: {
          todoItems: {
            map: function (doc) {
              if (doc._id.slice(0, 6) === 'todo-item@3:') emit(doc._id.slice(6), doc.isDone)
            },
            reduce: '_count'
          }
        }
      }
    },
    // restrict access to template database
    _security: {
      members: {
        roles: [],
        names: ['admin']
      }
    },
    // initial documents
    'todo-item-group@1:default': {
      title: 'Welcome',
      schema: 'todo-item-group',
      version: 1
    },
    'todo-item@3:welcome': {
      title: 'Getting started with this todo app',
      schema: 'todo-item',
      version: 3,
      groupId: 'default'
    },
    // clean up obsolete versions of design documents and indices
    _delete: [
      '[email protected]',
      '[email protected]'
    ]
  },
  // setup a database for Alice
  // http://docs.couchdb.org/en/2.1.1/config/couch-peruser.html
  // userdb-{hex encoded [email protected]}
  'userdb-616c696365406578616d706c652e636f6d': {
    // local docs work well to remember state for services
    // http://docs.couchdb.org/en/2.1.1/api/local.html
    _local: {
      'todo-app-service@1': {
        lastSeq: null,
        status: 'active'
      }
    },
    // only Alice can access db
    _security: {
      members: {
        roles: [],
        names: [
          '[email protected]'
        ]
      },
      // admin is Alice
      admins: {
        roles: [],
        names: [
          '[email protected]'
        ]
      }
    }
  },
  // userdb-{hex encoded [email protected]}
  'userdb-626f62406578616d706c652e636f6d': {
    _local: {
      'todo-app-service@1': {
        lastSeq: null
      }
    },
    // Alice and Bob can access db
    _security: {
      members: {
        roles: [],
        names: [
          '[email protected]',
          '[email protected]'
        ]
      },
      // admin is Bob this time
      admins: {
        roles: [],
        names: [
          '[email protected]'
        ]
      }
    }
  }
}

This is currently not possible, bootstrap functions only accept filenames. Since https://github.com/jo/couchdb-compile/releases/tag/v1.10.0 couchdb-compile supports sources being objects.

Hello.
How can I safety use bootstrap on secured servers (server admins, db admins, users) ?
I can pass to URL http://login:pas@domain but I don't think, this is a good idea ?

In CLI and Node.js ?

Have a watch option which pusheds everytime a file is changed, similar to watchify.

Assuming a CouchDB server root on URL http://localhost/_couchdb
I get the error URL must point to the root of a CouchDB server

It seems it is using nano's url parsing option , dismissing the db part, since that is already configured from the couchdb-boostrap file setup).

If you pass in a nano config as the url, e.g. to set requestDefaults (#14):

bootstrap({
  url: 'http://localhost:5984',
  requestDefaults: {
    auth: {
      user: 'admin',
      pass: 'admin'
    }
  }
}, 'path/to/fs', cb)

… requestDefaults is lost after nano.use, for example in secure:

// ...
var dbname = utils.mapDbName(filename, options)
var db = couch.use(dbname)
console.log('couch: ', couch.config, '\ndb: ', db.config)
// =>
// couch:  { url: 'http://localhost:5984',
//  requestDefaults: { jar: false, auth: { user: 'admin', pass: 'admin' } },
//  defaultHeaders: { 'X-Couch-Full-Commit': 'true' } } 
// db:  { url: 'http://localhost:5984', db: '_users' }

In this case, this will cause subsequent requests to 401 (You are not authorized to access this db.).

Perhaps this is an issue with nano itself (apache/nano#278)?