garbados / comdb Goto Github PK

This is related to the excellent work in #28.

As part of a software update, I have a use case where I want to administratively deploy new data to a remote database that is fully encrypted by comdb. The resulting database will have a mix of encrypted and unencrypted data. I would like the unencrypted data to be encrypted upon syncing with user client application (without double encrypted existing data).

Hi,

tanks for this awesome lib!

I am trying to get the e2e encryption working in an angular 10 app. Just like you showed it here:
#4

But i have trouble with the pouchdb-adapter-memory.

When I import it as this:

import PouchDB from 'pouchdb'; // Works
import PouchAuth from 'pouchdb-authentication'; // Works
import comdb from 'comdb'; // Works
import PouchMemory from 'pouchdb-adapter-memory'; // Does not

PouchDB.plugin(PouchAuth);
PouchDB.plugin(comdb);
PouchDB.plugin(PouchMemory);

I get this at app start:

Error: ./node_modules/sublevel-pouchdb/node_modules/readable-stream/readable.js
Module not found: Error: Can't resolve 'stream' in 'C:\source\ngcp\node_modules\sublevel-pouchdb\node_modules\readable-stream
'``

Then I tried to import this: 

import PouchMemory from 'pouchdb/dist/pouchdb.memory';

and got

memory adapter plugin error: Cannot find global "PouchDB" object! Did you remember to include pouchdb.js?
core.js:6210 ERROR Error: Invalid plugin: got "[object Object]", expected an object or a function

I simply don't get it. Do you have any idea? I would be very thankful.
Btw. what I actually try to accomplish in the end is storing data encrypted on the client while having the decrypted version on the server. :-)

Encrypted documents created with .post() fail to load using .loadEncrypted() if a version of that document already exists on the decrypted side. You can replicate this issue in a browser by running this code:

const db = new PouchDB('local')
db.setPassword(PASSWORD).then(async () => {
  await db.loadEncrypted()
  await db.post({ name: 'bob' })
})

This will work the first time. But if you run this in a browser and then reload the page, it will fail with a message like this:

Say you lose your password. ComDB doesn't store your password and can only verify it, so we can't provide it if you lose it. Unless we do store it when asked to, in an encrypted way.

Consider a method such as .addSecurityQuestion(name, question, answer). Using a hash of the answer as a password, we then create a new Crypt instance to encrypt the user's password. This encrypted value and the Crypt instance's export string are then stored in the _local/comdb document like this:

{
  _id: '_local/comdb',
  _rev: '...',
  security_questions: {
    house: {
      question: 'In the house you grew up in, what is buried in the garden?',
      exportString: '...',
      payload: '...'
    }
  }
}

A matching db.removeSecurityQuestion(name) would remove the specified security question from _local/comdb.

Thus, a user can retrieve their password by answering exactly any one security question. The user must only ever opt-in to this type of password protection!

I am experiencing weird behaviour with a database on my node.js app. I am using memdown and comdb to have in-memory database that stores encrypted copy to disk.

I am trying to write timestamp to database on startup.

If I store the timestamp like this it always stays the same and does not update.:

PouchDB.plugin(comdb);
const InMemPouchDB = PouchDB.defaults({
 db: memdown
});

const start = async () => {
 const key = 'timestamp';
 const secret = 'random_secret';

 await db.setPassword(secret, {
   name: 'encrypted-db'
 });

 await db.loadEncrypted();

 try {
   const doc = await db.get(key);
   await db.put({
     ...doc,
     timestamp: now.toISOString()
   });
 } catch (err) {
   await db.put({
     _id: documentKey,
     timestamp: now.toISOString()
   });
 }
};

No error is raised and if I console.log the document after the db.put call it shows up updated but when the app is started again the doc has the old value and revision that it had when it was first stored to database.

However if I get all docs and put them back to database without modifying them (in this case there is only one doc, but if there is more than one they all need to be put back to database before they start updating) the put call start to update the timestamp correctly.

PouchDB.plugin(comdb);
const InMemPouchDB = PouchDB.defaults({
 db: memdown
});

const start = async () => {
 const key = 'timestamp';
 const secret = 'random_secret';

 await db.setPassword(secret, {
   name: 'encrypted-db'
 });

 await db.loadEncrypted();
 
 const docs = await db.allDocs({ include_docs: true });
 await db.bulkDocs(docs.rows.map(row => row.doc));

try {
   const doc = await db.get(key);
   await db.put({
     ...doc,
     timestamp: now.toISOString()
   });
 } catch (err) {
   await db.put({
     _id: documentKey,
     timestamp: now.toISOString()
   });
 }
};

Any idea what could cause this behaviour?

When generating the id for the encrypted record it looks like you are hashing all the data in the record:

https://github.com/garbados/comdb/blob/master/index.js#L101-L103

I understand the reason for needing the ID to be deterministic but why use the hash of the entire record? It seems if you change any data on a record that would create a new record vs doing a revision on the previous record. I'm unsure what impact this has on syncing but at the very least it means compaction won't work as from PouchDB's perspective those are two different records rather than the current record and a previous record.

Would it be perhaps better to just hash the cleartext ID to create an obfuscated deterministic ID for the encrypted record? If you were worried about someone determining the cleartext ID (which might contain sensitive data since PouchDB encourages intelligent keys) via rainbow table or something you could always generate a per-record salt to be stored with the encrypted data.

Just wanted to inquire to make sure I'm not missing anything obvious on why that is a bad idea.

Given code like the following, running in a web page:

const PouchDB = require('pouchdb')
PouchDB.plugin(require('pouchdb-adapter-memory'))
PouchDB.plugin(require('comdb'))

const PASSWORD = 'blahzeblaht'
const db = new PouchDB('local', { adapter: 'memory' })
db.setPassword(PASSWORD).then(async () => {
  await db.loadEncrypted()
  await db.post({ name: 'bob' })
})

This will work the first time, but subsequent runs will encounter a "Could not decrypt!" error. This is because ComDB stores an exportString in a local document in the decrypted database, which it uses along with the given password to decrypt the encrypted database. Because the in-memory database is wiped each time the program ends, this exportString value is also wiped, essentially orphaning all your encrypted data. This means ComDB does not currently work with an in-memory adapter, pending architectural changes.

Hello @garbados , Diana, excellent work you have done here.

I wonder how difficult would be to have End to End Encryption, that way information is safe on both sides. WIth State Management Containers like Redux or Vuex you can have on Ram decrypted information.

I wonder if this makes sense to you.

Once again, thanks for the great job!!

The plugin looks good.

It should be possible for a user to reset their password and rotate their encryption keys. They might do this to lock out a device that has become compromised.

The process requires the old and new passwords. Given those, each document in the encrypted copy is re-encrypted using the new password and key. The executor of this rotation sets a rotating property on the _local/comdb document, and then unsets it once the rotation completes. The database will be in an intermediate state during this period; applications that encounter "Could not decrypt!" errors at this time should prompt the user for the new password and then wait for the _local/comdb) document to lose the rotating property.

Hi,

I thought to have understood this nice project, but somehow...

I'm working local in my js app and I simply want to keep only the decrypted database in indexed db when the app isn't used.

So I do

const db = new PouchDB('test');
db.setPassword('my');

now working with the app -> when closing:

db.loadDecrypted();
db.destroy({ unencrypted_only: 'true' });`

When returning

const db = new PouchDB('test');
db.setPassword('my');
db.loadEncrypted();

This is working fine until there are attachments in the db.
Then it fails saying

"Failed to execute 'readAsArrayBuffer' on 'FileReader': parameter 1 is not of type 'Blob'."
Any help would be very appreciated!

Best Regards

Transform-Pouch is broken against CouchDB, and as a result this occurs when running the CouchDB example in this repository:

$ node examples/couchdb.js 
...
TypeError: Cannot read property 'doc' of undefined
    at /home/garbados/code/comdb/examples/couchdb.js:40:12
    at processTicksAndRejections (node:internal/process/task_queues:96:5)

There is a pending fix here. The dependency in this project will need to be updated as soon as that patch lands.

Now that #21 is complete, I'm SO excited for this!

One issue is that I have many remote DBs with encrypted data on them. Because of the way replication works, I'm not sure how to encrypt the data in place. Do you have any suggestions?

Edit: (I MEANT to say DBs with unencrypted data... this confusion was resolved below)