me-box / core-arbiter Goto Github PK

Post v0.1. A result of last night's discussions with @sevenEng. As it turns out, ocaml-macaroons macaroons (de)serialisation is bugged, in that it doesn't (de)serialise macaroons the same way that libmacaroons, macaroons.js, and others (in the sidebar at http://macaroons.io/) do. This makes ocaml-macaroons incompatible with other macaroon libraries, until they fix this (or we do and PR it). Technically, macaroon specs don't say how it should be done, but the code shows they tried to follow the "standard" set by the other libraries, but it's not consistent. This is a problem.

• The long-term solution is for ocaml-macaroons to be fixed.
• The short-term solution is for the arbiter to make an exception when the target is databox-export-service and use a simpler encoding, such as plain URL-safe base 64, and the export service can manually construct macaroons from this data OR
• We write a shim into the export service that translates from one format to the other
• The super short-term solution is to disable macaroon verification in the export service until after the launch, just so the API is usable at least (where we are now).

An alternative long-term solution is to not depend on ocaml-macaroons at all, and instead put a store in front of the export service (free auth and logging) and have the export service subscribe to anything that goes into it, making the export service a special app instead of a special store. It would need to support WebSockets for this, and stores would need to support wildcard subscriptions.

Config var name needs a refactor (didn't want to do it while you were in the middle of development) since "registry" is reserved by NPM (referring to the registry where NPM packages live) and when I tried setting it, I remember it screwing something up. More details in commit body here: 1eafc67

Edit: I think the issue was because NPM expects it to be an NPM registry URL, it does some kind of validation on it.

Depending on what we do, important lines of code are here:

https://github.com/me-box/databox-store-blob/blob/master/src/lib/macaroon/macaroon-verifier.js#L13
https://github.com/yousefamar/node-databox/blob/master/lib/utils.js#L80

I know I keep suggesting these API-breaking changes where we have to modify CM and store to match, but I figure better now than later, so please bear with me.

1. The path caveat should be split by HTTP method (GET, POST, maybe others but not yet)
2. We should change GET/POST /read/* to GET /*
3. We should change GET/POST /write/* to POST /*

I know that just the other day in a different GitHub issue, I was arguing the exact opposite, but the more I think about it, the more I believe that I was wrong before and that distinguishing between methods is better than encoding behaviour into paths.

This issue was prompted by me trying to add extra special cases and exceptions for authorisation on the /cat endpoint since GET and POST (and PUT and DELETE) to the same path have a different meaning. Aside from Hypercat, other RESTful API do indeed differentiate too, as well as front-ends to DBs (which only occurred to me when we had that discussion on CRUD in stores the other day).

Otherwise if we leave things as they are now, I'll just patch something on to deal with that one special case for now.

Hi @yousefamar ,

You have moved the arbiter over to https with a self-signed cert. Are we sure this is the right way to go?

It means that anything that needs to communicate with the arbiter must ignore cert errors. Which means it's a lot of overhead for little or no protection (MITM attacks are trivial it the certs aren't verified ). I've updated the CM in the Hyperrcat branch to support it, for now, but I'm not sure its the best plan.

one option might be https://github.com/mozilla/node-srp

EDIT: just had a chat with Chris and he thinks https is the way forward. However,

 1. The Arbiter certs should be generated at runtime (Probably by the CM) and passed in. So the private key is not leaked.
 2. All containers should receive the public key in as an Env var. This should then be trusted (in node i think we can use the options pram on request.https to add it to cert property)

Cheers

Tosh

As mentioned in #31, we no longer need the response from this endpoint to be base64 encoded. The advantage of removing this, is that the body of the response will be smaller, and store implementations don't need to worry about decoding the secrets before use.

This is not backwards compatible however, and will affect all current store implementations, so it's probably best to leave this until we have me-box/databox#6 resolved, and can have the stores depend on older versions, so they don't all have to update at once.

Right now macaroon path caveats are just a root wildcard (see here). Ideally, the SLA info that is passed to the arbiter by the CM should be encoded here instead.

Similarly, currently macaroons last forever, and should instead have a caveat that enforces an expiry timestamp.

Hi,

Would it be possible to add a simple UI to the arbiter? Just something that lists all the permissions granted would very useful for debugging.

https://databox-arbiter/ui --> formatted HTML output of permmissions

Cheers

Tosh