rescrv / libmacaroons Goto Github PK
View Code? Open in Web Editor NEWMacaroons are flexible authorization credentials that support decentralized delegation, attenuation, and verification.
License: BSD 3-Clause "New" or "Revised" License
Macaroons are flexible authorization credentials that support decentralized delegation, attenuation, and verification.
License: BSD 3-Clause "New" or "Revised" License
Macaroons are Better Than Cookies! ================================== This library provides an implementation of macaroons[1], which are flexible authorization tokens that work great in distributed systems. Like cookies, macaroons are bearer tokens that enable applications to ascertain whether their holders' actions are authorized. But macaroons are better than cookies! Why Macaroons? -------------- Macaroons are great for authorization because they're similar enough to cookies to be immediately usable by developers, but they include several features not present in cookies or other token-based authorization schemes. In particular: - Delegation with Contextual Caveats (i.e., confinement of the usage context): Macaroons support delegation. Give your macaroon to another user, and they can act on your behalf, with the same authority. Cookies permit delegation as well, but the remaining features of macaroons make it much more safe and practical to pass around macaroons than cookies. In particular, macaroons can limit when, where, and by whom the delegated authority can be exercised (e.g., within one minute, from a machine that holds a certain key, or by a certain logged-in user), by using attenuation and third-party caveats. - Attenuation: Macaroons enable users to add caveats to the macaroon that attenuate how, when, and where it may be used. Unlike cookies, macaroons permit their holder to attenuate them before delegating. Whereas cookies and authorization tokens enable an application to get access to all of your data and to perform actions on your behalf with your full privileges, macaroons enable you to restrict what they can do. Those questionable startups that "just want the address book, we swear it," become a whole lot more secure when the target application supports macaroons, because macaroons enable you to add caveats that restrict what the application can do. - Proof-Carrying: Macaroons are efficient, because they carry their own proof of authorization---cryptographically secured, of course. A macaroon's caveats are constructed using chained HMAC functions, which makes it really easy to add a caveat, but impossible to remove a caveat. When you attenuate a macaroon and give it to another application, there is no way to strip the caveats from the macaroon. It's easy for the entity that created a macaroon to verify the embedded proof, but others cannot. - Third-Party Caveats: Macaroons allow caveats to specify predicates that are enforced by third parties. A macaroon with a third-party caveat will only be authorized when the third party certifies that the caveat is satisfied. This enables loosely coupled distributed systems to work together to authorize requests. For example, a data store can provide macaroons that are authorized if and only if the application's authentication service says that the user is authenticated. The user obtains a proof that it is authenticated from the authentication service, and presents this proof alongside the original macaroon to the storage service. The storage service can verify that the user is indeed authenticated, without knowing anything about the authentication service's implementation---in a standard implementation, the storage service can authorize the request without even communicating with the authentication service. - Simple Verification: Macaroons eliminate complexity in the authorization code of your application. Instead of hand-coding complex conditionals in each routine that deals with authorization, and hoping that this logic is globally consistent, you construct a general verifier for macaroons. This verifier knows how to soundly check the proofs embedded within macaroons to see if they do indeed authorize access. - Decoupled Authorization Logic: Macaroons separate the policy of your application (who can access what, when), from the mechanism (the code that actually upholds this policy). Because of the way the verifier is constructed, it is agnostic to the actual underlying policies it is enforcing. It simply observes the policy (in the form of an embedded proof) and certifies that the proof is correct. The policy itself is specified when macaroons are created, attenuated, and shared. You can easily audit this code within your application, and ensure that it is upheld everywhere. The rest of this document walks through the specifics of macaroons and see just how easy authorization can be. So pour a fresh cup of espresso to enjoy alongside your macaroons and read on! Installing Macaroons -------------------- This library makes it easy to get started with using macaroons in your service. To use the library you must first install it. You'll need to somehow install libbsd[2]. It's packaged in some Linux distributions, and can be installed from source on most *NIX platforms. Once you have libbsd installed, installing macaroons is straight forward. If you're installing from git you'll need to install autoconf, automake, libtool, and autoconf-archive and execute the first command. if you're installing from a release tarball, you should skip the first command and start with configure. $ autoreconf -i # only when installing from Git $ ./configure --enable-python-bindings $ make # make install This will install macaroons onto your system and give you both the C and Python interfaces to libmacaroons. In the rest of this document, we'll show examples using the Python interface, but the code could easily be translated into C. Creating Your First Macaroon ---------------------------- Imagine that you ran a bank, and were looking to use macaroons as the basis of your authorization logic. Assuming you already installed libmacaroons, you can create a macaroon like this: >>> import macaroons >>> secret = 'this is our super secret key; only we should know it' >>> public = 'we used our secret key' >>> location = 'http://mybank/' >>> M = macaroons.create(location, secret, public) We've created our first macaroon! You can see here that it took three pieces of information to create a macaroon. We start with a secret. Here, we just have English text, but in reality we would want to use something more random and less predictable (see below for a suggestion on how to generate one). The public portion tells us which secret we used to create the macaroon, but doesn't give anyone else a clue as to the contents of the secret. Anyone in possession of the macaroon can see the public portion: >>> M.identifier 'we used our secret key' This public portion, known as the macaroon's identifier, can be anything that enables us to remember our secret. In this example, we know that the string 'we used our secret key' always refers to this secret. We could just as easily keep a database mapping public macaroon identities to private secrets, or encrypt the public portion using a key known only to us. The only requirement here is that our application be able to remember the secret given the public portion, and that no one else is able to guess the secret. Our macaroon includes some additional information that enables us to add caveats and figure out where the macaroon should be used. The macaroon's location gives a hint to the user about where the macaroon is accepted for authorization. This hint is a free-form string maintained to help applications figure out where to use macaroons; the libmacaroons library (and by extension, the Python bindings) do not ascribe any meaning to this location. Each macaroon also has a signature that is the key used to add caveats and verify the macaroon. The signature is computed by the macaroons library, and is unique to each macaroon. Applications should never need to directly work with the signature of the macaroon. Any entity in possession of the macaroon's signature should be assumed to possess the macaroon itself. Both of these pieces of information are publicly accessible: >>> M.location 'http://mybank/' >>> M.signature 'e3d9e02908526c4c0039ae15114115d97fdd68bf2ba379b342aaf0f617d0552f' We can share this macaroon with others by serializing it. The serialized form is pure-ASCII, and is safe for inclusion in secure email, a standard HTTPS cookie, or a URL. We can get the serialized form with: >>> M.serialize(format=1) 'MDAxY2xvY2F0aW9uIGh0dHA6Ly9teWJhbmsvCjAwMjZpZGVudGlmaWVyIHdlIHVzZWQgb3VyIHNlY3JldCBrZXkKMDAyZnNpZ25hdHVyZSDj2eApCFJsTAA5rhURQRXZf91ovyujebNCqvD2F9BVLwo' Of course, this serialized form can be displayed in a more human-readable form for easy debugging: >>> print M.inspect() location http://mybank/ identifier we used our secret key signature e3d9e02908526c4c0039ae15114115d97fdd68bf2ba379b342aaf0f617d0552f Adding Caveats -------------- At this point, we have an unconstrained macaroon that authorizes everything within our bank. In practice, such a macaroon is dangerous, because very few people should hold such power. Let's add a caveat to our macaroon that restricts it to just the account number 3735928559. >>> M = M.add_first_party_caveat('account = 3735928559') This new macaroon includes the same identifier and location that our old macaroon from our initial macaroon, but includes the additional caveat that restricts the bank account. The signature of this new macaroon is different, and incorporates the new caveat we've just added. An entity in possession of this new macaroon cannot simply remove our new caveat to construct the old macaroon: >>> print M.inspect() location http://mybank/ identifier we used our secret key cid account = 3735928559 signature 1efe4763f290dbce0c1d08477367e11f4eee456a64933cf662d79772dbb82128 Of course, we can add a few more caveats, and the macaroon's signature will change with each of them. >>> M = M.add_first_party_caveat('time < 2020-01-01T00:00') >>> M.signature 'b5f06c8c8ef92f6c82c6ff282cd1f8bd1849301d09a2db634ba182536a611c49' >>> M = M.add_first_party_caveat('email = [email protected]') >>> M.signature 'ddf553e46083e55b8d71ab822be3d8fcf21d6bf19c40d617bb9fb438934474b6' >>> print M.inspect() location http://mybank/ identifier we used our secret key cid account = 3735928559 cid time < 2020-01-01T00:00 cid email = [email protected] signature ddf553e46083e55b8d71ab822be3d8fcf21d6bf19c40d617bb9fb438934474b6 The combination of all caveats in this macaroon authorize [email protected] to access account 3735928559 until the end of 2019. Alice may present this macaroon to the bank any time she wishes to prove to the bank that she is authorized to access her account. Ideally, she'll transmit the serialized form of the macaroon to the bank: >>> msg = M.serialize(format=1) >>> # send msg to the bank Verifying Macaroons ------------------- Our bank application's purpose is to protect users accounts from unauthorized access. For that reason, it cannot just accept anything that looks like a macaroon---that would defeat the point of using macaroons in the first place. So how can we ensure that only authorized users access the bank? We can determine whether a request is authorized through a process called verification. First, we construct a verifier that can determine whether the caveats on macaroons are satisfied. We can then use our verifier to determine whether a given macaroon is authorized in the context of the request. For example, our bank account application knows the account number specified in the request, and can specify ``account = #'' when building the verifier. The verifier can then check that this matches the information within the macaroon, and authorize the macaroon if it does indeed match. Let's walk through the verification process for Alice's macaroon that we constructed in the previous section. The first step, of course, is for the bank to deserialize the macaroon from the message. This converts the macaroon into a form we can work with. >>> M = macaroons.deserialize(msg) >>> print M.inspect() location http://mybank/ identifier we used our secret key cid account = 3735928559 cid time < 2020-01-01T00:00 cid email = [email protected] signature ddf553e46083e55b8d71ab822be3d8fcf21d6bf19c40d617bb9fb438934474b6 We have the same macaroon that Alice believes authorizes her to access her own account, but we must verify this for ourselves. One (very flawed) way we could try to verify this macaroon would be to manually parse it and authorize the request if its caveats are true. But handling things this way completely sidesteps all the crypto-goodness that macaroons are built upon. Another approach to verification would be to use libmacaroons's built-in verifier to process the macaroon. The verifier hides many of the details of the verification process, and provides a natural way to work with many kinds of caveats. The verifier itself is constructed once, and may be re-used to verify multiple macaroons. >>> V = macaroons.Verifier() >>> V # doctest: +ELLIPSIS <macaroons.Verifier object at ...> Let's go ahead and try to verify the macaroon to see if the request is authorized. To verify the request, we need to provide the verifier with Alice's macaroon, and the secret that was used to construct it. In a real application, we would retrieve the secret using ``M.identifier''; here, we know the secret and provide it directly. A verifier can only ever successfully verify the macaroon when provided with the macaroon and its corresponding secret---no secret, no authorization. Intuitively, our verifier should say that this macaroon is unauthorized because our verifier cannot prove that any of the caveats we've added are satisfied. We can see that it fails just as we would expect: >>> V.verify(M, secret) Traceback (most recent call last): Unauthorized: macaroon not authorized We can inform the verifier of the caveats used by our application using two different techniques. The first technique is to directly provide the verifier with the caveats that match the context of the request. For example, every account-level action in a typical banking application is performed by a specific user and targets a specific account. This information is fixed for each request, and is known at the time of the request. We can tell the verifier directly about these caveats like so: >>> V.satisfy_exact('account = 3735928559') >>> V.satisfy_exact('email = [email protected]') Caveats like these are called ``exact caveats'' because there is exactly one way to satisfy them. Either the account number is 3735928559, or it isn't. At verification time, the verifier will check each caveat in the macaroon against the list of satisfied caveats provided to ``satisfy_exact''. When it finds a match, it knows that the caveat holds and it can move onto the next caveat in the macaroon. Generally, you will specify multiple true statements as exact caveats, and let the verifier decide which are relevant to each macaroon at verification time. If you provide all exact caveats known to your application to the verifier, it becomes trivial to change policy decisions about authorization. The server performing authorization can treat the verifier as a black-box and does not need to change when changing the authorization policy. The actual policy is enforced when macaroons are minted and when caveats are embedded. In our banking example, we could provide some additional satisfied caveats to the verifier, to describe some (or all) of the properties that are known about the current request. In this manner, the verifier can be made more general, and be "future-proofed", so that it will still function correctly even if somehow the authorization policy for Alice changes; for example, by adding the three following facts, the verifier will continue to work even if Alice decides to self-attenuate her macaroons to be only usable from her IP address and browser: >>> V.satisfy_exact('IP = 127.0.0.1') >>> V.satisfy_exact('browser = Chrome') >>> V.satisfy_exact('action = deposit') Although it's always possible to satisfy a caveat within a macaroon by providing it directly to the verifier, doing so can be quite tedious. Consider the caveat on access time embedded within Alice's macaroon. While an authorization routine could provide the exact caveat ``time < 2020-01-01T00:00'', doing so would require inspecting the macaroon before building the verifier. Just like using MD5 to hash passwords, inspecting a macaroon's structure to build a verifier for it is considered to be very bad practice, and should be violently demonized in Hacker News discussions with vague, slightly inaccurate allusions to pbkdf2. So how can we tell our verifier that the caveat on access time is satisfied? We could provide many exact caveats of the form ``time < YYYY-mm-ddTHH:MM'', but this reeks of inefficiency. The second technique for satisfying caveats provides a more general solution. Called ``general caveats'', the second technique for informing the verifier that a caveat is satisfied allows for expressive caveats. Whereas exact caveats are checked by simple byte-wise equality, general caveats are checked using an application-provided callback that returns true if and only if the caveat is true within the context of the request. There's no limit on the contents of a general caveat, so long as the callback understands how to determine whether it is satisfied. We can verify the time caveat on Alice's macaroon by writing a function that checks the current time against the time specified by the caveat: >>> import datetime >>> def check_time(caveat): ... if not caveat.startswith('time < '): ... return False ... try: ... now = datetime.datetime.now() ... when = datetime.datetime.strptime(caveat[7:], '%Y-%m-%dT%H:%M') ... return now < when ... except: ... return False ... This callback processes all caveats that begin with ``time < '', and returns True if the specified time has not yet passed. We can see that our caveat does indeed return True when the caveat holds, and False otherwise: >>> check_time('time < 2020-01-01T00:00') True >>> check_time('time < 2014-01-01T00:00') False >>> check_time('account = 3735928559') False We can provide the ``check_time'' function directly to the verifier, so that it may check time-based predicates. >>> V.satisfy_general(check_time) It's finally time to verify our macaroon! Now that we've informed the verifier of all the various caveats that our application could embed within a macaroon, we can expect that the verification step will succeed. >>> V.verify(M, secret) True More importantly, the verifier will also work for macaroons we've not yet seen, like one that only permits Alice to deposit into her account: >>> N = M.add_first_party_caveat('action = deposit') >>> V.verify(N, secret) True Equally important is the verifier's ability to reject improper macaroons because they are not authorized. An improper macaroon could have additional caveats not known to the verifier, or have false general caveats. Worse, an unauthorized macaroon could be an attempt by a determined attacker to break into our bank. The verifier we've built will reject all of these scenarios: >>> # Unknown caveat >>> N = M.add_first_party_caveat('OS = Windows XP') >>> V.verify(N, secret) Traceback (most recent call last): Unauthorized: macaroon not authorized >>> # False caveat >>> N = M.add_first_party_caveat('time < 2014-01-01T00:00') >>> V.verify(N, secret) Traceback (most recent call last): Unauthorized: macaroon not authorized >>> # Bad secret >>> V.verify(M, 'this is not the secret we were looking for') Traceback (most recent call last): Unauthorized: macaroon not authorized >>> # Incompetent hackers trying to change the signature >>> N = macaroons.deserialize('MDAxY2xvY2F0aW9uIGh0dHA6Ly9teWJhbmsvCjAwMjZpZGVudGlmaWVyIHdlIHVzZWQgb3VyIHNl\nY3JldCBrZXkKMDAxZGNpZCBhY2NvdW50ID0gMzczNTkyODU1OQowMDIwY2lkIHRpbWUgPCAyMDIw\nLTAxLTAxVDAwOjAwCjAwMjJjaWQgZW1haWwgPSBhbGljZUBleGFtcGxlLm9yZwowMDJmc2lnbmF0\ndXJlID8f19FL+bkC9p/aoMmIecC7GxdOcLVyUnrv6lJMM7NSCg==\n') >>> print N.inspect() location http://mybank/ identifier we used our secret key cid account = 3735928559 cid time < 2020-01-01T00:00 cid email = [email protected] signature 3f1fd7d14bf9b902f69fdaa0c98879c0bb1b174e70b572527aefea524c33b352 >>> M.signature == N.signature False >>> V.verify(N, secret) Traceback (most recent call last): Unauthorized: macaroon not authorized Using Third-Party Caveats ------------------------- Like first-party caveats, third-party caveats restrict the context in which a macaroon is authorized, but with a different form of restriction. Where a first-party caveat is checked directly within the verifier, a third-party caveat is checked by the third-party, who provides a discharge macaroon to prove that the original third-party caveat is true. The discharge macaroon is recursively inspected by the verifier; if it verifies successfully, the discharge macaroon serves as a proof that the original third-party caveat is satisfied. Of course, nothing stops discharge macaroons from containing embedded first- or third-party caveats for the verifier to consider during verification. Let's rework the above example to provide Alice with access to her account only after she authenticates with a service that is separate from the service processing her banking transactions. As before, we'll start by constructing a new macaroon with the caveat that is limited to Alice's bank account. >>> secret = 'this is a different super-secret key; never use the same secret twice' >>> public = 'we used our other secret key' >>> location = 'http://mybank/' >>> M = macaroons.create(location, secret, public) >>> M = M.add_first_party_caveat('account = 3735928559') >>> print M.inspect() location http://mybank/ identifier we used our other secret key cid account = 3735928559 signature 1434e674ad84fdfdc9bc1aa00785325c8b6d57341fc7ce200ba4680c80786dda So far, so good. Now let's add a third party caveat to this macaroon that requires that Alice authenticate with ``http://auth.mybank/'' before being authorized access to her account. To add a third-party caveat we'll need to specify a location hint, generate a secret, and somehow be able to identify this secret later. Like the location used when creating a macaroon, the location used when adding a third-party caveat is simply a hint to the application that is uninterpreted by libmacaroons. The secret and identity are handled differently than during creation, however, because they need to be known to the third party. To do this, we'll provide the third-party with a randomly generated ``caveat_key'' and the predicate we wish for it to check. In response, it will give us an identifier that we can embed within the macaroon. Later, when we need to construct a discharge macaroon, we can provide it with just this identifier, from which it can recall the key and predicate to check. Here's what this looks like in code: >>> # you'll likely want to use a higher entropy source to generate this key >>> caveat_key = '4; guaranteed random by a fair toss of the dice' >>> predicate = 'user = Alice' >>> # send_to_auth(caveat_key, predicate) >>> # identifier = recv_from_auth() >>> identifier = 'this was how we remind auth of key/pred' >>> M = M.add_third_party_caveat('http://auth.mybank/', caveat_key, identifier) >>> print M.inspect() location http://mybank/ identifier we used our other secret key cid account = 3735928559 cid this was how we remind auth of key/pred vid AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA027FAuBYhtHwJ58FX6UlVNFtFsGxQHS7uD_w_dedwv4Jjw7UorCREw5rXbRqIKhr cl http://auth.mybank/ signature d27db2fd1f22760e4c3dae8137e2d8fc1df6c0741c18aed4b97256bf78d1f55c We now have a macaroon with a third-party caveat. The most interesting thing to note about this macaroon is that it includes no information that reveals the predicate it encodes. The third-party service knows the key and the predicate, and internally associates them with the identifier, but the identifier itself is arbitrary and betrays no information about the predicate to others. The service at ``http://auth.mybank/'' can authenticate that the user is Alice, and provide proof that the caveat is satisfied, without revealing Alice's identity. Other services can verify M and its associated discharge macaroon, without knowing the predicates the third-parties verified. The process for discharging third party caveats starts with the holder of the initial root macaroon, Alice. Alice looks at the macaroon for the list of third party caveat (location, identifier) pairs that must be addressed. >>> M.third_party_caveats() [('http://auth.mybank/', 'this was how we remind auth of key/pred')] In a real application, we'd look at these third party caveats, and contact each location to retrieve the requisite discharge macaroons. We would include the identifier for the caveat in the request itself, so that the server can recall the secret used to create the third-party caveat. The server can then generate and return a new macaroon that discharges the caveat: >>> D = macaroons.create('http://auth.mybank/', caveat_key, identifier) >>> D = D.add_first_party_caveat('time < 2020-01-01T00:00') >>> print D.inspect() location http://auth.mybank/ identifier this was how we remind auth of key/pred cid time < 2020-01-01T00:00 signature 2ed1049876e9d5840950274b579b0770317df54d338d9d3039c7c67d0d91d63c This new macaroon enables the verifier to determine that the third party caveat is satisfied. Our target service added a time-limiting caveat to this macaroon that ensures that this discharge macaroon does not last forever. This ensures that Alice (or, at least someone authenticated as Alice) cannot use the discharge macaroon indefinitely and will eventually have to re-authenticate. Once Alice has both the root macaroon and the discharge macaroon in her possession, she can make the request to the target service. Making a request with discharge macaroons is only slightly more complicated than making requests with a single macaroon. In addition to serializing and transmitting all involved macaroons, there is a preparation step that binds the discharge macaroons to the root macaroon. This binding step ensures that the discharge macaroon is useful only when presented alongside the root macaroon. The root macaroon is used to bind the discharge macaroons like this: >>> DP = M.prepare_for_request(D) If we were to look at the signatures on these prepared discharge macaroons, we would see that the binding process has irreversibly altered their signature(s). >>> D.signature '2ed1049876e9d5840950274b579b0770317df54d338d9d3039c7c67d0d91d63c' >>> DP.signature 'd115ef1c133b1126978d5ab27f69d99ba9d0468cd6c1b7e47b8c1c59019cb019' The root macaroon ``M'' and its discharge macaroons ``MS'' are ready for the request. Alice can serialize them all and send them to the bank to prove she is authorized to access her account. The bank can verify them using the same verifier we built before. We provide the discharge macaroons as a third argument to the verify call: >>> V.verify(M, secret, [DP]) True Without the ``prepare_for_request'' call, the verification would fail: >>> V.verify(M, secret, [D]) Traceback (most recent call last): Unauthorized: macaroon not authorized Choosing Secrets ---------------- For clarity, we've generated human-readable secrets that we use as the root keys of all of our macaroons. In practice, this is terribly insecure and can lead to macaroons that can easily be forged because the secret is too predictable. To avoid this, we recommend generating secrets using a sufficient number of suitably random bytes. Because the bytes are a secret key, they should be drawn from a source with enough entropy to ensure that the key cannot be guessed before the macaroon falls out of use. The macaroons module exposes a constant that is the ideal number of bytes these secret keys should contain. Any shorter is wasting an opportunity for security. >>> macaroons.SUGGESTED_SECRET_LENGTH 32 To generate a suitable key is very simple using the python standard library: >>> import os >>> import binascii >>> binascii.hexlify(os.urandom(macaroons.SUGGESTED_SECRET_LENGTH)) # doctest: +ELLIPSIS '...' Which gives a long string of hex bytes which can be passed into the create and validate functions as-is. Third-Party Caveats with Public Keys ------------------------------------ Public key cryptography can enable much more efficient schemes for adding third-party caveats. In the above example where we added a third-party caveat, the caveat's identifier was generated by the third party and retrieved with in one round trip. We can eliminate the round trip when the third party has a well-known public key. We can encrypt the caveat's secret, and the predicate to be checked using this public key, and use the ciphertext directly as the caveat's identifier. This saves a round trip, and frees the third party from having to remember an association between identifiers and key/predicate pairs. [1] http://research.google.com/pubs/pub41892.html [2] https://libbsd.freedesktop.org/wiki/
To hash two items together with a key, I would expect the algorithm
to be hmac(key, item1 + item2). This is the algorithm used in the
macaroons paper in addCaveatHelper (MAC(sig, vId + cId)).
Instead, macaroon_hash2 uses hmac(key, hmac(key, item1) + hmac(key, item2)).
This is unexpected, and probably less efficient than the simpler approach.
Hi,
the current serialization format of macaroons uses Base64, as described in RFC 1521.
When using macaroons as bearer tokens in HTTP requests (e.g. query params)
this may cause problems, because Base64 uses characters like + and / for encoding.
There is an alternative: RFC 4648 (Oct.2006),
which describes Base 64 Encoding with URL and filename safe alphabet.
The changes in encoding are minimal: two alphabet characters changed and no padding is applied.
See also:
https://tools.ietf.org/html/rfc4648#section-5
What do you think about changing the serialization to base64url (the preferred name, as stated in the RFC).
Regards
Martin
But handling things this way completely
sidesteps all the crypto-goodness that macaroons are built upon.
what is this crypto-goodness? I see hmac and secretbox in the code, but how it is actually applied to the caveats is not obvious (from glancing at the code)
my guess something like you hmac the secret, then hmac each caveat in turn (which would mean you could never remove a caveat but you could add one)? Is this the correct basic idea? is there any documentation about how it works?
All that said though, I do appreciate the simplicity,
ASCII-visibility and efficiency of the current format.
Most possible format changes would compromise
on at least one of the above qualities to some degree.
I'm raising this issue because I think the subject
should be at least discussed.
When we pass an unknown format to macaroon.serialize, it produces this error:
>>> m.serialize(format='json1')
Exception KeyError: ('json1',) in 'macaroons.Macaroon.version' ignored
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "bindings/python/macaroons.pyx", line 179, in macaroons.Macaroon.serialize (bindings/python/macaroons.c:2215)
if self.is_json(format):
File "bindings/python/macaroons.pyx", line 289, in macaroons.Macaroon.is_json (bindings/python/macaroons.c:3914)
'1': False}[v]
KeyError: 'json1'
It would be nice if it made it clearer what the error signifies (for example by raising an exception which mentions that it's an unknown format, possibly stating the possible format values, as there is no documentation for that, AFAICS)
As far as I can tell, the following script should print:
checked conditions: ['splendid', 'wonderful', 'top of the world']
but it actually prints:
checked conditions: ['top of the world', 'wonderful']
One first party caveat does not appear to be checked.
import macaroons
m1 = macaroons.create("", "root-key", "root-id")
m1 = m1.add_first_party_caveat("wonderful")
m1 = m1.add_third_party_caveat("bob", "bob-caveat-root-key", "bob-is-great")
m2 = macaroons.create("bob", "bob-caveat-root-key", "bob-is-great")
m2 = m2.add_first_party_caveat("splendid")
m3 = macaroons.create("bob", "bob-caveat-root-key", "bob-is-great")
m3 = m3.add_first_party_caveat("top of the world")
m2 = m1.prepare_for_request(m2)
m3 = m1.prepare_for_request(m3)
checked = []
def check(cond):
checked.append(cond)
return True
v = macaroons.Verifier()
v.satisfy_general(check)
v.verify(m1, "root-key", [m2, m3])
checked.sort()
print('checked conditions: {}'.format(checked))
In macaroon_add_third_party_caveat_raw(...)
, I'm wondering if the buffer for enc_cyphertext
should be the same size as that for enc_plaintext
?
Lines 342 to 352 in fb878b9
The call to macaroon_secretbox(...)
appears to be attempting to store data beyond the end of enc_cyphertext
when SECRET_BOX_OVERHEAD
is not included when initialising the buffer size.
Lines 384 to 390 in fb878b9
In macaroon_verify_inner_3rd(...)
, enc_cyphertext
buffer size matches enc_plaintext
:
Lines 717 to 730 in fb878b9
Note: SECRET_BOX_OVERHEAD
is define
d as the difference between MACAROON_SECRET_TEXT_ZERO_BYTES
and MACAROON_SECRET_BOX_ZERO_BYTES
.
Lines 329 to 331 in fb878b9
There is a macaroon_serialize_json function, but no macaroon_deserialize_json function.
Apologies, I haven't dug into this so I don't know if this is a trivial fix or not. I get this error when trying to build:
vagrant@wesdev:~/libmacaroons$ make
make --no-print-directory all-am
PYX bindings/python/macaroons.c
Error compiling Cython file:
------------------------------------------------------------
...
MACAROON_NOT_AUTHORIZED = 2055
MACAROON_NO_JSON_SUPPORT = 2056
macaroon* macaroon_create(unsigned char* location, size_t location_sz, unsigned char* key, size_t key_sz, unsigned char* id, size_t id_sz, macaroon_returncode* err)
void macaroon_destroy(macaroon* M)
int macaroon_validate(const macaroon* M)
^
------------------------------------------------------------
bindings/python/macaroons.pyx:55:40: Expected ')', found '*'
make[1]: *** [bindings/python/macaroons.c] Error 1
make: *** [all] Error 2
Am I missing something?
First of all, great initiative! :)
That said, the Python bindings have an "unsafe" API that makes it too easy to use the library in an incorrect and hence insecure way. The signature of the verify()
API returns a boolean result, which makes code like this seem correct but insecure:
m.verify()
do_something_assuming_it_is_verified()
The approach cryptography (the python project) takes avoid these kind of problems (see e.g. http://cryptography.readthedocs.org/en/latest/hazmat/primitives/mac/hmac/) by raising appropriate exceptions instead. In that case, code like the above would be secure, and code that needs to handle the verification error must do so explicitly:
try:
m.verify()
except macaroon.VerificationError:
pass # do something sensible here
The most common way to install Python packages is via pip
, a package manager.
The README explains how to compile Python bindings, but it would be helpful if it explained how to install via pip
, or what stanza I need to add to setup.py
or requirements.txt
in order to install this library.
Thanks!
Getting the following error when trying to build the python bindings using python 3.6:
/usr/bin/ld:bindings/python/.libs/macaroons.ver:2: syntax error in VERSION script
Said file contains the following:
{ global:
local: *; };
Same issue as #16 that was closed without actually being addressed.
Hi,
in the readme.md of libmacaroon, there is an example of a Base64 serialized macaroon (look for "M.serialize()").
When decoding this Base64 string everything is human readable, except the signature.
The signature is 32 bytes, which may contain everything, from white spaces to Greek letters.
My team and I plan to adopt macaroons for our growing microservice application landscape. As written in the research paper, there are a lot of benefits, which are very attractive for us.
BUT, when implementing, debugging and tracing macaroons based requests, we prefer to see human readable plain text only. It makes their handling easier. (IMHO, Base64 is transparent, I want to inspect macaroons)
So, I propose to change the signature and 'vid' packets to be packed as hex strings (e.g. cafebabe42),
which encourages better human readability.
What do you think?
PS: You may copy&paste the Base64 example to www.base64decode.org and see for yourself.
Hi,
Are there any plans for a v0.4.0?
I have an application that I'm working on which will start to depend on libmacaroons. I've noticed a few API breaks between current master and the latest tag -- will these be present in future released versions?
Additionally, I plan to package libmacaroons for Fedora/EPEL. Any upstream objections to that?
Thanks,
Brian
It seems that the vid field in third party caveats is not correctly encoded, which leads to an exception when trying to serialize macaroons in JSON format.
>>> m = macaroons.create("my location", "my secret key", "my identifier")
>>> m = m.add_third_party_caveat("tp location", "tp root key", "tp caveat")
>>> m.serialize(format='json')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "bindings/python/macaroons.pyx", line 181, in macaroons.Macaroon.serialize (bindings/python/macaroons.c:2259)
return bytes(data[:data_sz]).decode('utf8')
UnicodeDecodeError: 'utf8' codec can't decode byte 0x9c in position 90: invalid start byte
I added a print statement before the exception was raised, and it printed this:
('got serialized data', '{"v":2,"l":"my location","i":"my identifier","c":[{"i":"tp caveat","l":"tp location","v":"\x9cJ\xb4jr\x11\xa5\x13\x8e\xba\x13\xe2y\xb4\x829n\x96\x0c1b\x9e\x13\xff\x8a\x89\x10B\xe2\xff+\xf7hh\xe3W\xef\xd3SO\xe0i\x062\xcf\x93\xb1\xc3\xd8r\xa1\xf5\xb5m\\g\xad\xf1\x96\xa4\x164U\xde\t\xb0\x89\x15\xb7\x12"}],"s64":"FodXLpedS2YbvCiUOqNbArA2Edw4Plc7SBFPNHc9gBI"}')
It looks like the vid field should have been base64 encoded and then the v64 field used.
The base64 does not have the required padding characters to
allow it to be parsed by standard base64 decoders.
Verify by taking some serialization from the example README.md:
echo 'MDAxY2xvY2F0aW9uIGh0dHA6Ly9teWJhbmsvCjAwMjZpZGVudGlmaWVyIHdlIHVzZWQgb3VyIHNlY3JldCBrZXkKMDAyZnNpZ25hdHVyZSDj2eApCFJsTAA5rhURQRXZf91ovyujebNCqvD2F9BVLwo' | base64 -d
produces an error.
gets this issue when i try to import
import macaroons
Traceback (most recent call last):
File "", line 1, in
ImportError: libmacaroons.so.0: cannot open shared object file: No such file or directory
Given that the version 1 JSON format is used in production in various places,
it would be useful to support that as well as version 2.
I keep getting this error on my during ./configure
./configure: line 16287: ANAL_WARNINGS: command not found
./configure: line 16290: syntax error near unexpected token `SODIUM,'
./configure: line 16290: `PKG_CHECK_MODULES(SODIUM, libsodium >= 0.4)'
I've installed libsodium from here: https://download.libsodium.org/libsodium/releases/
Happens on my ubuntu machine (x86_64)
Hi Robert,
I just discovered your project and it's really interesting ๐
I'm just a little bugged with the cookies comparison in the project description. I know I'm being a bit picky but cookies are just a key/value transport mean with a promise that a browser will send them back. They can be used to transport anything and even if we often use them to transport session IDs they cannot be compared IMO. It's the same when people compare JWT with cookies, you can transport JWT in a cookie.
I think it would be better to oppose macaroons with session IDs or maybe I haven't catched well the behaviour of macaroons yet.
WDYT?
I'm not a Go expert, but the second else if
here:
https://github.com/rescrv/libmacaroons/blob/master/bindings/go/macaroons/macaroons.go#L231-L235
is either a leftover or wrong.
I found out that when building libmacaroons on OS X I do get this error:
https://github.com/rescrv/libmacaroons/blob/master/macaroons.c#L46
On OS X you should be including util.h
#include <util.h>
I'm trying to install libmacaroons on ubuntu trusty.
Dependencies:
apt-get install -y make wget tar autoconf libtool pkg-config
apt-get install -y python3 python3-dev python3-software-properties build-essential
pip3 install cython
Sodium:
wget https://github.com/jedisct1/libsodium/releases/download/0.6.1/libsodium-0.6.1.tar.gz
tar xzvf libsodium-0.6.1.tar.gz
cd libsodium-0.6.1 && ./configure && make && make check && sudo make install
Macaroons:
wget -O libmacaroons.tar.gz https://github.com/rescrv/libmacaroons/archive/master.tar.gz
tar xzvf libmacaroons.tar.gz
cd libmacaroons-master && autoreconf -i && ./configure --enable-python-bindings && make && sudo make install
I get the following error when running make though:
/usr/bin/ld:bindings/python/.libs/macaroons.ver:2: syntax error in VERSION script
collect2: error: ld returned 1 exit status
make[1]: *** [bindings/python/macaroons.la] Error 1
make: *** [all] Error 2
Any thoughts? Thanks!
Hi all,
I am trying to install libmacaroons in my local machine (Mac OS Catalina), but I am having issues, due to an error telling me that pkg-config is not found. Don't know what is happening but I have already installed all the requirements for this library as you can check below:
~ autoconf --version
autoconf (GNU Autoconf) 2.71
Copyright (C) 2021 Free Software Foundation, Inc.
License GPLv3+/Autoconf: GNU GPL version 3 or later
<https://gnu.org/licenses/gpl.html>, <https://gnu.org/licenses/exceptions.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Written by David J. MacKenzie and Akim Demaille.
โ ~ automake --version
automake (GNU automake) 1.16.4
Copyright (C) 2021 Free Software Foundation, Inc.
License GPLv2+: GNU GPL version 2 or later <https://gnu.org/licenses/gpl-2.0.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Written by Tom Tromey <[email protected]>
and Alexandre Duret-Lutz <[email protected]>.
โ ~ glibtool --version
glibtool (GNU libtool) 2.4.6
Written by Gordon Matzigkeit, 1996
Copyright (C) 2014 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
โ ~ pkg-config --version
0.29
And here below, the outputs I get when I trying to install libmacaroons. Thinking the problem is related to the m4 macros that for some reason the installation script is not able to find in my computer. Any suggestion to solve this issue?
~ git clone https://github.com/rescrv/libmacaroons.git
Cloning into 'libmacaroons'...
remote: Enumerating objects: 615, done.
remote: Counting objects: 100% (5/5), done.
remote: Compressing objects: 100% (5/5), done.
remote: Total 615 (delta 1), reused 0 (delta 0), pack-reused 610
Receiving objects: 100% (615/615), 248.70 KiB | 684.00 KiB/s, done.
Resolving deltas: 100% (362/362), done.
โ ~ cd libmacaroons
โ libmacaroons git:(master) autoreconf -i
glibtoolize: putting auxiliary files in '.'.
glibtoolize: copying file './ltmain.sh'
glibtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'.
glibtoolize: copying file 'm4/libtool.m4'
glibtoolize: copying file 'm4/ltoptions.m4'
glibtoolize: copying file 'm4/ltsugar.m4'
glibtoolize: copying file 'm4/ltversion.m4'
glibtoolize: copying file 'm4/lt~obsolete.m4'
configure.ac:69: warning: The macro `AC_PYTHON_DEVEL' is obsolete.
configure.ac:69: You should run autoupdate.
m4/ax_python_devel.m4:72: AC_PYTHON_DEVEL is expanded from...
configure.ac:69: the top level
configure.ac:16: installing './compile'
configure.ac:16: installing './config.guess'
configure.ac:16: installing './config.sub'
configure.ac:12: installing './install-sh'
configure.ac:12: installing './missing'
Makefile.am: installing './depcomp'
โ libmacaroons git:(master) autoupdate
โ libmacaroons git:(master) โ autoreconf -i
โ libmacaroons git:(master) โ ./configure --enable-python-bindings
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a race-free mkdir -p... ./install-sh -c -d
checking for gawk... no
checking for mawk... no
checking for nawk... no
checking for awk... awk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking whether make supports nested variables... (cached) yes
checking for a Python interpreter with version >= 2.6... python
checking for python... /usr/bin/python
checking for python version... 2.7
checking for python prefix... /System/Library/Frameworks/Python.framework/Versions/2.7
checking for python exec_prefix... /System/Library/Frameworks/Python.framework/Versions/2.7
checking for python platform... darwin
checking for python script directory... /Library/Python/2.7/site-packages
checking for python extension module directory... /Library/Python/2.7/site-packages
checking build system type... x86_64-apple-darwin19.6.0
checking host system type... x86_64-apple-darwin19.6.0
checking how to print strings... printf
checking whether make supports the include directive... yes (GNU style)
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether the compiler supports GNU C... yes
checking whether gcc accepts -g... yes
checking for gcc option to enable C11 features... none needed
checking whether gcc understands -c and -o together... yes
checking dependency style of gcc... gcc3
checking for a sed that does not truncate output... /usr/bin/sed
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for fgrep... /usr/bin/grep -F
checking for ld used by gcc... /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld
checking if the linker (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld) is GNU ld... no
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 196608
checking how to convert x86_64-apple-darwin19.6.0 file names to x86_64-apple-darwin19.6.0 format... func_convert_file_noop
checking how to convert x86_64-apple-darwin19.6.0 file names to toolchain format... func_convert_file_noop
checking for /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld option to reload object files... -r
checking for objdump... objdump
checking how to recognize dependent libraries... pass_all
checking for dlltool... no
checking how to associate runtime and link libraries... printf %s\n
checking for ar... ar
checking for archiver @FILE support... no
checking for strip... strip
checking for ranlib... ranlib
checking command to parse /usr/bin/nm -B output from gcc object... ok
checking for sysroot... no
checking for a working dd... /bin/dd
checking how to truncate binary pipes... /bin/dd bs=4096 count=1
checking for mt... no
checking if : is a manifest tool... no
checking for dsymutil... dsymutil
checking for nmedit... nmedit
checking for lipo... lipo
checking for otool... otool
checking for otool64... no
checking for -single_module linker flag... yes
checking for -exported_symbols_list linker flag... yes
checking for -force_load linker flag... yes
checking for stdio.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for strings.h... yes
checking for sys/stat.h... yes
checking for sys/types.h... yes
checking for unistd.h... yes
checking for dlfcn.h... yes
checking for objdir... .libs
checking if gcc supports -fno-rtti -fno-exceptions... yes
checking for gcc option to produce PIC... -fno-common -DPIC
checking if gcc PIC flag -fno-common -DPIC works... yes
checking if gcc static flag -static works... no
checking if gcc supports -c -o file.o... yes
checking if gcc supports -c -o file.o... (cached) yes
checking whether the gcc linker (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld) supports shared libraries... yes
checking dynamic linker characteristics... darwin19.6.0 dyld
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... yes
checking whether to build static libraries... yes
checking for gcc... (cached) gcc
checking whether the compiler supports GNU C... (cached) yes
checking whether gcc accepts -g... (cached) yes
checking for gcc option to enable C11 features... (cached) none needed
checking whether gcc understands -c and -o together... (cached) yes
checking dependency style of gcc... (cached) gcc3
checking for g++... g++
checking whether the compiler supports GNU C++... yes
checking whether g++ accepts -g... yes
checking for g++ option to enable C++11 features... none needed
checking dependency style of g++... gcc3
checking how to run the C++ preprocessor... g++ -E
checking for ld used by g++... /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld
checking if the linker (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld) is GNU ld... no
checking whether the g++ linker (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld) supports shared libraries... yes
checking for g++ option to produce PIC... -fno-common -DPIC
checking if g++ PIC flag -fno-common -DPIC works... yes
checking if g++ static flag -static works... no
checking if g++ supports -c -o file.o... yes
checking if g++ supports -c -o file.o... (cached) yes
checking whether the g++ linker (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld) supports shared libraries... yes
checking dynamic linker characteristics... darwin19.6.0 dyld
checking how to hardcode library paths into programs... immediate
checking pkg-config m4 macros... no
configure: error:
pkg-config is required.
See pkg-config.freedesktop.org
The assert
macro is used to check certain conditions, including that the calling application is behaving as intended by passing in proper input. Any application-triggerable assert
calls should be turned into failures that return the error.
Sorry if this is a really silly issue but the following two seemingly paradoxical paragraphs are killing me.
Both:
More importantly, the verifier will also work for macaroons we've not yet seen,
like one that only permits Alice to deposit into her account:>>> N = M.add_first_party_caveat('action = deposit') >>> V.verify(N, secret) True
and
Equally important is the verifier's ability to reject improper macaroons because
they are not authorized. An improper macaroon could have additional caveats not
known to the verifier, or have false general caveats. Worse, an unauthorized
macaroon could be an attempt by a determined attacker to break into our bank.
The verifier we've built will reject all of these scenarios:>>> # Unknown caveat >>> N = M.add_first_party_caveat('OS = Windows XP') >>> V.verify(N, secret) Traceback (most recent call last): Unauthorized: macaroon not authorized
(emphasis mine)
cannot be true. It cannot be true that we can add a caveat that the verifier doesn't know about, and have it both succeed and fail, and indeed if you add a caveat like action = deposit
to the macaroon and try to use the verifier without updating it, verfication fails.
What am I missing?
There seems to be nothing in the repository that documents how
to build the libmacaroons library.
I got as far as:
autoconf
./configure
make
At this point, we get this error:
make --no-print-directory all-am
CC base64.lo
CC macaroons.lo
macaroons.c:48:2: error: #error portability problem
#error portability problem
^
make[1]: *** [macaroons.lo] Error 1
make: *** [all] Error 2
because libbsd (which doesn't exist on Linux) is required.
It seems that this is to get the arc4random_buf function,
which doesn't exist in Linux. How about using getrandom instead in Linux? ... except that getrandom
doesn't appear to exist in Ubuntu 14.04 which I'm using. Perhaps
the only decent solution is to open /dev/urandom directly.
If that include is deleted, we then get a bunch of errors like this:
port.c: In function 'macaroon_bin2hex':
port.c:130:5: error: 'for' loop initial declarations are only allowed in C99 mode
for (size_t i = 0; i < bin_sz; ++i)
^
port.c:130:5: note: use option -std=c99 or -std=gnu99 to compile your code
make[1]: *** [port.lo] Error 1
make: *** [all] Error 2
because declaring variables inside for loops isn't pre-C99 standard C.
Having fixed that, it finally compiles (but really it shouldn't because
there's no definition of arc4random_buf available - if the declaration in
port.c was removed, then at least the compile should fail correctly,
but it doesn't because somehow the warnings for use of undeclared functions
seem to be disabled).
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.