diafygi / acme-tiny Goto Github PK

FreeBSD foo.u 10.1-RELEASE-p24 FreeBSD 10.1-RELEASE-p24 #0: Mon Nov 2 12:17:28 UTC 2015 [email protected]:/usr/obj/usr/src/sys/GENERIC amd64
apache24-2.4.17

if challenge dir has default ownership, apache can not see it

# ls -ld challenges
drwx--x---  2 acme  staff  512 Dec  6 01:42 challenges/

You don't have permission to access /.well-known/acme-challenge/ on this server

so i hack it to www:www

# ls -ld challenges
drwx--x---  2 www  www  512 Dec  6 01:42 challenges/

and a browser can see the challenges dir, e.g.

Index of /.well-known/acme-challenge

but i run script as user acme and

foo.u:/home/acme> python acme_tiny.py --account-key ./account.key --csr ./domain.csr --acme-dir /home/acme/challenges > .signed.crt
Parsing account key...
Parsing CSR...
Registering account...
Already registered!
Verifying cache0.sea.rpki.net...
Traceback (most recent call last):
  File "acme_tiny.py", line 195, in <module>
signed_crt = get_crt(args.account_key, args.csr, args.acme_dir, LOGGER)
  File "acme_tiny.py", line 125, in get_crt
wellknown_path, wellknown_url))
ValueError: Wrote file to /home/acme/challenges/rSy_-pozern-II3T2jbHeclPoqvfxy7pQF1AeOCj6v8, but couldn't download http://foo.u/.well-known/acme-challenge/rSy_-pozern-II3T2jbHeclPoqvfxy7pQF1AeOCj6v8

no help in apache error log. clue bat please

You asked me to read the source code, so I did:

As far as I can see, if LE issue a challenge token of '../../../../../etc/passwd' (or some writable file) you'll blat that file.

Ummhh..Maybe am i overlooking something. Doesn't matter, just let me ask ... could Apache servers willing to use acme-tiny have their little place in the acme-tiny documentation in "a example" kind of ? I'm willing to participate !

thanks beforehand

The suggested renewal script includes:

wget lets-encrypt-x1-cross-signed.pem

This will work the first time. The subsequent runs will create lets-encrypt-x1-cross-signed.pem.1, and so on.

Suggested fixes:

wget -p
or
wget --backups=1

or, in fact, use --backups=1 and then check to see if there's an update:
diff --backups=1 lets-encrypt-x1-cross-signed.pem{,.1}

Sorry not really an issue with this project.. but is there a way to convert the account key json file into a pem format that can be used with acme-tiny?

I've a bootstrapping issue, my domain uses HSTS and redirects 80 to 443, meaning I cannot easily validate via HTTP. I can however very easily poke my DNS API with a TXT record output by acme-tiny. Please consider, thanks.

For security reasons, I have added a letsencrypt user with no valid shell. I run acme-tiny like this:

su -s /usr/bin/python -c "/home/letsencrypt//acme-tiny/acme-tiny.py --account-key ... etc." letsencrypt

The -s option sets a temporary shell for the user run by su. However, for some reason, it doesn't like it when I do that:

  File "<string>", line 1
    /home/letsencrypt/acme-tiny/acme-tiny.py --account-key ...
    ^
SyntaxError: invalid syntax

I'm unsure if this is a python or acme-tiny bug. It runs fine when I do it this way:

    su -s /bin/sh -c "python /home/letsencrypt//acme-tiny/acme-tiny.py --account-key ... etc." letsencrypt

But that involves an unnecessary extra step via an sh shell. Can it be avoided?

Thanks a lot for the script!

Unfortunately the last step fails with the error below.

Note: I am using the private key generated by le.

Signing certificate...
Traceback (most recent call last):
  File "acme_tiny.py", line 199, in <module>
    main(sys.argv[1:])
  File "acme_tiny.py", line 195, in main
    signed_crt = get_crt(args.account_key, args.csr, args.acme_dir, log=LOGGER, CA=args.ca)
  File "acme_tiny.py", line 162, in get_crt
    raise ValueError("Error signing certificate: {0} {1}".format(code, result))
ValueError: Error signing certificate: 400 {"type":"urn:acme:error:malformed","detail":"Error creating new cert :: Certificate public key must be different than account key","status":400}

Could you document the fuse requirements in the tests?

It's not installed on the OS by default, which makes it harder to reproduce the tests locally. Would be nice to refactor out in order to run the tests without this system dependency.

re: "You must have a public key registered with Let's Encrypt and sign your requests with the corresponding private key"

I've used the official client several times now (since public beta) and not ever had to register an account level public key. Was that a requirement prior to the public beta?

However, I'd like to use acme-tiny instead of the official version for some automation, because its much easier to understand, and have no trouble generating my own keys/CSRs. I'm just curious if we need to remove the parts that handle the account.key....

If we still need the account.key registered with LE, how is that accomplished? Like I said, used the official client several times without registerign a public key, and don't see any obvious means to do that through LE (either their site or the client)

hello,

if you want, you may use this default nginx config:

server {
    listen 80;
    server_name _;

    location / {
        return 301 https://$host$request_uri;
    }
    location ~ ^/\.well-known/acme-challenge/([-_a-zA-Z0-9]+)$ {
        default_type text/plain;
        return 200 "$1.ACCOUNT_THUMBPRINT";
        break;
    }
}

or just add location wherever you want.

I like to add some information - apache configuration, location of openssl.cnf on other systems than Debian - to your README.md, but I fear that will grow too big.

Where shall I put them?

I'm not sure if this can be used in python 3. Can someone review this script to see what needs to be changed to add python 3 support?

It would be wonderful if the script would return i. e. a 1 exit code if everything went well and a certificate has actually been updated by a renewal. This would ensure that the reload of the webserver only happens when it is really necessary.

I'd like to run this script locally and then upload the files to the proper server (that way I never need to put my account key on the actual servers where).

acme-tiny should offer an option (-o or --out) to write the obtained certificate to a file upon success, instead of writing to the standard output.

For reference, the current documented usage is as follows:
python acme_tiny.py --account-key ./account.key --csr ./domain.csr --acme-dir /usr/share/nginx/html/.well-known/acme-challenge/ > signed.crt

The current workflow (writing to stdout and letting the user redirect to a file) is problematic for automated generations of certificates.
When using the example command, signed.crt is (over)written even when the generation fails.
If the cert generation fails while trying to update a existing cert, the existing cert will be overwritten and lost.

I realize the user could workaround this (redirecting to temp file, then somehow verify that the file is a valid cert, then replacing the old cert), but it would be much easier to just support the option in acme-tiny

I know that one of the goals of this project would be to stay below 200 lines of code (at the time of this issue script has 198 lines already), but would there be a chance for the script to support specifying an email for account registration? That should be helpful for account recovery in the future.

For users who need to upload the challenge to another host that does the serving (e.g. with Google App Engine), it would be helpful to pause, print the challenge, and wait for user input before the challenge continues.

If I'm not mistaken, this would happen around line 119

A simple pause, giving the user time to upload the challenge e.g.:

# import os, above
print "Challenging {} for token {}".format(
  wellknown_url, keyauthorization
)
os.system('pause')

It might make sense if this coder were run only when a --manual param is given (much like letsencrypt-auto).

Hi,

Cronjobs are deprecated and are replaced by systemd Timers. So, it would be cool to document in https://github.com/diafygi/acme-tiny/blob/master/README.md how you use systemd Timers with acme-tiny instead of some cronjobs.

Thanks.

Would be great for automated updates and in general a more sane setup. ;-)

I used the intermediate like you to create the chained PEM file:

wget -O - https://letsencrypt.org/certs/lets-encrypt-x1-cross-signed.pem > intermediate.pem
cat signed.crt intermediate.pem > chained.pem

https://www.ssllabs.com claims that the chain is incomplete.

This server's certificate chain is incomplete. Grade capped to B.

I couldn't figure out which additional certificates must be inserted to the chain.

I have generated the current SSL certificate on my laptop with the certonly --manual options in letsencrypt, but I prefer using acme-tiny directly on the server (that runs Nginx) so it's possible to retrive the private key and the CSR from /etc/letsencrypt/ ??

I'd rather not have the account key on publicly accessible server. Thus I need to create the certificates locally and copy them over to the server later. However, in order to make the domain verification pass I need the respective challenge files on the server.

Thanks for the great client.

I would like to have an parameter to upload the weil known file to an other loaction. For example call:

/opt/acme-tiny/acme_tiny.py --account-key ./account.key \
     --csr ./domain.csr --acme-dir /var/tmp/acme-rsync/ \
     --acme-dir-action 'scp /var/tmp/acme-rsync/ webost:/var/www/acme-root/' > ./signed.crt

The command should be called after writing the wellknown_file

          token = re.sub(r"[^A-Za-z0-9_\-]", "_", challenge['token'])
          keyauthorization = "{0}.{1}".format(token, thumbprint)
          wellknown_path = os.path.join(acme_dir, token)
          with open(wellknown_path, "w") as wellknown_file:
              wellknown_file.write(keyauthorization)

          EXECUDE_THE_CMD_HERE
          # check that the file is in place

I'm not sure what should be changed to add support for python 2.6. Can someone please review this and see what, if anything needs to be changed?

it is possible to not use port 80 for verify?

I have an website already using HTTPS with valid Let's Encrypt certificates. The server listens on HTTPS and HTTP is disabled. I don't want to set an HTTP server just for renewing the certificate (which is possible, but a lot complicated in my case). Looking some issues in LetsEncrypt repository seems it's possible, but depends on the client software.

As part of the registration object you can set an email address for the account which can later be used for account recovery in the event of losing your account key. It could be beneficial to add support for setting an email address.

{
  "resource": "new-reg",
  "contact": [
    "mailto:[email protected]",
    "tel:+12025551212"
  ],
  "agreement": "https://example.com/acme/terms",
  "authorizations": "https://example.com/acme/reg/1/authz",
  "certificates": "https://example.com/acme/reg/1/cert",
}

https://letsencrypt.github.io/acme-spec/#rfc.section.5.2

But it's shown in the nginx config. Maybe add it as another note in step 5?

I'm new to LetsEncrypt and tried to understand it's protocol by reading this document https://letsencrypt.org/howitworks/technology/.

Is it possible to authenticate the account.key with the DNS as the above document suggest ?
This would allow to generate keys not intended to be used for https sites.

I think this may have been alluded to in #14, but it appears that this script fails on sites that are configured to redirect all HTTP traffic to HTTPS when running under Python 2, presumably due to a limitation of the Python2 urllib2.

E.g. I have Apache setup to send temporary 302 redirects to send all HTTP traffic on my site to HTTPS. When I run the script using Python 2 I get:

$ python src/acme-tiny/acme_tiny.py --account-key ./account.key --csr ./request.csr --acme-dir /var/www/.well-known/acme-challenge/ > ./signed.crt
Parsing account key...
Parsing CSR...
Registering account...
Already registered!
Verifying www.andysayler.com...
Traceback (most recent call last):
  File "src/acme-tiny/acme_tiny.py", line 198, in <module>
    main(sys.argv[1:])
  File "src/acme-tiny/acme_tiny.py", line 194, in main
    signed_crt = get_crt(args.account_key, args.csr, args.acme_dir, log=LOGGER, CA=args.ca)
  File "src/acme-tiny/acme_tiny.py", line 123, in get_crt
    wellknown_path, wellknown_url))
ValueError: Wrote file to /srv/www/.well-known/acme-challenge/i4yT4IdRFELqbe3MK9RjKrWB_Ke0c8pESDsFFypE2IM, but couldn't download http://www.andysayler.com/.well-known/acme-challenge/i4yT4IdRFELqbe3MK9RjKrWB_Ke0c8pESDsFFypE2IM

Running the same command with Python 3 works correctly:

$ python3 src/acme-tiny/acme_tiny.py --account-key ./account.key --csr .request.csr --acme-dir /srv/www/.well-known/acme-challenge/ > ./signed.crt
Parsing account key...
Parsing CSR...
Registering account...
Already registered!
Verifying www.andysayler.com...
www.andysayler.com verified!
Verifying andysayler.com...
andysayler.com verified!
Signing certificate...
Certificate signed!

Since redirecting all HTTP traffic to HTTPS is likely a pretty common practice amongst LE users, and since Python2 is still fairly pervasive - this might be worth address -- although I'm not sure of the best solution short of using better HTTP libraries or adding extra redirect handling code to make up for the limitations of urllib2.

Short of an actual fix, it might be nice to document this limitation in the README.

I'm happy to take a crack at a fix, but I'm not sure it can be done without adding additional lines of code over your 200-line limit...

We import acme-tiny to a script on our servers, run by cron. I would like to have no output if everything is okay but sure, if errors occure, I would like to see them.

At the moment, acme-tiny writes regular status messages to stderr, so to get rid of them, I would have to trash stderr. This would hide all errors, too.

I made a proof of concept in my local fork but that pushes the code to 201 lines, so I'm not gonna make a pull request. See bwurst/acme-tiny@20be6f6 for my current solution.

#!/usr/bin/sh
python /PATH/TO/acme_tiny.py --account-key /PATH/TO/acme-tiny/account.key --csr /PATH/TO/acme-tiny/domain.csr --acme-dir /var/www/challenges > /PATH/TO/acme-tiny/signed.crt
wget -O - https://letsencrypt.org/certs/lets-encrypt-x1-cross-signed.pem > /PATH/TO/acme-tiny/intermediate.pem
cat /PATH/TO/acme-tiny/signed.crt /PATH/TO/acme-tiny/intermediate.pem > /PATH/TO/cme-tiny/chained.pem
rsync -v -b --backup-dir=/etc/letsencrypt/XXXXX/BACKUP/ --suffix=".$(date +%F)" /PATH/TO/acme-tiny/chained.pem /etc/letsencrypt/XXXX/chained.pem
service nginx reload

might be helpful to use rsync -b to backup the existing cert before you replace it with the new cert (just in case something goes wrong). It'll move the old cert to the /BACKUP/ folder and rename it with the current date. Not a big deal, I just find it reassuring to have the old cert available just in case.

I used acme-tiny to get a certificate signed by letsencrypt but I just realized that I forgot to add a subdomain. How can I revoke the certificate before getting a new one signed? There seems not to be an option using acme-tiny.
Anyone here who is able to help me? Thanks!

Why duplicate efforts if there is https://github.com/kuba/simp_le? I think we pretty much have the same goal...

So when I run the script it claims that it wrote the file to /var/www/challenges/ ...but when I check it didn't actually write it so then the challenge obviously fails. Presumably there should be error handling to check if the challenge was correctly written? Seems right now it just assumes it did/could write it and then it carries on based on that assumption until the check fails.

python acme_tiny.py --account-key ./account.key --csr ./domain.csr --acme-dir /var/www/challenges/ > ./signed.crt
Parsing account key...parsed!
Parsing CSR...parsed!
Registering account...already registered!
Verifying XXXXXX...Traceback (most recent call last):
  File "acme_tiny.py", line 195, in <module>
    signed_crt = get_crt(args.account_key, args.csr, args.acme_dir)
  File "acme_tiny.py", line 127, in get_crt
    wellknown_path, wellknown_url))
ValueError: Wrote file to /var/www/challenges/YYYYYYYYYYYY, but couldn't download http://XXXXXXX/.well-known/acme-challenge/YYYYYYYYYYYY

Hello,

Let's Encrypt now support ECC keys,is it possible for you to update this script ?
You can check here:
https://community.letsencrypt.org/t/ecdsa-testing-on-staging/8809

Thank you for your kind reply.
Li

Hi,

I'm trying to use acme_tiny, but I can't get it to work. If I use python3, it spits a long traceback about a bad header, so I tried using python2.7. Here is the traceback I get :

Parsing account key...
Parsing CSR...
Registering account...
Already registered!
Verifying <domain>...
Traceback (most recent call last):
  File "/usr/bin/acme_tiny.py", line 198, in <module>
    main(sys.argv[1:])
  File "/usr/bin/acme_tiny.py", line 194, in main
    signed_crt = get_crt(args.account_key, args.csr, args.acme_dir, log=LOGGER, CA=args.ca)
  File "/usr/bin/acme_tiny.py", line 118, in get_crt
    resp_data = resp.read().decode('utf8').strip()
  File "/usr/lib64/python2.7/encodings/utf_8.py", line 16, in decode
    return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode byte 0x80 in position 0: invalid start byte

My domain is a pretty standard ascii one, nothing fancy. My system's locale is en_US.UTF-8

I don't know how well this fits with the goal of keeping under 200 lines, but: I'm getting frequent errors like:

urllib2.URLError: <urlopen error [Errno 101] Network is unreachable>

which are usually transient -- waiting a short time and retrying makes it go away. It would be nice if acme-tiny could take care of this itself, because otherwise it puts more complexity in the autorenew cron job.

Make acme_tiny produce error messages suitable for end user consumption. Any error message goes to stderr must be suitable for end user. For debugging and logging purposes, add the option '--log filename' to output verbose messages to 'filename'.

Any error messages returned by Let's Encrypt should be safe to be presented directly to end users. Others should generate generic error messages.

Hello,

From the ACME spec (emphasis is mine):

For challenges where the client can tell when the server has validated the challenge (e.g., by seeing an HTTP or DNS request from the server), the client SHOULD NOT begin polling until it has seen the validation request from the server.

Sure, “SHOULD” is used and not “MUST”, so it's not such a big issue I guess.

I understand that by polling every 2 seconds, your implementation is simpler, so it's not likely to change.

My suggestion is to do the following:

• Acknowledge this point in the readme, and point to alternatives (see below);
• Maybe use a random time between each poll? I am not sure if that would help against overwhelming the server. Alternatively, increase the delay?

I believe I will end up using your code but modifying it a little bit so that it does not do polling. I would like to have a list of possibles ways to do so for the Simple HTTP validation challenges. Here is what I came up with, but I am sure I am missing a few ;-)

• Instead of serving static files for /.well-known/acme-challenge/, have some code that also notices when the files are being requested;
• Watching / parsing the logs of the web server;
• Watching for the time of last access (atime) of the file (does not seem to be reliable).

Any other ideas?

I don't know enough Python to fix this effectively, but when I ran the script on my server (CentOS running Python 2.6.6), the HTTPError object that I got back when checking if my account had been registered didn't have a reason property. As a result, it crashed on line 62 (https://github.com/diafygi/acme-tiny/blob/master/acme_tiny.py#L62) when it tries to return the result.

I got around this by just replacing the second call to getattr() with a string literal, but since the result isn't actually used anywhere in the script that I can see, other than for debugging logs (everything else branches off the status code), maybe it could be eliminated? Or the e.reason.__str__ default could be wrapped in another call to getattr() with a sensible default?

I'm reliably getting this error:

Parsing account key...
Parsing CSR...
Registering account...
Already registered!
Verifying [myactualserver.example.org]...
Traceback (most recent call last):
  File "~/acme-tiny/acme_tiny.py", line 198, in <module>
    main(sys.argv[1:])
  File "~/acme-tiny/acme_tiny.py", line 194, in main
    signed_crt = get_crt(args.account_key, args.csr, args.acme_dir, log=LOGGER, CA=args.ca)
  File "~/acme-tiny/acme_tiny.py", line 128, in get_crt
    "keyAuthorization": keyauthorization,
  File "~/acme-tiny/acme_tiny.py", line 62, in _send_signed_request
    return e.code, e.read()
AttributeError: 'URLError' object has no attribute 'code'

An error in the error-handling, perhaps, or am I missing something? This is python 2.7.5 on CentOS 7; a similar error happens with python 3.4.

I was trying to investigate further, but I think I've gotten myself onto a blocklist for over-trying. :)

While it's just the local path which can be anything, it will be rather less confusing if the example matches the actual URL which is accessed, which is /.well-known/challenge/ (without the final "s").

If the "s" is actually intentional, I'd suggest just dropping the .well-known part from the local path.

acme-tiny always overwrites the existing cert. If a 429 error is returned from the CA, indicating denial due to the rate-limiting policy, an existing cert will be overwritten with a zero-length file.

To reproduce: call the renewal script several times in a day, until the rate-limiting policy is triggered. 11 times in a row should do it.

Desired behavior: when receiving an error, an existing valid cert is not overwritten.

Hello!

tried your script on pfSense.... and see this:

[2.2.5-RELEASE][root@gw]/root/acme-tiny: python acme_tiny.py --account-key ./account.key --csr ./domain.csr --acme-dir /usr/local/etc/nginx/letsencrypt/ > ./signed.crt
Parsing account key...
Parsing CSR...
Registering account...
Registered!
Verifying Common...
Traceback (most recent call last):
  File "acme_tiny.py", line 198, in <module>
    main(sys.argv[1:])
  File "acme_tiny.py", line 194, in main
    signed_crt = get_crt(args.account_key, args.csr, args.acme_dir, log=LOGGER, CA=args.ca)
  File "acme_tiny.py", line 104, in get_crt
    raise ValueError("Error requesting challenges: {0} {1}".format(code, result))
ValueError: Error requesting challenges: 400 {"type":"urn:acme:error:malformed","detail":"Error creating new authz :: DNS name does not have enough labels","status":400}

Line 146 of README.md:

ssl_dhparam /path/to/server.dhparam;

I assume it's basically

openssl dhparam -rand /dev/urandom 4096 -out server.dhparam

but since everything else was explicitly mentioned I figure this should be as well...

The requests library solves a lot of HTTP communication problems, but most notable among these might be automatic redirection (especially from http -> https`) for the challenge.

http://docs.python-requests.org/en/latest/user/quickstart/#redirection-and-history

It should be a reasonably straightforward drop-in.

Cheers

Let's Encrypt doesn't guarantee the use of a specific intermediate certificate for signing. Therefore it's currently not possible for a user of acme-tiny to create a chain.pem and fullchain.pem without the risk of breaking his setup when Let's Encrypt (suddenly) switches to Let’s Encrypt Authority X2. To solve this problem, a ACME server must send a Link: <https://example.com/acme/ca-cert>;rel="up";title="issuer" header when retrieving a signed certificate.

A possible solution I can think of is, that acme-tiny follows this link, appends all intermediate certificates to the signed certificate and therefore actually outputs/returns a fullchain.pem.

Installing instructions are not covering the generation of the dhparam at all, but it's refered to in step 6 when installing the certificate.