taoeffect / empress Goto Github PK

• All data should be in /root/.tarsnap, this includes the tarsnap.key (#53) and the cache directory
• The /root/.tarsnaprc should look like so:

keyfile ~/.tarsnap/tarsnap.key
cachedir ~/.tarsnap/cache
exclude ~/.tarsnap/cache
humanize-numbers
print-stats
totals

Having Tarsnap's key and cache directory in separate locations is ugly and confusing. Plus currently the cache directory is world-readable, and that's not good as it contains plaintext filenames.

Users should follow the tarsnap instructions and place the key in /root/.tarsnap/tarsnap.key. Having it look for the key in roles/tarsnap/files/decrypted_tarsnap.key is potentially dangerous (could get committed and pushed somewhere it's not supposed to), and it's just a weird location for it. I think we already did this with the SSL certs, so let's do it for tarsnap.

• Reference /root/.tarsnap/tarsnap.key in .tarsnaprc
• Update README.md as needed

Some packages should be installed from backports or unstable. Nginx is a good candidate, since the version in stable is ancient.

However, as per our experiments installing emacs24-nox from backports, we need to make sure that we have the right pinning, otherwise the installation seems to not work. The pin should be equivalent to whatever is the default for stable (I think) to fix this.

Empress can copy over a file into /etc/apt/preferences.d to enforce this.

Once we get #8 (nginx) we should add whatever the DSPAM web interface is.

Leftover from sovereign.

Requirements:

• All traces of Apache server must be obliterated (see also #6)
• Must not destroy or overwrite existing nginx installations / configurations
• Must be secure (see bettercrypto and mozilla's thing)
• Nginx config must NOT enable HSTS.
• Primarily used for the purpose of Mailpile (#3) without stepping on the toes of other sites on the server.
• Install from backports (depends on #7).

At the time of posting, for TLS certs relevant file is here (at a specific commit, so might be outdated).

That, once #37 is closed, will all be put into a single folder (which can be rm -rf'd). But right now the public/private keys and certs are in separate locations:

• /etc/ssl/private/wildcard_private.key
• /etc/ssl/certs/wildcard_public_cert.crt
• /etc/ssl/certs/wildcard_ca.pem
• /etc/ssl/certs/wildcard_combined.pem
• /etc/ssl/private/wildcard.csr
• /etc/ssl/private/openssl.cnf (only needs to be deleted if domains change; possibly create separate issue)

Plus there's the OpenDKIM stuff that could be reset:

• /etc/opendkim/keys/{{ stuff }} (plus the signing tables and config files)

I don't know why sovereign doesn't already do this (given that they linked to this blog post that uses it), but it seems like installing and setting up dovecot-antispam might be a good idea.

• It creates a million backups each time it is run because it runs tarsnap for each DIR. Make it run the command only once.
• The silly -L option (#55)
• It shouldn't run pg_dumpall if it doesn't exist.
• Run something for mariadb

See comments in sovereign/sovereign#318.

ssl_protocols = !SSLv2 !SSLv3 !TLSv1 doesn't work with Apple Mail.

ssl_protocols = !SSLv2 !SSLv3 does.

So, related to sovereign/sovereign#251, and related to my comment here (which i'll quote here):

For a future PR, let's move the keys to one folder (both the .key and the .crt), and let's put it in a place that's recommended by dovecot, which I believe @al3x also created an issue for in sovereign.

Also worth doing, as part of this issue or a separate one, moving roles/common/files/wildcard_private.key (the user's key) to a top level folder called secrets instead of buried within the roles.

So this is a two parter:

1. Place .key and .crt into "the right place" on the server, and make that place a single folder so that it's easy to re-generate keys by simply deleting it.
2. Create a secrets folder in this repo at the top level and tell users to put their private key. It's best to not distribute a "default key" the way sovereign is currently doing, as that is ... how you say... something that people should be sued over (default passwords = negligence).

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

• /decrypted should be a variable defined in vars/defaults.yml and overriden in vars/user.yml
• Its permissions must be very well thought out, especially since it might be used with other sovereign modules as per #18.

note to @al3x: we got rid of encfs and don't recommend it or any other system whereby a private key is stored on the same server as the encrypted data.

In some setups on webhosts (like DreamHost's), all "root folders" are for some reason created as sub-folders of INBOX.

Apple Mail seems to automatically detect this and set an INBOX prefix:

Thunderbird, however, does not, and will show them all as subfolders of INBOX. To fix this, users need to manually go to Account Settings > [account] > Server Settings > Advanced... and specify INBOX/ for the IMAP Server Directory:

Not something we should be depending on or doing.

It would be ideal to not "brand" the machine with domain as defined in vars/user.yml, but instead simply have a list of domains with no "primary" one.

If a primary one is required, its implications (for example, the fact that using mail from the terminal will have a FROM: header of {{ domain }}) need to be thoroughly mapped out and documented. Any unnecessary externalities must be eliminated.

Ideally, this is how it should work:

1. Select desired functionality (ex: send mail, receive mail, webmail, migration, etc.)
2. For each functionality, select desired implementation (ex: postfix vs exim, dspam vs spamassassin, mailpile vs roundcube, etc.)
3. For each implementation, select desired options and dependencies (ex: backend for email from [sqlite, mariadb, pqsql, etc.]; for migration [enter emails/passwds]; for basics [email accounts & passwords]; etc.)

Click Go in some interface.

cc @PiPeep

dsync, even with the experimental 2.2.15 build, results in:

"start": "2014-11-16 09:22:50.461509", "stderr": "dsync([email protected]): Error: Synchronization corrupted index 
header: (in-memory index)\ndsync([email protected]): Warning: fscking index file (in-memory index)\ndsync(greg@some
website.com): Error: Synchronization corrupted index header: (in-memory index)\ndsync([email protected]): Warning: 
fscking index file (in-memory index)", "stdout": ""}

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

The docs don't specify where to clone the repo to.

There are two options:

1. Clone to deploy machine (locally, your laptop, etc.)
2. @bnvk's (and my) use case (for email migration etc) clone to remote machine.

These two options need to be documented, probably on a wiki that's linked from the readme.

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Goes with #9.

We're not using postgres: https://github.com/taoeffect/empress/blob/master/vars/user.yml

To speed up the migration, and to fix migration errors, users should install dovecot from Debian testing or unstable (or whatever's necessary to get version 2.2+).

Speeding up also involves removing the {# fetch-headers #} jinja comment in migration/tasks/main.yml.

Currently Sovereign/Empress only support one wildcard cert for a single domain.

We should instead support one wildcard cert per domain. A single server can have multiple domains that it's managing email for, and each of those will have a different domain.

Obviously, we should also do SHA256 hashes.

Here's how to do a self-signed wildcard cert w/sha256:

1. Copy one of these files:

/usr/lib/ssl/openssl.cnf
/usr/share/doc/dovecot-core/dovecot/dovecot-openssl.cnf
/usr/share/dovecot/openssl.cnf

Edit: some research needs to be done to decide which one.

2. Make sure it has the following sections in it

[ v3_req ]

# Extensions to add to a certificate request

basicConstraints = CA:FALSE
keyUsage = nonRepudiation, digitalSignature, keyEncipherment
subjectAltName = @alt_names

[ alt_names ]

DNS.1 = {{ domain }}
DNS.2 = mail.{{ domain }}

EDIT: An alternative is to include all the domains as alt_names, and use a single SSL cert...

3. Create self-signed wildcard cert for the domain

# openssl genrsa 2048 > {{ domain }}.key
# openssl req -new -out {{ domain }}.csr -key ./{{ domain }}.key -config ../openssl.cnf -sha256
# openssl x509 -req -days 1460 -in {{ domain }}.csr -signkey ./{{ domain }}.key -sha256 -out {{ domain }}-wildcard-ss.pem -extensions v3_req -extfile openssl.cnf
# openssl x509 -fingerprint -text -noout < {{ domain }}-wildcard-ss.pem > {{ domain }}-wildcard-ss.pem.info

For StartSSL signed certs

If they go with StartSSL, they do something similar to the above, except after generating the CSR they give it to StartSSL and then download their public cert, plus StartSSL's intermediate cert, and then do:

# cat {{ domain }}.pem sub.class1.server.ca.pem > {{ domain }}-unified.crt
# openssl x509 -fingerprint -text -noout < {{ domain }}-unified.crt > {{ domain }}-unified.crt.info

Related issues: #50, #52, #47

1. Migrate using larch --all
2. Iterate over subfolders of INBOX using doveadm mailbox list -u {{ email }} "INBOX.*"
3. If a mailbox of the same name as that subfolder exists in the parent directory already (mailbox A), move all of the email inside of A into the subfolder (mailbox B), delete A, then move B up one directory

Note that some people actually have INBOX.INBOX. If this is found, skip it and all subfolders of INBOX.INBOX.

As discussed and explained in #18, we need to modularize all of the services provided into individual roles that can be, at the user's discretion, commented out and not run. HT to @PiPeep for suggesting this approach.

So that implies we must get rid of the common role completely and split all of its services out into separate roles.

StartSSL doesn't support free wildcard certs. Need to document how to create free TLS certs w/CSRs for arbitrary domains with StartSSL.

• Explain what part of the vars/user.yml is responsible for migration and give an example
• Warn the user that during the migration their password will be exposed via server logs (since it's passed in on the command line as a parameter), and that any other users on the system may be able to see it while it's running via ps auwwwx, htop, etc.
• Make sure that the migration is run locally on the server (in tmux) and not remotely from their control machine (as is currently being done). Issue #28 created for that.

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

If you're using Ansible to help other people, this convention of separating everything into the smallest possible service-focused roles is the Right Thing to Do™.

We're going to do this for issue #56 Get rid of common.

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

I've gotten four of these emails today and DSPAM still declares "Viagra_Cialis" (of all things) "Innocent" with high confidence!

X-DSPAM-Result: Innocent
X-DSPAM-Processed: Mon Nov 24 23:19:33 2014
X-DSPAM-Confidence: 0.9899
X-DSPAM-Probability: 0.0000

In fact, I'm not sure it's classified anything as spam yet... Related to #22.

It takes way too long (especially the migration part, which will take hours for most people) and the potential for something to go wrong because of a dropped connection is not worth the risk.

@PiPeep says we should do this when we've got nginx.

And ideally a replacement that doesn't use Java...

Larch will preserve the INBOX. prefix that's found on many servers, whereas dsync, with the way we're using it, won't. This will result in a situation where if both methods are used to sync mail, a whole bunch of duplicate folders will be created.

Meaning, larch will create .INBOX.foobar and dsync will create .foobar.

Since larch does this on its own and there doesn't seem to be a way to fix it, I think we should change dsync to behave like larch does (since I'm guessing that's a possibility).

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

@MacLemon points out that C2S should be TLSv1.2. Our issues with TLSv1.2 were S2S (server-to-server). Figure out what parts can be safely made TLSv1.2 while preserving current defaults for everything else.

Yes.

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

During my tests with ansible 1.7.1 (perhaps that's the problem?) authdb.sqlite didn't get filled. The command is correct (since that's how I manually fixed it), but for whatever reason it's not working via ansible.

postfix/postscreen[11215]: close database /var/lib/postfix/postscreen_cache.db: No such file or directory (possible Berkeley DB bug)

?

Running ansible-playbook -vvv migration.yml locally on the server (in tmux), I noticed:

• the migration will run regardless of the existence of dovecot.index.log
• it will continue running the background if you ^C it, and the only safe way I could figure of killing it was to first systemctl stop dovecot, then kill -9 the dsync processes, restart dovecot, comment out the creates: thing (since it wasn't doing anything anyway), and restart the sync process.

We need to:

1. Figure out why creates: is being ignored
2. Make sure that when the user ^C's the dsync, it actually stops the sync.

Right now it assumes postgresql is installed and tries dumping the database and backing that up.

It should:

• Backup MariaDB/MySQL (only if it exists)
• PostgreSQL (only if it exists)

This is a requirement if we want to support a web-admin interface in the future. We don't want some web framework software running as root.

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Related to #11.

Need to have in the README a step-by-step guide of what to setup, what files to edit, example session, etc.

By default we'll use Mailpile, but if someone wants to use Roundcube instead they should be able to choose that.

Obviously depends on #8 (nginx).

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

These can be inferred from mail_virtual_users, right?

cron stuff gets lost currently, php stuff could get lost, etc.

So mail_virtual_domains is often looped through in configuration files. In some places mail. is prepended and in some places it's not. For example, the recent PR #35 loops through this variable in openssl.cnf:

https://github.com/taoeffect/empress/pull/35/files#diff-fc4a7a00381d9bd47edfe77044dbca15R43

The user should be the one specifying what their mail server is called, and this should not be mutated by the scripts.

This screen illustrates the problem as well:

Cron sends these messages:

Can't locate Net/IP.pm in @INC (@INC contains: /etc/perl /usr/local/lib/perl/5.14.2 /usr/local/share/perl/5.14.2 /usr/lib/perl5 /usr/share/perl5 /usr/lib/perl/5.14 /usr/share/perl/5.14 /usr/local/lib/site_perl .) at /opt/check-rbl.pl line 33.
BEGIN failed--compilation aborted at /opt/check-rbl.pl line 33.

Sovereign does way too many things, and because of that, each individual thing it does isn't as good as it could be. If sovereign instead were a "package" system for specific ansible goals (setting up email, setting up calendars, setting up personal dropbox, etc.), then the individual packages would be free to focus and specialize on what they do best.

Ideally:

• You'd be able to run each individual sovereign module independently if that's all you wanted (without using sovereign)
• You'd be able to use sovereign and enable different modules (perhaps via git submodule system) that you wanted.

This would make development easier as well, since Al3x would not be burdened with the responsibility of managing all these different tasks; they could instead be delegated out to experts who focus on those tasks specifically.

cc @PiPeep

See vars/defaults.yml and vars/disabled_extras.yml for a complete list. We might want to get rid of the latter file as well.

These should be configurable:

• Number of monthly/weekly backups
• What folders to backup
• What flags should be sent to tarsnap (certainly not -L, see #55).
• Arbitrary scripts to run along with the cron thing

Anything else?

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

1. Someone who has no idea what Ansible is should be able to quickly get up and running.
2. It should be clearly spelled out what files should be edited.
3. It should be clearly spelled out what Empress will do to someone's system.
4. We should link to our own wiki.

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.