dsqmoore / appengine-mailer Goto Github PK

appengine-mailer is an RFC2822-over-HTTP proxy for running in Google App Engine.

appengine-mailer was written by Toby White <[email protected]> (@tow21) for use on the Timetric (http://timetric.com) platform. It was heavily inspired by an earlier proof of concept by Mat Clayton (@matclayton)

Google App Engine includes an API for sending emails, which lets you send up to 7000 emails a day for free; above that it costs $0.0001 per email. (Prices correct as of July 2010).

appengine-mailer exposes this API behind a URL, so that a client can POST serialized email/mime-multipart messages, and have GAE send them on its behalf; in effect using GAE as a mail proxy. 

Anti-spam measures
==================

To avoid spam, Google don't let you send arbitrary email messages. There are several important caveats:

1. The From address must be the email of a registered developer on the GAE app. That is - any From address you want to use, you have to register as a developer on the app. This limits you to 50 possible From addresses. (There are some limitations on which addresses may be used - see Known Issues)

2. Google don't allow you to set arbitrary headers. The list of headers you have control over can be seen at XXX

3. Google don't give you full control over the multipart structure of the email. You can have:
  * plain text emails
  * plain text/HTML multipart/alternative emails
  * and an arbitrary number of attachments, all of which will be appended to the end of the message.

4. There's an upper limit on the number of To/Cc/Bcc addresses that may be used on a single message. It's not documented, but in practice seems to be about 100.

Authorization
=============

Authorization is handled by means of a shared secret between client and server. Each request is sent, signed by the client's secret, and the signature is checked on the server. The server can keep several valid secrets concurrently, allowing key changes to occur seamlessly.

Deploying the server
====================

* Change app.yaml to use an application name controlled by you.

* You need to supply at least one secret key. The server will look for these in a file called GMAIL_SECRET_KEY, one key on each line.

* Optionally, you can supply a default From address, to be used when the address on the email is unsuitable. The server will look for this in a file called GMAIL_DEFAULT_SENDER.

* Having added these files, deploy to GAE.

Deploying the client
====================

The client can be used in three ways:

* Direct from a Python program

* From the command line, like the sendmail command

* As an Exim transport.

In all cases, the client needs to know the URL of the server to talk to, and a secret to use for signing the message. Sometimes, this can be specified - if unspecified, it will always fall back to looking for the server URL first in the environment variable GMAIL_PROXY_URL, then in the file /etc/envdir/GMAIL_PROXY_URL. Similarly, it will look for the secret key first in the environment variable GMAIL_SECRET_KEY, then in the file /etc/envdir/GMAIL_SECRET_KEY.

Finally, since the interface is simple HTTP, you can write a client speaking to the documented interface.

Python client
=============

The Python client only needs the module "gmail.py".

# Using the standard Python email class
from email.message import Message
msg = Message()
msg['From'] = "Toby at work <[email protected]>"
msg['To'] = "Toby at home <[email protected]>"
msg['Subject'] = "Testing"
msg.set_payload("This is the body of the message")

# And passing them message through to gmail
from gmail import GmailProxy, MessageSendingFailure
# You need to specify the SECRET_KEY and APPENGINE_PROXY_URL
gmail_proxy = GmailProxy(SECRET_KEY, EMAIL_APPENGINE_PROXY_URL, fail_silently=False)
try:
    gmail_proxy.send_mail(msg)
except MessageSendingFailure, e:
    print "Failed to send message!\n%s" % e

Django example
==============

As of Django 1.2, there is support for swapping out the default email backend. Add the whole appengine-mailer directory as an app to your Django project, and then update your settings to include:

EMAIL_BACKEND = 'appengine_mailer.django_email_backend.GmailBackend'
EMAIL_APPENGINE_PROXY_URL = "http://your_domain.appspot.com"
# The SECRET_KEY used will be your Django project's SECRET_KEY

Alternatively, if your servers are configured to use Exim, you can configure Exim as below, and then use Django's unmodified email backend to send mail via GAE.


Command-line client
===================

The file "gmail.py" can be used as a partial replacement for /usr/bin/mail. It has no dependencies beyond the Python stdlib (tested with 2.6, should work with earlier versions). For details of options it understands, call it with "--help".

Exim transport
==============

Because gmail.py can be used instead of /usr/bin/mail, it can be dropped into place as a transport, so that all mail going through Exim will be routed via the GAE interface. (subject to Google's limited interface.) An example transport is shown in the file 50_exim4_config_gmail_pipe. If you're using a Debian system:

* Move gmail.py to /usr/bin/gmail, and ensure that it's executable.
* Configure Exim4 to send via a smarthost.
* Replace /etc/exim4/conf.d/transport/30_exim4-config_remote_smtp_smarthost with 50_exim4_config_gmail_pipe.
* Restart Exim. 

Exim will not read environment variables, so this will only look in /etc/envdir/GMAIL_PROXY_URL and /etc/envdir/GMAIL_SECRET_KEY for the relevant settings.

HTTP Interface
==============

You can send messages via the HTTP interface. The server accepts POST on its root URL, and expects two parameters in the body of the POST request. One parameter is 'msg', and is the serialized email message; the other is 'signature', and is the signature of the message, according to the signing algorithm given in Python by:

    base64.encodestring(hmac.new(SECRET_KEY, msg, hashlib.sha1).digest()).strip()

The server will respond with a 204 in case of success, a 400 for client error (missing parameters, or malformed signature) and a 500 otherwise. Clients should be prepared to deal with occasional server-side failures, due to GAE downtime; use of a local queue is recommended.


------------

Known issues
============

* The From address needs to be authorized - that is, it needs to be associated with a Google account, and listed as an administrator of the GAE app. For google mail accounts (or for google Apps For Your Domain accounts). The check seems to be done by strict string comparison, not by using any nicknames or aliases. This means that you *can't* send email from:
  * A google address with a +suffix on the username.
  * An address which is a "nickname" of another address.
  * An address which resolves to a google group.

* Reply-To won't work for people reading the email in gmail: See  http://www.google.com/support/forum/p/gmail/thread?tid=74d00d5e2605242d&hl=en