Git Product home page Git Product logo

twoifbysea's Introduction

twoifbysea

send notifications about events with ease

About

twoifbysea is a server that receives messaging requests from clients. The goals for the project are:

  • Rapid development for clients, abstracting the annoying details of messaging away
  • Supporting a variety of messaging channels

Installation

To install PyPI dependencies, run:

$ pip install -r /path/to/twoifbysea/requirements.txt

Troubleshooting pycrypto installation

If you see an error message such as:

blake2/blake2module.c:1:20: fatal error: Python.h: No such file or directory

 #include <Python.h>

                    ^

compilation terminated.

error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

----------------------------------------
Cleaning up...
Command /usr/bin/python -c "import setuptools,
tokenize;__file__='/tmp/pip-build-Br8maE/blake2/setup.py';exec(compile(getattr(tokenize,
'open', open)(__file__).read().replace('\r\n', '\n'), __file__,
'exec'))" install --record /tmp/pip-EL6ID6-record/install-record.txt
--single-version-externally-managed --compile failed with error code 1
in /tmp/pip-build-Br8maE/blake2

This may indicate that you are lacking important source code files for Python-based development. This can be rectified for some Linux distributions by running:

$ apt-get update && apt-get install python-dev

Usage

Clients must connect to the server using a supported connection mechanism; currently, only HTTP is supported. (Please create a GitHub issue to request additional connectors.) Once started, the server can then service client requests to send notifications. The notification script must be scheduled as a cron job to clear the queue.

Starting the webserver

Currently the only connector available for the server is via HTTP. The webserver that receives and processes notification requests can be started as follows:

$ cd /path/to/twoifbysea/master ; make start

Scheduling the notification daemon

Notification requests by clients are added to a queue and will be sent when the notify.py script is run. To regularly empty the queue, this script should be run as a cron job. The notify.py script takes no arguments and will find the queue database file on its own using the appdir Python module.

Ex.:

Send queued notifications every 5 minutes:

5 * * * * /path/to/twoifbysea/master/twoifbysea/notify.py

Client examples

See the examples/ directory for general and Python-specific examples of connecting to the notification server.

Logs

Check the twoifbysea.log file for error messages, such as those concerning missing environment variables for notification credentials. The location of this log file is determined by the (appdirs)[https://pypi.python.org/pypi/appdirs/1.4.3] Python module:

MacOS: ~/Library/Application Support/twoifbysea/twoifbysea.log

Windows (non-roaming profiles): C:\Documents and Settings<User>\Application Data\Local Settings\atlas\twoifbysea\twoifbysea.log

Windows (roaming profiles): C:\Documents and Settings<User>\Application Data\atlas\twoifbysea\twoifbysea.log

Linux: ~/.local/share/twoifbysea/twoifbysea.log

Project cleaning

To delete the notification database (primarily for development purposes), use:

$ cd /path/to/twoifbysea/master/twoifbysea ; make clean

Supported communication channels

  • Email (unencrypted, GMail only)
  • Telegram bot (unencrypted)

Important TODOs

  • Add support for daily digest messages
  • Add negative notifications (e.g. "no new messages for app-x today, all is good")
  • Allow Python clients to initiate delivery of some or all types of messages without running a listening server
  • Implement generic email accounts apart from GMail (need to figure out some spam filtering stuff)
  • Add new communication channels, such as PGP-encrypted emails, Slack channels, etc.
  • Refactor to minimize client footprint
  • Create packaged installer for server

twoifbysea's People

Watchers

James Cloos avatar Kristov Atlas avatar  avatar

twoifbysea's Issues

Trouble installing pycrypto on shared hosted environment

e.g. https://bugs.launchpad.net/pycrypto/+bug/1294670

checking whether we are cross compiling... configure: error: in `/tmp/pip-build-root/pycrypto':

configure: error: cannot run C compiled programs.

If you meant to cross compile, use `--host'.

See `config.log' for more details

Solution: TMPDIR=~/tmp python -m pip install pycrypto

Seems like some cheap hosts (in my case Site5), has /tmp mounted with noexec. I of course don't have any permissions to change this. [...]
there is of course a C-flag to change the temp dir.

crypto review of secret storage

Would like someone with crypto background to review how I currently store secrets

https://github.com/kristovatlas/twoifbysea/blob/master/twoifbysea/crypt.py

Encryption specifications:
    * os.urandom is used for random data. "This function returns random bytes
      from an OS-specific randomness source. The returned data should be
      unpredictable enough for cryptographic applications, though its exact
      quality depends on the OS implementation. On a UNIX-like system this will
      query /dev/urandom, and on Windows it will use CryptGenRandom(). If a
      randomness source is not found, NotImplementedError will be raised."
    * Message is padded with random prefix and suffix to defend known-plaintext
      attacks, probably unnecessarily, given that a random IV is used.
    * A random IV is used for each encryption operation, to be stored along with
      the ciphertext for decryption.
    * AES-256 is used in CBC mode.

Also, application secrets are stored in the server's database as a blake2 hash:

def get_key_val(self, app_id, app_secret, key):
"""Get plaintext of encrypted val from database
Args:
app_id (str): Base64-encoded string identifying the app for whom we
are storing data
app_secret (str): Base64-encoded string unique to the app that is
used to encrypt and decrypt data
key (str): The plaintext name of the value being stored. Not to be
confused with the encryption key.
The following are stored in the database:
* app_id
* hash of key with hash-key app_secret; keying the hash helps protect
the contents of the key being stored against an attacker that does
not know the app_secret.
* value_iv, the iv used to encrypt the value using AES256-CBC
* encrypted value, encrypted with value_iv using AES256-CBC and padding
Returns: str: The plaintext value decrypted
Raises: DecryptionFailError: If the plaintext value cannot be returned
"""
assert isinstance(app_id, str)
assert isinstance(app_secret, str)
assert isinstance(key, str)
common.b64decode(app_id)
common.b64decode(app_secret)
hashed_key = blake2.blake2(data=key, hashSize=64, key=app_secret)
stmt = 'SELECT value, value_iv FROM {0} WHERE app_id = ? AND key = ?'.format(
TBL_CRYPT_KV_STORE.name)
arglist = (app_id, hashed_key)
try:
row = self.fetch_one_row(stmt, arglist)
except DatabaseReadError, err:
if str(err) == DB_SELECT_RETURNED_NULL_MSG:
raise DecryptionFailError('Failed to decrypt value')
assert len(row) == 2
value_ciphertext = str(row[0])
value_iv = str(row[1])
common.b64decode(value_ciphertext)
common.b64decode(value_iv)
value = decrypt(ciphertext=value_ciphertext, key=app_secret, iv=value_iv)
#This sanity check is probably not required, but we add a magic prefix
#value to ensure that the plaintext proudced from decryption looks good.
if value[0:len(ENCRYPTION_MAGIC_NUMBER)] != ENCRYPTION_MAGIC_NUMBER:
raise DecryptionFailError('Failed to decrypt value')
return value[len(ENCRYPTION_MAGIC_NUMBER):]

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.