csaftoiu / slack-email-digest Goto Github PK

Scripts to summarize Slack chat history into an email digest.

License: The Unlicense

Python 100.00%

slack-email-digest's Issues

Group conversations

Instead of lumping all messages in a day together, it would be nice to have break-lines between conversations that are separated by a minimum amount of idle activity (say 15 minutes).

Linked gifs and images

Slack embeds gifs and images automatically in the channel. Will this be the case for digest emails?

Better Message-ID handling

Right now messages ids vary only by date. They don't vary by slack group name or channel. They should, so separate threads are kept separate.

slack.team.info().body['id'] gives the unique team id.

Remaining Pieces

Avatars next to each message
Timestamps
~~Pad table cells~~ they look fine on normal browser, Yahoo! apparently ignores this attribute anyway
Avoid blockquote so yahoo doesn't swallow it
Splitting to avoid 64 kB limit
- Done properly
Auto-emailing

Proper fall-back text-only version

Ideally the sent emails should be multi-part, with text as the first part and html as the second part. The text would be an acceptable rendering of the html.

Not sure if this is worth supporting, though.

Meta info in footer or header

Include meta info like the following in the header of footer of every digest email:

Link to joining Slack conversation for lurkers: https://actualfreedom-slackin.herokuapp.com
Disclaimers? Policy for referencing?

Support markdown in attachments

e.g. in mailclerk's attachments, there is some markdown

Assign same month digests to same email thread

Assign digests from the same month to the single conversation thread so they appear 'grouped' in the Yahoo! Group.

The chief benefit of doing this is that it will lead to less cluttering of Yahoo! Group message archive page, when viewed by-thread basis.

Quote linked messages

For example:

srid: @solvann: example here: https://actualfreedom.slack.com/archives/general/p1470071346001458

In Slack, that embeds the linked message. But in the digest no such thing happens. Would be nice to do it though.

Refactor generation code

Probably should add automated tests before doing the following:

Use templates for all aspects of it
Text-only generation should then be easy by just having (text-only) jinja2 templates with the same names
All pieces should access the one SlackScraper instance and use it to get info, instead of pieces being passed around.
All pieces should operate based on the date being processed, instead of start or end timestamp. Date info should come with the messages being rendered.

Postmark doesn't preserve Message-ID and In-Reply-To headers

If I deliver a message with postmark, the message-id and in-reply-to headers aren't preserved.

I tested sending the same message, but with smtp delivery, and the headers were preserved on the yahoo groups interface - so it isn't yahoo that is changing them. The SMTP method does indeed preserve topics even with different subjects:

We'll need to figure this out in order to be able to send threaded messages with postmark. Until then, SMTP delivery from Heroku should work - at least, I tested it via heroku run bash and it worked, will see if the scheduled task delivers it also.

Store shortened URL cache in redis

This file shortened_url_cache.json will go away frequently due to dyno cycling (every day). Best to store these URLs in redis. There may already be libraries that does this.

SSL error when running the script

Not sure how this is happening:

 ./myenv/bin/python scripts/slack-email-digest.py -v [email protected] [email protected] --token "snip" --delivery=stdout
Getting messages from 2016-08-05 19:51:08-04:00 to 2016-08-06 19:51:08-04:00
Retrieved 68 messages
Getting shortened URL for https://secure.gravatar.com/avatar/7858d1ce739fca5301435fb302794c5e.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2F3654%2Fimg%2Favatars%2Fava_0018-72.png...
Traceback (most recent call last):
  File "/Users/srid/code/slack-email-digest/myenv/lib/python3.5/site-packages/requests-2.10.0-py3.5.egg/requests/packages/urllib3/connectionpool.py", line 578, in urlopen
    chunked=chunked)
  File "/Users/srid/code/slack-email-digest/myenv/lib/python3.5/site-packages/requests-2.10.0-py3.5.egg/requests/packages/urllib3/connectionpool.py", line 351, in _make_request
    self._validate_conn(conn)
  File "/Users/srid/code/slack-email-digest/myenv/lib/python3.5/site-packages/requests-2.10.0-py3.5.egg/requests/packages/urllib3/connectionpool.py", line 814, in _validate_conn
    conn.connect()
  File "/Users/srid/code/slack-email-digest/myenv/lib/python3.5/site-packages/requests-2.10.0-py3.5.egg/requests/packages/urllib3/connection.py", line 289, in connect
    ssl_version=resolved_ssl_version)
  File "/Users/srid/code/slack-email-digest/myenv/lib/python3.5/site-packages/requests-2.10.0-py3.5.egg/requests/packages/urllib3/util/ssl_.py", line 308, in ssl_wrap_socket
    return context.wrap_socket(sock, server_hostname=server_hostname)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/ssl.py", line 377, in wrap_socket
    _context=self)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/ssl.py", line 752, in __init__
    self.do_handshake()
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/ssl.py", line 988, in do_handshake
    self._sslobj.do_handshake()
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/ssl.py", line 633, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:645)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/srid/code/slack-email-digest/myenv/lib/python3.5/site-packages/requests-2.10.0-py3.5.egg/requests/adapters.py", line 403, in send
    timeout=timeout
  File "/Users/srid/code/slack-email-digest/myenv/lib/python3.5/site-packages/requests-2.10.0-py3.5.egg/requests/packages/urllib3/connectionpool.py", line 604, in urlopen
    raise SSLError(e)
requests.packages.urllib3.exceptions.SSLError: [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:645)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "scripts/slack-email-digest.py", line 167, in <module>
    main()
  File "scripts/slack-email-digest.py", line 142, in main
    html_renderer = HTMLRenderer(scraper)
  File "/Users/srid/code/slack-email-digest/slack_email_digest/HTMLRenderer.py", line 119, in __init__
    self.load_avatars()
  File "/Users/srid/code/slack-email-digest/slack_email_digest/HTMLRenderer.py", line 123, in load_avatars
    self.avatars[name] = get_shortened_url(info['profile']['image_72'])
  File "/Users/srid/code/slack-email-digest/slack_email_digest/memoize.py", line 31, in wrapped
    res = f(arg)
  File "/Users/srid/code/slack-email-digest/slack_email_digest/HTMLRenderer.py", line 77, in get_shortened_url
    res = pyshorteners.Shortener('Isgd', timeout=5).short(url)
  File "/Users/srid/code/slack-email-digest/myenv/lib/python3.5/site-packages/pyshorteners-0.6.0-py3.5.egg/pyshorteners/shorteners/__init__.py", line 108, in short
  File "/Users/srid/code/slack-email-digest/myenv/lib/python3.5/site-packages/pyshorteners-0.6.0-py3.5.egg/pyshorteners/shorteners/isgd.py", line 18, in short
  File "/Users/srid/code/slack-email-digest/myenv/lib/python3.5/site-packages/pyshorteners-0.6.0-py3.5.egg/pyshorteners/shorteners/base.py", line 25, in _get
  File "/Users/srid/code/slack-email-digest/myenv/lib/python3.5/site-packages/requests-2.10.0-py3.5.egg/requests/api.py", line 71, in get
    return request('get', url, params=params, **kwargs)
  File "/Users/srid/code/slack-email-digest/myenv/lib/python3.5/site-packages/requests-2.10.0-py3.5.egg/requests/api.py", line 57, in request
    return session.request(method=method, url=url, **kwargs)
  File "/Users/srid/code/slack-email-digest/myenv/lib/python3.5/site-packages/requests-2.10.0-py3.5.egg/requests/sessions.py", line 475, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/srid/code/slack-email-digest/myenv/lib/python3.5/site-packages/requests-2.10.0-py3.5.egg/requests/sessions.py", line 606, in send
    history = [resp for resp in gen] if allow_redirects else []
  File "/Users/srid/code/slack-email-digest/myenv/lib/python3.5/site-packages/requests-2.10.0-py3.5.egg/requests/sessions.py", line 606, in <listcomp>
    history = [resp for resp in gen] if allow_redirects else []
  File "/Users/srid/code/slack-email-digest/myenv/lib/python3.5/site-packages/requests-2.10.0-py3.5.egg/requests/sessions.py", line 179, in resolve_redirects
    **adapter_kwargs
  File "/Users/srid/code/slack-email-digest/myenv/lib/python3.5/site-packages/requests-2.10.0-py3.5.egg/requests/sessions.py", line 585, in send
    r = adapter.send(request, **kwargs)
  File "/Users/srid/code/slack-email-digest/myenv/lib/python3.5/site-packages/requests-2.10.0-py3.5.egg/requests/adapters.py", line 477, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:645)