mathrb / mycollect Goto Github PK

Restarting collector seems to break sending emails.

The configuration had changed, the email was sent at a different time

Example:

1. Get ready for the next stage of warfare with this free strategy MMO! https://t.co/jy8NLEY8G3 https://t.co/81ByR4po42
   https://gog-news.kingsgroupgames.com/api/cms/p?id=OTUxODEwMQ==

(1) Get ready for the next stage of warfare with this free strategy MMO! https://t.co/wOmiH4QXOP https://t.co/IXF1eMnbhZ
https://gog-news.kingsgroupgames.com/api/cms/p?id=OTUxOTIyOA==

Should be able to run at different times. Defined by crons.
They should store their results somewhere that output can collect and send.
Outputs should register what has been sent, so they don't send multiple times the same thing

After a period of time (approximately a day and a couple of hours) the twitter collect stops without any information.

Should organize categories in the email alphabetically.

python async
(15) Lambas! OOP! Async/await! @hockeygeekgirl and @GeekTrainer bring you MORE #Python for Beginners (the intermediate… https://t.co/41TZ9wd8OS
https://twitter.com/i/web/status/1256683132230696960

The example above target a Twitter status where we can find the real link.
Need to fix this

This aggregator will need to use clusterization or topic modeling to group items.
Once it is done, we need to find a good textual representation for each topic. And also score the topics in order to be displayed properly.

Should not contain your categories, setup some basic ones