foxmask / django-th Goto Github PK
View Code? Open in Web Editor NEW:snake: Trigger Happy - The bus :bus: for your internet services
Home Page: https://foxmask.org/tag/triggerhappy.html
License: BSD 3-Clause "New" or "Revised" License
:snake: Trigger Happy - The bus :bus: for your internet services
Home Page: https://foxmask.org/tag/triggerhappy.html
License: BSD 3-Clause "New" or "Revised" License
check why there is no split on the comma between tag string, to avoid to meet :
EDAMUserException(errorCode=2, parameter='Tag.name')
which means there is a comma in the tag name.
something like
python add_service.py foobar
will have to add :
'django_th.services.my_foobar.ServiceFoobar',
TH_FOOBAR = (
'consumer_key': 'abcdefghijklmnopqrstuvwxyz',
'consumer_secret': 'abcdefghijklmnopqrstuvwxyz',
)
and probably something with the TH_WIZARD_TPL tuple too.
to filter by anything we want, for example by target service, by provider service, by triggered date etc -
Have to see from here http://django-haystack.readthedocs.org
When 2 services are choosen, create an action.
eg.
next dev will be from http://www.foxmask.info/post/2013/07/13/sondage-quels-services-utilisez-vous-le-plus/
when packaging to pypi html are not provided
when creating a note to evernote the sanitize function which call pytidy ; return a valid XML but drop ALL the UTF-8/non-ascii chars ...
Need to make some test with LXML
this will permit to publish data on a dedicated server/channel with a given username and a key if needed
lib to try https://pypi.python.org/pypi/irc
the trigger list displays the last triggered date but is h+2
use this http://pythonhosted.org/feedparser/bozo.html to check if the feeds are ok then displays a warn to the user if the feeds is not well formed
We could keep the log of the last event that has been triggered and stored to a service
For example this :
"From ProviderServiceA: string of the event - To ConsumerServiceB"
Could be view from a new Provider/Consumer page that will list the last triggered data
check if it's still mandatory otherwise add it in settings.py
Actually, when a token is expired, fire.py exit with an exception.
We have to check the token in a try/catch to avoid to exit.
Also we need to show on the service page, that the service is disable because of the expired token
add a paginator to the list of Triggers
add the source of the data in the body like :
"from foobar.com" foloowing by the link of the page
when grabbing source data, check if a title is provided to avoid something like this:
Traceback (most recent call last):
File "./fire.py", line 123, in <module>
main()
File "./fire.py", line 119, in main
go()
File "./fire.py", line 72, in go
service.consummer.token, title, content, service.id, extra)
File ".../django_th/services/my_evernote.py", line 120, in save_data
created_note = note_store.createNote(note)
File ".../local/lib/python2.7/site-packages/evernote/api/client.py", line 138, in delegate_method
)(**dict(zip(arg_names, args)))
File "..../local/lib/python2.7/site-packages/evernote/edam/notestore/NoteStore.py", line 4793, in createNote
return self.recv_createNote()
File "..../local/lib/python2.7/site-packages/evernote/edam/notestore/NoteStore.py", line 4817, in recv_createNote
raise result.userException
evernote.edam.error.ttypes.EDAMUserException: EDAMUserException(errorCode=2, parameter='Note.title')
if the title is empty, do not handling this data and continue to the next record
to be done on the admin panel
dont stuck the user with the tag which should not be mandatory.
on the list of triggers
if < 2 dont permit to create a trigger at all as 2 services at least are needed
thus this will avoid an error 500 and an Exception with UserService model.
add 2 links to be able to disable / enable all the triggers in a row
this should permit to change the URL of the provider
With multiprocessing this gives those error https://gist.github.com/foxmask/9113543
The script works in single process or with multiprocess but with SQLite.
Same for MySQL 5.5 - multiprocessing does not work.
I put here a peace of code as a starting point :
cms_app.py to put in at the same level of models folder
# -*- coding: utf-8 -*-
from django.utils.translation import ugettext_lazy as _
from cms.app_base import CMSApp
from cms.apphook_pool import apphook_pool
class TriggerhappyApp(CMSApp):
name = _("Trigger Happy") # give your app a name, this is required
urls = ["django_th.urls"] # link your app to url configuration(s)
app_name = "django_th" # this is the application namespace
apphook_pool.register(TriggerhappyApp) # register your app
cms_plugins to put at the same level of models folder:
# -*- coding: utf-8 -*-
from cms.plugin_base import CMSPluginBase
from cms.plugin_pool import plugin_pool
from django_th.models import TriggerHappyPlugin
from django.utils.translation import ugettext as _
class CMSTriggerHappyPlugin(CMSPluginBase):
model = TriggerHappyPlugin # Model where data about this plugin is saved
name = _("Trigger Happy Plugin") # Name of the plugin
# template to render the plugin with
render_template = "cms_plugin.html"
def render(self, context, instance, placeholder):
context.update({'instance': instance})
return context
# register the plugin
plugin_pool.register_plugin(CMSTriggerHappyPlugin)
Template to create:
{% extends "base.html" %}
{% load static %}
{% load url from future %}
{% load i18n %}
{% load django_th_extras %}
{% block title %}{% if user.is_authenticated %}{% trans "My Triggers" %} - {{ user.username }} - {{ current_site.name }}{% else %}{% trans "Home" %} - {{ current_site.name }}{% endif %}{% endblock %}
{% block content %}
<!-- Fixed navbar -->
<div class="navbar navbar-default navbar-fixed-top" role="navigation">
<div class="container">
<div class="navbar-header">
<button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-collapse">
<span class="sr-only">Toggle navigation</span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="{% url 'base' %}" title="{% trans "Home" %}">Trigger Happy</a>
</div>
<div class="navbar-collapse collapse">
{% if user.is_authenticated %}
<ul class="nav navbar-nav">
<li><a href="{% url 'base' %}" title="{% trans "Home" %}"><span class="glyphicon glyphicon-home"></span></a></li>
<li>{% if nb_services > 1 %}<a href="{% url 'create_service' %}" title="{% trans "Create a new trigger" %}">{% else %}<a href="#" title="You can't create a new trigger until 2 services are activated">{% endif %}<span class="glyphicon glyphicon-plus"></span></a></li>
<li><a href="{% url 'user_services' %}" title="{% trans "List of your own activated services" %}"><span class="glyphicon glyphicon-tasks"></span></a></li>
{% block filter_trigger %}
{% include "filter.html" with trigger_filter_by=trigger_filter_by %}
{% endblock %}
{% url 'profiles_profile_detail' request.user.username as profiles_profile_detail %}
<li><a href="{% url 'logout' %}" title="{% trans "log out" %}"><span class="glyphicon glyphicon-off"></span></a></li>
</ul>
{% endif %}
</div><!--/.nav-collapse -->
</div>
</div>
{% if user.is_authenticated %}
<div class="col-md-12">
<div class="col-md-5">
<h2>{% trans "My Triggers" %}</h2>
</div>
<div class="col-md-6 trigger-summary">
{% trans "Summary" %} :
<span class="label label-info">{{ nb_services }}</span> {% trans "activated" %} <span class="label label-default">{% trans "services" %}</span>
<span class="label label-info">{{ nb_triggers.enabled }}</span> {% trans "enabled" %} <span class="label label-default">{% trans "triggers" %}</span>
<span class="label label-warning">{{ nb_triggers.disabled }}</span> {% trans "disabled" %} <span class="label label-default">{% trans "triggers" %}</span>
<hr/>
{% trans "Actions" %} : <a href="{% url 'trigger_switch_all_to' 'off' %}" title="{% trans "this action will set all the triggers off"%}">{% trans "Mark all Off" %}</a> - <a href="{% url 'trigger_switch_all_to' 'on' %}" title="{% trans "this action will set all the triggers on"%}">{% trans "Mark all On" %}</a>
</div>
</div>
{% if triggers_list %}
{% if is_paginated %}
<div class="col-md-12 pagination">
<ul>
{% if page_obj.has_previous %}
<li><a href="{% url 'home' %}?page={{ page_obj.previous_page_number }}">{% trans "previous" %}</a></li>
{% endif %}
<li class="active"><a >
{% blocktrans with page_number=page_obj.number total_of_pages=page_obj.paginator.num_pages %}
Page {{ page_number }} of {{ total_of_pages }}
{% endblocktrans %}</a>
</li>
{% if page_obj.has_next %}
<li><a href="{% url 'home' %}?page={{ page_obj.next_page_number }}">{% trans "next" %}</a></li>
{% endif %}
</ul>
</div>
{% endif %}
{% for trigger in triggers_list %}
<article>
<div class="trigger-record {% if trigger.status %}trigger-enable{% else %}trigger-disable{% endif %} col-md-12">
<div class="col-md-7 trigger-text">
<a href="{% url 'edit_provider' trigger.id %}" title="{% trans 'Edit' %} {{ trigger.provider.name|service_readable|lower }}" >{{ trigger.provider.name|service_readable }}</a> <span class="glyphicon glyphicon-chevron-right"></span> <a href="{% url 'edit_consumer' trigger.id %}" title="{% trans 'Edit' %} {{ trigger.consumer.name|service_readable|lower }}" >{{ trigger.consumer.name|service_readable }}</a>
<h3><a href="{% url 'edit_trigger' trigger.id %}" title="{% trans 'Edit the description' %} ">{{ trigger.description|safe|escape }}</a></h3>
</div>
<div class="col-md-5">
{% if trigger.status %}
<a class="btn btn-lg btn-info" role="button" href="{% url 'edit_provider' trigger.id %}" title="{% trans 'Edit your service' %} {{ trigger.provider.name|service_readable|lower }}" ><span class="glyphicon glyphicon-pencil icon-white"></span> {{ trigger.provider.name|service_readable|lower }}</a>
<a class="btn btn-lg btn-info" role="button" href="{% url 'edit_consumer' trigger.id %}" title="{% trans 'Edit your service' %} {{ trigger.consumer.name|service_readable|lower }}" ><span class="glyphicon glyphicon-pencil icon-white"></span> {{ trigger.consumer.name|service_readable|lower }}</a>
<a class="btn btn-primary btn-lg" role="button" href="{% url 'trigger_on_off' trigger.id %}" title="{% trans 'Set this trigger off' %}"><span class="glyphicon glyphicon-off icon-white"></span></a>
{% else %}
<a class="btn btn-lg btn-info disabled" role="button" href="#" title="{% trans 'service disabled' %} {{ trigger.provider.name|service_readable|lower }}" ><span class="glyphicon glyphicon-pencil icon-white"></span> {{ trigger.provider.name|service_readable|lower }}</a>
<a class="btn btn-lg btn-info disabled" role="button" href="#" title="{% trans 'service disabled' %} {{ trigger.consumer.name|service_readable|lower }}" ><span class="glyphicon glyphicon-pencil icon-white"></span> {{ trigger.consumer.name|service_readable|lower }}</a>
<a class="btn btn-success btn-lg" role="button" href="{% url 'trigger_on_off' trigger.id %}" title="{% trans 'Set this trigger on' %}"><span class="glyphicon glyphicon-off icon-white"></span></a>
{% endif %}
<a class="btn btn-lg btn-danger" role="button" href="{% url 'delete_trigger' trigger.id %}" title="{% trans "Delete this trigger ?" %}"><span class="glyphicon glyphicon-trash icon-white"></span></a><br/>
</div>
<footer>
<div class="col-md-12">
<p><span class="glyphicon glyphicon-calendar"></span>{% trans "Created" %} {{ trigger.date_created }} -
<span class="glyphicon glyphicon-calendar"></span>{% trans "Triggered" %} {% if trigger.date_triggered %}{{ trigger.date_triggered }}{% else %} {% trans "Never triggered" %}{% endif %}</p>
</div>
</footer>
</div>
</article>
{% endfor %}
{% if is_paginated %}
<div class="col-md-12 pagination">
<ul>
{% if page_obj.has_previous %}
<li><a href="{% url 'home' %}?page={{ page_obj.previous_page_number }}">{% trans "previous" %}</a></li>
{% endif %}
<li class="active"><a >
{% blocktrans with page_number=page_obj.number total_of_pages=page_obj.paginator.num_pages %}
Page {{ page_number }} of {{ total_of_pages }}
{% endblocktrans %}</a>
</li>
{% if page_obj.has_next %}
<li><a href="{% url 'home' %}?page={{ page_obj.next_page_number }}">{% trans "next" %}</a></li>
{% endif %}
</ul>
</div>
{% endif %}
{% else %}
<div class="trigger-record col-md-12">
<div class="alert alert-info">
<button type="button" class="close" data-dismiss="alert">×</button>
{% trans 'No trigger yet' %}
</div>
{% if nb_services < 2 %}
<div class="alert alert-danger">{% trans "Before creating a trigger" %} <span class="label label-danger">{% trans "you have to activate" %}</span> <a href="{% url 'add_service' %}">{% trans "at least 2 services" %}</a>. </div>
{% else %}
<p>{% trans "You can now proceed by creating" %}<a href="{% url 'create_service' %}"> {% trans "your first trigger" %}</a></p>
{% endif %}
</div>
{% endif %}
{% else %}
<div class="jumbotron">
<p>
<strong>Trigger Happy</strong> is up !
</p>
<p>Have Fun</p>
<a class="btn btn-primary" href="{% url 'login' %}">{% trans "Log in" %}</a>
</div>
{% endif %}
{% endblock %}
use https://pypi.python.org/pypi/twitter
this will need to make a dedicated module django-th-twitter
this should be used to :
the drop down should not contains "Service" eg ServiceRSS ServiceEvernote etc.
send an email to the Armin when new user registers. setup this feature from the admin
Add the feature that permits to load a OPML file like http://planet.python.org/opml.xml and create one trigger for each.
the wizard will be :
on top of the main page ; display the total of triggers created and number of activated one
useful when we start to have more than one page to turn ;)
if, from the external service, we drop the authorization we grant to Trigger Happy, then Trigger Happy fails to handle the job.
We Need to add a button to renew the token.
start it here https://github.com/foxmask/django-th-twitter
Currently just 2 services exist so there is one wizard between each of them but this can't be scalable.
So I need to build a dynamic wizard system that will be able to link the services between each other
when fire.py run, we should add the user at first then the service provider ; consummer ; number of new data ; then the description of the trigger
smth like :
[user] [provider] [consummer] [num_of_new_datas] [description]
thus :
we'll be able to track the stability of the service for each user.
When a provider does not have a date of creation (or something like that), trigger-happy creates the same data to the consummer part, at each launch of fire.py ...
eg : RSS feeds does that.
But other provider does not have a date and are able to get fresh datas between two launches.
eg : pocket does that
So several possibilities :
When a service raises an exception, return False then:
so
consumer(
service.consumer.token, service.id, **data)
become
status = consumer(
service.consumer.token, service.id, **data)
if status if false
also initialse
status = False
after
to_update = False
and
if to_update:
logger.info("user: {} - provider: {} - consumer: {} - {} = {} new data".format(
service.user, service.provider.name.name, service.consumer.name.name, service.description, count_new_data))
update_trigger(service)
else:
logger.info("user: {} - provider: {} - consumer: {} - {} nothing new".format(
service.user, service.provider.name.name, service.consumer.name.name, service.description))
will become
if to_update:
logger.info("user: {} - provider: {} - consumer: {} - {} = {} new data".format(
service.user, service.provider.name.name, service.consumer.name.name, service.description, count_new_data))
update_trigger(service)
else:
if status:
logger.info("user: {} - provider: {} - consumer: {} - {} nothing new".format(
service.user, service.provider.name.name, service.consumer.name.name, service.description))
else:
logger.info("user: {} - provider: {} - consumer: {} - {} AN ERROR OCCURS ".format(
service.user, service.provider.name.name, service.consumer.name.name, service.description))
For each service, display :
this will help the user to know what is the goal of the service he can activate
Now that the Wizard is working great, I have to drop the FBV I did to handle RSS+Evernote and make a generic (FC)BV that will generate a generic form from the 2 models we plug each other during the FormWizard.
currently when we enable / disable a trigger on page 2,3,4 etc... we go back to page 1.
Manage the action with ajax then we'll stay of the page we are.
From the service provider calls line 41, we have to get data from "content", thus line 46 we could read data in the for loop like this :
content = data.content.value or data.content
instead of the actual
content = data.content[0].value
which is very "RSS" oriented ;)
This will permit to integrate many other services (provider) more easyly.
ATOM feeds do not always have updated/issued node thus fire.py failed and crashed
File "./fire.py", line 63, in go
to_datetime(data.published_parsed) >= service.date_triggered:
File "../lib/python2.7/site-packages/feedparser.py", line 417, in __getattr__
We need to check if published_parsed exists before using it otherwise try to find another date from : http://pythonhosted.org/feedparser/date-parsing.html
split __init__.py
from models and forms in a base.py module
thus this will avoid to load the content on each __init__.py
with things that are not expected.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.