smart-facility / cognicity-reports Goto Github PK

If we hit the tweeting rate limit, handle this and back off sending so that we don't get our sending account blocked or keep increasing the rate limited period.

refer smart-facility/cognicity-reports-powertrack#27 (comment)

Allow data sources to perform some startup verification actions which can fail startup. E.g., the powertrack data source used to check the outgoing connection to twitter and check the configured tweet message lengths were acceptable. If either of these fails, an error when starting is more useful than an error later on once running.

See fix for smart-facility/cognicity-server#34

The README.md needs more info about what the project is about and how it inter-relates, including hyperlinks to and discussion of submodules.

The connection to Twitter should only reside in either the Twitter or the PowerTrack module. Reasoning:

• this error reporting structure not used in practice, with monitoring of AWS systems & logs etc.
• when using PM2 if an error is reported via Twitter- numerous tweets are sent (PM2 restarts execution loop)
• not with fit of new separated architecture as discussed (i.e. future work may see only Detik reports used, for example, in which case user may not have a Twitter account).

As the parent module for the Detik submodule, this repo is also affected by this bug:
smart-facility/cognicity-reports-detik#2

This error on Travis:

$ git submodule update
Cloning into 'detik'...
Warning: Permanently added the RSA host key for IP address '192.30.252.128' to the list of known hosts.
Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights and the repository exists.
Clone of '[email protected]:smart-facility/cognicity-reports-detik' into submodule path 'detik' failed

I guess we need to use a 'deploy key', see http://docs.travis-ci.com/user/private-dependencies/#Using-an-existing-key , to allow Travis to checkout our SSH URL submodules.

Could we use HTTPS URLs for the submodules perhaps?

If the DB throws an error (which we see on Amazon environment switch IIRC), our error handler which caches incoming data and tries to reconnect to the database fails under pg 3.0.0 (at least on Windows - comments indicate this may not be universal). It worked on 2.0.0.

This describes the issue:
brianc/node-postgres#746

And this is the fix:
brianc/node-postgres#843
which was in >= 4.4.2

What was the reason for the initial PG upgrade? It would be nice to get this reconnection behaviour working again. Testing if it fails on the deployment environment would be interesting. But upgrading the pg dependency may be simplest.

• Need more documentation on how to activate submodules including how to specify which config file they use.
• PM2 needs to be looked at as well to make sure it is consistent with the new startup process. I think it is.
• If submodules cannot be started up in standalone mode separate from the submodule then raise separate issues there to remove their PM2 processes.json and relevant text from README.md