smart-facility / cognicity-server Goto Github PK

The test of the server code (see cb5f335) relies on config.pg.conString being invalid, in order to (as the commit comment says) test all the code up until that point. This is why the test works on travis-ci and jenkins as there's no database present. If, however, it is valid (i.e. database has been set up and config.pg.conString , then the server starts up and keeps running, so the timeout in the test is hit and the test fails (#14). Annoyingly, this then made it hard to reproduce the issue, as, unless you kill that node process, then the second time you test the server will fail because it couldn't open the TCP port, and so the test passes in a different way. To get the desired behaviour for the test we should set config.pg.conString=null in test-config.js

Keep the time when the status is changed and serve this status in the response to the 'flooded' query with each region

.

Zooming out on the map leaves a blank page in firefox, chrome and safari (all tested on desktop only so far) despite successful 200 requests for tiles.

On realising that I'd put robots.txt in the wrong location, I went to fix this in petajakarta-web 2d021f4 but realised (of course) it's not that simple: cognicity-server needs to be modified to serve this static file under /
@talltom, is there a clean way of doing this?

https://travis-ci.org/smart-facility/cognicity-server
@talltom don't forget to fix master and then also merge into 2.0.x if required

Description

This is with node v4.1.2 (latest available with homebrew brew update && brew install node) but I suspect also common to newer io.js-derived node.js versions across platforms that care about properties being read only. I suspect strict mode is enable now by default and that is throwing the error because of the way those variables are defined.

Output of npm test

52 passing (883ms)
1 failing

1. getHistoricalCountByArea validation should call the database if parameters are valid:
   TypeError: Cannot assign to read only property 'start_time' of ale
   at chainQueries (CognicityServer.js:433:21)
   at Object.server.getCountByArea (test/testCognicityServer.js:631:4)
   at Object.CognicityServer.getHistoricalCountByArea (CognicityServer.js:452:8)
   at Context. (test/testCognicityServer.js:642:10)

Update confirmed reports SQL query as per the fork at https://github.com/matthewberryman/cognicity-server

Output from /data/API/v2/infrastructure/floodgauges gives timestamps with timezone set by PostgreSQL database server option, and not in local time (WIB / UTC+7).

E.g. {"measuredatetime":"2015-12-03 15:00:00+11","depth":115,"warninglevel":4,"warningnameid":"SIAGA IV "}

To be consistent with other API endpoints timestamps should be forced into local time, e.g.:
{"measuredatetime":"2015-12-03 11:00:00","depth":115,"warninglevel":4,"warningnameid":"SIAGA IV "}

The default log path does not exist on all systems (see #14). Some options here:

1. Improve clarity of the README.md documentation around the need to change it.
2. Test if the directory exists, either
   a. using this package - I am not concerned about the race condition as we're talking about a system directory that's either there or not; Or
   b. alternatively we could detect when winston fails to open the log file and open it in the current working directory.
3. Change the default to be the current working directory and then modify .ebextensions to make sure it can be bundled off to an s3 bucket on AWS as per the docs for that.

I propose 3 and then still clarify (1).

v1 API calls need to be redirected to v2. @talltom has proposed using 303 in 2b0bc36 however according to the rfc either 302, a temporary redirect, or 301, a permanent redirect, should be used. This change for us should be done using a 301 as we are permanently (as best as we can foresee) deprecating v1 in favour of v2 and v2 handles v1 requests. A vanilla 301 would be cached indefinitely (in practice, until a browser eventually runs out of cache space). If we ever do change our minds, or in case of handover of the domain, etc. I think it's better to put a timeout on that. Initially one hour (just in case, even after testing, we've broken something) and then going up to a month.

JS & SQL code in these two functions is now near equivalent, potentially could be collapsed into one method which optional parameters as required.

-> This is important because currently getReport() does not return all parameters for a true representation of a report, potentially breaking client side processing of report objects if they expect complete object as returned by getReports().

For tweets confirmed by a BPBD retweet, we should consider, due to the time taken from tweet -> retweet, resetting the time window to start from the time of the BPBD retweet; we know at that point of time that flooding is ongoing. I've put this in cognicity-server but maybe we want to change the cognicity-reports-powertrack code to assist with this?

• There's a mix of dynamic and static content under /banjir/language_code/data/api/, which means that any robots file will have to specify subfolders of the api folder and be upgraded if they ever change / are added to. Also means that nginx rules for static content paths (to speed up serving and reduce CPU load) would be unnecessarily difficult.
• Having api (for actual API calls, not the static content currently at /banjir/language_code/data/api/ ) as a top level path instead of sitting under /banjir also helps with nginx configuration to speed up static content and reduce separate code.
• The API is duplicated under each language code. Having it separate to /banjir/ would help with that too. Having the site return pages based on browser request instead of having them as separate paths is not good either. This aspect is is covered in part by petajakarta-web issue #46, but you'd also need to make necessary changes to the server code, I guess.

Running with default config

config.logger.logDirectory = '/var/log/nodejs/';

As this DIR isn't available on my local machine the application fails to start but does this silently with no console output and no logfile output.

Time zone output of time stamp data in the API is hard-coded, should be an optional parameter within the configuration file.

We need reset the default time window in server.js at places identified with CHANGEME before release. Currently set to 6 hours for testing but we probably want something shorter for production. Tom had it as 1 hour in 928810e but that may be a little too short during low-medium level flooding.

The API should use sensor_data.distance with sensor_metadata.height_above_riverbed to calculate water depth.

getReport() for individual reports, doesn't return the full compliment of metadata required by the map to categorise reports, display images etc.

• related: #27

CognicityServer.js#L126 introduced in 649badd throws the following errors in conjunction with cognicity-schema/schema.sql in 2.0.x as applied to a fresh database using fresh-cognicity-database.sh:
2015-08-24T00:37:49.676Z - error: dataQuery: Database query failed, column "source" does not exist, queryObject={"text":"SELECT 'FeatureCollection' As type, array_to_json(array_agg(f)) As features FROM (SELECT 'Feature' As type, ST_AsGeoJSON(lg.the_geom)::json As geometry, row_to_json( (SELECT l FROM (SELECT pkey, created_at at time zone 'ICT' created_at, source, url, image_url, title, text) As l) ) As properties FROM tweet_reports As lg WHERE created_at >= to_timestamp($1) AND created_at <= to_timestamp($2) ORDER BY created_at DESC LIMIT $3 ) As f ;","values":["-1","1440376669",null]}
2015-08-24T00:37:49.677Z - error: Express error: undefined, Database query error, Error: Database query error
at null.callback (/home/matthew/cognicity-server/CognicityServer.js:78:16)
at Query.handleError (/home/matthew/cognicity-server/node_modules/pg/lib/query.js:99:17)
at null. (/home/matthew/cognicity-server/node_modules/pg/lib/client.js:176:26)
at emitOne (events.js:77:13)
at emit (events.js:169:7)
at Socket. (/home/matthew/cognicity-server/node_modules/pg/lib/connection.js:109:12)
at emitOne (events.js:77:13)
at Socket.emit (events.js:169:7)
at readableAddChunk (_stream_readable.js:146:16)
at Socket.Readable.push (_stream_readable.js:110:10)

I am guessing you probably want to update config.js#L87 to refer to the all_reports table or otherwise update tweet_reports (although why have a source column for that). Something else to consider is whether there are any unused tables with all the other new ones introduced.

Suspect config values for default language redirection do not work

Hello,
I'm trying to install the cognicity-server on ubuntu 14.04.
I've changed the configuration file in order to connect to localhost database.
When I run npm test from inside the cognicity-server directory I've 1 failing with this error.
52 passing (10s)
1 failing

1. server.js should launch with default configuration and fail with an error:
   Error: timeout of 10000ms exceeded. Ensure the done() callback is being called in this test.

npm ERR! [email protected] mocha: mocha test
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] mocha script.
npm ERR! This is most likely a problem with the cognicity-server package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR! mocha test
npm ERR! You can get their info via:
npm ERR! npm owner ls cognicity-server
npm ERR! There is likely additional logging output above.
npm ERR! System Linux 3.13.0-32-generic
npm ERR! command "/usr/bin/node" "/usr/bin/npm" "run" "mocha"
npm ERR! cwd /home/simone/cognicity-server
npm ERR! node -v v0.10.40
npm ERR! npm -v 1.4.28
npm ERR! code ELIFECYCLE
npm ERR!
npm ERR! Additional logging details can be found in:
npm ERR! /home/simone/cognicity-server/npm-debug.log
npm ERR! not ok code 0
npm ERR! Test failed. See above for more details.
npm ERR! not ok code 0

But I can reach the web page connectin to the 8081 port

If I run the node server.js config.js no node process starts at all