bbc / newslabs-newsquery Goto Github PK

Create a function to return the list of sources (refereed to as "products" internally) the Triplestore knows about as an array of Names and URI's.

e.g.

[ {
"name" : "Sky News",
"uri" : "http://www.bbc.co.uk/ontologies/bbc/SkyNews"
} ]

NB: This may not currently exactly match what we are actually indexing, but that's something we are looking to resolve.

This would make it easy to plot mentions over time on a graph, and identify trending correlations between one topic and another in the news.

Add unit tests for all methods.

Note: An API key will still need to be provided before tests can be run.

Currently it returns the best lexical match, it should prioritise by date if no date range is specified

Thinking of using a set of mongo collection for caching if caching is set to enabled.

I'm thinking would be sensible default collection names and it would connect to the db "newsquery" on "localhost", but it would be possible to change these as options.

I'm inclined to use mongo for ease of integration with other projects (current and future) but open to alternative suggestions. Also considering a NoSQL equivalent to SQLLite:

http://stackoverflow.com/questions/7233057/lightweight-javascript-db-for-use-in-node-js

Hey Iain!

I was looking at a use-case for newsQuery to display some of the API functionality that we have in the Juicer and make the Juicer demo more awesome. At the moment, it seems that newsQuery is only usable from Node.js. However, it would be pretty cool if this library was generic, in the sense that it would implement the common API scenarios, but leave the specifics of HTTP and caching up to the user.

For the use-case I have in mind, I would like to have just a HTML page with some Javascript, deployed to S3 without the baggage of deploying a backend.

I can imagine it being a sort of parametrised library, allowing the user to provide the HTTP mechanism and caching framework and any other configurable components upon instantiation. That way it could be usable in the backend and frontend as well.

What do you think? Would it be relatively easy/hard? What's the best practise around such parametrisation in the JS/Node community?

hello Iain
Sadly it seems the api isn't available for people outside of bbc? Do you have any information about this or how to obtain an api key (besides waiting paciently)?
Thanks for your insight
Jeremy

Yo Iain,
The following code...

var apiKey = 'myBBCNewsLabsApiKey';
var newsQuery = require('newsquery')(apiKey); 
newsQuery.getConceptOccurrencesOverTime("http://dbpedia.org/resource/Ukraine", "2014-06-24", "2014-06-28")
.then(function(occurrences) {
    console.log(occurrences);
});

Returns this weird result:

[ { date: '2014-06-24', value: 0 },
{ date: '2014-06-25', value: 0 },
{ date: '2014-06-26', value: 0 },
{ date: '2014-06-27', value: 0 } ]

Note that the API key value works fine with BBC News Labs API.
Cheers mate

Some of the tests fail on first run because the timeout period is too short (the result is then cached so always works on the second run).

2 seconds really should be long enough but going to have to bump it to 5-10 seconds to be sure.

As the tests run concurrently this won't make the tests take any longer.

Have had feedback internally the term "Related Concepts" is too vague as a method name, and worse - may imply that the concepts are related from an ontological perspective - and am minded to agree.

Intend to rename getRelatedConcepts() to something like getCoOccurrences() or getCoOccuringConcepts(). The latter is more consistent but is a bit of a mouthful.

NB: After the change will leave in getRelatedConcepts() as a deprecated call for a while.

Feedback welcome!