Currently Wirecloud is accessed using http, but it should be https.

Comet apparently has a strict limit on the number of values, on the other side browsers can only handle a certain maximum number of values.
We need an operator that reduces the number of entries in a list by replacing chunks of data with their average.
Idea: Config setting with maximum number of values. If the data collection exceeds this value, its reduces to about this given size
Update/Remark: Best implemented as additional feature within Comet. This would limit the data volume exchanged between Comet and Wirecloud

For every incoming message there needs to be an entity and an IDAS device

Automatically import the daily Feedback-Now report into the SDPG

If there are no inbound data package of specific period of time an alert should be raised

Install Apache Spark and connect it to the installed version of Hadoop

Setup and enable FIWARE Security as part of the IoT Platform

Select appropriate tooling (eg. Jmeter)
Write scripts for automatic onboarding and removal of sensors
Define testplans

Perform a series of performance tests on different container layouts in order to find the optimal layout for FIWARE components

Install Jupyter with Spark Integration in order to analyse the data stored in the SDPG

Within FIWARE, Comet is used to access historical data. However, Comet only works with MongoDB. Thus, develop a component exposing the same query interface as Comet but using data from Hadoop

Create new docker image
Test new version
Migrate data

Provide a mean to automatically convert the daily Excel report from FeedbackNow into a CSV file that allows for automatic import into the SCDP

Internal, critical services like Orion, Comet, Cygnus, IDAS, ...) should only be accessible via PEP. Thus, make sure that these services are only reachable within nodes on Kubernetes (Cluster-IP)

In order to deploy the FIWARE container network (docker-compose), containers and networks need to re-configered, in order to run a kubernetes infrastructure

yaml files

Define a set of rules for all the data stored in the SDPG (FeedbackNow, Traffic, Air Quality) such that data can stored in an efficient way over a long period of time

The SOAP Interface provided by the traffic department needs to be queried in regular intervals and the the resulting data forwarded to the SCDP.
This could possibly also be done using NodeRED

Cugnus is currently using the Hadoop Namenode's REST Service (WebHDFS), which is apparently way to slow for high loads. This should be refactored, using a "native" Hadoop interface.

Check whether NodeRED is a suitable platform for full-custom code needed throughout this project

Research and evaluate Kubernetes for use as a basic infrastructure to run and manage our docker containers for hadoop and FIWARE

Create a dashboard to visualize feedback now data in the most appropriate way

The NameNode is a Single Point of Failure. In order to provide high availability a standby namenode is setup.
Steps that have to be performed:

• Create new docker images
• Configure Automatic Failover with Zookeeper
• Test HA cluster
• Upgrade the cluster on Kubernetes
• Migrate data

Since the backup-system only supports Mongo, we need a decent way to get this data:

• back into the production system (mongodump, mongorestore)
• into hadoop (e.g. by converting the exported text file right into the Hadoop csv format)

Visualize all Air Quality Data produced by the real-life sensor network

Different deployment layouts shall be developed and performance tested in order to find one with an optimal throughput

In order to prevent accential deletion of entities, devices, ..., bring up a confirm dialog.

Data collected by the proprietary IoT System connected to the Air-Quality sensor network is pushed in real-time to our FIWARE IoT-Platform. This REST-Interface will receive this data, translate its payload into Ultralight and send it to IDAS.

Sensor data , Orion and Cygnus should use a consistent and correct time zone
Affected components: POI Beautifier

Description of the initial Hadoop setup

Do some thorough research to identify tools that could possibly be used for enforcing data policies

Create a suitable visualisation of traffic data

IDAS requires API-Key and device ID to be passed as query parameters. This disables XACML rules. Thus, Node-RED shall be used to rewrite URL-Parameters to Query-Parameters.
PEP will then speak to Node-RED rather than IDAS

Find a way to define dependencies between Kubernetes service, so that they start-up in a defined sequence.

For automatically testing the availability of our NodeRED end-points there should be special request that does not produce any data but returns 200 if everything is OK. This request should be sent by systems like Nagios

Describe possible, performant ways how to access (read/write) HDFS from outside the cluster

A major part of the SDPG will be Hadoop and its relevant Plugins. All required nodes should be rolled out on the local cloud using kubernetes

Setup some sample notebooks demonstrating the use of machine learning on SDPG data

This will create the necessary collections in order to support aggregate queries.

For entity attributes in ORION a datatype can/needs to be specified ("Int", "Float", ...).
However, ORION queries always return strings. So, what are these types good for?
Also check COMET aggregate queries