dojot / device-manager Goto Github PK

View Code? Open in Web Editor NEW

0.0 2.0 1.0 712 KB

Device management service for dojot

Home Page: http://www.dojot.com.br

License: Apache License 2.0

Python 98.17% Shell 1.16% Mako 0.21% Dockerfile 0.46%

dojot iot core

device-manager's Introduction

dojot

dojot was conceived with the goal of developing and demonstrating technologies for smart cities. Initially focusing on the pillars of public safety, urban mobility and health. It intends to build a multidisciplinary ecosystem in these areas.

dojot is a Brazilian IoT Platform that came up with an open source proposal in order to facilitate the development of solutions and the IoT ecosystem with local content focused on Brazilian needs.

The platform is one of the outcomes of the project "Plataforma Aberta para IoT e suas Aplicações", which is supported by FUNTTEL via Finep, and led by CPqD, its main executor, in partnership with other science and technology institutions: Instituto Atlântico, Centro de Tecnologia da Informação Renato Archer - CTI, and Fundação de Apoio à Capacitação em Tecnologia da Informação - FACTI.

dojot Manifesto

dojot is a derivation of dojo (or dôjo), a Japanese word that, basically, means "the place where Japanese martial arts are trained".

Dojo is not just a simple training area, it is also a place to be respected as the practitioners' home. Therefore, culturally, it is very common to see the practitioner bowing before entering this area, just as they do before entering their homes.

Linking this concept to the digital world and applying it to software development, dojo represents the place of a developer meeting, seeking for good practices and absorbing the vision of software development as an art.

By combining this concept with the Internet of Things, we pursue IoT software development best practices, bringing together multidisciplinary experts to accelerate solution delivery and reduce the risks inherent in innovation.

dojot = dojô, IoT, and connectivity.

Repository Overview

When the project started, it was decided to create a repository for each service that made up the dojot IoT Platform. However, over time this approach has been shown to have a high maintenance cost. Therefore, all new services are being created in this monorepository (https://github.com/dojot/dojot) and our goal is to migrate the other services to this one in the future.

Getting Help

Contributing

We welcome contributions, but there are just a few small guidelines you need to follow.

Bug Reports

Before open a bug report, please search the issues list to see if it has already been raised.

If the bug is a new one, go ahead and open an issue at https://github.com/dojot/dojot/issues.

If you find a bug and know how to fix it, please don't hesitate to contribute, raise a pull-request with the bug fix!

Feature Requests

If you are demanding a specific feature, you may open an issue at https://github.com/dojot/dojot/issues describing it. Our team will analyse your needs, and it might be added to our Roadmap.

You may also develop this feature and raise a pull-request, but before doing this, please open an issue to discuss with the dojot team the best solution. Without a previous discussion, your pull-request might be rejected.

Pull-Requests

To submit a pull-request, you must follow some steps:

Fork the github repository at https://github.com/dojot/dojot if you haven't already.
Clone your fork, create a new branch, push changes to the branch.
Add documentation and unit tests as part of the change if it makes sense.
Check whether your change doesn't break any existing unit tests.
Open a pull request against the development branch.

After raising a pull-request, it will be reviewed by dojot developers and some changes might be requested before merging it.

Copyright and license

The dojot IoT Platform is based on well-known open source software such as Apache Kafka, RabbitMQ, PostgreSQL, MongoDB, Redis, Kong Gateway and VerneMQ, which have their own licenses.

The services developed by the dojot team to integrate these open-source software components and implement the business logics are copyrighted by CPQD <www.cpqd.com.br> and till release v0.5.1 were licensed under the GPLv3. From release v0.5.2 onwards, these services are being licensed under the Apache License, Version 2.0.

You can use and/or modify the software for free (no license costs apply).

device-manager's People

Contributors

Watchers

Forkers

elgohr-update

device-manager's Issues

Search for templates without associated devices

I'm submitting a ...
- bug report
- feature request
- support request
Do you want to request a feature or report a bug?

Feature

What is the current behavior?

The current behavior is that there is no possibility to look for templates filtering by the ones that doesn't have associated devices.

What is the expected behavior?

With the addition of some parameter, for example, ?has_devices=false, return only templates that do not have associated devices.

What is the motivation / use case for changing the behavior?

If there are lots of templates created and we want to do a cleanup removing all the unused templates, today we would have to try to remove each of the templates and handle the error messages returned by the ones with devices.

Template provided values for devices, and devices' own configuration values

As reported by @rascaraficci , currently there is an understanding/implementation gap related to attribute values on templates and devices in dojot.

The proposed behavior of the system is summarized as the following:

Templates provide the attribute (i.e. field) modeling for devices
Templates may provide global default values for all fields it models
A device (i.e. an instantiation of a set of templates) may override the values of any template-modeled field
If a device possesses a device-specific value override, changes to the template-assigned values must not affect that device's configuration.

This behavior, as outlined above, is currently not working as of tag 0.2.0-20180118

Setting static attribute values when creating a new device is painful

I'm submitting a ...
- bug report
- feature request
- support request
Do you want to request a feature or report a bug?
Feature
What is the current behavior?
DeviceManager needs the attribute ID from a template when setting a static attribute from a new device. This absolutely makes no sense - there is already a verification to check whether there is no conflict between template attribute names, so what's the point of having its attribute ID as well? This is certain to create problems and confusion when creating devices via API.
If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem
What is the expected behavior?
What is the motivation / use case for changing the behavior?
Please tell us about your environment:
- Version: 0.2.x
- Environment: [kubernetes, docker-compose, standalone]
- Operating system: [Ubuntu 16.04, Fedora 28, Ubuntu on VirtualBox (Windows 10), etc.]
Other information (e.g. detailed explanation, stacktraces, related issues, suggestions how to fix, links for us to have context, eg. stackoverflow, gitter, etc)

dojot GUI and DeviceManager have different naming rules for devices

GUI is more strict when checking device names - DeviceManager allows more stuff, but there is no way to tell (from a third-party developer standpoint) which one is right.

Device update is not working properly

Issuing a PUT /device/:id with updated device information is not updating device data, returning an error.

This has been reported by @cfrancisco

Add the template.update message to documentation

[Security] Workflow main.yml is using vulnerable action actions/checkout

The workflow main.yml is referencing action actions/checkout using references v1. However this reference is missing the commit a6747255bd19d7a757dbdda8c654a9f84db19839 which may contain fix to the some vulnerability.
The vulnerability fix that is missing by actions version could be related to:
(1) CVE fix
(2) upgrade of vulnerable dependency
(3) fix to secret leak and others.
Please consider to update the reference to the action.

Attributes and settings listed in the same list

Separate in different lists settings and configurations

Database access error when updating container version

After updating the device-manager from the tag version 0.2.0-nightly20180321 to tag version 0.2.0-nightly20180326 the database model became inconsistent with the module returning the following error:

 [SQL: 'SELECT anon_1.templates_id AS anon_1_templates_id, anon_1.templates_label AS anon_1_templates_label, anon_1.templates_created AS anon_1_templates_created, anon_1.templates_updated AS anon_1_templates_updated, attrs_1.id AS attrs_1_id, attrs_1.label AS attrs_1_label, attrs_1.created AS attrs_1_created, attrs_1.updated AS attrs_1_updated, attrs_1.type AS attrs_1_type, attrs_1.value_type AS attrs_1_value_type, attrs_1.static_value AS attrs_1_static_value, attrs_1.template_id AS attrs_1_template_id, attrs_1.parent_id AS attrs_1_parent_id \nFROM (SELECT templates.id AS templates_id, templates.label AS templates_label, templates.created AS templates_created, templates.updated AS templates_updated \nFROM templates \n LIMIT %(param_1)s OFFSET %(param_2)s) AS anon_1 LEFT OUTER JOIN attrs AS attrs_1 ON anon_1.templates_id = attrs_1.template_id'] [parameters: {'param_1': 20, 'param_2': 0}] (Background on this error at: http://sqlalche.me/e/f405)
10.46.0.11 - - [26/Mar/2018:16:15:26 +0000] "GET /template HTTP/1.1" 500 25 "http://10.50.11.32/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:59.0) Gecko/20100101 Firefox/59.0"
[2018-03-26 16:15:47,457] ERROR in app: Exception on /template [GET]
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1193, in _execute_context
    context)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 507, in do_execute
    cursor.execute(statement, parameters)
psycopg2.ProgrammingError: column attrs_1.parent_id does not exist
LINE 1: ...alue, attrs_1.template_id AS attrs_1_template_id, attrs_1.pa...

To avoid this kind of issue it is necessary implementing some sort of database versioning and migration functionalities for when there are model changes.

DeviceManager is not creating subscriptions in STH

Actuation returning internal error

I'm submitting a ...
- bug report
- feature request
- support request
Do you want to request a feature or report a bug?
Report a bug.
What is the current behavior?
Device-manager returns internal error when trying to actuate over a device, but, when the service restarts, it works again.
If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem
I was trying to actuate with this request:
curl -X PUT http://localhost:8000/device/44f3cb/actuate -H "Authorization: Bearer ${JWT}" -H 'Content-Type:application/json' -d ' {
"attrs": {
"FWUpdate-Update" : "e"
}
}'
Here are the logs:
What is the expected behavior?
To work normally.
What is the motivation / use case for changing the behavior?
Please tell us about your environment:
- Version: 0.2.x
- Environment: [kubernetes, docker-compose, standalone]
- Operating system: [Ubuntu 16.04, Fedora 28, Ubuntu on VirtualBox (Windows 10), etc.]
Other information (e.g. detailed explanation, stacktraces, related issues, suggestions how to fix, links for us to have context, eg. stackoverflow, gitter, etc)

Template and device creation problem

While experimenting with the platform I encountered some problems with the creation of devices and with creation and listing of templates. I still don't know if I'm doing something wrong while configuring the Dojot plataform or something alike.

I used the following command to create the device:

curl -X POST http://localhost:8000/device -H "Authorization: Bearer ${JWT}" -H 'Content-Type:application/json' -d '{
    "label": "SensorModel",
    "attrs": [
        {
            "label": "temperature",
            "type": "dynamic",
            "value_type": "float"
        }
    ] }'

This is the reply from the Device-Manager:

{"msg": "Internal Error"}

By looking with the docker-composer I got the following error:

auth_1            | 2018-02-05 15:31:20,937 - INFO - user 1 permit to POST on /device/
auth_1            | 172.19.0.17 - - [05/Feb/2018:15:31:20 +0000] "POST /pdp HTTP/1.1" 200 38 "-" "LuaSocket 3.0-rc1"
device-manager_1  | [2018-02-05 15:31:20,943] ERROR in app: Exception on /device [POST]
device-manager_1  | Traceback (most recent call last):
device-manager_1  |   File "/usr/local/lib/python2.7/site-packages/Flask-0.12-py2.7.egg/flask/app.py", line 1982, in wsgi_app
device-manager_1  |     response = self.full_dispatch_request()
device-manager_1  |   File "/usr/local/lib/python2.7/site-packages/Flask-0.12-py2.7.egg/flask/app.py", line 1614, in full_dispatch_request
device-manager_1  |     rv = self.handle_user_exception(e)
device-manager_1  |   File "/usr/local/lib/python2.7/site-packages/Flask-0.12-py2.7.egg/flask/app.py", line 1517, in handle_user_exception
device-manager_1  |     reraise(exc_type, exc_value, tb)
device-manager_1  |   File "/usr/local/lib/python2.7/site-packages/Flask-0.12-py2.7.egg/flask/app.py", line 1612, in full_dispatch_request
device-manager_1  |     rv = self.dispatch_request()
device-manager_1  |   File "/usr/local/lib/python2.7/site-packages/Flask-0.12-py2.7.egg/flask/app.py", line 1598, in dispatch_request
device-manager_1  |     return self.view_functions[rule.endpoint](**req.view_args)
device-manager_1  |   File "/usr/src/app/DeviceManager/DeviceManager.py", line 128, in create_device
device-manager_1  |     device_data, json_payload = parse_payload(request, device_schema)
device-manager_1  |   File "/usr/src/app/DeviceManager/SerializationModels.py", line 72, in parse_payload
device-manager_1  |     data, errors = schema.load(json_payload)
device-manager_1  | ValueError: need more than 1 value to unpack
device-manager_1  | 172.19.0.17 - - [05/Feb/2018:15:31:20 +0000] "POST /device HTTP/1.1" 500 25 "-" "curl/7.35.0"

Creating or listing templates also gives an error, giving the same error message to a POST, but with the following output on docker-composer:

GET:

 curl -X GET http://localhost:8000/template -H "Authorization: Bearer ${JWT}"

Reply:

{"msg": "Internal Error"}

Docker-Composer output:

auth_1            | 2018-02-05 15:33:45,495 - INFO - user 1 permit to GET on /template/
auth_1            | 172.19.0.17 - - [05/Feb/2018:15:33:45 +0000] "POST /pdp HTTP/1.1" 200 38 "-" "LuaSocket 3.0-rc1"
device-manager_1  | [2018-02-05 15:33:45,515] ERROR in app: Exception on /template [GET]
device-manager_1  | Traceback (most recent call last):
device-manager_1  |   File "/usr/local/lib/python2.7/site-packages/Flask-0.12-py2.7.egg/flask/app.py", line 1982, in wsgi_app
device-manager_1  |     response = self.full_dispatch_request()
device-manager_1  |   File "/usr/local/lib/python2.7/site-packages/Flask-0.12-py2.7.egg/flask/app.py", line 1614, in full_dispatch_request
device-manager_1  |     rv = self.handle_user_exception(e)
device-manager_1  |   File "/usr/local/lib/python2.7/site-packages/Flask-0.12-py2.7.egg/flask/app.py", line 1517, in handle_user_exception
device-manager_1  |     reraise(exc_type, exc_value, tb)
device-manager_1  |   File "/usr/local/lib/python2.7/site-packages/Flask-0.12-py2.7.egg/flask/app.py", line 1612, in full_dispatch_request
device-manager_1  |     rv = self.dispatch_request()
device-manager_1  |   File "/usr/local/lib/python2.7/site-packages/Flask-0.12-py2.7.egg/flask/app.py", line 1598, in dispatch_request
device-manager_1  |     return self.view_functions[rule.endpoint](**req.view_args)
device-manager_1  |   File "/usr/src/app/DeviceManager/TemplateManager.py", line 36, in get_templates
device-manager_1  |     templates = template_list_schema.dump(page.items).data
device-manager_1  | AttributeError: 'list' object has no attribute 'data'
device-manager_1  | 172.19.0.17 - - [05/Feb/2018:15:33:45 +0000] "GET /template HTTP/1.1" 500 25 "-" "curl/7.35.0"

Setting static values in device creation

During the device creation isn't possible to set a value for static attributes.

Someone has registered dojot/device-manager with the SCAN project

Hi,

A user with email address "rascaraficci" [email protected] has requested to be maintainer of this project with the SCAN service at https://scan.coverity.com. I can not find this user amongst the project's contributors so I am double checking.

Also requested dojot/docker-monitor

Please reply to [email protected] to approve or deny the request