Feature Request: Get Update status at the reporter from worker only when update is performed ( failed or successful ). about worker HOT 11 CLOSED

Issue-Label Bot is automatically applying the label feature_request to this issue, with a confidence of 0.95. Please mark this comment with 👍 or 👎 to give our bot feedback!

Links: app homepage, dashboard and code for this bot.

from worker.

So to summarize the request (and make sure I'm getting it correctly) your asking to have a flag in the worker configuration that when set will only send reports on updates.

something like:

• report_on_update_only: false - the default setting, will send all reports
• report_on_update_only: true - will only send a report if a change happened

from worker.

Hello @naorlivne

Thanks for the Response

So to summarize the request (and make sure I'm getting it correctly) your asking to have a flag in the worker configuration that when set will only send reports on updates.

something like:

• report_on_update_only: false - the default setting, will send all reports
• report_on_update_only: true - will only send a report if a change happened

Yes this solves the problem that's mentioned in the first paragraph of the feature request comment

I want the complete device update history and store only those report which contains the status of updates. ( i.e. Fail or Success ) This data can be purged after 11 or 12 months if required.

Is it possible to do what I have mentioned below =>

To elaborate the above two points I want the current behavior where I get the device state ( for example if container is running, ram, CPU, etc ) continuously according to the NEBULA_MANAGER_CHECK_IN_TIME which I can purge after some time ( for example six months ) but for another behavior I want the update report ( in this I am expecting when was device was updated and which release it has ) only if the end device is updated successfully or the update failed. I want to maintain the data for second behavior for more time comparatively than the first behavior.

To summarize, is it possible to have two reports i.e. worker reports about the update only when the update happens on Edge device ( Either fail or success ) and also another report where worker continuously reports the device state ( i.e. Report's that it is sending currently, for example, CPU and ram consumption. ) depending on the NEBULA_MANAGER_CHECK_IN_TIME.

from worker.

having two reports at the same time would make for a lot of redundant data so it's not something I feel comfortable adding in, I will however won't mind having another field added that states if each document updated anything or not in the same reports stream which can be used to filter with a query param on.

from worker.

Hello @naorlivne

Thanks for the Response.

having two reports at the same time would make for a lot of redundant data so it's not something I feel comfortable adding in, I will however won't mind having another field added that states if each document updated anything or not in the same reports stream which can be used to filter with a query param on.

Yes, It seems quite convincing about the redundant data.

To summarize, is it possible to have two reports i.e. worker reports about the update only when the update happens on Edge device ( Either fail or success ) and also another report where worker continuously reports the device state ( i.e. Report's that it is sending currently, for example, CPU and ram consumption. ) depending on the NEBULA_MANAGER_CHECK_IN_TIME.

Since the current report data contains a lot of relevant information about the device state, it could be possible to use the reports for monitoring the Edge Device.

So to summarize the request (and make sure I'm getting it correctly) your asking to have a flag in the worker configuration that when set will only send reports on updates.

something like:

report_on_update_only: false - the default setting, will send all reports
report_on_update_only: true - will only send a report if a change happened

Considering the use case I mentioned if one is only interested in a device update this seems to be a better solution.

from worker.

I'll implement 2 things in Nebula in regards to this:

The first being the report_on_update_only as we discussed above & The second is I'll add a query param filter to the report that will filter the reports down only to those where Nebula had an ID changed on the worker configuration (when it was really updated), let's call this field "updated":

• When "updated" is true it only returns those reports that had a change in them
• When "updated" is false it only returns those fields that didn't had a change in them
• When "updated" is undeclared (or possibly a wildcard *) it returns everything

So for your use case you'll likely want to do something as follows:

1. Set report_on_update_only as false
2. Whenever you want to check the devices cpu\mem\etc you send the /reports query without the updated field
3. Whenever you want to ensure a device updated you send the query with the updated field set to true and only get reports which had the device updated in them

from worker.

Okay

from worker.

Added the features discussed above to the latest\master branch\release.

from worker.

Hello @naorlivne

Thanks for accepting the feature request.

I have cloned the worker repository on the raspberry pi.

I am facing an issue when I give REPORT_ON_UPDATE_ONLY: true as the boolean value under the environment section inside the worker's docker-compose.yml.

docker-compose.yml =>

version: '3'
services:
  worker:
    container_name: worker
    build:
      context: .
      dockerfile: Dockerfile
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    restart: unless-stopped
    hostname: worker
    environment:
      REGISTRY_HOST: < registry_url >
      REGISTRY_AUTH_USER: < registry_username >
      REGISTRY_AUTH_PASSWORD: < registry_password >
      MAX_RESTART_WAIT_IN_SECONDS: 0
      NEBULA_MANAGER_AUTH_USER: < nebula_manager_username >
      NEBULA_MANAGER_AUTH_PASSWORD: < nebula_manager_password >
      NEBULA_MANAGER_HOST: < nebula_manager_url >
      NEBULA_MANAGER_PORT: < nebula_manager_server_port >
      NEBULA_MANAGER_PROTOCOL: < nebula_manager_protocol >
      NEBULA_MANAGER_CHECK_IN_TIME: 5
      DEVICE_GROUP: test
      KAFKA_BOOTSTRAP_SERVERS: < nebula_kafka_cluster >:9092
      KAFKA_TOPIC: nebula-reports
      REPORT_ON_UPDATE_ONLY: true

I get the following error =>

ERROR: The Compose file './docker-compose.yml' is invalid because:
services.worker.environment.REPORT_ON_UPDATE_ONLY contains true, which is an invalid type, it should be a string, number, or a null

But when I give REPORT_ON_UPDATE_ONLY: 'true' it works properly. Also, I have tested if I get the report only when an update is done and it is working properly.

So to confirm if the value type for REPORT_ON_UPDATE_ONLY key is string I referred to this documentation and I found that it is given as a boolean.

from worker.

That's a docker\linux issue, when you setting the environment field your actually setting the environment variables of it, which in linux can only be a string.

Because your using docker-compose (which is based on YAML) when you pass true or false the YAML interpreter docker-compose is using goes with boolean, which fails once it reaches the point of the docker-compose that actually tries setting the envvars to it.

The param is boolen (so if you where to set it via a JSON\YAML\ETC config file you would need to set it to boolen) but as your using docker-compose to create the envvars you have to hardcode it to be a string first as you found out above.

TL;DR:
It's the expected behaviour, noting wrong.

from worker.

Okay Thank you @naorlivne

from worker.