jbouwh / omnikdatalogger Goto Github PK

Home assistant logs show message like:

2021-11-19 09:34:36 WARNING (MainThread) [homeassistant.components.sensor] sensor.laatste_statusupdate is providing a string for its state, while the device class is 'timestamp', this is not valid and will be unsupported from Home Assistant 2022.2. Please create a bug report at https://github.com/home-assistant/core/issues?q=is%3Aopen+is%3Aissue+label%3A%22integration%3A+mqtt%22

MQTT publishing must be updated to comply with the new standard.

The example:

docker run --rm --serialnumber NLDN123456789012 --listenaddress 0.0.0.0 --omniklogger 176.58.117.69 --omnikloggerport 10004

The ouptput:

unknown flag: --serialnumber
See 'docker run --help'.

With the first solar measurement of the day today_energy starts again. The last_reset time stamp is provided, but home assistant does not seem to reset the value correctly.

Background

When the sun is down, omnik inverters stop providing data. Omnikdatalogger caches the last total_energy and today_enery. Other attributes are omitted. This causes warning in the log wen using the MQTT auto discovery in Home Assistant.

2021-10-07 06:35:08 WARNING (MainThread) [homeassistant.helpers.template] Template variable warning: 'dict object' has no attribute 'operation_hours' when rendering '{{(value_json.operation_hours)}}'
2021-10-07 06:35:08 WARNING (MainThread) [homeassistant.helpers.template] Template variable warning: 'dict object' has no attribute 'current_power_pv' when rendering '{{(value_json.current_power_pv)}}'
2021-10-07 06:40:08 WARNING (MainThread) [homeassistant.helpers.template] Template variable warning: 'dict object' has no attribute 'current_power' when rendering '{{(value_json.current_power)}}'
2021-10-07 06:40:08 WARNING (MainThread) [homeassistant.helpers.template] Template variable warning: 'dict object' has no attribute 'power_ac1' when rendering '{{(value_json.power_ac1)}}'
2021-10-07 06:40:08 WARNING (MainThread) [homeassistant.helpers.template] Template variable warning: 'dict object' has no attribute 'power_pv1' when rendering '{{(value_json.power_pv1)}}'
2021-10-07 06:40:08 WARNING (MainThread) [homeassistant.helpers.template] Template variable warning: 'dict object' has no attribute 'power_pv2' when rendering '{{(value_json.power_pv2)}}'

Cause

Omnikdatalogger publishes al inverter states over one shared state topic, but all entities listening to this state topic will be triggered. When certain attributes are omitted (the ones shown in the log), Home Assistant cannot update the state, which it should not, but it still causes a warning.

Possible solutions:

Possible solutions are:

• Use a unique state topic for each attribute.
• Publish all attributes, even at night, using cached values. Power attributes can be set to 0.

Using a state per attribute will give much more payload, and will break usecases for MQTT-users not using Home Assistant.
Adding this missing attributes will take away the warnings. A disadvantage is that updates need to be processed, even when there is no update. For users that do not want to retain MQTT messages this will be good, since the state will recover automatically for these attributes.

P.S. The warnings do not impact presenting the correct values over MQTT.

I am struggling with the Omnik Data logger Automation in HACS.
I am running Home Assistant running on a RPI and I have all software (HA-Core, Supervisor, HACS etc) updated to the latest releases.
I have:

• installed Appdaemon4
• updated /config/appdaemon/appdaemon.yaml (file appended as .txt)
• update /config/appdaemon/apps/apps.yaml (file appended as .txt)
• installed the integration DSMR with an Omnik v2.2. which is running and providing 5 sensors

But I have still an error in the Appdaemon4, see the last part of the log in appended log file
appdeamon log.log
appdaemon.txt
apps.txt

Do you need more info?
Can you advise me what I am missing or doing wrong.?

Kind regards,

I've noticed some strange behaviour of consumed energy after inverter shuts down after sunset.

See the attached screenshot:

It is visible on the pvoutput.org website when you open the stats of one of the previous days. When you look at the outputs, in this case for 04-12, you see the last inverter update is at 04:35 pm. After that the used energy column changes from 6.637 to 12.720 kWh. It stays at this value until the end of the day.

I'm not sure yet if it is the omniklogger script or something in PVoutput, but another account which is provided by the dsmr entity doesn't show this behaviour

I'm trying to output to an InfluxDB instance, but I'm getting 401's back on every config combination I try.
InfluxDB version: 2.1.1
Data logger version: 1.10.5

config attempt 1:

  output.influxdb:
    host: <some_IP>
    port: 8086
    username: <username>
    password: <password>
    database: solar
    use_temperature: true

config attempt 2:

  output.influxdb:
    host: <some_IP>
    port: 8086
    jwt_token: <token>
    database: solar
    use_temperature: true

token is created in Data --> API Tokens

log output:

WARNING:omnik.datalogger:W 2022-02-11T15:54:21.734385 Unhandled RequestException error: 401 Client Error: Unauthorized for url: http://<some_IP>:8086/write?db=solar

Am I missing something obvious?

Is it possible to connect to the inverter without a SN?
Since a software upgrade, my inverter is stripped from his serial number, the field is empty in the web interface, also external logging is broken.

I wanted to try to read the values on my local network via tcpclient.
The inverter is listening on port 8899.

I'm using the old serial number in the config file.
But i'm getting the following result:

INFO:omnik.datalogger:I 2023-04-15T19:48:43.310815 Initializing: Trying to reach the inverter for plant 123 over port 8899.
INFO:omnik.datalogger:I 2023-04-15T19:49:23.340375 Inverter is not reachable over port 8899, this is normal when it is dark. timed out

Hello Jan,

I have to tried to configure your tool to for use with my LSW3 datalogger, but unfortunately encounter an error message when using the tcpclient:
INFO:omnik.datalogger:I 2022-11-07T14:00:42.527266 Using configured data configuration from '/home/pi/omnikdatalogger/apps/omnikdatalogger/data_fields.json'.
INFO:omnik.datalogger:I 2022-11-07T14:00:42.527926 Using energy cache file './persistant_cache.json'.
INFO:omnik.datalogger:I 2022-11-07T14:00:42.552706 Initializing client : tcpclient.
INFO:omnik.datalogger:I 2022-11-07T14:00:42.554804 Client enabled: TCPclient
INFO:omnik.datalogger:I 2022-11-07T14:00:42.555128 Output plugins configured: ['influxdb'].
INFO:__main__:I 2022-11-07T14:00:42.575398 Daemon interval 360 seconds.
ERROR:omnik.datalogger:E 2022-11-07T14:00:43.577822 Unexpected error during data handling: '2746864'
INFO:__main__:I 2022-11-07T14:00:49.584578 Stopping Omnikdatalogger...

I am not sure if my hardware is supported yet?
logger_sn: 178****
inverter_port: 8899
inverter_sn: 234**** resp. SP1ES108***

Logger firmware: LSW3_15_270A_1.50

Thanks in advance
Sebastian

I try to make a mqqt connection to a cloud based mqtt broker. I use this for a remote omnik installation. For some reason I only get "MQTT disconnected" messages in the log, but nothing has been published to the mqtt broker. The messages are like these:

"INFO:omnik.datalogger:I 2021-10-26T14:07:45.747265 MQTT disconnected"

Actually this is more a question, is there support for mqtt using secure protocols (TLS, using port 8883)?

Hi Jan,

For years I'm a using modified Python-script to read the actual data from /js/status.js
I recently started playing around with your omnikdatalogger because I would like to use the tcpclient setup for retrieving data from my 3 inverters.

After playing around with the config.yaml file I can now read data from the two inverters for which the inverter_sn is starting with NLxxx, and the logger_sn starting with 602xxxxxx and 634xxxxxx.
But the one inverter with an inverter_sn starting with AALNxxxx I get a 'timed out' in the logging. This inverter is running with logger_sn starting with 644xxxxxx
Can it be that the omnikdatalogger is testing if the inverter_sn is starting with NLxxx, or is this an issue related to the logger_sn?

Regards George

I've updated my docker container to the latest version, as the home assistant log stated there were some legacy options used in the previous version (mdi icons). It seems to run fine (after adjusting the config directories to the new location). However, the docker container needs to be restarted daily to function normally.

Problem: The docker version of the datalogger does not connect to inverter after sunset / sunrise sequence.

When started, the logger works as aspected. The log shows:
INFO:omnik.datalogger:I 2021-11-29T09:00:00.954385 New message received from inverter ''
INFO:omnik.datalogger:I 2021-11-29T09:00:00.962378 Update for plant UTC 2021-11-29 08:00:00.952819+00:00

After sunrise the next day, the logger does not automatically connect to the inverter. The log shows:
INFO:omnik.datalogger:I 2021-11-29T08:15:26.050094 Combining cached logging for plant with DSRM data.
INFO:omnik.datalogger:I 2021-11-29T08:20:26.138647 Net data update for plant UTC 2021-11-29 07:20:01+00:00

The above message keeps repeating every 5 minutes, I dont see the message "New message received from inverter ''" in the logs until the docker container is restarted.

I did a pull from hub.docker.com and I build the container myself, both gave the same result. I've looked in the documentation for a config option that I may have missed, but couldn't find one.

Jan,

I am having a Trannergy inverter; afaik, same family as Omnik. For the Trannergy inverter, I don't need the localproxy work around. You can set a "remote server" which points to the server which runs the data collection script. See attached screenshot.

I was looking for a means to collect data at a higher frequency....I run into your tcpclient, which works for my Trannergy inverter; thanks!

Cheer
Hans

Installed HACS and appdeamon and then datalogger in HA 11. Appdeamon crashes when using dsmr but that is not a problem. The problem is that I can’t seem to start the app. Hereby the log from appdeamon:

2023-10-24 21:47:21.823580 WARNING Error: ------------------------------------------------------------
2023-10-24 21:50:43.298566 WARNING Error: ------------------------------------------------------------
2023-10-24 21:50:43.298916 WARNING Error: Unexpected error loading module: /config/appdaemon/apps/omnikdatalogger/omniklogger.py:
2023-10-24 21:50:43.299195 WARNING Error: ------------------------------------------------------------
2023-10-24 21:50:43.305116 WARNING Error: Traceback (most recent call last):
File "/usr/lib/python3.11/site-packages/appdaemon/app_management.py", line 1000, in check_app_updates
await utils.run_in_executor(self, self.read_app, mod["name"], mod["reload"])
File "/usr/lib/python3.11/site-packages/appdaemon/utils.py", line 304, in run_in_executor
response = future.result()
^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/appdaemon/app_management.py", line 783, in read_app
self.modules[module_name] = importlib.import_module(module_name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/importlib/init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "", line 1204, in _gcd_import
File "", line 1176, in _find_and_load
File "", line 1147, in _find_and_load_unlocked
File "", line 690, in _load_unlocked
File "", line 940, in exec_module
File "", line 241, in _call_with_frames_removed
File "/config/appdaemon/apps/omnikdatalogger/omniklogger.py", line 13, in
from omnik.datalogger import DataLogger
File "/config/appdaemon/apps/omnikdatalogger/omnik/datalogger.py", line 20, in
from .dsmr import DSRM
File "/config/appdaemon/apps/omnikdatalogger/omnik/dsmr/init.py", line 1, in
from dsmr_parser import obis_references
ModuleNotFoundError: No module named 'dsmr_parser'

2023-10-24 21:50:43.305558 WARNING Error: ------------------------------------------------------------

Some assistant ?

Latest Home Assistant introduced new Energy Grid module. But DSMR entities produced by "omnikdatalogger" does not apper in the list of "Consumed Energy" entities list

I'm running the lates version (v1.6.4) of omnikdatalogger

Also couple of suggestions:

1. disable polling omnik inverter in tcpclient mode from dusk to dawn to not spam the log
2. keep updating "combined DSMR" entities event if there is no updates coming from Inverter.

Anyway, thanks for amazing tool, I really enjoy it!!

Hi Jan,

I currently have a HomeWizard Wi-Fi P1 meter connected to the central meter to export some power data. Now, this device sends it both to an remote system and exposes the data via a local, unprotected endpoint. I would like to see if it would be possible to integrate this data with the data from our solar panel inverters. Would it be possible to support such endpoints?

I realize of course that it is impossible to support such devices all from the datalogger but maybe if users supply their own field mapping for the endpoint, assuming the data is exposed in JSON, it could work?

I'm currently looking at the following exported data:

{
    "smr_version": 42,
    "meter_model": "Kaifa KAIFA-METER",
    "wifi_ssid": "<REDACTED>",
    "wifi_strength": 54,
    "total_power_import_t1_kwh": 19076.256,
    "total_power_import_t2_kwh": 16463.774,
    "total_power_export_t1_kwh": 3110.004,
    "total_power_export_t2_kwh": 7837.318,
    "active_power_w": 826,
    "active_power_l1_w": 219,
    "active_power_l2_w": 4,
    "active_power_l3_w": 603,
    "total_gas_m3": null,
    "gas_timestamp": null
}

Unfortunately I do not have this information for the gas part as it's not a "smart" gasmeter.

If this cannot be supported, could I do the same data combining afterwards via my InfluxDB collections? Or does the logger do some magic internally to match the two sources together?

Thanks,
Ian

First of all: thank You for this fantastic logger!

This week I started with the omnikdatalogger in my existing home assistant application.
I can connect by port 8899 with my inverter and can scan every 60 seconds..
It is working very well, but only in the morning I see a strange behaviour, the appdeamon log displays errors and the current_power value is 0 Watt.

2022-04-17 07:57:34.354666 INFO omnik_datalogger: No sunshine postponing till down next dawn 2022-04-17 06:01:55.739746+02:00.
2022-04-17 07:57:35.003055 INFO omnik_datalogger: Update for plant rb12 UTC 2022-04-17 05:57:34.977773+00:00

But when I look directly in the Omnik inverter (with a webbrowser) it show that the value is not 0 Watt, but 142 Watt. this value is also presented in the sensor AC power L1.