unpoller / dashboards Goto Github PK

I'm not sure if the root cause of this issue is in parsing the Unifi data or in the ddashboard itself, but I'm reporting here since its where I see the issue.

In the screenshot above, the data for my Google Nest Minis is split. It looks like the OUI got split causing everything to be misaligned by one column. Unifi reports the proper OUI is "Google, Inc.", so my gut says this is a CSV parsing issue (or similar) where the , isn't being escaped or quoted properly. I get the same issue for other OUI names with commas too, but this is the most obvious.

Let me know if I can provide any additional info or if there's a better repo to report this issue. Not sure which step in the process caused the problem.

Happy to dig around for a fix and put together a PR as well if I can get a bit of a head-start from maintainers on where to start looking.

Hi, i noticed, that - from time to time - the "Client DPI" (and other IDS related) charts are empty (while others are fine on the same page). First I thougt it had to do something with the grafana dashboard, so I tried exploring the influx data. I found out, that unpoller still writes data, but there are no changes in the counter so that the lines in the charts resulting in a flat zero line. Then I tought it could be some error in the mongodb included in the unifi controller. But unpoller is still reading from there and doesn´t report any errors - even in debug mode (btw. there are no errors in all unifi logs).

While I was trying to understand whats going on, I played around with the unifi mobile app, too. And suddenly the charts started to becaome alive agian. After that I waited for the next failure in the charts and tried to reproduce, what made them working again.

Sometimes the charts are working for hours, sometimes they break after a few minutes. So it was a game of chance to catch the next error. But after some time the charts hung again and I tried, what I did before:

• start the unifi mobile app
• switch to the "client devices" tab
• select some random client
• view the "insights" tab

It seems the app stops working, because it takes some time to display the DPI data, but after some seconds the data appears - AND then the DPI Charts in grafana start working, too.

Can someone reproduce this?

I think there is special API call, that "resets" some kind of broken processing or something like that. If so, perhaps it is possible, that unpoller could send this call from time to time, to prevent this behaviour?

kind regards
and thanks for this tool

I have just updated my UXG Lite to the UGX Max Gateway device.
The switch went well, only when I open the UniFi-Poller: USG Insights - InfluxDB Dashboard, I am presented with the following dashoard.

I think that this was caused by me renaming the gateway after the installation.
The current name is ICANGateway. No matter what I try I cannot seem to remove the other two Gateway names.
I did try to delete these but then it also ended up deleting the ICANGateway as well.
Looking at the separate dashboards, two of the 3 have no data. The ICANGateway has data as expected.
How can I delete the second two dashboards. Below are all 3 dashboards.

Thanks
Lawrence

I'm new to Grafana, and actually Unifi as well, I've loaded 4 of the other dashboards and all work well. The Client DPI dashboard is not populating the Sites menu. I'm running the latest Grafana 7.0.3_armhf on a Raspberry Pi 4, using InfluxDB, not sure the version there. Of course, since there is no site selected, it shows no data available.
Thanks for listening to my newb-ness!

Running this dashboard, and have DPI enabled, but no data is populating. All the other Unifi Poller dashboards are working fine. I'm using InfluxDB.

is it possible to add Unpoller stats to HA? When i try to add InfluxDB to HA it tells me to add a database specific for HA in the yaml file. But i don't know how i can fill this db with Unpoller stats.

No data in UniFi / UniFi-Poller: Client DPI - InfluxDB

All the other Dashboards are populated.

Any clues ?

in the US and have the USW Insights dashboard loaded and I get the above error on the "Switch Temperature"

I manually had to remove the 'WHERE "site"=/^$site$/' from all graphs in the v2 Dashboards to again see values.

Hello,

Many thanks for all this work.

New released "Gateway Max" identified as type 'uxg', compared to my previous model, 'usg'.
Grafana variables need to be updated to show related metrics in USG dashboard:

Controller --> label_values(unpoller_device_info{type=~"udm|usg|ugw|uxg"},source)	
Site --> label_values(unpoller_device_info{source=~"$Controller", type=~"udm|usg|ugw|uxg"},site_name)	
Gateway --> label_values(unpoller_device_info{site_name=~"$Site", type=~"udm|usg|ugw|uxg"},name)	

I guess it is the same for new "UniFi Cloud Gateway Ultra" that may match type = 'ucg' (I do not own such device to validated the type value)

For example USW Insights, Data Transfer chart show no data when time range is 2 days, but if I change the time range to 1 week suddenly the data appear.

It seems something related to a recent change. I run unpoller since 1y and I never noticed this behavior.

Any idea?

Seems related to scaping interval (Mine is set to 30s in Grafana Agent) and minimum interval in dashboard (set to 30s on empty charts and to 2min on others)

Just setup and installed unifi-poller, grafana, and influxDB and imported 'UniFi Client DPI: InfluxDB Dashboard' Imported fine but nothing will load, the "site" selection is empty. All of the other dashboards on "https://grafana.com/grafana/dashboards?dataSource=influxdb&search=unifi-poller" are working ok but not this one.

I am running this against an UDM-PRO, maybe something to do with the way it handles sites?

I guess this might be more of a suggestion for an improvement but I know personally my Client dashboard pulldown has a ton of MAC address-only entries at the top of the Client dashboard variable pulldown:

Played around with the dashboard variable settings for Client and seems using this value for regex works to exclude them:

    "regex": "/^(?!(?:[a-fA-F0-9]+:)+)(.+)$/",

See this regex101 example but I was trying to do a negative look ahead match for any MAC address and otherwise include any positive match.

Best I can tell from looking at the dashboard export JSON model, it seems like the entry for the Client variable has no value for regex now and setting the above one will result in stripping out all MAC address but that behavior might not be desired.

I feel like there may be a better solution in the form of pulling out MAC addresses conditionally or having another variable that is just MAC addresses but that is about the extent of my knowledge with Grafana dashboards at this point. If it is helpful I can share the full exported dashboard JSON as well.

There is already support for unpoller to fetch the metrics for the Unifi PDU Pro.
For prometheus, they are made available at unpoller_device_outlet_.

Is it possible to have a dashboard made for the pdu?

I am very new to grafana and unnifi-poller.
I have installed the unifi-poller on influxdb. I downloaded the following dashboards.

Client DPI 10419 -- shows no data
Sites 10414 -- shows data
USW 10417 -- shows data
USG 10416 -- shows data
UAP 10415 -- shows data
Clients 10418 -- shows data

As you can see above, all the dashboards work and deliver data very well, only the Client DPI shows no data.

Below is something that I find strange but don't know how to change/update. As you can see, under the Site selection, there is no site possible, it is showing as None.

Is there something I missed in the setup, how can I investigate the no data.

A very BIG Thank You for a great functional product. Very impressive.
Thanks
Lawrence

I have just reinstalled influxdb and downloaded all the dashboards again.
FYI, I replaced my old USG 3 with the new UXG Lite
There are a few items that are blank, but I will ask about those in a different issue.
For the dashboard "UniFi-Poller: Client DPI - InfluxDB" all measurements are blank and showing no data.

Can you please provide me some guidance to see if I can try to figure this out.

Thanks
Lawrence

I loaded all 5 Grafana, UniFi, plugins. Four of them appear to be working fine, but the UniFi-Poller: Client DPI - InfluxDB plugin only provides: No Data To Show. I do have DPI enabled on my USG, so there should be data.

Dear David,

Thank you again for such great dashboards. Only in the last months I have noticed that there seems to be a warning that has popped up after I upgraded Grafana the last time. See image below.
"Angular plugin
This plugin uses a deprecated, legacy platform based on AngularJS and will stop working in future releases of Grafana."

I am a complete novice when it comes to Grafana and the dashboards. How can I resolve this issue.
Is there a fix for this?

Thanks
Lawrence

I have loaded all unifi-poller dashboards and all of them work excellent except the client DPI dashboard doesn't work. No data gets loaded.

Recently upgraded unifi to version 8.0.28 so my unifi-poller stopped working. Upgraded to unpoller.

Seems to be collecting data OK so far, but this is spamming the logs. Is this ok?
tail -f -n100 /var/log/syslog /var/log/messages | grep unpoller
tail: cannot open '/var/log/messages' for reading: No such file or directory
Feb 7 04:52:39 grafana unpoller[537983]: 2024/02/07 04:52:39 [ERROR] influxdb.Write(points): {"error":"partial write: field type conflict: input field "rx_bytes-r" on measurement "subsystems" is type integer, already exists as type float dropped=5"}
Feb 7 04:53:09 grafana unpoller[537983]: 2024/02/07 04:53:09 [ERROR] influxdb.Write(points): {"error":"partial write: field type conflict: input field "rx_bytes-r" on measurement "subsystems" is type integer, already exists as type float dropped=5"}
Feb 7 04:53:39 grafana unpoller[537983]: 2024/02/07 04:53:39 [ERROR] influxdb.Write(points): {"error":"partial write: field type conflict: input field "rx_bytes-r" on measurement "subsystems" is type integer, already exists as type float dropped=5"}
Feb 7 04:54:09 grafana unpoller[537983]: 2024/02/07 04:54:09 [ERROR] influxdb.Write(points): {"error":"partial write: field type conflict: input field "rx_bytes-r" on measurement "subsystems" is type integer, already exists as type float dropped=5"}
Feb 7 04:54:39 grafana unpoller[537983]: 2024/02/07 04:54:39 [ERROR] influxdb.Write(points): {"error":"partial write: field type conflict: input field "rx_bytes-r" on measurement "subsystems" is type integer, already exists as type float dropped=5"}
Feb 7 04:55:09 grafana unpoller[537983]: 2024/02/07 04:55:09 [ERROR] influxdb.Write(points): {"error":"partial write: field type conflict: input field "rx_bytes-r" on measurement "subsystems" is type integer, already exists as type float dropped=5"}
Feb 7 04:55:39 grafana unpoller[537983]: 2024/02/07 04:55:39 [ERROR] influxdb.Write(points): {"error":"partial write: field type conflict: input field "rx_bytes-r" on measurement "subsystems" is type integer, already exists as type float dropped=5"}

Just installed "UniFi-Poller: Client DPI - InfluxDB" and I have no data in any of the graphs. I do have DPI switched on in unifi.

All the other dashboards work great.

Lawrence

Do we have any dashboards for monthly bandwidth usage yet in Grafana?

For example, the UniFi portal now allows you to search by a range of days and specific clients.

Looks like the query's for the dashboards need to be improved to allow this for InfluxDB.

Also you should redirect dashboard questions from the https://github.com/unpoller/unpoller/issues repo here.

As a example of what I have attempted to create. The other issue I noticed is that if the UDM Pro reboots, then the sum'ed total that is carried forward gets reset and you get this large negative values.

https://pastebin.com/5Cw4m5sd

Hi! Amazing project :) Thank you so much for all the work you folks have put into this!

I have all the dashboards working, except for the DPI one. I havb UP_UNIFI_DEFAULT_SAVE_DPI set to true, and that seems to be working fine, since I see this in the unpoller logs in docker:

2022/12/25 04:43:47 [INFO]    => Save Sites true / Save DPI true (metrics)

I also have DPI enabled in UnifiOS:

However, I see these errors in Graphana:

I have influxdb v 1.8.4 as recommended in the docs. I couldn't find any reference to this error anywhere else, so I'm not sure what i could be missing.

Thanks in advance!

The USG dashboard doesn't currently support gateways of type uxg. At the time of opening this issue, I believe there are now three such gateways available: UXG-Lite, UXG-Max and UXG-Pro. I got some of the dashboard to work by simply including uxg as an option for the defined dashboard variables.

label_values(unpoller_device_info{type=~"udm|usg|ugw|uxg"},source)
label_values(unpoller_device_info{source=~"$Controller", type=~"udm|usg|ugw|uxg"},site_name)
label_values(unpoller_device_info{site_name=~"$Site", type=~"udm|usg|ugw|uxg"},name)

I'm running a UXG-Max, and can therefore run some tests if that's helpful.

I have just updated my UXG Lite to the UGX Max Gateway device.
The switch went well, only when I open the UniFi-Poller: USG Insights - InfluxDB Dashboard, I am presented with the following dashoard.

I think that this was caused by me renaming the gateway after the installation.
The current name is ICANGateway. No matter what I try I cannot seem to remove the other two Gateway names.
I did try to delete these but then it also ended up deleting the ICANGateway as well.
Looking at the separate dashboards, two of the 3 have no data. The ICANGateway has data as expected.
How can I delete the second two dashboards. Below are all 3 dashboards.

Thanks
Lawrence

Bug(?):
Found that the clients in the "Client / IP" list are listed twice when you select "Wireless: All" and "Wired: All" in the Variables at the top of the dashboard.
It seems that the following two changes help solve the issue:
Step 1: remove the ".*" in the "Custom all variable" field for the Wired and Wireless variables.
Step 2a: replace the AND is_wired=true with AND "is_wired"='true' in the query for the Wired variable.
Step 2b: replace the AND is_wired=false with AND "is_wired"='false' in the query for the Wireless variable.
(Note: I don't have wireless clients in my network and thus cannot verify that this works for the Wireless variable - but it prevents loading all the wired clients into the Wireless variable.)

Suggestion:
I found setting the Wired and Wireless variables to "Sort"="case insensitive, asc" instead of "Sort"="asc" makes this list more readable. That way, a "crApple iPad" is listed between "Carla's PC" and "Curt's Server" - and not after the "XBOX".

By the way:
A very big "Thank you!" for developing this UnFi-Poller tool. It helped me identify and fix several issues in our network - and most of it I wouldn't have found if I only relied on the UI of my Unifi DreamMachine Pro!

In the v2.0.1 of UniFi-Poller: Client Insights - InfluxDB the dashboard table is not current with the selected time filter.

For example, the Rx and Tx rates should change when changing the time filter. Seems the data is stuck to looking back over a static period of time.

Hi, I just installed your unifi-poller (influxdb) and the rest of the charts work fine, except for the Client-DPI (#10419).

Am i missing anything here?

Nothing loads and it just says No data to show.

Let me know what other information you require :)

Hello,

i've been testing everything with 6.0.20 and there is a lot of information missing. Client DPI

The Guest count for the 'UAP Insights - InfluxDB' dashboard displays as 0/zero.

To fix change the query to is_guest evaluation to AND is_guest = 'true' note single quotes rather than double.