Comments (41)
Hi @bb-Ricardo
Thanks for your information. I didn't run Influx v2 at the moment. I use InfluxDB v1.8.10 because I thought it could be, because I setup a database inside the bucket. This means, that I have the utilization and temperature problem with v1.8.10 and v2.22.2.
I will now test with your former version of fritzinflux. It's now running and I will wait for some more data. Will come back then.
from fritzinfluxdb.
you can disable the collection of the Lua values here:
https://github.com/karrot-dev/fritzinfluxdb/blob/rewrite-adding-async/fritzinfluxdb.py#L96
Just comment it out and then the Lua data won't be requested.
Thank you for looking into this.
maybe cutting down the precession from "ms" to "s" helps.
from fritzinfluxdb.
you can disable the collection of the Lua values here: https://github.com/karrot-dev/fritzinfluxdb/blob/rewrite-adding-async/fritzinfluxdb.py#L96
Just comment it out and then the Lua data won't be requested.
I will give it a try...
from fritzinfluxdb.
I change the precision to "s" in the branch. Can you pull the the branch again and give it a try?
Thank you
already did... now running the container... will come back to you later...
from fritzinfluxdb.
now running with this lua services... maybe with this I will not run in high temp and utilization ;o)
fritzbox_services.append(
{
"name": "System Stats",
"page": "ecoStat",
"value_instances": {
"cpu_temp": {
"data_path": "data.cputemp.series.0.-1",
"type": int
},
"cpu_utilization": {
"data_path": "data.cpuutil.series.0.-1",
"type": int
},
"ram_usage_fixed": {
"data_path": "data.ramusage.series.0.-1",
"type": int
},
"ram_usage_dynamic": {
"data_path": "data.ramusage.series.1.-1",
"type": int
},
"ram_usage_free": {
"data_path": "data.ramusage.series.2.-1",
"type": int
}
}
})
# every 2 minutes
fritzbox_services.append(
{
"name": "Active network hosts",
"page": "netDev",
"params": {
"useajax": 1,
"xhrId": "all",
"xhr": 1,
"initial": True
},
"interval": 120,
"value_instances": {
# "active_host": {
# "data_path": "data.active",
# "type": list,
# "next": extract_single_host
# },
"num_active_host": {
"type": int,
"value_function": lambda data: len(data.get("data", {}).get("active", [])),
}
}
}
)
from fritzinfluxdb.
hi @bb-Ricardo looks good... with the reduced lua values I didn't face the temp and utilization problem. for me it's fine and I close this case...
from fritzinfluxdb.
Hey @bb-Ricardo
Nice to hear. I will definitely do it. But I need a bit time, I have limited access to my pi at the moment.
As soon as I set up the actual version I will give you a response.
BR
Ronald
from fritzinfluxdb.
Hello @ronaldn1969,
I noticed some similar issues that loading the dashboard got quite slow as soon as there was more data collected.
I would NOT think that this is an issue of fritzinfux and more a Influx/grafana issue.
First I blamed it on the shard group duration which was set to to something like 168 hours instead of 24. Now on creation of new buckets with the fritzinfluxdb they get set to 24H. -> d314333#diff-7e286a2207fa26a8ed2169a4f16250b27fb2564f10118236da39f3f7dc0ff006L210
Maybe you can check the shard group duration for your bucket in influxdb.
from fritzinfluxdb.
morning @bb-Ricardo
after collecting data over the weekend and check this morning, I didn't faced the issue on my Pi3 with the former version of fritzinflux...
the data appears directly and no impact to cpu utilization and temperature...
it's looks like, that this is maybe related to the "disordered" database structure or the lua values. Is it possible for me, to switch off the lua services? are the lua values responsible for the disordered database structure or is this because of influxdb v2?
from fritzinfluxdb.
Setup the latest version from Git and comment out L96.
I got following message...
2022-05-02 08:06:23,349 - INFO: Starting fritzinfluxdb v1.0.0-alpha2 (2022-04-11)
2022-05-02 08:06:23,351 - INFO: Done reading config files
2022-05-02 08:06:23,416 - INFO: Successfully parsed config
2022-05-02 08:06:23,436 - INFO: Database 'fritzbox' not found, trying to create it
2022-05-02 08:06:23,491 - INFO: Connection to InfluxDB 1.8.10 established and database present
2022-05-02 08:06:25,644 - INFO: Successfully established FritzBox TR-069 session
2022-05-02 08:06:26,093 - INFO: Successfully established FritzBox Lua session
2022-05-02 08:06:26,093 - INFO: Successfully connected to FritzBox '192.168.178.1' (fritz.box) Model: FRITZ!Box 7590 - FW: 7.29
2022-05-02 08:06:26,096 - INFO: Starting main loop
!! 2022-05-02 08:06:27,010 - INFO: Requested invalid action 'GetInfo' for service: WANIPConnection:1
!! 2022-05-02 08:06:27,010 - INFO: Querying action 'GetInfo' will be disabled
!! 2022-05-02 08:06:27,010 - INFO: All actions for service 'WANIPConnection:1' are unavailable. Disabling service.
I took a look inside my fritzconnection output and find following:
Service: WANIPConnection1
Action: GetInfo
Parameters:
Name direction data type
NewEnable out -> boolean
NewConnectionStatus out -> string
NewPossibleConnectionTypes out -> string
NewConnectionType out -> string
NewName out -> string
NewUptime out -> ui4
NewLastConnectionError out -> string
NewRSIPAvailable out -> boolean
NewNATEnabled out -> boolean
NewExternalIPAddress out -> string
NewDNSServers out -> string
NewMACAddress out -> string
NewConnectionTrigger out -> string
NewRouteProtocolRx out -> string
NewDNSEnabled out -> boolean
NewDNSOverrideAllowed out -> boolean
Is this is related to ":" in between?
from fritzinfluxdb.
Hi, this is a desired behaviour. WANIPConnection
only works on FritzBoxes with a cable modem. Have to add a message that this is OK if the Model uses DSL and vice versa.
from fritzinfluxdb.
oh... yes... Monday morning... I see... WANPPPConnection:1 is for my FB...
Even without the Lua Service the database is not like in the old version... Is still fragmented... Maybe this is the reason for the issue. with the old unfragmented database structure I didn't face the issue. Could it be, that because of fragmentaion, Grafana/InfluxDB need more time and system ressources to display the data?
Setup: act. Fritzinfluxdb, InfluxDB v1.8.10, Grafana 8.5.0
from fritzinfluxdb.
I wouldn't call it 'fragmented'. It is more accurate compared to the old method.
maybe this page has some hints: https://docs.influxdata.com/influxdb/v2.2/write-data/best-practices/
I could try to reduce the precision to "seconds" instead of "milliseconds"
from fritzinfluxdb.
okay... understand... but I'm running v1.8.10 not v2.2
I could try to reduce the precision to "seconds" instead of "milliseconds"
if it's not to much for you... I'm here to test it...
from fritzinfluxdb.
It would reduce the points of data a bit. And it would be an easy change.
Ultimately I could reimplement the old behavior and could compare it as well.
from fritzinfluxdb.
Ultimately I could reimplement the old behavior and could compare it as well.
this means, that the database structure would be se same as in the former version? If you don't mind, let us start with the first of your suggestions ;o) maybe this solves the issue... Thanks a lot for your work, it's a great project...
from fritzinfluxdb.
I change the precision to "s" in the branch. Can you pull the the branch again and give it a try?
Thank you
from fritzinfluxdb.
Today the dashboard get a little bit slower and the CPU Utilization is around 35% and Temperature increase aboz 10°C as log as I display the dashboard.
from fritzinfluxdb.
Hi Ricardo
Unfortunately the issue is not solved by changing from ms to s....
from fritzinfluxdb.
Can you check how the data looks like in the database now?
from fritzinfluxdb.
of course... looks like before...
from fritzinfluxdb.
when you look at the time
column you can see that the precision is now down to seconds.
Your view results probably from the different tags attached to each measurement.
And if you disable the lua_connection now and leave it running?
from fritzinfluxdb.
still some spaces
from fritzinfluxdb.
hi @bb-Ricardo
checked today again... without Lua I have no temp and utilization issue...
from fritzinfluxdb.
Hi,
Yes, that is correct. Just wanted to see if it improves the situation. Temp and cpu values are only available via LUa connection.
from fritzinfluxdb.
sorry... yes, what I want to say is, as log as I didn't collect Lua values, I didn't face the temperature and utilization issue, what I reported here. so, if I activate the Lua collection, I have the high temperature and utilization issue again.
from fritzinfluxdb.
Ahh, you're talking about your Raspi? I thought you ment your FritzBox.
Would be interesting to know which query causes the issue. Maybe I did something wrong creating the panels.
from fritzinfluxdb.
Yes... talking about Raspi... It happens already with this open row...
and there are only three lua values... cpu temp, utilization and log **... and I didn't belive, that there is something wrong in this panels... ;o)
But I can give it a tray... I switch on the lua collection and than run the dashboard without the lua panels. then we will see, what happens...
** and RAM and aktiv devices
Edit: Now running with lua collection and dashboard without lua panels. now have to wait... will come back to you...
from fritzinfluxdb.
Thank you for all the time and testing, this is highly valued.
from fritzinfluxdb.
morning Ricardo
It's in my own interest... And you did already a great job by publishing this project....
I took a look... The increase of temperature an utilization on the Raspi is not related to lua panels. Even without I have the issue...
from fritzinfluxdb.
It's looks like, that it is a Lua based issue. Maybe because the database have a lot more rows per timestamp instead of the old version, were it have only one row per timestamp.
from fritzinfluxdb.
Mmmmh, this is all not really good as it is a show stopper for the whole rewrite. I pretty much stopped now. Because if this continues then it is nice to have the values, you just can't look at them 🤷.
I'm out of ideas in how to enhance the data collection to not cause this issue.
from fritzinfluxdb.
I think, you can't do anything. Imho it's related to the fact, that it need one line for every data like energy consumption and active/passive hosts.
your project is absolutely great. so, my next step will be to comment out that lua data, which need more then one line and keep something like CPU temperature and utilization, number of active user and maybe the eventlog and then check again...
btw: today I took a look into my fritzinfluxdb log and found following:
2022-05-25 11:19:37,350 - ERROR: Failed to write to InfluxDB '192.168.178.34': 400: {"error":"partial write: max-values-per-tag limit exceeded (100003/100000): measurement=\"fritzbox\" tag=\"ipv4_last_used\" value=\"1653475846\" dropped=15"}
2022-05-25 11:21:37,738 - ERROR: Failed to write to InfluxDB '192.168.178.34': 400: {"error":"partial write: max-values-per-tag limit exceeded (100003/100000): measurement=\"fritzbox\" tag=\"ipv4_last_used\" value=\"1653475846\" dropped=15"}
2022-05-25 11:23:37,842 - ERROR: Failed to write to InfluxDB '192.168.178.34': 400: {"error":"partial write: max-values-per-tag limit exceeded (100003/100000): measurement=\"fritzbox\" tag=\"ipv4_last_used\" value=\"1653475846\" dropped=15"}
2022-05-25 11:25:39,628 - ERROR: Failed to write to InfluxDB '192.168.178.34': 400: {"error":"partial write: max-values-per-tag limit exceeded (100003/100000): measurement=\"fritzbox\" tag=\"ipv4_last_used\" value=\"1653475846\" dropped=15"}
do you know the meaning of it?
from fritzinfluxdb.
hey @ronaldn1969,
Thank you for testing and your encouraging words.
I might have found the issue with the high cpu utilisation. Can you pull the latest commits from the rewrite branch? I also added an updated dashboard with more information.
Thank you.
from fritzinfluxdb.
version 1.0.0 has been released
from fritzinfluxdb.
Hey @bb-Ricardo
Faster as thought. Build a docker container with version 1.0.0 from rewrite-adding-async and launched on my RasPi... Will now collected a few days and will come back and give you an update.
Edit: Checked the new Dashboard... Thanks for the rebuild an adding VPN/WLAN and modified Network Hosts.
from fritzinfluxdb.
In the reamde is a link to an prebuilt container on hub.docker.com.
Edit: but no Arm images, true. Let's see if I can build some as well.
from fritzinfluxdb.
Hi @bb-Ricardo
Thanks for the hint, I saw it later in the readme. But no problem to build it myself...
from fritzinfluxdb.
Now also multi arch linux
from fritzinfluxdb.
hi @bb-Ricardo, back from holiday and checking the cpu utilization and temperature from my pi after collecting 15 days data. both in normal condition and displaying of data is fast, when I switch between the sections or going back to dashboard.
looks like your adjustments were successful. thanks for your efforts and further development of the project.
from fritzinfluxdb.
These are great news. Thank you for all your help and testing efforts. This was very valuable.
from fritzinfluxdb.
Related Issues (20)
- FritzBox Cable - Channel Information / Modulation HOT 9
- Update Description HOT 2
- Unable to perform request to 'http://192.168.178.1/data.lua' HOT 6
- FritzInfluxDB does not run as systemd service, only as a script HOT 5
- DeprecationWarning: There is no current event loop HOT 3
- Get more DSL diag stats HOT 6
- Data from LTE HOT 7
- Site to Site VPNs in VPN overview HOT 5
- Readme contains dead link HOT 3
- Unknown FritzBox link type 'Ethernet' HOT 4
- Unable to convert FritzBox Lua value HOT 28
- FritzOS version 7.57 not compatible HOT 4
- Dashboard is loading slow with big databases HOT 3
- Influxdb 3.0 support HOT 1
- Permit connection to Influxdb without user and password HOT 2
- INFO: Service 'DSL Info not applicable for this FritzBox Model Link type 'Cable' HOT 4
- Getting fritzinfluxdb to work with InfluxDBv2 HOT 7
- 2 minor issues found that need to be fixed! HOT 10
- Can't run the script HOT 5
- Update for FW7.50 HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from fritzinfluxdb.