spoulson / pifan Goto Github PK

Display version with --version|-v.

After shutting down all VMs and the host ESXi server for a storm, I powered everything back on. When restarting the PiFan session, I saw it was reporting both CPUs as having 54C and 56C temps on each and every poll. This did not match the temps reported in iDRAC web admin, showing temps varying in the mid to high 60's.

I checked using racadm getsensorinfo, which seemed to show the same numbers as iDRAC. Meanwhile, ipmitool is still returning 54,56 every time.

Problem with ipmitool or IPMI protocol?

Can IPMI calls be more efficient with a native implementation?

Optionally allow PiFan to run as a daemon.

Provide installation method to run as daemon under PiOS.

After each poll, print a summary of recent fan speeds and cpu temperatures. Summary to include multiple time aggregations (similar to load averages). This provides the admin with at-a-glance insight into stability of server temperatures.

Time aggregations: 1 hour, 1 day, 1 week.

Somehow I never noticed PiFan doesn't enable manual fan control at runtime. I must've always done it in a terminal at some point and then ran PiFan.

When my R720 reboots, it reverts back to default automatic fan control. Then, PiFan was not able to effectively change fan speeds.

To fix, add an additional command when change fan speed:

ipmitool -I lanplus -H SERVERS_IP_HERE -U IDRAC_USERNAME -P 'IDRAC_PASSWORD_HERE' raw 0x30 0x30 0x01 0x00

Such as: System Board Exhaust Temperature

R720 does not monitor other items, like disk or mainboard temperatures. By monitoring exhaust temperature, we can determine whether there's excess heat soak that needs to be removed with increased fan speed.

Use the same fan speed suggestion algorithm to come up with a speed to address exhaust temperature. Then, set fan speed to the maximum of each suggested speed.

I found that the Observium network monitoring system is able to extract the necessary temperature and fan speed metrics via SNMP out of the box. I should be able to find the MIB necessary to understand how to get these metrics and poll them via SNMP. This would be more streamlined as ipmitool is a special install requirement and it is slow to poll.

Print an error message if min/max fan speeds are found to exceed a threshold.

Default threshold? 25%?

CPU temps can sometimes spike momentarily. If PiFan catches a spike, it'll ramp up the fans, then immediately drop then down on the next poll. To run smoother, aggregate and average of n samples to smooth out the CPU temp sampling.

Default to 3 samples. Configurable with command line arg --samples n.

Suggested fan speed: 6%
...
FanSensor: name=Fan1, id=0x30, rpm=1800, max=23640, percent=7.6%
FanSensor: name=Fan2, id=0x31, rpm=1800, max=23640, percent=7.6%
FanSensor: name=Fan3, id=0x32, rpm=1800, max=23640, percent=7.6%
FanSensor: name=Fan4, id=0x33, rpm=1680, max=23640, percent=7.1%
FanSensor: name=Fan5, id=0x34, rpm=1800, max=23640, percent=7.6%
FanSensor: name=Fan6, id=0x35, rpm=1800, max=23640, percent=7.6%

Additionally, the iDRAC web management shows fan speed percentage matches "suggested", not FanSensor.

Root Cause:
Dell does not expose the actual percentage in ipmitool sdr get command. So, the percent values on FanSensor lines are computed from the rpm and max values. The method or algorithm for Dell's fan percentage is not yet known.

Optional argument to specify an easing number that ranges from linear to varying degrees of parabolic curve.

To be able to monitor status, the runtime status should be optionally exported into Prometheus.

Since PiFan operates as CLI command, it is necessary for it to push metrics to a Prometheus Pushgateway. This will ensure metrics are accessible to Prometheus even after the CLI command completes.

As a side effect of CLI command structure, it will currently not be possible to maintain runtime state for counters or summary metrics that aggregate data. Only gauge metrics will be delivered.

Metrics:

• Fan speeds RPM
• Suggested fan speed %
• CPU temperatures C

Enable Prometheus Pushgateway integration via CLI argument:

--prometheus-push <host>:<port>

Cron typically does not have ability to schedule less than 1 minute intervals. A workaround is to be creative with sleep:

* * * * * /usr/local/bin/pifan ...
* * * * * sleep 15 && /usr/local/bin/pifan ...
* * * * * sleep 30 && /usr/local/bin/pifan ...
* * * * * sleep 45 && /usr/local/bin/pifan ...

Instead of using --one, there should be the ability to set --interval n and a count. This way, I can schedule a single pifan cron job that would run 4 polls at 15 second intervals.

Implement a -c | --count option to specify the number of polls. Default is 0 for no limit.

Control multiple servers on each poll.

By default, PiFan will run endlessly. To support a cron-based deployment, a one-time mode will be necessary.

Implement CLI option --one to poll once and exit.

Update README.md to outline a sample cron deployment.

Reasons to deploy with cron:

• Manage multiple servers concurrently with independent intervals.
• Cron could instead launch a script of sequential pifan --one commands.