enterprisedb / system_stats Goto Github PK

View Code? Open in Web Editor NEW

99.0 10.0 21.0 165 KB

A Postgres extension for exposing system metrics such as CPU, memory and disk information

License: Other

Makefile 1.28% C 98.72%

system_stats's People

Contributors

Stargazers

Watchers

system_stats's Issues

`make installcheck` doesn't work

Out of habit, I used make check after compiling and installing the package. That told me to run make installcheck instead, so I did that. However, I got this:

perhan: system_stats 0$ make installcheck
"Platform is: linux"
/home/alvherre/Code/pgsql-install/REL_11_STABLE/lib/pgxs/src/makefiles/../../src/test/regress/pg_regress --inputdir=./ --bindir='/home/alvherre/Code/pgsql-install/REL_11_STABLE/bin'    --dbname=contrib_regression system_stats
(using postmaster on Unix socket, port 55442)
============== dropping database "contrib_regression" ==============
DROP DATABASE
============== creating database "contrib_regression" ==============
CREATE DATABASE
ALTER DATABASE
============== running regression test queries        ==============
test system_stats                 ... /bin/sh: 1: cannot open /home/alvherre/Code/system_stats/sql/system_stats.sql: No such file

The sql/system_stats.sql file does indeed not exist and it has never existed in the repository.

All load information is misunderstood and incorrectly set.

Most of the system's load information statistics only have a time range of 1-5-15, and our repo mistakenly gave a very strange time range of 10 minutes.

The value given to 10 minutes on the Windows platform points to a variable that has not undergone any updates at all.
Now load is fixed of 0 on windows, for 10 minute value.
https://github.com/EnterpriseDB/system_stats/blob/84f50fd8b6282369fd853a9cf2feedc01dbd2847/windows/load_avg.c#L125C69-L125C69

PSutil's code repository has the same implementation, but their scope is only 1-5-15
https://github.com/giampaolo/psutil/blob/e9dabbbcae38a52a725722c6e3df0dc94c6051ab/psutil/arch/windows/wmi.c#L118

Complete Makefile

Please complete the Makefile, so that make install installs all the necessary files.

diff --git a/Makefile b/Makefile
index 03f6a8e..3725801 100644
--- a/Makefile
+++ b/Makefile
@@ -65,7 +65,7 @@ PG_LDFLAGS= -framework IOKit -framework CoreFoundation
 endif
 
 EXTENSION = system_stats
-DATA = system_stats--1.0.sql uninstall_system_stats.sql
+DATA = system_stats--1.0--2.0.sql  system_stats--1.0.sql  system_stats--2.0.sql uninstall_system_stats.sql
 PGFILEDESC = "system_stats - system statistics functions"

Incorrect extension version 2.0 after installing version 2.1

SQL Error [42710]: ERROR: role "monitor_system_stats" already exists

Hi,

first of all I want to thank you for this great extension. This is something really useful for people using the community edition of PostgreSQL.

Back to the issue:
It seems that the monitor_system_stats role is not dropped if you drop the extension itself.

Steps to reproduce:

create extension system_stats;
drop extension system_stats;
create extension system_stats;

You get the error:
SQL Error [42710]: ERROR: role "monitor_system_stats" already exists

It's not a big deal. One can simply execute a "drop role monitor_system_stats", but I think the idea is that drop extension should do this kind of "garbage collection".

But If you want to use the extension in more than one database in the same PostgreSQL cluster, then this is not possible, because you get this error as soon as you want to create the extension a second time.

So the best solutions from my point of view would be, if "Create extension" checked for an already existing role "monitor_system_stats" and skipped the creation of the role if so.

Cheers
James

Inaccurate Memory Usage Reporting Due to Not Considering MemAvailable

Environment

pgAdmin Version: 8.4
System Stats Extension Version: 2.1
OS: Ubuntu 22.04
Browser: Google Chrome

Description:

The system_stats extension seems to be reporting used memory by subtracting the 'MemFree' value from the 'MemTotal' value in /proc/meminfo. However, this does not account for the 'MemAvailable' value, which provides a more realistic figure of how much memory is truly available for new processes, as it includes cached and buffered memory that can be reallocated.

Steps to Reproduce:

Open pgAdmin's dashboard to view system statistics.
Compare the memory usage reported by pgAdmin with the output of cat /proc/meminfo in the terminal.

Expected behavior:

The memory usage reported by the system_stats extension should consider 'MemAvailable' to provide an accurate representation of how much memory is available for new processes, without causing swapping.

Current behavior:

The system_stats extension reports memory usage based on 'MemTotal' minus 'MemFree', leading to a potential misunderstanding of the actual available memory, especially in scenarios where a significant portion of the memory is used for caches and buffers but can be made available.

Possible Fix:

Modify the memory usage calculation in the system_stats extension to take 'MemAvailable' into account when computing available memory. This would align the reported memory usage with modern Linux memory reporting standards.

Additional context:

Here is a sample output from /proc/meminfo:

MemTotal: 131569360 kB
MemFree: 724984 kB
MemAvailable: 123998908 kB

In this example it says 124GB memory used and 802MB free, even tho it's actually only around 7.6GB used and around 123GB available (as seen in the output from /proc/meminfo/):

Could not find clang

Hello,

First of all - I would like to say that this is a long awaited extensions for PostgreSQL that really gives important system views for easier monitoring - so thank you for this.

I have tried to install according to instructions but I got an error - I am sure I have done something wrong in the process I just dont know what. These are the steps I have taken (CentOS 7 PG 12) - all with user "root":

yum install clang -y
git clone https://github.com/EnterpriseDB/system_stats.git
export PATH=/usr/pgsql-12/bin:$PATH
make install USE_PGXS=1

These is the error I got:

/opt/rh/llvm-toolset-7/root/usr/bin/clang -Wno-ignored-attributes -fno-strict-aliasing -fwrapv -O2  -I. -I./ -I/usr/pgsql-12/include/server -I/usr/pgsql-12/include/internal  -D_GNU_SOURCE -I/usr/include/libxml2  -I/usr/include -flto=thin -emit-llvm -c -o system_stats.bc system_stats.c
make: /opt/rh/llvm-toolset-7/root/usr/bin/clang: Command not found
make: *** [system_stats.bc] Error 127

What have I missed?

Thanks!

Redundant logic.

Is it normal to set the value of MaxClockSpeed to l2cache?
Although l2cache was set correctly afterwards.

system_stats/windows/cpu_info.c

Line 136 in 18033e7

hres = result->lpVtbl->Get(result, L"MaxClockSpeed", 0, &query_result, 0, 0);

Memory leak on windows

After my test, if function is implement from WMI, the all logic will memory leak after Get

VARIANT type need correct Clear

Read doc from Microsoft, we can get correct fixed. Release function is VariantClear.

https://learn.microsoft.com/en-us/windows/win32/api/wbemcli/nf-wbemcli-iwbemclassobject-get

(BTW: Indeed very few people running commercial databases on Windows.)

Postgres 16

Hi all,
I tried to install the extension on Unbuntu 22.04 LTS. The installation went fine, but when I try to install the extension with "CREATE EXTENSION system_stats;" I get the following error: ERROR: extension "system_stats" has no installation script nor update path for version "2.0"

Thnaks,
Paolo

PostgreSQL 16 support

Hi,

Can you please push a new release for v16 compatibility? It is due tomorrow.

Thanks!

Regards, Devrim

Could increase the statistics of CPU waiting time during SQL execution

Could add IO indicators（IO throughput of read and write） in pg_sys_cpu_memory_by_process()

Building for PG 16 not working

I am trying to build the extension for PG 16rc1.

It fails with an error.

Are you planning to update the extension any time soon?

make installcheck fails due to missing test file sql/system_stats.sql

Hi,

It looks like the normal sql/ and expected/ directories are missing.

I get this error when trying to run "make installcheck":

$ make installcheck
"Platform is: linux"
/usr/lib/postgresql/13/lib/pgxs/src/makefiles/../../src/test/regress/pg_regress --inputdir=./ --bindir='/usr/lib/postgresql/13/bin'    --dbname=contrib_regression system_stats
(using postmaster on Unix socket, default port)
============== dropping database "contrib_regression" ==============
DROP DATABASE
============== creating database "contrib_regression" ==============
CREATE DATABASE
ALTER DATABASE
============== running regression test queries        ==============
test system_stats                 ... /bin/sh: 1: cannot open /home/joel/postgresql-extensions/system_stats/sql/system_stats.sql: No such file
diff: /home/joel/postgresql-extensions/system_stats/expected/system_stats.out: No such file or directory
diff: /home/joel/postgresql-extensions/system_stats/results/system_stats.out: No such file or directory
diff command failed with status 512: diff  "/home/joel/postgresql-extensions/system_stats/expected/system_stats.out" "/home/joel/postgresql-extensions/system_stats/results/system_stats.out" > "/home/joel/postgresql-extensions/system_stats/results/system_stats.out.diff"
make: *** [/usr/lib/postgresql/13/lib/pgxs/src/makefiles/pgxs.mk:420: installcheck] Error 2

Installation problems

Hey all, I'm here because PgAdmin has a little text saying that I can get more stats if I install system_stats extension but I have no idea how to install this.

There are some small instructions on the readme file

tar -zxvf system_stats-1.0.tar.gz
cd system_stats-1.0
PATH="/usr/local/pgsql/bin:$PATH" make USE_PGXS=1
sudo PATH="/usr/local/pgsql/bin:$PATH" make install USE_PGXS=1

but I'm not sure how to use this, where is this system_stats-1.0.tar.gz file?

Am I supposed to do/build something to get it first?

I'd love it if someone can lend a helping hand. I also think it'd be a good idea to have the install instructions from scratch somewhere in the repo or in some documentations page (I apologize if it exists and I couldn't find it).

Let me know if there more in depth instructions somewhere but I couldn't find it.

'kIOMainPortDefault' is only available on macOS 12.0

Hello,

I am trying to install this on macOS 14.2 (23C64) on a 2018 15" MacBook Pro 2.6 GHz 6-Core Intel Core i7 and getting the following error:

user ~/Downloads/system_stats-2.0  $ PATH="/Applications/Postgres.app/Contents/Versions/16/bin:$PATH" make USE_PGXS=1
"Platform is: darwin"
gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Werror=unguarded-availability-new -Wendif-labels -Wmissing-format-attribute -Wcast-function-type -Wformat-security -fno-strict-aliasing -fwrapv -Wno-unused-command-line-argument -Wno-compound-token-split-by-macro  -Os -mmacosx-version-min=10.13 -arch arm64 -arch x86_64  -fvisibility=hidden -I. -I./ -I/Applications/Postgres.app/Contents/Versions/16/include/postgresql/server -I/Applications/Postgres.app/Contents/Versions/16/include/postgresql/internal -I/Applications/Postgres.app/Contents/Versions/16/share/icu -I/Applications/Postgres.app/Contents/Versions/16/include/libxml2 -I/Applications/Postgres.app/Contents/Versions/16/include -I/Applications/Postgres.app/Contents/Versions/16/include  -I/Applications/Postgres.app/Contents/Versions/16/include  -c -o darwin/io_analysis.o darwin/io_analysis.c
darwin/io_analysis.c:44:35: error: 'kIOMainPortDefault' is only available on macOS 12.0 or newer [-Werror,-Wunguarded-availability-new]
        if (IOServiceGetMatchingServices(kIOMainPortDefault,
                                         ^~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/IOKit.framework/Headers/IOKitLib.h:112:19: note: 'kIOMainPortDefault' has been marked as being introduced in macOS 12.0 here, but the deployment target is macOS 10.13.0
const mach_port_t kIOMainPortDefault
                  ^
darwin/io_analysis.c:44:35: note: enclose 'kIOMainPortDefault' in a __builtin_available check to silence this warning
        if (IOServiceGetMatchingServices(kIOMainPortDefault,
                                         ^~~~~~~~~~~~~~~~~~
1 error generated.
make: *** [darwin/io_analysis.o] Error 1

PostgreSQL 16 as an RPM YUM

Hello! I'm hoping for an RPM for RHEL9 with fresh 16.1 install. I do know how to work with make files and the like, but sense that an RPM from an official YUM repo a less risky "blessed" path. It would also be helpful if this repo's README would provide the name of the RPM. For the record, sudo yum info *system_stats* shows me:

Updating Subscription Management repositories.
Last metadata expiration check: 3:44:50 ago on Wed 13 Dec 2023 01:03:16 PM EST.
Available Packages
Name         : system_stats_15
Version      : 1.0
Release      : 2.rhel9
Architecture : x86_64
Size         : 76 k
Source       : system_stats_15-1.0-2.rhel9.src.rpm
Repository   : pgdg15
Summary      : A Postgres extension for exposing system metrics such as CPU, memory and disk information
URL          : https://github.com/EnterpriseDB/system_stats
License      : PostgreSQL
Description  : system_stats is a Postgres extension that provides functions to access system
             : level statistics that can be used for monitoring. It supports Linux, macOS and
             : Windows.
             : 
             : Note that not all values are relevant on all operating systems. In such cases
             : NULL is returned for affected values.

I thank you.

No such file or directory pgxs.mk

When I try to make this on a Rocky Linux 9.3 using postgres-16, I get

# PATH="/usr/pgsql-16/bin:$PATH" make USE_PGXS=1
"Platform is: linux"
Makefile:85: /usr/pgsql-16/lib/pgxs/src/makefiles/pgxs.mk: No such file or directory
make: *** No rule to make target '/usr/pgsql-16/lib/pgxs/src/makefiles/pgxs.mk'.  Stop.