Currently, LB does not have notion of admin users to pass through the requests without additional checks. We need to add admin users to LB so that admins can look into dashboards of different users. We should do same thing as CEEMS API, sync admin users from Grafana.

Move all this logic into a middleware and add it to frontend handler.

The idea is that the load balancer should be able to use DB or make a request to CEEMS api server to check owner ship of the unit.

Imagine a situation where user associated with a project at some point of time and submitted a compute unit during that time. If the user loses membership on that project, they should be able to look into units of that project anymore.

However, the way we check for user project association, that user will have access to that project (who they are not members anymore) until the DB entry has been purged. This should be improved.

We can create a table for user project association and sync with resource manager regularly and we check the association based on that table.

Cray has its own IPMI implementation called capmc. We can support this in IPMI collector to get node power stats. An example output of command is as follows:

> capmc get_system_power -w 600
{
"start_time":"2015-04-01 17:02:10",
"avg":5942,
"min":5748,
"max":6132,
"window_len":600,
"e":0,
"err_msg":""
}

Ref: https://cug.org/proceedings/cug2015_proceedings/includes/files/pap132.pdf
API Docs: https://github.com/Cray-HPE/hms-capmc/blob/release/csm-1.0/api/swagger.yaml

I feel it would help advertise the project.
The corresponding tool for OAR does it: https://github.com/oar-team/colmet?tab=readme-ov-file#introduction

Users/operators should be able to choose which country's emission factor they would like to use when making query.

REST API versions seem to be diverged between 0.0.37 and latest 0.0.41. Try to support v0.0.37 and v0.0.40 that can potentially support other versions too.

To indicate whether hyperthreading is on or off on the host

This will help to identify stale projects to clean up the DB

We can improve query batching by adjusting batch size dynamically based on scrape interval, duration of query and TSDB's query-max.-samples. This ensures that we operate in a safe zone and avoid OOM errors on TSDB side and fetching metrics all the time on CEEMS side.