woodcoder / circa Goto Github PK

The current code is a bit 'smelly' being, as it is, in one main function.

It needs to be refactored into functions.

If the system running ca goes to sleep (e.g. a laptop) during the call to sleep() the command waits for too long and misses its window.

This appears to be because the API doesn't account for time when the system is suspended.

The ca code currently has the region hardwired as eastus. This needs to be changeable!

This could be done:

• as a default for the server in a config file (see #2)
• as a command line option --location

I think that it probably needs to be supported as a command line option, as it might well need to be supplied with an additional region.

e.g. it's perfectly feasible that a user might want to wait for carbon intensity to be low in two places (e.g. where a database server and report server are located in different places).

Along with the config file(s) and the command line parameters, supplying default locations and API urls by environment variable would probably be useful in some scenarios.

The existing ca code assumes a default run-time window of 30 minutes.

Folk are likely to want to specify different expected run times (shorter, longer) depending on the job.

Which could be done with something like a --window parameter.

The code uses the forecast API, which currently responds in a granularity of 5 minutes and always starts in the future.

This means that the earliest 'optimal' result is always in the next five minute block - which could cause the command to sleep for 4m59s if called at, for example 14:31:01.

Either it could have a minimum sleep of 5 minutes, or possibly ask for the data for 'now' as well?

All good unix commands have manual pages.

A single location can be specified for finding the lowest carbon intensity.

However, some tasks will involve multiple locations (e.g. database or webserver in a different region) -- therefore it would be good to be able to specify multiple locations on the command line.

Either replacing any default location, or in addition to the default.

This will involve working out when the best overlapping time would be - and might also need some sort of percentage weighting to be specified too, to take into account asymmetric processing.

Running the example powershell script fails with:
Cannot find type [Windows.Data.Xml.Dom.XmlDocument]: verify that the assembly containing this type is loaded

Looks like it's probably to do with this:
https://stackoverflow.com/a/73834316/3168602

As with any project aiming to optimise anything, it would be great to sanity check that this project actually helps save carbon.

One possible way to check this might be to setup two small servers and run a head to head test - one using ca and cron and one just using cron.

I don't quite know how you'd instrument or measure this but it would need to do something like:

1. measure the amount and timestamp of energy used on both servers
2. cross-check the historic actual carbon intensity data from the carbon aware SDK

I guess (1) could be done with a simple 'how long did the job take' metric - but this might not be very accurate.

Maybe using a tool like Scaphandre might work?

The goal is to show that the, hopefully small, overhead added by ca is actually helping the overall goal of reducing emissions on a day by day basis!

Needs to be checked on a linux machine and the appropriate debian packages (or similar) for each of the dependencies added to the README?

Arch Linux (and friends) use the pacman package manager, that has different dependency names.

The end time of the timeframe is always relative to when the command is run. It would be nice to allow a fixed end time of 09:00 in localtime or 18:00Z in UTC to be given.

I was keen for circa to use as few dependencies as possible, with the goal of making it as lightweight as the coreutils programs it aspires to be like.

Linking directly to the Carbon Aware SDK directly (which involves bringing in the whole of .NET core) felt quite counter to that design goal, so I started with accessing the API directly using libcurl.

This means that if you wanted to talk to a local Carbon Aware SDK instance, you'd have to set it up running on a webserver on localhost. Presumably that's not too bad given that it can be run in a docker container.

However, it might be nice if the SDK was installed on the local machine, if circa could exec the ./CarbonAware.CLI command directly.

Currently the URL is hardwired to the hackathon's hosted version of the URL (redirected from https://grnsft.org/hack22/api).

This needs to be configurable:

• ideally at a server-wide level (/etc config file?)
• or possibly a user level (~/.circa)
• and/or as a command line switch (--api-host)

The end time for the last point as which the command can be run is defined as a number of hours between 1 and 24 from 'now'.

It would be nicer if you could specify 90m or 3d or even 09:00 as the target end time.

Some method of getting the system to record or log statistics about the commands it's running.

This could be something like time taken to run, energy usage, machine load, how much the task was time shifted by.

Ideally, these things could be used to learn more about a command (e.g. average or variation in runtime, its effect on the load of the server) - especially if run repeatedly from a task scheduler.

This probably would mean something like forking the process to run the new command, and waiting until it is finished to fetch and log this sort of data.

See:
https://github.com/stedolan/jq/wiki/C-API:-jq-program-invocation

I am not sure that the code will build under windows - particularly:
https://github.com/woodcoder/circa/blob/main/circa.c#L190

The autoconf script creates a config.h with a bunch of #defines so it feels like this should be easy enough to make work with a few #ifdefs.