The current IO data structure is based on COMPETES. Change to the Tulipa format.

Unit tests currently run on the large EU dataset and take a long time. Make a smaller dataset. It should cover all of the corner cases.

On large datasets the calculations can take a long time. Add progress bars to make it more clear that progress is being made

Instead of working with a DataFrame, operate on the DuckDB table directly.
Probably the split function can do that, and probably the output as well.

When removing IO from this package, the input validation is not being done.
A similar issue on TEM: TulipaEnergy/TulipaEnergyModel.jl#636.
We should coordinate.

See what needs to be exported to TulipaIO, if anything, and use that instead.
Remove reading functions from here and update tests and docs using TulipaIO.
Use the DuckDB connection to read necessary data.

Implement k-means clustering in. a way that allows to use other ways of finding the representatives and their weights.

• #2
• #3
• #4
• #5

Dirac weights can also be found from another type of weights by using a projection. Implement a projection operator onto a discrete space of representatives.

cf. TulipaEnergy/TulipaEnergyModel.jl#676

Update the relevant names and files to match TEM. I identified so far:

• asset in the table should probably be profile_name (TulipaEnergy/TulipaEnergyModel.jl#660)
• time_step should be timestep
• EU/assets-profiles.csv to EU/profiles.csv

Gnomonic clustering should in theory work better with conical (both bounded and unbounded) and convex weights. It prioritizes extreme periods in the convex/conical hull of the dataset

This function is not covered by the tests. Implement a test for it.

The rep_periods_data is a table in TEM containing

• rep_period: The representative period (Hopefully will change to rep_period soon: TulipaEnergy/TulipaEnergyModel.jl#661
• num_timesteps: Number of time steps for this rp. (= period_duration).
• resolution: Time scaling of this rp (or length in hours of a timestep).

This can be created in this package, or even required as input.

When the number of representative periods is not valid (more than the number of periods, or fewer than 1), currently the clustering method throws an error. It should throw a custom-made error

This issue is used to trigger TagBot; feel free to unsubscribe.

If you haven't already, you should update your TagBot.yml to include issue comment triggers.
Please see this post on Discourse for instructions and more details.

If you'd like for me to do this for you, comment TagBot fix on this issue.
I'll open a PR within a few hours, please be patient!

The docs currently just show how to contribute, etc.
There should be a basic tutorial as well as how-tos (if necessary).

Read this for guidance on how to write tutorials and how-tos and structure docs in general. :)
https://diataxis.fr/

We should have a function that takes a Dictionary like this one, for example:
-> For periods of length 24 create 5 representatives
-> For periods of length 168 create 2 representatives

To create representatives of different length

These common docs we just want to keep/update in 1 place.
Say something like "We follow the same guidelines as our main repo, TulipaEnergyModel(url)."

What's the package do? Add it to the landing page (README) and the Home section of Docs.