The data that is shown in the GeoServer API should have their proper License attached.

Using great expectations we should add tests for our source data so that we are notified if something changes. Also the documentation of the MaStR Source data set should be extended. The .yml files for the source description in dbt can then be transformed to OEMetadata schema.

• Rename duplicated environment variables (like pwd vs. DATA_WAREHOUSE_POSTGRES_PASSWORD)

• How to update local version of open-dapro - git pull
• How to update python packages - rebuild user code container

There seems to be an error in naming the host correctly within the dbt production environment.

When running docker-compose up, the dbt deps command is executet when the container docker_user_code is built. However after the building process, the volume for the dagter app data is mounted, overwriting the dbt package data created when running dbt deps initially.

Solution

Run dbt deps on container startup.

Source: https://pandas.pydata.org/docs/whatsnew/v2.2.0.html#upcoming-changes-in-pandas-3-0

When pandas 3.0 is introduced, there will be major changes to read_sql and to_sql. As these functions are used often in open-dapro we need to have this in mind.

Delete the following assets:

• buildings_to_datasette
• zensus, base_zensus, stg_zensus

stg_solar seems already good, however the other technologies need to be adapted accordingly.

stg_charging_points throws error

Error log:
Database Error in model stg_charging_points (models/staging/charging_points/stg_charging_points.sql)
function replace(double precision, unknown, unknown) does not exist
LINE 33: ...T(REPLACE("Längengrad", ',', '.') AS float), CAST(REPLACE("B...
^
HINT: No function matches the given name and argument types. You might need to add explicit type casts.
compiled Code at target/run/energy_dbt/models/staging/charging_points/stg_charging_points.sql

mastr__temporal_expansion throws error

Database Error in model mastr__temporal_expansion (models/marts/marktstammdatenregister/mastr__temporal_expansion.sql)
column "power" does not exist
LINE 36: power,

Several sensitive information (like passwords etc) should also be configurable with the .env file when Dagster is run using docker-compose.

• Make path of volumes an .env variable

• Add a changelog
• Clean repository, delete unused code

Do this as a new dbt model

MaStR units have a column saying if the plant is still active. We should filter our staging pipeline depending on the fact if units are still active.

Decide which functions can be tested, and write according tests.

Right now, some services are hosted at 0.0.0.0 using the docker compose setup. This should be changed so that they are only accessible over 127.0.0.1

A better setup would be to have nginx as a proxy and redirect to 127.0.0.1:PORTNUMBER

/dagster could redirect to the dagster UI
/geoserver could redirect to the GeoServer API

Probably the docker setup can also be run by using docker compose command (which is installed together with docker). By that we can get rid of the additional installation of the standalone docker-compose command.

It would be great if someone could document oepn-dapro in our compendium. This will help to keep an overview of all our tools, and by providing a short description, users can better understand what this tool does in the context of the OE family.

The compendium is a simple mkdocs page hosted by github pages. It is maintained in this repo. : https://github.com/OpenEnergyPlatform/organisation
Content can be added by simply writing Markdown.
We use https://squidfunk.github.io/mkdocs-material/

I assume this tool could be documented on this site? What do you think?

https://github.com/OpenEnergyPlatform/organisation/blob/production/compendium/family_members/open-process-integration.md

The rendered website looks like this:
https://openenergyplatform.github.io/organisation/family_members/open-process-integration/

If you don't want to go to the trouble of adding content to the compendium, it would be nice if you could provide the descriptive text in this issue so I can add it later.

• Link to new documentation page
• Delete detailed installation instructions
• Add logo of fortiss and etwin project