Figure out relationship to dataportals.org about dataportals-registry HOT 5 OPEN

@augusto-herrmann The difference, I think, is how projects collect the data. My goal is to create something similar to re3data, but not especially with advanced web UI since this registry is a sub-project of the open data search engine.

I think about it as a roadmap:

1. Registry of open data portals (we are here)
2. Raw indexes of collected metadata
3. Search index combined from raw indexes with different data schemas
4. Search engine over search index
5. Data archival and backup of the indexed datasets.

I can't dedicate much time to the web interface of the data catalog. To me, it's a data product, not a website. If I take care of Dataportals.org, it will be a complete project re-launch. I think it should be more about data quality monitoring and not just a catalog. But I am not sure that there are many resources on it.

So I think that we could start by using data from this registry to fuel dataportals.org.

from dataportals-registry.

Planned to add dataportals.org identifier to each data catalog. Will start catalogs mapping soon

from dataportals-registry.

Thanks for explaining your roadmap with this project.

Data quality monitoring is much more challenging than just maintaining a data catalog, which is already hard enough with the current (very small) time resources we have. That said, I think it makes total sense to use the registry of open data portals here to fuel dataportals.org.

from dataportals-registry.

@augusto-herrmann I've first attempted to map Dataportals.org records to the Common Data Index registry. Only 131 records were successfully mapped.
mapped_tsv.zip

There are a few reasons.

• a lot of records in Dataportals.org are actually not data portals but other open data initiatives, educative or organizational websites
• too many outdated records in Dataportals.org. Some websites are unavailable. For example, data.pm.go.th and e-data.gov.ua are unavailable for now.

I will keep looking deeper and start to integrate identifiers.

from dataportals-registry.

@augusto-herrmann Updated registry data and mapped identifiers. Here is an example: Australia data portal https://registry.commondata.io/catalog/cdi00000098 and https://registry.commondata.io/catalog/cdi00000098.json as JSON.

from dataportals-registry.