dewberry / blobfish Goto Github PK

• Create a new tag in repo (v0.9) to act as permanent home for the AORC ontology and SHACL files
• Rewrite AORC ontology TTL file. DON'T use code to do this, just manually write the TTL. Ultimately saves time and ensures readability
• Create SHACL shapes graph to constrain source and mirror dataset values
• Add function for acquiring spatial extent of RFC offices
• Revise AORC download process to use asynchronous verification of URLs and async streaming of zips to s3
• Add method for fetching git information to use in populating metadata
• Improve method for fetching docker information to use in populating metadata
• Add method to graph class which uploads data to CKAN instance
• Write a parser for CKAN linked data which creates RDF using AORC ontology

In transposition metadata collection, we gather s3 URI for DSS files as an output of SST modeling
Would like to either:

• Edit SST modeling (stormcloud) repo to directly produce and upload data to CKAN for serialization
• This is probably the best option as it doesn't decouple data and metadata production and allows for metadata collection during the data creation, rather than creating and then later reinspecting data

OR

• Enable DSS file inspection to verify the temporal coverage and spatial resolution of transposition datasets

• Create URIs for repeated and complex references such as RFCs and source datasets

https://github.com/Dewberry/blobfish/security/dependabot/5

Potentially revise dataset metadata parsing and serializing to create individual serializations of metadata for each dataset and record the collection of these metadata as a catalog, to better reflect structure of source data and reduce use of blank nodes

• Improve debug capabilities by adding logging
• Add docstrings to all functions and classes

• Implement SHACL validation of datasets previous to CKAN upload

Links used right now in SST extension are invalid placeholders, to be updated with valid URLs to CKAN data.

Add method for testing the results of GeoSPARQL querying against spatial attributes in AORC datasets (subclasses of dct:Location)

Run through the creation of composite dataset metadata in order to be able to link to SST data

Create script which pulls docs from s3, parses them, provides placeholder data where necessary, and serializes to JSON-LD format as specified in SST DCAT-US extension