stac-extensions / datacube Goto Github PK

Storage formats like Zarr and NetCDF allow variables to be chunked along one or more dimensions (zarr docs).

Would this extension be an appropriate place to capture that information? Building on #6:

{
  "cube:variables": {
    "prcp": {
      "type": "data",
      "description": "...",
      "extent": [0, null],
      "unit": "mm",
      "dimensions": ["time", "y", "x"],
      // new chunking information below //
      "shape": [365, 1000, 1000],
      "chunks": [1, 100, 100]
    }
}

The expectation is that if chunks is specified, it's an array of integers equal to the number of dimensions. The integers indicate the chunk size. Zarr, at least, doesn't allow non-uniform chunks, so a simple list of integers is used. If non-uniform chunking is desired, you would need a list of list of integers.

This overlaps somewhat with proj:shape, but I'm not really sure how extensions interact with each other.

You can specify "values" (i.e. dimension labels) in each dimension. The spec describes it in various ways, but always includes

"a set of all potential values"

Recently, we realized this may imply to people that the values are unordered: Open-EO/openeo-python-client#277 (comment)

This is unintentional, at least from my POV as the original author of this extension. This should probably be something like:

an ordered list of all values

The spec says an AdditionalDimension object can have type 'temporal'. It's unclear to me how one would look at the JSON of a Temporal or Additional dimension with type 'temporal' and determine which one it is. Are client libraries checking if the values are ISO 8601 compliant and making the determination off of that?

Origin: radiantearth/stac-spec#713

Two things came up recently that could be integrated into the data cube extension:

1. Add variables in addition to dimensions. Some data cubes expose variables, some don't. We don't need this for openEO (yet?), but Google Earth Engine @simonff would probably use them. I'm also looking at netCDF and other formats, which afaik support variables as addition to dimensions. Maybe there's space for alignment also with the ESM collection spec "fork" from @rabernat.
2. For dimensions it might be useful to specify the number of cells (see openEO UDF discussions).

Allow a way to provide information about the data cube values, right now this extension is focused on dimensions, variables and labels.
Should probably be aligned with raster:bands. (We can't use raster:bands as not all datacubes may have bands.)

Potential fields:

• Data type
• Statistics
• ...

I had a discussion with @gilbertocamara on the topic. In the Brazil Data Cube we work with three dimensions + the eo:bands. We used this document as a basis, at the time we created the STAC service.

What do you think about including the datatype, missing-data, scale and chunking schema to the variable definition?

Originally posted by @gqueiroz in #6 (comment)

Related issue: #7

See OSGeo/PROJ#4088

The json schema has changed based on the scientific schema and we need to check if this extension json schema needs to change as well.

related pr:
stac-extensions/scientific#2

At ITS_LIVE we have a list of Zarr data cubes (~3k items) and we'd like to generate a static STAC catalog, preferably using this extension. I'm a bit confused on the semantics of what a data cube means at the collection level, my idea is to have 1 top level collection that describes the dataset and each of the ~3000 zarr cubes are items of the collection that use the datacube extension to describe the dimensions, variables etc. I've only seen 1 cube per collection, so I'm not sure if I'm thinking this wrong. Any advice is welcome.

Hi,

We're building a STAC catalog for CMIP6 data and had a question.

Should time_bnds, lat_bnds and lon_bnds be defined as having type data or auxiliary ? I would have put those as auxiliary, but in the daymet example time_bnds has data type.

Thanks,