Ignore missing dims when mapping over tree about datatree HOT 5 OPEN

Continuing from related discussion in https://discourse.pangeo.io/t/xarray-and-collections-of-forecasts/3054/6

It would also be helpful to have it on .sel for my usage.

I haven't dug around in the guts of datatree enough to understand how it's mapping functions to each group, but would it be possible to add a missing_dims kwarg at the mapping level? Then use it to decide if to catch KeyErrors from the underlying dataset methods or not?

If I'm understanding things right, Datatree uses a mixin (MappedDatasetMethodsMixin) to manage mapping methods to datasets. Could map_over_subtree pick missing_dims off the kwargs?

from datatree.

Hi @abkfenris !

datatree.mapping is where the guts of the mapping occurs. The mixin just steals certain methods from xarray.Dataset and wraps them with map_over_subtree. The mapping code is basically just this:

def map_over_subtree(func, dt, *args, **kwargs):
    new_tree = ...
    for node in dt.subtree
        result_ds = func(node.ds, *args, **kwargs)
        new_tree[node.path] = result_ds

but generalised to potentially map over multiple trees simultaneously (e.g. for binary operations like __add__), with error checking, and usable as a decorator.

would it be possible to add a missing_dims kwarg at the mapping level?

We could, but missing_dims wouldn't make sense for every function we might map - that's the challenge here. That's why I suggested we might want to add something to map_over_subtree that allows you to ignore any KeyError? Or another approach would be to modify .sel upstream.

from datatree.

I wonder if ignoring KeyError might be too broad and could catch more than intended (I'm thinking Dask or fsspec KeyErrors bubbling up). Might be worth exploring getting more tightly defined errors upstream.

from datatree.

from datatree.

See pydata/xarray#8949 for a much more thought-out solution to this problem

from datatree.