Git Product home page Git Product logo

Comments (4)

jgrss avatar jgrss commented on September 28, 2024

Hey @mmann1123 I think I follow your setup, except for the shape of your data. What is the shape of array[self.index_to_write]? Is array 4d (time x bands x height x width) and then you slice to get (1 x bands (1?) x height x width), and then squeeze to (time x height x width)?

You can control a bit of the output profile in your user function. For example, the output band count is set by the TimeModule.count (by default, the output band count is 1). And that gets passed here to the rasterio profile.

If you want a multi-band output then you can specify that in your user function. I don't think you need index_to_write, so I replaced it with count below in your __init__ method and in the return. But I think that assumes you are processing multi-temporal, single band data. See my comment at the bottom.

class interpolate_nan(gw.TimeModule):
    def __init__(self, missing_value=None, interp_type="linear", count=1):
        super(interpolate_nan, self).__init__()
        self.missing_value = missing_value
        self.interp_type = interp_type
        # Overrides the default output band count
        self.count = count

    def calculate(self, array):
        # check if missing_value is not None and not np.nan
        if self.missing_value is not None:
            if not np.isnan(self.missing_value):
                array = jnp.where(array == self.missing_value, np.NaN, array)
            if self.interp_type == "linear":
                array = np.apply_along_axis(_interpolate_nans_linear, axis=0, arr=array)
        # Return the interpolated array (3d -> time/bands x height x width)
        # If the array is (time x 1 x height x width) then squeeze to 3d
        return array.squeeze()

Then, you should be able to use it by:

with gw.series(
    files,
    nodata=9999,
) as src:
    src.apply(
        func=interpolate_nan(
            missing_value=0, 
            # not sure if your output length matches your input file length
            # whatever your case is, this is where you define the output band count
            count=len(src.filenames)
        ),
        outfile=f"/home/mmann1123/Downloads/test.tif",
        num_workers=5,
        # Note that this is the band, or bands, to read
        bands=1,
    )

Note that you can only write a 3d array. Therefore, you can either write a single interpolated date and multiple bands, or all the dates for a single band.

from geowombat.

jgrss avatar jgrss commented on September 28, 2024

Did you also try xarray's interpolate method?

with gw.open(files, chunks={'time': -1}) as src:
    interp = (
        src.interpolate_na(dim='time', method='linear', fill_value='extrapolate')
        .bfill(dim='time')
        .ffill(dim='time')
        # Interpolate to new grid
        #.interp(time=smooth_range, method='slinear')
    )

from geowombat.

mmann1123 avatar mmann1123 commented on September 28, 2024

I had been using xarrays interpolate_na but now I have files that are too big to bring into memory, and I am not sure how to apply it to chunks and write it out.

from geowombat.

mmann1123 avatar mmann1123 commented on September 28, 2024

Ok you are a life saver as always. Yes I was applying the interpolation to a single band for multiple periods. Your apply example worked! Thanks again

from geowombat.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.