icesat-2hackweek / cloudmask Goto Github PK
View Code? Open in Web Editor NEWFetch, classify, and label ICESat-2 data
License: BSD 2-Clause "Simplified" License
Fetch, classify, and label ICESat-2 data
License: BSD 2-Clause "Simplified" License
The function read_atl06_fromfile
in utils_atl06
access an h5 file and returns the data frames. However, some of the h5 files do not have all the features we request included, leading to the following problem:
I am still not sure if this is a problem of the data itself, in icepyx, or in the way I retrieve ATL06 data. This may also be related to this another issue, since I am not specifying the name of the granules I want to access.
For now, I will add a try
command in read_atl06_fromfile
to avoid this problem, but probably this is no the best solution.
The name of some features in the ATL06 dictionary have been changed in order to have a more descriptive name or sometimes a shorter name. This is in utils_atl06.py
. For example:
Should the names of the variables in the dataframes coincide with the ones in the ATL06 data dictonary?
The current assimilation of ICESat-2 with VIIRS is done via the Ball Tree method. The function 'associate' in 'utils_viirs.py' just calls the method Ball Tree on Sklearn:
In the particular case where (at least) one of the datasets consists on a grid of points (eg, satellite images) the assimilation can be done in a more efficient way, exploiting the structure of the grid.
This algorithm probably is useful in more general application and the method could be an important contribution for other teams to.
The following two functions can be used to compute a time object given the seconds in GPS, that is, the total seconds since 1980-06-01 00:00. The first function uses the package astropy and the second datetime.
However, we can see that they result in different times from the same timestamp (in this case, 0 seconds should match with 1980-06-01 00:00):
The haversine distance from sklearn we use to compute the distance between in 'CloudMask.ipynb' different grids takes pairs of the form [latitude, longitude]. However, in the code the grids are build as lists of the form [longitude, latitude].
Also, when we check the filter of spatial distmatch < tolerance, this tolerance is in the wrong units.
Parallelize the for
loops in VIIRS_Bash.ipynb
and in the assimilation between climate and ICESat-2 data.
As it was already reported in one of the issues in icepyx, we received the following error after we try to select the variables in ATL06 we want to retrieve:
In utils_atl06
we found the code used to retrieve the ATL06 data and create the associated Pandas data frames. If now we comment the lines where we select the name of the variables in the ATL06 product we want to sublet, then we don't received that error anymore:
Furthermore, it seems that without the subletting, we retrieve all the requiered information in the ATL06 product. However, we would like to ask for specific variables, in order no to download features that we are not going to use.
In the last version of Icepyx, the way to obtain the avail granules has changed. Now, we have to do
import icepyx as ipx
region_a = ipx.Query(short_name, spatial_extent, date_range)
See an example here. We need to change the function that retrieves the ATL data to this and also the notebooks that import icepyx.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.