Git Product home page Git Product logo

pygedi's Introduction

https://spotthestation.nasa.gov/

Introduction

The new Global Ecosystem Dynamics Investigation GEDI launched on December 5th, 2018 is operating onboard the International Space Station ISS producing a high-resolution laser collecting 3D data around Earth on forest canopy height, canopy vertical structure, and surface elevation. As it is collecting daily data, a stable and fast platform is essential. For this reason, the library pyGEDI is developed in Python as it can utilize multiple CPUs, GPUs, and is supported by C and GDAL.

pyGEDI provides a high performance, lower cognitive load, and cleaner and more transparent code for data extraction, analysis, processing, and visualization of GEDI's products.

Package Overview

pyGEDI has multiple functions for visualization, processing, analysis, and data extraction all in one package. pyGEDI package contains the following functions:

  • Connection to NASA's server.
  • Downloading GEDI data.
  • Clipping of your specific area.
  • Generate multiple files like .csv, .shp, and .tif for GEDI's products.
  • Visualization of waveforms with canopy height and profile metrics.
  • Process .h5 files.
  • Generate histograms, raster’s of different GEDI metrics.
  • Graph in 3D cloud points.
  • Demonstrative products (correlation GEDI and Airborne Laser Scanning).

https://github.com/EduinHSERNA/pyGEDI/blob/master/blog/graphics.jpeg

Dependencies

  • numpy
  • gdal
  • h5py
  • pandas
  • matplotlib

Installation

  • pip install pyGEDI

Loading pyGEDI Package

from pyGEDI import *

Session NASA

Connect with NASA's server Register for an Earthdata Login Profile (if you do not have an account register for a free account by providing your email).

username='write your user name'
password='write your password'

session=sessionNASA(username,password)

Set Parameters

Define the coordinates of your specific area of interest. The next line will exhibit an example using coordinates in Colombia.

ul_lat= 2.96845  
ul_lon=-73.32586
lr_lat=-1.26845
lr_lon=-70.23869  

bbox=[ul_lat,ul_lon,lr_lat,lr_lon]

Download GEDI Data

Now that the box is defined, download all GEDI trajectories that have flown over the area(s) of interest. Currently, there are only three products available with 25m resolution.

This includes:

  • Level 1 - Geolocated Waveforms
  • Level 2- Footprint Level Canopy Height
  • Level 2- Profile Metrics

For more information check GEDI Finder

Note: GEDI Trajectories contain heavy amounts of data. It is recommended to have good internet for it to download in a more timely manner. Depending on your area of interest and trajectories this may take a couple hours illustrated by the download bar.

product_1B='GEDI01_B'
product_2A='GEDI02_A'
product_2B='GEDI02_B'

version='001'

outdir_1B='data/'+product_1B+'.'+version+'/'
outdir_2A='data/'+product_2A+'.'+version+'/'
outdir_2B='data/'+product_2B+'.'+version+'/'

gediDownload(outdir_1B,product_1B,version,bbox,session)
gediDownload(outdir_2A,product_2A,version,bbox,session)
gediDownload(outdir_2B,product_2B,version,bbox,session)

Once the download starts it will show a download bar for each file

Created the subdirectory   
data/GEDI01_B.001/2019.08.05/
GEDI01_B_2019217151359_O03661_T02309_02_003_01.h5 | 7.405GB | 24.19%   
[███████████████████......................................................]

Research Products for Analysis, Processing, and Visualization

Access the following notebooks for each respective product.

Notebook:

Pendings for v0.3:

  • Time series.
  • Export data to Google Earth Engine.
  • Analysis for LEVEL 3 gridded canopy height metrics and variability.
  • Analysis for LEVEL 4A and 4B Footprint and gridded aboveground carbon estimates.

References

Acknowledgements

We would like to thank The University of Maryland and NASA's Goddard Space Flight Center for developing GEDI's mission and for providing free and open data.

Reporting Issues

Please report any issue regarding the pyGEDI package to:

  • 137eduin[at]gmail[dot]com
  • andreshs[at]umd[dot]edu

pyGEDI blog forthcoming.

Call for Contributions

pyGEDI appreciates help from a wide range of different backgrounds. Small improvements or fixes are always appreciated. Kindly report any issues with labeling or processing. If you are considering larger contributions outside the traditional coding work, please contact us through the mailing list.

Other functions

pyGEDI has several functions and is a continuously improved package. If it does not exist in pyGEDI please ask us for a new feature, and depending on the feasibility we will be glad to implement your requested feature.

Citing pyGEDI

Eduin H.SERNA; Andres Hernandez-Serna. pyGEDI: NASA's Global Ecosystem Dynamics Investigation (GEDI) mission data extraction, analysis, processing and visualization. version 0.2, April. 5th 2020, available at: https://pypi.org/project/pyGEDI/

Disclaimer

pyGEDI package has not been developed by the GEDI team. It comes with no guarantee, expressed or implied, and the authors hold no responsibility for its use or reliability of its outputs.

pygedi's People

Contributors

andreshs137 avatar eduinhserna avatar tim-devereux avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

pygedi's Issues

ground height calculation

In the notebook GEDI_Waveforms.ipynb why the heights for rh level are calculated by adding abs(min(h5_2A[beam]['rh'][shot_number_id])). Isn't the actual ground height at elev_lowestmode and not rh0? Similarly, canopy height rh100 is elevated by abs(min(h5_2A[beam]['rh'][shot_number_id])).

heights=h5_2A[beam]['rh'][shot_number_id]+abs(min(h5_2A[beam]['rh'][shot_number_id]))

All h5 files downloaded are 1kb and cannot be opened

I tried to download GEDI data from NE India. I followed the instructions from your README, but all the files are 1kb and do not open with h5py. Here's my code. Removed my username and password.

from pyGEDI import *

#removed my username and password
#username='
#password='

session=sessionNASA(username,password)

ul_lat= 23.4
ul_lon=96
lr_lat=27.3
lr_lon=90.3

bbox=[ul_lat,ul_lon,lr_lat,lr_lon]

product_1B='GEDI01_B'
product_2A='GEDI02_A'
product_2B='GEDI02_B'

version='001'

outdir_1B='data/'+product_1B+'.'+version+'/'
outdir_2A='data/'+product_2A+'.'+version+'/'
outdir_2B='data/'+product_2B+'.'+version+'/'

gediDownload(outdir_1B,product_1B,version,bbox,session)
gediDownload(outdir_2A,product_2A,version,bbox,session)
gediDownload(outdir_2B,product_2B,version,bbox,session)

Not able to download data !!!

Hi @EduinHSERNA @andreshs137 @tim-devereux
I tried to download GEDI data following the same way as https://github.com/EduinHSERNA/pyGEDI.

But I am getting the following error, please let me know how to resolve it. It feels like getfinder is not working.

Created the subdirectory data/GEDI01_B.001/

JSONDecodeError Traceback (most recent call last)
File ~/anaconda3/envs/gdal/lib/python3.9/site-packages/requests/models.py:971, in Response.json(self, **kwargs)
970 try:
--> 971 return complexjson.loads(self.text, **kwargs)
972 except JSONDecodeError as e:
973 # Catch JSON-related errors and raise as requests.JSONDecodeError
974 # This aliases json.JSONDecodeError and simplejson.JSONDecodeError

File ~/anaconda3/envs/gdal/lib/python3.9/json/init.py:346, in loads(s, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, **kw)
343 if (cls is None and object_hook is None and
344 parse_int is None and parse_float is None and
345 parse_constant is None and object_pairs_hook is None and not kw):
--> 346 return _default_decoder.decode(s)
347 if cls is None:

File ~/anaconda3/envs/gdal/lib/python3.9/json/decoder.py:337, in JSONDecoder.decode(self, s, _w)
333 """Return the Python representation of s (a str instance
334 containing a JSON document).
335
336 """
--> 337 obj, end = self.raw_decode(s, idx=_w(s, 0).end())
338 end = _w(s, end).end()

File ~/anaconda3/envs/gdal/lib/python3.9/json/decoder.py:355, in JSONDecoder.raw_decode(self, s, idx)
354 except StopIteration as err:
--> 355 raise JSONDecodeError("Expecting value", s, err.value) from None
356 return obj, end

JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

JSONDecodeError Traceback (most recent call last)
Cell In[8], line 13
9 outdir_1B='data/'+product_1B+'.'+version+'/'
10 # outdir_2A='data/'+product_2A+'.'+version+'/'
11 # outdir_2B='data/'+product_2B+'.'+version+'/'
---> 13 gediDownload(outdir_1B,product_1B,version,bbox,session)

File ~/anaconda3/envs/gdal/lib/python3.9/site-packages/pyGEDI/init.py:71, in gediDownload(outdir, product, version, bbox, session)
69 url='https://lpdaacsvc.cr.usgs.gov/services/gedifinder?product='+product+'&version='+str(version)+'&bbox='+str(bbox)+'&output=json'
70 content=requests.get(url)
---> 71 listh5=content.json().get('data')
73 for url in listh5:
74 url_response(outdir,url,session)

File ~/anaconda3/envs/gdal/lib/python3.9/site-packages/requests/models.py:975, in Response.json(self, **kwargs)
971 return complexjson.loads(self.text, **kwargs)
972 except JSONDecodeError as e:
973 # Catch JSON-related errors and raise as requests.JSONDecodeError
974 # This aliases json.JSONDecodeError and simplejson.JSONDecodeError
--> 975 raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)

JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Downloading and processing data

I think this is maybe a feature request rather than an issue, but even for modest areas of interest, downloading and processing a small region of GEDI requires a large quantity of local storage. Using the current methods, the entirety of the swaths have to be downloaded and then processing to subset to an AOI can be done. Would it be possible to extend this method to download a swath, and clip it to an AOI, and then cleanup the download, retaining only the spot points which intersect an area of interest?

Currently I can't actually process an AOI because I keep filling up my hard-drive(s).

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.