umr-lops / sarwaveifrproc Goto Github PK

View Code? Open in Web Editor NEW

2.0 3.0 1.0 1.48 MB

SAR sea state Ifremer processor

Home Page: https://cerweb.ifremer.fr/datarmor/sarwave/documentation/processor/sar/sarwaveifrproc/html/

License: MIT License

Makefile 6.18% Python 93.82%

neural-network oceanography period sar sea-state-estimation sentinel-1 wave-height waves

sarwaveifrproc's Introduction

sarwaveifrproc

SAR Sentinel-1 ESA mission sea state Ifremer processor

Free software: MIT license
Documentation: https://sarwaveifrproc.readthedocs.io.

Features

predicts sea state geophysical quantities from Level-1B or Level-1C Ifremer SARWAVE Sentinel-1 (WV,IW,EW) products using empirical function learnt on numerical hindcasts (WAVEWATCH III):

significant wave height (Hs)

mean wave period (t0m1)

significant wave height of the wind-sea (pshs0)

save results in a netCDF file per sub-swath.

Credits

This Sentinel-1 Level-2 sea state processor is developed/maintained by Ifremer - LOPS laboratory. This work is co-funded by ESA through the SARWAVE project (https://www.sarwave.org/). The processor development benefits from support and contributions from Sentinel-1 Mission Performance Cluster team (https://sar-mpc.eu/about/activities-and-team/).

sarwaveifrproc's People

Contributors

Stargazers

Watchers

Forkers

alxmouche

sarwaveifrproc's Issues

Fix unit test Hs predictions intraburst

When comparing intraburst hs_mean values from a CPU prediction on the file safe=/home/datawork-cersat-public/cache/project/sarwave/data/products/tests/iw/slc/l1b/3.7.6/S1A_IW_XSP__1SDV_20231128T035702_20231128T035729_051412_063451_3781.SAFE/
using this cmd:
(/opt/conda-envs/mamba-l2awinddir-dev) agrouaze@grougrou1 12:26:55 ~/git/sarwaveifrproc L2-wave-processor --save_directory ${SCRATCH}/workspace_prediction/seastate/ --product_id val01cpu --input_safe $safe --overwrite
we have:

_, _, _, _, _,
  _, _, _, _, _,
  _, _, _, _, _,
  _, _, _, _, _,
  _, _, _, _, _,
  _, _, 0.704757439330426, 0.829383210159725, 0.772311780099223,
  _, _, _, 1.01369066360322, 1.04560130899908,
  _, _, _, _, 0.893044782992965,
  _, _, _, _, _ ;
  } // group intraburst

while in the reference file ./sarwaveifrproc/reference_data/s1a-iw2-slc-dv-20231128t035702-20231128t035727-051412-063451-e01.nc (generated from GPU), we have:

_, _, _, _, _,
  _, _, _, _, _,
  _, _, _, _, _,
  _, _, _, _, _,
  _, _, _, _, _,
  _, _, 0.645451854292777, 0.762869749876333, 0.717675525029661,
  _, _, _, 0.979326482544491, 0.982501818281662,
  _, _, _, _, 0.881201161077921,
  _, _, _, _, _ ;
  } // group intraburst

SAFE name have the L1B slug not replaced.

fix the function to get output safe name
extend the unit test to check such issue
observed : ./2020/007/S1B_IW_WAV__2SDV_20200107T154013_20200107T154040_019713_025450_C73D_A09_E04.SAFE
expected : ./2020/007/S1B_IW_WAV__2SDV_20200107T154013_20200107T154040_019713_025450_C73D_E04.SAFE

some L2 wave products (E04) have an issue in the measurement filename

We saw that there was products E04 with measurement filelie this:
l2-s1a-iw1-wav-dv-vv-20220507t162439...
while we want:
l2-s1a-iw1-wav-dv-20220507t162439...

Make sure tiles over land have wave parameters filled by NaN

We have to check whether the latest version of the processor behave as we decided to handle tiles over land:

all variables should be present but for the wave parameters they should be filled with NaN

fix pypi workflow

permission to fix.

doc update

cite missing for stopa2017
correct ATBD "basis"
credits SARWAVE and ESA in the readme.md
ATBD review (voir "PAPER IGARSS 2024 - SARWAVE" overleaf)

clarify which polarisations are usable for prediction

latest L1B without `burst` dimension are not handle by the processor

When using Level-1B recent product (without burst dimension) as input of the sea state Level-2 processor, one can get such error:

Traceback (most recent call last):                                                                                                                                                                             
  File "/usr/local/bin/L2-wave-processor", line 8, in <module>                                                                                                                                                     sys.exit(main())                                                                                                                                                                                           
  File "/usr/local/lib/python3.10/dist-packages/sarwaveifrproc/main.py", line 82, in main                                                                                                                      
    process_files(input_safe, output_safe, model_intraburst, model_interburst, scaler_intraburst, scaler_interburst, bins_intraburst, bins_interburst, predicted_variables, product_id)                        
  File "/usr/local/lib/python3.10/dist-packages/sarwaveifrproc/utils.py", line 164, in process_files                                                                                                           
    l2_product = generate_l2_wave_product(xdt, model_intraburst, model_interburst, scaler_intraburst, scaler_interburst, bins_intraburst, bins_interburst, predicted_variables)                                
  File "/usr/local/lib/python3.10/dist-packages/sarwaveifrproc/l2_wave.py", line 39, in generate_l2_wave_product                                                                                               
    ds_intraburst = generate_intermediate_product(xdt['intraburst'].ds, intraburst_model, intraburst_scaler, intraburst_bins, predicted_variables, kept_variables)                                             
  File "/usr/local/lib/python3.10/dist-packages/sarwaveifrproc/l2_wave.py", line 69, in generate_intermediate_product                                                                                              tiles_stacked = tiles.stack(all_tiles = ['burst', 'tile_line','tile_sample'], k_phi = ['phi_hf', 'k_gp'])                                                                                                  
  File "/usr/local/lib/python3.10/dist-packages/xarray/core/dataset.py", line 5279, in stack                                                                                                                   
    result = result._stack_once(dims, new_dim, index_cls, create_index)                                                                                                                                        
  File "/usr/local/lib/python3.10/dist-packages/xarray/core/dataset.py", line 5193, in _stack_once                                                                                                                 shape = [self.sizes[d] for d in vdims]                                                                                                                                                                     
  File "/usr/local/lib/python3.10/dist-packages/xarray/core/dataset.py", line 5193, in <listcomp>                                                                                                                  shape = [self.sizes[d] for d in vdims]                                                                                                                                                                     
  File "/usr/local/lib/python3.10/dist-packages/xarray/core/utils.py", line 427, in __getitem__                                                                                                                    return self.mapping[key]                                                                                                                                                                                   
KeyError: 'burst'

investigate usefulness of `onnx`

https://onnx.ai/ is both a format to store NN models independently of the pytorch or keras version and a "runtime" that allow to do predictions.
This ticket to investigate potential usage of this "framework" to replace "keras" on the inference part.