ncar / gmet Goto Github PK

Gridded Meteorological Ensemble Tool

Fortran 100.00%

gmet's Introduction

Gridded Meteorological Ensemble Tool (GMET)

Overview

GMET is a software for created gridded meteorological datasets for precipitation and temperature. The current applications have been at a daily timestep, yielding daily total precipitation, mean temperature and temperature range. The algorithm is based on locally-weighted spatial regression, applied independently for each day and each output grid cell to a sample of nearby stations. The approach yields ensemble meteorological fields for which the mean and spread vary in time and space.

GMET is organized into two programs that are run in sequence: (1) sp_regression, and (2) ens_generation. The first applies the spatial regression, generating an output file for the domain that contains all the regression coefficients. The second uses this file to create ensemble members by using spatially correlated random fields to sample the regression uncertainty.

Several test cases are included (in tar bundles) to give examples of the application of GMET. These are compatible with GMET v2.

Example 1 is for a 2 degree by 2 degree domain and does not use cross-validation or dynamic predictors
Example 2 is for a domain including much of northern California, uses cross validation and dynamic predictors (from HRRR). This test case is described in the paper: Bunn et al. (2021)

GMET has been used successfully in a number of applications, including supporting real-time and retrospective hydrologic modeling and forecasting and research in the western US.

Existing GMET Datasets

1/8th degree 100-member NLDAS-domain implementation (Newman et al, 2015), version 2, https://doi.org/10.5065/D6TH8JR2
1 km 100-member Hawaii dataset from 1990-2014, https://doi.org/10.5065/D6SB44JV
2 km 100-member Alaska dataset from 1980-2012, https://doi.org/10.5065/D61Z42T0
1/8 degree 100-member NLDAS-domain ensemble dressing for NLDASv2 (EDN2), ds613.0 | https://doi.org/10.5065/KARJ-0E19
1/8th and 1/16th degree 36-member ensembles for a western US domain (125-92W, 25-53N), 1970-2020 (with some real-time components). Contact: Andy Wood, NCAR

Notes

This code is a work in progress and is provided without guarantee of fitness for any particular application.
The master branch contains the most vetted release
The SHARP branch is used operationally to generate real-time ensemble model forcings at NCAR. The only direct pushes to the SHARP branch, however, are made from the operational clone after testing in the operational context.
The develop branch will contain up to date general improvements between new release updates in the master branch

References

GMET development and major datasets

Bunn, PTW, AW Wood, AJ Newman, H Chang, CL Castro, MP Clark and JR Arnold (2021) Improving in situ observation-based ensemble surface meteorological analyses using numerical weather prediction: A case study of the Oroville Dam crisis precipitation event. AMS J. Hydromet. (in review)
Newman, A. J. et al. (2020) ‘Probabilistic Spatial Meteorological Estimates for Alaska and the Yukon’, Journal of Geophysical Research: Atmospheres, 125(22), pp. 1–21. doi: 10.1029/2020JD032696.
Newman, A. J. et al. (2019) ‘Use of daily station observations to produce high-resolution gridded probabilistic precipitation and temperature time series for the Hawaiian Islands’, Journal of Hydrometeorology, 20(3), pp. 509–529. doi: 10.1175/JHM-D-18-0113.1.
Newman, AJ, MP Clark, J Craig, B Nijssen, AW Wood, E Gutmann, N Mizukami, L Brekke, and JR Arnold, 2015, Gridded Ensemble Precipitation and Temperature Estimates for the Contiguous United States, J. Hydromet., doi: http://dx.doi.org/10.1175/JHM-D-15-0026.1
Clark, M. P. and Slater, A. G. (2006) ‘Probabilistic Quantitative Precipitation Estimation in Complex Terrain’, Hydrometeorology, Journal O F, (2000), pp. 3–22.

Applications or Research using GMET

Huang, C, AJ Newman, MP Clark, AW Wood and X Zheng, 2016, Evaluation of snow data assimilation using the ensemble Kalman Filter for seasonal streamflow prediction in the Western United States, Hydrol. Earth Syst. Sci. 21, 635-650, 2017, http://www.hydrol-earth-syst-sci.net/21/635/2017/, doi:10.5194/hess-21-635-2017
Mendoza, PA, AW Wood, EA Clark, E Rothwell, MP Clark, B Nijssen, LD Brekke, and JR Arnold, 2017, An intercomparison of approaches for improving predictability in operational seasonal streamflow forecasting, Hydrol. Earth Syst. Sci., 21, 3915–3935, 2017
Liu, Hongli, AW Wood, AJ Newman and MP Clark, 2021, Ensemble dressing of meteorological fields: using spatial regression to estimate uncertainty in deterministic gridded meteorological datasets, AMS J. Hydromet. (submitted)

gmet's People

Contributors

Stargazers

Watchers

Forkers

amfox37 hydrofcst martynpclark anewman89 h294liu guoqiang-fork envon-x hydrospheric guoqiang-tang shinginangombe

gmet's Issues

Desired improvements to SCRFs

In today's group meeting, we discussed a few issues that would be nice to address in the SCRF code.

cc @anewman89 @gutmann @martynpclark

proposals for metrics and laugh tests

At our group meeting today, @martynpclark asked me to start a discussion where we develop a list of standard GMET evaluation metrics and a series of easy-to-produce laugh tests. I'll come back to this issue after the meetings

cc @gutmann, @martynpclark, @anewman89

xref: jhamman/storylines#2

explore vectorization operations in ens. generation

right now many scalar operations and if statements are applied w/ loops where they could be faster vector or array operations

bring to the surface all buried/hardwired constants & decisions

these include correlation length for temperature, error fraction used in max pcp cap, threshold for slope to switch between regressions and so on.

change ens generation control file to a namelist

right now it's an order specific text file

Compare GMET SCRF algorithm to others in the literature

In today's group meeting, we discussed that it would be beneficial to explore other existing SCRF methods that have been developed since @martynpclark wrote the algorithm included in this repository.

@martynpclark @gutmann @anewman89

split scrf generation + ensemble forcing generation

Allow for saving SCRF #s for reuse
Consider converting this piece to another language, eg R or python

turn high precip truncation and temp limits into user supplied terms (including option not to use).

consider writing outputs incrementally (per timestep)

Currently GMET's spatial regression writes no output until the entire run is done. If a run dies before the final timestep is processed, all output is lost.

Letting GMET write each timestep after the regression is complete would also reduce the size of the memory arrays needed -- right now large GMET domains can require using big-memory nodes to complete.

There would be a slight run time hit but not much.

create modules so as to separate all the bulky interfaces from the code files

this would improve readability a great deal. I tried to do this at one point but then couldn't get it to compile, so it needs more work.

let exponent in station weighting function be a user option or else station density or terrain dependent

The station weighting function is hardwired to an inverse cubic. This may exclude (by downweighting) too many stations in some case, thus having the exponent be data-responsive or user controlled may be beneficial. The flexible gaussian-esque form may be sufficient without changing further. Along with the user specified max distance, this is effectively the same weighting fcn as in Werner et al (2004), with alpha & lambda params.

The current 'USE_STATION_WEIGHTING' could be generalized/eliminated with this option, since specifying an exponent of 0 means there is no station weighting; an exponent of 1 means it is linear with distance.

let the user supply the temperature autocorrelation and temp-precip cross correlations for the regression step

For a short run (eg a few weeks to a few months) which might be used in a real-time system, these correlations are likely to be poorly estimated, thus it would be better to let the user supply them in the config file, perhaps as estimated for the target region based on a longer run.

Also, at present the mean correlations are calculated across all stations, but if a larger domain of stations is given in the input file than is used in the target domain, the estimate will not be optimal for the local region. It's worth calculating the correlations after the analysis to find the nearest neighbors for all the grid cells in the target domain.

Of course the bigger issue is that the full domain mean long term correlation is a huge approximation, and could be refined locally and seasonally for better results (should be another issue).

remove mode 1 from GMET?

mode 1 (downscaling grids) has not been maintained since GARD was written. It could be removed from the code.

take allocation/deallocation back out of inner time/grid loop

This was removed once, and put back in, but currently a set of fixed arrays are allocated and deallocated 10s of millions of times, when they can be allocated and deallocated just once -- in estimate_forcing_regression.f90.