Comments (9)
ThomasMoreHamill
commented on August 23, 2024
2
I have done a quick random check (I have not checked all folders) but it appears as all the files related to subset of fields above 700mb are missing for the D>10 days forlders.
Chiara, all pressure levels after day+10 are in one file without the "_abv700". The documentation will be updated to reflect this.
from open-data-registry.
zflamig
commented on August 23, 2024
1
Reaching out to see if I can find out the answer to this
from open-data-registry.
chiaral
commented on August 23, 2024
1
This issue has been addressed by @ThomasMoreHamill - with the new documentation https://noaa-gefs-retrospective.s3.amazonaws.com/Description_of_reforecast_data.pdf
Thanks so much!
from open-data-registry.
chiaral
commented on August 23, 2024
Hi Zac,
I also noticed that the lead time resolution goes from 3 hourly to 6 hourly when I move from D1-10 to D>10 . This is not in the documentation.
from open-data-registry.
chiaral
commented on August 23, 2024
Hello - I am expanding this issue, because I am finding more un-documented things as i move through the data.
In particular, for the Day 1-10 the data should be at a 3 hourly interval. That makes for 80 time steps (8 per day for the first 10 days).
I have quite a few variables, however, that when read with pynio engine in xarray - I am exploring them with wgrib2 as well however, have 2 different coordinate dimensions:
<xarray.Dataset>
Dimensions: (forecast_time0: 40, forecast_time1: 40, lat_0: 721, lon_0: 1440)
Coordinates:
* forecast_time1 (forecast_time1) timedelta64[ns] 0 days 03:00:00...
* lat_0 (lat_0) float32 90.0 89.75 89.5 ... -89.75 -90.0
* lon_0 (lon_0) float32 0.0 0.25 0.5 ... 359.5 359.75
* forecast_time0 (forecast_time0) timedelta64[ns] 0 days 06:00:00...
Data variables:
ACPCP_P11_L1_GLL0_acc6h (forecast_time0, lat_0, lon_0) float32 dask.array<chunksize=(1, 721, 1440), meta=np.ndarray>
ACPCP_P11_L1_GLL0_acc3h (forecast_time1, lat_0, lon_0) float32 dask.array<chunksize=(40, 721, 1440), meta=np.ndarray>
The two forecast_time however seem to be read as lagged of 3 hours:
xarray.DataArray 'forecast_time0' (forecast_time0: 40)>
array([ 6., 12., 18., 24., 30., 36., 42., 48., 54., 60., 66.,
72., 78., 84., 90., 96., 102., 108., 114., 120., 126., 132.,
138., 144., 150., 156., 162., 168., 174., 180., 186., 192., 198.,
204., 210., 216., 222., 228., 234., 240.])
Coordinates:
* forecast_time0 (forecast_time0) timedelta64[ns] 0 days 06:00:00 ... 10 d...
<xarray.DataArray 'forecast_time1' (forecast_time1: 40)>
array([ 3., 9., 15., 21., 27., 33., 39., 45., 51., 57., 63.,
69., 75., 81., 87., 93., 99., 105., 111., 117., 123., 129.,
135., 141., 147., 153., 159., 165., 171., 177., 183., 189., 195.,
201., 207., 213., 219., 225., 231., 237.])
Coordinates:
* forecast_time1 (forecast_time1) timedelta64[ns] 0 days 03:00:00 ... 9 da...
What does happen if I plot the two time series?
They seem to be actually accumulated every 3 hourly for one, and every 6 hourly for the other. (this is the global average, clearly one is the double of the other).
for another variable where the quantities is not an accumulated measure but an average (but for which I have the same double coordinates) I have:
<xarray.Dataset>
Dimensions: (forecast_time0: 40, forecast_time1: 40, lat_0: 721, lon_0: 1440)
Coordinates:
* forecast_time1 (forecast_time1) timedelta64[ns] 0 days 03:00:00...
* lat_0 (lat_0) float32 90.0 89.75 89.5 ... -89.75 -90.0
* lon_0 (lon_0) float32 0.0 0.25 0.5 ... 359.5 359.75
* forecast_time0 (forecast_time0) timedelta64[ns] 0 days 06:00:00...
Data variables:
DLWRF_P11_L1_GLL0_avg6h (forecast_time0, lat_0, lon_0) float32 dask.array<chunksize=(1, 721, 1440), meta=np.ndarray>
DLWRF_P11_L1_GLL0_avg3h (forecast_time1, lat_0, lon_0) float32 dask.array<chunksize=(40, 721, 1440), meta=np.ndarray>
<xarray.DataArray 'forecast_time0' (forecast_time0: 40)>
array([ 6., 12., 18., 24., 30., 36., 42., 48., 54., 60., 66.,
72., 78., 84., 90., 96., 102., 108., 114., 120., 126., 132.,
138., 144., 150., 156., 162., 168., 174., 180., 186., 192., 198.,
204., 210., 216., 222., 228., 234., 240.])
Coordinates:
* forecast_time0 (forecast_time0) timedelta64[ns] 0 days 06:00:00 ... 10 d...
<xarray.DataArray 'forecast_time1' (forecast_time1: 40)>
array([ 3., 9., 15., 21., 27., 33., 39., 45., 51., 57., 63.,
69., 75., 81., 87., 93., 99., 105., 111., 117., 123., 129.,
135., 141., 147., 153., 159., 165., 171., 177., 183., 189., 195.,
201., 207., 213., 219., 225., 231., 237.])
Coordinates:
* forecast_time1 (forecast_time1) timedelta64[ns] 0 days 03:00:00 ... 9 da...
When I look into the file through a wgrib2 reader, I have 80 time steps (corresponding to the 80 3 hourly interval in 240 hours) but these are the deltas (sorry I will copy everything for completeness) :
1:0:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:0-3 hour acc fcst:ENS=low-res ctl
2:325317:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:0-6 hour acc fcst:ENS=low-res ctl
3:720943:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:6-9 hour acc fcst:ENS=low-res ctl
4:1037396:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:6-12 hour acc fcst:ENS=low-res ctl
5:1409304:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:12-15 hour acc fcst:ENS=low-res ctl
6:1739118:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:12-18 hour acc fcst:ENS=low-res ctl
7:2124405:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:18-21 hour acc fcst:ENS=low-res ctl
8:2461831:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:18-24 hour acc fcst:ENS=low-res ctl
9:2847727:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:24-27 hour acc fcst:ENS=low-res ctl
10:3178852:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:24-30 hour acc fcst:ENS=low-res ctl
11:3563279:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:30-33 hour acc fcst:ENS=low-res ctl
12:3899970:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:30-36 hour acc fcst:ENS=low-res ctl
13:4289611:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:36-39 hour acc fcst:ENS=low-res ctl
14:4632471:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:36-42 hour acc fcst:ENS=low-res ctl
15:5029422:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:42-45 hour acc fcst:ENS=low-res ctl
16:5375796:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:42-48 hour acc fcst:ENS=low-res ctl
17:5770451:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:48-51 hour acc fcst:ENS=low-res ctl
18:6107738:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:48-54 hour acc fcst:ENS=low-res ctl
19:6500107:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:54-57 hour acc fcst:ENS=low-res ctl
20:6839749:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:54-60 hour acc fcst:ENS=low-res ctl
21:7232102:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:60-63 hour acc fcst:ENS=low-res ctl
22:7576151:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:60-66 hour acc fcst:ENS=low-res ctl
23:7974133:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:66-69 hour acc fcst:ENS=low-res ctl
24:8322998:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:66-72 hour acc fcst:ENS=low-res ctl
25:8723627:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:72-75 hour acc fcst:ENS=low-res ctl
26:9069130:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:72-78 hour acc fcst:ENS=low-res ctl
27:9467807:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:78-81 hour acc fcst:ENS=low-res ctl
28:9815931:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:78-84 hour acc fcst:ENS=low-res ctl
29:10218210:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:84-87 hour acc fcst:ENS=low-res ctl
30:10571367:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:84-90 hour acc fcst:ENS=low-res ctl
31:10978922:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:90-93 hour acc fcst:ENS=low-res ctl
32:11335902:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:90-96 hour acc fcst:ENS=low-res ctl
33:11743572:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:96-99 hour acc fcst:ENS=low-res ctl
34:12092049:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:96-102 hour acc fcst:ENS=low-res ctl
35:12495277:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:102-105 hour acc fcst:ENS=low-res ctl
36:12847432:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:102-108 hour acc fcst:ENS=low-res ctl
37:13255942:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:108-111 hour acc fcst:ENS=low-res ctl
38:13607490:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:108-114 hour acc fcst:ENS=low-res ctl
39:14013142:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:114-117 hour acc fcst:ENS=low-res ctl
40:14365292:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:114-120 hour acc fcst:ENS=low-res ctl
41:14767992:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:120-123 hour acc fcst:ENS=low-res ctl
42:15113945:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:120-126 hour acc fcst:ENS=low-res ctl
43:15512978:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:126-129 hour acc fcst:ENS=low-res ctl
44:15862333:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:126-132 hour acc fcst:ENS=low-res ctl
45:16265666:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:132-135 hour acc fcst:ENS=low-res ctl
46:16616099:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:132-138 hour acc fcst:ENS=low-res ctl
47:17023212:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:138-141 hour acc fcst:ENS=low-res ctl
48:17377777:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:138-144 hour acc fcst:ENS=low-res ctl
49:17784248:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:144-147 hour acc fcst:ENS=low-res ctl
50:18133857:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:144-150 hour acc fcst:ENS=low-res ctl
51:18538700:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:150-153 hour acc fcst:ENS=low-res ctl
52:18892132:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:150-156 hour acc fcst:ENS=low-res ctl
53:19303274:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:156-159 hour acc fcst:ENS=low-res ctl
54:19665780:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:156-162 hour acc fcst:ENS=low-res ctl
55:20081576:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:162-165 hour acc fcst:ENS=low-res ctl
56:20439920:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:162-168 hour acc fcst:ENS=low-res ctl
57:20850288:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:168-171 hour acc fcst:ENS=low-res ctl
58:21202842:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:168-174 hour acc fcst:ENS=low-res ctl
59:21609977:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:174-177 hour acc fcst:ENS=low-res ctl
60:21969286:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:174-180 hour acc fcst:ENS=low-res ctl
61:22382397:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:180-183 hour acc fcst:ENS=low-res ctl
62:22743627:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:180-186 hour acc fcst:ENS=low-res ctl
63:23158467:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:186-189 hour acc fcst:ENS=low-res ctl
64:23519584:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:186-192 hour acc fcst:ENS=low-res ctl
65:23930481:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:192-195 hour acc fcst:ENS=low-res ctl
66:24284666:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:192-198 hour acc fcst:ENS=low-res ctl
67:24698501:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:198-201 hour acc fcst:ENS=low-res ctl
68:25064181:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:198-204 hour acc fcst:ENS=low-res ctl
69:25481643:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:204-207 hour acc fcst:ENS=low-res ctl
70:25844541:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:204-210 hour acc fcst:ENS=low-res ctl
71:26264820:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:210-213 hour acc fcst:ENS=low-res ctl
72:26628166:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:210-216 hour acc fcst:ENS=low-res ctl
73:27042627:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:216-219 hour acc fcst:ENS=low-res ctl
74:27399562:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:216-222 hour acc fcst:ENS=low-res ctl
75:27811628:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:222-225 hour acc fcst:ENS=low-res ctl
76:28172038:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:222-228 hour acc fcst:ENS=low-res ctl
77:28592598:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:228-231 hour acc fcst:ENS=low-res ctl
78:28957647:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:228-234 hour acc fcst:ENS=low-res ctl
79:29379645:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:234-237 hour acc fcst:ENS=low-res ctl
80:29740164:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:234-240 hour acc fcst:ENS=low-res ctl
Note how:
1:0:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:0-3 hour acc fcst:ENS=low-res ctl
2:325317:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:0-6 hour acc fcst:ENS=low-res ctl
3:720943:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:6-9 hour acc fcst:ENS=low-res ctl
4:1037396:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:6-12 hour acc fcst:ENS=low-res ctl
5:1409304:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:12-15 hour acc fcst:ENS=low-res ctl
6:1739118:d=2000010100:ACPCP Convective Precipitation [kg/m^2]:surface:12-18 hour acc fcst:ENS=low-res ctl
I have the first 3 hourly interval, then a 6 hourly, then a 3 hourly from 6-9, then a 6hourly... and so on.. so the 3-6, is missing, so is the 9-12 - and it is instead swapped with the 6hourly accumulation. I also want to note that the Delta time steps in the grib files do NOT correspond to the delta time step of the xarray dataset, I think xarray is struggling to figure out what is the right interval. And so am I!
Can we have someone from NOAA chiming in? Thanks SOOO much!
from open-data-registry.
chiaral
commented on August 23, 2024
for completeness - the delta of a regular variable with just one forecast_time coordinate looks like this (albeit it's not an interval).
1:0:d=2000010400:TMP Temperature [K]:surface:3 hour fcst:ENS=low-res ctl
2:550304:d=2000010400:TMP Temperature [K]:surface:6 hour fcst:ENS=low-res ctl
3:1103954:d=2000010400:TMP Temperature [K]:surface:9 hour fcst:ENS=low-res ctl
4:1661552:d=2000010400:TMP Temperature [K]:surface:12 hour fcst:ENS=low-res ctl
5:2223727:d=2000010400:TMP Temperature [K]:surface:15 hour fcst:ENS=low-res ctl
6:2785544:d=2000010400:TMP Temperature [K]:surface:18 hour fcst:ENS=low-res ctl
7:3348267:d=2000010400:TMP Temperature [K]:surface:21 hour fcst:ENS=low-res ctl
from open-data-registry.
chiaral
commented on August 23, 2024
After reading the table with more attention - and not the documentation only - this behavior is documented! so it's all good - it might require some wrangling in the zarr version but I will figure it out.
The missing files tho and the different resolution for D>10 doesn't seem to be in the documentation.
from open-data-registry.
chiaral
commented on August 23, 2024
Ok - here I finalize the list of missing documentation to the best of my knowledge - so far - in the dataset.
The link to the Presentation "Meanwhile, a presentation on GEFSv12 can be found here." listed in the documentation is not publicly available - I get a "you need access" warning. Probably some of these missing details are clarified in that presentation, but I cannot be sure.
-
the files related to the 5 variables for fields above 700mb are missing for the D>10 days folders - in fact you have 59 files in D 1-10 and 54 in the D 10-16/35 - I am pretty sure this is true for the whole dataset (i have done some random folders check and 100% of those I checked had missing files).
-
In more than one presentation I have learned that for D>10 the temporal resolution goes from 3hours to 6 hours. That is in fact the case in the files, but there is no mention of this in the documentation; it simply states the text below, please update the documentation to reflect that:
For most grib2 files, the data are provided on a grid with a 0.25-degree grid spacing,
archived every 3 hours for the first 10 days of the forecast; beyond 10 days, 0.50 degrees grid
spacing is used.
- The last table of the documentation lists the "Single-level reforecast variables (and units). Archived at 0.25 degree resolution to 10 days, 0.5 degree resolution beyond. " variables. In particular for 5 variables, it mentions that the width of the forecast window is not homogeneous (for the D1-10 case, when the resolution is expected to be 3 hourly):
in fact, if I download one file for accumulated convective precip and use wgrib2 to explore it:
1:0:d=2000012900:ACPCP Convective Precipitation [kg/m^2]:surface:0-3 hour acc fcst:ENS=low-res ctl
2:350244:d=2000012900:ACPCP Convective Precipitation [kg/m^2]:surface:0-6 hour acc fcst:ENS=low-res ctl
3:770020:d=2000012900:ACPCP Convective Precipitation [kg/m^2]:surface:6-9 hour acc fcst:ENS=low-res ctl
4:1098391:d=2000012900:ACPCP Convective Precipitation [kg/m^2]:surface:6-12 hour acc fcst:ENS=low-res ctl
5:1479463:d=2000012900:ACPCP Convective Precipitation [kg/m^2]:surface:12-15 hour acc fcst:ENS=low-res ctl
6:1821766:d=2000012900:ACPCP Convective Precipitation [kg/m^2]:surface:12-18 hour acc fcst:ENS=low-res ctl
7:2218345:d=2000012900:ACPCP Convective Precipitation [kg/m^2]:surface:18-21 hour acc fcst:ENS=low-res ctl
for Tmax
1:0:d=2000012900:TMAX Maximum Temperature [K]:2 m above ground:0-3 hour max fcst:ENS=low-res ctl
2:752876:d=2000012900:TMAX Maximum Temperature [K]:2 m above ground:0-6 hour max fcst:ENS=low-res ctl
3:1501605:d=2000012900:TMAX Maximum Temperature [K]:2 m above ground:6-9 hour max fcst:ENS=low-res ctl
4:2226956:d=2000012900:TMAX Maximum Temperature [K]:2 m above ground:6-12 hour max fcst:ENS=low-res ctl
5:2951909:d=2000012900:TMAX Maximum Temperature [K]:2 m above ground:12-15 hour max fcst:ENS=low-res ctl
6:3374354:d=2000012900:TMAX Maximum Temperature [K]:2 m above ground:12-18 hour max fcst:ENS=low-res ctl
7:4108243:d=2000012900:TMAX Maximum Temperature [K]:2 m above ground:18-21 hour max fcst:ENS=low-res ctl
8:4851238:d=2000012900:TMAX Maximum Temperature [K]:2 m above ground:18-24 hour max fcst:ENS=low-res ctl
However I have noticed this varying resolution in other variables that are not listed in the documentation, for the averaging period:
Downward Long-Wave Rad. Flux
1:0:d=2000010100:DLWRF Downward Long-Wave Rad. Flux [W/m^2]:surface:0-3 hour ave fcst:ENS=low-res ctl
2:464462:d=2000010100:DLWRF Downward Long-Wave Rad. Flux [W/m^2]:surface:0-6 hour ave fcst:ENS=low-res ctl
3:925251:d=2000010100:DLWRF Downward Long-Wave Rad. Flux [W/m^2]:surface:6-9 hour ave fcst:ENS=low-res ctl
4:1409397:d=2000010100:DLWRF Downward Long-Wave Rad. Flux [W/m^2]:surface:6-12 hour ave fcst:ENS=low-res ctl
5:1876656:d=2000010100:DLWRF Downward Long-Wave Rad. Flux [W/m^2]:surface:12-15 hour ave fcst:ENS=low-res ctl
6:2376494:d=2000010100:DLWRF Downward Long-Wave Rad. Flux [W/m^2]:surface:12-18 hour ave fcst:ENS=low-res ctl
7:2854612:d=2000010100:DLWRF Downward Long-Wave Rad. Flux [W/m^2]:surface:18-21 hour ave fcst:ENS=low-res ctl
Downward Short-Wave Radiation Flux
1:0:d=2000010100:DSWRF Downward Short-Wave Radiation Flux [W/m^2]:surface:0-3 hour ave fcst:ENS=low-res ctl
2:326949:d=2000010100:DSWRF Downward Short-Wave Radiation Flux [W/m^2]:surface:0-6 hour ave fcst:ENS=low-res ctl
3:705168:d=2000010100:DSWRF Downward Short-Wave Radiation Flux [W/m^2]:surface:6-9 hour ave fcst:ENS=low-res ctl
4:904778:d=2000010100:DSWRF Downward Short-Wave Radiation Flux [W/m^2]:surface:6-12 hour ave fcst:ENS=low-res ctl
5:1121675:d=2000010100:DSWRF Downward Short-Wave Radiation Flux [W/m^2]:surface:12-15 hour ave fcst:ENS=low-res ctl
6:1335049:d=2000010100:DSWRF Downward Short-Wave Radiation Flux [W/m^2]:surface:12-18 hour ave fcst:ENS=low-res ctl
7:1562761:d=2000010100:DSWRF Downward Short-Wave Radiation Flux [W/m^2]:surface:18-21 hour ave fcst:ENS=low-res ctl
8:1784273:d=2000010100:DSWRF Downward Short-Wave Radiation Flux [W/m^2]:surface:18-24 hour ave fcst:ENS=low-res ctl
Ground Heat Flux
1:0:d=2000010100:GFLUX Ground Heat Flux [W/m^2]:surface:0-3 hour ave fcst:ENS=low-res ctl
2:445442:d=2000010100:GFLUX Ground Heat Flux [W/m^2]:surface:0-6 hour ave fcst:ENS=low-res ctl
3:887615:d=2000010100:GFLUX Ground Heat Flux [W/m^2]:surface:6-9 hour ave fcst:ENS=low-res ctl
4:1340996:d=2000010100:GFLUX Ground Heat Flux [W/m^2]:surface:6-12 hour ave fcst:ENS=low-res ctl
5:1788279:d=2000010100:GFLUX Ground Heat Flux [W/m^2]:surface:12-15 hour ave fcst:ENS=low-res ctl
6:2242787:d=2000010100:GFLUX Ground Heat Flux [W/m^2]:surface:12-18 hour ave fcst:ENS=low-res ctl
7:2690135:d=2000010100:GFLUX Ground Heat Flux [W/m^2]:surface:18-21 hour ave fcst:ENS=low-res ctl
8:3146470:d=2000010100:GFLUX Ground Heat Flux [W/m^2]:surface:18-24 hour ave fcst:ENS=low-res ctl
Latent Heat Net Flux
1:0:d=2000010100:LHTFL Latent Heat Net Flux [W/m^2]:surface:0-3 hour ave fcst:ENS=low-res ctl
2:820379:d=2000010100:LHTFL Latent Heat Net Flux [W/m^2]:surface:0-6 hour ave fcst:ENS=low-res ctl
3:1629555:d=2000010100:LHTFL Latent Heat Net Flux [W/m^2]:surface:6-9 hour ave fcst:ENS=low-res ctl
4:2453074:d=2000010100:LHTFL Latent Heat Net Flux [W/m^2]:surface:6-12 hour ave fcst:ENS=low-res ctl
5:3270124:d=2000010100:LHTFL Latent Heat Net Flux [W/m^2]:surface:12-15 hour ave fcst:ENS=low-res ctl
6:4101893:d=2000010100:LHTFL Latent Heat Net Flux [W/m^2]:surface:12-18 hour ave fcst:ENS=low-res ctl
7:4920923:d=2000010100:LHTFL Latent Heat Net Flux [W/m^2]:surface:18-21 hour ave fcst:ENS=low-res ctl
8
Water Runoff
1:0:d=2000010100:WATR Water Runoff [kg/m^2]:surface:0-3 hour acc fcst:ENS=low-res ctl
2:315053:d=2000010100:WATR Water Runoff [kg/m^2]:surface:0-6 hour acc fcst:ENS=low-res ctl
3:669777:d=2000010100:WATR Water Runoff [kg/m^2]:surface:6-9 hour acc fcst:ENS=low-res ctl
4:866710:d=2000010100:WATR Water Runoff [kg/m^2]:surface:6-12 hour acc fcst:ENS=low-res ctl
5:1087674:d=2000010100:WATR Water Runoff [kg/m^2]:surface:12-15 hour acc fcst:ENS=low-res ctl
6:1285684:d=2000010100:WATR Water Runoff [kg/m^2]:surface:12-18 hour acc fcst:ENS=low-res ctl
7:1508968:d=2000010100:WATR Water Runoff [kg/m^2]:surface:18-21 hour acc fcst:ENS=low-res ctl
Momentum Flux, V-Component
1:0:d=2000010100:VFLX Momentum Flux, V-Component [N/m^2]:surface:0-3 hour ave fcst:ENS=low-res ctl
2:522581:d=2000010100:VFLX Momentum Flux, V-Component [N/m^2]:surface:0-6 hour ave fcst:ENS=low-res ctl
3:1047662:d=2000010100:VFLX Momentum Flux, V-Component [N/m^2]:surface:6-9 hour ave fcst:ENS=low-res ctl
4:1585483:d=2000010100:VFLX Momentum Flux, V-Component [N/m^2]:surface:6-12 hour ave fcst:ENS=low-res ctl
5:2109696:d=2000010100:VFLX Momentum Flux, V-Component [N/m^2]:surface:12-15 hour ave fcst:ENS=low-res ctl
6:2656558:d=2000010100:VFLX Momentum Flux, V-Component [N/m^2]:surface:12-18 hour ave fcst:ENS=low-res ctl
7:3186532:d=2000010100:VFLX Momentum Flux, V-Component [N/m^2]:surface:18-21 hour ave fcst:ENS=low-res ctl
Upward Short-Wave Radiation Flux
1:0:d=2000010100:USWRF Upward Short-Wave Radiation Flux [W/m^2]:surface:0-3 hour ave fcst:ENS=low-res ctl
2:236614:d=2000010100:USWRF Upward Short-Wave Radiation Flux [W/m^2]:surface:0-6 hour ave fcst:ENS=low-res ctl
3:510055:d=2000010100:USWRF Upward Short-Wave Radiation Flux [W/m^2]:surface:6-9 hour ave fcst:ENS=low-res ctl
4:779534:d=2000010100:USWRF Upward Short-Wave Radiation Flux [W/m^2]:surface:6-12 hour ave fcst:ENS=low-res ctl
5:1070198:d=2000010100:USWRF Upward Short-Wave Radiation Flux [W/m^2]:surface:12-15 hour ave fcst:ENS=low-res ctl
6:1335793:d=2000010100:USWRF Upward Short-Wave Radiation Flux [W/m^2]:surface:12-18 hour ave fcst:ENS=low-res ctl
7:1619012:d=2000010100:USWRF Upward Short-Wave Radiation Flux [W/m^2]:surface:18-21 hour ave fcst:ENS=low-res ctl
Upward Long-Wave Rad. Flux
1:0:d=2000010100:ULWRF Upward Long-Wave Rad. Flux [W/m^2]:top of atmosphere:0-3 hour ave fcst:ENS=low-res ctl
2:396192:d=2000010100:ULWRF Upward Long-Wave Rad. Flux [W/m^2]:top of atmosphere:0-6 hour ave fcst:ENS=low-res ctl
3:794185:d=2000010100:ULWRF Upward Long-Wave Rad. Flux [W/m^2]:top of atmosphere:6-9 hour ave fcst:ENS=low-res ctl
4:1224182:d=2000010100:ULWRF Upward Long-Wave Rad. Flux [W/m^2]:top of atmosphere:6-12 hour ave fcst:ENS=low-res ctl
5:1629407:d=2000010100:ULWRF Upward Long-Wave Rad. Flux [W/m^2]:top of atmosphere:12-15 hour ave fcst:ENS=low-res ctl
6:2075690:d=2000010100:ULWRF Upward Long-Wave Rad. Flux [W/m^2]:top of atmosphere:12-18 hour ave fcst:ENS=low-res ctl
7:2501165:d=2000010100:ULWRF Upward Long-Wave Rad. Flux [W/m^2]:top of atmosphere:18-21 hour ave fcst:ENS=low-res ctl
8
Upward Long-Wave Rad. Flux
1:0:d=2000010100:ULWRF Upward Long-Wave Rad. Flux [W/m^2]:surface:0-3 hour ave fcst:ENS=low-res ctl
2:379050:d=2000010100:ULWRF Upward Long-Wave Rad. Flux [W/m^2]:surface:0-6 hour ave fcst:ENS=low-res ctl
3:774095:d=2000010100:ULWRF Upward Long-Wave Rad. Flux [W/m^2]:surface:6-9 hour ave fcst:ENS=low-res ctl
4:1155969:d=2000010100:ULWRF Upward Long-Wave Rad. Flux [W/m^2]:surface:6-12 hour ave fcst:ENS=low-res ctl
5:1538232:d=2000010100:ULWRF Upward Long-Wave Rad. Flux [W/m^2]:surface:12-15 hour ave fcst:ENS=low-res ctl
6:1925982:d=2000010100:ULWRF Upward Long-Wave Rad. Flux [W/m^2]:surface:12-18 hour ave fcst:ENS=low-res ctl
7:2310651:d=2000010100:ULWRF Upward Long-Wave Rad. Flux [W/m^2]:surface:18-21 hour ave fcst:ENS=low-res ctl
Momentum Flux, U-Component
1:0:d=2000010100:UFLX Momentum Flux, U-Component [N/m^2]:surface:0-3 hour ave fcst:ENS=low-res ctl
2:552611:d=2000010100:UFLX Momentum Flux, U-Component [N/m^2]:surface:0-6 hour ave fcst:ENS=low-res ctl
3:1101462:d=2000010100:UFLX Momentum Flux, U-Component [N/m^2]:surface:6-9 hour ave fcst:ENS=low-res ctl
4:1663743:d=2000010100:UFLX Momentum Flux, U-Component [N/m^2]:surface:6-12 hour ave fcst:ENS=low-res ctl
5:2211408:d=2000010100:UFLX Momentum Flux, U-Component [N/m^2]:surface:12-15 hour ave fcst:ENS=low-res ctl
6:2781189:d=2000010100:UFLX Momentum Flux, U-Component [N/m^2]:surface:12-18 hour ave fcst:ENS=low-res ctl
7
Total Cloud Cover [%]
1:0:d=2000010100:TCDC Total Cloud Cover [%]:entire atmosphere:0-3 hour ave fcst:ENS=low-res ctl
2:755020:d=2000010100:TCDC Total Cloud Cover [%]:entire atmosphere:0-6 hour ave fcst:ENS=low-res ctl
3:1559621:d=2000010100:TCDC Total Cloud Cover [%]:entire atmosphere:6-9 hour ave fcst:ENS=low-res ctl
4:2054574:d=2000010100:TCDC Total Cloud Cover [%]:entire atmosphere:6-12 hour ave fcst:ENS=low-res ctl
5:2570873:d=2000010100:TCDC Total Cloud Cover [%]:entire atmosphere:12-15 hour ave fcst:ENS=low-res ctl
6:3082468:d=2000010100:TCDC Total Cloud Cover [%]:entire atmosphere:12-18 hour ave fcst:ENS=low-res ctl
Sensible Heat Net Flux
1:0:d=2000010100:SHTFL Sensible Heat Net Flux [W/m^2]:surface:0-3 hour ave fcst:ENS=low-res ctl
2:777289:d=2000010100:SHTFL Sensible Heat Net Flux [W/m^2]:surface:0-6 hour ave fcst:ENS=low-res ctl
3:1538479:d=2000010100:SHTFL Sensible Heat Net Flux [W/m^2]:surface:6-9 hour ave fcst:ENS=low-res ctl
4:2316889:d=2000010100:SHTFL Sensible Heat Net Flux [W/m^2]:surface:6-12 hour ave fcst:ENS=low-res ctl
5:3087962:d=2000010100:SHTFL Sensible Heat Net Flux [W/m^2]:surface:12-15 hour ave fcst:ENS=low-res ctl
6:3880391:d=2000010100:SHTFL Sensible Heat Net Flux [W/m^2]:surface:12-18 hour ave fcst:ENS=low-res ctl
Large-Scale Precipitation (non-convective)
1:0:d=2000010100:NCPCP Large-Scale Precipitation (non-convective) [kg/m^2]:surface:0-3 hour acc fcst:ENS=low-res ctl
2:313251:d=2000010100:NCPCP Large-Scale Precipitation (non-convective) [kg/m^2]:surface:0-6 hour acc fcst:ENS=low-res ctl
3:700820:d=2000010100:NCPCP Large-Scale Precipitation (non-convective) [kg/m^2]:surface:6-9 hour acc fcst:ENS=low-res ctl
4:896668:d=2000010100:NCPCP Large-Scale Precipitation (non-convective) [kg/m^2]:surface:6-12 hour acc fcst:ENS=low-res ctl
5:1016498:d=2000010100:NCPCP Large-Scale Precipitation (non-convective) [kg/m^2]:surface:12-15 hour acc fcst:ENS=low-res ctl
6:1225682:d=2000010100:NCPCP Large-Scale Precipitation (non-convective) [kg/m^2]:surface:12-18 hour acc fcst:ENS=low-res ctl
For some of these variables it makes sense (I expect NCPCP to have the same resolution as the other accumulate precip products) but I was not expecting some of the others.
Two are the cases - either these variables are indeed accumulated/averaged with this alternating resolution, or the grib files have an erroneous forecast time width.
For the precip/tmax it's obvious the different resolution (especially for the accumulated variables), in fact, as I reported above, the 2 time series (the one referring to 3hours width forecast period, and the one for the 6hour width data) are one half the other. Below I plot the 2 time series, in particular they are the global average of the files with 2000010100_c00 start time/ensemble - xarray.open_dataset(... engine='pynio') automatically recognizes them as 2 variables. As you can see the accumulated variables have one time series about double the other:
for APCP
for NPCP
Water runoff too...
but for the average variables it's harder to say whether they are averaged on different time periods:
for LHTFL
for TCDC
I think this is all from me now.
from open-data-registry.
ThomasMoreHamill
commented on August 23, 2024
I have done a quick random check (I have not checked all folders) but it appears as all the files related to subset of fields above 700mb are missing for the D>10 days forlders.
In the documentation it says:
For most grib2 files, the data are provided on a grid with a 0.25-degree grid spacing,
archived every 3 hours for the first 10 days of the forecast; beyond 10 days, 0.50 degrees grid
spacing is used. For pressure-level data above 700 hPa, even during the first 10 days of the
forecast, data are saved at 0.5-degree grid spacing in order to conserve space. The grid
proceeds from 90°N to 90°S and from 0E to 359.75 E.I initially thought that because of the similar lat/lon grid they had one grib file for the whole Day 1-16/35 period, but nope, the file in the Day 1-10 folder has only lead times for 3 to 240 hours.
The file names are
hgt_pres_abv700mb_YYYYMMDDHH_ens.grib2 spfh_pres_abv700mb_YYYYMMDDHH_ens.grib2 tmp_pres_abv700mb_YYYYMMDDHH_ens.grib2 ugrd_pres_abv700mb_YYYYMMDDHH_ens.grib2 vgrd_pres_abv700mb_YYYYMMDDHH_ens.grib2 vvel_pres_abv700mb_YYYYMMDDHH_ens.grib2where for
ensi mean the ensemble members,c00andp##The same issue seems to be present for those days with 11 members and forecast times up to Day 35.
from open-data-registry.
Related Issues (20)
- Search Datasets text field not filtering results HOT 1
- Missing NOAA GHCN daily data for 2023 HOT 2
- GEFS v12 AWS missing data HOT 2
- OpenAQ updates HOT 1
- Not able to download Mevadata from AWS HOT 2
- Raster data class categories information HOT 2
- AWS Public blockchain data is no longer being updated. HOT 1
- DOWNLOADING NEX-GDDP-CMIP6 DATA FOR PARTICULAR LATITUDE and LONGITUDE
- Error when trying to read smithsonian-open-access bucket HOT 2
- USGS Landsat SNS topics have changed HOT 2
- Open Data Sponsorship Program, Argo marine dataset HOT 3
- Looking for Data set on brute force attack HOT 1
- GEFS Re-forecast files missing HOT 24
- Download speed fluctuations HOT 1
- Download a specific tiff based on a location point
- Error when trying to read synpuf-omop open-access bucket HOT 1
- Collab page missing dataset in "managed by" group HOT 2
- Access denied for NYC taxi dataset
- Access Denied for BDSP IIIC Sparcnet dataset
- How to check the status of the uploaded dataset, after pull request
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from open-data-registry.