Comments (6)
JohnLCaron
commented on June 19, 2024
Hi Robert:
The way that works is that a Grib1Customizer customizes behavior when its
not standard WMO GRib1. So if you look at NcepTables you see it does handle
timeRangeIndicator=130. OTOH, if its copernicus, its probably not NCEP so
you'll have to track down the correct tables, since theres no reason to
assume that they are using the same definition as NCEP. Which is why we
cant just "look around" for some table with timeRangeIndicator=130.
In any case, you have to look at the grib record, figure out what table it
needs, then configure the code to recognize what table to use for that file:
https://docs.unidata.ucar.edu/netcdf-java/current/userguide/grib_tables.html#grib-1-table-handling
If you want to post the file, Im sure the Unidata folks can help. If
its huge, and you know how (ToolsUI can do it, but its obscure), you can
extract a sample record that shows the problem.
Regards,
John
…On Sun, Oct 8, 2023 at 8:15 PM R. Schmunk ***@***.***> wrote:
Versions impacted by the bug
I am working with a devo copy of netCDF-Java 5.3.4, snapshot downloaded
early August.
What went wrong?
A user pinged me because netCDF-Java (and thus Panoply) would not open a
GRIB-1 dataset, with an exception thrown because the data use time range
indicator 130. See stack trace below.
In looking at the netCDF-Java code, I see that Grib1ParamTime rejects and
time range indicator with value > 125. However, in looking for more info
about this, I found that there are at least thirteen time range indicators
greater than that limit, as shown on e.g. this NOAA/NCEP webpage
<https://www.nco.ncep.noaa.gov/pmb/docs/on388/table5.html>
Although the above link is to a NOAA webpage, the user originally
downloaded the data from an EU Copernicus site
<https://cds.climate.copernicus.eu/cdsapp#!/dataset/reanalysis-era5-single-levels-monthly-means?tab=overview>.
Unfortunately, I don't know what specific field input he used there to
acquire the problem dataset. Instead he provided me a copy of the file.
Is there any chance that netCDF-Java developers can look into this in the
foreseeable future? I haven't looked at the code in Grib1ParamTime
closely enough to see if I could puzzle it out for myself, and bugs in
another app are calling for my immediate attention.
Relevant stack trace
Caused by: java.lang.IllegalArgumentException: PDS: Unknown Time Range Indicator 130
at ucar.nc2.grib.grib1.Grib1ParamTime.<init>(Grib1ParamTime.java:399)
at ucar.nc2.grib.grib1.tables.Grib1Customizer.getParamTime(Grib1Customizer.java:118)
at ucar.nc2.grib.grib1.Grib1Record.getParamTime(Grib1Record.java:124)
at ucar.nc2.grib.collection.Grib1CollectionBuilder.filterIntervals(Grib1CollectionBuilder.java:160)
at ucar.nc2.grib.collection.Grib1CollectionBuilder.makeGroups(Grib1CollectionBuilder.java:107)
at ucar.nc2.grib.collection.GribCollectionBuilder.createMultipleRuntimeCollections(GribCollectionBuilder.java:128)
at ucar.nc2.grib.collection.GribCollectionBuilder.createIndex(GribCollectionBuilder.java:120)
at ucar.nc2.grib.collection.GribCdmIndex.openGribCollectionFromDataFile(GribCdmIndex.java:821)
at ucar.nc2.grib.collection.GribCdmIndex.openGribCollectionFromDataFile(GribCdmIndex.java:804)
at ucar.nc2.grib.collection.GribCdmIndex.openGribCollectionFromRaf(GribCdmIndex.java:774)
at ucar.nc2.grib.collection.GribIosp.build(GribIosp.java:183)
at ucar.nc2.NetcdfFiles.build(NetcdfFiles.java:815)
at ucar.nc2.NetcdfFiles.open(NetcdfFiles.java:754)
at ucar.nc2.NetcdfFiles.open(NetcdfFiles.java:276)
Relevant log messages
*No response*
If you have an example file that you can share, please attach it to this
issue.
If so, may we include it in our test datasets to help ensure the bug does
not return once fixed?
Note: the test datasets are publicly accessible without restriction.
Yes
Code of Conduct
- I agree to follow the UCAR/Unidata Code of Conduct
—
Reply to this email directly, view it on GitHub
<#1246>, or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAEVZQCZBVNMJ5Z6I3PIYDTX6NM5XAVCNFSM6AAAAAA5YD2PESVHI2DSMVQWIX3LMV43ASLTON2WKOZRHEZTEMJZGQZDSNI>
.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***>
from netcdf-java.
rschmunk
commented on June 19, 2024
@JohnLCaron, Depends on your definition of huge. The sample file I was provided is 170 MB.
from netcdf-java.
JohnLCaron
commented on June 19, 2024
hmm i dont see the file
from netcdf-java.
rschmunk
commented on June 19, 2024
@JohnLCaron, I didn't include a link in my original post about this. Here's a Dropbox link.
from netcdf-java.
rschmunk
commented on June 19, 2024
And today I get a complaint from someone downloading data from the same Copernicus website that they are getting an "Unknown Time Range Indicator 128" exception trying to open the download. Sorry, no sample file provided this time.
from netcdf-java.
JohnLCaron
commented on June 19, 2024
Its clear that ECMWF has started using some non-standard grib1 time
parameters.
Someone has to go find out what they are, and perhaps also check on whether
the parameter tables have changed.
Its fair to ask your customer to do so, presumably they are the ones who
care, and perhaps might know enough to tell if they are right.
Then send that information into Unidata support, who should be able to add
the code to implement it.
This will happen for as long as you try to support GRIB (esp GRIB1), so you
could try to standardize it, and point people at a FAQ or something. Or you
could stop trying to support GRIB1.
We did make a deliberate decision to let things break, rather than allow
incorrect data. But we could add a debug flag allowing people to see the
data, and set the time range to something like Jan 1, 1970.
It also occurs to me that you are looking at a legacy product for
which there is a newer, maybe GRIB2 equivalent.
…On Wed, Oct 11, 2023 at 9:30 AM R. Schmunk ***@***.***> wrote:
And today I get a complaint from someone downloading data from the same
Copernicus website that they are getting an "Unknown Time Range Indicator
128" exception trying to open the download. Sorry, no sample file provided
this time.
—
Reply to this email directly, view it on GitHub
<#1246 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAEVZQH7SO4RWOAXXETYPTDX623PPANCNFSM6AAAAAA5YD2PEQ>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
from netcdf-java.
Related Issues (20)
- [5.5.4-SNAPSHOT]: Checksum32 throwing checksum exception accessing var in H5 file HOT 2
- [5.5.4+]: CF 1.9 and Geostationary projection HOT 5
- [5.5-SNAPSHOT]: WMS error: Coordinate values must increase or decrease monotonically HOT 9
- Add Standardizer preprocessor to netcdf dataset enhancements HOT 5
- [5.5.3]: java.io.EOFException while reading some BUFR files (solution: just update the bufr tables) HOT 2
- Allow coverage axes with same name in different case
- Allow coverages with non-canonical time/ensemble axis order HOT 1
- [5.5.3]: UnsupportedOperationException while reading a variable from BUFR, but no error in NCdumpW HOT 2
- NoSuchMethodError for NetCDF Builder [VERSION]: 5.5.3 HOT 1
- Fix S3 NcML datasets
- DAP4: Session and Request Header Support for netcdf-java HOT 2
- [VERSION]: 5.5.2 Exception thrown when accessing char data from NetcdfDatasets.openNcmlDataset
- Exception when opening twoD grib collection as a coverage
- netcdfAll-5.5.3.jar Unable to release handle after reading file HOT 1
- Build failed due to `gretty:2.0.1` HOT 3
- Some NetCDF data cannot be read by `ucar.nc2.dataset.NetcdfDataset` HOT 7
- DOC: Oudated link to ncml XSD in NCML documentation HOT 1
- Unlimited dimension cannot work without a specified length HOT 3
- Lambert conformal projection does not respect units of the false northing/easting
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from netcdf-java.