Hi,
I would like some help with a problem I am having with ADAGUC Server.
I am unable to visualise temperature data from a multi temporal NetCDF file that is served over a THREDDS Data Server v4.6.13 (TDS) via the OpenDAP protocol. The NetCDF file is generated from two separate GRIB version 1 files that originated from the ECMWF. They are:
D1D12191200122309001
D1D12191200122312001
They contain data from the same model run date and time (19/12/2018 12:00) but for two different forecast times (23/12/2018 9am and 12pm). If I convert any one of them to a NetCDF file using the eccodes grib_to_netcdf terminal command, and serve them individually over TDS. They will visualise as expected.
However if I concatenate them using the following terminal command:
cat D1D12191200122309001 D1D12191200122312001 > ECMWF_HRES_3HR_AU_VIC_20181219_1200.grib1
Then convert the resulting file to NetCDF before serving it over TDS. The result will not visualise in the ADAGUC Viewer.
If I open the same OpenDAP link in Panoply it will visualise the two time dimensions with no issues. If I take the problematic NetCDF file and place it in the ADAGUC data directory and serve it as a file rather than an OpenDAP link it will visualise the two time dimensions with no issues. The problem is only when this multi time dimension file is served over OpenDAP, in that case ADAGUC will not visualise any of the time slices.
Can someone help me with this?
I am using the latest openearth/adaguc-server docker hub image. A GetCapabilities request returns a version of 2.0.37.
My TDS catalog featureCollection configuration:
<featureCollection name="HRES 3hr Australia VIC" featureType="FMRC" path="NetCDF/ECMWF/HRES/3HR/AU/VIC">
<metadata inherited="true">
<serviceName>all</serviceName>
</metadata>
<collection spec="/usr/local/tomcat/content/thredds/public/ecmwf_nc/ECMWF_HRES_3HR_AU_VIC_#yyyyMMdd_HHmm#.nc$"
name="ECMWF_NC_HRES_3HR_AU_VIC"
timePartition="file"
olderThan="5" />
</featureCollection>
My ADAGUC configuration file:
<?xml version="1.0" encoding="UTF-8" ?>
<Configuration>
<Layer type="database">
<FilePath>http://thredds:8080/thredds/dodsC/NetCDF/ECMWF/HRES/3HR/AU/VIC/files/ECMWF_HRES_3HR_AU_VIC_20181219_1200.nc</FilePath>
<Variable>t2m</Variable>
<Dimension name="time" interval="PT3H" default="max">time</Dimension>
</Layer>
</Configuration>
The GetCapabilities request has the correct layer 't2m' listed and the correct time range:
<Dimension name="time" units="ISO8601" default="2018-12-23T12:00:00Z" multipleValues="1" nearestValue="0" current="1">2018-12-23T09:00:00Z/2018-12-23T12:00:00Z/PT3H</Dimension>
The raw request looks like this:
http://localhost:8090//adaguc-services/adagucserver?DATASET=ecmwf_2&SERVICE=WMS&VERSION=1.3.0&REQUEST=GetMap&LAYERS=t2m&WIDTH=1035&HEIGHT=321&CRS=EPSG%3A4326&BBOX=-49.17696610818382,91.662023552629,-17.282936741172634,194.49791263317908&STYLES=auto%2Fnearest&FORMAT=image/png&TRANSPARENT=TRUE&time=2018-12-23T09%3A00%3A00Z&0.3807920303767882
It returns this error message:
<?xml version='1.0' encoding="ISO-8859-1" standalone="no" ?>
<ServiceExceptionReport version="1.3.0" xmlns="http://www.opengis.net/ogc" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.opengis.net/ogc http://schemas.opengis.net/wms/1.3.0/exceptions_1_3_0.xsd">
<ServiceException code="OperationNotSupported">
nc_get_vara (typeconversion): : NetCDF: Access failure;
Problem with variable t2m of type CDF_DOUBLE (requested CDF_DOUBLE):;
nc_get_var: : NetCDF: Access failure;
Unable to read data for variable t2m;
Unable to read data for variable t2m in file http://thredds:8080/thredds/dodsC/NetCDF/ECMWF/HRES/3HR/AU/VIC/files/ECMWF_HRES_3HR_AU_VIC_20181219_1200.nc;
Could not open file: http://thredds:8080/thredds/dodsC/NetCDF/ECMWF/HRES/3HR/AU/VIC/files/ECMWF_HRES_3HR_AU_VIC_20181219_1200.nc;
warpImage for layer t2m failed;
Unable to convert datasource t2m to image;
Returning from line 1934;
WMS GetMap Request failed;
</ServiceException>
</ServiceExceptionReport>
Output from running adaguc-server-updatedatasets.sh:
*** Starting update for /data/adaguc-datasets/ecmwf_2
[D: CRequest.cpp, 116 in CRequest] Dataset name based on passed configfile is [ecmwf_2]
[D: CAutoResource.cpp, 81 in CAutoResource] Found dataset /data/adaguc-datasets//ecmwf_2.xml
[D: CDBFileScanner.cpp, 938 in CDBFileScanner] *** Starting update layer 't2m' ***
[D: CDBFileScanner.cpp, 940 in CDBFileScanner] Using path [http://thredds:8080/thredds/dodsC/NetCDF/ECMWF/HRES/3HR/AU/VIC/files/ECMWF_HRES_3HR_AU_VIC_20181219_1200.nc], filter [] and tailpath []
[D: CDBFileScanner.cpp, 117 in CDBFileScanner] Checking dim [time]
[D: CDataReader.cpp, 1298 in CDataReader] Warning no standard name given for dimension time, using variable name instead.
[D: CDataReader.cpp, 1298 in CDataReader] Warning no standard name given for dimension time, using variable name instead.
[D: CDBFileScanner.cpp, 366 in CDBFileScanner] Found dimension 0 with name time of type 2, istimedim: [1]
[D: CDBFileScanner.cpp, 388 in CDBFileScanner] Marking table done for dim 'time' with table 't20190326t044525769_6afjqayh5htc9xatr7oe'.
[D: CDBFileScanner.cpp, 395 in CDBFileScanner] Found 1 files
[D: CDBFileScanner.cpp, 831 in CDBFileScanner] The database contains 1 files
[D: CDBFileScanner.cpp, 841 in CDBFileScanner] Comparing lists
[D: CDBFileScanner.cpp, 844 in CDBFileScanner] Found 0 files in DB which are missing
[D: CDBFileScanner.cpp, 1014 in CDBFileScanner] *** Finished update layer 't2m' ***
[D: CRequest.cpp, 3446 in CRequest] ***** Finished DB Update *****
[D: CDirReader.cpp, 394 in CCachedDirReader] Cleaningup dirReaders
The log file entries relating to the failed request:
[D: CRequest.cpp, 817 in CRequest] 2019-03-26T08:39:54Z/385 WMS GETMAP (0) t2m
[D: CTime.cpp, 172 in CTime] 2019-03-26T08:39:54Z/385 Initializing CTime with seconds since 1970
[D: CRequest.cpp, 1469 in CRequest] 2019-03-26T08:39:54Z/385 Using maxquerylimit 512
[D: CDataSource.cpp, 1234 in CDataSource] 2019-03-26T08:39:54Z/385 _currentStyle == NULL
[D: CAutoConfigure.cpp, 495 in CAutoConfigure] 2019-03-26T08:39:54Z/385 Loading header [http://thredds:8080/thredds/dodsC/NetCDF/ECMWF/HRES/3HR/AU/VIC/files/ECMWF_HRES_3HR_AU_VIC_20181219_1200.nc]
[D: CDataReader.cpp, 751 in CDataReader] 2019-03-26T08:39:54Z/385 Working on [http://thredds:8080/thredds/dodsC/NetCDF/ECMWF/HRES/3HR/AU/VIC/files/ECMWF_HRES_3HR_AU_VIC_20181219_1200.nc]
[D: CDataReader.cpp, 573 in CDataReader] 2019-03-26T08:39:54Z/385 Assuming that for variable t2m the x dim equals longitude and the y dim equals latitude based on their position and name.
[D: CDataReader.cpp, 599 in CDataReader] 2019-03-26T08:39:54Z/385 Using variable longitude as X variable and variable latitude as Y variable.
[D: CDataReader.cpp, 626 in CDataReader] 2019-03-26T08:39:54Z/385 No stride defined in the RenderSettings, using a default stride of 1.
[W: CDataReader.cpp, 179 in CDataReader] 2019-03-26T08:39:54Z/385 No correct projection found, using by default the geographic coordinate system (latitude and longitude).
[D: CDataReader.cpp, 216 in CDataReader] 2019-03-26T08:39:54Z/385 Using the geographic coordinate system (latitude and longitude)
[E: CCDFNetCDFIO.cpp, 255 in class NetCDFReader] nc_get_vara (typeconversion): : NetCDF: Access failure
2019-03-26T08:39:54Z/385 [E: CCDFNetCDFIO.cpp, 269 in NetCDFReader] 2019-03-26T08:39:54Z/385 Problem with variable t2m of type CDF_DOUBLE (requested CDF_DOUBLE):
[E: CCDFNetCDFIO.cpp, 270 in class NetCDFReader] nc_get_var: : NetCDF: Access failure
2019-03-26T08:39:54Z/385 [E: CCDFVariable.cpp, 351 in Variable] 2019-03-26T08:39:54Z/385 Unable to read data for variable t2m
[E: CDataReader.cpp, 1008 in CDataReader] 2019-03-26T08:39:54Z/385 Unable to read data for variable t2m in file http://thredds:8080/thredds/dodsC/NetCDF/ECMWF/HRES/3HR/AU/VIC/files/ECMWF_HRES_3HR_AU_VIC_20181219_1200.nc
[D: CDataReader.cpp, 1011 in CDataReader] 2019-03-26T08:39:54Z/385 time 1 1 1
[D: CDataReader.cpp, 1011 in CDataReader] 2019-03-26T08:39:54Z/385 latitude 0 81 1
[D: CDataReader.cpp, 1011 in CDataReader] 2019-03-26T08:39:54Z/385 longitude 0 181 1
[E: CImageDataWriter.cpp, 1459 in CImageDataWriter] 2019-03-26T08:39:54Z/385 Could not open file: http://thredds:8080/thredds/dodsC/NetCDF/ECMWF/HRES/3HR/AU/VIC/files/ECMWF_HRES_3HR_AU_VIC_20181219_1200.nc
[E: CImageDataWriter.cpp, 2011 in CImageDataWriter] 2019-03-26T08:39:54Z/385 warpImage for layer t2m failed
[E: CRequest.cpp, 1933 in CRequest] 2019-03-26T08:39:54Z/385 Unable to convert datasource t2m to image
[E: CRequest.cpp, 2185 in CRequest] 2019-03-26T08:39:54Z/385 Returning from line 1934
[E: CRequest.cpp, 3144 in CRequest] 2019-03-26T08:39:54Z/385 WMS GetMap Request failed
This zip file contains three folders 'nc_file1', 'nc_file2', and 'nc_file1_and_file2'. The problematic NetCDF file is located 'nc_file1_and_file2/ECMWF_HRES_3HR_AU_VIC_20181219_1200.nc' and is the result of concatenating the two grib files and converting them to NetCDF. The file by the same name in nc_file1/ and nc_file2/ is just each of those GRIB files converted to NetCDF and included for the purpose of demonstrating that each of those files on their own work just fine.
adaguc_opendap_problem_files.zip