Comments (12)
mh-geus-dk
commented on May 30, 2024
I have been looking on the data from QGIS. The EGDI platform we do not visualise this kind of data using WCS but by accessing the data directly after they (most probably) have been redden into the PostgreSQL database.
The data will by default be made available on the platforms through the web GIS and for download.
We recommend data like this to be uploaded to EGDI and not delivered by services as this kind of services tend to fail over time.
Have you in the project had any thoughts about the thematisation of the data set? I suppose you want to be able to show all the time steps using the same colour scale.
The engine behind EGDI (MapServer) supports WCS services so making the data available as a WCS could be an option. A perhaps more user friendly way to deliver the data could WMS with the year as option. This could deliver the images using a same colour scale across the span of years.
from project-support-wp8.
nmtoken
commented on May 30, 2024
This issue (#297) is not about WCS, this issue is whether the NetCDF output (file) that HOVER have provided and is already available for public download (since July 2018), can be uploaded into the EGDI system and whether it works in whatever tools you have to display it.
from project-support-wp8.
mh-geus-dk
commented on May 30, 2024
Well in the text above Marcus writes "They are considering setting up a WCS to publish this data". The issue is also created at nearly the same time as issue #298 Create WCS based from gridded data in NetCDF format for HOVER WP5. So I supposed they were in some way related.
In QGIS I get this warning when opening the NetCDF.
"This layer appears to have no projection specification." So the there seems to miss some information in the file.
from project-support-wp8.
nmtoken
commented on May 30, 2024
The commonality between the issues is the data not the delivery mechanism.
In #282 we have recorded that EGDI will support
NetCDF Classic format (CDF-1)
NetCDF 64-bit Offset Format (CDF-2)
NetCDF 64-bit Data Format (CDF-5)
NetCDF-4 Format / HDF5
ncdump -k tells me this is data is netCDF-4, so according to #282 it should be supported.
The original question that lead to the list given in #282 was whether there was a requirement for any profiles such as NetCDF CF for NetCDF supplied to EGDI.
We know that this data doesn't conform to NetCDF CF Metadata Conventions.
ncdump -c (header and coordinate information) gives me:
netcdf Nstor {
dimensions:
lon = 718 ;
lat = 279 ;
time = 101 ;
variables:
double lon(lon) ;
lon:units = "degrees_east" ;
lon:long_name = "lon" ;
lon:axis = "X" ;
double lat(lat) ;
lat:units = "degrees_north" ;
lat:long_name = "lat" ;
lat:axis = "Y" ;
double time(time) ;
time:units = "years since 1899" ;
time:long_name = "time" ;
time:axis = "T" ;
double Nstor(time, lat, lon) ;
Nstor:units = "kg N" ;
Nstor:_FillValue = 1.e+32 ;
Nstor:long_name = "nitrate_storage_vadose_zone" ;
// global attributes:
:title = "Global modelled estimates of nitrate stored in the vadose zone" ;
:institution = "British Geological Survey" ;
:references = "Ascott, M. J., Gooddy, D. C., Wang, L., Stuart, M. E., Lewis, M. A., Ward, R. S., & Binley, A. M. (2017). Global patterns of nitrate storage in the vadose zone. Nature Communications, 8(1), 1416." ;
:history = "M.J. Ascott, Thu Jun 14 09:45:32 2018" ;
data:
lon = -178.75, -178.25, -177.75, -177.25, -176.75, -176.25, -175.75,
-175.25, -174.75, -174.25, -173.75, -173.25, -172.75, -172.25, -171.75,
-171.25, -170.75, -170.25, -169.75, -169.25, -168.75, -168.25, -167.75,
-167.25, -166.75, -166.25, -165.75, -165.25, -164.75, -164.25, -163.75,
-163.25, -162.75, -162.25, -161.75, -161.25, -160.75, -160.25, -159.75,
-159.25, -158.75, -158.25, -157.75, -157.25, -156.75, -156.25, -155.75,
-155.25, -154.75, -154.25, -153.75, -153.25, -152.75, -152.25, -151.75,
-151.25, -150.75, -150.25, -149.75, -149.25, -148.75, -148.25, -147.75,
-147.25, -146.75, -146.25, -145.75, -145.25, -144.75, -144.25, -143.75,
-143.25, -142.75, -142.25, -141.75, -141.25, -140.75, -140.25, -139.75,
-139.25, -138.75, -138.25, -137.75, -137.25, -136.75, -136.25, -135.75,
-135.25, -134.75, -134.25, -133.75, -133.25, -132.75, -132.25, -131.75,
-131.25, -130.75, -130.25, -129.75, -129.25, -128.75, -128.25, -127.75,
-127.25, -126.75, -126.25, -125.75, -125.25, -124.75, -124.25, -123.75,
-123.25, -122.75, -122.25, -121.75, -121.25, -120.75, -120.25, -119.75,
-119.25, -118.75, -118.25, -117.75, -117.25, -116.75, -116.25, -115.75,
-115.25, -114.75, -114.25, -113.75, -113.25, -112.75, -112.25, -111.75,
-111.25, -110.75, -110.25, -109.75, -109.25, -108.75, -108.25, -107.75,
-107.25, -106.75, -106.25, -105.75, -105.25, -104.75, -104.25, -103.75,
-103.25, -102.75, -102.25, -101.75, -101.25, -100.75, -100.25, -99.75,
-99.25, -98.75, -98.25, -97.75, -97.25, -96.75, -96.25, -95.75, -95.25,
-94.75, -94.25, -93.75, -93.25, -92.75, -92.25, -91.75, -91.25, -90.75,
-90.25, -89.75, -89.25, -88.75, -88.25, -87.75, -87.25, -86.75, -86.25,
-85.75, -85.25, -84.75, -84.25, -83.75, -83.25, -82.75, -82.25, -81.75,
-81.25, -80.75, -80.25, -79.75, -79.25, -78.75, -78.25, -77.75, -77.25,
-76.75, -76.25, -75.75, -75.25, -74.75, -74.25, -73.75, -73.25, -72.75,
-72.25, -71.75, -71.25, -70.75, -70.25, -69.75, -69.25, -68.75, -68.25,
-67.75, -67.25, -66.75, -66.25, -65.75, -65.25, -64.75, -64.25, -63.75,
-63.25, -62.75, -62.25, -61.75, -61.25, -60.75, -60.25, -59.75, -59.25,
-58.75, -58.25, -57.75, -57.25, -56.75, -56.25, -55.75, -55.25, -54.75,
-54.25, -53.75, -53.25, -52.75, -52.25, -51.75, -51.25, -50.75, -50.25,
-49.75, -49.25, -48.75, -48.25, -47.75, -47.25, -46.75, -46.25, -45.75,
-45.25, -44.75, -44.25, -43.75, -43.25, -42.75, -42.25, -41.75, -41.25,
-40.75, -40.25, -39.75, -39.25, -38.75, -38.25, -37.75, -37.25, -36.75,
-36.25, -35.75, -35.25, -34.75, -34.25, -33.75, -33.25, -32.75, -32.25,
-31.75, -31.25, -30.75, -30.25, -29.75, -29.25, -28.75, -28.25, -27.75,
-27.25, -26.75, -26.25, -25.75, -25.25, -24.75, -24.25, -23.75, -23.25,
-22.75, -22.25, -21.75, -21.25, -20.75, -20.25, -19.75, -19.25, -18.75,
-18.25, -17.75, -17.25, -16.75, -16.25, -15.75, -15.25, -14.75, -14.25,
-13.75, -13.25, -12.75, -12.25, -11.75, -11.25, -10.75, -10.25, -9.75,
-9.25, -8.75, -8.25, -7.75, -7.25, -6.75, -6.25, -5.75, -5.25, -4.75,
-4.25, -3.75, -3.25, -2.75, -2.25, -1.75, -1.25, -0.75, -0.25, 0.25,
0.75, 1.25, 1.75, 2.25, 2.75, 3.25, 3.75, 4.25, 4.75, 5.25, 5.75, 6.25,
6.75, 7.25, 7.75, 8.25, 8.75, 9.25, 9.75, 10.25, 10.75, 11.25, 11.75,
12.25, 12.75, 13.25, 13.75, 14.25, 14.75, 15.25, 15.75, 16.25, 16.75,
17.25, 17.75, 18.25, 18.75, 19.25, 19.75, 20.25, 20.75, 21.25, 21.75,
22.25, 22.75, 23.25, 23.75, 24.25, 24.75, 25.25, 25.75, 26.25, 26.75,
27.25, 27.75, 28.25, 28.75, 29.25, 29.75, 30.25, 30.75, 31.25, 31.75,
32.25, 32.75, 33.25, 33.75, 34.25, 34.75, 35.25, 35.75, 36.25, 36.75,
37.25, 37.75, 38.25, 38.75, 39.25, 39.75, 40.25, 40.75, 41.25, 41.75,
42.25, 42.75, 43.25, 43.75, 44.25, 44.75, 45.25, 45.75, 46.25, 46.75,
47.25, 47.75, 48.25, 48.75, 49.25, 49.75, 50.25, 50.75, 51.25, 51.75,
52.25, 52.75, 53.25, 53.75, 54.25, 54.75, 55.25, 55.75, 56.25, 56.75,
57.25, 57.75, 58.25, 58.75, 59.25, 59.75, 60.25, 60.75, 61.25, 61.75,
62.25, 62.75, 63.25, 63.75, 64.25, 64.75, 65.25, 65.75, 66.25, 66.75,
67.25, 67.75, 68.25, 68.75, 69.25, 69.75, 70.25, 70.75, 71.25, 71.75,
72.25, 72.75, 73.25, 73.75, 74.25, 74.75, 75.25, 75.75, 76.25, 76.75,
77.25, 77.75, 78.25, 78.75, 79.25, 79.75, 80.25, 80.75, 81.25, 81.75,
82.25, 82.75, 83.25, 83.75, 84.25, 84.75, 85.25, 85.75, 86.25, 86.75,
87.25, 87.75, 88.25, 88.75, 89.25, 89.75, 90.25, 90.75, 91.25, 91.75,
92.25, 92.75, 93.25, 93.75, 94.25, 94.75, 95.25, 95.75, 96.25, 96.75,
97.25, 97.75, 98.25, 98.75, 99.25, 99.75, 100.25, 100.75, 101.25, 101.75,
102.25, 102.75, 103.25, 103.75, 104.25, 104.75, 105.25, 105.75, 106.25,
106.75, 107.25, 107.75, 108.25, 108.75, 109.25, 109.75, 110.25, 110.75,
111.25, 111.75, 112.25, 112.75, 113.25, 113.75, 114.25, 114.75, 115.25,
115.75, 116.25, 116.75, 117.25, 117.75, 118.25, 118.75, 119.25, 119.75,
120.25, 120.75, 121.25, 121.75, 122.25, 122.75, 123.25, 123.75, 124.25,
124.75, 125.25, 125.75, 126.25, 126.75, 127.25, 127.75, 128.25, 128.75,
129.25, 129.75, 130.25, 130.75, 131.25, 131.75, 132.25, 132.75, 133.25,
133.75, 134.25, 134.75, 135.25, 135.75, 136.25, 136.75, 137.25, 137.75,
138.25, 138.75, 139.25, 139.75, 140.25, 140.75, 141.25, 141.75, 142.25,
142.75, 143.25, 143.75, 144.25, 144.75, 145.25, 145.75, 146.25, 146.75,
147.25, 147.75, 148.25, 148.75, 149.25, 149.75, 150.25, 150.75, 151.25,
151.75, 152.25, 152.75, 153.25, 153.75, 154.25, 154.75, 155.25, 155.75,
156.25, 156.75, 157.25, 157.75, 158.25, 158.75, 159.25, 159.75, 160.25,
160.75, 161.25, 161.75, 162.25, 162.75, 163.25, 163.75, 164.25, 164.75,
165.25, 165.75, 166.25, 166.75, 167.25, 167.75, 168.25, 168.75, 169.25,
169.75, 170.25, 170.75, 171.25, 171.75, 172.25, 172.75, 173.25, 173.75,
174.25, 174.75, 175.25, 175.75, 176.25, 176.75, 177.25, 177.75, 178.25,
178.75, 179.25, 179.75 ;
lat = 83.25, 82.75, 82.25, 81.75, 81.25, 80.75, 80.25, 79.75, 79.25, 78.75,
78.25, 77.75, 77.25, 76.75, 76.25, 75.75, 75.25, 74.75, 74.25, 73.75,
73.25, 72.75, 72.25, 71.75, 71.25, 70.75, 70.25, 69.75, 69.25, 68.75,
68.25, 67.75, 67.25, 66.75, 66.25, 65.75, 65.25, 64.75, 64.25, 63.75,
63.25, 62.75, 62.25, 61.75, 61.25, 60.75, 60.25, 59.75, 59.25, 58.75,
58.25, 57.75, 57.25, 56.75, 56.25, 55.75, 55.25, 54.75, 54.25, 53.75,
53.25, 52.75, 52.25, 51.75, 51.25, 50.75, 50.25, 49.75, 49.25, 48.75,
48.25, 47.75, 47.25, 46.75, 46.25, 45.75, 45.25, 44.75, 44.25, 43.75,
43.25, 42.75, 42.25, 41.75, 41.25, 40.75, 40.25, 39.75, 39.25, 38.75,
38.25, 37.75, 37.25, 36.75, 36.25, 35.75, 35.25, 34.75, 34.25, 33.75,
33.25, 32.75, 32.25, 31.75, 31.25, 30.75, 30.25, 29.75, 29.25, 28.75,
28.25, 27.75, 27.25, 26.75, 26.25, 25.75, 25.25, 24.75, 24.25, 23.75,
23.25, 22.75, 22.25, 21.75, 21.25, 20.75, 20.25, 19.75, 19.25, 18.75,
18.25, 17.75, 17.25, 16.75, 16.25, 15.75, 15.25, 14.75, 14.25, 13.75,
13.25, 12.75, 12.25, 11.75, 11.25, 10.75, 10.25, 9.75, 9.25, 8.75, 8.25,
7.75, 7.25, 6.75, 6.25, 5.75, 5.25, 4.75, 4.25, 3.75, 3.25, 2.75, 2.25,
1.75, 1.25, 0.75, 0.25, -0.25, -0.75, -1.25, -1.75, -2.25, -2.75, -3.25,
-3.75, -4.25, -4.75, -5.25, -5.75, -6.25, -6.75, -7.25, -7.75, -8.25,
-8.75, -9.25, -9.75, -10.25, -10.75, -11.25, -11.75, -12.25, -12.75,
-13.25, -13.75, -14.25, -14.75, -15.25, -15.75, -16.25, -16.75, -17.25,
-17.75, -18.25, -18.75, -19.25, -19.75, -20.25, -20.75, -21.25, -21.75,
-22.25, -22.75, -23.25, -23.75, -24.25, -24.75, -25.25, -25.75, -26.25,
-26.75, -27.25, -27.75, -28.25, -28.75, -29.25, -29.75, -30.25, -30.75,
-31.25, -31.75, -32.25, -32.75, -33.25, -33.75, -34.25, -34.75, -35.25,
-35.75, -36.25, -36.75, -37.25, -37.75, -38.25, -38.75, -39.25, -39.75,
-40.25, -40.75, -41.25, -41.75, -42.25, -42.75, -43.25, -43.75, -44.25,
-44.75, -45.25, -45.75, -46.25, -46.75, -47.25, -47.75, -48.25, -48.75,
-49.25, -49.75, -50.25, -50.75, -51.25, -51.75, -52.25, -52.75, -53.25,
-53.75, -54.25, -54.75, -55.25, -55.75 ;
time = 1900, 1901, 1902, 1903, 1904, 1905, 1906, 1907, 1908, 1909, 1910,
1911, 1912, 1913, 1914, 1915, 1916, 1917, 1918, 1919, 1920, 1921, 1922,
1923, 1924, 1925, 1926, 1927, 1928, 1929, 1930, 1931, 1932, 1933, 1934,
1935, 1936, 1937, 1938, 1939, 1940, 1941, 1942, 1943, 1944, 1945, 1946,
1947, 1948, 1949, 1950, 1951, 1952, 1953, 1954, 1955, 1956, 1957, 1958,
1959, 1960, 1961, 1962, 1963, 1964, 1965, 1966, 1967, 1968, 1969, 1970,
1971, 1972, 1973, 1974, 1975, 1976, 1977, 1978, 1979, 1980, 1981, 1982,
1983, 1984, 1985, 1986, 1987, 1988, 1989, 1990, 1991, 1992, 1993, 1994,
1995, 1996, 1997, 1998, 1999, 2000 ;
}
My version of QGIS interprets the CRS as CRS:84, and the data displays correctly georeferenced against another dataset.
from project-support-wp8.
bgsmase
commented on May 30, 2024
I will see what happens when I try uploading this file to EGDI test platform.
from project-support-wp8.
nmtoken
commented on May 30, 2024
Doesn't seem possible to test upload of anything other than GeoPackage or shapefile at the moment using the dev EGDI Admin tools
from project-support-wp8.
mh-geus-dk
commented on May 30, 2024
It also work for PDF files, Document by DOI and pictures.
from project-support-wp8.
nmtoken
commented on May 30, 2024
@mh-geus-dk you are correct, I should have been more precise. :)
For data sets, only geopackage and shapefiles appear possible. I tried too using the search and create option, but that appears to only look at pre-loaded data (I guess already in a database?)
For documents and images etc PDF, JPG and CSV are supported for upload or DOI by reference.
For this issue (#297) about uploading a NetCDF-4 file (a data set), we trying to test if we could upload it using the EGDI tools, but current status is that we can't; is that correct?
from project-support-wp8.
mh-geus-dk
commented on May 30, 2024
Yes, that is correct. There is currently no upload facility for NetCDF's. The administration module is more than an upload facility, when spatial data are uploaded, they are also transferred to the EGDI database, and configured as layers on the portal. Right now, only about two projects (a third might be coming up) will deliver data on NetCDF format. Therefore, we start streamlining the functionality for shapefiles and GeoPackages. Depending on the amount of project and the time available we will either develop an upload functionality for NetCDF (and GeoTIFF) or do it by hand.
from project-support-wp8.
bgsmase
commented on May 30, 2024
Similarly to #278 we don't currently have a route for gridded data to be uploaded to the EGDI Portal. Can @mh-geus-dk advise whether science projects should send them test files or wait for facility to be developed.
from project-support-wp8.
bgsmase
commented on May 30, 2024
Presumably EGDI platform will have same problems as in #298 if the NetCDF file does not have CF-conventions. Can @mh-geus-dk confirm whether or not EGDI will be able to handle the NetCDF file without the CF-conventions?
from project-support-wp8.
bgsmase
commented on May 30, 2024
From email communication:
There is no upload for NetCDF and there will not be developed one. NetCDF must be sent to me (Martin Hansen). We will load the "layers" into the PosgreSQL database and store them as rasters. NetCDF should only be used if the file has several "layers" belonging to the same dataset like the RESOURCE dataset that describes the hydrology of Europe in a 10 km grid. If the projects wants to deliver standalone maps they should be delivered as GeoTIFF.
This issue is being closed as project has sent test data directly to EGDI
from project-support-wp8.
Related Issues (20)
- WMS of EGDI WMS/WFS Service fails INSPIRE validation HOT 1
- error uploading shapefile HOT 28
- details table in EGDI map HOT 8
- Upload failed TIF file HOT 3
- Re: Tactic WP3: Upload of data to Egdi HOT 2
- Re[2]: Tactic WP3: Upload of data to Egdi HOT 1
- EDGI webmap display HOT 13
- 3DGEO-EU raster data in EGDI test environment HOT 4
- Geopackage upload not working on final EGDI Admin HOT 4
- Mapserver Style HOT 6
- Scale rules in EGDI HOT 25
- Failed to open MS_ERRORFILE (production platform) HOT 1
- Raster style is not applied HOT 1
- Error: Unable to access file (Development platform)
- Error: unable to edit / delete layers from owner postgres HOT 11
- Import with style from geopackage HOT 3
- WFS layer doesn't work on QGIS HOT 4
- Search for a report - imports of raw materials/minerals from Russia into the EU HOT 1
- Data Download EGDI HOT 10
- Extract Maps from EGDI HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from project-support-wp8.