trondkr / era5-roms Goto Github PK
View Code? Open in Web Editor NEWDownload, and create ERA5 data forcing for ROMS
Download, and create ERA5 data forcing for ROMS
When testing the current output of this package for an Arctic ROMS model it seems that ROMS has either 1) a problem with one of the variables that prevent seasonality to emerge in the ROMS solution 2) the internal interpolation in ROMS is broken.. This needs to be further investigated
Hi Trond,
Thanks for this great piece of code, but I came into some issue that ROMS model (I'm using COAWST actually) can't read the forcing data. It seems ROMS doesn't like the latitude to be from 90 to -90, I modified "ECMWF_convert_to_ROMS.py" with following:
vnc = f1.createVariable('lat', 'd', 'lat', fill_value=fill_val)
vnc.long_name = 'Latitude'
vnc.units = 'degree_north'
vnc.standard_name = 'latitude'
vnc[:] = latitude[::-1]
vnc = f1.createVariable(metadata['roms_name'], 'd', (metadata['time_name'], 'lat', 'lon'), fill_value=fill_val)
vnc.long_name = metadata["name"]
vnc.standard_name = metadata["short_name"]
vnc.coordinates = "lon lat {}".format(metadata['time_name'])
vnc.units = var_units
vnc.missing_value = fill_val
vnc[:, :, :] = data_array[:,::-1,:]
and ROMS is running. Not sure if it's an error with ROMS or COAWST, but I think it may be worth bringing it to you.
Cheers,
Yi
Hi Trondkr. I'm using your script to download the ERA5 data to use in my ROMS domain. However, the download is apparently doing very slowly, and after some time the process is killed unexpectedly, at different times (in my failed attempts to download and process the data). It can be something wrong with the solicitation? do you know if we have a limit of downloads of ERA5 data?
I am trying to download:
The download starts, it is downloaded 1998 u_wind, v_wind, they are converted to ROMS, but it stops when it finishes the 2m-temperature, or before...
Hi Trond,
Could you please check whether there's a bug in ECMWF_tools.py. I specified both the start and end dates to be 2015 in ECMWF_querry.py, but when the download completed it started to grab data for 2016. Thanks again for this tool!
Regards,
Joseph K. Ansong
Hello sir,
I am trying to use your script ERA5-ROMS. Thanks a lot for your such a generous contribution. Although I had an issue. After running the toolbox using according to Run the toolbox section of Readme.md file, I am not sure where the file is being saved(i mean its location and filename). May be it's a bug or maybe I am unable to use it in a proper way. Also is there a way that I can use this script to create forcing on my own grid present in netcdf format ?
Thanks a lot for your help in advance @trondkr
Thanks for this great project, but I encountered a small issue while using the code.
In the ECMWF_convert_to_ROMS.py, line 99, function write_to_ROMS_netcdf_file is defined and netcdf_file is used as a parameter, which gives the name of the netcdf file to be written in line 120.
However, in line 120,the code is netcdf_roms_filename = f"{out_filename[0:-3]}_roms.nc", which use out_filename instead of netcdf_file, causing the problem: NameError: name 'out_filename' is not defined.
I wonder if the code may need to be modified. Looking for reply. Thanks
I try to download data for ROMS Rutgers. I can use it but I can download 10 variables. My other problem is the code does not change the time properly and my header file is for 1900. When I start running the model, I faced a problem.
netcdf era5_msnlwrf_year_2014 {
dimensions:
longitude = 36 ;
latitude = 29 ;
time = 4380 ;
variables:
float longitude(longitude) ;
longitude:units = "degrees_east" ;
longitude:long_name = "longitude" ;
float latitude(latitude) ;
latitude:units = "degrees_north" ;
latitude:long_name = "latitude" ;
int time(time) ;
time:units = "hours since 1948-01-01 00:00:00.0" ;
time:long_name = "time" ;
time:calendar = "gregorian" ;
short msnlwrf(time, latitude, longitude) ;
msnlwrf:scale_factor = 0.00339088604676628 ;
msnlwrf:add_offset = -91.074113655914 ;
msnlwrf:_FillValue = -32767s ;
msnlwrf:missing_value = -32767s ;
msnlwrf:units = "W m**-2" ;
msnlwrf:long_name = "Mean surface net long-wave radiation flux" ;
// global attributes:
:Conventions = "CF-1.6" ;
:history = "2022-06-04 09:23:44 GMT by grib_to_netcdf-2.24.3: /opt/ecmwf/mars-client/bin/grib_to_netcdf -S param -o /cache/data1/adaptor.mars.internal-1654334612.669915-11911-5-14831270-9472-4c48-9faf-6bc9abe5ac36.nc /cache/tmp/14831270-9472-4c48-9faf-6bc9abe5ac36-adaptor.mars.internal-1654334292.9382155-11911-7-tmp.grib" ;
data:
longitude = 12, 12.25, 12.5, 12.75, 13, 13.25, 13.5, 13.75, 14, 14.25, 14.5,
14.75, 15, 15.25, 15.5, 15.75, 16, 16.25, 16.5, 16.75, 17, 17.25, 17.5,
17.75, 18, 18.25, 18.5, 18.75, 19, 19.25, 19.5, 19.75, 20, 20.25, 20.5,
20.75 ;
latitude = 45.9, 45.65, 45.4, 45.15, 44.9, 44.65, 44.4, 44.15, 43.9, 43.65,
43.4, 43.15, 42.9, 42.65, 42.4, 42.15, 41.9, 41.65, 41.4, 41.15, 40.9,
40.65, 40.4, 40.15, 39.9, 39.65, 39.4, 39.15, 38.9 ;
time = 999312, 999314, 999316, 999318, 999320, 999322, 999324, 999326,
999328, 999330, 999332, 999334, 999336, 999338, 999340, 999342, 999344,
999346, 999348, 999350, 999352, 999354, 999356, 999358, 999360, 999362,
999364, 999366, 999368, 999370, 999372, 999374, 999376, 999378, 999380,
999382, 999384, 999386, 999388, 999390, 999392, 999394, 999396, 999398,
999400, 999402, 999404, 999406, 999408, 999410, 999412, 999414, 999416,
999418, 999420, 999422, 999424, 999426, 999428, 999430, 999432, 999434,
999436, 999438, 999440, 999442, 999444, 999446, 999448, 999450, 999452,
Great code!But I had a problem when I ran it: the download was too slow (20~100kb/s). I simply modified the code, now I can call Internet Download Manager (IDM) for multi-threaded download, the speed can reach 8mb/s. But the whole process of data download and processing is not elegant enough (need to run twice and comment out some statements). I wonder if the author has any plans to add a multi-threaded download module to the code. Thanks.
Hello @trondkr sorry for the bother.
I don't understand what is happening, I am setting
self.start_year = 2017
self.end_year = 2020
self.project = "IEA"
with
return {'IEA': '80/-40/45/15',
'A20': '90/-180/40/180'}[project]
and I am getting the following:
2023-02-10 12:52:12,079 INFO Request is failed
2023-02-10 12:52:12,079 ERROR Message: the request you have submitted is not valid
2023-02-10 12:52:12,080 ERROR Reason:
request['verbose'][0]: False is not of type 'string'
2023-02-10 12:52:12,080 ERROR Traceback (most recent call last):
2023-02-10 12:52:12,080 ERROR File "/opt/cdstoolbox/cdscompute/cdscompute/cdshandlers/services/handler.py", line 59, in handle_request
2023-02-10 12:52:12,080 ERROR result = cached(context.method, proc, context, context.args, context.kwargs)
2023-02-10 12:52:12,080 ERROR File "/opt/cdstoolbox/cdscompute/cdscompute/caching.py", line 108, in cached
2023-02-10 12:52:12,080 ERROR result = proc(context, *context.args, **context.kwargs)
2023-02-10 12:52:12,080 ERROR File "/opt/cdstoolbox/cdscompute/cdscompute/services.py", line 124, in call
2023-02-10 12:52:12,080 ERROR return p(*args, **kwargs)
2023-02-10 12:52:12,080 ERROR File "/opt/cdstoolbox/cdscompute/cdscompute/services.py", line 60, in call
2023-02-10 12:52:12,080 ERROR return self.proc(context, *args, **kwargs)
2023-02-10 12:52:12,080 ERROR File "/home/cds/cdsservices/services/mars/mars.py", line 48, in internal
2023-02-10 12:52:12,080 ERROR return mars(context, request, **kwargs)
2023-02-10 12:52:12,080 ERROR File "/home/cds/cdsservices/services/mars/mars.py", line 18, in mars
2023-02-10 12:52:12,080 ERROR **kwargs)
2023-02-10 12:52:12,080 ERROR File "/home/cds/cdsservices/services/mars/preprocess_request.py", line 24, in preprocess_request
2023-02-10 12:52:12,080 ERROR requests = enforce_minimal_schema(requests, context)
2023-02-10 12:52:12,080 ERROR File "/home/cds/cdsservices/services/mars/request_checking.py", line 54, in enforce_minimal_schema
2023-02-10 12:52:12,080 ERROR return enforce_schema(requests, schema, context)
2023-02-10 12:52:12,080 ERROR File "/opt/cdstoolbox/cds-common/cds_common/request_schemas/enforce_schema.py", line 29, in enforce_schema
2023-02-10 12:52:12,080 ERROR from None
2023-02-10 12:52:12,080 ERROR cdsinf.exceptions.BadRequestException:
2023-02-10 12:52:12,080 ERROR request['verbose'][0]: False is not of type 'string'
The funny thing is that this very same code was working very well last December.
Any help and suggestion you could provide?
Thank you very much in advance.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.