An SQLAlchemy ORM for GSSHA model files and a toolkit to convert gridded input into GSSHA input.
License: BSD 3-Clause "New" or "Revised" License
An SQLAlchemy ORM for GSSHA model files and a toolkit to convert gridded input into GSSHA input.
Documentation can be found: http://gsshapy.readthedocs.io/en/latest
I can't get gsshapy to build for Python 3.7 on conda forge due to one it's dependencies not being built for 3.7 (rapidpy). There may be other issues too.
See: https://dev.azure.com/conda-forge/feedstock-builds/_build/results?buildId=38198
For now, it'll be configured to only build Python 3.6, but we should address the roadblocks at some point. One way around this would be to make rapidpy and other similar dependencies optional. It doesn't seem to get used in the core ORM.
I have two questions, both related to confusion newcomers might experience when starting with gsshapy. I'm asking the two questions together in case there's an underlying design reason for both.
First (unless I have missed it!), the documentation doesn't seem to mention that installing gssha is required. Would a PR on http://gsshapy.readthedocs.io/en/latest/intro.html#installation adding something along the lines of the following be any good?
Installation
First, you will need to download and install GSSHA, and then ensure that the
gsshaexecutable is on your PATH.
Second, it seems like failure to find the gssha binary should result in an obvious, early exception rather than the exception being caught and only logged as a warning as happens at present. Particularly when (according to the docs)
GsshaPy uses the default Python logging module [and by] default, nothing is logged anywhere.
In general it seems like quite a lot of exceptions are caught and logged throughout gsshapy. It's great to see a project taking logging seriously because that can be very useful. However, as a user I'd also like to find out immediately about things like missing the simulator or a critical file required for the run.
I'm not sure what the knock-on effects of changing this throughout the repository would be - might not be worth addressing.
Use of relative paths would make sharing examples and model inputs across users/computers a whole lot easier. Is there something fundamental that makes this difficult or is it a case of going through the codebase and making path handling more robust.
ORIGINAL SECTION:
MULTI_LAYER_SOIL "combo"
NUM_IDS 16
MAX_SOIL_ID 10000
ID DESCRIPTION1 DESCRIPTION2 HYD_COND PORE_INDEX CAPIL_HEAD POROSITY FIELD_CAP WILTING_PT RESID_SAT SOIL_MOIST DEPTH
2001 clay loam corn/soy .0076669 0.390000 20.880000 0.464000 0.318000 0.117000 0.075000 0.250000 43.180000
.1090112 0.418000 19.020000 0.466000 0.318000 0.174000 0.047000 0.250000 43.180000
.1445873 0.412000 14.885000 0.463000 0.294000 0.157000 0.051000 0.250000
2002 clay loam alfalfa/grass .0076669 0.390000 20.880000 0.464000 0.318000 0.117000 0.075000 0.250000 43.180000
.1090112 0.418000 19.020000 0.466000 0.318000 0.174000 0.047000 0.250000 43.180000
.1445873 0.412000 14.885000 0.463000 0.294000 0.157000 0.051000 0.250000
2003 clay loam wetlands, water 0.100000 0.390000 20.880000 0.464000 0.318000 0.117000 0.075000 0.250000 43.180000
0.573000 0.418000 19.020000 0.466000 0.318000 0.174000 0.047000 0.250000 43.180000
0.760000 0.412000 14.885000 0.463000 0.294000 0.157000 0.051000 0.250000
2004 clay loam developed 0.010000 0.390000 20.880000 0.464000 0.318000 0.117000 0.075000 0.250000 43.180000
0.573000 0.418000 19.020000 0.466000 0.318000 0.174000 0.047000 0.250000 43.180000
0.760000 0.412000 14.885000 0.463000 0.294000 0.157000 0.051000 0.250000
2005 clay loam forest 0.150000 0.390000 20.880000 0.464000 0.318000 0.117000 0.075000 0.250000 43.180000
0.573000 0.418000 19.020000 0.466000 0.318000 0.174000 0.047000 0.250000 43.180000
0.760000 0.412000 14.885000 0.463000 0.294000 0.157000 0.051000 0.250000
3001 loam corn/soy .1152894 0.434000 8.890000 0.463000 0.270000 0.117000 0.027000 0.250000 35.560000
.1445873 0.412000 14.885000 0.463000 0.294000 0.157000 0.051000 0.250000 43.180000
.1445873 0.412000 14.885000 0.463000 0.294000 0.157000 0.051000 0.250000
3002 loam alfalfa/grass .1152894 0.434000 8.890000 0.463000 0.270000 0.117000 0.027000 0.250000 35.560000
.1445873 0.412000 14.885000 0.463000 0.294000 0.157000 0.051000 0.250000 43.180000
.1445873 0.412000 14.885000 0.463000 0.294000 0.157000 0.051000 0.250000
3003 loam wetlands, water 0.606000 0.434000 8.890000 0.463000 0.270000 0.117000 0.027000 0.250000 35.560000
0.760000 0.412000 14.885000 0.463000 0.294000 0.157000 0.051000 0.250000 43.180000
0.760000 0.412000 14.885000 0.463000 0.294000 0.157000 0.051000 0.250000
3004 loam developed 0.010000 0.434000 8.890000 0.463000 0.270000 0.117000 0.027000 0.250000 35.560000
0.760000 0.412000 14.885000 0.463000 0.294000 0.157000 0.051000 0.250000 43.180000
0.760000 0.412000 14.885000 0.463000 0.294000 0.157000 0.051000 0.250000
3005 loam forest 0.606000 0.434000 8.890000 0.463000 0.270000 0.117000 0.027000 0.250000 35.560000
0.760000 0.412000 14.885000 0.463000 0.294000 0.157000 0.051000 0.250000 43.180000
0.760000 0.412000 14.885000 0.463000 0.294000 0.157000 0.051000 0.250000
5001 mucky silt loam corn/soy .0091318 0.486000 16.680000 0.501000 0.330000 0.117000 0.015000 0.250000 20.320000
.0684887 0.436000 21.620000 0.478000 0.338000 0.179000 0.043000 0.250000 55.880000
.0684887 0.436000 21.620000 0.478000 0.338000 0.179000 0.043000 0.250000
5002 mucky silt loam alfalfa/grass 0.156570 0.486000 16.680000 0.501000 0.330000 0.117000 0.015000 0.250000 20.320000
0.360000 0.436000 21.620000 0.478000 0.338000 0.179000 0.043000 0.250000 55.880000
0.360000 0.436000 21.620000 0.478000 0.338000 0.179000 0.043000 0.250000
5003 mucky silt loam wetlands, water 0.156570 0.486000 16.680000 0.501000 0.330000 0.117000 0.015000 0.250000 20.320000
0.360000 0.436000 21.620000 0.478000 0.338000 0.179000 0.043000 0.250000 55.880000
0.360000 0.436000 21.620000 0.478000 0.338000 0.179000 0.043000 0.250000
5004 mucky silt loam developed 0.010000 0.486000 16.680000 0.501000 0.330000 0.117000 0.015000 0.250000 20.320000
0.360000 0.436000 21.620000 0.478000 0.338000 0.179000 0.043000 0.250000 55.880000
0.360000 0.436000 21.620000 0.478000 0.338000 0.179000 0.043000 0.250000
5005 mucky silt loam forest 0.156570 0.486000 16.680000 0.501000 0.330000 0.117000 0.015000 0.250000 20.320000
0.360000 0.436000 21.620000 0.478000 0.338000 0.179000 0.043000 0.250000 55.880000
0.360000 0.436000 21.620000 0.478000 0.338000 0.179000 0.043000 0.250000
10000 unknown unknown 0.048330 0.390000 20.880000 0.464000 0.318000 0.117000 0.075000 0.250000 43.180000
0.573000 0.418000 19.020000 0.466000 0.318000 0.174000 0.047000 0.250000 43.180000
0.760000 0.412000 14.885000 0.463000 0.294000 0.157000 0.051000 0.250000
GSSHAPY OUTPUT:
MULTI_LAYER_SOIL "combo"
NUM_IDS 16
ID DESCRIPTION1 DESCRIPTION2 HYD_COND PORE_INDEX CAPIL_HEAD POROSITY FIELD_CAP WILTING_PT RESID_SAT SOIL_MOIST
2001 clay loam corn/soy 0.007667 0.390000 20.880000 0.464000 0.318000 0.117000 0.075000 0.250000 43.180000
2002 clay loam alfalfa/grass 0.007667 0.390000 20.880000 0.464000 0.318000 0.117000 0.075000 0.250000 43.180000
2003 clay loam wetlands, water 0.100000 0.390000 20.880000 0.464000 0.318000 0.117000 0.075000 0.250000 43.180000
2004 clay loam developed 0.010000 0.390000 20.880000 0.464000 0.318000 0.117000 0.075000 0.250000 43.180000
2005 clay loam forest 0.150000 0.390000 20.880000 0.464000 0.318000 0.117000 0.075000 0.250000 43.180000
3001 loam corn/soy 0.115289 0.434000 8.890000 0.463000 0.270000 0.117000 0.027000 0.250000 35.560000
3002 loam alfalfa/grass 0.115289 0.434000 8.890000 0.463000 0.270000 0.117000 0.027000 0.250000 35.560000
3003 loam wetlands, water 0.606000 0.434000 8.890000 0.463000 0.270000 0.117000 0.027000 0.250000 35.560000
3004 loam developed 0.010000 0.434000 8.890000 0.463000 0.270000 0.117000 0.027000 0.250000 35.560000
3005 loam forest 0.606000 0.434000 8.890000 0.463000 0.270000 0.117000 0.027000 0.250000 35.560000
5001 mucky silt loam corn/soy 0.009132 0.486000 16.680000 0.501000 0.330000 0.117000 0.015000 0.250000 20.320000
5002 mucky silt loam alfalfa/grass 0.156570 0.486000 16.680000 0.501000 0.330000 0.117000 0.015000 0.250000 20.320000
5003 mucky silt loam wetlands, water 0.156570 0.486000 16.680000 0.501000 0.330000 0.117000 0.015000 0.250000 20.320000
5004 mucky silt loam developed 0.010000 0.486000 16.680000 0.501000 0.330000 0.117000 0.015000 0.250000 20.320000
5005 mucky silt loam forest 0.156570 0.486000 16.680000 0.501000 0.330000 0.117000 0.015000 0.250000 20.320000
10000 unknown unknown 0.048330 0.390000 20.880000 0.464000 0.318000 0.117000 0.075000 0.250000 43.180000
0.109011 0.418000 19.020000 0.466000 0.318000 0.174000 0.047000 0.250000 43.180000
0.144587 0.412000 14.885000 0.463000 0.294000 0.157000 0.051000 0.250000
0.109011 0.418000 19.020000 0.466000 0.318000 0.174000 0.047000 0.250000 43.180000
0.144587 0.412000 14.885000 0.463000 0.294000 0.157000 0.051000 0.250000
0.573000 0.418000 19.020000 0.466000 0.318000 0.174000 0.047000 0.250000 43.180000
0.760000 0.412000 14.885000 0.463000 0.294000 0.157000 0.051000 0.250000
0.573000 0.418000 19.020000 0.466000 0.318000 0.174000 0.047000 0.250000 43.180000
0.760000 0.412000 14.885000 0.463000 0.294000 0.157000 0.051000 0.250000
0.573000 0.418000 19.020000 0.466000 0.318000 0.174000 0.047000 0.250000 43.180000
0.760000 0.412000 14.885000 0.463000 0.294000 0.157000 0.051000 0.250000
0.144587 0.412000 14.885000 0.463000 0.294000 0.157000 0.051000 0.250000 43.180000
0.144587 0.412000 14.885000 0.463000 0.294000 0.157000 0.051000 0.250000
0.144587 0.412000 14.885000 0.463000 0.294000 0.157000 0.051000 0.250000 43.180000
0.144587 0.412000 14.885000 0.463000 0.294000 0.157000 0.051000 0.250000
0.760000 0.412000 14.885000 0.463000 0.294000 0.157000 0.051000 0.250000 43.180000
0.760000 0.412000 14.885000 0.463000 0.294000 0.157000 0.051000 0.250000
0.760000 0.412000 14.885000 0.463000 0.294000 0.157000 0.051000 0.250000 43.180000
0.760000 0.412000 14.885000 0.463000 0.294000 0.157000 0.051000 0.250000
0.760000 0.412000 14.885000 0.463000 0.294000 0.157000 0.051000 0.250000 43.180000
0.760000 0.412000 14.885000 0.463000 0.294000 0.157000 0.051000 0.250000
0.068489 0.436000 21.620000 0.478000 0.338000 0.179000 0.043000 0.250000 55.880000
0.068489 0.436000 21.620000 0.478000 0.338000 0.179000 0.043000 0.250000
0.360000 0.436000 21.620000 0.478000 0.338000 0.179000 0.043000 0.250000 55.880000
0.360000 0.436000 21.620000 0.478000 0.338000 0.179000 0.043000 0.250000
0.360000 0.436000 21.620000 0.478000 0.338000 0.179000 0.043000 0.250000 55.880000
0.360000 0.436000 21.620000 0.478000 0.338000 0.179000 0.043000 0.250000
0.360000 0.436000 21.620000 0.478000 0.338000 0.179000 0.043000 0.250000 55.880000
0.360000 0.436000 21.620000 0.478000 0.338000 0.179000 0.043000 0.250000
0.360000 0.436000 21.620000 0.478000 0.338000 0.179000 0.043000 0.250000 55.880000
0.360000 0.436000 21.620000 0.478000 0.338000 0.179000 0.043000 0.250000
0.573000 0.418000 19.020000 0.466000 0.318000 0.174000 0.047000 0.250000 43.180000
0.760000 0.412000 14.885000 0.463000 0.294000 0.157000 0.051000 0.250000
Look into http://xarray.pydata.org
The long-term simulation LONGITUDE card must always be positive.
Need to add changes in 2.1 to documentation.
Look into using http://aospy.readthedocs.io/en/latest/index.html
Copy cmt file to execute/result directory, change file paths for index maps to point to parent folder.
Create function to resample any input grid to the model grid. Most likely via rasterio.
Look into improving projecting the grid.
Need to convert cloud cover from percentage to decimal when using it to calculate cloud cover...
I've marked many of the tests with xfail (expected fail) b/c most of them use almost equal comparisons on files with very small tolerances. As a result, the tests fail on many systems due to the natural variance from system to system.
When marked as xfail, these tests will still run, but won't cause the test suite to fail when they fail.
Moving the .gag file to the run folder causes WMS to not be able to find it when opening the project from the project folder.
Alan,
I had the MAX_COURANT_NUMBER card in my source project file. It didn't get replicated to the run project file.
The current version of gridded hmet fails as it is missing the NODATA_value
Alan,
You might want to add these cards to the scripting if they aren't there already.
Clay
-----Original Message-----
From: Follum, Michael L ERDC-RDE-CHL-MS CIV
Sent: Wednesday, February 08, 2017 7:14 AM
To: Lahatte, Clay W ERDC-RDE-CHL-MS CIV
Subject: RE: Snow in GSSHA
Hey Clay, it does depending on the version. Here are the cards:
SNOW_SWE_FILE "SR_SnowWaterEquivalent.swd"
SNOW_SD_FILE "SR_SnowDepth.sdd"
Mike
-----Original Message-----
From: Lahatte, Clay W ERDC-RDE-CHL-MS CIV
Sent: Tuesday, February 07, 2017 3:34 PM
To: Follum, Michael L ERDC-RDE-CHL-MS CIV
Cc: Turnbull, Stephen J ERD-MS; Downer, Charles W ERDC-RDE-CHL-MS CIV
Subject: Snow in GSSHA
Mike, does GSSHA output a raster that shows what cells got snow? That would be so nice. I couldn't find a reference to that in the GSSHA wiki.
Fix printing gssha output to console for Python 3.
LINK 44
LAKE
MINWSE 200.000000
INITWSE 230.000000
MAXWSE 239.800000
NUMPTS 127
83 66 82 66 82 67 81 67 81 68 81 69 82 69 83 69 84 66 84 69
80 67 79 67 78 67 84 68 78 66 77 66 77 67 83 68 76 67 75 67
84 67 79 66 75 68 76 66 82 68 80 68 76 68 85 66 77 68 74 68
81 70 75 69 73 68 77 69 74 69 76 69 73 67 77 70 76 70 75 70
72 67 85 67 71 67 74 70 70 67 77 71 80 69 69 67 75 71 81 71
76 71 73 69 73 70 79 68 69 68 72 70 74 71 72 69 72 68 73 71
71 70 71 68 71 69 71 71 78 68 75 72 77 72 70 68 72 71 70 69
79 69 75 66 74 72 69 69 76 72 78 69 69 70 68 70 80 70 81 72
68 69 79 70 71 66 68 71 70 66 71 72 81 66 72 72 68 72 73 72
69 66 76 73 67 71 81 73 80 66 83 67 67 72 72 66 74 67 67 73
78 72 68 73 82 70 68 74 68 68 68 75 67 75 85 68 80 73 80 72
85 69 67 76 82 72 67 77 66 77 68 67 66 78 86 69 76 65 78 70
69 71 78 71 65 78 76 74 70 70 86 70 68 66
NUMRATE 1617
195.700 0.000 44515.500
195.700 0.130 48157.600
195.800 0.630 51799.800
ERROR:
DataError at /apps/reservoir-prediction-gssha/
(psycopg2.DataError) invalid input syntax for integer: "NUMRATE"
LINE 1: ...reservoir_points ("reservoirID", i, j) VALUES (1, 'NUMRATE',...
^
[SQL: 'INSERT INTO cif_reservoir_points ("reservoirID", i, j) VALUES (%(reservoirID)s, %(i)s, %(j)s) RETURNING cif_reservoir_points.id'] [parameters: {'i': 'NUMRATE', 'reservoirID': 1, 'j': '1617'}]
When I run "activate gssha" the result is...
Z:\Projects\FY16\Korea_North\AirForce\AOIs\NK_Hawaii\GSSHA\Stream_Attribs\20170316_Thiessen>activate gssha
The system cannot find the path specified.
(gssha) C:\Program Files\Anaconda3\envs\gssha\etc\conda\activate.d>set "GDAL_DRIVER_PATH="
(gssha) Z:\Projects\FY16\Korea_North\AirForce\AOIs\NK_Hawaii\GSSHA\Stream_Attribs\20170316_Thiessen>
Need to add documentation for DB Tools
Need to add ./ to output files for WMS to read it in.
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.