nrcan / canflood Goto Github PK

French documentation is referenced from a different repository to enable Read the Docs to switch between English and French versions in the same link. Search an alternate solution so the french documentaion is under the same repository.

Some revisions to the dike module are not reflected in Tutorial 6 documentation (e.g., only 1 dike influence zone).

Hi Seth,

As the last step in the case study, I came across this error with my own inventory data with the following log messages. Appreciate any help.

A nice CI primer can be found here. Once setup, this will reduce development time and reduce bugs for the users (improving their experience).

Need to figure out how to setup a pyqgis build on GitHub. I'm guessing we won't be able to execute UI tests remotely. If this is the case, we can only include API/backend tests (which we'll need to write.. see issue 65).

no error. Seems to be an issue with new PyQt version

Under section 6.8.3:

The following image is what the manual states that the per-asset results (‘passet’) data file should look like:

This does not match with what is found in my risk1_Tut4a_run1_r_passet.csv file:

The data under the ‘0.02’ and the ‘ead’ column were not the same, however the rest of the data was consistent. This also resulted in the final polygon vector layer to have values that differ greatly from what is shown in the manual.

However, I did not experience the same issue when I worked through Tutorial 4b; all my results did match with what is shown in the manual.

I am running QGIS 3.16.14 (ltr) with the following packages:

Issue by cefect
Friday Apr 09, 2021 at 21:02 GMT # Sample: Friday Sep 13, 2013 at 22:58 GMT
Originally opened as IBIGroupCanWest/CanFlood#79

Issue by jdngibson
Tuesday Apr 21, 2020 at 17:44 GMT # Sample: Friday Sep 13, 2013 at 22:58 GMT
Originally opened as IBIGroupCanWest/CanFlood#40

I'm getting an error when trying to generate the 'evals.csv' in the Event Variables Tab. The issue appears to be caused by only having one rastor layer.

• install new QGIS version supported by CanFlood (3.22.8)
• manually install new CanFlood dev version (need to create a zip of CanFlood folder. build dialog should read '1.2.0')
• Review and edit the new toolset documentation (5.3.4 Report Automator)
• Complete, review, and edit tutorial 2a (section 6.2)

Note I've created a new branch for this work.

Check the boxes here and create a pull request once this is finished.

Issue by cefect
Wednesday Aug 19, 2020 at 00:21 GMT # Sample: Friday Sep 13, 2013 at 22:58 GMT
Originally opened as IBIGroupCanWest/CanFlood#47

download the OSGeo4W Network Installer
select 'Advanced Install option'
On the 'Select Packages' page, type 'qgis-ltr-full' into the search bar, expand 'Desktop' and ensure the 'New' (or 'Current') version shows 3.10-2 or higher.

OPTIONAL: check the dependency versions listed here (e.g. type 'python3-pandas' into the search bar, and ensure 'New' shows 1.0.1-1 or higher.)

Click through and this should install Qgis 3.10 nicely

CanFlood’s Inventory Construction Helper is challenging to use (we were not able to make it work), and some more documentation or examples in tutorial would be helpful.

Hi @cefect, I have combined the model files as suggested, and have now successfully run the risk L2 model without the res_per_asset box being checked. When I check the box, however, I get the following error: 'got negative extrapolation on '2002044': -912.55'.

Do you know what this is referring to? I have checked asset 2002044 in the model files and don't see anything out of the ordinary.

Originally posted by @blhumphreys in #47 (comment)

I am trying to follow tutorial 2a with my own data. Several of the nested 'cap' fields in my inventory feature class are entirely null, which leads to a warning message and the finv CSV not being stored in the working folder. I have tried manually exporting the inventory attribute table to CSV, but found that the model could not be validated when using this.

The following warning message is generated when I attempt to store the inventory data:
WARNING cf.BuildDialog.finv_to_csv: CanFlood_RiskL2_SimpleEvents1_SFD_R0a dropping 3 (of 44) empty fields

The following warning message is generated when I attempt to validate the model:
WARNING cf.BuildDialog.run_validate: risk2 error 2:
no value provided for 'risk_fps.dmgs'. optional=False
INFO cf.BuildDialog.set_cf_pars: updated control file w/ 2 pars at :
FILE_PATH\CanFlood_RiskL2_SimpleEvents1_SFD_R0a.txt
INFO cf.BuildDialog.cf_mark: risk2=False
INFO cf.BuildDialog.run_validate: passed 0 (of 2) validations. see log for errors

Additional info:

• CanFlood version 1.1.0
• QGIS version 3.16.16
• I have been able to get the model to validate when using data that doesn't have null cap fields.
• Inventory data is clipped to hazard rasters.

revise the rst files so there is only one sentence per line. This makes tracking future changes and reading the source code easiser. This should be fairly quick with a Ctrl+F applied to blocks.

Please help

Often, a user will have a damage function library with many 'extra' curves not referenced in the model. It would be nice to provide a simple tool on the Build - Inventory tab to remove any damage functions from the xls not found in the finv.

dev branch is taking ~3 seconds to load on my machine. This is quite slow to other comparable plugins.

TODO: investigate plugin initialization and identify opportunities for optimization (e.g., moving some setup functions to first load or first run).

Change copyright to Natural Resources Canada (from NRCan).

QGIS ver 3.16.10
CanFlood ver 1.0.2
Python ver 3.9.5
Pandas ver 1.3.0
PyQt5 ver 5.15.4
Numpy ver 1.20.2
xlrd ver 1.2.0
xlsxwriter ver 1.2.6
scipy ver 1.6.2
openpyxl ver 3.0.6

I was following along with the first tutorial in the CanFlood user manual, using the tut1 data and when i got to the Inventory Compiler in the CanFlood Build Toolset (pg. 68), I hit the "Store" button and I got the following Python Error:

It seems like there is an issue with the to_replace parameter in Pandas, where it cannot except a Qvariant object in more recent versions of Pandas. Initially, I was using Pandas 1.3.0 (the latest version) and I so I tried to reinstall two older versions (1.2.3 and 1.1.3) but I got the same issue.

I am trying to run an L2 risk model using my own data. I am able to build and fully validate the model using 7 hazard rasters and complex failure polygons, but when I go to run L2 risk I get one of the following errors, depending on whether I set event_rels to 'mutually exclusive' or 'independent':

• Mutually exclusive - 'x and y arrays must have at least 2 entries'
• Independent - 'attribution not implemented for event_rels='indep''

Could you please help us resolve these issues? It was mentioned in a previous post that the 'x and y arrays must have at least 2 entries' error can be resolved by not using enough hazard events, but that doesn't appear to be the issue here.

Thanks

• Unit tests should be added here. Draft feature documentation is here
• Further, this feature should be added to the tutorial 2A integration test here. Draft tutorial documentation is here

documentation should be edited if necessary.

find a city with lot's of public data (Calgary?)
walk through of downloading data, formatting, etc.
this could be a blog post instead of in the manual

with the migration to pytest, we have some pretty big gaps in the test coverage. Especially the tutorials (currently, we only have tutorial 2a). I suppose we can just add these each time we revisit a section of the code... but this exposes us to code breaks on dependency upgrades.

action: add tests

We encourage users to post their issues to receive help and aide others who may have similar problems. Please follow these steps when posting issues:

0) Review the documentation

1) Check to see if a similar issue has already been posted. If so, do not create a new ticket. Instead, respond to the existing thread if you still need help.

2) Check to see if you have the latest version of CanFlood installed and the matching 'tested' version of QGIS (usually the LTR). If not, install these and see if your issue persists.

3) Create a new issue, assign the appropriate labels, and include the following info:

3a) CanFlood version, QGIS version, and OS version.

3b) Screen shot of the Error (e.g. First Aid dialogue) is applicable

3c) Output from QGIS debug log file, if available (to locate this file: Settings > Options > file path should be displayed next to 'QGIS_LOG_FILE' environment variable). To set this up, see the post here.

3d) Any inputs necessary to replicate the issue. Try and make these as small as possible to replicate the issue (e.g., clip the data spatially, remove fields). If you are unable to share your data for some reason (e.g., privacy protections), re-create a dummy dataset with an identical structure that replicates the error.

3e) Links to project documentation (if applicable).

3f) Description of the issue including the actions required to generate the error.

0) Consultez la documentation.

1) Vérifiez si un problème comparable a déjà été signalé. Si c’est le cas, ne créez pas de billet. Répondez plutôt au fil de discussion existant si vous avez encore besoin d’aide.

2) Vérifiez si vous avez installé la dernière version de CanFlood et la version « testée » correspondante de QGIS (habituellement la LTR). Si vous ne les avez pas installées, installez-les et voyez si votre problème persiste.

3) Créez un billet, attribuez-lui les étiquettes appropriées et incluez les renseignements suivants :

3a) Version de CanFlood, version de QGIS et version d’OS.

3b) Capture d’écran de l’erreur (p. ex. dialogue First Aid) est applicable

3c) Extrant du fichier de débogage de QGIS, si disponible (pour localiser ce fichier : Settings > Options > le chemin du fichier devrait être affiché à côté de la variable d’environnement « QGIS_LOG_FILE »). Pour configurer cela, voir la publication ici.

3d) Tous les intrants nécessaires pour reproduire le problème. Dans la mesure du possible, tentez de réduire au minimum les intrants nécessaires pour reproduire le problème (p. ex., découpez les données dans l’espace, supprimez des champs). Si vous ne pouvez pas communiquer vos données pour une raison quelconque (p. ex., mesures de protection de la vie privée), recréez un ensemble de données factice avec une structure identique qui reproduit l’erreur.

3e) Liens vers la documentation à l’appui (s’il y a lieu).

3f) Description du problème, y compris les actions requises pour générer l’erreur.

see the control field 'name'. add this to the header

Integrate changes to dev branch

• ensure the version tag is updated on \canflood\build\build.ui

• execute all pytests. investigate warnings. fix errors.

• perform a 'person test' by having a non-developer follow relevant tutorials. investigate warnings and fix errors.

• Once these tests are complete, a pull request should be completed and the dev branch merged into the main.

Prepare main branch for publication

Now that all the code is tested and in the main branch, perform the following:

• update the README.md to summarize any new features for users

• similarly update canflood\metadata.txt

Publish the plugin

• delete all instances of _pycache_\ in the source code

• zip the \canflood subfolder to some temporary directory

• login to plugins.qgis.org using the CanFlood credentials (ask Nicky). Navigate to Upload a plugin and select the zip file.

• In QGIS, refresh the repository and ensure that the new version is available (may take ~10mins for the version to be available). Upgrade and check that it works.

Publish release on git-hub

• in git-hub, create a new release tag (e.g., v1.2.0), summarize new features for developers. upload the same zip file.

• notify the management team

3.28 goes stable March 3rd. At that point, we need to test, update, and re-release a CanFlood update.

Issue by yipjackie
Wednesday Feb 26, 2020 at 17:21 GMT # Sample: Friday Sep 13, 2013 at 22:58 GMT
Originally opened as IBIGroupCanWest/CanFlood#6

After a clean install of QGIS 3.10 (downloaded the long-term release for Mac OS from https://www.qgis.org/en/site/forusers/download.html) and the First Aid plug-in, the installing of the CanFlood plug-in gave an error as shown below - suggesting that it needs pandas 0.25.3 or higher.

Also included here the About QGIS page for reference

NOTE the first page is not a 'Section'

Create a nice clean homepage for CanFlood.

could include project updates.
page on potential resources and processing advice for input data.

Hi,

I am currently learning CanFlood based on hazard rasters from a project and I may have to generate inventory data myself in a specific area in BC.

According to the manual it seems we can create data template via inventory constructor or use some reference from NPRI/GAR15.

Is there a website for us to directly download data to make this easier?

Thanks,

Tong
Tong Cao, P.Eng.
Water Resource Engineer
Associated Engineering (B.C.) Ltd.
#500 - 2889 East 12th Avenue, Vancouver, BC V5M 4T5
Tel: 604.293.1411 | Dir: 236.317.2207

@.***https://www.ae.ca/

You may unsubscribe from Associated's electronic @.***?subject=Unsubscribe+from+AE+-+CASL> at any time.

Originally posted by @TC-999 in #32 (comment)

links still point to old repository. need to move the data and fix the readme

https://github.com/IBIGroupCanWest/CanFlood/tree/master/committee

QGIS version: 3.16.14-Hannover Hannover, 4279198632
OS version: Windows 10, 20H2
This message came right after CanFlood installation. I am not sure which version of CanFlood was installed as I just searched the plugin from the QGIS repository according to the Installation Instructions.

I am getting the error "impact values decreasing." I am trying to run L2 analysis of buildings for which I have f0_elv = height of first floor above ground level (m), f0_scale= floor area (m2), f0_cap = assessment value ($), 4 water surface level events with different return periods, and a DEM file. The values in the expos*.csv file are correct; the values is gels*.csv file look correct as the ground elevation height at each point. I am using a damage curve file with expos_var as water height above floor, impact_var as building repair and restoration cost, and scale_var as floor area. The inventory felv is set to "ground." I have tried changing the f0_cap values to zero but still get the error message. Is there something amiss with my data assignments and/or logic of what should be happening?

My QGIS is 3.16.14, CanFlood is 1.1.0 Attached are 2 screenshots of the error and the debug log file.

qgis_debug.log

Issue by petecampbel
Tuesday Aug 18, 2020 at 22:39 GMT # Sample: Friday Sep 13, 2013 at 22:58 GMT
Originally opened as IBIGroupCanWest/CanFlood#46

Running QGIS 3.10 A Coruna. Installed all python plugins using OSGeo4W.bat and the latest CanFlood plugin (CanFlood_031_20200415.zip) within QGIS. CanFlood installs but then returns an error message and won't load.

when running hazard sampler raster prep, QGIS crashes (sometimes without error) during final stages:

D_16-16.31.39 cf.BuildDialog.run_rPrep: loading 4 result layers onto widget
D_16-16.31.39 cf.BuildDialog.load_toCanvas: loading '<class 'qgis._core.QgsRasterLayer'>': <QgsRasterLayer: 'GAR2015flood_hazard_100_yrp_prepd' (gdal)>

Inventory tab is confusing, in terms of which files are needed and what is optional, as well as the sequence of required input (i.e., which file requirement is based upon a previous selection).

not clear that the DTM Sampler is optional

Hi,

I am trying to run an L2 risk model using 7 hazard rasters and complex failure polygons. I am able to validate the model and run impacts L2, but when I run risk L2 I get the following error: 'some complex event probabilities exceed 1'.

I have tried a different event_rels assumption (mutually exclusive instead of independent) and have confirmed that my exlikes values are all small (max 0.1659), as suggested by @cefect in issue #38.

Could you please help troubleshoot this issue?

Thanks.

In addition to this, we will need to run 6 breach scenarios for 3 different events (total of 18 rasters) for our analysis. Is there a way to run all 18 rasters together? It seems that the tool is either limited to 8 (max inputs for the Conditional P tab), or until the total for each row in the exlikes row is greater than 1.

Please let me know if you'd like me to post this as a new issue.

Originally posted by @blhumphreys in #46 (comment)

Issue by mbrwn
Wednesday Feb 24, 2021 at 21:39 GMT # Sample: Friday Sep 13, 2013 at 22:58 GMT
Originally opened as IBIGroupCanWest/CanFlood#76

Following Tutorial 2a but with my own datasets. All steps were successful (with some trial-and-error) until this point. Validation of R1 was successful. Then when tried to run R1 got error:

Run of L2 worked ok with the same files/set-up. Then run of R2 got this error:

currently, L1 per-event impacts are stored in the r_passet file and no _dmgs file is created (like an L2). The expos file does not include any scaling.

Proposal:
L1 should create a _dmgs file for formatting consistency w/ L2

Hello,

We are trying to use the IBI 2020 curve for a project in BC. However, we do not know how to assign building categories (fX_tag) to our building data.

In the IBI 2020 curve, the building categories are listed as mfSlab150, mfSlab151, etc. We cannot find adequate information in the CanFlood documentation to determine which buildings these codes stand for. We also check the Flood Risk Assessment for BC’s Lower Mainland report (IBI 2020) and its Appendix A, but still did not find enough information.

Is it possible to add the relevant information somewhere in the CanFlood documentation if it is not already there? Thank you!

ISSUE:
geopackages get flagged as 'edited' by GIT each time they are opened (regardless of whether there are changes). Also, geopackages don't support GIT change tracking.

PROPOSAL:
convert all tutorial vector layers to geojson. will need to revise tests and documentation.

https://canflood.readthedocs.io/en/latest/toolsets.html#model

tables need a table number (e.g., table 1.1)

Issue by cefect
Friday Jan 29, 2021 at 16:55 GMT # Sample: Friday Sep 13, 2013 at 22:58 GMT
Originally opened as IBIGroupCanWest/CanFlood#66

A user reported the following error when installing 0.4.2

When running an L1 assessment for multiple flood scenarios (1:2yr, 1:5yr, 1:10yr, 1:20yr, 1:100yr), I receive an AssertionError: passed rtail value (0.50) not > max aep (0.50) error. (I suspect it would also fail for L2, but haven't tested that.)

Exact error as follows:

AssertionError: passed rtail value (0.50) not > max aep (0.50)
Traceback (most recent call last):
  File "C:\Users\XXX\AppData\Roaming\QGIS\QGIS3\profiles\default\python\plugins\canflood\model\dialog.py", line 164, in run_risk1
    res_ttl, res_df = model.run(res_per_asset=self.checkBox_r1_rpa.isChecked())
  File "C:\Users\XXX\AppData\Roaming\QGIS\QGIS3\profiles\default\python\plugins\canflood\model\risk1.py", line 268, in run
    res_df = self.calc_ead(bres_df)
  File "C:\Users\XXX\AppData\Roaming\QGIS\QGIS3\profiles\default\python\plugins\canflood\model\riskcom.py", line 807, in calc_ead
    assert aep_val > df.columns.max(), 'passed rtail value (%.2f) not > max aep (%.2f)'%(
AssertionError: passed rtail value (0.50) not > max aep (0.50)

Removing the 1:2yr scenario and updating related elements clears the error.

Workaround:
Before running the L1 assessment, change the value for the following line in the control file from 0.5 to 0.6:
rtail = 0.5 #EAD extrapolation: right trail treatment (high prob low damage)

While this workaround helps, we will be conducting several coastal flood studies with modelling for the 1:2yr event for several locations across Canada and it would be nice to get this working by default for high probability, low impact events.

Getting the following error when installing CanFlood 1.1.0 plugin:

Using QGIS 3.16.14 Hanover. Here is the 'About' info:

Hi, I'm getting the following critical errors when inputting my data:

2022-04-12T13:22:46 WARNING cf.BuildDialog.vlay_get_fdata: got no results! from 'finv_tuk_finv'
2022-04-12T13:22:46 CRITICAL allow_none=FALSE and no results

Any ideas?

Issue by cefect
Tuesday Feb 25, 2020 at 00:25 GMT # Sample: Friday Sep 13, 2013 at 22:58 GMT
Originally opened as IBIGroupCanWest/CanFlood#4

mFieldComboBox_cid is trying to refresh on an empty canvas.

We're using some pretty difficult syntax and features... not your fault... and largely owing to having to port from MS word a few years ago.

I added a ./docs/CONTRIBUTING.md with some ideas how we can improve the document and workflow. Specifically to the number of tables (and figures) this means using the 'numref' role to automatically number things (so we don't have to renumber the whole section anytime something changes. Similarly, I noticed many of the cross-references have weird spacing (e.g., FIgure1-2 instead of Figure 1-2). Finally, many of the tables are unreadable owing to excessive column widths (e.g., the Table "CanFlood model level summaries").

The following should be addressed per the new CONTRIBUTING.md guidelines:

• revise all figure and table references to use the 'numref' roll (rather than manually number). Not all figures need a number. We may lose the section number, but this is fine.
• revise all cross-references to be human readable e.g. Table1.1 should be revised to Table 1.1 (numref will automatically do this for tables and figures, but sections may need to be cleaned manually).
• check the width of the tables. Any that are unreadably long should be converted to a csv-table to get the text to wrap.