Git Product home page Git Product logo

lakecat's Introduction

DOI Badge

LakeCat

Description:

The LakeCat Dataset (https://www.epa.gov/national-aquatic-resource-surveys/lakecat-dataset) provides summaries of natural and anthropogenic landscape features for ~378,088 lakes and their associated catchments across the conterminous US.. This repo contains code used in LakeCat to process a suite of landscape rasters to watersheds for lakes and their associated catchments (local reach contributing area) within the conterminous USA using the NHDPlus Version 2 as the geospatial framework.

Necessary Python Packages and Installation Tips

The scripts for LakeCat rely on several python modules a user will need to install such as numpy, pandas, gdal, fiona, rasterio, geopandas, shapely, pysal, and ArcPy with an ESRI license (minimal steps still using ArcPy). We used the conda package manager to install necessary python modules. Note that package configurations and dependencies are sensitive and can change - in particular, setting up an environment with a working version of both geopandas and arcpy can be challenging. Our working version of the conda environment is contained in the StreamCat.yml file in the repository, and our essential packages and versions when code was last used are listed below - note that other configurations may work, we simply have verified this particular combination (Windows 64 and Python 3.7.10):

Package Version
python 3.7.10
fiona 1.8.18
gdal 3.1.4=py37
geopandas 0.9.0
geos 3.8.1
libgdal 3.1.4
numpy 1.19.5
pandas 1.2.3
pyproj 2.6.1
rasterio 1.2.1=py37
shapely 1.7.1

If you are using Anaconda, creating a new, clean 'StreamCat' environment with these needed packages can be done one of several ways:

  • In your conda shell, add one necessary channel and then download the streamcat environment from the Anaconda cloud:

    • conda config --add channels conda-forge
    • conda env create mweber36/StreamCat
  • Alternatively, using the streamcat.yml file in this repository, in your conda shell cd to the directory where your streamcat.yml file is located and run:

    • conda env create -f StreamCat.yml
  • To build environment yourself, we followed the steps suggest here which are:

    • conda create -n StreamCat -c conda-forge python=3.7 anaconda gdal=3.1.4 vs2015_runtime=14.28.29325 numpy=1.19.5 jupyter pandas geopandas matplotlib cartopy beautifulsoup4 shapely rpy2=3.4.1 simplegeneric r-raster=3.4_5 r-dismo=1.3_3 r-maptools pyproj=2.6.1.post1 rasterio
  • Activate the new environment:

    • conda activate StreamCat
  • To open Spyder, type the following at the conda prompt

    • activate Streamcat

    Then

    • Spyder

Finally, to use arcpy in this new environment, you will need to copy several ArcPro files and folders to your new environment as follows:

  • C:/Program Files/ArcGIS/Pro/bin/Python/envs/arcgispro-py3/Lib/site-packages/Arcgisscripting

  • C:/Program Files/ArcGIS/Pro/bin/Python/envs/arcgispro-py3/Lib/site-packages/arcpy_wmx

  • C:/Program Files/ArcGIS/Pro/bin/Python/envs/arcgispro-py3/Lib/site-packages/Gapy

To your environment directory which should look something like:

  • C:/Users/mweber/AppData/Local/Continuum/anaconda3/envs/StreamCat/Lib/site-packages

Note that the exact paths may vary depending on the version of ArcGIS and Anaconda you have installed and the configuration of your computer

How to Run Scripts

The scripts make use of 'control tables' to pass all the particular parameters to the two primary scripts:

In turn, these scripts rely on a set of functions in LakeCat_functions.py.

A table with all required parameters is used to process landscape layers in LakeCat:

Running LakeCat.py to generate new LakeCat metrics

After editing the control tables to provide necessary information, such as directory paths, the following stesps will excecute processes to generate new watershed metrics for the conterminous US. All examples in the control table are for layers (e.g., STATSGO % clay content of soils) that were processed as part of the LakeCat Dataset. This example assumes run in Anaconda within Conda shell.

  1. Edit ControlTable_LakeCat and set desired layer's "run" column to 1. All other columns should be set to 0
  2. Open a Conda shell and type "activate LakeCat"
  3. At the Conda shell type: "Python"
  4. Drag and drop "LakeCat.py" to the Conda shell from a file manager followed by another space
  5. Drag and drop the control table to the Conda shell

Final text in Conda shell should resemble this: python C:\some_path\LakeCat.py C:\some_other_path\ControlTable.csv

EPA Disclaimer

The United States Environmental Protection Agency (EPA) GitHub project code is provided on an "as is" basis and the user assumes responsibility for its use. EPA has relinquished control of the information and no longer has responsibility to protect the integrity , confidentiality, or availability of the information. Any reference to specific commercial products, processes, or services by service mark, trademark, manufacturer, or otherwise, does not constitute or imply their endorsement, recommendation or favoring by EPA. The EPA seal and logo shall not be used in any manner to imply endorsement of any commercial product or activity by EPA or the United States Government.

lakecat's People

Contributors

debboutr avatar jqortiz avatar justin-luke avatar mhweber avatar ryan-hill avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

lakecat's Issues

NSurp

  • Allocation and Accumulation
  • FinalTables
  • ZIPs
  • metadata

NLCD 2016

  • Allocation and Accumulation
  • FinalTables
  • ZIPs
  • metadata

TN_Load

  • Allocation and Accumulation
  • FinalTables
  • ZIPs
  • metadata

NLCD 2004

  • Allocation and Accumulation
  • FinalTables
  • ZIPs
  • metadata

pptSurplus

  • Allocation and Accumulation
  • FinalTables
  • ZIPs
  • metadata

AgDrain

  • Allocation and Accumulation
  • FinalTables
  • ZIPs
  • metadata

NLCD 2008

  • Allocation and Accumulation
  • FinalTables
  • ZIPs
  • metadata

Framework naming conventions

the following are the files that we deliver in the framework along w/ the attribute names, I thought it might be worth looking at these to go over the names we are currently using, and if we want we can rename them now, some of them come from the NHD directly, but others are put there by us. You can copy and paste the text into a comment below if you want names to change

Off-network.shp
############

  • COMID - Unique ID label to each waterbody in NHDPlusV2
  • FTYPE - NHDPlusV2 feature Type
  • RPU - UnitID of the RPU which the NHDPlusV2 waterbody is found
  • UID - Unique Identifier that is used for LakeCat raster creation
  • VPU_moved - UnitID of the VPU that geom exists spatially.
  • VPU_orig - UnitID of the VPU waterbody file that geom is found.
  • catAREASQK - Area of the Catchment where the centroid of the Waterbody falls
  • catCOMID - Catchment where the centroid of the Waterbody falls
  • oversized - delineated wshed basin larger than assoc. catchment

allBasins.shp
##########

  • COMID - Unique ID label to each basin created from the NHDPlusV2 waterbody
  • UID - Unique Identifier that is used in LakeCat rasters
  • BSN_COUNT - number of cells represented in the wshed raster
  • AreaSqKM - Area of the basin that was delineated from the Watershed tool
  • RPU - UnitID of the RPU which the basin is found

wtsheds_xxx.tif
############

  • VALUE - Unique ID label to each basin created from the NHDPlusV2 waterbody
  • COUNT - number of cells represented in each delineated watershed

Also, are we good with using LkCat_Frame_min for the zipfile name?? It might be better to use something like LakeCat_Framework for the name of the zip file

NLCD 2001

  • Allocation and Accumulation
  • FinalTables
  • ZIPs
  • metadata

NLCD 2006

  • Allocation and Accumulation
  • FinalTables
  • ZIPs
  • metadata

NANI

  • Allocation and Accumulation
  • FinalTables
  • ZIPs
  • metadata

NLCD 2011

  • Allocation and Accumulation
  • FinalTables
  • ZIPs
  • metadata

Average Slope

Add average slope as a metric in LakeCat using NHDPlusV2 elevation rasters

  • Allocation and Accumulation
  • FinalTables
  • ZIPs
  • metadata

septic 1990

  • Allocation and Accumulation
  • FinalTables
  • ZIPs
  • metadata

RockN

  • Allocation and Accumulation
  • FinalTables
  • ZIPs
  • metadata

Water Input

  • Allocation and Accumulation
  • FinalTables
  • ZIPs
  • metadata

NLCD 2013

  • Allocation and Accumulation
  • FinalTables
  • ZIPs
  • metadata

wdrw_onag

  • Allocation and Accumulation
  • FinalTables
  • ZIPs
  • metadata

Aquifer

  • Allocation and Accumulation
  • FinalTables
  • ZIPs
  • metadata

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.