Request site data from NWIS for stations about hydrofunctions HOT 2 OPEN

It turns out that different sites have different data on their stream stats pages ("gagepage"). Some sites have lots of data.

One original source is an old thing called the "Basin Characteristics File" that was first created in 1986! I found mentions of the file here: https://catalog.data.gov/dataset/watstore-stream-flow-basin-characteristics-file. This has links to the file on WATSTORE. It is in really old formats like .e00 and SDTS! These can be opened, but it doesn't seem worth it.

I checked several other packages to see if they have a way to access the gagepages:

• USGS-python/dataretrieval: they have some streamstats code, but it doesn't touch this file. It's more about getting geoJSON files, I think.
• cheginit/hydrodata: He has some routines that will calculate landcover for a watershed from the NLCD, but nothing with your slopes. You might find this package useful for you- the idea is that you provide a station, and this will retrieve the watershed and a lot of data for it.
• earthlab/streamstats: this finds the HUC8 watershed when you give it a point. It uses the streamstats service, but not the pages you want.
• Dewberry/usgs-tools: this uses the streamstats service too. It might be able to create a new watershed for you and calculate all of the statistics. It doesn't use the pre-calculated statistics that you found for the stations.

Since there doesn't seem to be anything that reads these pages, we could try to do it ourselves.

To read directly from the "gagepage" at streamstats, we could use Beautiful Soup, which parses HTML. I've never used it before, but here is a nice guide: https://www.pluralsight.com/guides/extracting-data-html-beautifulsoup It seems relatively easy to do:

pip install requests beautifulsoup4

import requests

from bs4 import BeautifulSoup as bs

url = "https://streamstatsags.cr.usgs.gov/gagepages/html/03335000.htm"

html_content = requests.get(url).text

soup = bs(html_content, "lxml")

The soup object has a few methods that allow you to select things...

• soup.title
• or soup.title.text for just the content and not the title tag.
  The tutorial page has an example that pulls data from a table. It shouldn't be too hard to figure out how to modify that to pull the data you want.

from hydrofunctions.

The site file is retrieved by hf.site_file(site) with commit 76f92cf.
It would still be useful to get the 'gagepages' because the site file mostly just contains the size of the watershed.

from hydrofunctions.