makepath / xarray-spatial Goto Github PK
View Code? Open in Web Editor NEWRaster-based Spatial Analytics for Python
Home Page: https://xarray-spatial.readthedocs.io/
License: MIT License
Raster-based Spatial Analytics for Python
Home Page: https://xarray-spatial.readthedocs.io/
License: MIT License
images
directoryDEM | SLOPE | CURVATURE |
HILLSHADE | ZONAL | |
span to set global shading min/ max range
to_ogc_tile_metadata
which outputs a tile GetCapabilities.xml
file: http://schemas.opengis.net/wmts/1.0/profiles/wmts-simple/examples/wmtsGetCapabilities_response_OSM.xmlto_esri_tile_metadata
which outputs a tile MapServer
file (json):{
"serviceDescription" : "Test Map Service Description",
"mapName" : "Street Map Pro Data",
"description": "Street Map USA",
"copyrightText" : "ESRI",
"layers": [
{"id" : 0, "name" : "Cities", "defaultVisibility" : true, "parentLayerId" : -1, "subLayerIds" : null},
{"id" : 1, "name" : "States", "defaultVisibility" : true, "parentLayerId" : -1, "subLayerIds" : null},
{"id" : 2, "name" : "Counties", "defaultVisibility" : false, "parentLayerId" : -1, "subLayerIds" : [3, 4]},
{"id" : 3, "name" : "Large Counties", "defaultVisibility" : false, "parentLayerId" : 2, "subLayerIds" : null},
{"id" : 4, "name" : "Small Counties", "defaultVisibility" : false, "parentLayerId" : 2, "subLayerIds" : null}
],
"spatialReference" : {"wkid" : 4326},
"singleFusedMapCache" : true,
"tileInfo": {
"rows" : 512, "cols" : 512, "dpi" : 96, "format" : "JPEG", "compressionQuality" : 75,
"origin" : {"x" : -130.0, "y" : 50.0},
"spatialReference" : {"wkid" : 4326},
"lods": [
{"level" : 0, "resolution" : 8.46, "scale" : 32000.0 },
{"level" : 1, "resolution" : 4.23, "scale" : 16000.0 },
{"level" : 2, "resolution" : 2.11, "scale" : 8000.0 },
{"level" : 3, "resolution" : 1.05, "scale" : 4000.0 },
{"level" : 4, "resolution" : 0.52, "scale" : 2000.0 }
]
},
"initialExtent" : {
"xmin" : -109.55, "ymin" : 25.76, "xmax" : -86.39, "ymax" : 49.94,
"spatialReference" : {"wkid" : 4326}
},
"fullExtent" : {
"xmin" : -130.0, "ymin" : 24.0, "xmax" : -65.0, "ymax" : 50.0,
"spatialReference" : {"wkid" : 4326}
},
"units" : "esriDecimalDegrees",
"supportedImageFormatTypes": "PNG32,PNG24,PNG,JPG,DIB,TIFF,EMF,PS,PDF,GIF,SVG,SVGZ",
"documentInfo": {
"Title" : "StreetMap USA.mxd",
"Author" : "ESRI Data Team",
"Comments" : "ESRI Data and Maps 2004",
"Subject" : "Street level data for the US",
"Category" : "vector",
"Keywords" : "StreetMap USA"
}
}```
- [ ] add mbtile output
Looks like the text and example don't match up. To fix these:
git clone && cd && pip install -e .
returns new xr.DataArray
where we store the value of the nearest feature
...instead of distance_to_nearest
from xrspatial.proximity import allocation
allocations = allocation(points_agg)
allocations.shape == points_agg.shape
Reproduction:
import datashader as ds
from xrspatial import proximity
df1 = pd.DataFrame(dict(x=[-74.39], y=[43.0], val=[122]))
canvas = ds.Canvas(plot_height=300, plot_width=300, x_range=(-74, -75), y_range=(42,44))
df1_agg = canvas.points(df1, 'x', 'y', ds.sum('val'))
df1_agg.data = df1_agg.data.astype('float64')
df1_agg.data[np.isnan(df1_agg.data)] = 0.0
df1_proximity = proximity(df1_agg, distance_metric='GREAT_CIRCLE')
np.nanmin(df1_proximity)
Looks like @perrygeo has an awesome library for this...we just want it implemented in numba...@thuydotm let's make sure to give attribution
https://github.com/perrygeo/jenks/blob/master/jenks.pyx
xrspatial/tests/test_focal.py::test_apply_invalid_input
xrspatial/tests/test_focal.py::test_apply_mean
xrspatial/tests/test_focal.py::test_apply_mean
xrspatial/tests/test_focal.py::test_apply_mean
xrspatial/tests/test_focal.py::test_apply_mean
xrspatial/tests/test_focal.py::test_apply_mean
xrspatial/tests/test_focal.py::test_apply_mean
xrspatial/tests/test_focal.py::test_apply_mean
xrspatial/tests/test_focal.py::test_apply_mean
xrspatial/tests/test_focal.py::test_apply_mean
xrspatial/tests/test_focal.py::test_apply_mean
xrspatial/tests/test_focal.py::test_apply_mean
xrspatial/tests/test_focal.py::test_apply_mean
xrspatial/tests/test_focal.py::test_apply_mean
xrspatial/tests/test_focal.py::test_apply_mean
xrspatial/tests/test_focal.py::test_apply_mean
xrspatial/tests/test_focal.py::test_apply_sum
xrspatial/tests/test_focal.py::test_apply_sum
xrspatial/tests/test_focal.py::test_apply_sum
xrspatial/tests/test_focal.py::test_apply_sum
xrspatial/tests/test_focal.py::test_apply_sum
xrspatial/tests/test_focal.py::test_apply_sum
xrspatial/tests/test_focal.py::test_apply_sum
xrspatial/tests/test_focal.py::test_apply_sum
xrspatial/tests/test_focal.py::test_apply_sum
xrspatial/tests/test_focal.py::test_apply_sum
xrspatial/tests/test_focal.py::test_apply_sum
xrspatial/tests/test_focal.py::test_apply_sum
xrspatial/tests/test_focal.py::test_apply_sum
xrspatial/tests/test_focal.py::test_apply_sum
xrspatial/tests/test_focal.py::test_apply_sum
xrspatial/tests/test_focal.py::test_apply_with_nan
xrspatial/tests/test_focal.py::test_apply_with_nan
xrspatial/tests/test_focal.py::test_hotspot_invalid
xrspatial/tests/test_focal.py::test_hotspot
/Users/me/xarray-spatial/xrspatial/focal.py:62: Warning: Raster distance unit not provided. Use meter as default.
'Use meter as default.', Warning)
xrspatial/tests/test_focal.py::test_apply_mean
xrspatial/tests/test_focal.py::test_apply_mean
xrspatial/tests/test_focal.py::test_apply_mean
xrspatial/tests/test_focal.py::test_apply_mean
xrspatial/tests/test_focal.py::test_apply_mean
xrspatial/tests/test_focal.py::test_apply_mean
xrspatial/tests/test_focal.py::test_apply_mean
xrspatial/tests/test_focal.py::test_apply_mean
xrspatial/tests/test_focal.py::test_apply_mean
xrspatial/tests/test_focal.py::test_apply_mean
xrspatial/tests/test_focal.py::test_apply_mean
xrspatial/tests/test_focal.py::test_apply_mean
xrspatial/tests/test_focal.py::test_apply_mean
xrspatial/tests/test_focal.py::test_apply_mean
xrspatial/tests/test_focal.py::test_apply_mean
xrspatial/tests/test_focal.py::test_apply_sum
xrspatial/tests/test_focal.py::test_apply_sum
xrspatial/tests/test_focal.py::test_apply_sum
xrspatial/tests/test_focal.py::test_apply_sum
xrspatial/tests/test_focal.py::test_apply_sum
xrspatial/tests/test_focal.py::test_apply_sum
xrspatial/tests/test_focal.py::test_apply_sum
xrspatial/tests/test_focal.py::test_apply_sum
xrspatial/tests/test_focal.py::test_apply_sum
xrspatial/tests/test_focal.py::test_apply_sum
xrspatial/tests/test_focal.py::test_apply_sum
xrspatial/tests/test_focal.py::test_apply_sum
xrspatial/tests/test_focal.py::test_apply_sum
xrspatial/tests/test_focal.py::test_apply_sum
xrspatial/tests/test_focal.py::test_apply_sum
xrspatial/tests/test_focal.py::test_apply_with_nan
xrspatial/tests/test_focal.py::test_apply_with_nan
xrspatial/tests/test_focal.py::test_hotspot_invalid
xrspatial/tests/test_focal.py::test_hotspot
/Users/me/xarray-spatial/xrspatial/focal.py:125: Warning: Raster distance unit not provided. Use meter as default.
'Use meter as default.', Warning)
returns new xr.DataArray
where we store the direction (0.0-259.0 degrees) to the nearest feature
...instead of distance_to_nearest
from xrspatial.proximity import direction
directions = direction(points_agg)
directions.shape == points_agg.shape
For each one of the validations below:
Hillshade
against example outputSlope
against example outputAspect
against example outputCurvature
against example outputZonal Statistics
against example outputViewshed
against example outputEdge pixels
handled in validation data vs. Xarray-Spatial? Use slope
as the example for your investigation.Polygon Rasterization
(datashader's) against the output in the test dataTake Zonal Examples for User Guide, then expanded to have additional zonal functions
We are going to port from PySAL and then give them appropriate credit...
multispectral.py
moduleRight now proximity requires x
y
named fields in the xarray.DataArray. Lets expose x_field
and y_field
optional arguments as overrides.
At what height, width do we are dropping polygons:
What is the smallest feature in the geodataframe by area?
If we were to represent that feature in a min of 25 pixels...what would the overall height width of the image be?
Code Idea
from xrspatial import suggest_zonal_canvas
my_polygon_df = GeoDataFrame(1M Polygons)
canvas_width, canvas_height = suggest_zonal_canvas(full_extent, min_feature_area, tolerance=.05)
cvs = ds.Canvas(width=canvas_width, height=canvas_height, ...)
cvs.polygons(my_polygon_df)
what should the size of my Canvas be to not drop polygons?
What should the canvas height width be so that the smallest polygon is rasterized with at least 25 pixels?
How about to not drop polygons and achieve a minimum pixel count?
Pharmacy Notebook has a decent example of this...
Let's consider adding https://pypi.org/project/tifffile/
as a dependency for tif io.
read_tif
to readers.py
which mirrors xr.open_rasterio
DEM data is stored as COG on S3
url = 'https://esip-pangeo-uswest2.s3-us-west-2.amazonaws.com/sciencebase/Southern_California_Topobathy_DEM_1m_cog.tif'
da = xr.open_rasterio(url)
da
<xarray.DataArray (band: 1, y: 225217, x: 316120)>
[71195598040 values with dtype=float32]
Coordinates:
* band (band) int64 1
* y (y) float64 3.825e+06 3.825e+06 3.825e+06 ... 3.6e+06 3.6e+06
* x (x) float64 1.795e+05 1.795e+05 1.795e+05 ... 4.956e+05 4.956e+05
Attributes:
transform: (1.0, 0.0, 179523.99999999822, 0.0, -1.0, 3824832.0)
crs: +init=epsg:26911
res: (1.0, 1.0)
is_tiled: 1
nodatavals: (-3.4028230607370965e+38,)
scales: (1.0,)
offsets: (0.0,)
Check out the overviews:
src = rasterio.open(url, 'r')
[src.overviews(i) for i in src.indexes]
[[4, 8, 16, 32, 64, 128, 256, 512, 1023]]
taken from: https://nbviewer.jupyter.org/gist/rsignell-usgs/0f96bb9c0ca34a5dd0fc8131a7bbae1c
I am curious about the project and personally find that web accessible docs are very helpful
Learn how to add arbitrary routes to a Panel applications.
reference: https://docs.bokeh.org/en/latest/docs/dev_guide/server.html
Implement the tile server using url:
/tile/{z}/{x}/{y}.png
and returns Content-Type image/png
.aefbc63827acb2991471098ab.png
)examples/zonal-stats-panel
The holy grail for this example is:
zonal-stats -f dem.tif
Support for dask arrays will allow for distributed spatial analysis.
def distributed_hillshade(block):
return hillshade(block)
g = da.overlap.overlap(mydaskarray, depth={0: 2, 1: 2},
boundary={0: 'periodic', 1: 'periodic'})
g2 = g.map_blocks(distributed_hillshade)
result = da.overlap.trim_internal(g2, {0: 2, 1: 2})
I want to be able to create the input dataset necessary for the ipyleaflet
velocity layer.
Here is an example of what we want to get to:
from ipyleaflet import Map, TileLayer, basemaps
from ipyleaflet.velocity import Velocity
import xarray as xr
import os
if not os.path.exists('wind-global.nc'):
url = 'https://github.com/benbovy/xvelmap/raw/master/notebooks/wind-global.nc'
import requests
r = requests.get(url)
wind_data = r.content
with open('wind-global.nc', 'wb') as f:
f.write(wind_data)
center = [0, 0]
zoom = 1
m = Map(center=center, zoom=zoom, interpolation='nearest', basemap=basemaps.CartoDB.DarkMatter)
ds = xr.open_dataset('wind-global.nc')
display_options = {
'velocityType': 'Global Wind',
'displayPosition': 'bottomleft',
'displayEmptyString': 'No wind data'
}
There is an interesting example of the approach implemented in Rust: https://github.com/mthh/contour-rs
Started a repo here for it: https://github.com/makepath/xarray-spatial-feedstock
I think proximity with GREAT_CIRCLE looks still broken…can you try proximity with GREAT_CIRCLE using the example in user_guide.ipynb and confirm its good?
New Jupyter Notebook in examples
called xarray-spatial-basic-ml.ipynb
The general template for ML examples:
Use an Xarray-Spatial Analysis Function
Take outputs and use as features for an scikit-learn model.
Example 1 Clustering: Proximity To Pharmacies KMeans
Regression Example
Clustering Example using proximity and KMeans.
/examples/user_guide/
/examples/user_guide/0_Getting_Setup.ipynb
/examples/user_guide/1_Surface.ipynb
/examples/user_guide/2_Proximity.ipynb
/examples/user_guide/3_Zonal.ipynb
/examples/user_guide/4_Focal.ipynb
/examples/user_guide/5_Classification.ipynb
/examples/user_guide/6_Conversion.ipynb
()/examples/user_guide/7_Tiles.ipynb
(Generate tileset)/examples/user_guide/8_Remote_Sensing.ipynb
(NDVI, Contrast Enhancment, Segmentation)A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.