Git Product home page Git Product logo

go-whosonfirst-spatial-www's Introduction

go-whosonfirst-spatial-www

Opinionated web application for the go-whosonfirst-spatial packages.

Documentation

Documentation is incomplete at this time.

Example

package main

import (
	_ "github.com/whosonfirst/go-reader-cachereader"
)

import (
	"context"
	"github.com/whosonfirst/go-whosonfirst-spatial-www/application/server"
	"log"
)

func main() {

	ctx := context.Background()
	logger := log.Default()

	err := server.Run(ctx, logger)

	if err != nil {
		logger.Fatal(err)
	}
}

The default server implementation uses an in-memory RTree-based spatial index that needs to be populated when the server is started.

There are also server implementations that use SQLite and Protomaps derived spatial databases:

Tools

$> make cli
go build -mod vendor -o bin/server cmd/server/main.go

server

$> ./bin/server -h
  -authenticator-uri string
    	A valid sfomuseum/go-http-auth URI. (default "null://")
  -cors-allow-credentials
    	Allow HTTP credentials to be included in CORS requests.
  -cors-origin value
    	One or more hosts to allow CORS requests from; may be a comma-separated list.
  -custom-placetypes string
    	A JSON-encoded string containing custom placetypes defined using the syntax described in the whosonfirst/go-whosonfirst-placetypes repository.
  -enable-cors
    	Enable CORS headers for data-related and API handlers.
  -enable-custom-placetypes
    	Enable wof:placetype values that are not explicitly defined in the whosonfirst/go-whosonfirst-placetypes repository.
  -enable-geojson
    	Enable GeoJSON output for point-in-polygon API calls.
  -enable-gzip
    	Enable gzip-encoding for data-related and API handlers.
  -enable-www
    	Enable the interactive /debug endpoint to query points and display results.
  -is-wof
    	Input data is WOF-flavoured GeoJSON. (Pass a value of '0' or 'false' if you need to index non-WOF documents. (default true)
  -iterator-uri string
    	A valid whosonfirst/go-whosonfirst-iterate/v2 URI. Supported schemes are: directory://, featurecollection://, file://, filelist://, geojsonl://, null://, repo://. (default "repo://")
  -leaflet-initial-latitude float
    	The initial latitude for map views to use. (default 37.616906)
  -leaflet-initial-longitude float
    	The initial longitude for map views to use. (default -122.386665)
  -leaflet-initial-zoom int
    	The initial zoom level for map views to use. (default 14)
  -leaflet-max-bounds string
    	An optional comma-separated bounding box ({MINX},{MINY},{MAXX},{MAXY}) to set the boundary for map views.
  -log-timings
    	Emit timing metrics to the application's logger
  -map-provider-uri string
    	A valid aaronland/go-http-maps/provider URI. (default "leaflet://?leaflet-tile-url=https://tile.openstreetmap.org/{z}/{x}/{y}.png")
  -path-api string
    	The root URL for all API handlers (default "/api")
  -path-data string
    	The URL for data (GeoJSON) handler (default "/data")
  -path-ping string
    	The URL for the ping (health check) handler (default "/health/ping")
  -path-pip string
    	The URL for the point in polygon web handler (default "/point-in-polygon")
  -path-prefix string
    	Prepend this prefix to all assets (but not HTTP handlers). This is mostly for API Gateway integrations.
  -properties-reader-uri string
    	A valid whosonfirst/go-reader.Reader URI. Available options are: [cachereader:// fs:// null:// repo:// stdin://]. If the value is {spatial-database-uri} then the value of the '-spatial-database-uri' implements the reader.Reader interface and will be used.
  -server-uri string
    	A valid aaronland/go-http-server URI. (default "http://localhost:8080")
  -spatial-database-uri string
    	A valid whosonfirst/go-whosonfirst-spatial/data.SpatialDatabase URI. options are: [rtree://] (default "rtree://")

For example:

$> bin/server \
	-spatial-database-uri 'rtree:///?strict=false' \
	-enable-www \	
	/usr/local/data/sfomuseum-data-architecture
	
11:44:31.902988 [main][index] ERROR 1159157931 failed indexing, (rtreego: improper distance). Strict mode is disabled, so skipping.
11:44:32.073804 [main] STATUS finished indexing in 744.717822ms

When you visit http://localhost:8080 in your web browser you should see something like this:

If you don't need, or want, to expose a user-facing interface simply remove the -enable-www and -nextzen-apikey flags. For example:

$> bin/server \
	-enable-geojson \
	-spatial-database-uri 'rtree:///?strict=false' \
	/usr/local/data/sfomuseum-data-architecture

And then to query the point-in-polygon API you would do something like this:

$> curl -XPOST http://localhost:8080/api/point-in-polygon -d '{"latitude": 37.61701894316063, "longitude": -122.3866653442383}'

{
  "places": [
    {
      "wof:id": 1360665043,
      "wof:parent_id": -1,
      "wof:name": "Central Parking Garage",
      "wof:placetype": "wing",
      "wof:country": "US",
      "wof:repo": "sfomuseum-data-architecture",
      "wof:path": "136/066/504/3/1360665043.geojson",
      "wof:superseded_by": [],
      "wof:supersedes": [
        1360665035
      ],
      "mz:uri": "https://data.whosonfirst.org/136/066/504/3/1360665043.geojson",
      "mz:latitude": 37.616332,
      "mz:longitude": -122.386047,
      "mz:min_latitude": 37.61498599208708,
      "mz:min_longitude": -122.38779093748578,
      "mz:max_latitude": 37.61767331604971,
      "mz:max_longitude": -122.38429192207244,
      "mz:is_current": 0,
      "mz:is_ceased": 1,
      "mz:is_deprecated": 0,
      "mz:is_superseded": 0,
      "mz:is_superseding": 1,
      "wof:lastmodified": 1547232156
    }
    ... and so on
}    

By default, results are returned as a list of "standard places response" (SPR) elements. You can also return results as a GeoJSON FeatureCollection by including a format=geojson query parameter. For example:

$> curl -H 'Accept: application/geo+json' -XPOST http://localhost:8080/api/point-in-polygon -d '{"latitude": 37.61701894316063, "longitude": -122.3866653442383}'

{
  "type": "FeatureCollection",
  "features": [
    {
      "type": "Feature",
      "geometry": {
        "type": "MultiPolygon",
        "coordinates": [ ...omitted for the sake of brevity ]
      },
      "properties": {
        "mz:is_ceased": 1,
        "mz:is_current": 0,
        "mz:is_deprecated": 0,
        "mz:is_superseded": 0,
        "mz:is_superseding": 1,
        "mz:latitude": 37.616332,
        "mz:longitude": -122.386047,
        "mz:max_latitude": 37.61767331604971,
        "mz:max_longitude": -122.38429192207244,
        "mz:min_latitude": 37.61498599208708,
        "mz:min_longitude": -122.38779093748578,
        "mz:uri": "https://data.whosonfirst.org/136/066/504/3/1360665043.geojson",
        "wof:country": "US",
        "wof:id": 1360665043,
        "wof:lastmodified": 1547232156,
        "wof:name": "Central Parking Garage",
        "wof:parent_id": -1,
        "wof:path": "136/066/504/3/1360665043.geojson",
        "wof:placetype": "wing",
        "wof:repo": "sfomuseum-data-architecture",
        "wof:superseded_by": [],
        "wof:supersedes": [
          1360665035
        ]
      }
    }
    ... and so on
  ]
}  

Indexing "plain old" GeoJSON

There is early support for indexing "plain old" GeoJSON, as in GeoJSON documents that do not following the naming conventions for properties that Who's On First documents use. It is very likely there are still bugs or subtle gotchas.

For example, here's how we could index and serve a GeoJSON FeatureCollection of building footprints:

$> bin/server
	-spatial-database-uri 'rtree:///?strict=false' \
	-iterator-uri featurecollection:// \
	/usr/local/data/footprint.geojson

And then:

$> curl -s -XPOST 'http://localhost:8080/api/point-in-polygon '{"latitude": 37.61686957521345, "longitude": -122.3903158758416}' \

| jq '.["places"][]["spr:id"]'

"1014"
"1031"
"1015"
"1026"

Support for returning results in the properties or geojson format is not available for "plain old" GeoJSON records at this time.

See also

go-whosonfirst-spatial-www's People

Contributors

dependabot[bot] avatar thisisaaronland avatar tomtaylor avatar

Stargazers

 avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

go-whosonfirst-spatial-www's Issues

Replace all the ?format= stuff with middleware handlers

Let's assume we're talking about PIP operations.

  • Everything should start by being run through /api/point-in-polygon.
  • If there is a transformation it should happen in a separate middleware handler.
  • We can test this idea by re-enabling geojson output as something like /api/point-in-polygon/geojson.
  • If that works we'll need to do the same for ?format=properties

We'll need to update the /api/point-in-polygon handler to accept an optional "next" handler.

Better max bounds controls

"Better" as in the max boundaries controls are being triggered inside of the (max) boundary.

Maybe draw a mask inside/outside the max boundary area to show what's going on.

Allow empty go-whosonfirst-index -mode flag

Principally for use with the go-whosonfirst-spatial-http-sqlite package where the assumption is the -spatial-database-uri flag may reference a database that has already been indexed.

At present the work-around is do this:

go run -mod vendor cmd/server/main.go -... -mode directory://

Where we set the -mode flag but don't pass any files to index.

Fix ?format=properties output

?format=properties&properties=foo,bar,bax does the right thing, namely returns results wrapped in a properties dictionary.

?format=properties returns the default SPR places dictionary.

HTTP response error when hammering the server

I'm sometimes seeing the following response from the client when making up to 16 PIP request at once, filling the CPU of a 16 core machine:

2022/08/12 09:56:06 http: superfluous response.WriteHeader call from github.com/whosonfirst/go-whosonfirst-spatial-pip/api.PointInPolygonHandler.func1 (http.go:126)

I don't think this is a client bug - I've disabled keep alives and am closing the connection after every request.

Add a /data handler to return WOF records from a whonsonfirst/go-reader interface

Basically we need a way for the web application to fetch SPR records from the /point-in-polygon endpoint and then fetch the WOF GeoJSON records over the wire.

As of this writing the application defaults to requesting ?format=geojson records but these are often large and take to long to deliver. Better instead to do a quick PIP query and then inflate the records on the fly.

Data race when HTTP server boots

I was trying to track down a bug where infrequently connections to the PIP endpoint would return EOF in my HTTP Client, so I booted the server up with -race to see if this might be a server error. I don't think it is, but I did spot this in the logs as the server boots.

[spatial] app.go:179: Register /data/ handler
[spatial] app.go:204: Register /api/point-in-polygon handler
==================
WARNING: DATA RACE
Write at 0x00c0001946d0 by goroutine 8:
  github.com/whosonfirst/go-whosonfirst-spatial-www/server.(*HTTPServerApplication).RunWithFlagSet.func1()
      /Users/tom/projects/wof/go-whosonfirst-spatial-www/server/app.go:124 +0x84

Previous write at 0x00c0001946d0 by main goroutine:
  github.com/whosonfirst/go-whosonfirst-spatial-www/server.(*HTTPServerApplication).RunWithFlagSet()
      /Users/tom/projects/wof/go-whosonfirst-spatial-www/server/app.go:161 +0xd74
  github.com/whosonfirst/go-whosonfirst-spatial-www/server.(*HTTPServerApplication).Run()
      /Users/tom/projects/wof/go-whosonfirst-spatial-www/server/app.go:69 +0x1d0
  main.main()
      /Users/tom/projects/wof/go-whosonfirst-spatial-www/cmd/server/main.go:24 +0x4c

Goroutine 8 (running) created at:
  github.com/whosonfirst/go-whosonfirst-spatial-www/server.(*HTTPServerApplication).RunWithFlagSet()
      /Users/tom/projects/wof/go-whosonfirst-spatial-www/server/app.go:122 +0x9bc
  github.com/whosonfirst/go-whosonfirst-spatial-www/server.(*HTTPServerApplication).Run()
      /Users/tom/projects/wof/go-whosonfirst-spatial-www/server/app.go:69 +0x1d0
  main.main()
      /Users/tom/projects/wof/go-whosonfirst-spatial-www/cmd/server/main.go:24 +0x4c
==================
2022/08/12 08:58:19 index 101874113#:0 [-0.26, -0.15]x[52.88, 52.93]
2022/08/12 08:58:19 index 1175612693#:0 [-2.49, -2.25]x[53.42, 53.54]
2022/08/12 08:58:19 index 101872113#:0 [-1.97, -1.85]x[53.28, 53.36]
2022/08/12 08:58:19 index 101853557#:0 [-0.64, -0.58]x[51.68, 51.73]
2022/08/12 08:58:19 index 101854787#:0 [-0.39, -0.33]x[51.79, 51.84]
[etc]

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.