pdok / goaf Goto Github PK

View Code? Open in Web Editor NEW

8.0 12.0 3.0 16.4 MB

Implementation of the OCG API - Features specification

License: MIT License

Dockerfile 0.28% Go 29.86% Smarty 0.79% HTML 7.42% CSS 61.56% Shell 0.09%

postgis docker docker-compose oaf golang go openapi openapi-features oapif geopackage

goaf's Introduction

GOAF

ARCHIVED, See GoKoala which is the successor of GOAF.

GOAF is a OGC API - Features implementation in golang.

For who is it?

If you are looking for a very fast OGC API - Features application and are not afraid of some data tuning, this could be the application for you!

Datasources

GOAF supports the following datasources:

OGC GeoPackage
PostGIS (*Postgresql 9.4+)

PDOK server implementation of OGCAPI-FEATURES

A a GeoJSON implementation with a Geopackage as a data provider.

The specification is a preliminary one, with go generate the routing based on api spec, provider interfaces en types structs and convenient parameter extractions are generated to stay easily up to date.

FeatureCollection is overridden in the GeoPackage provider to use the GeoJSON equivalent for decoding blobs
https://github.com/opengeospatial/ogcapi-features/blob/master/core/openapi/ogcapi-features-1.yaml

Build

docker build -t pdok/goaf:latest .

GeoPackage

The GeoPackage provider is a minimal config for GeoPackages that tend to be relative small e.g. < 3 GB.

docker run --rm -v `pwd`/example:/example -e CONFIG=/example/config-addresses-gpkg-minimal.yaml -p 8080:8080 pdok/goaf:latest

PostGIS

More elaborate config optimised performance for huge db (10M+ features/collection)

docker run -v `pwd`/example:/example -e CONFIG=/example/config-addresses-postgis-localhost.yaml' -p 8080:8080 pdok/goaf:latest

Example table

CREATE TABLE addresses.addresses
(
    fid text COLLATE pg_catalog."default" NOT NULL,
    offsetid bigint NOT NULL,
    properties jsonb,
    geom geometry,
    bbox geometry,
      
    CONSTRAINT addresses_addresses_pk PRIMARY KEY (fid)
)
WITH (
    OIDS = FALSE
)

CREATE INDEX addresses_geom_sidx ON addresses.addresses USING GIST (geom);
CREATE INDEX addresses_offsetid_idx ON addresses.addresses(offsetid);

Generate

Some of the code is generated based on the given oas.yaml:

codegen/provider.go
codegen/types.go
server/routing.gen.go

go generate generate/gen.go

Test

go test ./... -covermode=atomic

How to Contribute

Make a pull request...

License

Distributed under MIT License, please see license file within the code for more details.

Thanks

Inspiration and code copied from:

The main differences with regards to jivan is the data provider setup, some geopackage query speedups for larger Geopackages and some tweaks for scanning the SQL features

goaf's People

Contributors

Stargazers

Watchers

Forkers

purple-polar-bear marcoteixeira1977

goaf's Issues

add schema-org markup to html templates

Currently not implemented; see spec

ETL for generating data in (simple) alternative encoding

INSPIRE addresses in alternative encoding have been generated using the following SQL query (assuming databases with normalized datamodel voor INSPIRE addresses) according to these transformation rules:

create schema oaf_poc_2021;
create view oaf_poc_2021.simple_inspire_ad as
select ''                                                                         as alternativeIdentifier,
       ad.validfrom,
       ad.validto,
       null                                                                       as beginLifespanVersion,
       null                                                                       as endLifespanVersion,
       ''                                                                         as building,
       tfname.thoroughfarename                                                    as component_ThoroughfareName,
       pdesc.postaldescriptor                                                     as component_PostalDescriptor,
       adaname.addressareaname                                                    as component_AddressAreaName,
       ''                                                                         as component_AdminUnitName_1,
       ''                                                                         as component_AdminUnitName_2,
       ''                                                                         as component_AdminUnitName_3,
       ''                                                                         as component_AdminUnitName_4,
       ''                                                                         as component_AdminUnitName_5,
       ''                                                                         as component_AdminUnitName_6,
       ad.designator_type_two                                                     as locator_designator_addressNumber,
       ad.designator_type_three                                                   as locator_designator_addressNumberExtension,
       ad.designator_type_four                                                       locator_designator_addressNumber2ndExtension,
       'unit level'                                                               as locator_level,
       'http://inspire.ec.europa.eu/codelist/LocatorLevelValue/unitLevel'         as locator_href,
       ''                                                                         as locator_designator_buildingIdentifier,
       ''                                                                         as locator_designator_buildingIdentifierPrefix,
       ''                                                                         as locator_designator_cornerAddress1stIdentifier,
       ''                                                                         as locator_designator_cornerAddress2ndIdentifier,
       ''                                                                         as locator_designator_entranceDoorIdentifier,
       ''                                                                         as locator_designator_floorIdentifier,
       ''                                                                         as locator_designator_kilometrePoint,
       ''                                                                         as locator_designator_postalDeliveryIdentifier,
       ''                                                                         as locator_designator_staircaseIdentifier,
       ''                                                                         as locator_designator_unitIdentifier,
       ''                                                                         as locator_name,
       ''                                                                         as parcel,
       ''                                                                         as parentAddress,
       ad.geom                                                                    as geom,
       'entrance'                                                                 as position_specification,
       'http://inspire.ec.europa.eu/codelist/GeometrySpecificationValue/entrance' as position_specification_href,
       'by administrator'                                                         as position_method,
       'http://inspire.ec.europa.eu/codelist/GeometryMethodValue/byAdministrator' as position_method_href,
       true                                                                       as position_default,
       ''                                                                         as status,
       ''                                                                         as status_href
from inspire_ad
         .address ad
         INNER JOIN inspire_ad.thoroughfarename tfname on ad.thoroughfarename_id = tfname.localid
         INNER JOIN inspire_ad.postaldescriptor pdesc on ad.postaldescriptor_id = pdesc.localid
         INNER JOIN inspire_ad.addressareaname adaname on ad.addressareaname_id = adaname.localid
;

Using this view a GeoPackage can be generated with ogr2ogr:

export PGPASSWORD="postgres"
export PGCONN="PG:dbname='pdok' host='my-db-host' port='5432` user='postgres'"; ogr2ogr -f GPKG data.gpkg "$PGCONN" -oo ACTIVE_SCHEMA=oaf_poc_2021  -sql "select * from simple_inspire_ad where geom && 
ST_MakeEnvelope(4.6445,52.9685,4.9466,53.2027,4258)" -nln simple_inspire_ad

Note that with the above command the data is still in its source projection, namely EPSG:4258.

Cluster data on geohash for better usability when browsing data (features will be in same area when paging through features):

CREATE INDEX simple_inspire_ad_texel_geohash ON oaf_poc_2021.simple_inspire_ad_texel (ST_GeoHash(ST_Transform(geom,4326)));
CLUSTER oaf_poc_2021.simple_inspire_ad_texel USING simple_inspire_ad_texel_geohash;

application/geo+json regarding Requirements Class "GeoJSON"

http://docs.opengeospatial.org/DRAFTS/17-069r4.html -> 8.3. Requirements Class "GeoJSON"

generate jsonld representation for all resources

something goes wrong with the generation of the next links

This is with a PostGIS datasource
It does seem to work with a GPKG datasource

"custom" configurable Links

Make it possible to configure Links.
One of the practical uses would be a Link pointing towards the "full" dataset (URL containing a prepared download)

Fail according to spec on requests with invalid content-type

See Requirement 9 /req/core/query-param-invalid: The server SHALL respond with a response with the status code 400, if the request URI includes a query parameter that has an invalid value.

Two known issues (possibly more):

goaf defaults to respond with json even if an invalid content-type is requested
requesting non-existing collections respoonds with 200 response

Bevindingen high5 GeoNovum-PDOK

vraag: Wat te doen met structureel lege velden in alternative encoding? (want in brondata ontreken deze velden)
vraag: Is het logisch om bij het uitserveren van data in INSPIRE simplified encoding te linken naar de volledige download van de geharmonizeerde encoding versie? Of is een simplified encoding van een geharmonizeerde dataset een eigen dataset?
Als je beide aanbiedt in service als full download, hoe maak je onderscheid tussen de twee verschillende versies (geharmonizeerd en simplified)?
Spec lijkt niets te zeggen over response body in het geval van errors. Wat is best practice? Is dit altijd een json response, of respecteert dit de request content-type in grote lijnen (geojson,jsonld,json-> json, html>html) http://docs.opengeospatial.org/is/17-069r3/17-069r3.html#http_status_codes

use Accept request header instead ""Content-Type"

See

goaf/core/util.go

Line 122 in 532ca5f

reqContentType := r.Header.Get("Content-Type")

Content-Type is not a request header, but a response header...

update viewers to use webmercator and wgs84

update the feature viewers to use webmercator for basemap and wgs84 for the features retrieved from (g)oaf.

mixed case in yaml config

need to lowercase the yaml config files

flexible way of consuming the oaf.yml

The yaml contains now a lot of Kadaster/PDOK specific configuration

update the example data

The example data used now is a very specific Dutch database, it would be better to swap it out with a more 'common' known dataset like Addresses/Building (or something along those lines)

template oas spec to facilitate service metadata

Run OAF validator on goaf

See OGC validator

Refactor viewer to eliminate code duplication

Currently the viewer is duplicated twice:

Refactor so both components use the same viewer js template, but with some config.

is it required to set license on service level?

I would assume that licenses are only applicable on dataset level, maybe can be set on service level if licenses for all collections are the same.

See https://demo.pygeoapi.io/master

update the Conformance classes

the current classes are using the old wfs-1/3.0/.. URI's

rebranding wfs-3.0 -> goapi-features

like the title suggest we need to rename the application/repo/parts of the code, to scrub out the old OAF name of WFS3.0.

cleanup old WFS(3) references

created #13

proper postgis docker-compose example

The postgis implementation is layered around the jsonb type, while this is a fast solution. It does require some effort regarding the preperation of the data, this also applies to our docker-compose example.
To get the example working without issues/obstakels when need to prepare some example data that can be imported efficiently in to postgis

refactor app config into one config file to rule them all

Currently some config options can only be from the environment or as CLI param.

All config options should be configurable from one yaml file.

replace text/template lib with html/template

zie text/template docs:

To generate HTML output, see package html/template, which has the same interface as this package but automatically secures HTML output against certain attacks.

Om te voorkomen dat er in de toekomst nog template-gerelateerde lijken uit de kast gaan vallen.