go-whosonfirst-sqlite-index's Introduction

go-whosonfirst-sqlite-index

Go package for indexing SQLite databases using table constucts defined in the aaronland/go-sqlite/v2 package and records defined by the whosonfirst/go-whosonfirst-iterate/v2 package.

Documentation

Tools

$> make cli
go build -mod vendor -o bin/example cmd/example/main.go

example

$> ./bin/example -h
Usage of ./bin/example:
  -database-uri string
    	 (default "modernc://mem")
  -emitter-uri string
    	A valid whosonfirst/go-whosonfirst-iterate/v2 URI. Valid schemes are: directory://,featurecollection://,file://,filelist://,geojsonl://,null://,repo://. (default "repo://")
  -live-hard-die-fast
    	Enable various performance-related pragmas at the expense of possible (unlikely) database corruption (default true)
  -post-index
    	Enable post indexing callback function
  -timings
    	Display timings during and after indexing

For example:

$> ./bin/example -dsn 'modernc://cwd/test.db' /usr/local/data/sfomuseum-data-architecture/
2021/02/18 11:34:58 time to index paths (1) 403.514656ms

$> sqlite3  test.db 
SQLite version 3.28.0 2019-04-15 14:49:49
Enter ".help" for usage hints.
sqlite> .tables
example

sqlite> SELECT COUNT(id) FROM example;
12751

Watchers

go-whosonfirst-sqlite-index's Issues

missing dependency?

make tools
go build -mod vendor -o bin/wof-sqlite-index-features cmd/wof-sqlite-index-features/main.go
build command-line-arguments: cannot load github.com/whosonfirst/go-whosonfirst-sqlite-index: open /code/whosonfirst/go-whosonfirst-sqlite-features-index/vendor/github.com/whosonfirst/go-whosonfirst-sqlite-index: no such file or directory
Makefile:6: recipe for target 'tools' failed
make: *** [tools] Error 1

manually entering vendor/github.com/whosonfirst and cloning the github.com/whosonfirst/go-whosonfirst-sqlite-index repo resolved the issue and I was able to build successfully.

How are the dependencies managed? I tried using dep ensure github.com/whosonfirst/go-whosonfirst-sqlite-index but it gave the error could not find project Gopkg.toml, use dep init to initiate a manifest, so I'm guessing you're not using that to manage the deps?

geojson rows missing from sqlite distributions

The continent of Asia is not present in the geojson table, but it is available in the spr table:

sqlite> select * from geojson where id = 102191569;

sqlite> select * from spr where id = 102191569;
102191569||-4251112170830360571|multipolygon|XX||34.5444195|0.0|-12.199965|-180.0|34.5444195|180.0|-1|-1|-1|-1|-1|||-1

Further checking of the whosonfirst-data-admin-xy-latest.db database reveals that this is not the only record missing:

sqlite> SELECT COUNT(*) FROM geojson;
16052

sqlite> SELECT COUNT(*) FROM spr;
18572

sqlite> SELECT COUNT(*) FROM spr LEFT JOIN geojson USING(id) WHERE geojson.id IS NULL;
2520

I downloaded whosonfirst-data-admin-us-latest.db to check if it was an isolated issue, unfortunately, it isn't:

sqlite> SELECT COUNT(*) FROM geojson;
234651

sqlite> SELECT COUNT(*) FROM spr;
273935

sqlite> SELECT COUNT(*) FROM spr LEFT JOIN geojson USING(id) WHERE geojson.id IS NULL;
39284

Recommend Projects

whosonfirst / go-whosonfirst-sqlite-index Goto Github PK

go-whosonfirst-sqlite-index's Introduction

go-whosonfirst-sqlite-index

Documentation

Tools

example

See also

go-whosonfirst-sqlite-index's People

Watchers

go-whosonfirst-sqlite-index's Issues

missing dependency?

geojson rows missing from sqlite distributions

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent