Git Product home page Git Product logo

maps.zoondka.com's Introduction

maps.zoondka.com

This repository is a collection of Ansible roles for the configuration & provisioning of our tile server @ maps.zoondka.com.

The Stack

Our tile server is a single robust machine composed of various tools & applications for all things vector tiles. It acts as a database for OSM & other map data, a vector tile generator, a database for generated tiles, and a server for both tiles and our zoondka-maps app which consumes them.

Here is an overview of our software stack:

For more details of the various components & their configurations, you can take a look through the roles directory.

Requirements

Ansible
Vagrant (for dev VM)

Usage

We use this same Ansible playbook in production @ Zoondka, so it includes two different sets of variables - dev & production. For testing, feel free to use the default dev variables. I've included a Vagrantfile & you can get up & running with a test VM easily.

Dev VM

With Vagrant installed, you can easily setup a dev VM that mirrors our production environment @ maps.zoondka.com (excluding the actual hardware) . This is useful for testing the software stack:

git clone https://github.com/zoondka/maps.zoondka.com
cd maps.zoondka.com
vagrant up

With the VM up & running, you can re-provision it anytime with vagrant provision or by linking to the vagrant inventory file & using the ansible-playbook command directly:

ln -s .vagrant/provisioners/ansible/inventory/vagrant_ansible_inventory dev
ansible-playbook -i dev playbook.yml 

Now, you can vagrant ssh into the machine to start generating & serving tiles.

Importing Data

Included in this playbook is some OSM test data on Lichtenstein...we can use that as an example:

vagrant ssh
sudo su postgres
cd /srv/data
osm2pgsql --create --slim --flat-nodes nodes.bin -C 2000 --number-processes 1 --hstore liechtenstein-latest.osm.pbf
. post-import-steps.sh # add indexes and custom functions according to osm-bright.tm2source @ https://github.com/kartotherian/osm-bright.tm2source
exit

Generating Tiles

We use Tilerator for generating our vector tiles. For detailed usage, take a look @ their README.

Once you've imported some data, you can vagrant ssh into your machine & fire up tilerator (workers) and tilerator-ui (ui only) services:

sudo systemctl enable tilerator tilerator-ui
sudo systemctl start tilerator tilerator-ui

To talk to Tilerator from your host machine, you can forward the tilerator-ui port via SSH:

vagrant ssh -- -L 6533:localhost:16533

Now, on your host, you can post to localhost:6533 to add a job to Tilerator:

curl -X POST 'http://localhost:6533/add?generatorId=gen&storageId=v2&zoom=0&fromZoom=0&beforeZoom=15&biggerThan=0&parts=12'

Or, in a web browser, navigate to localhost:6533/jobs to take a look at job progress.

Serving Tiles

Once you've generated some tiles, you can serve them with Kartozoa. Simply enable & start the Kartozoa service:

sudo systemctl enable kartozoa
sudo systemctl start kartozoa

Tiles should now be available internally @ localhost:16532/osm-intl/{zoom}/{x}/{y}.pbf and externally @ https://maps.zoondka.com/kartozoa/osm-intl/{zoom}/{x}/{y}.pbf. Take a look @ Kartozoa for more info.

Launching Zoondka Maps

As with Kartozoa, simply enable & start the zoondka-maps service to host our mapping app:

sudo systemctl enable zoondka-maps
sudo systemctl start zoondka-maps

The app should now be available @ localhost:16531 or https://maps.zoondka.com.

Hacking

If you'd like to use parts of this playbook as a starting point for another machine, feel free to fork the repository & enjoy. You will want to create your own production variables and then groom the Ansible roles as you desire. For example, if you'd like to set up a machine just for generating and serving vector tiles using Kartotherian, then this a great repository to start with.

In production, depending on the machine you are using & the dataset you'd like to import, you will probably want to tune your osm2pgsql command, for this take a look at osm2pgsql usage

maps.zoondka.com's People

Contributors

leblowl avatar

Stargazers

 avatar

Watchers

James Cloos avatar  avatar  avatar

maps.zoondka.com's Issues

Map out a proper provisioning plan

The idea of this repository is to record the build of our tile server & allow for us to get the same tile server policy up & running on any machine. I'm currently checking out Ansible. The idea is to setup an Ansible playbook for deploying on to baremetal & then using Vagrant with Ansible for testing that playbook on a VM as we only have one server @ the moment.

Get an idea for how we can balance our resources on maps.zoondka.com

So we have one machine to act as an osm database, tile generator, tile server, and web app server. It would be great to get an idea of how many resources each part of the whole requires, how many cores & how much RAM is needed for the various processes. Load testing the tile server/ web app server would be a great start.

Zero downtime deploys

Zero downtime deploys are awesome & the only way to go when you have folks using your service. I'd like to figure out a simple way to do this.

OSM Updates

We need to set up a job to update our OSM database and re-render changed tiles.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.