Git Product home page Git Product logo

idb-osm-extraction-tool's Introduction

IDB OSM Extraction Tool

Description and Context


Netlify Status

Friendly front-end for querying OSM features around Guyana from the Overpass API and extracting as a Shapefile.

User Guide


The development environment is completely encapsulated in the Vagrant, Ansible, and Docker files contained in this respository. All management scripts can be found in the ./scripts directory.

Installation Guide

Dependencies

  • Vagrant 1.8+
  • VirtualBox 4.3
  • Ansible 2.1+

Getting Started

Install the application and all required dependencies.

./scripts/setup

Development

Rebuild Docker images and run application.

vagrant up
vagrant ssh
./scripts/update
./scripts/server

Ports

Service Port
Webpack Dev Server 4567

Testing

./scripts/test

Scripts

Name Description
cibuild Build project for CI
clean Free disk space by cleaning up dangling Docker images
console Run interactive shell inside application container
lint Lint source code
server Run Docker Compose services
setup Provision Vagrant VM and run update
test Run unit tests
update Build Docker images

Adding NPM Packages

To add a new NPM package to the project:

  • Manually add the package to the project's package.json file, ensuring that you pin it to a specific version.
  • Run ./scripts/update in the VM.
  • Commit the changes to the following files to git:
    • package.json
    • yarn.lock
  • We usually pin packages to a specific version to minimize build errors.

Creating a Release with Git Flow

This project uses git flow for releases. You can find the latest version of git-flow to install here.

Once it's installed, you'll need to enable git flow in your local version of the repo by typing:

git flow init

Use the default values provided; master for the current release, develop for the next release. Since the release process entails pulling changes from develop into a release, then merging that into master, you should ensure that develop builds successfully on Travis before commencing a release.

After you've enabled git flow, you can use the following commands to make a release, replacing "1.2.3" with the version you're releasing and updating the CHANGELOG.md and package.json to match that version.

git flow release start 1.2.3
vim CHANGELOG.md
vim src/package.json
git add CHANGELOG.md src/package.json
git commit -m "1.2.3"
git flow release publish 1.2.3
git flow release finish 1.2.3

After you've completed the git flow steps, you'll need to push the changes from your local master and develop branches back to the main repository and push the release tags to finalize the release:

git push origin master:master
git push origin develop:develop
git push --tags

Once you've pushed the tags, the new release will appear in the project's "Releases" tab in GitHub. There you can edit the tagged release to add release notes.

Feature configuration

OSM tags are grouped into high level features in the config file. The configuration can take all values of a tag, only certain keys of a tag, and multiple tags. A comprehensive example of a feature configuration is:

{
        label: 'Shop/Business',
        entities: [
            { tag: 'shop' },
            { tag: 'craft' },
            { tag: 'office' },
            {
                tag: 'building',
                values: ['office'],
            },
            {
                tag: 'amenity',
                values: ['restaurant', 'cafe', 'internet_cafe', 'bar', 'biergarten', 'fast_food', 'marketplace', 'fuel'],
            },
        ],
    },

The OSM tags will be converted to an Overpass API query and fetched from that service.

Code of Conduct


Authors


For full list see Contributors

License


The Documentation of Support and Use of the software is licensed under Creative Commons IGO 3.0 Attribution-NonCommercial-NoDerivative (CC-IGO 3.0 BY-NC-ND)

The codebase of this repo uses AM-331-A3 Software License.

Limitation of responsibilities


The IDB is not responsible, under any circumstance, for damage or compensation, moral or patrimonial; direct or indirect; accessory or special; or by way of consequence, foreseen or unforeseen, that could arise:

I. Under any concept of intellectual property, negligence or detriment of another part theory; I

ii. Following the use of the Digital Tool, including, but not limited to defects in the Digital Tool, or the loss or inaccuracy of data of any kind. The foregoing includes expenses or damages associated with communication failures and / or malfunctions of computers, linked to the use of the Digital Tool.

idb-osm-extraction-tool's People

Contributors

eneedham avatar kellyi avatar rajadain avatar rbreslow avatar

Stargazers

 avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

idb-osm-extraction-tool's Issues

Make new release

Once #42 is merged in, create a new release to deploy those changes to the production url

Setup map & sidebar

Following the wireframes, set up the map and sidebar components

Configure the map to use the OSM basemap & be centered in the proper place

Configure the sidebar to have a header & extract button, but no dropdowns, roughly matching the wireframes

Plot OSM query response geojson shapes on map

In #18 we created a stock OSM Overpass API query to execute, then plotted the response's points on the map.

However, when we set up the app to execute real queries against the OSM API, we'll need to update the display of the geojson to allow for lines and polygons as well as points.

Depends on & blocked by #10

Add visual indication that successful requests with no features returned no features

Per #39 (comment)

there's not much good feedback for the user when a query completes successfully but does not return any features -- such as looking for Emergency Infrastructure in the last month. That request will 200 but since there are no features, no shapefile gets generated. I'm going to make an enhancement card for adding a visual indication that "No features were found".

We should add some kind of visual indication when no features are found from the API request.

Add OSM based geocoder

Additionally, allow the geocoder to work on coordinates as Guyana has many remote areas which may not be present in a Gazetteer.

Create actual OSM query from filters

Using the dropdown filters and the area of interest, replace the mocked OSM query from #6 with one that's generated according to the user's selections. Ensure that #7 and #8 continue to work as before

Problem in reading persian characters.

Hello. Thanks for your good program. Just a problem has occurred. When I downloaded shapefile of an area with persian names (i.e. locations in Iran for example) name tags other than ones written in English characters are not not readable in attribute table of the shapefile. It would be very kind of you to help me solve this problem or you, dear developers, resolve the bug, if any.

Click and Release doesn't work for drawing on IE11

In Chrome, after selecting Box Drawing, one can either click and drag a box, or click to start and click again to end drawing the box.

On IE11, each click starts a new box, that never finishes drawing:

2018-08-01 16 04 17

Click and drag works correctly.

Add sidebar dropdown select filters

As depicted in the wireframes, add the dropdown select filters to the sidebar and track the state of things in Redux. Currently we have:

  • one feature dropdown
  • one date range selector dropdown

Replace object prop-types with custom shapes

The app currently uses some object prop-types by disabling the linter for those lines only. However, it would be better if those object prop-types were redefined as distinct shapes.

Add GitHub Issue Templates

As we have done in other projects, we should add GitHub Issue Templates here to make it easier to clearly report bugs or request new features.

Add tests

There are no tests. Test execution is commented out in STRTA.

Relates to #20

Use date range filter to modify overpass queries

The date range filter is wired up to the UI but selecting a different option currently doesn't change the Overpass query. For this card: make the selected date range in the UI limit the overpass query results.

Shapefiles produced from extract are not valid

Upon downloading a shapefile converted from OSM->JSON feature results in errors when opening. The error from multiple systems indicates that this is due to a discrepancy between the count of spatial features and the rows in the DBF. The geojson that is being converted appears to be valid.

Investigate if the current shapefile library is being used incorrectly

  • are we settings the points | lines | polygon properties correctly?
  • are we handling the tags attribute of properties properly? Do they need to be flattened?

If the problem seems endemic to this library, experiment with alternatives.

Add custom date range option to date select dropdown

Along with the set of prepared options for the date range select dropdown, the wireframes indicate also having a "Custom" option which would bring up a date picker component to allow a user to select custom start and end dates.

For this card: implement the custom start and end date feature.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.