Git Product home page Git Product logo

spatialmerge's Introduction

spatialmerge

A (relatively) fast library for spatial joining and merging data in JavaScript.

spatialmerge

The library is largely influenced by the geopandas merge and sjoin methods. Just as geopandas, spatialmerge utilizes a static spatial index provided by Flatbush (R-tree) to narrow down the number of features to test for intersection.

import { merge, sjoin } from 'spatialmerge'
// join the properties of a spatial dataset (GeoJSON FeatureCollection) with a
// non-spatial dataset (e.g. contents of a CSV file parsed using d3-dsv) 
// based on a common variable
const resultsMerge = merge(countryShapes, countryNames, { on: 'iso_a3' })

// join the properties of two spatial datasets (GeoJSON FeatureCollections)
const mergeSjoin = sjoin(citiesPoints, countryShapes)

Working examples, and an in-depth explanation of the two functions can be found in the user guide notebook.

Installing

npm install spatialmerge
# or
yarn add spatialmerge

For vanilla HTML in modern browsers, import spatialmerge from Skypack:

<script type="module">

import { merge, sjoin } from 'https://cdn.skypack.dev/spatialmerge'
// ...

</script>

For legacy environments, you can load spatialmerge’s UMD bundle from an npm-based CDN such as jsDelivr; a spatialmerge global is exported:

<script src="https://cdn.jsdelivr.net/npm/spatialmerge"></script>

User Guide

The User Guide is hosted as an interactive ObservableHQ notebook
https://observablehq.com/@chrispahm/hello-spatialmerge

API reference

spatialmerge.merge(leftFC, rightFC_or_arrayOfObjects, options = { on: <property>, mutate: false }])

Join the attributes of a GeoJSON FeatureCollection with an array of objects or another GeoJSON FeatureCollection based on a common variable (object key).

Parameters:

spatialmerge.sjoin(leftFC, rightFC[, options = { how: 'inner', op: 'intersects', matches: 'all', lsuffix: 'left', rsuffix: 'right' }])

Spatial join of two GeoJSON FeatureCollections.
See the User Guide for details.

Parameters:

  • leftFC, rightFC: <GeoJSON FeatureCollection>, required
  • options: <object>, optional
    • how: <string>, default: 'inner'
      The type of join:
      • β€˜left’: use keys from left_df; retain only left_df geometry column
      • β€˜right’: use keys from right_df; retain only right_df geometry column
      • β€˜inner’: use intersection of keys from both dfs; retain only left_df geometry column
    • op: <string>, default: 'intersects'
      Binary predicate. Internally uses the corresponding turf.js modules.
      • 'intersects'
      • 'contains'
      • 'within'
      • 'crosses'
      • 'overlaps'
    • matches: <string>, default: 'all' Whether to output all results of the join operation, or only the first.
      • 'all'
      • 'first'
    • lsuffix: <string>, default: 'left'
      Suffix to apply to overlapping column names (left GeoJSON).
    • rsuffix: <string>, default: 'right'
      Suffix to apply to overlapping column names (right GeoJSON).
    • inclLeftIndex: <boolean>, default: false
      Whether to include the left index as a property value in the resulting GeoJSON FeatureCollection.

Performance

Spatially merging datasets is (almost) always a computationally intensive task. If you plan to use spatialmerge on the client side or in a Node.js / Deno server, be sure to wrap it in a WebWorker or worker_thread to avoid blocking the Rendering / Event Loop.

Resources:

TODOs

  • Fix tests for sjoin including large files, potentially using git-lfs
  • Example using WebWorker
  • Performance comparison with @turf/tag and geopandas

Contribution

Contribution is highly appreciated πŸ‘
Please open an issue in case of questions / bug reports or a pull request if you implemented a new feature / bug fix. In the latter case, please make sure to run npm test (and adapt test/test.js to your changes) and / or update the README πŸ™‚

License

MIT @Christoph Pahmeyer

spatialmerge's People

Contributors

brahn avatar chrispahm avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

spatialmerge's Issues

sjoin leaves field values undefined when there are duplicate field names

I encountered the same problem as #1: when the left and right objects share field names, the result of sjoin leaves right-side field values undefined.

For a test case, use:

import { readFileSync } from 'fs';
import { join } from 'path';
import { sjoin } from 'spatialmerge';

const readJson = (filepath: string) => JSON.parse(readFileSync(filepath, 'utf8'));
const neighborhoods = readJson(join(__dirname, 'data/Neighborhoods.geojson'));
const restrooms = readJson(join(__dirname, 'data/Parks_Public_Restrooms.geojson'));
const mergeResult = sjoin(restrooms, neighborhoods, { how: "inner", op: "intersects" });

console.log(mergeResult.features[0].properties)
// console output:
{
  OBJECTID: 216,
  Facility: 'AIRPORT LITTLE LEAGUE PARK',
  JURISDICTION: 'City',
  WithinPark: 'Yes',
  GlobalID: '{75922410-F154-4E4A-8315-E418D5D7E6E6}',
  index_right: undefined,
  NAME: undefined,
  SHAPE_Length: undefined,
  SHAPE_Area: undefined
}

I would expect the undefined fields to be populated with values from the properties of the matching neighborhood:

console.log(neighborhoods.features[110].properties)
// console output:
{
  OBJECTID: 111, 
  NAME: 'Golf Course Terrace', 
  SHAPE_Length: 0.08183311421559303, 
  SHAPE_Area: 0.0001623340769874396
}

I believe the problem may be a typo on line 24 of helpers.js. Should that line use the value object2[key] rather than object1[key]?

keys2.forEach(key => {
if (duplicates.indexOf(key) > -1) {
feature[key + '_' + rsuffix] = object2[key]
} else {
feature[key] = object1[key]
}
})
return feature


I started putting together a pull request for a fix but then I encountered some additional issues, which gave me enough pause that I thought I should stop and check my understanding. The additional issues:

  • The lsuffix and rsuffix arguments are not populated in this call to mergeWith, which results in fields being overwritten rather than disambiguated:

    spatialmerge/src/sjoin.js

    Lines 116 to 122 in dff5f13

    properties: mergeWith(leftFeature.properties, {
    ...(how === 'right' && { index_left: matchingRightIndex }),
    ...(how !== 'right' && { index_right: matchingRightIndex }),
    ...(how === 'right' && inclLeftIndex && { index_right: leftIndex }),
    ...(how !== 'right' && inclLeftIndex && { index_left: leftIndex }),
    ...rightGeojson.features[matchingRightIndex].properties
    })

  • This calculation of duplicates doesn't seem right:

    const duplicates = allKeys.filter(key => uniq.indexOf(key) === -1)

    For example:

    const allKeys = ['a', 'b', 'c', 'c', 'a'];
    const uniq = [...new Set(allKeys)];
    const duplicates = allKeys.filter(key => uniq.indexOf(key) === -1);
    console.log(duplicates);
    // actual output: []
    // expected output: ['a', 'c']  (or some permutation)

sjoin right side field values are undefined

The code below couldn't receive field values of parcel. Thus, the fields of parcel are in the output but their values are undefined.
sjoin(buildingGeoJSON, parcel, {how: 'inner', op: 'intersects'});

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.