Git Product home page Git Product logo

spatialparquet's Introduction

Spatial Parquet

This contains the code associated with the paper:

Majid Saeedan and Ahmed Eldawy. 2022. Spatial Parquet: A Column File Format for Geospatial Data Lakes. In The 30th International Conference on Advances in Geographic Information Systems (SIGSPATIAL โ€™22), November 1โ€“4, 2022, Seattle, WA, USA. ACM, New York, NY, USA, 4 pages. Link

How to Use

To build, run the following on the root directory:

mvn clean install

edu.ucr.cs.bdlab.spatialParquet This package contains the reader/writer implementation of Spatial Parquet. Check the Main class for how they are used.

edu.ucr.cs.bdlab.geoParquet This package contains a reader/writer implementation of the GeoParquet structure.

To use it with FP-Delta encoding, must used a modified version of Parquet. This is done as follows:

  • Change the dependency version of org.apache.parquet from 1.12.2 to 1.13.0-SNAPSHOTpom.xml
  • Go to the folder parquet-mr and follow the build instructions.
  • Build Spatial Parquet again and use it similar to the Main file.

This is currently required because Parquet doesn't provide the option to set a custom encoding. The current implementation of the encoding is experimental and can see some performance improvement in terms of read/write time and providing more options for encoding to provide more flexibility.

spatialparquet's People

Contributors

majidsas avatar

Stargazers

yalei avatar Christos avatar Nikos Koutroumanis avatar

Watchers

 avatar

Forkers

aske-w

spatialparquet's Issues

Custom filter

Hi Majid,

In this class you mention about custom filter in line 404. Do you have any example about this? Is it possible to declare a predicate pushdown filter for repated values that handle the geometry object?

Thank you,
Nikos

parquet-mr mvn install error

I get an error when i'm running mvn install in the parquet-mr folder.
Is there a source available to download it?

Thank you,
Nikos

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.