Git Product home page Git Product logo

geo1101.2020.ahn3's Introduction

GitHub license DOI Documentation Status

Improved AHN3 Gridded DTM/DSM

Comparison between current DTM and improved version

GIF created using JuxtaposeJS


The current gridded DTM and DSM for the Netherlands has some flaws, showing many missing pixels where the interpolation method was unable to determine a value, and has holes where buildings and water bodies were removed from the dataset.

The goal of this project is to improve upon the results available by creating DSM/DTM results that no longer have no-data values. This creates rasters with complete coverage of their target area, ensuring that every pixel value has an accurate height assigned to it.

This has mainly been made possible by the following Python packages and binaries:

Usage

The tool is made to run an amount of threads in parallel to ensure fast processing. Please note that each thread needs around 20GB of memory to run. Optimizations can be made if memory use for quadrant-based IDW is reduced, and/or for Startin.

  1. Install all packages specified in requirements.txt
  2. Configure your settings in the config.ini
    1. Global parameters and folder paths are essential to change
    2. Further parameters are optimized for use with AHN3 dataset
  3. Run main.py

Documentation and help

Read the full documentation at http://geo11012020ahn3.rtfd.io/

For any hiccups you encounter, please create an issue

Notes

For more interpolation methods that have been implemented in the process of this project, see this repository. It holds six different interpolation methods that can be used to replace the ones featured here, and also contains information on how to use them.

The chosen interpolation methods in this repository have been proven to provide the best results for our datasets, and are the most robust encountered.

For DTM a Laplace interpolation is used based on Startin's Delaunay Triangulation, which in turn is based on Rust, making it extremely fast to execute.

For DSM a quadrant-based IDW is used, which is implemented in Python. This method is limited by the amount of raster cells it needs to interpolate for and the speed at which Python can do so. Depending on how small your base raster cell size is this will take a while to run.

Error handling is partially implemented; no errors should occur that will break the processing loop, though some of these errors may result in partial outcomes.

Some of the files include a NO_DATA value defined at the top of the file as -9999, if this needs changing this is where to find it.

Troubleshooting

  • PDAL/GDAL won't install for various reasons
    • Start from a clean Python 3.7 Anaconda virtual environment
    • First install PDAL and GDAL, then the other requirements

Contributors

k  halhoz kriskenesei mdjong1 lkeurentjes mpa-tudelft hugoledoux

geo1101.2020.ahn3's People

Contributors

dependabot[bot] avatar hugoledoux avatar lkeurentjes avatar mdjong1 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

Forkers

simrit1 robodrome

geo1101.2020.ahn3's Issues

Download tiles?

Nice repo!
I was wondering if you have processed all the tiles, and if you offer links to download these tiles?
If not, can you guide me with the naming pattern of original tiles? For instance, tiles are named like this - 06HZ2, 11DN1, etc. Is there a way I can get all the tile names, or if there is a pattern being followed that I am missing?
Thanks.

Flattening of Water

I've found an issue with the flattening of water in the DSM file.

This happend with the new AHN4 09DZ1, subtile 1 (with default 4x5 subtiling).

There are some 6m high ridges in the resulting geotif as you can see here:
image

This is not present in the DTM file, so also tried to create the DSM with Laplace interpolation instead of the QuadIDW:
image
With Laplace the results are different but still a 6m difference.

Maybe this is an result of the overhanging tree which is classified as water? However, the trees are filtered out the LAS for the DTM, which does not have this problem:
image

Could this be handled with filtering the outliers with different parameters?

Edit:
It seems that there's a hole in the data where these ridges appear.
image
image

Would it be possible to only use ground and water for the interpolation? And ignore the other classes for this?
image

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.