Git Product home page Git Product logo

convert_geotiff's Introduction

Build Status convert_geotiff

This is a small commandline utility for converting data from GeoTIFF to geogrid format used by WRF.

Getting the prerequisites

This program requires GeoTIFF and LibTIFF development libraries. These should be available in most if not all package managers. To install in Ubuntu, you just need to install the package libgeotiff-dev like this

sudo apt-get install libgeotiff-dev

Using Homebrew on Mac OSX,

brew install libgeotiff

It is also possible to install the dependencies from source, but you may need to help the configure script below to find the libraries by setting configuration variables CFLAGS="-I$PREFIX/include" and LDFLAGS="-L$PREFIX/lib".

Compiling the source

Download the latest release tarball from here and extract it. In the extracted directory, run ./configure && make. If everything built correctly, you should now have convert_geotiff in the current directly. You can either move this file into your PATH or type sudo make install to install it.

Using convert_geotiff

Running convert_geotiff with no arguments will produce a usage description as follows.

Usage: convert_geotiff [OPTIONS] FileName

Converts geotiff file `FileName' into geogrid binary format
into the current directory.

Options:
-h         : Show this help message and exit
-c NUM     : Indicates categorical data (NUM = number of categories)
-b NUM     : Tile border width (default 3)
-w [1,2,4] : Word size in output in bytes (default 2)
-z         : Indicates unsigned data (default FALSE)
-t NUM     : Output tile size (default 100)
-s SCALE   : Scale factor in output (default 1.)
-m MISSING : Missing value in output (default 0., ignored for categorical data)
-u UNITS   : Units of the data (default "NO UNITS")
-d DESC    : Description of data set (default "NO DESCRIPTION")

All of the files will be created in the current directory, so it is best to run the program from an empty directory.
A more detailed description of the arguments to this program follows.

  • -b :

    The data tiles in the geogrid binary format are allowed to overlap by a fixed number of grid points. The extra border around the tile is called the halo, and this argument sets the width of the halo. For instance with a halo of size three, the file named 00101-00200.00051-00100 would actually contain columns 98-203 and rows 48-103 of the full dataset. This halo is necessary for the interpolation scheme inside of WPS. The default should be acceptable for most situations.

  • -w :

    The number of bytes to represent each data point as an integer. These integers are scaled by the scaling parameter before being truncated to an integer. scaledA lower value will make the output data smaller, at the cost of accuracy or the dynamic range of the input.

  • -m :

    Any grid point that is missing data, such as the outer border of the edge tiles, or grid points that the GeoTIFF file indicates as missing will be set to this value. This argument is currently ignored when the categorical flag is set, instead missing data will be set to the maximum category + 1.

    *-s :

    Because the data is always stored as an integer, a scaling parameter is needed to represent fractional numbers or large values. The data set will be divided by this number prior to being truncated to an integer. If the data set has an accuracy of 2 decimal places, a reasonable scale to use would be 0.01.

    *-u, -d :

    The units and a small description of the data set should be included as arguments. Multi-word arguments should be quoted as follows.

    -u meters -d "elevation above sea level"

  • FileName :

    The final argument must always be present. This is the (absolute or relative) path to the GeoTIFF file to be converted.

If you get an error that says something like "error while loading shared libraries: libgeotiff.so...", this means that GeoTIFF was compiled in as a shared library. You just need to tell the system where to find this library. This can be done by adding the path to the GeoTIFF library to the environment variable LD_LIBRARY_PATH. For example,

export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:${PREFIX}/lib

where $PREFIX is the location where you installed GeoTIFF.

Limitations

The current code has some limitations which are listed here.

  • Datasets must not contain more than 99,999 grid points in each axis. This is a limitation of the geogrid format itself, due to the naming convention of the tiles. However, it is possible (but inefficient) to split a single dataset into multiple directories for this purpose. A better solution would be to resample the data to a lower spatial resolution prior to converting.
  • This program cannot convert between geographic projections, so the input data must be in a projection supported by WPS. All of the projections supported by WPS should work for this conversion program; however, only UTM, Albers equal area, and lat-lon have been tested. In addition, data sources may not conform to EPSG standards in their projection tags; the output should always be checked before use.

Bitdeli Badge

convert_geotiff's People

Contributors

bitdeli-chef avatar fergui avatar janmandel avatar jbeezley avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

convert_geotiff's Issues

dx, dy can be uninitialised in index file

I have found that dx, dy in the index file can be non-deterministic and sometimes incorrect in the index file. For example, here is the output from 2 consecutive runs:

dx = 0.000000e+00
dy = inf
dx = 0.000000e+00
dy = 3.193589e+04

I think the reason for this is that the the line https://github.com/openwfm/convert_geotiff/blob/master/read_geotiff.c#L65 can fail and since idx,dx and idx.dy are not initialised they take on random values. If these values are < 0 then everything is OK because there is code that catches this later on.

How to create an index for my files?

Hello,

In your website it seems like you have a convert_geotiff.x executable that creates the index file for you. However I could not find it in the download...and convert_geotiff only seems to do the conversion...

WPS seems to have troubels finding "truelats"

Hello,

I used the script and it did provide both binary as well as index files, but I can't get the WPS/geogrid to process the data (output HGT_M are all zeroes). I get this message error in the geogrid.log and don't have any clue about the problem's origin.

Processing HGT_M
WARNING: In GEOGRID.TBL, no specification for truelat1 in entry 1.
WARNING: In GEOGRID.TBL, no specification for truelat2 in entry 1.
WARNING: In GEOGRID.TBL, no specification for stdlon in entry 1.

Here is the created index file :

type = continuous
signed = yes
projection = mercator
dx = 0.0025
dy = 0.0025
known_x = 1.0
known_y = 601
known_lat = -18,30125
known_lon = -150.26125
row_order = top_bottom
wordsize = 8
tile_x = 625
tile_y = 601
tile_z = 1
tile_bdr=0
endian = little
units="meters MSL"
description="Topography height"

This is what my namelist.wps looks like ("THT" being the new high-res. elevation data I want to add) :

&share
 wrf_core = 'ARW',
 max_dom = 4,
 start_date = 2005-01-01_00:00:00,2005-01-01_00:00:00,2005-01-01_00:00:00,2005-01-01_00:00:00,
 end_date   = 2005-01-01_12:00:00,2005-01-01_12:00:00,2005-01-01_12:00:00,2005-01-01_12:00:00,
 interval_seconds = 21600,
 io_form_geogrid = 2,
 opt_output_from_geogrid_path = '/home3/datawork/pmauger/CONFIGS/V3_wrfout_base_coarse/WRF_FILES/WPS_DATA/HIGHRES',
 debug_level = 0
/

&geogrid
 parent_id         =   1,   1,  2,  3,
 parent_grid_ratio =   1, 5, 3,3,
 i_parent_start    =   1, 47, 323,40,
 j_parent_start    =   1, 15, 65,41,
 e_we              =   211, 441, 121,121,
 e_sn              =   79, 166, 121,121,
 geog_data_res     =   30s, 30s, THT+30s,THT+30s,
 dx = 105000,
 dy = 105000,
 map_proj = 'mercator',
 ref_lat   = -10.0000
 ref_lon   = -158.4
 truelat1  = -10.0000,
 truelat2  = 0.0,
 stand_lon = -158.4,
 geog_data_path = '/home3/datawork/pmauger/DATA/geog'
 opt_geogrid_tbl_path ='/home1/datawork/sjullien/MODELS/WRF/WPSV3.6.1_paul/WPS/geogrid'
/

Have you any idea what could have gone wrong ? Thank you.

configure error

I've installed the dependency library (libtiff and geotiff) and gave the path of CFLAGS and LDFLAGS. But there are still error when configuring. Kindly help me resolve the issue.
Looking forward to any kind of help.
image
I guess maybe my dependency library installation was successful, but incomplete.

-lm missing in makefile and problem with geotiff headers not being found

I've had a pair of problems installing this (Ubuntu, Debian and openSuSE)

  • On the three systems, after configure, the makefile is missing the "-lm" in LIBS so that the mathematical library is loaded. This causes an error in "make".

  • On openSuSE, GEOTIFF headers (geotiffio.h) was not found and have to provide the path with:
    export CPPFLAGS="-I/usr/include/geotiff/"

You might consider provide fixes for this.

Thank you and keep up the good work!

problem on make

when doing make, I have this problem

gcc -DRELATIVE_GTIFF=geotiff/ -I/include -L/lib -o convert_geotiff convert_geotiff.o geogrid_tiles.o read_geotiff.o write_geogrid.o -lgeotiff -ltiff
/usr/bin/ld: geogrid_tiles.o: undefined reference to symbol 'ceil@@GLIBC_2.2.5'
//lib/x86_64-linux-gnu/libm.so.6: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status
Makefile:356: recipe for target 'convert_geotiff' failed
make: *** [convert_geotiff] Error 1

rounding error in dx and dy in the index file

I recently used convert_geotiff to transform the 250m CORINE land cover to wps format.

I found out that there is a small rounding error in the dx and dy values in the "index" file. The program writes:

dx = 3.246043e-03
dy = 3.247010e-03

But gdalinfo returns:

dx = 0.003247108021919
dy = 0.003247108021919

The error is small, but enough to cause a small shift of about 1 km to the west in the data for the city of Madrid.

MISSING -m ignored for categorical data

"-m MISSING : Missing value in output (default 0., ignored for categorical data)"

I'm pretty sure, that ignoring missing data for categorical type is wrong. I tried to use Corine landuse for Poland, but it makes false data at the borders (for Russia and Belaruse), like there are no cities, and always ice cover, etc.. If there will be missing data (like -1 or something), WRF geogrid could use different data source for landuse, like MODIS. Right?
Now Corine data is useless at the border of UE.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.