Git Product home page Git Product logo

go-grib2's Introduction

GRIB2 library for Go programming language (golang)

History

Once upon time I had a need to read the GRIB2 file in golang. I used Google search and found the following libraries:

Library nilsmagnus/grib looked reassuringly. I downloaded gfs.t00z.pgrb2.0p25.f001 file from nomads.ncep.noaa.gov, ran grib export and got error format is not supported (((.

I decided to develop GRIB2 library by myself. I looked at nilsmagnus/grib and libraries for Java and Python. All libraries are attempts to understand how GRIB2 is arranged inside and to develop GRIB2 engine using appropriated programming language.

There was the reference GRIB2 implementation - wgrib2 tool. Look at wgrib2 source code. It's a lot of "crazy" C language code ))). As for me it's very difficult to understand how it works in details. There was the way to call wgrib2 from golang application. But I rejected that due to the following reasons:

  • wgrib2 is not thread-safe. There are a lot of global read/write variables inside code.

  • I prefer pure golang libraries

I saw the only one way - to port wgrib2 to golang. I did not want to repeat approach of another libraries. I was not going to rewrite wgrib2 to golang. I decided to port wgrib2 engine C code to golang with minimal correction and develop simple wrapper. I used results of wgrib2 export to CSV as the reference for testing. That approach allows:

  • safe time (porting spent few days only)
  • easy apply new features and patches from original wgrib2 development stream

I used grib2-v2.0.6c version. Here is the result of my work. The work is on final stage.

How-to

go get -u github.com/amsokol/go-grib2

It contains only one function that reads GRIB2 file:

func Read(data []byte) ([]GRIB2, error)

where GRIB2 is the structure with parsed data:

// GRIB2 is simplified GRIB2 file structure
type GRIB2 struct {
    RefTime     time.Time
    VerfTime    time.Time
    Name        string
    Description string
    Unit        string
    Level       string
    Values      []Value
}

// Value is data item of GRIB2 file
type Value struct {
    Longitude float64
    Latitude  float64
    Value     float32
}

Unlike wgrib2 It's thread-safe.

See the following usage examples:

Roadmap

Major work is finished. go-grib2 is ready to use. I have a little bit to port:

  • parse longitude and latitude are stored as gauss, space_view, irr_grid
  • parse data are stored using complex and run_length algorithms

Issues

go-grib2 does not support jpeg, png and aec data package formats. The reason is wgrib2 uses external C libraries to read these formats. If you have an idea how to easy port jpeg, png and aec please let me know.

Need help

I need help is testing. Please test go-grib2 with several GRIB2 files from several providers. Use wgrib2 CSV export to validate results. Please create issues here.

If you need fully functional GRIB2 library look at the my golang wrapper for ecCodes

go-eccodes

go-grib2's People

Contributors

lhemala avatar nilsmagnus avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

go-grib2's Issues

MeteoFrance AROME

Hi, I saw in the other opened issue that you were working on meteofrance data.
Right now it's not working, giving :

$ go run main.go AROME_0.01_SP1_37H_201711231200.grib2
panic: runtime error: index out of range
goroutine 1 [running]:
github.com/amsokol/go-grib2.Read(0xc42206e000, 0x1835c86, 0x1fffe00, 0x1835c86, 0x1fffe00, 0x0, 0x0, 0xc42003dc00)
/home/user/go/src/github.com/amsokol/go-grib2/reader.go:97 +0x9af
main.main()
/home/user/main.go:27 +0x1eb
exit status 2

Just in case I tried with the opened pullrequest :

$ go run main.go AROME_0.01_SP1_37H_201711231200.grib2
2017/11/23 21:33:04 packing type [40] not supported
exit status 1

read the .grib2 data failed

Hello,

I try to open test.grib2 file and get this error message:
Failed to unpack data: packing type [40] not supported

Please advise.

Thank you.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.