Git Product home page Git Product logo

fitsio's Introduction

fitsio

GitHub release CI codecov GoDoc

fitsio is a pure-Go package to read and write FITS files.

Installation

$ go get github.com/astrogo/fitsio

Documentation

https://godoc.org/github.com/astrogo/fitsio

Contribute

astrogo/fitsio is released under BSD-3. Please send a pull request to astrogo/license, adding yourself to the AUTHORS and/or CONTRIBUTORS file.

Example

import fits "github.com/astrogo/fitsio"

func dumpFitsTable(fname string) {
    r, err := os.Open(fname)
    if err != nil {
        panic(err)
    }
    defer r.Close()
	f, err := fits.Open(r)
	if err != nil {
		panic(err)
	}
	defer f.Close()

	// get the second HDU
	table := f.HDU(1).(*fits.Table)
	nrows := table.NumRows()
    rows, err := table.Read(0, nrows)
    if err != nil {
        panic(err)
    }
    defer rows.Close()
	for rows.Next() {
        var x, y float64
        var id int64
        err = rows.Scan(&id, &x, &y)
        if err != nil {
            panic(err)
        }
        fmt.Printf(">>> %v %v %v\n", id, x, y)
	}
    err = rows.Err()
    if err != nil { panic(err) }
    
    // using a struct
    xx := struct{
        Id int     `fits:"ID"`
        X  float64 `fits:"x"`
        Y  float64 `fits:"y"`
    }{}
    // using a map
    yy := make(map[string]interface{})
    
    rows, err = table.Read(0, nrows)
    if err != nil {
        panic(err)
    }
    defer rows.Close()
	for rows.Next() {
        err = rows.Scan(&xx)
        if err != nil {
            panic(err)
        }
        fmt.Printf(">>> %v\n", xx)

        err = rows.Scan(&yy)
        if err != nil {
            panic(err)
        }
        fmt.Printf(">>> %v\n", yy)
	}
    err = rows.Err()
    if err != nil { panic(err) }
    
}

TODO

  • [DONE] add support for writing tables from structs
  • [DONE] add support for writing tables from maps
  • [DONE] add support for variable length array
  • provide benchmarks wrt CFITSIO
  • add support for TUNITn
  • add support for TSCALn
  • add suport for TDIMn
  • add support for (fast, low-level) copy of FITS tables

fitsio's People

Contributors

airnandez avatar bjet007 avatar brandondube avatar rickbassham avatar sbinet avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

fitsio's Issues

handle big.Int

FITS files headers may hold integers bigger than what int64 can encode.

e.g.:

QID     = 40002100000000422948
QOBSEQID= 30001200000000003545

in that case: convert to math/big.Int

view-fits: support for pan

it should be possible to pan the image (when it has been zoomed in).

probably with the usual combination of holding Shift/Ctrl plus left-button of the mouse

view-fits: support for zoom-in/zoom-out

it should be possible to zoom-in/zoom-out inside an image displayed by view-fits:

  • + to zoom-in
  • - to zoom-out
  • use the mouse's wheel to zoom-in/zoom-out
  • use the mouse's left button to create a box to zoom-in

fitsio: Precision lost when writing small float64 value in the header

Hi,

I'm trying to generate a fits file float64 value in the header. Since most of my values are small number with a lot of precision, decimal get truncated because of the way the float64 get printed in the document using %20f.

Looking at the specification (https://fits.gsfc.nasa.gov/standard40/fits_standard40aa-le.pdf), the limitation is on the number of bytes used, so I was expecting that I get the same value that I've specified in my go program.

Is it possible that the writer should use %20g instead of %20f so that value don't lose precision if value fit within the length of the spec.

I've create a small sample here to show the behaviour
https://play.golang.org/p/oqFmTQBeNOD

add support for image.Image

The Image HDU should have an Image() image.Image method so clients could directly display the image using the standard library image package.

fitsio: Expected behaviour with precision on floating point number

Hi,

I'm trying to generate a fits file float64 value in the header. Since most of my values are small number with a lot of precision, decimal get truncated because of the way the float64 get printed in the document using %20f.

Looking at the specification (https://fits.gsfc.nasa.gov/standard40/fits_standard40aa-le.pdf), the limitation is on the number of bytes used, so I was expecting that I get the same value that I've specified in my go program.

Is it possible that the writer should use %20g instead of %20f so that value don't lose precision if value fit within the length of the spec.

I've create a small sample here to show the behaviour
https://play.golang.org/p/oqFmTQBeNOD

Data is lost when writing/reading table from map

Hi, I found that when writing and reading binary tables from a map, the recovered data is always zero value. For example strings are empty "" and numeric values are 0.

This can be easily replicated with the TestTableMapsRW in table_test.go. Even though the test passes, the values that are scanned to the map do not match with the original data.

I'm still a beginner at Go, but I think that calling reflect.Value.Interface returns the default value of the interface, so that's why it reads numeric values as 0 and strings as empty strings.

Panic on Duplicate Keys

This is obviously by design.

Some of my images saved by Sequence Generator Pro have DEWPOINT specified twice. Since this library panics on duplicate keywords, I can't use this to parse my fits files. I've sent them a request to fix it on their end.

I'm not sure doing a panic is the right thing to do here, although short of just logging it out or silently ignoring it, neither of which are good options, I'm not sure what the best approach to handle this is. I'll probably fork and silently ignore for my own work, but I'm not sure what the rest of the community would prefer on this.

Thanks for the awesome lib!

Creating an empty fits file

Hi,

I would like to create empty fits files, but I didn't manage to do it : I got the 252 status.

Here is what I do :

from fitsio import FITS

fits = FITS('test.fits', 'rw', clobber=True)
fits.close()
fits = FITS('test.fits')
---------------------------------------------------------------------------
IOError                                   Traceback (most recent call last)
<ipython-input-41-d0bdd6cc9830> in <module>()
----> 1 fits = FITS('test.fits')

/usr/local/anaconda3/envs/py27/lib/python2.7/site-packages/fitsio/fitslib.pyc in __init__(self, filename, mode, **keys)
    351                 raise IOError("File not found: '%s'" % filename)
    352 
--> 353         self._FITS =  _fitsio_wrap.FITS(filename, self.intmode, create)
    354 
    355     def close(self):

IOError: FITSIO status = 252: 1st key not SIMPLE or XTENSION
Extension doesn't start with SIMPLE or XTENSION keyword. (ffrhdu)
END
Error while closing HDU number 0 (ffchdu).
Extension doesn't start with SIMPLE or XTENSION keyword. (ffrhdu)
END
ffopen could not interpret primary array header of file: 
test.fits
This does not look like a FITS file.

Any idea on how to do that ?

Thanks in advance,
Thomas.

fitsio: reflection heavy r/w code

encoding/decoding tables relies heavily on reflect (by way of gonuts/binary).

drop the dependency on gonuts/binary and replace with fast-path handwritten code for all the []T types.

fitsio: card value as float64

#42 exposed an interesting edge case: values that look like ints (e.g. 4) but should really be handle as float64.

perhaps Value should grow an AsFloat64() or Float64() method that converts an int into a float64?

Is it possible to read only the Header Unit and not the Data Unit?

I'm evaluating astrogo/fitsio for scanning the header units of hundreds of thousands of FITS files. I'm not interested in the contents of the data units.

I noticed that the method DecodeHDU reads not only the Header Unit but also its associated Data Unit.

This behavior is inconvenient for my use case. The structure of each one of the 800+K FITS files I need to scan is basically:

  • Each file has a single HDU
  • Size of the Header Unit: 14400 bytes
  • Size of the Data Unit: 18187200 bytes
  • Total file size: 18+ MB

Is there any way to tell astrogo/fitsio not to consume (i.e. read from disk) the data unit (i.e. 18MB in this particular case) but only the header unit (i.e. 14 KB in this particular case)? Given the number and the structure of the files I need to scan, that would greatly help.

view-fits: support for remote files

it would be great to be able to open remote files, served, say, over http(s).

protocols which could be supported at first:

  • http://
  • https://
  • file:// (so, local files too)

do you want to give it a shot, @tbellembois ?

Handling of COMMENT, HISTORY, END cards differs wrt cfitsio

The returned value of Header.Keys() does not include cards of type END, COMMENT, HISTORY:

func (hdr *Header) Keys() []string {

I understand that there is a method for retrieving the comments and history cards of a FITS header, I would suggest that for compatibility with cfitsio, those cards should be also be returned by Header.Keys().

In addition, it seems that astrogo/cfitsio assumes that in a given HDU there is only one card of type HISTORY or COMMENT. This is not the case with cfitsio.

In case it matters, for my tests I'm using cfitsio v3.370.

An example of a file with several COMMENT cards in the same HDU can be found at:

ftp://legacy.gsfc.nasa.gov/fits_info/sample_files/images/rosat_pspc_rdf2_3_bk1.fits

A problem in file "fitsio/decode.go"

I find "fitsio.Open()" can't open large files. When I use "fitsio.Open()" to open a 1.2 Gigabytes file, I encounter an error, which is:

fitsio: error loading binary table: fitsio: read too few bytes (1073741824). expected 1122704640

I think the problem is in file "decode.go", line 249, which is n, err := dec.r.Read(block)
After I change the line to io.ReadAtLeast(dec.r, block, len(block)) , the problem is solved.

I realize 1073741824=2^30, so I think the problem is about the "Read()" function. I'm a newbie to Golang, so I'm not sure if "Read()" funciton can only read up to 2^30 once. In short, I think there should be some changes in source file "decode.go", line 249.

fitsio: support for RICE (de)compression algorithm

image: make imageHDU.Image concurrent friendly

decoding of imageHDU.Image() image.Image for BITPIX -32 and -64 relies on some global variables (the fmin32 et al. variables)

fix this by adding Min and Max fields to f32Image and f64Image which would be calculated when they are created, modifying the newF32Image constructor to take the raw buffer and iterate over it to get the min and max values.

use reflect.ArrayOf

go-1.5 may expose reflect.ArrayOf for general consumption.

if golang/go#5996 effectively lands, consider removing completely g_arrayTypes (or hide it with a build tag)

Reusing the I/O buffer when decoding a header unit would be beneficial

The current implementation of readBlock allocates a new memory buffer every time a new block of 2880 bytes is to be read (see here). This generates additional unnecessary work to the garbage collector when decoding thousands of files in a single execution of a program using astrogo/fitsio.

This situation could be improved by associating a buffer of the appropriate size to a streamDecoder and reuse that buffer for reading each block of the same FITS file.

Alternatively, a single buffer could be allocated by the DecodeHDU() method and that buffer could be reused to perform all the necessary read operations while decoding the entire HDU.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.