Git Product home page Git Product logo

tiffimages.jl's Introduction

"Don't get into a tiff with your images"

Stable release Documentation Build Status

This package aims to be a fast, minimal, and correct TIFF reader and writer written in Julia.

Features

  • Fast reading and writing of many common TIFFs
  • Extensible core for other TIFF packages to build on
  • Native integration with Colors.jl and the Julia Array ecosystem
  • Memory-mapping for loading images too large to fit in memory
  • BigTIFF standard (TIFFs larger than 4 GB)
  • Arbitrary bit depths (e.g. 12 or 14 bit cameras)
  • Common compression algorithms like LZW and Packbits

Installation

TiffImages.jl is available through Julia's general repository. You can install it by running the following commands in the Julia REPL:

using Pkg
Pkg.install("TiffImages")

Please see the documentation above for usage details and examples

tiffimages.jl's People

Contributors

chrstphrbrns avatar dcjones avatar gunnarfarneback avatar ianbutterworth avatar jaakkor2 avatar jaydevsr avatar johnnychen94 avatar kimikage avatar mdhe1248 avatar timholy avatar tlnagy avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

tiffimages.jl's Issues

Missing COMPRESSION_ADOBE_DEFLATE

As suggested by the error message, I am reporting the issue:

ERROR: Compression COMPRESSION_ADOBE_DEFLATE is not implemented. Please open an issue against TiffImages.jl.
Stacktrace:
  [1] error(::String, ::TiffImages.CompressionType, ::String)
    @ Base ./error.jl:42
  [2] read!(tf::TiffFile{UInt32, FileIO.Stream{FileIO.DataFormat{:TIFF}, IOStream, String}}, arr::Matrix{ColorTypes.Gray{FixedPointNumbers.Q0f31}}, #unused#::Val{TiffImages.COMPRESSION_ADOBE_DEFLATE})
    @ TiffImages ~/.julia/packages/TiffImages/YTEez/src/compression.jl:50
  [3] read!(tf::TiffFile{UInt32, FileIO.Stream{FileIO.DataFormat{:TIFF}, IOStream, String}}, arr::Matrix{ColorTypes.Gray{FixedPointNumbers.Q0f31}}, comp::TiffImages.CompressionType)
    @ TiffImages ~/.julia/packages/TiffImages/YTEez/src/compression.jl:8
  [4] read!(target::Matrix{ColorTypes.Gray{FixedPointNumbers.Q0f31}}, tf::TiffFile{UInt32, FileIO.Stream{FileIO.DataFormat{:TIFF}, IOStream, String}}, ifd::TiffImages.IFD{UInt32})
    @ TiffImages ~/.julia/packages/TiffImages/YTEez/src/ifds.jl:212
  [5] load(tf::TiffFile{UInt32, FileIO.Stream{FileIO.DataFormat{:TIFF}, IOStream, String}}, ifds::Vector{TiffImages.IFD{UInt32}}, ::Nothing; verbose::Bool)
    @ TiffImages ~/.julia/packages/TiffImages/YTEez/src/load.jl:63
  [6] load(tf::TiffFile{UInt32, FileIO.Stream{FileIO.DataFormat{:TIFF}, IOStream, String}}; verbose::Bool, mmap::Bool)
    @ TiffImages ~/.julia/packages/TiffImages/YTEez/src/load.jl:36
  [7] load(io::IOStream; verbose::Bool, mmap::Bool)
    @ TiffImages ~/.julia/packages/TiffImages/YTEez/src/load.jl:7
  [8] #6
    @ ~/.julia/packages/TiffImages/YTEez/src/load.jl:3 [inlined]
  [9] open(f::TiffImages.var"#6#7"{Bool, Bool}, args::String; kwargs::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ Base ./io.jl:330
 [10] open
    @ ./io.jl:328 [inlined]
 [11] #load#5
    @ ~/.julia/packages/TiffImages/YTEez/src/load.jl:2 [inlined]
 [12] load(filepath::String)
    @ TiffImages ~/.julia/packages/TiffImages/YTEez/src/load.jl:2
 [13] top-level scope
    @ REPL[4]:1

"Compression COMPRESSION_LZW is not implemented"

A screenshot of the error:

image

A text-copy of the error:

Compression COMPRESSION_LZW is not implemented. Please open an issue against TiffImages.jl.

    error(::String, ::TiffImages.CompressionType, ::String)@error.jl:44
    read!(::TiffImages.TiffFile{UInt32, FileIO.Stream{FileIO.DataFormat{:TIFF}, IOStream, String}}, ::SubArray{ColorTypes.RGBA{FixedPointNumbers.N0f8}, 1, Vector{ColorTypes.RGBA{FixedPointNumbers.N0f8}}, Tuple{UnitRange{Int64}}, true}, ::Val{TiffImages.COMPRESSION_LZW})@compression.jl:58
    read!(::TiffImages.TiffFile{UInt32, FileIO.Stream{FileIO.DataFormat{:TIFF}, IOStream, String}}, ::SubArray{ColorTypes.RGBA{FixedPointNumbers.N0f8}, 1, Vector{ColorTypes.RGBA{FixedPointNumbers.N0f8}}, Tuple{UnitRange{Int64}}, true}, ::TiffImages.CompressionType)@compression.jl:8
    read!(::Matrix{ColorTypes.RGBA{FixedPointNumbers.N0f8}}, ::TiffImages.TiffFile{UInt32, FileIO.Stream{FileIO.DataFormat{:TIFF}, IOStream, String}}, ::TiffImages.IFD{UInt32})@ifds.jl:236
    var"#load#17"(::Bool, ::typeof(TiffImages.load), ::TiffImages.TiffFile{UInt32, FileIO.Stream{FileIO.DataFormat{:TIFF}, IOStream, String}}, ::Vector{TiffImages.IFD{UInt32}}, ::Nothing)@load.jl:72
    var"#load#16"(::Bool, ::Bool, ::Bool, ::typeof(TiffImages.load), ::TiffImages.TiffFile{UInt32, FileIO.Stream{FileIO.DataFormat{:TIFF}, IOStream, String}})@load.jl:36
    load(::TiffImages.TiffFile{UInt32, FileIO.Stream{FileIO.DataFormat{:TIFF}, IOStream, String}})@load.jl:18
    var"#load#15"(::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}, ::typeof(TiffImages.load), ::IOStream)@load.jl:17
    [email protected]:17[inlined]
    #[email protected]:13[inlined]
    var"#open#378"(::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}, ::typeof(open), ::TiffImages.var"#13#14"{Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}}, ::String, ::Vararg{String})@io.jl:384
    [email protected]:381[inlined]

    #load#[email protected]:12[inlined]
    load(::String)@load.jl:11
    top-level scope@[Local: 1](http://localhost:1234/edit?id=df2be67c-5426-11ed-2840-d96aa871c7f4#)[inlined]
---

Key 0x0115 not found

Hello! I am hoping to use your package to analyze some images. It seems that there is a metadata tag in my image that isn't recognized (0x0115). My images are generated with some old software and I am no expert at TIFF. Seems that this tag denotes "SamplesPerPixel". I followed some of the suggestions on #23 to debug the issue:

julia> img = TiffImages.load("example.tif")
ERROR: KeyError: key 0x0115 not found

julia> tf = read(open(fpath), TiffImages.TiffFile)
TiffFile{UInt32, FileIO.Stream{FileIO.DataFormat{:TIFF}, IOStream, String}}("example.tif", FileIO.Stream{FileIO.DataFormat{:TIFF}, IOStream, String}(IOStream(<file example.tif>), "example.tif"), 130, false)

julia> first(tf)
IFD, with tags:
	Tag(SUBFILETYPE, 0)
	Tag(IMAGEWIDTH, 981)
	Tag(IMAGELENGTH, 1043)
	Tag(BITSPERSAMPLE, 32)
	Tag(COMPRESSION, COMPRESSION_NONE)
	Tag(PHOTOMETRIC, 1)
	Tag(IMAGEDESCRIPTION, REMOTE@580 String[] len=508)
	Tag(MODEL, REMOTE@62 String[] len=48)
	Tag(STRIPOFFSETS, 4096)
	Tag(ROWSPERSTRIP, 1043)
	Tag(STRIPBYTECOUNTS, 4092732)
	Tag(XRESOLUTION, REMOTE@484 Rational{UInt32}[] len=1)
	Tag(YRESOLUTION, REMOTE@492 Rational{UInt32}[] len=1)
	Tag(SOFTWARE, REMOTE@110 String[] len=20)
	Tag(DATETIME, REMOTE@30 String[] len=32)
	Tag(ARTIST, REMOTE@10 String[] len=20)
	Tag(SAMPLEFORMAT, 2)
	Tag(EXIFTAG_EXIFVERSION, "�7t�")
	Tag(UNKNOWN(36865), 0)
	Tag(UNKNOWN(36866), 0)
	Tag(EXIFTAG_DATETIMEORIGINAL, 1.8)
	Tag(EXIFTAG_DATETIMEDIGITIZED, 0.0)
	Tag(UNKNOWN(36870), "")
	Tag(UNKNOWN(36876), 0.0)
	Tag(UNKNOWN(36877), 0.0)
	Tag(UNKNOWN(36878), 0.0)
	Tag(UNKNOWN(36879), 0.0)
	Tag(UNKNOWN(36880), 0.0)
	Tag(UNKNOWN(37120), REMOTE@500 UInt32[] len=20)

I've included an example image here to help with diagnosis. I zipped it as I can't upload tif files here. I can send it in an alternative way if needed. Thanks!
example.tif.zip

Precompilation Warning: `Progress(n::Integer, dt::Real, desc::AbstractString = "Progress: ", barlen = nothing, color::Symbol = :green, output::IO = stderr; offset::Integer = 0)` is deprecated

┌ TiffImages [731e570b-9d59-4bfa-96dc-6df516fadf69]
│  ┌ Warning: `Progress(n::Integer, dt::Real, desc::AbstractString = "Progress: ", barlen = nothing, color::Symbol = :green, output::IO = stderr; offset::Integer = 0)` is deprecated, use `Progress(n; dt = dt, desc = desc, barlen = barlen, color = color, output = output, offset = offset)` instead.
│  │   caller = ip:0x0
│  └ @ Core :-1

Use new sizeof metrics to improve BigTIFF vs TIFF container selection

Currently, my estimation for whether I need a BigTIFF container for a dense image or not is extremely naive:

# this is only a crude estimate for the amount information that we can store
# in regular TIFF. The real value should take into account the size of the
# header, the size of a minimal set of tags in each IFDs.
if sizeof(T) * length(data) >= typemax(offset)
@info "Array too large to fit in standard TIFF container, switching to BigTIFF"
offset = UInt64
end

In f234035, I implemented a concrete determination of the size of an image on disk and I should use that logic instead. This should eliminate any weird edge cases when the image is very close to the maximum size of a normal TIFF container.

For example, here's how easy it is for mmapped images:

ondisk = sizeof(A.file)
ondisk += (size(A) == (0, 0, 0)) ? 0 : sum(sizeof.(A.ifds)) + sizeof(T) * reduce(*, size(A))

DEFLATE decoding is allocation-intensive

As one can see, the allocations are way out of proportion to the file size

julia> using TiffImages

julia> @time TiffImages.load("/home/chris/red_deflate.tif");
  1.064223 seconds (2.22 M allocations: 267.701 MiB, 8.74% gc time, 76.13% compilation time)

julia> @time TiffImages.load("/home/chris/red_deflate.tif");
  0.272420 seconds (1.01 M allocations: 183.330 MiB, 16.64% gc time)

(@v1.11) pkg> status TiffImages
Status `~/.julia/environments/v1.11/Project.toml`
  [731e570b] TiffImages v0.6.6
> ls -lh /home/chris/red_deflate.tif
-rw-rw-r-- 1 chris chris 89K Sep 26 07:10 /home/chris/red_deflate.tif
> file /home/chris/red_deflate.tif
/home/chris/red_deflate.tif: TIFF image data, little-endian, direntries=23, height=720, bps=19146, compression=deflate, PhotometricIntepretation=RGB, orientation=upper-left, width=1280

Decompress after reading in blocks

As identified in #17, a key area of weakness is my strategy for inflating compressed data. Currently, I inflate while reading the data, which is likely slower than reading a contiguous block followed by in-memory inflation. Plus, I should consider spinning out (some of) the compression code into separate package for reusability sake.

Towards baseline TIFF compatibility

In order to satisfy baseline TIFF compatibility (TIFF spec 6.0), we need:

Loading

  • Basic file layout
  • Multiple IFDs
  • Bilevel images
  • Basic compression
  • Strips
  • Palette-color images
  • Full-color images

Other niceties

  • Floating-point images
  • BigTIFF

Tiff is saved with uncompleted tag content

Hi, I try to write below omexml to TiffImages.IMAGEDESCRIPTION tag and save tiff, but saved tiff's IMAGEDESCRIPTION tag always lose final 2 chars! It is so weird... For example, if the original is ...</OME>\n, saved tiff will be ...</OME(julia take \n as a char ). But the dump xml from OMETIFF.jl and simple strings don't show this issue. As the same time, exiftool write tag normally.

run(`exiftool -ImageDescription=$tiffxml $img_name -overwrite_original`

The final goal is to save OMETIFF. To do that, I have to add two more chars in original xml: ...</OME>\n => ...</OME>\n>\n. It will be convenient and excited to write OMETIFF, so I'm looking for your idea.

"<?xml version=\"1.0\" encoding=\"utf-8\"?>\n<OME xsi:schemaLocation=\"http://www.openmicroscopy.org/Schemas/OME/2016-06 http://www.openmicroscopy.org/Schemas/OME/2016-06/ome.xsd\" UUID=\"urn:uuid:39163ae6-e2d4-4bd4-b45f-e29234238f3f\" Creato" ⋯ 579 bytes ⋯ "  </Channel>\n      <TiffData FirstT=\"0\" IFD=\"0\" FirstC=\"0\" PlaneCount=\"1\" FirstZ=\"0\">\n        <UUID FileName=\"4D-series.ome.tif\">urn:uuid:39163ae6-e2d4-4bd4-b45f-e29234238f3f</UUID>\n      </TiffData>\n    </Pixels>\n  </Image>\n</OME>\n"

Question: Opening a series of tiff files?

Hi,
I have a 3d image that consists of multiple 2d ome.tiff files.
Is there a simple way or function to load every tiff file in a single variable with mmap?
All I can think of is to try a for loop...

Trigger patch release

The version was already bumped in the Project.toml, but the release wasn't triggered.

Image is being incorrectly interpreted as 16bit instead of 8bit

Does this mean you get an all-zero channel after loading it even before visualization?

Exactly. Apart from that it seems to be taking the pixel values in each slice (z) from different channels randomly, meaning that in slice 21 I'd be getting the information from original channel 1 and for slice 57 for example the info from channel 3, which is at least weird.

Because there are multiple Image IO backends. Can you check if using TIffImages directly works?

using TiffImages
TiffImages.load(filepath)

Yeah I've tried loading with TiffImages backend and the probelm stills.

If it is still the all-zero case, there might be some issues with your specific image file and TiffImages. In that case, providing an example image would be helpful to diagnose.

If you don't mind I can send you the image I am trying to load and I will be very grateful if you could help me.

https://drive.google.com/file/d/1JEDVJXh6yV5V5WU7qyvk_xBd4E1_rInJ/view?usp=sharing

Thanks in advance!

Originally posted by @DiegoPerezDones in JuliaImages/Images.jl#998 (comment)

slow write performance for abstract array

using TestImages, FileIO, ImageCore
using BenchmarkTools

img = Float64.(testimage("cameraman"))

@btime save("tmp.tiff", img) #  1.840 ms (140 allocations: 519.67 KiB)
@btime save("tmp.tiff", reinterpret(Gray{Float64}, img)) # 20.758 ms (235 allocations: 2.01 MiB)
julia> versioninfo()
Julia Version 1.7.2
Commit bf53498635 (2022-02-06 15:21 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: 12th Gen Intel(R) Core(TM) i9-12900K
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-12.0.1 (ORCJIT, goldmont)

error loading file

i get the following error when trying to load a particular file. i think this line needs to handle the case where ifd[STRIPOFFSETS].data is not a vector, though that's just a guess as i know nothing about the innards of tiff files. happy to share the particular file in question if you tell me how best to get it to you. it is 12MB.

Fatal error:
ERROR: MethodError: no method matching diff(::UInt32)
Closest candidates are:
  diff(::AbstractRange{T}; dims) where T at multidimensional.jl:1007
  diff(::SparseArrays.AbstractSparseMatrixCSC; dims) at /groups/scicompsoft/home/arthurb/bin/julia-1.8.3/share/julia/stdlib/v1.8/SparseArrays/src/linalg.jl:1099
  diff(::StaticArraysCore.StaticArray{Tuple{N}, T, 1} where {N, T}) at /groups/scicompsoft/home/arthurb/.julia/packages/StaticArrays/B0HhH/src/mapreduce.jl:316
  ...
Stacktrace:
  [1] iscontiguous(ifd::TiffImages.IFD{UInt32})
    @ TiffImages /groups/scicompsoft/home/arthurb/.julia/packages/TiffImages/FEcMg/src/ifds.jl:134
  [2] read!(target::Matrix{Gray{N0f16}}, tf::TiffFile{UInt32, FileIO.Stream{FileIO.DataFormat{:TIFF}, IOStream, String}}, ifd::TiffImages.IFD{UInt32})
    @ TiffImages /groups/scicompsoft/home/arthurb/.julia/packages/TiffImages/FEcMg/src/ifds.jl:218
  [3] macro expansion
    @ /groups/scicompsoft/home/arthurb/.julia/packages/TiffImages/FEcMg/src/load.jl:86 [inlined]
  [4] macro expansion
    @ /groups/scicompsoft/home/arthurb/.julia/packages/ProgressMeter/sN2xr/src/ProgressMeter.jl:938 [inlined]
  [5] load(tf::TiffFile{UInt32, FileIO.Stream{FileIO.DataFormat{:TIFF}, IOStream, String}}, ifds::Vector{TiffImages.IFD{UInt32}}, N::Int64; verbose::Bool)
    @ TiffImages /groups/scicompsoft/home/arthurb/.julia/packages/TiffImages/FEcMg/src/load.jl:85
  [6] load(tf::TiffFile{UInt32, FileIO.Stream{FileIO.DataFormat{:TIFF}, IOStream, String}}; verbose::Bool, mmap::Bool, lazyio::Bool)
    @ TiffImages /groups/scicompsoft/home/arthurb/.julia/packages/TiffImages/FEcMg/src/load.jl:38
  [7] load(io::IOStream; kwargs::Base.Pairs{Symbol, Bool, Tuple{Symbol}, NamedTuple{(:verbose,), Tuple{Bool}}})
    @ TiffImages /groups/scicompsoft/home/arthurb/.julia/packages/TiffImages/FEcMg/src/load.jl:17
  [8] #13
    @ /groups/scicompsoft/home/arthurb/.julia/packages/TiffImages/FEcMg/src/load.jl:13 [inlined]
  [9] open(::TiffImages.var"#13#14"{Base.Pairs{Symbol, Bool, Tuple{Symbol}, NamedTuple{(:verbose,), Tuple{Bool}}}}, ::String, ::Vararg{String}; kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ Base ./io.jl:384
 [10] open
    @ ./io.jl:381 [inlined]
 [11] #load#12
    @ /groups/scicompsoft/home/arthurb/.julia/packages/TiffImages/FEcMg/src/load.jl:12 [inlined]
 [12] invokelatest(f::Any, args::Any; kwargs::Base.Pairs{Symbol, Bool, Tuple{Symbol}, NamedTuple{(:verbose,), Tuple{Bool}}})
    @ Base ./essentials.jl:731

Reading file from the network? Example code.

Hi, I wanted to load an image from an HTTP server without saving it to disk and came up with this snippet. I don't know if this is a supported usage or if it is wrong in some way / has some downside. I sort of matched the type signatures until I found a combination that worked. If it is good, maybe it could be added to the examples.

function load_tiff_without_saving(url)
  buffer = IOBuffer()
  Downloads.download(url, buffer)
  bufstream = TiffImages.getstream(format"TIFF", buffer)
  TiffImages.load(read(bufstream, TiffFile))
end

ERROR: KeyError: key (TIFF.SAMPLEFORMAT_INT, 0x0008) not found

One more error might need to be caught.
4D-series.ome.tif is sample file from OME-TIFF website.

julia> TIFF.load("4D-series.ome.tif")
ERROR: KeyError: key (TIFF.SAMPLEFORMAT_INT, 0x0008) not found
Stacktrace:
 [1] getindex(::Dict{Tuple{TIFF.SampleFormats,Int64},DataType}, ::Tuple{TIFF.SampleFormats,UInt16}) at ./dict.jl:477
 [2] output(::TIFF.IFD{UInt32}) at /home/hf/.julia/dev/TIFF/src/ifds.jl:113
 [3] load(::String) at /home/hf/.julia/dev/TIFF/src/load.jl:14
 [4] top-level scope at REPL[2]:1

Writing a LazyBufferedTIFF and can't save the added tags

Hi and thanks for the help in advance. I am trying to write additional tags to lazybufferedtiffs but I am not managing to get the tags saved. I am also writing some tiffs that are >4GB so I need bigtiff=true. Is there a way to do this? Here is some example code:

using TiffImages
using FixedPointNumbers
using Colors

random_data = rand(Gray{N0f16}, 100, 100)

lazy_tiff = empty(LazyBufferedTIFF, Gray{N0f16},"test1.tiff"; bigtiff=true) 
push!(lazy_tiff, random_data)
lazy_tiff.ifds[1][TiffImages.IMAGEDESCRIPTION] = "Hello world"

TiffImages.save("test2.tiff", lazy_tiff) 

normal_tiff = TiffImages.load("test2.tiff")
normal_tiff.ifds[1][TiffImages.IMAGEDESCRIPTION] = "Hello world"
TiffImages.save("test3.tiff", normal_tiff) 

test1_tiff = TiffImages.load("test1.tiff") # Doesn't have tag
test2_tiff = TiffImages.load("test2.tiff") # Doesn't have tag
test3_tiff = TiffImages.load("test3.tiff") # Has tag

Loading might be a little too lazy, leading to long access times

My current strategy has been to store the data in-memory exactly as on disk and lazily transform it on access to the expected dimensions and shape. While this leads to great loading times, access is quite slow (~100-150x slower than ImageMagick.jl). Alternatively, I could try to create both the target image location and a buffer that is sized to the maximum number of bytes expected. I could read into this buffer, apply transforms lazily, and then save into the target (realizing the transforms).

Bizarre strip image bug(s)

On the latest master, striped images behave inconsistently:

house.tif flagler.tif 4D-series.ome.tif
house flagler 4D-series

house.tif behaves as expected and renders properly. It has horizontal strips whose length matches the number of columns in the image exactly. flager.tif has strips are much longer than a single row so they must wrap properly. The strips in 4D-series are written in reverse!?

TiffImages fails to open Tiled TIFFs

It's more related to performance rather than inability to read I think but here the error seems slightly when I try it without FileIO and in a isolated env. When I used with FileIO, error was different as seen here:JuliaImages/ImageSegmentation.jl#94 (comment)
Noticed here: JuliaImages/ImageSegmentation.jl#94 as the OP wanted to load a very large 30k*10k image which is here: https://github.com/niccolo99mandelli/JHistint.jl/blob/main/output_example/SlideExample.tif

MWE:

julia> using TiffImages
[ Info: Precompiling TiffImages [731e570b-9d59-4bfa-96dc-6df516fadf69]

julia> img = TiffImages.load("SlideExample.tif")
ERROR: KeyError: key 0x0111 not found
Stacktrace:
  [1] getindex
    @ C:\Users\lenono\.julia\packages\OrderedCollections\vTcqn\src\ordered_dict.jl:385 [inlined]
  [2] getindex
    @ C:\Users\lenono\.julia\packages\TiffImages\lJfX2\src\ifds.jl:31 [inlined]
  [3] getindex
    @ C:\Users\lenono\.julia\packages\TiffImages\lJfX2\src\ifds.jl:30 [inlined]
  [4] read!(target::Matrix{ColorTypes.RGB{FixedPointNumbers.N0f8}}, tf::TiffFile{UInt32, FileIO.Stream{FileIO.DataFormat{:TIFF}, IOStream, String}}, ifd::TiffImages.IFD{UInt32})
    @ TiffImages C:\Users\lenono\.julia\packages\TiffImages\lJfX2\src\ifds.jl:118
  [5] macro expansion
    @ C:\Users\lenono\.julia\packages\TiffImages\lJfX2\src\load.jl:63 [inlined]
  [6] macro expansion
    @ C:\Users\lenono\.julia\packages\ProgressMeter\sN2xr\src\ProgressMeter.jl:938 [inlined]
  [7] load(tf::TiffFile{UInt32, FileIO.Stream{FileIO.DataFormat{:TIFF}, IOStream, String}}, ifds::Vector{TiffImages.IFD{UInt32}}, N::Int64; verbose::Bool)
    @ TiffImages C:\Users\lenono\.julia\packages\TiffImages\lJfX2\src\load.jl:62
  [8] load(tf::TiffFile{UInt32, FileIO.Stream{FileIO.DataFormat{:TIFF}, IOStream, String}}; verbose::Bool, mmap::Bool)
    @ TiffImages C:\Users\lenono\.julia\packages\TiffImages\lJfX2\src\load.jl:24
  [9] load(io::IOStream; verbose::Bool, mmap::Bool)
    @ TiffImages C:\Users\lenono\.julia\packages\TiffImages\lJfX2\src\load.jl:7
 [10] #8
    @ C:\Users\lenono\.julia\packages\TiffImages\lJfX2\src\load.jl:3 [inlined]
 [11] open(f::TiffImages.var"#8#9"{Bool, Bool}, args::String; kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ Base .\io.jl:384
 [12] open
    @ .\io.jl:381 [inlined]
 [13] #load#7
    @ C:\Users\lenono\.julia\packages\TiffImages\lJfX2\src\load.jl:2 [inlined]
 [14] load(filepath::String)
    @ TiffImages C:\Users\lenono\.julia\packages\TiffImages\lJfX2\src\load.jl:1
 [15] top-level scope
    @ REPL[4]:1

Question: pixel spacing?

Hi,
I would like to know how to update pixel spacing information (x, y, and z).
It seems that tifftag has XRESOLUTION and YRESOLUTION. I played with it, but the information is not being updated (when I check that in fiji).

Thanks,

Add support for GeoTIFFs

Thank you very much for this great package. I don't know much about the Tiff format, but I am sure that it represents a lot of work to implement such a complex format :-)

My use case is to read satellite data in the GeoTIFF format. The data I am working with has 11 (spectral) bands. I am currently using ArchGDAL to read this data. It seem that TiffImages.jl currently reads only the first band:

using TiffImages
tiff = TiffImages.load(download("https://data-assimilation.net/upload/Alex/TIFF/S2_1-12-19_48MYU_0.tif"));
eltype(tiff)
# output: ColorTypes.Gray{Float32}
size(tiff)
# output (256, 256)

Using the command line tool gdalinfo S2_1-12-19_48MYU_0.tif, I get these 11 bands

[...]
Band 1 Block=256x1 Type=Float32, ColorInterp=Gray
Band 2 Block=256x1 Type=Float32, ColorInterp=Undefined
Band 3 Block=256x1 Type=Float32, ColorInterp=Undefined
Band 4 Block=256x1 Type=Float32, ColorInterp=Undefined
Band 5 Block=256x1 Type=Float32, ColorInterp=Undefined
Band 6 Block=256x1 Type=Float32, ColorInterp=Undefined
Band 7 Block=256x1 Type=Float32, ColorInterp=Undefined
Band 8 Block=256x1 Type=Float32, ColorInterp=Undefined
Band 9 Block=256x1 Type=Float32, ColorInterp=Undefined
Band 10 Block=256x1 Type=Float32, ColorInterp=Undefined
Band 11 Block=256x1 Type=Float32, ColorInterp=Undefined

Is there a way to load also the other bands using TiffImages.jl ? Is this with-in scope of the package?

Ref: Alexander-Barth/TIFFDatasets.jl#1
I am using TiffImages v0.6.4, julia 1.8.3 on 64-bit Linux

Improve handling of duplicate tags

Working on tlnagy/OMETIFF.jl#12, I ran into an issue where tags can be duplicated. The first IFD in one of the test files has two different IMAGEDESCRIPTION entries:

Tag(IMAGEWIDTH, 256)
...
Tag(IMAGEDESCRIPTION, REMOTE@3983416 String[] len=10188)
Tag(IMAGEDESCRIPTION, REMOTE@3993604 String[] len=61)
...

The first one contains the OMEXML and second contains ImageJ info. With the current design, the first gets overwritten by the second so the OMEXML data is discarded. Unfortunately, to fix this, it's going to require a non-trivial amount of refactoring.

equivalent of `ImageMagick.getimageproperty()`

is there a way to do this with TiffImages.jl?

using ImageMagick
wand = MagickWand()
readimage(wand, "filename.tif")
my_custom_metadata_value = ImageMagick.getimageproperty(wand, "my_custom_metadata_key")

Towards an initial release

Now that #1 is closed, here's what I would like to have in the initial release

  • Basic writing (i.e. no compression) (progress in #14)
  • Streamlining of load function
  • Thorough testing
  • Benchmarking against libtiff and ImageMagick.jl
  • Memory mapped files (port code from OMETIFF.jl over)
  • Documentation of usage
  • Documentation of internals
  • Rename package for registration (#5 )

Can I keep the 3rd dimension with a length of one?

While saving as tiff, is it possible to keep a dimension with a length of one?
The below lose the third dimension after save.

using TiffImages, Images
fn = "3d_img.tif"
img1 = rand(RGB{N0f8}, 10,10,1)
save(fn, img1)
img2 = load(fn)
isequal(img1, img2) # results in `false`

On the other hand, the NRRD format keeps the dimension.

fn = "3d_img.nrrd"
save(fn, img1)
img2 = load(fn)
isequal(img1, img2) #results in `true`

TiffImages.SOFTWARE tag overwritten during saving

The Software tag seems to be fixed during saving to something like "TiffImages.jl v0.4.1" .
While I agree that this is the suitable spot, it would be great if this is already done before saving, so also shows up right after generating the DenseTaggedImage, instead of silently overwriting the current software tag during saving.

LoadError: KeyError when using TiffImages.load

Not a Julia-pro so forgive me if this is an issue with my implementation rather than the package.

using:

stack = TiffImages.load(filepath, mmap = true) or stack = TiffImages.load(filepath)

where the file in question is a standard .tif (default tiff option in ImageJ)

gives me the error:

LoadError: KeyError: key 0x0103 not found
Stacktrace:
 [1] getindex at C:\Users\ajsc4\.julia\packages\OrderedCollections\DqfZ7\src\ordered_dict.jl:346 [inlined]
 [2] getindex at C:\Users\ajsc4\.julia\packages\TiffImages\uTSzK\src\ifds.jl:18 [inlined]
 [3] getindex at C:\Users\ajsc4\.julia\packages\TiffImages\uTSzK\src\ifds.jl:17 [inlined]
 [4] output(::TiffImages.IFD{UInt32}) at C:\Users\ajsc4\.julia\packages\TiffImages\uTSzK\src\ifds.jl:135
 [5] load(::IOStream; verbose::Bool, mmap::Bool) at C:\Users\ajsc4\.julia\packages\TiffImages\uTSzK\src\load.jl:20
 [6] #4 at C:\Users\ajsc4\.julia\packages\TiffImages\uTSzK\src\load.jl:3 [inlined]
 [7] open(::TiffImages.var"#4#5"{Bool,Bool}, ::String; kwargs::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at .\io.jl:325
 [8] open at .\io.jl:323 [inlined]
 [9] #load#3 at C:\Users\ajsc4\.julia\packages\TiffImages\uTSzK\src\load.jl:2 [inlined]
 [10] top-level scope at C:\Users\ajsc4\Desktop\Code\tifread.jl:12
 [11] include_string(::Function, ::Module, ::String, ::String) at .\loading.jl:1088
in expression starting at C:\Users\ajsc4\Desktop\Code\tifread.jl:12

I'm not entirely sure how to interpret this.

Thanks for your work putting this all together!

Add support for limited loading as `Array` type

For packages like ImageIO where we are emitting Arrays, there should be an option for TiffImages to load the file in a simplified and compatible way, i.e. only the largest slice in a pyramidal image, etc. This would insulate users of those packages from TiffImages types and complexity.

This would be more efficient/more general than doing the conversion post-hoc.

Originally posted by @tlnagy in JuliaIO/ImageIO.jl#25 (comment)

TagBot trigger issue

This issue is used to trigger TagBot; feel free to unsubscribe.

If you haven't already, you should update your TagBot.yml to include issue comment triggers.
Please see this post on Discourse for instructions and more details.

If you'd like for me to do this for you, comment TagBot fix on this issue.
I'll open a PR within a few hours, please be patient!

Improve IFD handling API

Should switch away from calls like img.ifds to function calls like ifd(img) and make this behavior consistent across all TIFF images.

incompatible Float64 encoding?

This is only an issue of Float64 and not N0f8.

using ImageCore, TiffImages
img = rand(RGB{Float64}, 64, 64)
TiffImages.save("tmp.tif", img)
img_loadback = TiffImages.load("tmp.tif")

img == img_loadback # true

However, this "tmp.tif" file is not loadable by other image viewers (both macOS and Windows' default image viewer fail to open it).

image

ERROR: EOFError: read end of file

Missing TiffData attributes in ome-xml confuse TIFF.jl.
In ImageJ and Imaris, they seem to rebuild XYZCT structure from Pixels' attributes SizeT, SizeX, SizeY, SizeZ, which is a better idea.

I guess thatTIFF.jl isn't required to handle ome-xml, but it should catch this error and read pure tiff pixel info.

julia> TIFF.load("HE7-1NLS-22uw-const_1_s10.ome.tiff")
ERROR: EOFError: read end of file
Stacktrace:
 [1] unsafe_read at ./iostream.jl:376 [inlined]
 [2] unsafe_read at ./io.jl:669 [inlined]
 [3] read!(::IOStream, ::Array{UInt8,1}) at ./io.jl:681
 [4] read!(::TIFF.TiffFile{UInt64}, ::Array{UInt8,1}) at /home/hf/.julia/dev/TIFF/src/files.jl:51
 [5] load(::TIFF.TiffFile{UInt64}, ::TIFF.Tag{UInt64}) at /home/hf/.julia/dev/TIFF/src/tags.jl:18
 [6] load!(::TIFF.TiffFile{UInt64}, ::TIFF.IFD{UInt64}) at /home/hf/.julia/dev/TIFF/src/ifds.jl:26
 [7] load(::String) at /home/hf/.julia/dev/TIFF/src/load.jl:11
 [8] top-level scope at REPL[12]:1

xml info:

<?xml version="1.0" encoding="UTF-8"?>
<!-- Warning: this comment is an OME-XML metadata block, which contains crucial dimensional parameters and other important metadata. Please edit cautiously (if at all), and back up the original data before doing so. For more information, see the OME-TIFF web site: http://www.openmicroscopy.org/site/support/ome-model/ome-tiff/. -->
<OME xmlns="http://www.openmicroscopy.org/Schemas/OME/2016-06" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" Creator="OME Bio-Formats 5.2.2" UUID="urn:uuid:a5ae8c1b-ac04-4544-97c1-bbdd0bdf8629" xsi:schemaLocation="http://www.openmicroscopy.org/Schemas/OME/2016-06 http://www.openmicroscopy.org/Schemas/OME/2016-06/ome.xsd">
   <Image ID="Image:0" Name="4D-series.ome.tif">
      <Pixels BigEndian="true" DimensionOrder="XYZCT" ID="Pixels:0" SizeC="1" SizeT="134" SizeX="1300" SizeY="1900" SizeZ="20" Type="uint16" PhysicalSizeX="0.108" PhysicalSizeXUnit="µm" PhysicalSizeY="0.108" PhysicalSizeYUnit="µm" PhysicalSizeZ="0.5" PhysicalSizeZUnit="µm" TimeIncrement="600" TimeIncrementUnit="s">
         <Channel ID="Channel:0:0" SamplesPerPixel="1">
            <LightPath/>
         </Channel>
         <TiffData/>
         <TiffData/>
         <TiffData/>
         <TiffData/>
         <TiffData/>
         <lots of TiffData repeat>
         <TiffData/>
      </Pixels>
   </Image>
</OME>

Rename `memmap`?

memmap is used for incremental writing. However, as pointed out in #79,it's not actually using Mmap.mmap and perhaps should be renamed. What about empty_tiff?

Alternatively, you can use (warning: untested)

img = open(filename, "w") do io
    Mmap.mmap(io, Gray{N0f16}, sz3)
end
for i = 1:n
    img[:,:,i] = create_slice_data(i, sz2)   # this will write to the file
end

This should work at least on Linux despite apparently closing the file before values are assigned; Linux will defer the close until img gets garbage-collected. I am not sure about other platforms.

Of course, that doesn't create a TIFF. You'd need to do some work to write the header and set up empty chunks for the slice planes, essentially as MmappedTIFF does in #79. Basically, you mmap the entire file-buffer and then assign slices to correctly-placed views within the full buffer. Of course, if it's possible to pack all slice planes densely, then just a single view will suffice for an entire 3d array.

This may be more of a pain than any benefit you'd derive from it, so just renaming memmap seems expeditious.

support orientation tag

Currently the embedded orientation tag is ignored and the TIFF is always loaded in the storage orientation.

singleton third dimension with mmap=true

julia> img = TiffImages.load("myimage.tif");

julia> size(img)
(10512, 6336)

julia> img = TiffImages.load("myimage.tif", mmap=true);

julia> size(img)
(10512, 6336, 1)

julia> img = ImageMagick.load("myimage.tif");

julia> size(img)
(10512, 6336)

12-bit tiff support

I have a 12-bit grayscale camera that outputs TIFF stacks with the following tags:

        Tag(IMAGEWIDTH, 1024)
        Tag(IMAGELENGTH, 1024)
        Tag(BITSPERSAMPLE, 12)
        Tag(COMPRESSION, COMPRESSION_NONE)
        Tag(PHOTOMETRIC, 1)
        Tag(STRIPOFFSETS, REMOTE@1573010 UInt32[] len=205)
        Tag(ORIENTATION, 1)
        Tag(SAMPLESPERPIXEL, 1)
        Tag(ROWSPERSTRIP, 5)
        Tag(STRIPBYTECOUNTS, REMOTE@1573830 UInt32[] len=205)
        Tag(PLANARCONFIG, 1)

The rawtype_mapping dict doesn't have an entry for 12-bit images, so TiffImages throws an error on image load:

julia> TiffImages.load("filepath.tiff");
ERROR: KeyError: key (TiffImages.SAMPLEFORMAT_UINT, 0x000c) not found

I tried working around this by defining

TiffImages.rawtype_mapping[(TiffImages.SAMPLEFORMAT_UINT, 12)] = UInt16

which gets me past the previous error, but the data is not correctly aligned, so I now see ERROR: EOFError: read end of file.

mmap error: seems to be cutting off filename

This only happens when using mmap = true. The regular load function is working fine on the same file.

Weirdly is seems as though the filename is being cut off before the end. This seems to happen at the same point whether I'm using double backslashes or single forward-slashes in the file name.

filename = "C:\\Users\\Alexander Collins\\Desktop\\Code\\test_image.tif"
stack = TiffImages.load(filename, mmap = true)

942×1072×1 TiffImages.DenseTaggedImage{Gray{Normed{UInt16,16}},3,UInt32,TiffImages.DiskTaggedImage{Gray{Normed{UInt16,16}},UInt32,Array{Gray{Normed{UInt16,16}},2}}}:
[:, :, 1] =
Error showing value of type TiffImages.DenseTaggedImage{Gray{Normed{UInt16,16}},3,UInt32,TiffImages.DiskTaggedImage{Gray{Normed{UInt16,16}},UInt32,Array{Gray{Normed{UInt16,16}},2}}}:
ERROR: SystemError: opening file "C:\\Users\\Alexande": No such file or directory
Stacktrace:
 [1] systemerror(::String, ::Int32; extrainfo::Nothing) at .\error.jl:168
 [2] #systemerror#48 at .\error.jl:167 [inlined]
 [3] systemerror at .\error.jl:167 [inlined]
 [4] open(::String; lock::Bool, read::Nothing, write::Nothing, create::Nothing, truncate::Nothing, append::Nothing) at .\iostream.jl:284
 [5] open at .\iostream.jl:273 [inlined]
 [6] getindex(::TiffImages.DiskTaggedImage{Gray{Normed{UInt16,16}},UInt32,Array{Gray{Normed{UInt16,16}},2}}, ::Int64, ::Int64, ::Int64) at C:\Users\Alexander Collins\.julia\packages\TiffImages\XpHUY\src\types\mmap.jl:58
 [7] getindex at C:\Users\Alexander Collins\.julia\packages\TiffImages\XpHUY\src\types\dense.jl:28 [inlined]
 [8] getindex at .\subarray.jl:257 [inlined]
 [9] isassigned(::SubArray{Gray{Normed{UInt16,16}},2,TiffImages.DenseTaggedImage{Gray{Normed{UInt16,16}},3,UInt32,TiffImages.DiskTaggedImage{Gray{Normed{UInt16,16}},UInt32,Array{Gray{Normed{UInt16,16}},2}}},Tuple{Base.OneTo{Int64},Base.OneTo{Int64},Int64},false}, 
::Int64, ::Int64) at .\abstractarray.jl:408
 [10] alignment(::IOContext{REPL.Terminals.TTYTerminal}, ::SubArray{Gray{Normed{UInt16,16}},2,TiffImages.DenseTaggedImage{Gray{Normed{UInt16,16}},3,UInt32,TiffImages.DiskTaggedImage{Gray{Normed{UInt16,16}},UInt32,Array{Gray{Normed{UInt16,16}},2}}},Tuple{Base.OneTo{Int64},Base.OneTo{Int64},Int64},false}, ::Array{Int64,1}, ::Array{Int64,1}, ::Int64, ::Int64, ::Int64) at .\arrayshow.jl:67
 [11] print_matrix(::IOContext{REPL.Terminals.TTYTerminal}, ::SubArray{Gray{Normed{UInt16,16}},2,TiffImages.DenseTaggedImage{Gray{Normed{UInt16,16}},3,UInt32,TiffImages.DiskTaggedImage{Gray{Normed{UInt16,16}},UInt32,Array{Gray{Normed{UInt16,16}},2}}},Tuple{Base.OneTo{Int64},Base.OneTo{Int64},Int64},false}, ::String, ::String, ::String, ::String, ::String, ::String, ::Int64, ::Int64) at .\arrayshow.jl:196
 [12] print_matrix(::IOContext{REPL.Terminals.TTYTerminal}, ::SubArray{Gray{Normed{UInt16,16}},2,TiffImages.DenseTaggedImage{Gray{Normed{UInt16,16}},3,UInt32,TiffImages.DiskTaggedImage{Gray{Normed{UInt16,16}},UInt32,Array{Gray{Normed{UInt16,16}},2}}},Tuple{Base.OneTo{Int64},Base.OneTo{Int64},Int64},false}) at .\arrayshow.jl:169
 [13] show_nd(::IOContext{REPL.Terminals.TTYTerminal}, ::TiffImages.DenseTaggedImage{Gray{Normed{UInt16,16}},3,UInt32,TiffImages.DiskTaggedImage{Gray{Normed{UInt16,16}},UInt32,Array{Gray{Normed{UInt16,16}},2}}}, ::typeof(Base.print_matrix), ::Bool) at .\arrayshow.jl:304
 [14] print_array at .\arrayshow.jl:318 [inlined]
 [15] show(::IOContext{REPL.Terminals.TTYTerminal}, ::MIME{Symbol("text/plain")}, ::TiffImages.DenseTaggedImage{Gray{Normed{UInt16,16}},3,UInt32,TiffImages.DiskTaggedImage{Gray{Normed{UInt16,16}},UInt32,Array{Gray{Normed{UInt16,16}},2}}}) at .\arrayshow.jl:358
 [16] display(::REPL.REPLDisplay, ::MIME{Symbol("text/plain")}, ::Any) at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\REPL\src\REPL.jl:214
 [17] display(::REPL.REPLDisplay, ::Any) at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\REPL\src\REPL.jl:218
 [18] display(::Any) at .\multimedia.jl:328
 [19] #invokelatest#1 at .\essentials.jl:710 [inlined]
 [20] invokelatest at .\essentials.jl:709 [inlined]
 [21] print_response(::IO, ::Any, ::Bool, ::Bool, ::Any) at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\REPL\src\REPL.jl:238
 [22] print_response(::REPL.AbstractREPL, ::Any, ::Bool, ::Bool) at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\REPL\src\REPL.jl:223
 [23] (::REPL.var"#do_respond#54"{Bool,Bool,Atom.var"#246#247",REPL.LineEditREPL,REPL.LineEdit.Prompt})(::Any, ::Any, ::Any) at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\REPL\src\REPL.jl:822
 [24] #invokelatest#1 at .\essentials.jl:710 [inlined]
 [25] invokelatest at .\essentials.jl:709 [inlined]
 [26] run_interface(::REPL.Terminals.TextTerminal, ::REPL.LineEdit.ModalInterface, ::REPL.LineEdit.MIState) at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\REPL\src\LineEdit.jl:2355
 [27] run_frontend(::REPL.LineEditREPL, ::REPL.REPLBackendRef) at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\REPL\src\REPL.jl:1144
 [28] (::REPL.var"#38#42"{REPL.LineEditREPL,REPL.REPLBackendRef})() at .\task.jl:356

Thanks!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.