Git Product home page Git Product logo

Comments (8)

simonbyrne avatar simonbyrne commented on May 29, 2024 2

Most reliable option is to use cconvert/unsafe_convert:

julia> using HDF5

julia> fid = h5open("test.h5", "w")
🗂️ HDF5.File: (read-write) test.h5

julia> attr = create_attribute(fid, "attr-name", datatype(String), dataspace(String))
🏷️ HDF5.Attribute: attr-name

julia> val = Base.cconvert(Cstring, "attr-val") # ensures string is nul-terminated
"attr-val"

julia> GC.@preserve val begin
          p = Base.unsafe_convert(Cstring, val)
          write_attribute(attr, datatype(String), Ref(p))
       end

julia> close(fid)

from hdf5.jl.

simonbyrne avatar simonbyrne commented on May 29, 2024 1

oh, i misunderstood: by default we write them as fixed length strings...

from hdf5.jl.

ericphanson avatar ericphanson commented on May 29, 2024

I was able to workaround it with the following code:

julia> function write_variable_length_string_attribute(fid, attr_key::String, attr_value::String)
           attr = create_attribute(fid, attr_key, datatype(String), dataspace(String))
           v = Vector{UInt8}(attr_value)
           GC.@preserve v begin
               p = pointer(v)
               write_attribute(attr, datatype(String), Ref(p))
           end
           return nothing
       end
write_variable_length_string_attribute (generic function with 1 method)

julia> fid = h5open("test.h5", "w")
🗂️ HDF5.File: (read-write) test.h5

julia> write_variable_length_string_attribute(fid, "attr-key", "attr-value")

julia> close(fid)

shell> h5dump test.h5
HDF5 "test.h5" {
GROUP "/" {
   ATTRIBUTE "attr-key" {
      DATATYPE  H5T_STRING {
         STRSIZE H5T_VARIABLE;
         STRPAD H5T_STR_NULLTERM;
         CSET H5T_CSET_UTF8;
         CTYPE H5T_C_S1;
      }
      DATASPACE  SCALAR
      DATA {
      (0): "attr-value"
      }
   }
}
}

The context here is I need to write a variable-length string as an attribute, so some python code using h5py will interpret the attribute as a string and not a numpy byte array (xref https://docs.h5py.org/en/stable/strings.html).

from hdf5.jl.

ericphanson avatar ericphanson commented on May 29, 2024

I don't know if something specific should be done to ensure null termination. I added a branch here, though the output seems exactly the same:

julia> using HDF5

julia> fid = h5open("test.h5", "w")
🗂️ HDF5.File: (read-write) test.h5

julia> function write_variable_length_string_attribute(fid, attr_key::String, attr_value::String)
           attr = create_attribute(fid, attr_key, datatype(String), dataspace(String))
           v = Vector{UInt8}(attr_value)
           v[end] == 0 || push!(v, 0) # null termination?
           GC.@preserve v begin
               p = pointer(v)
               write_attribute(attr, datatype(String), Ref(p))
           end
           return nothing
       end
write_variable_length_string_attribute (generic function with 1 method)

julia> write_variable_length_string_attribute(fid, "attr-key", "attr-value")

julia> close(fid)

shell> h5dump test.input
h5dump error: unable to open file "test.input"

shell> h5dump test.h5
HDF5 "test.h5" {
GROUP "/" {
   ATTRIBUTE "attr-key" {
      DATATYPE  H5T_STRING {
         STRSIZE H5T_VARIABLE;
         STRPAD H5T_STR_NULLTERM;
         CSET H5T_CSET_UTF8;
         CTYPE H5T_C_S1;
      }
      DATASPACE  SCALAR
      DATA {
      (0): "attr-value"
      }
   }
}
}

from hdf5.jl.

simonbyrne avatar simonbyrne commented on May 29, 2024

We should update the docs to recommend everyone use attrs

attrs(fid)["attr-key"] = "attr-value"

I don't get why this is segfaulting though?

from hdf5.jl.

simonbyrne avatar simonbyrne commented on May 29, 2024

It looks like we have to pass a pointer to a string pointer.

from hdf5.jl.

simonbyrne avatar simonbyrne commented on May 29, 2024

See https://forum.hdfgroup.org/t/how-to-create-scalar-variable-length-string/10309/3

from hdf5.jl.

ericphanson avatar ericphanson commented on May 29, 2024

Is

function write_variable_length_string_attribute(fid, attr_key::String, attr_value::String)
           attr = create_attribute(fid, attr_key, datatype(String), dataspace(String))
           v = Vector{UInt8}(attr_value)
           v[end] == 0 || push!(v, 0) # null termination?
           GC.@preserve v begin
               p = pointer(v)
               write_attribute(attr, datatype(String), Ref(p))
           end
           return nothing
       end

safe/legit? It seems to work, but I don't really know what I am doing

from hdf5.jl.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.