Comments (8)
Most reliable option is to use cconvert
/unsafe_convert
:
julia> using HDF5
julia> fid = h5open("test.h5", "w")
🗂️ HDF5.File: (read-write) test.h5
julia> attr = create_attribute(fid, "attr-name", datatype(String), dataspace(String))
🏷️ HDF5.Attribute: attr-name
julia> val = Base.cconvert(Cstring, "attr-val") # ensures string is nul-terminated
"attr-val"
julia> GC.@preserve val begin
p = Base.unsafe_convert(Cstring, val)
write_attribute(attr, datatype(String), Ref(p))
end
julia> close(fid)
from hdf5.jl.
oh, i misunderstood: by default we write them as fixed length strings...
from hdf5.jl.
I was able to workaround it with the following code:
julia> function write_variable_length_string_attribute(fid, attr_key::String, attr_value::String)
attr = create_attribute(fid, attr_key, datatype(String), dataspace(String))
v = Vector{UInt8}(attr_value)
GC.@preserve v begin
p = pointer(v)
write_attribute(attr, datatype(String), Ref(p))
end
return nothing
end
write_variable_length_string_attribute (generic function with 1 method)
julia> fid = h5open("test.h5", "w")
🗂️ HDF5.File: (read-write) test.h5
julia> write_variable_length_string_attribute(fid, "attr-key", "attr-value")
julia> close(fid)
shell> h5dump test.h5
HDF5 "test.h5" {
GROUP "/" {
ATTRIBUTE "attr-key" {
DATATYPE H5T_STRING {
STRSIZE H5T_VARIABLE;
STRPAD H5T_STR_NULLTERM;
CSET H5T_CSET_UTF8;
CTYPE H5T_C_S1;
}
DATASPACE SCALAR
DATA {
(0): "attr-value"
}
}
}
}
The context here is I need to write a variable-length string as an attribute, so some python code using h5py
will interpret the attribute as a string and not a numpy byte array (xref https://docs.h5py.org/en/stable/strings.html).
from hdf5.jl.
I don't know if something specific should be done to ensure null termination. I added a branch here, though the output seems exactly the same:
julia> using HDF5
julia> fid = h5open("test.h5", "w")
🗂️ HDF5.File: (read-write) test.h5
julia> function write_variable_length_string_attribute(fid, attr_key::String, attr_value::String)
attr = create_attribute(fid, attr_key, datatype(String), dataspace(String))
v = Vector{UInt8}(attr_value)
v[end] == 0 || push!(v, 0) # null termination?
GC.@preserve v begin
p = pointer(v)
write_attribute(attr, datatype(String), Ref(p))
end
return nothing
end
write_variable_length_string_attribute (generic function with 1 method)
julia> write_variable_length_string_attribute(fid, "attr-key", "attr-value")
julia> close(fid)
shell> h5dump test.input
h5dump error: unable to open file "test.input"
shell> h5dump test.h5
HDF5 "test.h5" {
GROUP "/" {
ATTRIBUTE "attr-key" {
DATATYPE H5T_STRING {
STRSIZE H5T_VARIABLE;
STRPAD H5T_STR_NULLTERM;
CSET H5T_CSET_UTF8;
CTYPE H5T_C_S1;
}
DATASPACE SCALAR
DATA {
(0): "attr-value"
}
}
}
}
from hdf5.jl.
We should update the docs to recommend everyone use attrs
attrs(fid)["attr-key"] = "attr-value"
I don't get why this is segfaulting though?
from hdf5.jl.
It looks like we have to pass a pointer to a string pointer.
from hdf5.jl.
See https://forum.hdfgroup.org/t/how-to-create-scalar-variable-length-string/10309/3
from hdf5.jl.
Is
function write_variable_length_string_attribute(fid, attr_key::String, attr_value::String)
attr = create_attribute(fid, attr_key, datatype(String), dataspace(String))
v = Vector{UInt8}(attr_value)
v[end] == 0 || push!(v, 0) # null termination?
GC.@preserve v begin
p = pointer(v)
write_attribute(attr, datatype(String), Ref(p))
end
return nothing
end
safe/legit? It seems to work, but I don't really know what I am doing
from hdf5.jl.
Related Issues (20)
- Test failures in h5a_iterate HOT 1
- Changed requirements in HDF5_jll's `libhdf5.so` for `libcurl.so`? HOT 8
- Can't get HDF5.jl work with Julia running in docker (julia:1.8-alpine3.17) - can't find libmpi.so.12 HOT 6
- Add mid/high level interface for HDF5 Dimension Scale HOT 1
- Writing scalar datasets of compound types HOT 3
- freeze when `hdf5_type_id` on self-referential datatype HOT 17
- Get rid of HISTORY.md? HOT 1
- Feature request - add support for SparseMatrixCSC HOT 1
- Support szip (freely) HOT 5
- Installing HDF5.jl on ARM M1 HOT 5
- HDF5.jl triggers segfault in ccall with openmp+clang(m1) with julia 1.10 HOT 20
- Inconsistent writing of complex data inside compound type HOT 1
- `set_libraries!()` fails on fresh install HOT 7
- h5_is_library_threadsafe() gives unreliable results due to unspecified initial value HOT 1
- The HDF Group CI
- Examine error handling per thread
- The H5T_BITFIELD class should not be directly mapped to Bool HOT 1
- Would it be possible to be able to save @enum values?
- View to a subset of a dataset HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from hdf5.jl.