Can adios_write please be modified to accept 'const void * var'? I assume that it is safe to do so, in which case it'd be preferable to have an explicit assurance that it is safe to assume the data pointed to by var is not modified.

https://github.com/acbauer/ADIOS/tree/cmake_missing_files shows some files that are missing from the CMake build of ADIOS. Also, when I build this it doesn't have adios_config.

Failure to specify a large enough buffer when writing with the POSIX method results in junk data being written to the file. A warning is printed to stdout, but the code does not abort.

Python files are hard-coded to install to $(libdir)/python. On most distros, Python files have been versioned, so that the correct directory is (at least) python2.7 or python3.3, etc.

Fortunately, automake has support for Python, so it's just a matter of using the builtin machinery and things will be in the right place.

Looking at adios_config.flags.in, the compiler flag to add an include directory is hard coded as -I. Unfortunately, that is not always correct when searching for Fortran modules.

See this autoconf test where several options are documented. Fortunately, -I is correct in most cases.

I don't use any of these more exotic compilers, but I just thought I'd point this out.

ADIOS 1.8.0 or 1.9.0 fail to build with gcc 5.3.0. I have got the following error:

core/adios_internals.c: In function 'adios_generate_var_characteristics_v1':
core/adios_internals.c:4754:28: error: non-floating-point argument in call to function '__builtin_isnan'
             hist = (struct adios_hist_struct *) stats[map[adios_statistic_hist]].data; \
                            ^
core/adios_internals.c:4809:13: note: in expansion of macro 'ADIOS_STATISTICS'
             ADIOS_STATISTICS(int8_t,1)
             ^
core/adios_internals.c:4754:28: error: non-floating-point argument in call to function '__builtin_isfinite'
             hist = (struct adios_hist_struct *) stats[map[adios_statistic_hist]].data; \
                            ^
[...]
make: *** [all] Error 2

Is a there any particular reason for having the maximum aggregation buffer size MAX_AGG_BUF being fixed as a define or would it be possible to have it set at runtime, e.g. during startup?

First of all: Sorry for all the nitpicking! ADIOS is awesome, these are just some minor issues that it would be nice to have resolved :)

Is it possible to define adios_long_long and adios_cxx_bool?

I see that there is adios_long, but adios_long is not necessarily guaranteed to be 64-bit, and therefore my code uses 'long long', instead. Currently I have a runtime check that sizeof(long) == sizeof(long long), in order to process 'long long' data with ADIOS, but there is no guarantee of this.

Likewise, I am currently working around the lack of adios_cxx_bool (as is available in the latest MPI) by using adios_byte, but I do not believe that there is any guarantee that bool is just 1 byte.

I would like to make the following proposal / RFE for an upcoming 1.9 release:

It would be great if one could set the option for the chunk_size during reads alternatively via the C API in adios_read_init_method (params, third argument).

Right now, the only way I know of is to export of a real environment variable.
If not set, both ADIOS_READ_METHOD_BP and ADIOS_READ_METHOD_BP_AGGREGATE complains about it and aborts execution.

When my code creates multiple ADIOS groups, during clean-up, more than one call to adios_free_group (each call with a separate group id) triggers the error: "Err in adios_common_free_group()". Calling this code when only one of those groups is created does not trigger this error.

@yyalli first of all thank you for the many and great updates in the last months! ✨

It would be fantastic if a adios.__version__ attribute would be supported.
This version represents the version of the ADIOS Python module, so currently 1.9.1b6 (not the version of the ADIOS release, currently 1.9.0, nor the version of ADIOS used to write a specific file).

Defined in PEP0396 for modules (also briefly mentioned in PEP8).

Examples:

import numpy as np
import h5py as h5
import adios

print(np.__version__)
print(h5.__version__)
# print(adios.__version__) # not implemented

# my current work-around
import pkg_resources;
print(pkg_resources.get_distribution('adios').version)

I am using CMake to build ADIOS, and set BUILD_FORTRAN to OFF. When I run make install, it attempts to install the various ADIOS Fortran libraries anyway, which leads to an error.

I worked around this by commenting out the Fortran libraries:

install(FILES ${PROJECT_BINARY_DIR}/src/libadios.a
        ${PROJECT_BINARY_DIR}/src/libadios_nompi.a
        #${PROJECT_BINARY_DIR}/src/libadiosf.a
        #${PROJECT_BINARY_DIR}/src/libadiosf_v1.a
        ${PROJECT_BINARY_DIR}/src/libadiosread.a
        ${PROJECT_BINARY_DIR}/src/libadiosread_nompi.a
        #${PROJECT_BINARY_DIR}/src/libadiosreadf.a
        #${PROJECT_BINARY_DIR}/src/libadiosreadf_nompi.a
        #${PROJECT_BINARY_DIR}/src/libadiosreadf_nompi_v1.a
        #${PROJECT_BINARY_DIR}/src/libadiosreadf_v1.a
        #${PROJECT_BINARY_DIR}/src/libadiosf_nompi.a
        #${PROJECT_BINARY_DIR}/src/libadiosf_nompi_v1.a
        DESTINATION ${libdir})

Hi @yyalli,

are you planning to add Python3 support for the numpy wrapper?

Thanks,
Axel

Hi,
I would like to know if you plan to introduce the support of polygonal and polyhedral meshes in ADIOS. If I am right, currently, ADIOS supports unstructured meshes composed of only one kind of cell. In addition, the cell types are limited to lines, triangles, quads, hexahedrons, prisms, tetrahedrons, and pyramids. Is there some technical limitations to not support polygonal and polyhedral meshes?

I am practising with the manual (1.8.0) and the example code.
Section 14.8.1 of manual and examples/C/global-array,

seb56@c061hk:/pkg/adios-1.8.0/examples/C/global-array$ mpirun -n 4 ./adios_global
seb56@c061hk:/pkg/adios-1.8.0/examples/C/global-array$ bpls -latd adios_global.bp -n 10
integer NX scalar = 10
integer size scalar = 4
integer rank scalar = 0
double temperature {4, 10} = 0 / 39 / 19.5 / 11.5434
(0,0) 0 1 2 3 4 5 6 7 8 9
(1,0) 10 11 12 13 14 15 16 17 18 19
(2,0) 20 21 22 23 24 25 26 27 28 29
(3,0) 30 31 32 33 34 35 36 37 38 39

string temperature/description attr = "Global array written from 'size' processes"

So far so good.
However, converting to netcdf seems to corrupt the data.

seb56@c061hk:/pkg/adios-1.8.0/examples/C/global-array$ bp2ncd adios_global.bp
local[0]: 80
local[0]: 80
local[0]: 80
local[0]: 80
seb56@c061hk:/pkg/adios-1.8.0/examples/C/global-array$ ncdump adios_global.nc
netcdf adios_global {
dimensions:
NX = 10 ;
size = 4 ;
temperature_0 = 80 ;
rank = 1 ;
variables:
byte temperature(temperature_0) ;
data:

temperature = 0, 0, 0, 0, 0, 0, 62, 64, 0, 0, 0, 0, 0, 0, 63, 64, 0, 0, 0,
0, 0, 0, 64, 64, 0, 0, 0, 0, 0, -128, 64, 64, 0, 0, 0, 0, 0, 0, 65, 64,
0, 0, 0, 0, 0, -128, 65, 64, 0, 0, 0, 0, 0, 0, 66, 64, 0, 0, 0, 0, 0,
-128, 66, 64, 0, 0, 0, 0, 0, 0, 67, 64, 0, 0, 0, 0, 0, -128, 67, 64 ;
}

From the manual, I was expecting to see temperature ranging 0...39

I would recommend supporting floats for indexing with the Python Adios bindings, similar to the h5py package for HDF5 files in Python. See below.

In h5py, the following works:
In [25]: file_path5='file.h5'
In [26]: f5 = h5py.File(file_path5)
In [27]: d1 = f5[data_path][:,1,:]
In [28]: d2 = f5[data_path][:,1.,:]

In adios, we get an error when using a float to slice:
In [29]: file_path=''file.bp'
In [30]: f = ad.file(file_path)
In [31]: d1 = f[data_path][:,1,:]

In [32]: d2 = f[data_path][:,1.,:]

AttributeError Traceback (most recent call last)
in ()
----> 1 d2 = f[data_path][:,1.,:]

adios.pyx in adios.var.getitem (adios.cpp:14190)()

AttributeError: 'float' object has no attribute 'step'

@yyalli I am splitting this thread from issue #53 since I accidentally went OT there.

It would also be great if we could bring the general interface closer to h5py since this would make many scripts easily portable, accessing a huge community of people already using h5py and one can benefit from a very easy interface (that ADIOS can extend where necessary, e.g., for transport methods).

Some points:

• .file -> .File (upper case in the read API)
• .attr -> .attrs renaming
• emulating that each group and each variable object can in turn again have a .attrs member (that simply checks the global list of attributes for attributes that share the same prefix)

Here is an example that allows to create objects for individual (sub-)groups and variables, making programming way easier for users:

import adios as ad

f = ad.file("adios_test.bp")
t = f["temperature"] # type: adios.var

# here t could have a member dictionary attr(s) again
# which looks up all attributes starting with t.name
# now match: f.attrs["/temperature/description"]
print( t.attrs["description"] )

# the same should be possible for groups
g = f["someSubGroup/anOtherGroup/"] # type: adios.group
# now match: f.attrs["/someSubGroup/anOtherGroup/anOtherAttribute"]
print( g.attrs["anOtherAttribute"] )
# now match: f["/someSubGroup/anOtherGroup/anOtherVariable"]
print( g["anOtherVariable"] )

I know that attributes are currently stored as a list and the full path is the identifier in bp, nevertheless the numpy API can just abstract and "visualize" these objects by simple string-matching.

I am writing this because I am urgently in the need to have our openPMD validator scripts for ADIOS, too. But to keep them maintainable it would be extremely helpful to have the ADIOS numpy wrapper in a clean OOP style such as the one from h5py (also on github).

We are also using the numpy wrapper in our project pyDive that is already struggling with missing OOP in the wrapper. We would really like to include the wrapper in openPMD-viewer with the LBNL and DESY folks but also here, more OOP as in the example above would help a lot.

When working with the python community we need to be aware that Python is not C, so interfaces need to be in object-centric, with object methods rather than function calls and well-documented via DocStrings (PEP8 PEP257). Python users work interactively via ipython and query DocStrings, e.g., with object? and object.method?. Python developers like abstracting things into modules, so changing an import h5py as io to import adios as io should be all that is necessary for a first serial reader and writer to become "ADIOS". Also, documentation can be auto-generated via Sphinx/ReadTheDocs and we could add examples/tests and if you like travis-ci tests during development. It might also be a good idea to split the wrappers from the main repo and include adios as a git submodule to keep the changing APIs and updates controllable.

Last but not least, HDF Compass could get a very simple backend implementation with ADIOS as soon as the wrapper is a bit more object oriented. This would automatically "equip" ADIOS with a GUI such as users know it from HDFView.

adios_dataspaces.c:780 in ds_pack_group_info()
When packing attributes, it does not check if an attribute refers to a variable instead of having a direct value. Therefore a->characteristics->value is NULL, and the code seg.faults.

If we have a series of BP files with one timestep each, and then we try to read data from one of the files (not the first one) using the streaming API, we get a assertion failure in bp_utils.c:2336 in function bp_get_dimensions_generic().

The reason for this is that the file has one timestep, which is not 1 but something larger (the actual time index at writing). The read API uses time indexes (0,1,...) relative to the first step in the file. The while loop in the function before the assert runs over the entire characteristics set because the relative time does not equal the real timestep: fp->current_step+1 != time_index

The function has the protytype:

int adios_close (int64_t fd_p);

The retval is initialized with common_adios_close() but from what I can see it is never returned by adios_close(). Simply put, the return retval; is missing.

I am trying to integrate FindADIOS.cmake into my code. When I find_package(ADIOS), CMake reports:

-- Found 'adios_config': /Users/acorriga/adios/bin/adios_config
-- ADIOS linker flags (unparsed): -L/Users/acorriga/adios/lib -ladios /usr/lib/libm.dylib;/Users/acorriga/homebrew/lib/libmxml.dylib;/usr/lib/libz.dylib
-- Found adios in /Users/acorriga/adios/lib/libadios.a

When I link to ${ADIOS_LIBRARIES} using target_link_libraries it links to libadios.a, but none of its dependencies, and so I get linker errors due to not linking with libmxml.dylib and libz.dylib. I was able to workaround this by changing CMAKE_EXE_LINKER_FLAGS to -L/Users/acorriga/homebrew/lib -L/Users/acorriga/adios/lib -ladios -lm -lmxml -lz, but that defeats the purpose of FindADIOS.

What am I doing wrong?

I'm building 1.7.0 using configure (not CMake), and parallel builds (make -j8) will sporadically fail with the following error:

make[3]: Entering directory `/home/elliott/code/adios-1.7.0/tests/genarray'
rm -f gwrite_genarray.fh gread_genarray.fh
../../utils/gpp/gpp.py ./genarray3d.xml
mpif90 -DHAVE_CONFIG_H -I. -I../..  -I../../src    -g -O2 -c -o genarray2D-genarray2D.o `test -f 'genarray2D.F90' || echo './'`genarray2D.F90
rm -f gwrite_genarray.fh gread_genarray.fh
mpif90 -DHAVE_CONFIG_H -I. -I../..  -I../../src    -g -O2 -c -o copyarray2D-copyarray2D.o `test -f 'copyarray2D.F90' || echo './'`copyarray2D.F90
../../utils/gpp/gpp.py ./genarray3d.xml
test "." = "." || cp ./genarray3d.xml ./genarray.xml .
mpif90 -DHAVE_CONFIG_H -I. -I../..  -I../../src    -g -O2 -c -o genarray-genarray.o `test -f 'genarray.F90' || echo './'`genarray.F90
mpif90 -DHAVE_CONFIG_H -I. -I../..  -I../../src    -g -O2 -c -o copyarray-copyarray.o `test -f 'copyarray.F90' || echo './'`copyarray.F90
copyarray.F90:209.18:

    cache_end_time = MPI_WTIME()
                  1
Error: Symbol 'cache_end_time' at (1) has no IMPLICIT type
copyarray.F90:200.20:

    cache_start_time = MPI_WTIME()
                    1
Error: Symbol 'cache_start_time' at (1) has no IMPLICIT type
copyarray.F90:210.20:

    cache_total_time = cache_end_time - cache_start_time
                    1
Error: Symbol 'cache_total_time' at (1) has no IMPLICIT type
copyarray.F90:239.18:

    cache_end_time = MPI_WTIME()
                  1
Error: Symbol 'cache_end_time' at (1) has no IMPLICIT type
copyarray.F90:233.20:

    cache_start_time = MPI_WTIME()
                    1
Error: Symbol 'cache_start_time' at (1) has no IMPLICIT type
copyarray.F90:240.20:

    cache_total_time = cache_end_time - cache_start_time
                    1
Error: Symbol 'cache_total_time' at (1) has no IMPLICIT type
make[3]: *** [copyarray-copyarray.o] Error 1
make[3]: *** Waiting for unfinished jobs....
make[3]: Leaving directory `/home/elliott/code/adios-1.7.0/tests/genarray'
make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory `/home/elliott/code/adios-1.7.0/tests'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/home/elliott/code/adios-1.7.0'
make: *** [all] Error 2

A subsequent build with serial make completes fine. This is usually a clear sign that some dependency is missing from the Makefile.

Skel templates appear to install to $(prefix)/etc/skel/templates. But if prefix = /usr, then templates go in /usr/etc/skel/templates, which doesn't make much sense. They should be in /etc.

The simplest part to fix is utils/skel/Makefile.am and utils/skel/etc/Makefile.am, which should use @sysconfdir@ instead of @prefix@/etc. But there are (probably) other files that reference the templates which would need modifying, and of which I'm not aware.

Allocations by adios_read_hooks_init and adios_query_hooks_init leak memory. Simple test case (had to append .txt for github) and valgrind output attached. Adios 1.9.0, no MPI, valgrind 3.11, 64-bit Linux system.
adios.c.txt
valgrind.txt

Inside of <\global-bounds> the element is unknown:
WARN : config.xml: invalid xml element: 'gwrite'

Someone may want to check 'adios_err' after each adios call with

Comparing to the latest release, 1.9.0, the numpy python wrapper seems to be broken in master as of f67b888

I installed adios from both the v1.9.0 tag and master and tested the numpy wrapper from pip and wrapper/numpy itself.

The result is, that in the latest master

import adios as ad
f = ad.file("someFile.bp")

leads to a segmentation fault, killing the whole python instance.

Did something change in the adios library the wrapper interfaces with?
Unfortunately I do not have a python with debug symbols by hand to go further into the issue via gdb.

currently, configure only accepts path to lustre directory and not to include and lib directory separately unlike hdf or netcdf

on cray system, the liblustre exists under lustre-cray_gem_s//lib64, configure only checks under "lib"

For now I pass the correct path by modifying configure to look under lib64 or by passing it to LDFLAGS.

Hi,

I am trying to compile the ADIOS master branch on Kraken. And I have run into the following troubles. Any suggestions what to do to make it run?

make LDFLAGS=-L/opt/cray/lustre-cray_ss_s/default/lib64
CDPATH="${ZSH_VERSION+.}:" && cd . && /bin/sh /nics/d/home/smagg/src/adios/kraken-tau/config/missing --run aclocal-1.11 -I config
configure.ac:77: warning: AC_LANG_CONFTEST: no AC_LANG_SOURCE call detected in body
../../lib/autoconf/lang.m4:194: AC_LANG_CONFTEST is expanded from...
../../lib/autoconf/general.m4:2662: _AC_LINK_IFELSE is expanded from...
../../lib/autoconf/general.m4:2679: AC_LINK_IFELSE is expanded from...
config/ac_portals.m4:22: AC_PORTALS is expanded from...
configure.ac:77: the top level
configure.ac:77: warning: AC_LANG_CONFTEST: no AC_LANG_SOURCE call detected in body
../../lib/autoconf/lang.m4:194: AC_LANG_CONFTEST is expanded from...
../../lib/autoconf/general.m4:2662: _AC_LINK_IFELSE is expanded from...
../../lib/autoconf/general.m4:2679: AC_LINK_IFELSE is expanded from...
../../lib/m4sugar/m4sh.m4:606: AS_IF is expanded from...
../../lib/autoconf/general.m4:2662: _AC_LINK_IFELSE is expanded from...
../../lib/autoconf/general.m4:2679: AC_LINK_IFELSE is expanded from...
config/ac_portals.m4:22: AC_PORTALS is expanded from...
configure.ac:77: the top level
configure.ac:84: warning: AC_LANG_CONFTEST: no AC_LANG_SOURCE call detected in body
../../lib/autoconf/lang.m4:194: AC_LANG_CONFTEST is expanded from...
../../lib/autoconf/general.m4:2662: _AC_LINK_IFELSE is expanded from...
../../lib/autoconf/general.m4:2679: AC_LINK_IFELSE is expanded from...
config/ac_nssi.m4:22: AC_NSSI is expanded from...
configure.ac:84: the top level
cd . && /bin/sh /nics/d/home/smagg/src/adios/kraken-tau/config/missing --run automake-1.11 --gnu
configure.ac:77: warning: AC_LANG_CONFTEST: no AC_LANG_SOURCE call detected in body
../../lib/autoconf/lang.m4:194: AC_LANG_CONFTEST is expanded from...
../../lib/autoconf/general.m4:2662: _AC_LINK_IFELSE is expanded from...
../../lib/autoconf/general.m4:2679: AC_LINK_IFELSE is expanded from...
config/ac_portals.m4:22: AC_PORTALS is expanded from...
configure.ac:77: the top level
configure.ac:77: warning: AC_LANG_CONFTEST: no AC_LANG_SOURCE call detected in body
../../lib/autoconf/lang.m4:194: AC_LANG_CONFTEST is expanded from...
../../lib/autoconf/general.m4:2662: _AC_LINK_IFELSE is expanded from...
../../lib/autoconf/general.m4:2679: AC_LINK_IFELSE is expanded from...
../../lib/m4sugar/m4sh.m4:606: AS_IF is expanded from...
../../lib/autoconf/general.m4:2662: _AC_LINK_IFELSE is expanded from...
../../lib/autoconf/general.m4:2679: AC_LINK_IFELSE is expanded from...
config/ac_portals.m4:22: AC_PORTALS is expanded from...
configure.ac:77: the top level
configure.ac:84: warning: AC_LANG_CONFTEST: no AC_LANG_SOURCE call detected in body
../../lib/autoconf/lang.m4:194: AC_LANG_CONFTEST is expanded from...
../../lib/autoconf/general.m4:2662: _AC_LINK_IFELSE is expanded from...
../../lib/autoconf/general.m4:2679: AC_LINK_IFELSE is expanded from...
config/ac_nssi.m4:22: AC_NSSI is expanded from...
configure.ac:84: the top level
src/Makefile.am:32: library used but RANLIB' is undefined src/Makefile.am:32: The usual way to defineRANLIB' is to add AC_PROG_RANLIB' src/Makefile.am:32: toconfigure.ac' and run autoconf' again. tests/suite/programs/Makefile.am:148: variabletransforms_SOURCES' is defined but no program or
tests/suite/programs/Makefile.am:148: library has transforms' as canonical name (possible typo) tests/suite/programs/Makefile.am:150: variabletransforms_LDADD' is defined but no program or
tests/suite/programs/Makefile.am:150: library has transforms' as canonical name (possible typo) tests/suite/programs/Makefile.am:151: variabletransforms_LDFLAGS' is defined but no program or
tests/suite/programs/Makefile.am:151: library has transforms' as canonical name (possible typo) utils/adios_lint/Makefile.am:15: CC was already defined in condition TRUE, which includes condition USE_PARALLEL_COMPILER ... configure.ac:16: ...CC' previously defined here
utils/adios_lint/Makefile.am:16: CXX was already defined in condition TRUE, which includes condition USE_PARALLEL_COMPILER ...
configure.ac:18: ... CXX' previously defined here utils/bp2ascii/Makefile.am:15: CC was already defined in condition TRUE, which includes condition USE_PARALLEL_COMPILER ... configure.ac:16: ...CC' previously defined here
utils/bp2ascii/Makefile.am:16: CXX was already defined in condition TRUE, which includes condition USE_PARALLEL_COMPILER ...
configure.ac:18: ... CXX' previously defined here utils/bp2h5/Makefile.am:12: CC was already defined in condition TRUE, which includes condition USE_PARALLEL_COMPILER ... configure.ac:16: ...CC' previously defined here
utils/bp2h5/Makefile.am:13: CXX was already defined in condition TRUE, which includes condition USE_PARALLEL_COMPILER ...
configure.ac:18: ... CXX' previously defined here utils/bp2ncd/Makefile.am:11: CC was already defined in condition TRUE, which includes condition USE_PARALLEL_COMPILER ... configure.ac:16: ...CC' previously defined here
utils/bp2ncd/Makefile.am:12: CXX was already defined in condition TRUE, which includes condition USE_PARALLEL_COMPILER ...
configure.ac:18: ... CXX' previously defined here utils/bpdump/Makefile.am:15: CC was already defined in condition TRUE, which includes condition USE_PARALLEL_COMPILER ... configure.ac:16: ...CC' previously defined here
utils/bpdump/Makefile.am:16: CXX was already defined in condition TRUE, which includes condition USE_PARALLEL_COMPILER ...
configure.ac:18: ... CXX' previously defined here utils/bpsplit/Makefile.am:27: CC was already defined in condition TRUE, which includes condition USE_PARALLEL_COMPILER ... configure.ac:16: ...CC' previously defined here
utils/bpsplit/Makefile.am:28: CXX was already defined in condition TRUE, which includes condition USE_PARALLEL_COMPILER ...
configure.ac:18: ... CXX' previously defined here utils/skel/Makefile.am:11: whitespace following trailing backslash utils/skel/src/Makefile.am:6: library used butRANLIB' is undefined
utils/skel/src/Makefile.am:6: The usual way to define RANLIB' is to addAC_PROG_RANLIB'
utils/skel/src/Makefile.am:6: to configure.ac' and runautoconf' again.
make: *** [Makefile.in] Error 1

This are my module list:

module list
Currently Loaded Modulefiles:

1. modules/3.2.6.6
2. portals/2.2.0-1.0301.26633.6.9.ss
3. nodestat/2.2-1.0301.25918.4.1.ss
4. sdb/1.0-1.0301.25929.4.88.ss
5. MySQL/5.0.64-1.0301.2899.20.1.ss
6. lustre-cray_ss_s/1.8.4_2.6.27.48_0.12.1_1.0301.5943.18.1-1.0301.27524.1.24
7. Base-opts/1.0.2-1.0301.25878.4.1.ss
8. xtpe-network-seastar
9. PrgEnv-gnu/3.1.72
10. atp/1.4.1
11. xt-asyncpe/5.11
12. pmi/2.1.4-1.0000.8596.15.1.ss
13. xt-mpich2/5.3.5
14. xt-libsci/11.0.04
15. gcc/4.6.2
16. torque/2.5.11
17. moab/6.1.6
18. xtpe-istanbul
19. gold
20. hsi
21. altd/1.0
22. globus/5.0.4
23. xdusage/1.0-r7
24. DefApps
25. mag-utils
26. git/1.7.4.2
27. cmake/2.8.7
28. mag-evpath/current
29. mag-mxml-2.7/kraken-gnu
30. java/jdk1.6.0_24
31. tau/2.22.1
32. automake/1.11.1
33. autoconf/2.68
34. perl/5.14.2-cnl

This is how I configure ADIOS.

CFLAGS="-fPIC -g" MPICC=cc MPICXX=CC MPIFC=ftn ./configure --prefix=/nics/d/home/smagg/opt/adios/kraken/gnu/git-master --with-flexpath=/nics/d/home/smagg/opt/evpath/kraken/gnu --with-mxml=/nics/d/home/smagg/opt/mxml-2.7/kraken/gnu --with-lustre=/opt/cray/lustre-cray_ss_s/default --without-portals --without-infiniband

Thanks,
Magda

On Titan, writes with more than 2GB file size per local process using MPI_AGGREGATE fail due to int overflows (using with gcc/4.8.2 on that machine).

This issue is just opened to document parts in the code that @psychocoderHPC and me found in the last days, where critical assumptions about types and sizes of types are made:

General usage of int

The exact type of int is not defined, but is usually int32_t (sometimes int16_t but "very seldom" int64_t):

• in nearly all src/write methods, the adios_MPI_data_struct has an int size; member which is used in many calculations such as
  • adios_mpi_build_file_offset
  • adios_mpi_aggregate_should_buffer
  • adios_mpi_aggregate_close
  • and probably in the internal .bp binary file format itself, since bpls on such a file gives a ERROR: Invalid BP file detected. PG index offset (-2135412183) > file size (2159585741) (needs verification)
  • several element counters will only work up to 2 billion elements with int

Possible Affected Parts of the Code

Other parts that might need updates when consequently using int64_t types for sizes:

• several log-messages assume 32bit values for now, such as src/core/adios_bp_v1.c which then need to use %s: hashtbl=%p size=%lld
• the struct struct qhashtbl_s also has int* size members
• the structs adios_index_attribute_struct_v1 and adios_attribute_struct_v1 use the imprecise notion int for the counters nelems

Testing environment

• ORNL's Titan cluster
• modules:
  • gcc/4.8.2 (with PrgEnv-gnu/5.2.40)
  • cray-mpich/7.0.4
  • adios/1.8.0
    • mxml/2.9
    • dataspaces/1.4.0
• user software PIConGPU linking further:
  • cudatoolkit/5.5.51-1.0502.9594.3.1
  • boost/1.54.0
  • cray-hdf5-parallel/1.8.12
  • -lz/-lpng and the like (all not relevant for this issue)
  • ADIOS implemention

Data transforms with zlib compression were used with 8k processes, 2k aggregators and 500 OSTs. The option have_metadata_file=0 was set during writes.

The error occured "offline" while creating the meta file using bpmeta (hangs and allocates inf. amount of memory) and using bpls on the files in fileName.bp.dir/<fileAbove2GB>.bp.%d:

ERROR: Invalid BP file detected. PG index offset (-2135412183) > file size (2159585741)
ERROR: File open failed: fileName.bp.dir/<fileAbove2GB>.bp.<%d>

Affected Versions and Work-Arounds

The problem affects release 1.8.0 and the current version of master (3c3b80d).

A work-around is to increase the number of files (which makes aggregates files smaller) and to harm performance - by simply using more aggregators. This probably postpones the problem a bit but a fix to use the optimum performance is would be very appreciated :)

Please tell us if we can provide more information on the issue.

I just built and installed ADIOS 1.7, but adios_config -v, and thus FindADIOS reports version 1.6.

The culprit seems to be: https://github.com/ornladios/ADIOS/blob/master/CMakeLists.txt#L1272

I tried to build adios 1.7.0 with data transform (compression) support using ZLIB and SZIP.
It works fine for zlib but SZIP is not available (not reported by adios_config -m nor during runtime). However, configure reports that SZIP has been properly found:

configure:35394: checking for zlib.h
configure:35403: result: yes
configure:35455: === checking for BZIP2 ===
configure:35682: === checking for SZIP ===
configure:35742: checking szlib.h usability
configure:35759: /opt/pkg/compiler/gnu/gcc/4.6.2/bin/gcc-4.6.2 -c -g -O2 -g0 -O3 -m64 -I[home]/bin/szip-2.1/include conftest.c >&5
configure:35766: $? = 0
configure:35780: result: yes
configure:35784: checking szlib.h presence
configure:35799: /opt/pkg/compiler/gnu/gcc/4.6.2/bin/gcc-4.6.2 -E -g0 -O3 -m64 -I[home]/bin/szip-2.1/include conftest.c
configure:35806: $? = 0
configure:35820: result: yes
configure:35848: checking for szlib.h
configure:35857: result: yes

SZIP_CPPFLAGS='-I[home]/bin/szip-2.1/include'
SZIP_LDFLAGS='-L[home]/bin/szip-2.1/lib'
SZIP_LIBS='-lsz'

#define HAVE_SZIP 1

Output from adios_config -m is

Available data transformation methods (in XML transform tags in <var> elements):
    "none"      : No data transform
    "identity"  : Identity transform
    "zlib"      : zlib compression

readvar should raise an error when the variable isn't found, instead of simply printing 'No variable found'. Behavior should mimic getitem

Calls to adios_init_noxml unconditionally return 1, indicating an error if compared with "err_no_error" from public/adios_error.h or src/core/adiosf_defs_mod.f90. The call sequence is:

adios_init_noxml (src/core/adios.c:49)
common_adios_init_noxml (common_adios.c:54)
adios_local_config (adios_internals_mxml.c:2264)

The return value of 1 is located at adios_internals_mxml.c:2275. The return value should be changed to 0, or perhaps better yet to "err_no_error" from public/adios_error.h.

Many of the functions in src/core/adios_internals_mxml.c (and in other parts of ADIOS) return 1 for success and 0 to indicate an error. This is inconsistent with common practice and with the values in public/adios_error.h, where 0 indicates success and a non-zero value indicates an error.

Is it possible to distribute and automatically build MXML with ADIOS? I wouldn't make such a request about a large library like HDF5, but MXML is small and not that common either. This would be one less step in getting ADIOS set up and installed.

Alternatively, could the XML interface to ADIOS please be made optional? My code doesn't use the XML interface to ADIOS, so perhaps it is not necessary, and could be disabled at compile-time.

My build was failing because -ldl is not passed to the linker

I ran configure with LIBS set to "-ldl" but the final configuration shows

LIBS =

I looked through the configure script and it looks like the LIBS value is getting overwritten instead of appending new values to it as needed.

not passing in ldl is another issue.

the current master version as of 649ec37 still contains a few warnings that might need a fix before the next release.

Environment

Compiler: gcc 4.8.2 (on Titan/ORNL)
MPICH: 3.1.1
relevant modules (extract): craype/2.2.1 cray-mpich/7.0.4 craype-interlagos gcc/4.8.2 PrgEnv-gnu/5.2.40 mxml/2.9
configure:

LDFLAGS="-fPIC -pthread" CFLAGS="-fPIC -g -O2" CXXFLAGS="-fPIC -g -O2" \
  ./configure --prefix=$MEMBERWORK/$proj/lib/adios-pre1.9.0 --with-zlib \
    --with-mpi=$MPICH_DIR --enable-static --enable-shared \
    --with-mxml=$MXML_DIR --without-dataspaces

MPI-Related

write/adios_mpi_amr.c: In function 'adios_mpi_amr_ag_close':
write/adios_mpi_amr.c: In function 'adios_mpi_amr_ag_close':
write/adios_mpi_amr.c:2954:29: warning: passing argument 5 of 'MPI_Gatherv' from incompatible pointer type [enabled by default]
                             ,0, md->g_comm1);
                             ^
In file included from ./public/adios_mpi.h:16:0,
                 from write/adios_mpi_amr.c:24:
/opt/cray/mpt/7.0.4/gni/mpich2-gnu/48/include/mpi.h:972:5: note: expected 'const int *' but argument is of type 'uint64_t *'
 int MPI_Gatherv(const void *sendbuf, int sendcount, MPI_Datatype sendtype, void *recvbuf,
     ^
write/adios_mpi_amr.c:2954:29: warning: passing argument 5 of 'MPI_Gatherv' from incompatible pointer type [enabled by default]
                             ,0, md->g_comm1);
                             ^
In file included from ./public/adios_mpi.h:16:0,
                 from write/adios_mpi_amr.c:24:
/opt/cray/mpt/7.0.4/gni/mpich2-gnu/48/include/mpi.h:972:5: note: expected 'const int *' but argument is of type 'uint64_t *'
 int MPI_Gatherv(const void *sendbuf, int sendcount, MPI_Datatype sendtype, void *recvbuf,
     ^
write/adios_mpi_amr.c:2954:29: warning: passing argument 6 of 'MPI_Gatherv' from incompatible pointer type [enabled by default]
                             ,0, md->g_comm1);
                             ^
In file included from ./public/adios_mpi.h:16:0,
                 from write/adios_mpi_amr.c:24:
/opt/cray/mpt/7.0.4/gni/mpich2-gnu/48/include/mpi.h:972:5: note: expected 'const int *' but argument is of type 'uint64_t *'
 int MPI_Gatherv(const void *sendbuf, int sendcount, MPI_Datatype sendtype, void *recvbuf,
     ^
write/adios_mpi_amr.c:2954:29: warning: passing argument 6 of 'MPI_Gatherv' from incompatible pointer type [enabled by default]
                             ,0, md->g_comm1);
                             ^
In file included from ./public/adios_mpi.h:16:0,
                 from write/adios_mpi_amr.c:24:
/opt/cray/mpt/7.0.4/gni/mpich2-gnu/48/include/mpi.h:972:5: note: expected 'const int *' but argument is of type 'uint64_t *'
 int MPI_Gatherv(const void *sendbuf, int sendcount, MPI_Datatype sendtype, void *recvbuf,
     ^
write/adios_mpi_amr.c: In function 'adios_mpi_amr_ag_close':
write/adios_mpi_amr.c:2954:29: warning: passing argument 5 of 'MPI_Gatherv' from incompatible pointer type [enabled by default]
                             ,0, md->g_comm1);
                             ^
In file included from ./public/adios_mpi.h:16:0,
                 from write/adios_mpi_amr.c:24:
/opt/cray/mpt/7.0.4/gni/mpich2-gnu/48/include/mpi.h:972:5: note: expected 'const int *' but argument is of type 'uint64_t *'
 int MPI_Gatherv(const void *sendbuf, int sendcount, MPI_Datatype sendtype, void *recvbuf,
     ^
write/adios_mpi_amr.c:2954:29: warning: passing argument 6 of 'MPI_Gatherv' from incompatible pointer type [enabled by default]
                             ,0, md->g_comm1);
                             ^
In file included from ./public/adios_mpi.h:16:0,
                 from write/adios_mpi_amr.c:24:
/opt/cray/mpt/7.0.4/gni/mpich2-gnu/48/include/mpi.h:972:5: note: expected 'const int *' but argument is of type 'uint64_t *'
 int MPI_Gatherv(const void *sendbuf, int sendcount, MPI_Datatype sendtype, void *recvbuf

skeldump

In file included from skeldump.c:38:0:
skeldump.h:23:0: warning: "strndup" redefined [enabled by default]
 #  define strndup(str,len) strdup(str)
 ^
In file included from /usr/include/string.h:635:0,
                 from skeldump.c:26:
/usr/include/bits/string2.h:1344:0: note: this is the location of the previous definition
 #   define strndup(s, n) __strndup (s, n)
 ^
skeldump.c:107:5: warning: initialization from incompatible pointer type [enabled by default]
     {"hidden_attrs",         no_argument,          &hidden_attrs,    true}, 
     ^
skeldump.c:107:5: warning: (near initialization for 'options[5].flag') [enabled by default]

bpls

In file included from bpls.c:34:0:
bpls.h:16:0: warning: "strndup" redefined [enabled by default]
 #  define strndup(str,len) strdup(str)
 ^
In file included from /usr/include/string.h:635:0,
                 from bpls.c:23:
/usr/include/bits/string2.h:1344:0: note: this is the location of the previous definition
 #   define strndup(s, n) __strndup (s, n)
 ^

After make install, the following files exist in $(bindir) that don't really belong there. They are not executable, or don't actually run:

• ad_config.py - Doesn't do anything by itself. Python files should be put in site-packages. Automake has direct support for Python, so you can easily find where that is with pythondir.
• adios_config.flag - Not executable, and does nothing if sourced. This one could possibly be okay if it exported its variables. Then it could be sourced.
• git.status - Not executable, and the name is wayy too generic.
• list_methods, list_methods_nompi, list_methods_readonly, list_methods_readonly_nompi, set_method.sh - I don't actually think there's anything wrong with them per se; they just have slightly generic names.
• type_mapper.py - Again, this doesn't do anything. It should be put in site-packages.

The Fortran module defined in src/core/adiosf_write_mod.f90 declares an interface for a subroutine named "adios_init_noxml_withcomm". I believe this is actually the interface for the new "adios_init_noxml", which adds a "comm" argument to the old "adios_init_noxml" in 1.4.x. Currently, there is no interface declared in the module for "adios_init_noxml".

The interface should be renamed from "adios_init_noxml_withcomm" to "adios_init_noxml".

Default python interpreter is not given in the gpp.py file. For platforms with python3 as default interpreter (such as Arch Linux), the compilation leads to the following error:

../../utils/gpp/gpp.py ./genarray3d.xml
  File "../../utils/gpp/gpp.py", line 25
    print "Unable to find adios_lint. Proceeding with code generation."
                                                                      ^

Is it possible to use multiple groups in a single .bp file, each group containing different variables, mesh, etc. from each other?
If so, how should I proceed? I did not see any example in the repository.

This is the error message -
genarray.F90:185:0: fatal error: gwrite_genarray.fh: No such file or directory
#include "gwrite_genarray.fh"
^
compilation terminated.
copyarray.F90:207:0: fatal error: gread_genarray.fh: No such file or directory
#include "gread_genarray.fh"
^
compilation terminated.
make[3]: *** [genarray-genarray.o] Error 1
make[3]: *** Waiting for unfinished jobs....
make[3]: *** [copyarray-copyarray.o] Error 1
make[3]: Leaving directory /autofs/nccs-svm1_home1/anshuman/ADIOS/tests/genarray' make[2]: *** [all-recursive] Error 1 make[2]: Leaving directory/autofs/nccs-svm1_home1/anshuman/ADIOS/tests'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/autofs/nccs-svm1_home1/anshuman/ADIOS'
make: *** [all] Error 2

Hi,

I am compiling ADIOS (master branch) on Kraken. This is an attempt to instrument ADIOS with TAU. In general it is not important because the issue happens with one of the Fortran tests

...
Debug: Compiling with Instrumented Code
Executing> ftn -g -O2 -c genarray.F90 -DHAVE_CONFIG_H -I. -I../.. -I../../src -I/nics/d/home/smagg/opt/mxml-2.7/kraken/gnu/include -I/nics/d/home/smagg/opt/evpath/kraken/gnu/include -I/nics/d/home/smagg/opt/evpath/kraken/gnu/include -I/nics/d/home/smagg/opt/evpath/kraken/gnu/include -I/nics/d/home/smagg/opt/evpath/kraken/gnu/include -I/nics/d/home/smagg/opt/evpath/kraken/gnu/include -I/opt/cray/lustre-cray_ss_s/default/include -I/nics/d/home/smagg/opt/tau/2/include -I/opt/cray/mpt/5.3.5/xt/seastar/mpich2-gnu/46/include -I/nics/d/home/smagg/opt/tau/2/include -I/opt/cray/mpt/5.3.5/xt/seastar/mpich2-gnu/46/include -g -finstrument-functions -o genarray-genarray.o
genarray.F90:185:0: fatal error: gwrite_genarray.fh: No such file or directory
compilation terminated.
Error: Compilation Failed
Error: Command(Executable) is -- ftn
Error: Full Command attempted is -- ftn    -g -O2 -c  genarray.F90   -DHAVE_CONFIG_H -I. -I../.. -I../../src -I/nics/d/home/smagg/opt/mxml-2.7/kraken/gnu/include -I/nics/d/home/smagg/opt/evpath/kraken/gnu/include -I/nics/d/home/smagg/opt/evpath/kraken/gnu/include -I/nics/d/home/smagg/opt/evpath/kraken/gnu/include -I/nics/d/home/smagg/opt/evpath/kraken/gnu/include -I/nics/d/home/smagg/opt/evpath/kraken/gnu/include -I/opt/cray/lustre-cray_ss_s/default/include  -I/nics/d/home/smagg/opt/tau/2/include -I/opt/cray/mpt/5.3.5/xt/seastar/mpich2-gnu/46/include    -I/nics/d/home/smagg/opt/tau/2/include -I/opt/cray/mpt/5.3.5/xt/seastar/mpich2-gnu/46/include   -g -finstrument-functions   -o genarray-genarray.o
Error: Reverting to a Regular Make


make[4]: *** [all] Error 1
make[3]: *** [genarray-genarray.o] Error 1
make[3]: Leaving directory `/nics/d/home/smagg/src/adios/kraken-tau/tests/genarray'
make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory `/nics/d/home/smagg/src/adios/kraken-tau/tests'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/nics/d/home/smagg/src/adios/kraken-tau'
make: *** [all] Error 2

The missing file is likely here. Maybe gwrite_arrays.fh? Or something similar, but simple change of the file names in genarray.F90 and adding the location of examples/Fortran/arrays to FFLAGS, doesn't work so it is not gwrite_arrays.fh file

> find . -name "*.fh"./examples/Fortran/global-array/gwrite_temperature.fh
./examples/Fortran/global-array/gread_temperature.fh
./examples/Fortran/global-array-time/gread_restart.fh
./examples/Fortran/global-array-time/gwrite_restart.fh
./examples/Fortran/arrays/gwrite_arrays.fh
./examples/Fortran/arrays/gread_arrays.fh
./examples/Fortran/scalars/gwrite_scalars.fh
./examples/Fortran/scalars/gread_scalars.fh

Here is the trick that doesn't work

ftn    -g -O2 -c  genarray.F90   -DHAVE_CONFIG_H -I. -I../.. -I../../src -I/nics/d/home/smagg/opt/mxml-2.7/kraken/gnu/include -I/nics/d/home/smagg/opt/evpath/kraken/gnu/include -I/nics/d/home/smagg/opt/evpath/kraken/gnu/include -I/nics/d/home/smagg/opt/evpath/kraken/gnu/include -I/nics/d/home/smagg/opt/evpath/kraken/gnu/include -I/nics/d/home/smagg/opt/evpath/kraken/gnu/include -I/opt/cray/lustre-cray_ss_s/default/include  -I/nics/d/home/smagg/opt/tau/2/include -I/opt/cray/mpt/5.3.5/xt/seastar/mpich2-gnu/46/include    -I/nics/d/home/smagg/opt/tau/2/include -I/opt/cray/mpt/5.3.5/xt/seastar/mpich2-gnu/46/include -I/nics/d/home/smagg/src/adios/kraken-tau/examples/Fortran/arrays  -g -finstrument-functions   -o genarray-genarray.o

/nics/d/home/smagg/src/adios/kraken-tau/examples/Fortran/arrays/gwrite_arrays.fh:3.25:
    Included at genarray.F90:187:

                + 8 * (NX) * (NY) &
                         1
Error: Symbol 'nx' at (1) has no IMPLICIT type
/nics/d/home/smagg/src/adios/kraken-tau/examples/Fortran/arrays/gwrite_arrays.fh:3.32:
    Included at genarray.F90:187:

                + 8 * (NX) * (NY) &
                                1
Error: Symbol 'ny' at (1) has no IMPLICIT type
/nics/d/home/smagg/src/adios/kraken-tau/examples/Fortran/arrays/gwrite_arrays.fh:9.52:
    Included at genarray.F90:187:

call adios_write (adios_handle, "var_int_1Darray", p, adios_err)
                                                    1
Error: Symbol 'p' at (1) has no IMPLICIT type
/nics/d/home/smagg/src/adios/kraken-tau/examples/Fortran/arrays/gwrite_arrays.fh:8.55:
    Included at genarray.F90:187:

call adios_write (adios_handle, "var_double_2Darray", t, adios_err)
                                                       1
Error: Symbol 't' at (1) has no IMPLICIT type

Regards,

Hi,

I have recently successfully compiled ADIOS on kraken (branch master). However, during the compilation I have found two issues that the build process complained about missing include files. Added <sys/time.h> and <sys/types.h> solved those compilation issues.

diff --git a/examples/C/compression_test/genarray3D.c b/examples/C/compression_test/genarray3D.c
index 062b70c..e02577d 100755
--- a/examples/C/compression_test/genarray3D.c
+++ b/examples/C/compression_test/genarray3D.c
@@ -1,3 +1,4 @@
+#include <sys/time.h>
 #include <stdio.h>
 #include <stdlib.h>
 #include <string.h>
diff --git a/examples/C/transforms/adios_write_all_3D.c b/examples/C/transforms/adios_write_all_3D.c
index ebdf3ac..db32092 100644
--- a/examples/C/transforms/adios_write_all_3D.c
+++ b/examples/C/transforms/adios_write_all_3D.c
@@ -12,6 +12,7 @@
  * ADIOS config file: adios_global.xml
  *
 */
+#include <sys/types.h>
 #include <stdio.h>
 #include <string.h>
 #include "mpi.h"
@@ -19,7 +20,6 @@
 #include <stdint.h>
 #include <stdlib.h>
 #include <assert.h>
-
 #include <core/adios_internals.h>

 #define MAX_DIMS 5

It is perhaps an issue with the manual, rather than with gpp.py
There is examples/C/manual/gread_temperature.ch, but gpp.py is supposed to generate gread_temperatue.ch as well as gwrite_temperature.ch.

The file contained in the package reads:

adios_groupsize = 0;
adios_totalsize = 0;
adios_group_size (adios_handle, adios_groupsize, &adios_totalsize);
adios_buf_size = 4;
adios_read (adios_handle, "NX", &NX, adios_buf_size);
adios_buf_size = 8 * (NX);
adios_read (adios_handle, "/temperature", t, adios_buf_size);

and works ok.

When gread_temperatue.ch is generated by "gpp.py config.xml", it reads:

s = adios_selection_writeblock (rank);
adios_schedule_read (fp, s, "/temperature", 0, 1, t);
adios_perform_reads (fp, 1);
adios_selection_delete (s);

where s, fp are undefined and the example code with this ch file won't compile.

I would like to request a fully-functional FindADIOS.cmake, which detects not only libadios and adios.h, but also all the dependent libraries.

So far, I have manually performed ADIOS detection in my code's CMakeLists.txt. This works for a minimal installation of ADIOS, but fails for an installation with all the various optional dependencies included (lustre, various Cray libraries, HDF5, NETCDF, etc.) that ADIOS may or may not require. When that fails, I typically just copy and paste flags from adios_config.flags into CMAKE_EXE_LINKER_FLAGS, but that is not an idiomatic CMake way of doing things, and introduces considerable burden on users of my code who are used to the standard way of using CMake.

    find_path(ADIOS_INCLUDE_PATH adios.h)
    if(NOT ADIOS_INCLUDE_PATH)
        message(SEND_ERROR "Please set ADIOS_INCLUDE_PATH")
    endif()
    find_package(Threads)   # pthreads seems to be required by mxml
    list(APPEND libraries ${CMAKE_THREAD_LIBS_INIT})
    find_library(ADIOS_LIBRARY NAMES adios)
    if(NOT ADIOS_LIBRARY)
        message(SEND_ERROR "Please set ADIOS_LIBRARY")
    endif()
    find_library(ADIOS_MXML_LIBRARY NAMES mxml)
    if(NOT ADIOS_MXML_LIBRARY)
        message(SEND_ERROR "Please set ADIOS_MXML_LIBRARY")
    endif()
    list(APPEND libraries ${ADIOS_LIBRARY})
    list(APPEND libraries ${ADIOS_MXML_LIBRARY})
    include_directories(${ADIOS_INCLUDE_PATH})

skel source reports "Don't use adios timing", and produces a skeletal without timing code.

Need to remove debugging output and call the external timing function.

src/write/adios_nc4.c includes only netcdf.h

line 834 uses NC_COLLECTIVE

netcdf_par.h is needed for NC_COLLECTIVE

http://www.unidata.ucar.edu/software/netcdf/release-notes-4.1.2.html

"Users of parallel I/O with netCDF-4 please note: starting with the 4.1.2 release the parallel I/O functions are prototyped in netcdf_par.h, not netcdf.h. You must include netcdf_par.h BEFORE netcdf.h to use parallel I/O with netCDF-4."

configure is specifically looking for lustreapi.h.
Lustre 1.8.x does not have lustreapi.h but has liblustreapi.h

A cursory examination of liblustreapi.h in 2.4 indicates that liblustreapi.h has been renamed to lustreapi.h

Since ADIOS has no specific dependency on Lustre 2.4, configure should look for lustreapi.h and on failure must look for liblustreapi.h

In adios_mpi_amr_bg_close(), the MPI_Gather (currently line 1830) does not complete because it is called only by rank 0. If no attributes are used, this section of code is avoided, and the bug does not occur.

a call of
call adios_write (adios_handle, "/settings/chalt", chalt, adios_err)
results in error as variable name includes the path:
PHDF5 ERROR: can not create scalar /settings/chalt in hw_var!

check src/write/adios_phdf5.c:line 717
pvar->name == '/settings/chalt' instead of 'chalt'