Git Product home page Git Product logo

openpmd-api's Introduction

C++ & Python API for Scientific I/O with openPMD

Supported openPMD Standard Documentation Status Doxygen Gitter chat Supported Platforms License DOI

Linux/OSX Build Status dev Windows Build Status dev Coverity Scan Build Status Coverage Status CodeFactor

This library provides a high-level API for writing and reading scientific data according to openPMD. It allows to build logical file structures which drive scientific I/O libraries such as HDF5 and ADIOS through a common, intuitive interface. Where supported, openPMD-api implements both serial and MPI parallel I/O capabilities.

Usage

C++

C++11 C++11 API: Alpha

#include <openPMD/openPMD.hpp>
#include <iostream>

// ...

auto s = openPMD::Series("samples/git-sample/data%T.h5", openPMD::AccessType::READ_ONLY);

for( auto const& i : s.iterations ) {
    std::cout << "Iteration: " << i.first << "\n";

    for( auto const& m : i.second.meshes ) {
        std::cout << "  Mesh '" << m.first << "' attributes:\n";
        for( auto const& val : m.second.attributes() )
            std::cout << "    " << val << '\n';
    }

    for( auto const& p : i.second.particles ) {
        std::cout << "  Particle species '" << p.first << "' attributes:\n";
        for( auto const& val : p.second.attributes() )
            std::cout << "    " << val << '\n';
    }
}

Python

Python3 Python3 API: Alpha

import openpmd_api

# ...

series = openpmd_api.Series("samples/git-sample/data%T.h5", openPMD.Access_Type.read_only)

for k_i, i in series.iterations.items():
    print("Iteration: {0}".format(k_i))

    for k_m, m in i.meshes.items():
        print("  Mesh '{0}' attributes:".format(k_m))
        for a in m.attributes:
            print("    {0}".format(a))

    for k_p, p in i.particles.items():
        print("  Particle species '{0}' attributes:".format(k_p))
        for a in p.attributes:
            print("    {0}".format(a))

More!

Curious? Our manual shows full read & write examples, both serial and MPI-parallel!

Dependencies

Required:

  • CMake 3.11.0+
  • C++11 capable compiler, e.g. g++ 4.8+, clang 3.9+, VS 2015+

Shipped internally in share/openPMD/thirdParty/:

Optional I/O backends:

while those can be built either with or without:

  • MPI 2.1+, e.g. OpenMPI 1.6.5+ or MPICH2

Optional language bindings:

  • Python:
    • Python 3.5 - 3.7
    • pybind 2.2.4+
    • numpy 1.15+
    • mpi4py 2.1+

Installation

Spack Package Conda Package PyPI Package From Source

Choose one of the install methods below to get started:

Spack Status

# optional:               +python +adios1 -mpi
spack install openpmd-api
spack load -r openpmd-api

Conda Version Conda Status Conda Downloads

# serial version only
conda install -c conda-forge openpmd-api

PyPI Version PyPI Format PyPI Status PyPI Downloads

Behind the scenes, this install method compiles from source against the found installations of HDF5, ADIOS and/or MPI (in system paths, from other package managers, or loaded via a module system, ...). The current status for this install method is experimental. Please feel free to report how this works for you.

# optional:             --user
pip install openpmd-api

or with MPI support:

# optional:                                --user
openPMD_USE_MPI=ON pip install openpmd-api

From Source

Source Status

openPMD can then be installed using CMake:

git clone https://github.com/openPMD/openPMD-api.git

mkdir -p openPMD-api-build
cd openPMD-api-build

# optional: for full tests, with unzip
../openPMD-api/.travis/download_samples.sh

# for own install prefix append:
#   -DCMAKE_INSTALL_PREFIX=$HOME/somepath
# for options append:
#   -DopenPMD_USE_...=...
# e.g. for python support add:
#   -DopenPMD_USE_PYTHON=ON -DPYTHON_EXECUTABLE=$(which python)
cmake ../openPMD-api

cmake --build .

# optional
ctest

# sudo might be required required for system paths
cmake --build . --target install

The following options can be added to the cmake call to control features. CMake controls options with prefixed -D, e.g. -DopenPMD_USE_MPI=OFF:

CMake Option Values Description
openPMD_USE_MPI AUTO/ON/OFF Parallel, Multi-Node I/O for clusters
openPMD_USE_JSON AUTO/ON/OFF JSON backend (.json files)
openPMD_USE_HDF5 AUTO/ON/OFF HDF5 backend (.h5 files)
openPMD_USE_ADIOS1 AUTO/ON/OFF ADIOS1 backend (.bp files)
openPMD_USE_ADIOS2 AUTO/ON/OFF ADIOS2 backend (.bp files) 1
openPMD_USE_PYTHON AUTO/ON/OFF Enable Python bindings
openPMD_USE_INVASIVE_TESTS ON/OFF Enable unit tests that modify source code 2
openPMD_USE_VERIFY ON/OFF Enable internal VERIFY (assert) macro independent of build type 3
PYTHON_EXECUTABLE (first found) Path to Python executable

1 not yet implemented 2 e.g. changes C++ visibility keywords, breaks MSVC 3 this includes most pre-/post-condition checks, disabling without specific cause is highly discouraged

Additionally, the following libraries are shipped internally. The following options allow to switch to external installs:

CMake Option Values Library Version
openPMD_USE_INTERNAL_VARIANT ON/OFF MPark.Variant 1.4.0+
openPMD_USE_INTERNAL_CATCH ON/OFF Catch2 2.6.1+
openPMD_USE_INTERNAL_PYBIND11 ON/OFF pybind11 2.2.4+
openPMD_USE_INTERNAL_JSON ON/OFF NLohmann-JSON 3.5.0+

By default, this will build as a shared library (libopenPMD.[so|dylib|dll]) and installs also its headers. In order to build a static library, append -DBUILD_SHARED_LIBS=OFF to the cmake command. You can only build a static or a shared library at a time.

By default, the Release version is built. In order to build with debug symbols, pass -DCMAKE_BUILD_TYPE=Debug to your cmake command.

By default, tests and examples are built. In order to skip building those, pass -DBUILD_TESTING=OFF or -DBUILD_EXAMPLES to your cmake command.

Linking to your project

The install will contain header files and libraries in the path set with -DCMAKE_INSTALL_PREFIX.

CMake

If your project is using CMake for its build, one can conveniently use our provided Config.cmake package which is installed alongside the library.

First set the following environment hint if openPMD-api was not installed in a system path:

# optional: only needed if installed outside of system paths
export CMAKE_PREFIX_PATH=$HOME/somepath:$CMAKE_PREFIX_PATH

Use the following lines in your project's CMakeLists.txt:

# supports:                       COMPONENTS MPI NOMPI JSON HDF5 ADIOS1 ADIOS2
find_package(openPMD 0.1.0 CONFIG)

if(openPMD_FOUND)
    target_link_libraries(YourTarget PRIVATE openPMD::openPMD)
endif()

Alternatively, add the openPMD-api repository source directly to your project and use it via:

add_subdirectory("path/to/source/of/openPMD-api")

target_link_libraries(YourTarget PRIVATE openPMD::openPMD)

openpmd-api's People

Contributors

ax3l avatar c0nsultant avatar franzpoeschel avatar cfgrote avatar mpark avatar philsquared avatar stanczakdominik avatar nlohmann avatar wjakob avatar amundson avatar psychocoderhpc avatar sbastrakov avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.