Git Product home page Git Product logo

llnl / conduit Goto Github PK

View Code? Open in Web Editor NEW
199.0 22.0 62.0 41.93 MB

Simplified Data Exchange for HPC Simulations

Home Page: https://software.llnl.gov/conduit/

License: Other

Shell 0.13% CMake 1.91% Python 2.53% Fortran 3.58% C++ 82.95% C 8.44% HTML 0.05% Makefile 0.05% CSS 0.05% JavaScript 0.30% Dockerfile 0.02%
hpc scientific-computing cpp fortran python llnl json yaml hdf5 radiuss

conduit's Introduction

Conduit

Conduit: Simplified Data Exchange for HPC Simulations

Conduit is an open source project from Lawrence Livermore National Laboratory that provides an intuitive model for describing hierarchical scientific data in C++, C, Fortran, and Python. It is used for data coupling between packages in-core, serialization, and I/O tasks.

Travis CI Build Status Appveyor Build Status Coverage Status Static Analysis Status

Documentation

To get started building and using Conduit, check out the full documentation:

http://llnl-conduit.readthedocs.io/

Source Repo

Conduit's source is hosted on GitHub:

https://github.com/llnl/conduit

License

Conduit is released under a BSD-style license - for detailed license info, refer to:

https://llnl-conduit.readthedocs.io/en/latest/licenses.html

or the following files in the Conduit source tree:

Changelog

conduit's People

Contributors

aaroncblack avatar acbauer avatar adrienbernede avatar agcapps avatar ajkunen avatar andrewcorrigan avatar benson31 avatar bmhan12 avatar bradwhitlock avatar brugger1 avatar bryujin avatar cdl-ilight avatar charlesgueunet avatar christost avatar cyrush avatar gzagaris avatar haluskar avatar ianlee1521 avatar jameshcorbett avatar justinprivitera avatar kennyweiss avatar mathstuf avatar mclarsen avatar nicolemarsaglia avatar nselliott avatar publixsubfan avatar rpmcnally avatar ryujin1 avatar white238 avatar xjrc avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

conduit's Issues

Specifying Per-Material Fields Using Blueprint?

I'd like to use Blueprint to describe a mesh that has multiple materials in a single zone, where each material has its own value for a particular field (e.g. density). Does Blueprint have any mechanism for describing such a field? If not, what would be the best way to utilize Blueprint's existing schema to describe such a field? Thanks in advance!

replace relay mpi C99 allocs with c++ style allocs

    int intArray[2];
    MPI_Status status;

    int mpi_error = MPI_Recv(intArray, 2, MPI_INT, src, tag, comm, &status);
    CONDUIT_CHECK_MPI_ERROR(mpi_error);

    int schema_len = intArray[0];
    int data_len = intArray[1];

    char schema[schema_len + 1];
    char data[data_len + 1];

the above allocs for schema and data are working -- but they aren't typical C++ allocs. We should replace them

move fortran symbols into main conduit libs

right now we require an extra link for the fortran interfaces for each lib.

I think the symbols for the fortran interfaces can safely be put in the main libs.

If there is a need, we cold provide a way to discover if fortran support is enabled other than the existence of these libs. (The fortran modules files should fulfill this need)

remove Node::compact()

in favor of only compact_to()

This is to stay consistent with the types of operations we want to support as const

fetch_path_as_ptr arrays have wrong output size in in fortran

This pattern is wrong:

n = conduit_node_number_of_elements(cnode)
int32_c_ptr = c_conduit_node_fetch_path_as_int32_ptr(cnode,trim(path) // C_NULL_CHAR)
call c_f_pointer(int32_c_ptr, f_out, (/n/))

The size of f_out is set as the number of elements of the root, not the actually number of elements at the fetched path.

reconsider the names of Node::paths() and NodeIterator::path() methods

We are adding a path() function that returns a string with the full path to the given node

has_path and fetch path variants descend the tree, but, the new path() clashes a bit with a few other uses "path" in method names:

For example, in the Node and Schema classes:

  • the two paths() variants list the names of the node's direct children, not full paths to them.

In the NodeIterator class:

  • path() returns the name of the child, not the full path to the current child

I propose changing the path variants in the Node and Schema classes to

  • child_names() (or similar: names_of_children() , is closer in style to number_of_children())

And path() in NodeIterator to:

  • name()

It's an API breaking change, but I think 0.2.0 is the right time to do this.

remove monolithic web server "serve" call

in favor of setters on the web server class

we have quite a few options to configure before starring a relay web server, passing empty strings in the correct order will be error prone.

pull out options into setter methods (set_document_root(), etc), add add a simple serve() method.

required values are checked when serve() is called.

add filesystem utils related to dirs

conduit::utils::create_directory(const std::string &path, bool recursive=true);
conduit::utils::remove_directory(const std::string &path, bool recursive=true);

set() set_external(), =operator() overloads, etc are ambiguous on Windows (Visual Studio)

From Kathleen B.:
Wow, I have this in my test :
proc0["cycle"] = 120;
worked fine on linux, but Visual studio complains that it is ambiguous:
15> error C2593: 'operator =' is ambiguous
15> Node.h(923): could be 'conduit::Node &conduit::Node::operator =(conduit::float64)'
15> Node.h(922): or 'conduit::Node &conduit::Node::operator =(conduit::float32)'
15> Node.h(920): or 'conduit::Node &conduit::Node::operator =(conduit::uint64)'
15> Node.h(919): or 'conduit::Node &conduit::Node::operator =(conduit::uint32)'
15> Node.h(918): or 'conduit::Node &conduit::Node::operator =(conduit::uint16)'
15> Node.h(917): or 'conduit::Node &conduit::Node::operator =(conduit::uint8)'
15> Node.h(915): or 'conduit::Node &conduit::Node::operator =(conduit::int64)'
15> Node.h(914): or 'conduit::Node &conduit::Node::operator =(conduit::int32)'
15> Node.h(913): or 'conduit::Node &conduit::Node::operator =(conduit::int16)'
15> Node.h(912): or 'conduit::Node &conduit::Node::operator =(conduit::int8)'
15> d): or 'conduit::Node &conduit::Node::operator =(conduit::bool8)'
15> while trying to match the argument list '(conduit::Node, int)'

relay hdf5: support partial read of h5dset using conduit::DataType

hdf5_read(hid_t h5_id, const DataType &dt, Node &out)

this allows you to fetch using an offset, stride, etc and fetch as a different type than the hdf5 dataset

(for example, you can fetch an array of doubles from an offset of hdf5 dset that was created as bytes)

blueprint for nd iteration

for better compatibility with Sidre N-D Fortran array support and python ndarrays

start with the contiguous case:

+shape (list of ints)
+values (actually 1-d array with values)

problems with gfortran module creation and "make -j"

It seems the compiler is undermining threaded builds:

Fatal Error: Can't rename module file '../../../fortran/f_conduit_node.mod0' to '../../../fortran/f_conduit_node.mod': No such file or directory
make[2]: *** [tests/conduit/fortran/CMakeFiles/f_conduit_node_obj.dir/f_conduit_node_obj.f.o] Error 1
make[1]: *** [tests/conduit/fortran/CMakeFiles/f_conduit_node_obj.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 76%] /private/var/folders/wj/jjhjt_0d3kg1vmk56d5bq_38001kr8/T/spack-stage/spack-stage-J_e5zR/conduit/src/tests/conduit/fortran/f_conduit_node.f:411.6:

use f_conduit_node
1
Fatal Error: Can't open module file 'f_conduit_node.mod' for reading at (1): No such file or directory
make[2]: *** [tests/conduit/fortran/CMakeFiles/f_conduit_node.dir/f_conduit_node.f.o] Error 1

http://stackoverflow.com/questions/29728867/cmake-parallel-build-for-fortran-90-with-modules

consider void_array

A void_array (a DataArray<void*> instance) would be useful for the case were you want to generically stride though and access the bytes for any array w/o type checking.

The void_array will still have access to the full data type spec, so the parts of the DataArray inteface related to set, to_json, etc helpers could still work fine, but we need to look if it has implications on plumbing of the template use.

changes to enable read-the-docs

breathe (doxygen bridge) integration is the only thing that requires us to use a special configure step for the config.py passed to sphinx. Since that step leverages our CMake build system, it makes using read-the-docs out of reach.

We aren't really leveraging breathe -- it's work considering simplifying to a static config.py file

reenable civet and bb64 install on osx and linux

We shouldn't need to link these in client code b/c these are listed PRIVATE dependencies and statically linked. However when conduit is included by another CMake build system via its exported cmake targets these become link dependencies.

The easiest way to resolve this is to simply install them again.

add Schema::child_name()

for the object case,

add:
std::string child_name(index_t idx) const;

to complement:
index_t child_index(const std::string &path) const;

source file renames

ex:

change:
DataType.hpp
to
conduit_data_type.hpp

This is out of an abundance of caution to avoid conflicts -- since client codes need to "include/conduit" to their include paths

support set_external(const Node &data)?

set_external(Node &) effectively allows you to create a shallow copy the passed node.

Since it does not modify the structure of the passed Node, I think we should also support a variant that allows the passed node to be const.

const access strategy for a Node's children

conduit::Node provides const access to data, but does not provide const to traverse existing children.

Given the semantics of fetch and the []operator -- the best solution seems like a const variants of

Node* Node::fetch_ptr() and Node* Node::child_ptr()

We should also provide a const iterator.

see if we can support modifying the PYTHONPATH env var from cmake.

two use cases:

  1. if they can set additional python paths in a host-config this could help folks configure python packages (eg: numpy) without env vars & we could do this per platform in an centralized way.

  2. if we can figure out the best way to support PYTHONPATH env var for custom commands, this could also help us avoid the python module warning "you are installing your module to a place that isn't in your sys.path()" that shows up when we build the conduit python module

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.