Git Product home page Git Product logo

docker-config's Introduction

Flow123d · Jenkins release Jenkins debug Coveralls master Docker hub CI-HPC

Processes in Fractured Media

Flow123d is a simulator of underground water flow, transport, and mechanical processes in fractured porous media. Novelty of this software is support of computations on complex meshes consisting of simplicial elements of different dimensions. Therefore we can combine continuum models and discrete fracture network models. For more information see the project pages: flow123d.github.io.

Getting started

Please refer to a User Guide and Input Reference manual available at our official website where there is a entire section dedicated to this topic. You can find step-by-step tutorial explaining geometries, yaml input files and more. Below you can see a result from the tutorial problem.

Installation

You have several options when it comes to installation. You can use our prebuilt Docker images or build Flow123d from the source (requires a moderate experience with Linux OS).

For detailed instructions, see the installation guide.

Developers

Build

Two step build:

     host> bin/fterm     # start the docker developing container
container> make all      # produce "build_tree/bin/flow123d"
container> bin/flow123d  # lunch the simulator using a simple wrapper script

That would start the debug docker image and build the debug version of the simulator. For the release (optimized) version run:

bin/fterm rel
make all

For details see installation guide or the manual.

Troubleshooting

  • When problem occurs during the compilation process it may be due to a leftover files in a build folder. Cleaning this directory can solve this issue. You can either remove build-<branch> folder (the folder is located one level above repository root) via make clean-all, which removes build folders and also remove any symlinks.
    To clean all the build folders manually run rm -rf ../build-* while in a repository root.
    Running rm -rf can quite easily cause a lot of damage, double check that you're in a correct folder.

  • The build tools may fail if the root path contains folders with spaces.

  • During an installation under Windows, some scenarios can cause problems. Please refer to an installation guide for a Docker Toolbox. You can also check out Troubleshooting page where the most common error are described and solved.

Building the reference manual

The reference manual can be built by while in docker container

make ref-doc

To copy out reference manual from docker use command docker cp.

Singularity

Singularity is a container system targeting HPC applications. Singularity containers can be created from the docker images and in contrast to the docker one is not allowed to modify system of running containers.

For large images may be necessary set temp directory.

export SINGULARITY_TMPDIR="/some_absolute_path/tmp"

Run simulator in singularity container

singularity exec docker://flow123d/3.1.0 flow123d simulation.yaml

Parallel run, mpiexec out of the image

module add mpich-3.0.2-gcc
mpiexec -host host1,host2 -np 4 singularity exec docker://flow123d/3.1.0 flow123d simulation.yaml

Build

     host> git clone https://github.com/flow123d/flow123d.git # clone flow123d repository
     host> singularity shell -B flow123d/:/flow123d docker://flow123d/flow-dev-gnu-rel:3.1.0 # starts developing container
container> cd /flow123d
container> make all # produce "build_tree/bin/flow123d"
container> bin/flow123d  # lunch the simulator

Docker images

Hosted on dockerhub under organisation flow123d.

production images

tagged by the release version (e.g. 3.1.0)

  • flow123-gnu : based on gnu libraries and tools
  • flow123-intel : based on intel libraries (intelmpi, mkl, intel compilers)

alfa images

built by CI, tagged by {branch}-{commit}

docker-config's People

Contributors

janhybs avatar jbrezmorf avatar paulie14 avatar radeksrb avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

docker-config's Issues

tests for correct mpi builds and usage

Add tests for MPI performace in order to test performance of resulting images.

Installation of some packages may cause installation of system mpi incompatible with our own MPICH build.

Flow123d installation

  • separate creation of the install packages based on the full name of the image passed as a parameter
  • automatically create install packages for ci-gnu images, store them on astra
  • make github release script for:
    • creating git tag
    • release build and tests
    • image in flow123d/flow123d repository, appropriate installs
    • publication of the release on github
    • (optional) publication on pages
  • rebuild install of 3.1.0 branch

Use VISIP for building images

  • actions for performing image builds
  • actions for building libraries
  • creating working directories
  • all can be break into small persistent pieces, in order to minimize unnecessary rebuilds
  • demonstrate powers against 'doit'
  • base of the first practical tutorial

Finalize gnu-intel changes

Fix in PR #11:

  • tag release image as "flow123d_{env}:{images_version}" instead of "{images_version}_{env}"
  • check passing of 'destination' variable to package/makefile, should be the destination with env suffix, e.g. publish_gnu

Fix after merge of #11:

  • review content of bin, remove unused scripts

  • try to reduce duplicities of gnu/intel dockerfiles/makefiles...

  • move to archive:

  • review README.md in root and sub folders

fterms inconsistency

fterm and other files in docker-config/package/project/src/linux/ which are used in release version differ from files in
flow123d/bin/

Moreover, released files do not work properly. For instance, ./bin/flow123d should start flow123d, but it just starts the docker container, ...

flow123d release 3.1.0

  • Review and update release process of flow123d use new install image of the same version as the image used for release build of flow123d.
  • Test that Jenkins build proceed and the packaging artefacts are published on Astra: https://flow.nti.tul.cz/docker-images/.
  • Test the Linux installation.

Started from Jenkins CI build pro the release/package Jenkins job, however the job just clone docker-config and starts package build configured in package folder.

add gdbserver

Add gdb server to flow-dev* images.
It is necessary for debugging from Eclipse IDE.

prompt regression

The problem with disappearing command in prompt was probably fixed in 69e5d04.
However problem reappeared, current (1343faa) entry point has wrong PS1.

Try also to shorten the path, as space for command is too small.

New scheme of docker images

Goal: Change repository structure and dockerhub/flow123d image repositories to be consistent and allow build and publication of various content based on various environment. Content can include: debug and release build prerequisites, flow123d install image, Geomop/VISIP install image. Environment is specification of compiler and basic libraries used in builds, in particular we consider GNU and Intel variants.

Structure similar to previous instances:

Images by content

image base-{env} - basic environment common to development/build images and release images
image base-build-{env} - build tools and libraries common to debug and release build imagesstuff
image libs-{kind}-{env} - image for the actual build of flow123d prerequisites
image flow-dev-{kind}-{env} - image with installed built prerequisites for flow123d, PUBLISHED, image versioning
image install-{env} - base image for release image of flow123d, PUBLISHED, image versioning
image flow123d-{env} - release image, PUBLISHED, flow123d versioning

Notes

{env} - gnu or intel ecosystem
{kind} - dbg/rel = debug or release prerequisites, with or without debug symbols and checks respectively
flow123d versioning - follows flow123d numbered releases
image versioning - new versions prepared in longer periods than flow123d releases, but versions are mildly related to the flow123d versions

Repository structure

makefile - All docker images with exception of flow123d release image are built by the main makefile. Main makefile has to have separated rules for all images.

dockerfiles - folder with subdirectories for individual images, usually containing just the Dockerfile. Names of subdirectories follow images names possibly with a single parametrized dockerfile for different {kind} or {env} variants.

package -folder containing configuration and scritps for preparation of flow123d packages including flow123d-{env} images.

benchmarks - folder with scripts and configuration of flow123d benchmarks, mainly running on the Charon cluster

server - dockerfiles for the Jenkins CI server

Intel images PETSc error

flow123d software does not work in flow123d-intel images. It is possible to call flow123d --version, but tutorial examples failed.

PETSc error - linked against both OpenMPI and MPICH based MPI libraries.

Possible solution:
There is a different intel configuration in flow123d repository: flow123d/config/config_intel_release.cmake branch RS_intel

Reduce code duplicities

  • find a way how to propagate library versions consistently to different docker files
  • try to reduce duplicity between gnu and intel at least merge same files,
  • make subdirs for different environments

Prebuild tasks

  • uncomment bash link (/base-gnu/DockerFile, base-intel/DockerFile, line 16)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.