easybuilders / easybuild-tutorial Goto Github PK

Don't just say GCC is a gateway module, show it via module spider HDF5 first.

During the tutorial on June 23rd 2020 we ran into permission problems when trying to create files in the Docker container, because we used a directory that was created outside of the container as home directory in the container.

The problem was eventually easily fixed with the chmod 1777 command that was added in c24696f by @ocaisa.

@ChristianKniep @akesandgren Any more details on this, and suggestions for a proper long-term fix?

Should we just chmod a+w the directory after creating it, or is there a better way to ensure this always works?

Another problem with mounting a directory in the container is that the pre-downloaded sources that are pre-fetched into easybuild/sources in the home directory in the container image are no longer available (which could have prevented some of the SourceForge download trouble we hit).

• as an "appetizer", let people install something simple at the end of "Practical information" ?
• mention --dump-env-script in "Inspecting the build directory" ?
• mention how to handle multiple architectures in "Configuration" ?
• mention --subdir-user-modules when talking about HMNS and "Stacking software" ?

At the end of the base Dockerfile the pip installation is removed as EB installed itself. To get rid of the pip we can create a clean final step that only used the eb-installed version.

"Exercise 4.5 - Using installed software" mismatch between assignment and solution on the filename. mytestfile.hdf5 vs empty.hdf5

Checklist:

Day 1: Introduction to EasyBuild

• Practical Info
• Introduction: #135
• EasyBuild Terminology: #136
• Installation: #138
• Configuration
• Basic Usage of EasyBuild
• Writing Easyconfigs
• Troubleshooting
• Module Naming Schemes

Day 2: EasyBuild (advanced) + EESSI

EasyBuild (advanced)

• Contributing to EasyBuild]
• Customizing EasyBuild Using Hooks
• Implementing Easyblocks
• Submitting Installations as Slurm Jobs
• Using EasyBuild as a Python Library

Introduction to EESSI

• What is EESSI?
• High-level design of EESSI
• Current status of the project
• Getting access to EESSI
• Using EESSI
• Use Cases for EESSI
• The EESSI Community

In the eb-tutorial "Sanity check" section:
"Let us now retry, but use --module-only rather than redoing the whole installation."

This may warrant a more detailed explanation and reminding the reader what actually happened.
I.e. that the installation was actually done, and clarifying that --module-only runs the sanity check and creates the module.

This is still fairly early on...

eb SciPy-bundle-2020.11-foss-2020b.eb HDF5-1.10.7-gompi-2020b.eb

Ideally this goes into /easybuild/{software,modules/all}, or is symlinked there.

In the bullet point
"PythonBundle: a customized version of the Bundle"
"Bundle" is not properly defined yet as far as I've seen.
It should perhaps be added here and a good ref is Autotools

As discussed in #8: we might want to strip the base from any premature installations. In case EB base is used where no RDMA is to be seen...

Add an exercise to configure EasyBuild the way that is recommended for the remainder of the tutorial ($HOME/easybuild as prefix, /tmp/$USER as buildpath), and install bzip2-1.0.6.eb by means of test.

First bullet point under "Some things worth pointing out here" is missing the end ")"

"Length of module names":
what's kind of Pokémon is a "gompi" -> what kind of Pokémon is a "gompi"

No "'s" there

Text for installing eb-tutorial 1.1.0 says:

The sources are available via:

https://github.com/easybuilders/easybuild-tutorial/raw/master/docs/files/py-eb-tutorial-1.1.0.tar.gz

But the original is eb-tutorial-1.0.0.tar.gz, so py-eb-tutorial belongs to something else.

Just above "Required configure option":
See the output of "eb --avail-easyconfig-template"

According to eb --help it is "--avail-easyconfig-templates"

And just below "Required configure option":
"With sources defined, we can try and again." -> "With sources defined, we can try again."

There's now a container image for both haswell and ivybridge available, the latter should be compatible with more systems.

@kelseymh has tested this image on his old laptop, and it seems to work well

Whichever image we go with should be tagged as easybuilders/tutorial:isc20, and the "Practical information" page should be updated accordingly.

In the eb-tutorial "Easyblock" section:

The "easyblock =" line is usually at the top of the easyconfig file, but strictly speaking the order of the parameter definitions doesn't matter (unless one is defined the terms of another one).

Should be "(unless one is defined in terms of another one):"

Under "Using a pull request" 2nd paragraph:
"but you can specify only particular ones to use as well." -> "but you can specify particular ones to use as well." or "but you can specify which ones to use as well."

Just a couple of sentences, show what foss and intel are made of, point to https://easybuild.readthedocs.io/en/latest/Common-toolchains.html for more info

I.e. it doesn't work in the container setup.

eb HDF5-1.10.6-gompi-2020a.eb --robot --module-only
== temporary log file in case of crash /tmp/eb-aa7dv072/easybuild-yrxjqo2e.log
== found valid index for /scratch/ake/isc20/.local/easybuild/easyconfigs, so using it...
== resolving dependencies ...
== processing EasyBuild easyconfig /scratch/ake/isc20/.local/easybuild/easyconfigs/m/M4/M4-1.4.18.eb
== building and installing Core/M4/1.4.18...
  >> installation prefix: /easybuild/software/M4/1.4.18
== FAILED: Installation ended unsuccessfully (build directory: /scratch/ake/isc20/easybuild/build/M4/1.4.18/system-system): build failed (first 300 chars): Failed to create lock /easybuild/software/.locks/_easybuild_software_M4_1.4.18.lock: [Errno 30] Read-only file system: /easybuild/software/.locks/_easybuild_software_M4_1.4.18.lock (took 0 sec)

It fails first on the lock dir and once that has been redirected it fails to write the build log into the software dir
Failed to write to /easybuild/software/M4/1.4.18/easybuild/Core-M4-1.4.18-easybuild-devel:

• cover installation with --user and --prefix $HOME separately, so commands can just be copy-paste
• recommend using --prefix over --user
• pip3 install in prepared environment since pip is not installed, which also requires setting EB_PYTHON=python3
• using an EasyBuild other than the one included in /easybuild/modules is required for the HMNS part

At the moment it is very hard not to make a complete copy of older material so that you can create a new tutorial (since you will likely want to do things like shuffle things around and remove or add content).

With https://github.com/mondeja/mkdocs-include-markdown-plugin we could separate out the main content from the links that we provide at the bottom of each page. This gives you the freedom to restructure things while also allowing you to edit existing content (by using symlinks) rather than making entirely new files (which are a PITA to review in a PR as a result). When we archive a lesson we can break the links.

In the solution where it says: "We don't have this GCC version installed, but we do have GCC 9.3.0" a hint to module avail GCC is in place. It's still early and repetition is good