hpcugent / lmod-ugent Goto Github PK

View Code? Open in Web Editor NEW

8.0 8.0 7.0 204 KB

spec files of Lmod for UGent-HPC

Lua 19.71% Shell 6.14% Python 12.10% HTML 62.05%

lmod-ugent's People

Contributors

Stargazers

Watchers

Forkers

wpoely86 boegel wdpypere hpcuantwerpen vub-hpc stdweird

lmod-ugent's Issues

[Lmod 7 update] use new support for localisation

Lmod 7.1.11 has proper support for localisation via the errWarnMsgHook, which now has to be used since LmodWarning and LmodError no longer call msgHook...

tab completion on 'ml use' doesn't work

This is the result of ml use /us<tab>:

$ ml use /user-bash: _filedir: command not found

I guess we're missing the bash-completion.noarch RPM?

Hide cluster modules with hook

Instead of patching Lmod to hide the cluster modules, you can also use a hook.

Untested code below:

local function visible_hook(modT)
    -- modT is a table with: fullName, sn, fn and isVisible
    -- The latter is a boolean to determine if a module is visible or not

    if modT.fullName:find("cluster/%.") then
        modT.isVisible = false
    end
end

hook.register("isVisibleHook", visible_hook)

provide /usr/bin/modulecmd if possible

is it possible to ship a (small) wrapper /usr/bin/modulecmd that is either compatible with env-modules? not sure what it entails

all python code needs proper shebang

*** ERROR: ambiguous python shebang in /usr/share/lmod/8.2.5/libexec/run_lmod_cache.py: #!/usr/bin/python -E. Change it to python3 (or python2) explicitly.
error: Bad exit status from /var/tmp/rpm-tmp.Imca5x (%install)

easiest fix is to fix it in the spec file during the %build phase

lmod_cache monitoring needs initialized file

The lmod_cache check will fail when not run as root, since it tries to write and chown a file under /var/cache. Workaround:

touch /var/cache/run_lmod_cache.nagios.json.gz
chown vsc40003:nrpe /var/cache/run_lmod_cache.nagios.json.gz

The script should check the existance of this file and give a clear error message if this is missing

Test SitePackage.lua

We should test everything in SitePackage.lua. If internal structures of Lmod change, this might break something on our end.

provide `environment(modules)`

# dnf install python3-sphinx
Last metadata expiration check: 0:07:19 ago on Sat 29 Aug 2020 10:36:32 PM CEST.
Error: 
 Problem: package python3-sphinx-1:1.7.6-1.el8.noarch requires environment(modules), but none of the providers can be installed
  - package Lmod-8.2.5-2.ug.el8.x86_64 conflicts with environment-modules provided by environment-modules-4.1.4-4.el8.x86_64
  - package Lmod-8.2.5-2.ug.el8.x86_64 conflicts with environment-modules provided by environment-modules-4.1.4-1.el8.x86_64
  - problem with installed package Lmod-8.2.5-2.ug.el8.x86_64
  - conflicting requests
  - package Lmod-8.2.7-1.el8.x86_64 is filtered out by exclude filtering

the Lmod rpm/spec should mention a 'Provides: environment(modules)' if it does provide environment-modules compatibilty

Extend docs

Add error:

$ ml load cluster
Lmod has detected the following error:  No default cluster can be loaded. You need to choose one: 'ml spider cluster' gives an overview 
While processing the following module(s):
    Module fullname          Module Filename
    ---------------          ---------------
    cluster/.defaultcluster  /etc/modulefiles/vsc/cluster/.defaultcluster.lua

Which means that /etc/bashrc was not sourced.

Add to part about collections:

Collections make it very easy to work with your own compiled software. First load all the modules that you need to compile your software. Then build it and test it. If everything works fine, save the current set of loaded modules to a collection. In every job script that uses your software, you simple have to restore the collection. No more meddling with lots of module loads.

Test restore/swapping between clusters of collections. Collection saved on one cluster should work on another if all modules are present.
Should allow to swap between clusters with same modulepath (golett <=> phanpy, delcatty <=> raichu)

@boegel said: Currently, restoring a collection on a cluster that has the same $MODULEPATH should already work, but restoring on a different cluster (e.g. from golett to delcatty or vice versa) results in the modules being loaded from the wrong location)

knows caveats

list of knows caveats with Lmod compared to environment modules setup:

module load cluster is broken, must use module load cluster/$VSC_DEFAULT_CLUSTER_MODULE
changes to $MODULEPATH must be done with module use
- if not, Lmod may spit out The environment MODULEPATH has been changed in unexpected ways

bugs/issues:

output of module list and module --terse avail doesn't go do stdout as desired
manual module use on /apps/gent/.../modules/all results in spider cache not being found
- /apps/gent is a symlink, which gets resolved on manual module use, leading to path mismatch (bug in Lmod?)

better error message when loading conflicting modules

Users now get to see something like this, which is confusing them:

$ ml Python/2.7.11-intel-2016a

$ ml R/3.3.1-intel-2016b
Lmod has detected the following error:  Your site prevents the automatic swapping of modules with same name. You must explicitly unload the loaded version of "intel" before you can load the new
one. Use swap (or an unload followed by a load) to do this:

   $ module swap intel  intel/2016b

Alternatively, you can set the environment variable LMOD_DISABLE_SAME_NAME_AUTOSWAP to "no" to re-enable same name autoswapping. 
While processing the following module(s):
    Module fullname      Module Filename
    ---------------      ---------------
    R/3.3.1-intel-2016b  /apps/gent/SL6/sandybridge/modules/all/R/3.3.1-intel-2016b

If you don't understand the warning or error, contact the helpdesk at [email protected]

First of all, the swapping suggestion is confusing since it won't get them out of the woods.

@wpoely86 Can we provide a more meaningful error message here via a hook?