Git Product home page Git Product logo

atlas-containers's Introduction

atlas-containers

Docker to wsl2 for general purpose use with the ATLAS software stack. This wsl2 distro uses cvmfs.

Introduction

Contains a Dockerfile based on AlmaLinux 9 that installs all the necessary libraries for running the ATLAS software stack. This includes cvmfs. This container is used to create a wsl2 distro.

Who is this for:

  • Need a general container to run ATLAS software on Windows
  • Access to a high-speed, stable, internet connection: this container uses cvmfs for all ATLAS software. Quite a bit can be downloaded (e.g. GB's worth). While it is cached by the cvmfs software, who knows when the cache will expire and you'll need to re-download!

Prerequisites

You will need to have wsl2 and docker installed on your windows machine.

Usage

Everything is orchestrated via the script Install-ATLASwsl2.ps1:

PS> .\Install-ATLASWsl2.ps1 -?    
Install-ATLASWsl2.ps1 [-containerName] <string> [[-os] <string>] [<CommonParameters>]

An example:

PS C:\Users\gordo\Code\atlas\atlas-containers> .\Install-ATLASWsl2.ps1 atlas_centos7 centos7
[+] Building 257.1s (21/21) FINISHED                                                                                                       docker:default
 => [internal] load .dockerignore                                                                                                                    0.0s
 => => transferring context: 2B                                                                                                                      0.0s 
 => [internal] load build definition from Dockerfile                                                                                                 0.1s 
 => => transferring dockerfile: 1.84kB                                                                                                               0.0s 
 => [internal] load metadata for docker.io/library/centos:7                                                                                          0.5s 
 => [ 1/16] FROM docker.io/library/centos:7@sha256:be65f488b7764ad3638f236b7b515b3678369a5124c47b8d32916d6487418ea4                                  0.0s
 => [internal] load build context                                                                                                                    0.0s 
 => => transferring context: 211B                                                                                                                    0.0s 
 => CACHED [ 2/16] RUN yum -y install sudo passwd yum-utils krb5-workstation openssl man-db deltarpm xdg-utils libXft libSM patch cmake make g++ uu  0.0s 
 => CACHED [ 3/16] RUN yum -y install epel-release python2-pip && yum clean all                                                                      0.0s 
 => [ 4/16] RUN yum-config-manager --add-repo https://download.opensuse.org/repositories/home:/wslutilities/RHEL_7/home:wslutilities.repo && yum i  26.8s 
 => [ 5/16] RUN yum install -y https://ecsft.cern.ch/dist/cvmfs/cvmfs-release/cvmfs-release-latest.noarch.rpm && yum install -y cvmfs && yum clean  44.5s
 => [ 6/16] RUN cvmfs_config setup                                                                                                                   0.6s 
 => [ 7/16] COPY default.local /etc/cvmfs/default.local                                                                                              0.0s 
 => [ 8/16] RUN yum install -y https://linuxsoft.cern.ch/wlcg/centos7/x86_64/wlcg-repo-1.0.0-1.el7.noarch.rpm && yum clean all                       3.1s 
 => [ 9/16] RUN yum install -y --skip-broken HEP_OSlibs glibc-langpack && yum clean all                                                            123.4s 
 => [10/16] RUN mkdir /etc/atlas-cern                                                                                                                0.7s 
 => [11/16] COPY startup-atlas-sudo.sh /etc/atlas-cern/startup-atlas-sudo.sh                                                                         0.0s 
 => [12/16] COPY startup-atlas.sh /etc/profile.d                                                                                                     0.0s 
 => [13/16] COPY config-user.sh /etc/atlas-cern/config-user.sh                                                                                       0.0s 
 => [14/16] COPY sudoers-atlas-setup /etc/sudoers.d/sudoers-atlas-setup                                                                              0.0s 
 => [15/16] COPY wsl.conf /etc/wsl.conf                                                                                                              0.0s 
 => [16/16] RUN yum update -y && yum clean all && yum update -y                                                                                     53.0s 
 => exporting to image                                                                                                                               4.1s 
 => => exporting layers                                                                                                                              4.1s 
 => => writing image sha256:94a23c9c47e153a08d9d8ca195672717dbaceab10a2b837e60983c5ef39b45f2                                                         0.0s 
 => => naming to docker.io/library/atlas_centos7:latest                                                                                              0.0s 

What's Next?
  View a summary of image vulnerabilities and recommendations โ†’ docker scout quickview
atlas_centos7
atlas_centos7
Import in progress, this may take a few minutes.   
The operation completed successfully.
Enter username: gwatts
You will be prompted to enter a password for gwatts.
Changing password for user gwatts.
New password: 
Retype new password:
passwd: all authentication tokens updated successfully.
Adding user gwatts to group wheel
The operation completed successfully. 
PS C:\Users\gordo\Code\atlas\atlas-containers> C:\Users\gordo\OneDrive\Documents2\WindowsPowerShell\Scripts\atlas-scripts\Copy-LinuxFiles.ps1 atlas_centos7
Installing Globus certificates. You'll be prompted for the pass phrase for your p12 file
and then a pass phrase for the userkey.pem file.
Enter Import Password:
MAC verified OK
Enter PEM pass phrase:
Verifying - Enter PEM pass phrase:
And again for the p12 file to generate the usercert.pem
Enter Import Password:
MAC verified OK

Tests

The following have been tested:

  • lsetup rucio and voms-proxy-init and rucio ping
    • centos7
    • AL9
  • lsetup root xxx and root and then TBrowser b - using the default recommended version of root when this was last checked!
    • centos7
      • The X Windows just works with no further config (note you must start current version of root with root --web=off for this to work, otherwise it tries to use the new interface).
      • The web browser seems version starts ok, though wslutils gives a syntax error (bug report).
    • AL9
      • Same caveats as above.
      • The browser window was slow to open - it took 15 seconds on AL9. Much faster on CENTOS7.
  • xclock works (after installing with yum install xorg-x11-apps (this is not normally installed with this configuration).
    • centos7
      • First you must run sudo yum install xclock
    • AL9
      • First you must run sudo dnf --enablerepo=crb install -y xorg-x11-apps
      • The web pages talking about this are incorrect. I cannot find xclock for AL9. However, given the X-Windows version of root TBrowser works, I think the x-windowing system, wslg, is working just fine.
  • kinit and ssh lxplus.cern.ch without password for ssh.
    • centos7
    • AL9
  • Build the DiVertAnalysisR21 image using the gitlab build script.
    • centos07
    • Not clear this is possible on AL9 since this is R21.

atlas-containers's People

Contributors

gordonwatts avatar

Stargazers

 avatar  avatar

Watchers

 avatar  avatar  avatar

Forkers

timadye

atlas-containers's Issues

Can't install HEP_OSlibs on AL9

This is what happens when we try to do an install:

[gwatts@SurfacePhoto atlas-containers]$ sudo yum install -y HEP_OSlibs
[sudo] password for gwatts:
Last metadata expiration check: 0:08:29 ago on Sun Nov  5 05:40:39 2023.
Error: 
 Problem: cannot install the best candidate for the job
  - nothing provides gdbm-devel(x86-64) needed by HEP_OSlibs-9.1.0-2.el9.x86_64
  - nothing provides libtirpc-devel(x86-64) needed by HEP_OSlibs-9.1.0-2.el9.x86_64
  - nothing provides libxkbcommon-x11-devel(x86-64) needed by HEP_OSlibs-9.1.0-2.el9.x86_64
  - nothing provides xcb-util-devel(x86-64) needed by HEP_OSlibs-9.1.0-2.el9.x86_64
  - nothing provides xcb-util-image-devel(x86-64) needed by HEP_OSlibs-9.1.0-2.el9.x86_64
  - nothing provides xcb-util-keysyms-devel(x86-64) needed by HEP_OSlibs-9.1.0-2.el9.x86_64
  - nothing provides xcb-util-renderutil-devel(x86-64) needed by HEP_OSlibs-9.1.0-2.el9.x86_64
  - nothing provides xcb-util-wm-devel(x86-64) needed by HEP_OSlibs-9.1.0-2.el9.x86_64
(try to add '--skip-broken' to skip uninstallable packages or '--nobest' to use not only best candidate packages)
[gwatts@SurfacePhoto atlas-containers]$ 

Feels like I'm missing some development yum repository.

missing rpm expat-devel

Building Athena (Athena,main,r2024-03-07 sparse build) in WSL distro atlas_al9 gives an error in the cmake step:

-- Could NOT find Geant4 (found version "10.6.3"), checked the following files:
    /cvmfs/atlas-nightlies.cern.ch/repo/sw/main_Athena_x86_64-el9-gcc13-opt/2024-03-07T2101/AthenaExternals/25.0.1/InstallArea/x86_64-el9-gcc13-opt/lib/Geant4-10.6.3/Geant4Config.cmake (version 10.6.3)
    Reason given by package: Geant4 could not be found because dependency EXPAT could not be found.

An easy fix was to install:

sudo yum install expat-devel

This could be an Athena problem, but this package is available on lxplus and the setupATLAS -c el9 container, so it is a reasonable thing to have available.

do `yum clean` in a better way

Aside: If you want to take advantage of yum clean cleaning up and removing packages in a container then you need to run it in the same RUN command as the installation step. Each RUN command generates a new zipfile layer to the container image so if you clean after the RUN command that installed things you only cleaned the layer above, and things are still in the final image just sitting around as extra storage bloat.

From Matthew F!!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.