Issue with environments about longbow HOT 3 CLOSED

It has been reported that referring to the /etc/profile file does not work on at least one machine. This highlights the difference in how these environments are setup. There appears to be some range in how sys admins are linking various files and under what conditions.

This means that on the original comment under this thread only points 1 and 3 remain valid. These are copied below along with a new idea to investigate.

1. Pass information to the user through documentation, informing them to how to add the relevant line to their .bashrc. Have to be careful here, not all Linux distros, and not all sys admins do things the same. There is a bit of variance in how things are done so any solution put on the user needs proper instructions so they can deal with it.
2. Find out if there is a proper way to get a valid login shell when using python subprocess.
3. A better way could be as part of the tests on the machine, to include handling for when Linux modules can't be found. Then in this case, tests on the various files could reveal how the environment is loaded and then amend the ./bashrc

from longbow.

Newer generation of Hartree machines have a slightly different environment to machines that Longbow is currently used to operate with. It appears from initial discussions that there is a module hierarchy that has to be loaded to get access to other more system specific modules.

We should look at supporting this, it could be that supporting this properly will actually resolve the initial situation in this ticket.

from longbow.

Changing the connection tests to incorporate testing of the environment could be the best way. Here is a script that can identify this problem and "fix" it. In Longbow the fix could be done by appending a flag to toggle the sourcing command to on or off. When on this command is pre-pended to commands delivered over SSH.

#!/usr/bin/env python

import subprocess

machines = {
    "scarf": ["ssh", "-t", "scarf"],
    "archer": ["ssh", "-t", "archer-c01"],
    "iden": ["ssh", "-t", "hartree-iden"]
}

for machine in machines:

    print "Testing for environment on " + machine + "..."

    cmd = list(machines[machine])
    cmd.append("module avail")

    handle = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)

    stdout, stderr = handle.communicate()

    if handle.returncode != 0:

        if "bash: module: command not found" in stdout or "bash: module: command not found" in stderr:

            print "Environment is borked."

            print "attempt fix"

            cmd = list(machines[machine])
            cmd.append("source /etc/profile; module avail")

            handle = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)

            stdout, stderr = handle.communicate()

            if handle.returncode == 0:

                print "env is now ok"

            else:

                print "still busted."

    else:

        print "Env is ok"

from longbow.