parmentelat / apssh Goto Github PK

when targetting several boxes, if one or more of them are not reachable, there is no visible trace of that
for each of these, we should issue one line of the form

hostname: could not connect (reason)

I tried to run uptime in one host:

apssh -t 'testserver.org' uptime

But it failed with this error message:

Unexpected exception in create_connection Passphrase must be specified to import encrypted private keys
[email protected][22]:Connection failed:('UNHANDLED', KeyImportError('Passphrase must be specified to import encrypted private keys',))

My private SSH keys are already loaded in the SSH agent. ssh will work if apssh have run the command "ssh inf-p-trh001.mdc.gameloft.org". How can I make apssh run ssh without any check, to avoid the exception below?

Thanks.

Following up on a side-track of issue #2.

@Asher256

At this point I have no plan in parsing ssh’s config file because it would

• be an awful lot of work, and
• there is a lot of ssh config details that just don’t make sense to us

What I had in mind though, was the ability for users to define their own target files in, say, ~/.apssh

So you would for example create a file

cat ~/.apssh/desktops
[email protected]
[email protected]

And from then on you could just run
apssh -t desktops blabla

As far as I am concerned, I really need a way to create aliases for groups of nodes, after all that is what apssh is all about in the first place

Implementation would be straightforward, I would primarily just need to add this search location when resolving targets.

It might also make sense to allow a target file to contain target names that are not hostnames, but in turn target files

Would this kind of things be helpful in your case ?

When connecting to 2 nodes behind a gateway, apssh first create 2 connection to the gateway before connecting to the initial target. Which means that we have a SSH connection to the gateway per node behind it. These connections remain until the end of the python script instead of being kill when we close the 2nd hop distant proxy (which in nepi-ng is done by calling manually shudown() on the main scheduler).

Cause :

The _close_ssh() method only call the close() method of its connection and never call the one of its gateway.

Moreover, the _connect_direct() function works like :

self.conn, _ = [...](asyncssh.create_connection([...]),[...])

This mean that each time a node will try to connect its gateway, the parameter self.conn will be overwritten with a new connection.

Possible fix :

We cannot just call self.gateway.close() since the stored connection is maybe not the good one.

We can store all gateway connections but we would need a way to pair them with the second hop that has created it.

Or we can simply make the _connect_direct() return its newly created connection and store it in the second hop node. Then when we want to close the connection on the said node we just have to also close this stored gateway connection : example.

when one or more nodes are offline and do not answer at all, apssh displays a line like

distrait.pl.sophia.inria.fr: apssh WARNING - no result !
eagulls.pl.sophia.inria.fr: apssh WARNING - no result !

however this sometimes refers to nodes that are online !

similar to #20, we need more specials

• {linenl} which contains the trailing newline as produced by the remote
• {line} without the newline
• {nl} to add a newline

apssh does not work because of an exception raised by os.getlogin():

$ apssh
Traceback (most recent call last):
  File "/home/asher256/.local/bin/apssh", line 3, in <module>
    from apssh.apssh import Apssh
  File "/home/asher256/.local/lib/python3.5/site-packages/apssh/apssh.py", line 13, in <module>
    from .config import default_time_name, default_timeout, default_username, default_private_key, default_remote_workdir
  File "/home/asher256/.local/lib/python3.5/site-packages/apssh/config.py", line 5, in <module>
    default_username =              os.getlogin()
FileNotFoundError: [Errno 2] No such file or directory

According to http://stackoverflow.com/questions/4399617/python-os-getlogin-problem :

From the os.getlogin() docs: "Returns the user logged in to the controlling terminal of the process." Your script does not have a controlling terminal when run from cron. The docs go on to suggest: "For most purposes, it is more useful to use the environment variable LOGNAME to find out who the user is, or pwd.getpwuid(os.getuid())[0] to get the login name of the currently effective user id."

I suggest you to replace os.getlogin with:

import pwd
import os

getlogin = lambda: pwd.getpwuid(os.getuid())[0]
default_username = getlogin()

I my case, it solved the issue.

When using apssh on Mac, this snipet of code :

for i in range(1000):
    node = SshNode(hostname='127.0.0.1') 

Generate an error :

Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File "/usr/local/lib/python3.6/site-packages/apssh/nodes.py", line 165, in __init__
  File "/usr/local/lib/python3.6/site-packages/apssh/keys.py", line 157, in load_private_keys
  File "/usr/local/lib/python3.6/site-packages/apssh/keys.py", line 157, in <listcomp>
  File "/usr/local/lib/python3.6/site-packages/apssh/keys.py", line 35, in import_private_key
  File "/usr/local/Cellar/python/3.6.4_4/Frameworks/Python.framework/Versions/3.6/lib/python3.6/pathlib.py", line 1161, in open
  File "/usr/local/Cellar/python/3.6.4_4/Frameworks/Python.framework/Versions/3.6/lib/python3.6/pathlib.py", line 1015, in _opener
  File "/usr/local/Cellar/python/3.6.4_4/Frameworks/Python.framework/Versions/3.6/lib/python3.6/pathlib.py", line 387, in wrapped
OSError: [Errno 24] Too many open files: '/Users/[****]/.ssh/id_rsa'

Cause

There is a file descriptor that is not closed on the creation of a SshNode as we can see with :

[...]$ lsof -p [PythonPid]
[...]
Python  42078 [...]    4u    unix 0x4537b16394f7d0a3        0t0            ->0x4537b16394f7c67b ##asyncio loop
Python  42078 [...]    5u    unix 0x4537b16394f7c67b        0t0            ->0x4537b16394f7d0a3 ##asyncio loop
Python  42078 [...]    6u    unix 0x4537b16394f7b86b        0t0            ->0x4537b16394f7a8cb ##Remnant fd
Python  42078 [...]    7u    unix 0x4537b16394f7cbf3        0t0            ->0x4537b16394f7d61b ##Remnant fd
Python  42078 [...]    8u    unix 0x4537b16394f7cfdb        0t0            ->0x4537b16394f7c80b ##Remnant fd
Python  42078 [...]    9u    unix 0x4537b16394f7b613        0t0            ->0x4537b16394f7a993 ##Remnant fd

This come from the SSHAgentClient class from asyncssh. More particulary from the getkeys() method. In apssh code it is the await agent_client.get_keys() from the method load_agent_keys(...) in the keys.py file.

It is important to note that this remnant file descriptor happens only on mac.

Possible fix

Closing the SSHAgentClient seems to do the trick (example) but I will ask on asyncssh if it is an expected behavior.

Indeed, we can reproduce this simply by using asyncssh :

loop = asyncio.get_event_loop()
agent_path = os.environ.get('SSH_AUTH_SOCK', None)

async def aget_from_agent(loop,agent_path):
    clientAgent = asyncssh.SSHAgentClient(loop, agent_path)
    keys = await clientAgent.get_keys()

for i in range(1000):
    loop.run_until_complete(aget_from_agent(loop, agent_path))

When apssh crashes or is killed, it does not kill remote processes.

A common trick to ensure this is to force pseudo-TTY allocation, with ssh -tt. But it might have side-effects...

much like hostname has a -s option, it would be nice to be able to create a variant of ColonFormatter that only outputs the short hostname

we could even have

• ColonFormatter: to use short names
• LongColonFormatter: to use the hostname as-is, i.e. fqdn if that's what was provided in the targets

we need to have their stdout and stderr show up live instead, i.e. immediately when they are issued, not at the end of the process

for a more accurate implementation of the --window feature, does it make sense to use a nested scheduler now that asynciojobs supports it ?

SshNode has a keep_connection flag, that seems to make sense only in the context of asynciojobs 0.x, where running a scheduler automatically invoked a call to co_shutdown()

Now that this call is made explicit in asynciojobs 1.x, it seems like keep_connection can be removed
if not, at least we need a doc review about this topic

as of 0.20 one can target 4 nodes by doing

$ apssh -t "a b c d" some command

it would make sense to support also

$ apssh -t a,b,c,d some command

which would lighten the burden of using quotes

and a mix could be used as well

$ apssh -t "a,b c,d" some command

or

$ apssh -t "a b,c d" some command

the formatting specials read @host@ and similars, but with appull we have introduced a more usual syntax using {host} instead

so let us use this syntax everywhere as it is more compliant with

• f-strings,
• templating engines (jinja uses {{}})
• and even javascript `` which use ${}

in all cases there are {} involved so it is a better choice than @@

not --dry-run, but more like --list-targets
that would stop after resolving targets

it would be cool to be able to define a typical nepi-ng script using yaml for the scheduler structure
even if the various command line options would still be implemented in a Python script

in particular, this script here
https://github.com/sopnode/oai5g-rfsim
would be our first target for using this feature if/when it is available

I'm thinking it would be cool to be able to run parallel file copies; something like along the following lines.

this is obsolete, see new version of this feature in #19

how to denote remote files

the syntax uses {h}: as a way to describe remote files - actually anything matching {\w*}: including {}: will do the trick

pushing

copy the same local file over to a variety of hosts

• copy one local file on all remote home dirs

appush -t the_targets local_file {h}: 

• likewise but with several local files, and store remote copies in /etc/

appush -t the_targets local_file1 local_file2 {remote}:/etc

pulling

the other way around; the destination MUST BE a folder (so that one can store several files in there)

• copy all the /etc/fedora-release files, into subdir the-releases, named as <hostname>/fedora-release

appull -t targets {HOST}:/etc/fedora-release the-releases/

• same but fetch several files in one go

appull -t targets {HOST}:/etc/fedora-release {HOST}:/etc/fedora-release the-releases

• if the output directory is missing, it's like using apssh with the -d option

appull -t targets {HOST}:/etc/fedora-release {HOST}:/etc/fedora-release the-releases
2022-11-04@16:52

we currently have

• {host} for the fqdn (i.e. the hostname as provided)
• and {host-short} which is stripped of the domain part

let us replace this with

• {fqdn}
• {host}
  respectively

it's all in the title, we need

• more context on where
• and more info on why
  errors are found in yaml

Hi,
We are using below command to get the original login user. In some cases folks sudo as different user and run commands. In some cases folks use switch to different user (su - <>) and then run command. In all cases we would like to get original user who logged into the terminal to run the command. We are using below command to get the same.

user=os.getlogin()

We got three RHEL linux systems with Python 3 and strangely on two of them this command works fine. However only on third system it is failing with below error

OSError: [Errno 6] No such device or address

Not sure if it is some o/s env setup which is causing this command to fail. Any pointers to run this command will be of great help so that we can pin point the issue.

Please note on faiing system if I do python and then run the command it works fine without any issue. However if it is triggered through our Scheduler (Control M) it is failing with above message.

Give below is the command how Scheduler triggers the job.

/bin/su - edwadm -c /bin/sh -x <>

Thanks
Manick.

in the context of our demo script
https://github.com/fit-r2lab/r2lab-demos/tree/master/my-first-nepi-ng-script
we need (and this makes sense in general of course)

1. the ability to create RunScript instances that have their node set as a LocalNode()
2. the ability to create LocalNode instances through YAML

let me show a commad example

⋊> ~ pssh -l root -h hosts hostname
[1] 12:19:57 [SUCCESS] 192.168.14.14
[2] 12:19:57 [SUCCESS] 192.168.14.12
[3] 12:19:57 [SUCCESS] 192.168.14.19
[4] 12:19:57 [SUCCESS] 192.168.14.18
[5] 12:19:57 [SUCCESS] 192.168.14.22
[6] 12:19:57 [SUCCESS] 192.168.14.16
[7] 12:19:57 [SUCCESS] 192.168.14.24
[8] 12:19:57 [SUCCESS] 192.168.14.11
[9] 12:19:57 [SUCCESS] 192.168.14.25
[10] 12:19:57 [SUCCESS] 192.168.14.23
[11] 12:19:57 [SUCCESS] 192.168.14.28
[12] 12:19:57 [SUCCESS] 192.168.14.27
[13] 12:19:59 [FAILURE] 192.168.14.10 Exited with error code 255
[14] 12:19:59 [FAILURE] 192.168.14.21 Exited with error code 255
[15] 12:19:59 [FAILURE] 192.168.14.29 Exited with error code 255

⋊> ~ apssh -l root -t hosts hostname 
192.168.14.14:test.local 
192.168.14.12:test.local
192.168.14.19:test.local
192.168.14.24:test.local
192.168.14.16:test.local
192.168.14.22:test.local
192.168.14.18:test.local
192.168.14.11:test.local
192.168.14.25:test.local
192.168.14.23:test.local
192.168.14.28:test.local
192.168.14.27:test.local

pssh's output will show three down status hosts(192.168.14.10 , 192.168.14.21, 192.168.14.29) with failure, but apssh's output show nothing about them's status.

as of 0.20, we can

• have all the hosts be reached directly
• or have them all got through a gateway

there are times where one needs to either

• have only one node go through a gateway
• or have most nodes go through a gateway, while some others go through another (or none, for that matter)

proposal

• apssh -t "a b c" -g gate means gate→a gate→b gate→c
• apssh -t "a g2→b c" -g gate means gate→a g2→b gate→c
• ``apssh -t "a →b c" -g gatemeansgate→a b gate→c`

where either a, b, c or gate or g2 may all contain a username as in e.g.

apssh -t "u1@a u2@g2→b g3→u4@c u5@→d" -g ug@gate -l login means

• ug@gate→u1@a
• u2@g2→login@b
• ug@g3→u4@c
• u5@gate→login@d

also all occurrences of → (1 char) can be replaced with -> (2 chars)