sourcelibre / lunch Goto Github PK
View Code? Open in Web Editor NEWDistributed process launcher
License: Other
Distributed process launcher
License: Other
Let's port on Python 3.
We could very well use 2to3 https://docs.python.org/2/library/2to3.html
RIght now, we use Gtk+, and it doesn't easily work on macOS. (nor Windows) We should consider implementing another GUI with another toolkit that is more cross-platform.
the GUI should run on:
2010-08-17 13:02:59,505 commands ERROR [[AttributeError]]: Parsing a line from lunch-slave send_posture05_4: 80:d6:ba:15:44:07:c5:35:ec:3e:ea:58:8a:f0:a7:d0.
2010-08-17 13:02:59,506 commands ERROR [[AttributeError]]: Parsing a line from lunch-slave send_posture05_4: Please contact your system administrator.
2010-08-17 13:02:59,506 commands ERROR [[AttributeError]]: Parsing a line from lunch-slave send_posture05_4: Add correct host key in /home/posture/.ssh/known_hosts to get rid of this message.
2010-08-17 13:02:59,506 commands ERROR [[AttributeError]]: Parsing a line from lunch-slave send_posture05_4: Offending key in /home/posture/.ssh/known_hosts:14
2010-08-17 13:02:59,506 commands ERROR [[AttributeError]]: Parsing a line from lunch-slave send_posture05_4: RSA host key for 10.21.10.27 has changed and you have requested strict checking.
2010-08-17 13:02:59,507 commands ERROR [[AttributeError]]: Parsing a line from lunch-slave send_posture05_4: Host key verification failed.
2010-08-17 13:02:59,802 commands ERROR [[AttributeError]]: Parsing a line from lunch-slave send_posture05_4: @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
2010-08-17 13:02:59,803 commands ERROR [[AttributeError]]: Parsing a line from lunch-slave send_posture05_4: @ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @
2010-08-17 13:02:59,803 commands ERROR [[AttributeError]]: Parsing a line from lunch-slave send_posture05_4: @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
2010-08-17 13:02:59,803 commands ERROR [[AttributeError]]: Parsing a line from lunch-slave send_posture05_4: IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
2010-08-17 13:02:59,803 commands ERROR [[AttributeError]]: Parsing a line from lunch-slave send_posture05_4: Someone could be eavesdropping on you right now (man-in-the-middle attack)!
2010-08-17 13:02:59,804 commands ERROR [[AttributeError]]: Parsing a line from lunch-slave send_posture05_4: It is also possible that the RSA host key has just been changed.
2010-08-17 13:02:59,804 commands ERROR [[AttributeError]]: Parsing a line from lunch-slave send_posture05_4: The fingerprint for the RSA key sent by the remote host is
2010-08-17 13:02:59,804 commands ERROR [[AttributeError]]: Parsing a line from lunch-slave send_posture05_4: 80:d6:ba:15:44:07:c5:35:ec:3e:ea:58:8a:f0:a7:d0.
2010-08-17 13:02:59,805 commands ERROR [[AttributeError]]: Parsing a line from lunch-slave send_posture05_4: Please contact your system administrator.
2010-08-17 13:02:59,805 commands ERROR [[AttributeError]]: Parsing a line from lunch-slave send_posture05_4: Add correct host key in /home/posture/.ssh/known_hosts to get rid of this message.
2010-08-17 13:02:59,805 commands ERROR [[AttributeError]]: Parsing a line from lunch-slave send_posture05_4: Offending key in /home/posture/.ssh/known_hosts:14
2010-08-17 13:02:59,805 commands ERROR [[AttributeError]]: Parsing a line from lunch-slave send_posture05_4: RSA host key for 10.21.10.27 has changed and you have requested strict checking.
2010-08-17 13:02:59,806 commands ERROR [[AttributeError]]: Parsing a line from lunch-slave send_posture05_4: Host key verification failed.
We could create a new "state" for this.
Actually, it might be better to keep trying.
Hello,
Many thanks for your work towards pip install lunch https://pypi.org/project/lunch/.
I am using Mac OSx El Capitan. I am putting setting up Open TEE. One of the commands to run it lunch.
What I have done so far;
-installed pip
-ran $pip install lunch
-I got the error as attached herein.
Could you please advise on how to install it?
How to reproduce :
start a lunch with this file:
HOST = "brrr"
USER = "miville"
add_command("xeyes", env={"DISPLAY":":0.0"}, user=USER, host=HOST, title="remote_xeyes")
add_command("xlogo", env={"DISPLAY":":0.0"}, user=USER, host=HOST, title="remote_xlogo")
crash your computer.
The lunch-slave and xeyes stays alive on the remote host. (brrr)
What we could do to fix this :
Store the pid of the slave in /var/tmp/lunch/slave-default-1.pid
When launching a slave with the same identifier, read in that pidfile. If the process is alive, kill it.
try:
os.kill(pid, 0)
except OSError:
else:
os.kill(pid, 9) # SIGKILL? or SIGINT first?
Start the child as usual.
There is the PyPi page: https://pypi.python.org/pypi/lunch
There is the old wiki: http://code.sat.qc.ca/redmine/projects/lunch/wiki (See issue #13 )
There is the man page.
We should rotate the log files instead of creating a new one every several time interval.
Right now, the message is this one:
The SSH client asks for a passphrase to unlock your local private SSH key for which you have the corresponding public key on host 10.21.10.15. You should avoid this to be asked by providing that passphrase a first time using SSH by hand.
The line received from SSH is :
Enter passphrase for key '/home/posture/.ssh/id_rsa':
Can we provide better advices?
...to open a tail -F window.
Below is one of the possible examples. In this case, host RSA key has changed but there may be other related issues.
2011-12-01 10:28:53,104 commands ERROR AttributeError: Parsing a line from lunch-slave sat_video_recv_0: IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
2011-12-01 10:28:53,104 commands ERROR AttributeError: Parsing a line from lunch-slave sat_video_recv_0: Someone could be eavesdropping on you right now (man-in-the-middle attack)!
2011-12-01 10:28:53,104 commands ERROR AttributeError: Parsing a line from lunch-slave sat_video_recv_0: It is also possible that the RSA host key has just been changed.
2011-12-01 10:28:53,104 commands ERROR AttributeError: Parsing a line from lunch-slave sat_video_recv_0: The fingerprint for the RSA key sent by the remote host is
2011-12-01 10:28:53,104 commands ERROR AttributeError: Parsing a line from lunch-slave sat_video_recv_0: 50:1d:53:43:51:d6:dd:27:cc:e4:33:da:08:41:66:15.
2011-12-01 10:28:53,105 commands ERROR AttributeError: Parsing a line from lunch-slave sat_video_recv_0: Please contact your system administrator.
2011-12-01 10:28:53,105 commands ERROR AttributeError: Parsing a line from lunch-slave sat_video_recv_0: Add correct host key in /home/scenic/.ssh/known_hosts to get rid of this message.
2011-12-01 10:28:53,105 commands ERROR AttributeError: Parsing a line from lunch-slave sat_video_recv_0: Offending key in /home/scenic/.ssh/known_hosts:3
2011-12-01 10:28:53,105 commands ERROR AttributeError: Parsing a line from lunch-slave sat_video_recv_0: RSA host key for 206.167.169.239 has changed and you have requested strict checking.
2011-12-01 10:28:53,105 commands ERROR AttributeError: Parsing a line from lunch-slave sat_video_recv_0: Host key verification failed.
scenic@vlan:~$ ssh 206.167.169.239
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that the RSA host key has just been changed.
The fingerprint for the RSA key sent by the remote host is
50:1d:53:43:51:d6:dd:27:cc:e4:33:da:08:41:66:15.
Please contact your system administrator.
Add correct host key in /home/scenic/.ssh/known_hosts to get rid of this message.
Offending key in /home/scenic/.ssh/known_hosts:3
RSA host key for 206.167.169.239 has changed and you have requested strict checking.
Host key verification failed.
Process B and C depend on A. Fine. But what if we need to kill B if C dies, and kill C if B dies ? We must kill all the processes in the group "B&C".
with B
if any node in its group is STOPPED:
kill every node in the group
That would mean that every nodes in that group must be started at the exact same time, so that their state is never STOPPED when any of them is RUNNING.
Let's think more about this.
If process is a zombie, kill it !
See http://bitbucket.org/chrismiles/psi/src/tip/src/arch/linux_process.c in the Python module psi, maybe ? It reads in /proc to get some informations. I think it's when calling psi.process.Process.exists()
/** Decode the state character from /proc/<pid>/stat or /proc/<pid>/status
*
* Returns an int with the correct PROC_STATUS_* constant in.
*/
static int
decode_state(const char state)
{
int tmpi;
switch (state) {
case 'R':
tmpi = PROC_STATUS_RUNNING;
break;
case 'S':
tmpi = PROC_STATUS_SLEEPING;
break;
case 'D':
tmpi = PROC_STATUS_DISKSLEEP;
break;
case 'Z':
tmpi = PROC_STATUS_ZOMBIE;
break;
case 'T':
tmpi = PROC_STATUS_STOPPED;
break;
case 'W':
tmpi = PROC_STATUS_PAGING;
break;
case 'X':
tmpi = PROC_STATUS_DEAD;
break;
default:
[[PyErr]]_Format(PyExc_OSError, "Invalid/unknown state: %s", &state);
return -1;
}
return tmpi;
}
Remove the master-slave terminology
Criteria
This will be a breaking change, since it will change the interface of the software. (the name of the utilities, among others). A major release should follow.
YAML would be a good candidate. It would be similar to docker compose and Kubernetes, among others.
What was nice with Python as a configuration file language was the fact that we could use variables and loops. We could still provide a Python interface, but YAML would be better as a configuration syntax. We can very well write YAML files with scripts.
Question: What would be the syntax, exactly? (we can look at Kubernetes and Docker Compose, for inspiration)
$ git fsck
Checking object directories: 100% (256/256), done.
error in commit 4e1701fd6e5f028b8be0c936f776e498d967d580: invalid author/committer line - bad date
error in commit b1cf262a40f789077e91567295e54ae0f55d94d2: invalid author/committer line - bad date
error in commit e14382e8bb5b97144fe8df4765f9df074c3c65dd: invalid author/committer line - bad date
error in commit 8a2e0410ad7140816342a78c529fd80243b7ee0c: invalid author/committer line - bad date
error in commit 4463d986a481b49a1bee618844bf9fee49686578: invalid author/committer line - bad date
error in commit 7d53497feb3fb69d366ee7e47458808de0f828df: invalid author/committer line - bad date
error in commit 6ada189bf93a418f49131c9062d5f1d3ee66e9db: invalid author/committer line - bad date
error in commit a4becacbe3bc5c62e7924e9860e26ebec1771914: invalid author/committer line - bad date
error in commit 1002b70d2eafe88b45a7633f8c23dfa6e4d0a8f1: invalid author/committer line - bad date
error in commit 9ec4c4e860cebe53022ea9b3166da30031256565: invalid author/committer line - bad date
error in commit 16fee29616ff71e78b7ff162f5b43e111e1f5d29: invalid author/committer line - bad date
Rather, use Twisted's process launching utilities.
:0: UserWarning: You do not have a working installation of the service_identity module: 'No module named 'service_identity''. Please install it from <https://pypi.python.org/pypi/service_identity> and make sure all of its dependencies are satisfied. Without the service_identity module, Twisted can perform only rudimentary TLS client hostname verification. Many valid certificate/hostname mappings may be rejected.
Can we have some sort of automatic deletion of these files? Currently, the log appends to the file, making some very huge files. If we use the naming convention above (with the pid), we will instead accumulate many old files.
the GUI should run on:
Details:
lunch -k filename.py
Unhandled Error
Traceback (most recent call last):
File "/usr/local/lib/python3.9/dist-packages/lunch-0.7.0-py3.9.egg/EGG-INFO/scripts/lunch", line 42, in <module>
runner.run()
File "/usr/local/lib/python3.9/dist-packages/lunch-0.7.0-py3.9.egg/lunch/runner.py", line 132, in run
reactor.run()
File "/usr/local/lib/python3.9/dist-packages/Twisted-21.7.0-py3.9.egg/twisted/internet/base.py", line 1318, in run
self.mainLoop()
File "/usr/local/lib/python3.9/dist-packages/Twisted-21.7.0-py3.9.egg/twisted/internet/base.py", line 1328, in mainLoop
reactorBaseSelf.runUntilCurrent()
--- <exception caught here> ---
File "/usr/local/lib/python3.9/dist-packages/Twisted-21.7.0-py3.9.egg/twisted/internet/base.py", line 994, in runUntilCurrent
call.func(*call.args, **call.kw)
File "/usr/local/lib/python3.9/dist-packages/lunch-0.7.0-py3.9.egg/lunch/master.py", line 648, in _kill
pid = is_lunch_master_running(pid_file)
File "/usr/local/lib/python3.9/dist-packages/lunch-0.7.0-py3.9.egg/lunch/master.py", line 596, in is_lunch_master_running
if "python" in output:
builtins.TypeError: a bytes-like object is required, not 'str'
For the childs, an options like this should be ok:
working_dir = "/var/bar",
Support Microsoft Windows.
As a user, I want to be able to run Lunch on Windows.
If a slave tries to launch a command, and the command is not found, it should give up. Isn't it?
lunch-slave default_1> Command not found: glxgears
The authenticity of host '10.21.10.27 (10.21.10.27)' can't be established.
RSA key fingerprint is 80:d6:ba:15:44:07:c5:35:ec:3e:ea:58:8a:f0:a7:d0.
Are you sure you want to continue connecting (yes/no)?
2020-11-17 17:52:28,892 commands ERROR AttributeError: Parsing a line from lunch-slave dataviz-right: builtins.UnicodeDecodeError:
2020-11-17 17:52:36,768 commands ERROR AttributeError: Parsing a line from lunch-slave dataviz-right: Unhandled
2020-11-17 17:52:36,769 commands ERROR AttributeError: Parsing a line from lunch-slave dataviz-right: Traceback
This will make it easier to restart the child, especially when we need to restart the whole group.
./scripts/lunch-slave
msg Welcome to the lunch-slave console. Type 'help' for help.
ready
cmd echo hello
Unhandled Error
Traceback (most recent call last):
File "/Users/aalex/anaconda3/lib/python3.7/site-packages/twisted/python/log.py", line 103, in callWithLogger
return callWithContext({"system": lp}, func, *args, **kw)
File "/Users/aalex/anaconda3/lib/python3.7/site-packages/twisted/python/log.py", line 86, in callWithContext
return context.call({ILogContext: newCtx}, func, *args, **kw)
File "/Users/aalex/anaconda3/lib/python3.7/site-packages/twisted/python/context.py", line 122, in callWithContext
return self.currentContext().callWithContext(ctx, func, *args, **kw)
File "/Users/aalex/anaconda3/lib/python3.7/site-packages/twisted/python/context.py", line 85, in callWithContext
return func(*args,**kw)
--- <exception caught here> ---
File "/Users/aalex/anaconda3/lib/python3.7/site-packages/twisted/internet/selectreactor.py", line 149, in _doReadOrWrite
why = getattr(selectable, method)()
File "/Users/aalex/anaconda3/lib/python3.7/site-packages/twisted/internet/process.py", line 252, in doRead
return fdesc.readFromFD(self.fd, self.dataReceived)
File "/Users/aalex/anaconda3/lib/python3.7/site-packages/twisted/internet/fdesc.py", line 94, in readFromFD
callback(output)
File "/Users/aalex/anaconda3/lib/python3.7/site-packages/twisted/internet/process.py", line 256, in dataReceived
self.proc.childDataReceived(self.name, data)
File "/Users/aalex/anaconda3/lib/python3.7/site-packages/twisted/internet/_posixstdio.py", line 77, in childDataReceived
self.protocol.dataReceived(data)
File "/Users/aalex/anaconda3/lib/python3.7/site-packages/twisted/protocols/basic.py", line 572, in dataReceived
why = self.lineReceived(line)
File "./scripts/lunch-slave", line 459, in lineReceived
key = line.split(" ")[0].lower()
builtins.TypeError: a bytes-like object is required, not 'str'
We need to make sure the /var/tmp/lunch directory is world writable.
it should stop the process, change it and restart it, or something like that...
Instead of master-slave, use manager-agent.
See the discussion in #34
Users typically have commands ran first with respawn=False. Those are setup commands. We could add teardown commands, ran before quitting. (or restarting the whole thing)
They should not be all displayed in main GUI, but just a summary of them. Maybe a separate window could display their status and output value, or so?
We can know if a ssh key is installed on a remote host like this:
ssh [email protected] echo hello
If we are prompted for a password, maybe we should stop this process and pop up a xterm window, and wait for it to return ? Or we could use a GTK widget to ask for the password to unlock the key.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.