Comments (14)
@liukun1016 - the launcher looks good! Just have a couple comments/suggestions...
- We should make sure the close calls for the BufferedWriter and Socket are in finally blocks in case the preceding write calls fail.
- Let's move the block of code for sending the file back on the socket into the block of code that detects the file doesn't exist (so just after creating the file). The reason for this is because the user may change the connection mode from 'socket' to 'push' (although extremely unlikely), but if they don't remove the
--response-address
parameter from the kernel.json file, then the code will try to bind to a socket that won't exist - since Elyra has pushed the file. (Actually, it looks like code willexit(-1)
with an invalid socket address message since the value of that parameter will beERROR__NO__KERNEL_RESPONSE_ADDRESS
, but same effect.)
Also, have you had a chance to see if creating a Spark Context allows us to get around the Lazy load issue?
from enterprise_gateway.
If we want to modify the Toree, do we want to open an Issue on JIRA and later a PR?
from enterprise_gateway.
Yes - that is correct. However, I believe we should implement a launch application instead so as to provide us with unlimited flexibility and consistency (with no kernel update necessary). Once we produce a few such launch scripts (these are probably language specific items - not kernel specific), then a template can be documented for kernels of other languages to be produced.
from enterprise_gateway.
These "launchers" have 2 aspects:
- finding available local ports and creating the connection file (different per language)
- launching the actual kernel process (different per kernel)
In Spark we have compiled languages (Java, Scala) and scripted languages (Python, R). For the compiled languages we need to build and package a small launcher application as opposed to mere launcher scripts. For that we will need to add some build infrastructure, i.e. Maven or SBT to build and package jar files.
from enterprise_gateway.
+1 @ckadner
Regarding item 2, I was hoping the kernel process to launch would be conveyed as a parameter to the launch script/application, thereby only requiring language-specific launchers. I suspect you meant the same, but just want to be sure we only have an order of N thing and not NxM.
Also, it seems like the 'language' aspect of these launchers is really a function of the kernel's implementation language. For example, the Python and R kernels in Toree require that a Scala application be launched. As a result, the launcher for these kernels would still be written in Scala.
from enterprise_gateway.
Yes, I agree that we'd better create a launch application instead of changing the Toree kernel.
Would it be the easiest way if we do the similar thing as the launch_ipykernel.py script?
That is, simply add another Python script that handles the non-existent connection file and let the run.sh call this script before the spark-submit.
from enterprise_gateway.
Generation of the connection file needs to take place on the destination node and its via spark-submit in which that occurs. I don't know spark-submit well enough to know if it can launch a script/application in one language then have that thing start something in another language. I suspect spark-submit determines the interpreter based on the launch target - so I doubt switching languages would work. (Then again, I could be way off-base.)
from enterprise_gateway.
Kevin is correct about this:
spark-submit determines the interpreter based on the launch target
from enterprise_gateway.
I see your idea. Since connection file should be in the node where kernel will be running, we need to create the connection file after run.sh
is called.
I am going to develop a layer (let's call it ToreeLauncher
for now) around the Main method of Toree to generate the missing connection file, after spark-submit issued in run.sh
but before the kernel starts.
from enterprise_gateway.
@liukun1016 -- to better understand how the IPython kernel does find empty ports and create the connection file, start by taking a look at IPKernelApp.initialize(). For finding free ports you could also look at Akka (there your code is already in Scala), but for the session key I think we need to replicate what IPython does in ... jupyter_client/session.py
Just a nit-pick on terminology 😉 ... the "launch application" you are writing is not really a "wrapper" ... it will simply call the ToreeMain
application once it is done creating the connection file.
from enterprise_gateway.
@ckadner I see. Yes wrapper might not be a good name though. (updated the comment)
Will take a look at your references and thanks a lot!
from enterprise_gateway.
Please take a look at PR #43. That PR makes a change to the kernelspec format (where connection_file_mode can be specified) - so just wanted to bring that to your attention since it might affect your merge. Also, once pull mode is implemented, socket mode is a small step away.
from enterprise_gateway.
The source code of the Toree Launcher application: https://github.com/liukun1016/ToreeLauncher/blob/master/src/main/scala/launcher/ToreeLauncher.scala
Pull mode is already finished. Will test the socket mode.
from enterprise_gateway.
PR merged. We are going to maintain the Toree Launcher app source codes separately. So we don't have to include that in elyra building process.
Project created under https://github.com/SparkTC/elyra-toree-launcher
Close this issue now.
from enterprise_gateway.
Related Issues (20)
- Allow enterprise gateway to launch k8s kernels on remote clusters HOT 7
- Publish helm JEG on ArtifactHub HOT 1
- jupyter_client complains on wrogly passed kernel_id to Popen call HOT 1
- Enterprise gateway Spark Operator (Python) kernel fails to systematic restart HOT 3
- Avoid too many retried on kernel shutdown tries HOT 1
- Failed to start kernels (Spark Operator) keeps trying to connect indefinitely
- could not find elyra/enterprise-gateway:3.2.1 HOT 4
- Conda -> Venv for local development HOT 4
- DockerProcessProxy missing abstract method implementation
- Add Karpenter do-not-evict annotations to kernel-pod HOT 2
- Jupyter Enterprise Gateway to also create an Ingress resource along with service for each driver pod to access Spark UI HOT 3
- Add Mutual Authentication Option as a Parameter HOT 3
- Fix doc build HOT 1
- Build fail -> Error response from daemon: No such container: itest-yarn
- Unable to connect kernel HOT 6
- Jupyter lab error starting kernel HOT 8
- Add the option to terminate pending kubernetes kernels if they have events preventing them from starting HOT 4
- KIP cannot use ImagePullSecret when using containerd HOT 2
- kernel Status: Unknown in Jupyterhub 3.2.1 while connecting remote kernel vai Jupyter Enterprise Gateway 3.2.1 on K8s 1.25 AWS EKS HOT 3
- upgrade the os version used by the kernel-image-puller from Debian 11 to Debian 12 HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from enterprise_gateway.