osc / bc_osc_jupyter Goto Github PK
View Code? Open in Web Editor NEWBatch Connect - OSC Jupyter Notebook Server
Home Page: https://osc.github.io/Open-OnDemand/
License: MIT License
Batch Connect - OSC Jupyter Notebook Server
Home Page: https://osc.github.io/Open-OnDemand/
License: MIT License
This message is no longer needed. It is safe to remove it.
Not sure whether this is is being maintained by anyone now, but the template scripts use module
to purge the module environment.
For user who are using a Python library that requires other modules be loaded so the libraries are accessible to Python, e.g., gdal
, this breaks under the scheme presented here and would require that the installers of the Jupyter app also load all modules that might be needed by users.
Is there a reason not to use the environment that the user sets up with, for example, Lmod's default collection device? I am wondering if there was some specific reason for totally purging
all modules prior to Jupyter's start.
Some links in the README.md
are being redirected. These should be updated.
Currently the connection information is generated immediately after launching the Jupyter notebook server. But the server may not be fully loaded, and so the "Connect to Jupyter" button fails. That is why we warn the user with the following text:
The new Dashboard (not yet released) includes ood_core
v0.2.0 which provides a helper Bash function that can wait until a port is used. Calling this in after.sh
can postpone the connection file generation until the server is listening for connections.
We disabled it for an immediate fix to connecting to Jupyter as we launch using https://github.com/OSC/bc_osc_jupyter/blob/master/connection.html.mustache
The error actually said that the POST was missing _xsrf
. So I think we could actually set a hidden field _xsrf
with the proper value and it would work. Since we are starting the server, its possible we would know this ahead of time. However, implementing #8 would be another option and then we could just reenable the xsrf/csrf protection because we wouldn't be using POST via a form anymore from vncsim.
On a branch, modify the files necessary to get Jupyter running at a SLURM site. We can use an ssh tunnel to connect and if necessary install Jupyter in our home directory.
Having this working, modified version at another site will help us understand what parts are Torque and OSC specific and make it easier to discuss generalizing Jupyter or documenting how to fork and modify Jupyter to run elsewhere.
When largemem becomes available in jupyter expansion, this ticket is to enable that.
Replace the "RStudio Server" references that accidentally got copied over in the template/after.sh
file.
The function wait_until_port_used
in after.sh
prevent the a failed job from quitting until timeout. We should check if script.sh
is running before this endless loop.
We have a user that wants to launch Jupyter interactively and connect to it because they have their own conda environment and wish to use it instead of pip installed packages.
I'm wondering if we could just add a checkbox to launch under their own conda environment instead of under our modules and so on.
Get this error on Conda tab:
This is due to the code:
/usr/local/anaconda3/lib/python3.5/site-packages/nb_conda/envmanager.py
in particular on line 16:
name, version, build = s.rsplit('-', 2)
because s
is a dict generated from a json object instead of a string. The issue on this is tracked here: anaconda/nb_conda#39
We are waiting on them to merge in the pull request with the fix.
Update node_type help with pitzer expansion to indicate those nodes have 48 cores.
Is there support for singularity or apptainer images?
Now that we have 1.8.x in production, we can merge the two jupyter repositories. So this is to merge with https://github.com/OSC/bc_osc_jupyter_pitzer/.
Note that any merge should account for the different node types on the different clusters and the different ppn requests for those clusters.
Pitzer expansion nodes have up to 4 GPUs and Pitzer has 2. This ticket is to enable launching Jupyter and allocating N GPUs.
Formatting should be consistent, currently there is:
maybe just reduce it to:
https://blog.jupyter.org/2016/12/21/jupyter-notebook-4-3-1/
We have just released Jupyter Notebook 4.3.0 and 4.3.1 with some important security fixes.
4.3.0: Token authentication
The biggest change in 4.3 is the addition of token-based authentication, which is enabled by default.
This would allow us to re-enable xsrf (and csrf) protection in Jupyter because we can avoid using a POST request in a web form from the vncsim connection tab when launching Jupyter and still get single sign on.
Running module spider julia
over SSH appears to take longer than 1 minute in some cases causing an Apache timeout.
A number of solutions present themselves:
See https://discourse.osc.edu/t/jupyter-lmod-plugin/454 for a discussion on how this appears to not work in the plugin.
First ensure you can do this at OSC via a desktop session, like the Discourse topic indicates is possible. Second, see how the environment differs in the interactive app that results in this plugin not working.
Cuda needs to be enabled explicitly when launched on a node with a GPU:
module load cuda/8.0.44
Requested in SN Ticket: INC0315978
Notes on how to launch Spark within the batch job can be found here:
https://osu.app.box.com/notes/225694916910
Possibly UI in the form can be handled with a checkbox that says "Launch Spark with Jupyter", or a dropdown box in anticipation for including multiple frameworks.
The server should be running before generating the *.conn
file. Otherwise users won't be able to connect through the proxy right away without waiting a few seconds.
Migrate to use Dynamic JavaScript.
Child ticket of OSC/ondemand#1657
This was suggested by Shameema when working with the Jupyter + Spark app. This should also be carried over to this app.
These are closer to what happens:
I wonder if we can just get rid of jupyter notebook and boot lab directly? @ZQyou?
A user should be able to choose to launch Jupyter with either Python 2 or Python 3.
module load python/2.7
Requested in SN Ticket: INC0315978
Allow the user to choose the ability to have the Jupyter server timeout after 10 minutes of inactivity. This can be achieved with the configuration options:
NotebookApp.shutdown_no_activity_timeout
and
MappingKernelManager.cull_idle_timeout
See http://jupyter-notebook.readthedocs.io/en/stable/config.html for more information.
use a radio button for lab/notebook switch instead of the checkbox. The question doesn't quite read right.
Also update default to lab?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.