osc / bc_osc_jupyter Goto Github PK

This message is no longer needed. It is safe to remove it.

Not sure whether this is is being maintained by anyone now, but the template scripts use module to purge the module environment.

For user who are using a Python library that requires other modules be loaded so the libraries are accessible to Python, e.g., gdal, this breaks under the scheme presented here and would require that the installers of the Jupyter app also load all modules that might be needed by users.

Is there a reason not to use the environment that the user sets up with, for example, Lmod's default collection device? I am wondering if there was some specific reason for totally purging
all modules prior to Jupyter's start.

Some links in the README.md are being redirected. These should be updated.

Currently the connection information is generated immediately after launching the Jupyter notebook server. But the server may not be fully loaded, and so the "Connect to Jupyter" button fails. That is why we warn the user with the following text:

The new Dashboard (not yet released) includes ood_core v0.2.0 which provides a helper Bash function that can wait until a port is used. Calling this in after.sh can postpone the connection file generation until the server is listening for connections.

We disabled it for an immediate fix to connecting to Jupyter as we launch using https://github.com/OSC/bc_osc_jupyter/blob/master/connection.html.mustache

The error actually said that the POST was missing _xsrf. So I think we could actually set a hidden field _xsrf with the proper value and it would work. Since we are starting the server, its possible we would know this ahead of time. However, implementing #8 would be another option and then we could just reenable the xsrf/csrf protection because we wouldn't be using POST via a form anymore from vncsim.

On a branch, modify the files necessary to get Jupyter running at a SLURM site. We can use an ssh tunnel to connect and if necessary install Jupyter in our home directory.

Having this working, modified version at another site will help us understand what parts are Torque and OSC specific and make it easier to discuss generalizing Jupyter or documenting how to fork and modify Jupyter to run elsewhere.

When largemem becomes available in jupyter expansion, this ticket is to enable that.

Replace the "RStudio Server" references that accidentally got copied over in the template/after.sh file.

The function wait_until_port_used in after.sh prevent the a failed job from quitting until timeout. We should check if script.sh is running before this endless loop.

We have a user that wants to launch Jupyter interactively and connect to it because they have their own conda environment and wish to use it instead of pip installed packages.

I'm wondering if we could just add a checkbox to launch under their own conda environment instead of under our modules and so on.

Terminal should open in user's home directory. This is what happens now:

I get the following error when launching the Conda tab:

Get this error on Conda tab:

This is due to the code:

/usr/local/anaconda3/lib/python3.5/site-packages/nb_conda/envmanager.py

in particular on line 16:

name, version, build = s.rsplit('-', 2)

because s is a dict generated from a json object instead of a string. The issue on this is tracked here: anaconda/nb_conda#39

We are waiting on them to merge in the pull request with the fix.

Update node_type help with pitzer expansion to indicate those nodes have 48 cores.

Is there support for singularity or apptainer images?

Now that we have 1.8.x in production, we can merge the two jupyter repositories. So this is to merge with https://github.com/OSC/bc_osc_jupyter_pitzer/.

Note that any merge should account for the different node types on the different clusters and the different ppn requests for those clusters.

Pitzer expansion nodes have up to 4 GPUs and Pitzer has 2. This ticket is to enable launching Jupyter and allocating N GPUs.

Formatting should be consistent, currently there is:

• Jupyter notebook
• Jupyter Notebook
• Jupyter Notebook server
• Jupyter Notebook Server

maybe just reduce it to:

• Jupyter Notebook (when describing the app)
• Jupyter Notebook server (when discussing the actual server process that listens on a port)

https://blog.jupyter.org/2016/12/21/jupyter-notebook-4-3-1/

We have just released Jupyter Notebook 4.3.0 and 4.3.1 with some important security fixes.

4.3.0: Token authentication

The biggest change in 4.3 is the addition of token-based authentication, which is enabled by default.

This would allow us to re-enable xsrf (and csrf) protection in Jupyter because we can avoid using a POST request in a web form from the vncsim connection tab when launching Jupyter and still get single sign on.

Running module spider julia over SSH appears to take longer than 1 minute in some cases causing an Apache timeout.

A number of solutions present themselves:

• Move kernel generation out of ERB and into a script to be run on the compute node
• Keep the SSH command but add a form control asking the user to specify if they want to use Julia; and raise an error if the SSH command takes longer than a preset timeout
• Use a static list of available Julia versions

See https://discourse.osc.edu/t/jupyter-lmod-plugin/454 for a discussion on how this appears to not work in the plugin.

First ensure you can do this at OSC via a desktop session, like the Discourse topic indicates is possible. Second, see how the environment differs in the interactive app that results in this plugin not working.

Cuda needs to be enabled explicitly when launched on a node with a GPU:

module load cuda/8.0.44

Requested in SN Ticket: INC0315978

Notes on how to launch Spark within the batch job can be found here:

https://osu.app.box.com/notes/225694916910

Possibly UI in the form can be handled with a checkbox that says "Launch Spark with Jupyter", or a dropdown box in anticipation for including multiple frameworks.

The server should be running before generating the *.conn file. Otherwise users won't be able to connect through the proxy right away without waiting a few seconds.

Migrate to use Dynamic JavaScript.

Child ticket of OSC/ondemand#1657

This was suggested by Shameema when working with the Jupyter + Spark app. This should also be carried over to this app.

These are closer to what happens:

• plug is the icon for "connect"
• external-link is appropriate too becasue we open a new window

I wonder if we can just get rid of jupyter notebook and boot lab directly? @ZQyou?

A user should be able to choose to launch Jupyter with either Python 2 or Python 3.

module load python/2.7

Requested in SN Ticket: INC0315978

Allow the user to choose the ability to have the Jupyter server timeout after 10 minutes of inactivity. This can be achieved with the configuration options:

NotebookApp.shutdown_no_activity_timeout

and

MappingKernelManager.cull_idle_timeout

See http://jupyter-notebook.readthedocs.io/en/stable/config.html for more information.

use a radio button for lab/notebook switch instead of the checkbox. The question doesn't quite read right.

Also update default to lab?