Git Product home page Git Product logo

swarmspawner's Introduction

This project is deprecated, please check DockerSpawner.
==============================
SwarmSpawner

PyPI version

Python Versions

SwarmSpawner enables JupyterHub to spawn single user notebook servers in Docker Services.

More info about Docker Services here.

Prerequisites

Python version 3.3 and above is required.

Installation

Installation from GitHub

Configuration

You can find an example jupyter_config.py inside examples.

The spawner

Docker Engine in Swarm mode and the related services work in a different way compared to Docker containers.

Tell JupyterHub to use SwarmSpawner by adding the following lines to your `jupyterhub_config.py`:

What is jupyterhub_service_name?

Inside a Docker engine in Swarm mode the services use a name instead of a ip to communicate with each other. 'jupyterhub_service_name' is the name of ther service for the JupyterHub.

Networks

It's important to put the JupyterHub service (also the proxy) and the services that are running jupyter notebook inside the same network, otherwise they couldn't reach each other. SwarmSpawner use the service's name instead of the service's ip, as a consequence JupyterHub and servers should share the same overlay network (network across nodes).

Define the services inside jupyterhub_config.py

You can define container_spec, resource_spec and networks inside jupyterhub_config.py.

command and args depends on the image that you are using.

If you use one of the images from the Jupyter docker-stack you need to specify args as: /usr/local/bin/start-singleuser.sh

If you are using a specific image, well it's up to you to specify the right command.

Note: in a container spec, args sets the equivalent of CMD in the Dockerfile, command sets the equivalent of ENTRYPOINT. The notebook server command should not be the ENTRYPOINT, so generally use args, not command, to specify how to launch the notebook server.

See this issue for more info.

Bind a Host dir

With 'type':'bind' you mount a local directory of the host inside the container.

Remember that source should exist in the node where you are creating the service.

Mount a named volume

With 'type':'volume' you mount a Docker Volume inside the container. If the volume doesn't exist it will be created.

Named path

For both types, volume and bind, you can specify a {username} inside the source:

username will be the hashed version of the username.

Mount an anonymous volume

This kind of volume will be removed with the service.

Resource_spec

You can also specify some resource for each service

Using user_options

There is the possibility to set parameters using user_options

To control the creation of the services you have 2 ways, using jupyterhub_config.py or user_options.

Remember that at the end you are just using the Docker Engine API.

user_options, if used, will overwrite jupyter_config.py for services.

If you set c.SwarmSpawner.use_user_option = True the spawner will use the dict passed through the form or as json body when using the Hub Api.

The spawner expect a dict with these keys:

Names of the Jupyter notebook service inside Docker engine in Swarm mode

When JupyterHub spawns a new Jupyter notebook server the name of the service will be {service_prefix}-{service_owner}-{service_suffix}

You can change the service_prefix in this way:

Prefix of the service in Docker

service_owner is the hexdigest() of the hashed user.name.

In case of named servers (more than one server for user) service_suffix is the name of the server, otherwise is always 1.

Downloading images

Docker Engine in Swarm mode downloads images automatically from the repository. Either the image is available on the remote repository or locally, if not you will get an error.

Because before starting the service you have to complete the download of the image is better to have a longer timeout (default is 30 secs)

You can use all the docker images inside the Jupyter docker-stacks.

Contributing

If you would like to contribute to the project, please read contributor documentation.

For a development install, clone the repository and then install from source:

Credit

DockerSpawner

License

All code is licensed under the terms of the revised BSD license.

swarmspawner's People

Contributors

florianrhiem avatar isuftin avatar ken24 avatar mckinsel avatar minrk avatar twavv avatar wakonp avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

swarmspawner's Issues

Difference in mounts type between c.SwarmSpawner.container_spec and user_options?

This works:

mounts = [{
        'type': 'volume',
        'source': 'jupyterhub-user-{username}',
        'target': notebook_dir
    }]
    c.SwarmSpawner.container_spec = {
        'args': ['/usr/local/bin/start-singleuser.sh'],
        'Image': os.getenv('SINGLE_USER_IMAGE', 'jupyterhub/singleuser:latest'),
        'mounts': mounts
    }

This does not:

 mounts = [{
        'type': 'volume',
        'source': 'jupyterhub-user-{username}',
        'target': notebook_dir
    }]

    c.SwarmSpawner.use_user_options = True
    user_options = {
        'container_spec' : {
            'args': ['/usr/local/bin/start-singleuser.sh'],
            'Image': os.getenv('SINGLE_USER_IMAGE', 'jupyterhub/singleuser:latest'),
            'mounts': mounts,
            'placement' : ["node.Role == worker"],
            'network' : [os.getenv('SWARMSPAWNER_JUPYTERHUB_NETWORKS', 'jupyterhub_dlcc_network')],
            'name' : os.getenv('SWARMSPAWNER_JUPYTERHUB_SERVICE_NAME', 'jupyterhub_jupyterhub')
        }   
    }

Error seen:

jupyterhub_jupyterhub.1.rv400s2ujiec@myservice    |     Traceback (most recent call last):
jupyterhub_jupyterhub.1.rv400s2ujiec@myservice    |       File "/opt/conda/lib/python3.5/site-packages/tornado/web.py", line 1469, in _execute
jupyterhub_jupyterhub.1.rv400s2ujiec@myservice    |         result = yield result
jupyterhub_jupyterhub.1.rv400s2ujiec@myservice    |       File "/opt/conda/lib/python3.5/site-packages/jupyterhub/handlers/base.py", line 535, in get
jupyterhub_jupyterhub.1.rv400s2ujiec@myservice    |         yield self.spawn_single_user(current_user)
jupyterhub_jupyterhub.1.rv400s2ujiec@myservice    |       File "/opt/conda/lib/python3.5/site-packages/jupyterhub/handlers/base.py", line 328, in spawn_single_user
jupyterhub_jupyterhub.1.rv400s2ujiec@myservice    |         yield gen.with_timeout(timedelta(seconds=self.slow_spawn_timeout), f)
jupyterhub_jupyterhub.1.rv400s2ujiec@myservice    |       File "/opt/conda/lib/python3.5/site-packages/jupyterhub/user.py", line 261, in spawn
jupyterhub_jupyterhub.1.rv400s2ujiec@myservice    |         raise e
jupyterhub_jupyterhub.1.rv400s2ujiec@myservice    |       File "/opt/conda/lib/python3.5/site-packages/jupyterhub/user.py", line 229, in spawn
jupyterhub_jupyterhub.1.rv400s2ujiec@myservice    |         ip_port = yield gen.with_timeout(timedelta(seconds=spawner.start_timeout), f)
jupyterhub_jupyterhub.1.rv400s2ujiec@myservice    |       File "/opt/conda/lib/python3.5/site-packages/swarmspawner-0.1.0-py3.5.egg/cassinyspawner/swarmspawner.py", line 285, in start
jupyterhub_jupyterhub.1.rv400s2ujiec@myservice    |         for mount in self.container_spec['mounts']:
jupyterhub_jupyterhub.1.rv400s2ujiec@myservice    |     KeyError: 'mounts'

Question on docker volumes in multiple hosts

Hi @barrachri, thank you very much for the SwarmSpawner, it has helped me a lot!

I have successfully managed to run the Jupyterhub with your spawner, but I have one general question. Since it is a Swarm network, it is most likely that the spawned notebook will be spread across multiple hosts, and there is no guarantee that the same container will be spawned on the same host in the future. What is the strategy to ensure that the same user can access the previously created container?

Polling frequency when attempting to connect to notebook container

Is there a flag to allow setting the polling frequency for JupyterHub's polling of a starting notebook container? It seems to be polling 5 to 10 times per second which floods the service logs and I assume puts a decent load on the Docker network.

Example of JupyterHub's service logs when a container is being started (but the image not yet downloaded from the registry):

jupyterhub_jupyterhub.1.0fe08onaa24h@manager    | [W 2017-08-24 15:13:16.156 JupyterHub utils:88] Failed to connect to http://jupyter-5d9c68c6c50ed3d02a2fcf54f63993b6-1:8888/user/testuser ([Errno -2] Name or service not known)
jupyterhub_jupyterhub.1.0fe08onaa24h@manager    | [W 2017-08-24 15:13:16.375 JupyterHub utils:88] Failed to connect to http://jupyter-5d9c68c6c50ed3d02a2fcf54f63993b6-1:8888/user/testuser ([Errno -2] Name or service not known)
jupyterhub_jupyterhub.1.0fe08onaa24h@manager    | [W 2017-08-24 15:13:16.594 JupyterHub utils:88] Failed to connect to http://jupyter-5d9c68c6c50ed3d02a2fcf54f63993b6-1:8888/user/testuser ([Errno -2] Name or service not known)
jupyterhub_jupyterhub.1.0fe08onaa24h@manager    | [W 2017-08-24 15:13:16.808 JupyterHub utils:88] Failed to connect to http://jupyter-5d9c68c6c50ed3d02a2fcf54f63993b6-1:8888/user/testuser ([Errno -2] Name or service not known)
jupyterhub_jupyterhub.1.0fe08onaa24h@manager    | [W 2017-08-24 15:13:17.025 JupyterHub utils:88] Failed to connect to http://jupyter-5d9c68c6c50ed3d02a2fcf54f63993b6-1:8888/user/testuser ([Errno -2] Name or service not known)
jupyterhub_jupyterhub.1.0fe08onaa24h@manager    | [W 2017-08-24 15:13:17.239 JupyterHub utils:88] Failed to connect to http://jupyter-5d9c68c6c50ed3d02a2fcf54f63993b6-1:8888/user/testuser ([Errno -2] Name or service not known)
jupyterhub_jupyterhub.1.0fe08onaa24h@manager    | [W 2017-08-24 15:13:17.455 JupyterHub utils:88] Failed to connect to http://jupyter-5d9c68c6c50ed3d02a2fcf54f63993b6-1:8888/user/testuser ([Errno -2] Name or service not known)
jupyterhub_jupyterhub.1.0fe08onaa24h@manager    | [W 2017-08-24 15:13:17.672 JupyterHub utils:88] Failed to connect to http://jupyter-5d9c68c6c50ed3d02a2fcf54f63993b6-1:8888/user/testuser ([Errno -2] Name or service not known)
jupyterhub_jupyterhub.1.0fe08onaa24h@manager    | [W 2017-08-24 15:13:17.924 JupyterHub utils:88] Failed to connect to http://jupyter-5d9c68c6c50ed3d02a2fcf54f63993b6-1:8888/user/testuser ([Errno -2] Name or service not known)
jupyterhub_jupyterhub.1.0fe08onaa24h@manager    | [W 2017-08-24 15:13:18.138 JupyterHub utils:88] Failed to connect to http://jupyter-5d9c68c6c50ed3d02a2fcf54f63993b6-1:8888/user/testuser ([Errno -2] Name or service not known)
jupyterhub_jupyterhub.1.0fe08onaa24h@manager    | [W 2017-08-24 15:13:18.376 JupyterHub utils:88] Failed to connect to http://jupyter-5d9c68c6c50ed3d02a2fcf54f63993b6-1:8888/user/testuser ([Errno -2] Name or service not known)
jupyterhub_jupyterhub.1.0fe08onaa24h@manager    | [W 2017-08-24 15:13:18.590 JupyterHub utils:88] Failed to connect to http://jupyter-5d9c68c6c50ed3d02a2fcf54f63993b6-1:8888/user/testuser ([Errno -2] Name or service not known)
jupyterhub_jupyterhub.1.0fe08onaa24h@manager    | [W 2017-08-24 15:13:18.805 JupyterHub utils:88] Failed to connect to http://jupyter-5d9c68c6c50ed3d02a2fcf54f63993b6-1:8888/user/testuser ([Errno -2] Name or service not known)
jupyterhub_jupyterhub.1.0fe08onaa24h@manager    | [W 2017-08-24 15:13:19.028 JupyterHub utils:88] Failed to connect to http://jupyter-5d9c68c6c50ed3d02a2fcf54f63993b6-1:8888/user/testuser ([Errno -2] Name or service not known)
jupyterhub_jupyterhub.1.0fe08onaa24h@manager    | [W 2017-08-24 15:13:19.245 JupyterHub utils:88] Failed to connect to http://jupyter-5d9c68c6c50ed3d02a2fcf54f63993b6-1:8888/user/testuser ([Errno -2] Name or service not known)
jupyterhub_jupyterhub.1.0fe08onaa24h@manager    | [W 2017-08-24 15:13:19.460 JupyterHub utils:88] Failed to connect to http://jupyter-5d9c68c6c50ed3d02a2fcf54f63993b6-1:8888/user/testuser ([Errno -2] Name or service not known)
jupyterhub_jupyterhub.1.0fe08onaa24h@manager    | [W 2017-08-24 15:13:19.694 JupyterHub utils:88] Failed to connect to http://jupyter-5d9c68c6c50ed3d02a2fcf54f63993b6-1:8888/user/testuser ([Errno -2] Name or service not known)
jupyterhub_jupyterhub.1.0fe08onaa24h@manager    | [W 2017-08-24 15:13:19.909 JupyterHub utils:88] Failed to connect to http://jupyter-5d9c68c6c50ed3d02a2fcf54f63993b6-1:8888/user/testuser ([Errno -2] Name or service not known)
jupyterhub_jupyterhub.1.0fe08onaa24h@manager    | [W 2017-08-24 15:13:20.124 JupyterHub utils:88] Failed to connect to http://jupyter-5d9c68c6c50ed3d02a2fcf54f63993b6-1:8888/user/testuser ([Errno -2] Name or service not known)
jupyterhub_jupyterhub.1.0fe08onaa24h@manager    | [W 2017-08-24 15:13:20.350 JupyterHub utils:88] Failed to connect to http://jupyter-5d9c68c6c50ed3d02a2fcf54f63993b6-1:8888/user/testuser ([Errno -2] Name or service not known)
jupyterhub_jupyterhub.1.0fe08onaa24h@manager    | [W 2017-08-24 15:13:20.565 JupyterHub utils:88] Failed to connect to http://jupyter-5d9c68c6c50ed3d02a2fcf54f63993b6-1:8888/user/testuser ([Errno -2] Name or service not known)
jupyterhub_jupyterhub.1.0fe08onaa24h@manager    | [W 2017-08-24 15:13:20.782 JupyterHub utils:88] Failed to connect to http://jupyter-5d9c68c6c50ed3d02a2fcf54f63993b6-1:8888/user/testuser ([Errno -2] Name or service not known)
jupyterhub_jupyterhub.1.0fe08onaa24h@manager    | [W 2017-08-24 15:13:20.999 JupyterHub utils:88] Failed to connect to http://jupyter-5d9c68c6c50ed3d02a2fcf54f63993b6-1:8888/user/testuser ([Errno -2] Name or service not known)
jupyterhub_jupyterhub.1.0fe08onaa24h@manager    | [W 2017-08-24 15:13:21.218 JupyterHub utils:88] Failed to connect to http://jupyter-5d9c68c6c50ed3d02a2fcf54f63993b6-1:8888/user/testuser ([Errno -2] Name or service not known)
jupyterhub_jupyterhub.1.0fe08onaa24h@manager    | [W 2017-08-24 15:13:21.433 JupyterHub utils:88] Failed to connect to http://jupyter-5d9c68c6c50ed3d02a2fcf54f63993b6-1:8888/user/testuser ([Errno -2] Name or service not known)
jupyterhub_jupyterhub.1.0fe08onaa24h@manager    | [W 2017-08-24 15:13:21.646 JupyterHub utils:88] Failed to connect to http://jupyter-5d9c68c6c50ed3d02a2fcf54f63993b6-1:8888/user/testuser ([Errno -2] Name or service not known)
jupyterhub_jupyterhub.1.0fe08onaa24h@manager    | [W 2017-08-24 15:13:21.866 JupyterHub utils:88] Failed to connect to http://jupyter-5d9c68c6c50ed3d02a2fcf54f63993b6-1:8888/user/testuser ([Errno -2] Name or service not known)
jupyterhub_jupyterhub.1.0fe08onaa24h@manager    | [W 2017-08-24 15:13:22.082 JupyterHub utils:88] Failed to connect to http://jupyter-5d9c68c6c50ed3d02a2fcf54f63993b6-1:8888/user/testuser ([Errno -2] Name or service not known)
jupyterhub_jupyterhub.1.0fe08onaa24h@manager    | [W 2017-08-24 15:13:22.299 JupyterHub utils:88] Failed to connect to http://jupyter-5d9c68c6c50ed3d02a2fcf54f63993b6-1:8888/user/testuser ([Errno -2] Name or service not known)
jupyterhub_jupyterhub.1.0fe08onaa24h@manager    | [W 2017-08-24 15:13:22.512 JupyterHub utils:88] Failed to connect to http://jupyter-5d9c68c6c50ed3d02a2fcf54f63993b6-1:8888/user/testuser ([Errno -2] Name or service not known)
jupyterhub_jupyterhub.1.0fe08onaa24h@manager    | [W 2017-08-24 15:13:22.728 JupyterHub utils:88] Failed to connect to http://jupyter-5d9c68c6c50ed3d02a2fcf54f63993b6-1:8888/user/testuser ([Errno -2] Name or service not known)
jupyterhub_jupyterhub.1.0fe08onaa24h@manager    | [W 2017-08-24 15:13:22.946 JupyterHub utils:88] Failed to connect to http://jupyter-5d9c68c6c50ed3d02a2fcf54f63993b6-1:8888/user/testuser ([Errno -2] Name or service not known)
jupyterhub_jupyterhub.1.0fe08onaa24h@manager    | [W 2017-08-24 15:13:23.168 JupyterHub utils:88] Failed to connect to http://jupyter-5d9c68c6c50ed3d02a2fcf54f63993b6-1:8888/user/testuser ([Errno -2] Name or service not known)
jupyterhub_jupyterhub.1.0fe08onaa24h@manager    | [W 2017-08-24 15:13:23.383 JupyterHub utils:88] Failed to connect to http://jupyter-5d9c68c6c50ed3d02a2fcf54f63993b6-1:8888/user/testuser ([Errno -2] Name or service not known)
jupyterhub_jupyterhub.1.0fe08onaa24h@manager    | [W 2017-08-24 15:13:23.604 JupyterHub utils:88] Failed to connect to http://jupyter-5d9c68c6c50ed3d02a2fcf54f63993b6-1:8888/user/testuser ([Errno -2] Name or service not known)
jupyterhub_jupyterhub.1.0fe08onaa24h@manager    | [W 2017-08-24 15:13:23.821 JupyterHub utils:88] Failed to connect to http://jupyter-5d9c68c6c50ed3d02a2fcf54f63993b6-1:8888/user/testuser ([Errno -2] Name or service not known)
jupyterhub_jupyterhub.1.0fe08onaa24h@manager    | [W 2017-08-24 15:13:24.036 JupyterHub utils:88] Failed to connect to http://jupyter-5d9c68c6c50ed3d02a2fcf54f63993b6-1:8888/user/testuser ([Errno -2] Name or service not known)
jupyterhub_jupyterhub.1.0fe08onaa24h@manager    | [W 2017-08-24 15:13:24.264 JupyterHub utils:88] Failed to connect to http://jupyter-5d9c68c6c50ed3d02a2fcf54f63993b6-1:8888/user/testuser ([Errno -2] Name or service not known)
jupyterhub_jupyterhub.1.0fe08onaa24h@manager    | [W 2017-08-24 15:13:24.490 JupyterHub utils:88] Failed to connect to http://jupyter-5d9c68c6c50ed3d02a2fcf54f63993b6-1:8888/user/testuser ([Errno -2] Name or service not known)
jupyterhub_jupyterhub.1.0fe08onaa24h@manager    | [W 2017-08-24 15:13:24.706 JupyterHub utils:88] Failed to connect to http://jupyter-5d9c68c6c50ed3d02a2fcf54f63993b6-1:8888/user/testuser ([Errno -2] Name or service not known)
jupyterhub_jupyterhub.1.0fe08onaa24h@manager    | [W 2017-08-24 15:13:24.924 JupyterHub utils:88] Failed to connect to http://jupyter-5d9c68c6c50ed3d02a2fcf54f63993b6-1:8888/user/testuser ([Errno -2] Name or service not known)
jupyterhub_jupyterhub.1.0fe08onaa24h@manager    | [W 2017-08-24 15:13:25.143 JupyterHub utils:88] Failed to connect to http://jupyter-5d9c68c6c50ed3d02a2fcf54f63993b6-1:8888/user/testuser ([Errno -2] Name or service not known)
jupyterhub_jupyterhub.1.0fe08onaa24h@manager    | [W 2017-08-24 15:13:25.371 JupyterHub utils:88] Failed to connect to http://jupyter-5d9c68c6c50ed3d02a2fcf54f63993b6-1:8888/user/testuser ([Errno -2] Name or service not known)
jupyterhub_jupyterhub.1.0fe08onaa24h@manager    | [W 2017-08-24 15:13:25.591 JupyterHub utils:88] Failed to connect to http://jupyter-5d9c68c6c50ed3d02a2fcf54f63993b6-1:8888/user/testuser ([Errno -2] Name or service not known)
jupyterhub_jupyterhub.1.0fe08onaa24h@manager    | [W 2017-08-24 15:13:25.812 JupyterHub utils:88] Failed to connect to http://jupyter-5d9c68c6c50ed3d02a2fcf54f63993b6-1:8888/user/testuser ([Errno -2] Name or service not known)
jupyterhub_jupyterhub.1.0fe08onaa24h@manager    | [W 2017-08-24 15:13:26.024 JupyterHub utils:88] Failed to connect to http://jupyter-5d9c68c6c50ed3d02a2fcf54f63993b6-1:8888/user/testuser ([Errno -2] Name or service not known)
jupyterhub_jupyterhub.1.0fe08onaa24h@manager    | [W 2017-08-24 15:13:26.244 JupyterHub utils:88] Failed to connect to http://jupyter-5d9c68c6c50ed3d02a2fcf54f63993b6-1:8888/user/testuser ([Errno -2] Name or service not known)
jupyterhub_jupyterhub.1.0fe08onaa24h@manager    | [W 2017-08-24 15:13:26.458 JupyterHub utils:88] Failed to connect to http://jupyter-5d9c68c6c50ed3d02a2fcf54f63993b6-1:8888/user/testuser ([Errno -2] Name or service not known)
jupyterhub_jupyterhub.1.0fe08onaa24h@manager    | [W 2017-08-24 15:13:26.678 JupyterHub utils:88] Failed to connect to http://jupyter-5d9c68c6c50ed3d02a2fcf54f63993b6-1:8888/user/testuser ([Errno -2] Name or service not known)
jupyterhub_jupyterhub.1.0fe08onaa24h@manager    | [W 2017-08-24 15:13:26.895 JupyterHub utils:88] Failed to connect to http://jupyter-5d9c68c6c50ed3d02a2fcf54f63993b6-1:8888/user/testuser ([Errno -2] Name or service not known)
jupyterhub_jupyterhub.1.0fe08onaa24h@manager    | [W 2017-08-24 15:13:27.149 JupyterHub utils:88] Failed to connect to http://jupyter-5d9c68c6c50ed3d02a2fcf54f63993b6-1:8888/user/testuser ([Errno -2] Name or service not known)
jupyterhub_jupyterhub.1.0fe08onaa24h@manager    | [W 2017-08-24 15:13:27.419 JupyterHub utils:88] Failed to connect to http://jupyter-5d9c68c6c50ed3d02a2fcf54f63993b6-1:8888/user/testuser ([Errno -2] Name or service not known)
jupyterhub_jupyterhub.1.0fe08onaa24h@manager    | [W 2017-08-24 15:13:27.646 JupyterHub utils:88] Failed to connect to http://jupyter-5d9c68c6c50ed3d02a2fcf54f63993b6-1:8888/user/testuser ([Errno -2] Name or service not known)
jupyterhub_jupyterhub.1.0fe08onaa24h@manager    | [W 2017-08-24 15:13:27.860 JupyterHub utils:88] Failed to connect to http://jupyter-5d9c68c6c50ed3d02a2fcf54f63993b6-1:8888/user/testuser ([Errno -2] Name or service not known)

Unable to start a notebook....

I'm running Jupyterhub in a docker swarm, on an OpenStack base. I have 1 manager & 2 worker nodes.

I can happily create the 'Hub docker image, and log into it.... however that hub is not able to spawn notebooks.... and yes, the notebook has been pulled into the Swarm.

I'm getting the error: "json: cannot unmarshal string into Go value of type []string"

My docker file is:

FROM jupyterhub/jupyterhub
RUN apt-get update
RUN apt-get -y install python3-pip
# Temporary add these usful tools while developing
RUN apt-get -y install vim less net-tools
RUN useradd -ms /bin/bash test
RUN echo test:password | chpasswd
RUN git clone https://github.com/cassinyio/SwarmSpawner.git
WORKDIR SwarmSpawner
RUN pip install -r requirements.txt
RUN pip install .
WORKDIR /srv/jupyterhub
COPY config/jupyterhub_config.py /srv/jupyterhub/jupyterhub_config.py
CMD jupyterhub --debug > log.out 2>&1

and my config file is:

import subprocess
import os
import errno
import stat

c = get_config()
pwd = os.path.dirname(__file__)
c.JupyterHub.spawner_class = 'cassinyspawner.SwarmSpawner'
c.JupyterHub.ip = '0.0.0.0'
c.JupyterHub.hub_ip = '0.0.0.0'
c.JupyterHub.cleanup_servers = False
c.SwarmSpawner.start_timeout = 60 * 5
c.SwarmSpawner.jupyterhub_service_name = 'jupyterhub'
c.SwarmSpawner.networks = ["naas"]
notebook_dir = os.environ.get('NOTEBOOK_DIR') or '/home/test'
c.SwarmSpawner.notebook_dir = notebook_dir
mounts = [{'type' : 'volume',
           'source' : 'jupyterhub-user-{username}',
           'target' : notebook_dir}]
c.SwarmSpawner.container_spec = {
    'args' : '/usr/local/bin/start-singleuser.sh',
    'Image' : 'jupyterhub/singleuser',
    'mounts' : mounts
    }
c.SwarmSpawner.resource_spec = {
                'cpu_limit' : 1000,
                'mem_limit' : int(512 * 1e6),
                'cpu_reservation' : 1000,
                'mem_reservation' : int(512 * 1e6),
}

I start the hub with the following command:
docker service create --constraint 'node.role==manager' --network naas --name jupyterhub -p 8000:8000 -p 8001:8001 -p 8081:8081 --env DOCKER_HOST=192.168.1.11:2375 naasswarm/jupyterhub (where 192.168.1.11 is the outward facing IP of the swarm)

When I log in, I get the following error-stack:

[D 2017-04-21 14:39:40.317 JupyterHub swarmspawner:221] Getting Docker service 'jupyter-098f6bcd4621d373cade4e832627b4f6-1'
[I 2017-04-21 14:39:40.321 JupyterHub swarmspawner:229] Docker service 'jupyter-098f6bcd4621d373cade4e832627b4f6-1' is gone
[W 2017-04-21 14:39:40.321 JupyterHub swarmspawner:194] Docker service not found
[E 2017-04-21 14:39:40.353 JupyterHub web:1548] Uncaught exception POST /hub/login?next= (10.255.0.2)
    HTTPServerRequest(protocol='http', host='172.16.48.141:8000', method='POST', uri='/hub/login?next=', version='HTTP/1.1', remote_ip='10.255.0.2', headers={'Content-Type': 'application/x-www-form-urlencoded', 'X-Forwarded-Host': '172.16.48.141:8000', 'Accept-Language': 'en-US,en;q=0.5', 'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Firefox/52.0', 'Cookie': '_ga=GA1.1.103564032.1490776825; toggle=true', 'Upgrade-Insecure-Requests': '1', 'X-Forwarded-Proto': 'http', 'Content-Length': '31', 'Referer': 'http://172.16.48.141:8000/hub/login', 'X-Forwarded-Port': '8000', 'Connection': 'close', 'Accept-Encoding': 'gzip, deflate', 'Host': '172.16.48.141:8000', 'X-Forwarded-For': '10.255.0.2', 'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8'})
    Traceback (most recent call last):
      File "/opt/conda/lib/python3.5/site-packages/tornado/web.py", line 1469, in _execute
        result = yield result
      File "/opt/conda/lib/python3.5/site-packages/jupyterhub/handlers/login.py", line 83, in post
        yield self.spawn_single_user(user)
      File "/opt/conda/lib/python3.5/site-packages/jupyterhub/handlers/base.py", line 325, in spawn_single_user
        yield gen.with_timeout(timedelta(seconds=self.slow_spawn_timeout), f)
      File "/opt/conda/lib/python3.5/site-packages/jupyterhub/user.py", line 310, in spawn
        raise e
      File "/opt/conda/lib/python3.5/site-packages/jupyterhub/user.py", line 278, in spawn
        ip_port = yield gen.with_timeout(timedelta(seconds=spawner.start_timeout), f)
      File "/opt/conda/lib/python3.5/site-packages/cassinyspawner/swarmspawner.py", line 311, in start
        networks=networks)
      File "/opt/conda/lib/python3.5/concurrent/futures/_base.py", line 398, in result
        return self.__get_result()
      File "/opt/conda/lib/python3.5/concurrent/futures/_base.py", line 357, in __get_result
        raise self._exception
      File "/opt/conda/lib/python3.5/concurrent/futures/thread.py", line 55, in run
        result = self.fn(*self.args, **self.kwargs)
      File "/opt/conda/lib/python3.5/site-packages/cassinyspawner/swarmspawner.py", line 180, in _docker
        return m(*args, **kwargs)
      File "/opt/conda/lib/python3.5/site-packages/docker/utils/decorators.py", line 35, in wrapper
        return f(self, *args, **kwargs)
      File "/opt/conda/lib/python3.5/site-packages/docker/api/service.py", line 84, in create_service
        self._post_json(url, data=data, headers=headers), True
      File "/opt/conda/lib/python3.5/site-packages/docker/api/client.py", line 220, in _result
        self._raise_for_status(response)
      File "/opt/conda/lib/python3.5/site-packages/docker/api/client.py", line 216, in _raise_for_status
        raise create_api_error_from_http_exception(e)
      File "/opt/conda/lib/python3.5/site-packages/docker/errors.py", line 30, in create_api_error_from_http_exception
        raise cls(e, response=response, explanation=explanation)
    docker.errors.APIError: 500 Server Error: Internal Server Error ("json: cannot unmarshal string into Go value of type []string")

(and by dint of a bit of hacking, I can confirm that the

resp = yield self.docker('create_service',
                                     task_tmpl,
                                     name=self.service_name,
                                     networks=networks)

line is being fed the following data:

[I 2017-04-21 14:39:40.312 JupyterHub swarmspawner:307] Call Docker create_service task_template: {'ContainerSpec': {'Args': '/usr/local/bin/start-singleuser.sh',
                       'Command': None,
                       'Env': ['JPY_BASE_URL=/user/test',
                               'JPY_USER=test',
                               'JPY_COOKIE_NAME=jupyter-hub-token-test',
                               'JUPYTERHUB_CLIENT_ID=user-test',
                               'JPY_HUB_PREFIX=/hub/',
                               'JUPYTERHUB_API_TOKEN=e80c8e73010349bb85d846958b667bca',
                               'JUPYTERHUB_HOST=',
                               'NOTEBOOK_DIR=/home/test',
                               'JUPYTERHUB_OAUTH_CALLBACK_URL=/user/test/oauth_callback',
                               'JPY_API_TOKEN=e80c8e73010349bb85d846958b667bca',
                               'JPY_HUB_API_URL=http://jupyterhub:8081/hub/api'],
                       'Image': 'jupyterhub/singleuser',
                       'Mounts': [{'ReadOnly': False,
                                   'Source': 'jupyterhub-user-098f6bcd4621d373cade4e832627b4f6',
                                   'Target': '/home/test',
                                   'Type': 'volume'}]},
     'Resources': {'Limits': {'MemoryBytes': 512000000, 'NanoCPUs': 1000},
                   'Reservations': {'MemoryBytes': 512000000, 'NanoCPUs': 1000}}},
     name:'jupyter-098f6bcd4621d373cade4e832627b4f6-1',
     networks: ['naas']

)

I'm at a loss where the []string is, and actually what's tripping up....

Help?

mount type bind doesn't accept {username}

for mount with type 'bind' we can't add template via {username}, which doesn't permit any user specific isolation.

This code allows user specific isolation and templating with {username} for docker volumes:
https://github.com/cassinyio/SwarmSpawner/blob/master/cassinyspawner/swarmspawner.py#L302-L303

It can be used for mounting directories with template usernames for mount type with a very simple addition

https://www.dropbox.com/s/qx8u6e1p9aaa7f0/Screenshot%202017-03-23%2019.35.04.png?dl=0

its actually not a complete solution, because one has to create the hashed usernames apriori, which wouldn't make sense in a production setting where if an oauthenticator is used there can be many users and we can't know the names of all beforehand.

What's the Docker image used for JupyterHub?

I assume the SwarmSpawner is configured as part of a Docker service that runs JupyterHub. Could you share a sample configuration of that service, e.g. which base image is used?

Pulling images

Swarm directly controls the pulling of images, but JupyterHub should wait around 60 secs after you return ip and port of the spawned server.

So when you pull images that are not already available locally you can get some time-out error.

Kernel keeps restarting

When starting a new notebook we get the following error:

[I 2017-03-20 17:36:26.048 zzz sign:210] Writing notebook-signing key to /home/jovyan/.local/share/jupyter/notebook_secret
[W 2017-03-20 17:36:26.050 zzz manager:449] Notebook Accessing+elasticsearch+via+Python.ipynb is not trusted
[I 2017-03-20 17:36:27.821 zzz kernelmanager:89] Kernel started: 8525545b-0dfa-4735-8993-24bf4ea40337
[I 2017-03-20 17:36:30.822 zzz restarter:103] KernelRestarter: restarting kernel (1/5)
[I 2017-03-20 17:36:33.828 zzz restarter:103] KernelRestarter: restarting kernel (2/5)
[I 2017-03-20 17:36:36.833 zzz restarter:103] KernelRestarter: restarting kernel (3/5)
[W 2017-03-20 17:36:37.983 zzz handlers:232] Timeout waiting for kernel_info reply from 8525545b-0dfa-4735-8993-24bf4ea40337

The issue goes away if we switch back to using docker spawner.

name must be valid as a DNS name component

@barrachri

Issue:
Starting the notebook service via swarmspawner results in:

jupyterhub.1.ox704kq02e01@moby    | [I 2017-03-16 19:11:34.117 JupyterHub log:100] 200 GET /hub/login?next=%2Fhub%2Fuser%2Fanaqvi%2Fapi%2Fsessions%3F_%3D1489688263779 (@::ffff:10.255.0.3) 3.45ms
jupyterhub.1.ox704kq02e01@moby    | [I 2017-03-16 19:11:42.975 JupyterHub log:100] 302 GET / (@::ffff:10.255.0.3) 0.83ms
jupyterhub.1.ox704kq02e01@moby    | [I 2017-03-16 19:11:42.982 JupyterHub log:100] 302 GET /hub (@::ffff:10.255.0.3) 0.60ms
jupyterhub.1.ox704kq02e01@moby    | [I 2017-03-16 19:11:42.999 JupyterHub log:100] 302 GET /hub/ (@::ffff:10.255.0.3) 1.18ms
jupyterhub.1.ox704kq02e01@moby    | [I 2017-03-16 19:11:43.017 JupyterHub log:100] 200 GET /hub/login (@::ffff:10.255.0.3) 2.14ms
jupyterhub.1.ox704kq02e01@moby    | [I 2017-03-16 19:11:47.273 JupyterHub swarmspawner:260] Service 'jupyter-77e417a2b692b459f1deaac61b3bf5ba-1' is gone
jupyterhub.1.ox704kq02e01@moby    | [W 2017-03-16 19:11:47.273 JupyterHub swarmspawner:221] Service not found
jupyterhub.1.ox704kq02e01@moby    | [W 2017-03-16 19:11:47.810 JupyterHub swarmspawner:285] self.use_user_options:
jupyterhub.1.ox704kq02e01@moby    | [W 2017-03-16 19:11:47.810 JupyterHub swarmspawner:286] {}
jupyterhub.1.ox704kq02e01@moby    | [I 2017-03-16 19:11:47.819 JupyterHub swarmspawner:260] Service 'jupyter-77e417a2b692b459f1deaac61b3bf5ba-' is gone
jupyterhub.1.ox704kq02e01@moby    | [E 2017-03-16 19:11:48.738 JupyterHub user:282] Unhandled error starting anaqvi's server: 500 Server Error: Internal Server Error for url: http+docker://localunixsocket/v1.26/services/create ("rpc error: code = 3 desc = name must be valid as a DNS name component")

Explanation:
The hub can't find the service jupyter-77e417a2b692b459f1deaac61b3bf5ba-1 associated with a username anaqvi. It checks user options for a server_name:

https://github.com/cassinyio/SwarmSpawner/blob/master/cassinyspawner/swarmspawner.py#L292-L293

however it doesn't assign a value to server_name if there are no user options. That is if the c.SwarmSpawner.use_user_options = False in jupyterhub_config.py then it doesn't assign
the server_name anything. This results in the DNS name to be:

jupyter-77e417a2b692b459f1deaac61b3bf5ba-

which is against the regex for DNS names in Docker ^[a-zA-Z0-9](?:[-_]*[A-Za-z0-9]+)*$ as it contains a dash at the end.

This can be confirmed by changing this line:
https://github.com/cassinyio/SwarmSpawner/blob/master/cassinyspawner/swarmspawner.py#L156

from

        return "{}-{}-{}".format(self.service_prefix,

to

        return "{}-{}{}".format(self.service_prefix,

Possible solution:
I have a minor commit that fixes this issue by assigning the value of 1 to self.server_name after the if statement:
MAliNaqvi@49fb55b

Add a jupyter_config.py example

I always get failed because of the wrong configure,. Can you show me your jupyterhub_config.py about the swarmspawner's period ?

Add `placement` as a regular config option and not only for user_options

As with other options we can choose for configuring the swarm deployment, I feel as though placement should be available as a configuration option not only to be set through user_options.

An example of what I mean...

As a configuration level option, it currently does not work:
jupyterhub_config.py

[...]
c.JupyterHub.spawner_class = 'cassinyspawner.SwarmSpawner'
c.SwarmSpawner.placement = ["node.role == worker"]
[...]

Launch logs:

jupyterhub_jupyterhub.1.m9kw855bogr9@manager    | Adding user testuser
jupyterhub_jupyterhub.1.m9kw855bogr9@manager    | [I 2017-08-11 15:38:18.479 JupyterHub app:724] Loading cookie_secret from /srv/jupyterhub/jupyterhub_cookie_secret
jupyterhub_jupyterhub.1.m9kw855bogr9@manager    | [I 2017-08-11 15:38:18.545 JupyterHub app:892] Not using whitelist. Any authenticated user will be allowed.
jupyterhub_jupyterhub.1.m9kw855bogr9@manager    | [W 2017-08-11 15:38:18.560 JupyterHub configurable:168] Config option `placement` not recognized by `SwarmSpawner`.
jupyterhub_jupyterhub.1.m9kw855bogr9@manager    | [I 2017-08-11 15:38:18.636 JupyterHub swarmspawner:229] Docker service 'jupyter-90613d4f8204b906ead85556f075fd71-1' is gone
jupyterhub_jupyterhub.1.m9kw855bogr9@manager    | [W 2017-08-11 15:38:18.636 JupyterHub swarmspawner:194] Docker service not found
jupyterhub_jupyterhub.1.m9kw855bogr9@manager    | [W 2017-08-11 15:38:18.640 JupyterHub configurable:168] Config option `placement` not recognized by `SwarmSpawner`.
jupyterhub_jupyterhub.1.m9kw855bogr9@manager    | [I 2017-08-11 15:38:18.644 JupyterHub swarmspawner:229] Docker service 'jupyter-2f4c77557cd6c786d40e488583d77e79-1' is gone
jupyterhub_jupyterhub.1.m9kw855bogr9@manager    | [W 2017-08-11 15:38:18.645 JupyterHub swarmspawner:194] Docker service not found
jupyterhub_jupyterhub.1.m9kw855bogr9@manager    | [I 2017-08-11 15:38:18.652 JupyterHub app:1453] Hub API listening on http://0.0.0.0:8081/hub/
jupyterhub_jupyterhub.1.m9kw855bogr9@manager    | [I 2017-08-11 15:38:18.655 JupyterHub app:1176] Starting proxy @ http://0.0.0.0:443/
jupyterhub_jupyterhub.1.m9kw855bogr9@manager    | 15:38:18.859 - info: [ConfigProxy] Proxying https://0.0.0.0:443 to http://127.0.0.1:8081
jupyterhub_jupyterhub.1.m9kw855bogr9@manager    | 15:38:18.864 - info: [ConfigProxy] Proxy API at http://127.0.0.1:444/api/routes
jupyterhub_jupyterhub.1.m9kw855bogr9@manager    | [I 2017-08-11 15:38:18.964 JupyterHub app:1485] JupyterHub is now running at http://127.0.0.1:443/

But it does work if I add it under user_options and set to use user_options to true
jupyterhub_config.py

[...]
c.JupyterHub.spawner_class = 'cassinyspawner.SwarmSpawner'
# c.SwarmSpawner.placement = ["node.role == worker"]
c.SwarmSpawner.use_user_options = True
user_options = {
    'placement' : ["node.role == worker"]
}
[...]

Launch logs:

jupyterhub_jupyterhub.1.879snisqard1@manager    | Adding user testuser
jupyterhub_jupyterhub.1.879snisqard1@manager    | [I 2017-08-11 15:47:19.836 JupyterHub app:724] Loading cookie_secret from /srv/jupyterhub/jupyterhub_cookie_secret
jupyterhub_jupyterhub.1.879snisqard1@manager    | [I 2017-08-11 15:47:19.893 JupyterHub app:892] Not using whitelist. Any authenticated user will be allowed.
jupyterhub_jupyterhub.1.879snisqard1@manager    | [I 2017-08-11 15:47:19.973 JupyterHub swarmspawner:229] Docker service 'jupyter-90613d4f8204b906ead85556f075fd71-1' is gone
jupyterhub_jupyterhub.1.879snisqard1@manager    | [W 2017-08-11 15:47:19.973 JupyterHub swarmspawner:194] Docker service not found
jupyterhub_jupyterhub.1.879snisqard1@manager    | [W 2017-08-11 15:47:19.975 JupyterHub app:1097] isuftin not running.
jupyterhub_jupyterhub.1.879snisqard1@manager    | [I 2017-08-11 15:47:19.983 JupyterHub swarmspawner:229] Docker service 'jupyter-2f4c77557cd6c786d40e488583d77e79-1' is gone
jupyterhub_jupyterhub.1.879snisqard1@manager    | [W 2017-08-11 15:47:19.984 JupyterHub swarmspawner:194] Docker service not found
jupyterhub_jupyterhub.1.879snisqard1@manager    | [I 2017-08-11 15:47:19.996 JupyterHub app:1453] Hub API listening on http://0.0.0.0:8081/hub/
jupyterhub_jupyterhub.1.879snisqard1@manager    | [I 2017-08-11 15:47:19.999 JupyterHub app:1176] Starting proxy @ http://0.0.0.0:443/
jupyterhub_jupyterhub.1.879snisqard1@manager    | 15:47:20.208 - info: [ConfigProxy] Proxying https://0.0.0.0:443 to http://127.0.0.1:8081
jupyterhub_jupyterhub.1.879snisqard1@manager    | 15:47:20.213 - info: [ConfigProxy] Proxy API at http://127.0.0.1:444/api/routes

Force push?

Hi,

Did you do a force push at some point? My history has diverged at commit 59e92f7 which is missing from the main repo now.

Correct the example

            'cpu_limit' : 1, # (int) – CPU limit in units of 10^9 CPU shares.
            'mem_limit' : 512 * 1e6, # (int) – Memory limit in Bytes.
            'cpu_reservation' : 1000, # (int) – CPU reservation in units of 10^9 CPU shares.
            'mem_reservation' : 512 * 1e6, # (int) – Memory reservation in Bytes

They should be int (not float)

Problems with driver_configs

Commit c37c79f added support for driver_configs in mounts. But this actually made driver_configs stop working for me. I think there are two reasons. First, if m['driver_config'] is a docker.types.services.DriverConfig, then it is a dict with key Options not "options". And not every DriverConfig will have a "device" option; I think driver options vary by driver.

So this commit leads to KeyErrors for me. If I revert it, then driver_configs work again.

start_timeout and http_timeout ignored

I have set up c.SwarmSpawner.start_timeout=60 and c.SwarmSpawner.http_timeout=60 in jupyterhub_config.py, but it keeps timing out at 10 s. I could not see in the code that it overwrites the value from Spawner class. Is it not yet implemented?

SwarmSpawner Expects Different Format for inspect_service

Using what I think is a pretty standard installation... I can't quite get it to work.

jupyterhub.1.2fird3hjs8nf@bluebonnet    |     HTTPServerRequest(protocol='http', host='bluebonnet.eecs.umich.edu:8000', method='POST', uri='/hub/login?next=', version='HTTP/1.1', remote_ip='::ffff:10.255.0.2', headers={'Content-Length': '45', 'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36', 'X-Forwarded-Proto': 'http', 'Accept-Language': 'en-US,en;q=0.8', 'Cache-Control': 'max-age=0', 'Cookie': 'AMCV_4D6368F454EC41940A4C98A6%40AdobeOrg=-1330315163%7CMCIDTS%7C17238%7CMCMID%7C08480496867647127562896913634378802334%7CMCAAMLH-1489883949%7C7%7CMCAAMB-1489883949%7CNRX38WO0n5BH8Th-nqAG_A%7CMCOPTOUT-1489286349s%7CNONE%7CMCAID%7CNONE; s_pers=%20c19%3Dpr%253Apublications%253Aview%7C1489281780740%3B%20v68%3D1489279980474%7C1489281780743%3B%20v8%3D1489279989599%7C1583887989599%3B%20v8_s%3DFirst%2520Visit%7C1489281789599%3B; __utma=7269686.1234911664.1489279149.1489546566.1489546566.1; __utmz=7269686.1489546566.1.1.utmcsr=www-personal.umich.edu|utmccn=(referral)|utmcmd=referral|utmcct=/; _ceg.s=otpdxa; _ceg.u=otpdxa; gwlob=on; _ga=GA1.2.1234911664.1489279149; _gid=GA1.2.2068178188.1501350351', 'Upgrade-Insecure-Requests': '1', 'Content-Type': 'application/x-www-form-urlencoded', 'Referer': 'http://bluebonnet.eecs.umich.edu:8000/hub/login', 'Accept-Encoding': 'gzip, deflate', 'X-Forwarded-For': '::ffff:10.255.0.2', 'X-Forwarded-Host': 'bluebonnet.eecs.umich.edu:8000', 'Connection': 'close', 'X-Forwarded-Port': '8000', 'Host': 'bluebonnet.eecs.umich.edu:8000', 'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8', 'Origin': 'http://bluebonnet.eecs.umich.edu:8000'})
jupyterhub.1.2fird3hjs8nf@bluebonnet    |     Traceback (most recent call last):
jupyterhub.1.2fird3hjs8nf@bluebonnet    |       File "/opt/conda/lib/python3.5/site-packages/tornado/web.py", line 1511, in _execute
jupyterhub.1.2fird3hjs8nf@bluebonnet    |         result = yield result
jupyterhub.1.2fird3hjs8nf@bluebonnet    |       File "/opt/conda/lib/python3.5/site-packages/jupyterhub/handlers/login.py", line 99, in post
jupyterhub.1.2fird3hjs8nf@bluebonnet    |         yield self.spawn_single_user(user)
jupyterhub.1.2fird3hjs8nf@bluebonnet    |       File "/opt/conda/lib/python3.5/site-packages/jupyterhub/handlers/base.py", line 347, in spawn_single_user
jupyterhub.1.2fird3hjs8nf@bluebonnet    |         yield gen.with_timeout(timedelta(seconds=self.slow_spawn_timeout), f)
jupyterhub.1.2fird3hjs8nf@bluebonnet    |       File "/opt/conda/lib/python3.5/site-packages/jupyterhub/user.py", line 310, in spawn
jupyterhub.1.2fird3hjs8nf@bluebonnet    |         raise e
jupyterhub.1.2fird3hjs8nf@bluebonnet    |       File "/opt/conda/lib/python3.5/site-packages/jupyterhub/user.py", line 278, in spawn
jupyterhub.1.2fird3hjs8nf@bluebonnet    |         ip_port = yield gen.with_timeout(timedelta(seconds=spawner.start_timeout), f)
jupyterhub.1.2fird3hjs8nf@bluebonnet    |       File "/srv/jupyterhub/SwarmSpawner/cassinyspawner/swarmspawner.py", line 324, in start
jupyterhub.1.2fird3hjs8nf@bluebonnet    |         for line in service['Config']['Env']:
jupyterhub.1.2fird3hjs8nf@bluebonnet    |     KeyError: 'Config'

It looks like docker-py is returning it in this format, which doesn't contain a top level 'Config' object.

  "Version": {
    "Index": 141
  },
  "UpdatedAt": "2017-07-29T17:55:52.91611666Z",
  "Endpoint": {
    "VirtualIPs": [
      {
        "NetworkID": "nvtqnxhtejra3ofdgkeae501m",
        "Addr": "10.0.0.6/24"
      }
    ],
    "Spec": {}
  },
  "CreatedAt": "2017-07-29T17:55:52.843555567Z",
  "ID": "8dkpq7htzhlt8tfisykzrbize",
  "Spec": {
    "Mode": {
      "Replicated": {
        "Replicas": 1
      }
    },
    "Labels": {},
    "Networks": [
      {
        "Target": "nvtqnxhtejra3ofdgkeae501m"
      }
    ],
    "TaskTemplate": {
      "ForceUpdate": 0,
      "Runtime": "container",
      "ContainerSpec": {
        "Image": "jupyterhub/singleuser",
        "Env": [
          "JUPYTERHUB_API_TOKEN=b05caf788dd74c91af51fb86834348e9",
          "JUPYTERHUB_OAUTH_CALLBACK_URL=/user/jovyan/oauth_callback",
          "JUPYTERHUB_CLIENT_ID=user-jovyan",
          "JPY_HUB_API_URL=http://jupyterhub:8081/hub/api",
          "JPY_USER=jovyan",
          "JUPYTERHUB_HOST=",
          "JPY_COOKIE_NAME=jupyter-hub-token-jovyan",
          "JPY_BASE_URL=/user/jovyan",
          "JPY_HUB_PREFIX=/hub/",
          "JPY_API_TOKEN=b05caf788dd74c91af51fb86834348e9"
        ]
      }
    },
    "Name": "jupyter-45bfb35c3cbdca8d0c12590895195086-1"
  }
}

For reference, my jupyterhub_config.py is

import os
network_name = 'jupyterhub'

c.JupyterHub.log_level = 'DEBUG'

c.JupyterHub.spawner_class = 'cassinyspawner.SwarmSpawner'
c.JupyterHub.hub_ip = "0.0.0.0"
c.JupyterHub.proxy_api_ip = "0.0.0.0"
c.SwarmSpawner.jupyterhub_service_name = "jupyterhub"
c.SwarmSpawner.networks = ["jupyterhub"]
c.SwarmSpawner.container_spec = {
                  # The command to run inside the service
                  # 'args' : ['/usr/local/bin/start-singleuser.sh'], # (list)
                  'Image' : 'jupyterhub/singleuser',
                  'mounts' : []
          }

#c.JupyterHub.spawner_class = 'dockerspawner.DockerSpawner'
#c.DockerSpawner.use_internal_ip = True
#c.DockerSpawner.network_name = network_name
# Pass the network name as argument to spawned containers
#c.DockerSpawner.extra_host_config = { 'network_mode': network_name }
#c.DockerSpawner.extra_start_kwargs = { 'network_mode': network_name }


# User containers will access hub by container name on the Docker network
#c.JupyterHub.hub_ip = 'jupyterhub'
#c.JupyterHub.hub_port = 8000

# The docker instances need access to the Hub, so the default loopback port doesn't work:
#c.JupyterHub.hub_ip = 'jupyterhub'
#c.JupyterHub.ip = 'jupyterhub'
#c.DockerSpawner.container_ip = "0.0.0.0"
#c.DockerSpawner.hub_ip_connect = 'jupyterhub'


c.JupyterHub.authenticator_class = 'dummyauthenticator.DummyAuthenticator'
c.DummyAuthenticator.password = "your strong password"

and the image for Jupyterhub is derived from jupyterhub/jupyterhub.

Docker Service not found

Hi,
I tried out your SwarmSpawner, extending the Deploy Docker Repo and I quickly ran into some problems.

First of all, your example jupyter_config.py file must be updated on line 35, where you still use string args. Coming to my main problem, after install SwarmSpawner and launching the Jupyterhub as a Service in Docker Swarm everything seems to be okay. But when I login I get some weird errors which look like this:

JupyterHub is now running at http://127.0.0.1:443/
Getting Docker service 'jupyter-3b4bd7fa0504fe5d059e1b0f2655c56b-1'
Docker service 'jupyter-3b4bd7fa0504fe5d059e1b0f2655c56b-1' is gone
Docker service not found
user_options:
Getting Docker service 'jupyter-3b4bd7fa0504fe5d059e1b0f2655c56b-1'
Docker service 'jupyter-3b4bd7fa0504fe5d059e1b0f2655c56b-1' is gone
Unhandled error starting wakonp's server: 500 Server Error: Internal Server Error ("rpc error: code = 3 desc = invalid cpu value 1e-06: Must be at least 0.001")
Getting Docker service 'jupyter-3b4bd7fa0504fe5d059e1b0f2655c56b-1'
Docker service 'jupyter-3b4bd7fa0504fe5d059e1b0f2655c56b-1' is gone
Docker service not found
Uncaught exception POST /hub/login?next= (10.255.0.2)

This error tells me two things. The JupyterHub Service can't find the generated NotebookService and that my server has an internal server error because of an invalid cpu value. Lets check out my jupyterhub_config.py file.

import os 
import subprocess
import os
import errno
import stat

c = get_config()
pwd = os.path.dirname(__file__)

c.JupyterHub.ip = '0.0.0.0'
c.JupyterHub.port = 443
c.JupyterHub.hub_ip = '0.0.0.0'

c.JupyterHub.spawner_class = 'cassinyspawner.SwarmSpawner'
c.JupyterHub.cleanup_servers = False

c.SwarmSpawner.start_timeout = 60 * 5
c.SwarmSpawner.jupyterhub_service_name = 'jupyterhub_jupyterhub'


c.SwarmSpawner.use_user_options = False
c.SwarmSpawner.networks = ["jupyterhub-network"]
notebook_dir = os.environ.get('NOTEBOOK_DIR') or '/home/jovyan/work'
c.SwarmSpawner.notebook_dir = notebook_dir
mounts = [{'type' : 'volume',
           'source' : 'jupyterhub-user-{username}',
           'target' : notebook_dir}]	   
c.SwarmSpawner.container_spec = {
                  'args' : ['start-singleuser.sh'],
                  'Image' :'jupyter/scipy-notebook:bb222f49222e',
                  'mounts' : mounts
          }
c.SwarmSpawner.resource_spec = {
                'cpu_limit' : 1000, # (int) – CPU limit in units of 10^9 CPU shares.
                'mem_limit' : int(512 * 1e6), # (int) – Memory limit in Bytes.
                'cpu_reservation' : 1000, # (int) – CPU reservation in units of 10^9 CPU shares.
                'mem_reservation' : int(512 * 1e6), # (int) – Memory reservation in Bytes
                }

As you can see I use the your config for c.SwarmSpawner.resouce_spec and I set c.SwarmSpawner.use_user_options explicit to false. So I dont understand where the error Internal Server Error ("rpc error: code = 3 desc = invalid cpu value 1e-06: Must be at least 0.001") comes from.

Some other interesting files:
Dockerfile.Jupyterhub

# Copyright (c) Jupyter Development Team.
# Distributed under the terms of the Modified BSD License.
FROM jupyterhub/jupyterhub-onbuild:0.7.2


# Install dockerspawner and its dependencies
#    oauthenticator==0.5.* \
# dockerspawner==0.7.* \
RUN /opt/conda/bin/pip install \
	jupyterhub-ldapauthenticator

RUN git clone https://github.com/cassinyio/SwarmSpawner
WORKDIR ./SwarmSpawner
RUN pip install -r requirements.txt
RUN python setup.py install
WORKDIR ..

# install docker on the jupyterhub container
RUN wget https://get.docker.com -q -O /tmp/getdocker && \
    chmod +x /tmp/getdocker && \
    sh /tmp/getdocker

# Copy TLS certificate and key
ENV SSL_CERT /srv/jupyterhub/secrets/jupyterhub.crt
ENV SSL_KEY /srv/jupyterhub/secrets/jupyterhub.key
COPY ./secrets/*.crt $SSL_CERT
COPY ./secrets/*.key $SSL_KEY
RUN chmod 700 /srv/jupyterhub/secrets && \
    chmod 600 /srv/jupyterhub/secrets/*

COPY ./userlist /srv/jupyterhub/userlist

EXPOSE 8080
EXPOSE 8081

Docker-compose.yml (I run it with docker stack deploy -c docker-compose.yml jupyterhub, so the service name of Jupyterhub will be jupyterhub_jupyterhub)

version: "3.2"

services:
  jupyterhub:
    build:
      context: .
      dockerfile: Dockerfile.jupyterhub
    image: walki12/jupyterhub
    container_name: jupyterhub
    volumes:
      - type: bind
        source: /var/run/docker.sock
        target: /var/run/docker.sock
      - type: volume
        source: data
        target: /data
        volume: 
          nocopy: true
    ports:
      - "443:443"
    env_file:
      - .env
    deploy:
      mode: replicated
      replicas: 1
      restart_policy:
        condition: on-failure
        delay: 5s
        max_attempts: 3
        window: 120s
      update_config:
        parallelism: 1
        delay: 10s
        failure_action: continue
        monitor: 60s
        max_failure_ratio: 0.3
    command: >
      jupyterhub -f /srv/jupyterhub/jupyterhub_config.py

volumes:
  data:
    external:
      name: jupyterhub-data

networks:
  default:
    external: 
      name: jupyterhub-network

The Volume jupyterhub-data is available and the jupyterhub-network is an attachable overlay network.

Docker version 17.06.0-ce, build 02c1d87

I don't understand why this configuration does not work correctly and I really appreciate some decent explanation for this scenario.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.