Multiple Node Environment for Jupyterhub

[Sudipto Goswami]

I am having two hosts over which I am planning to build up an environment with Jupyterhub running on 2 nodes. Out of this, one node will be a docker server and the other to be base jupyerhub server. The main idea behind configuring this, is to establish a load balancing mechanism as well as clustering. Based on average load on the server, the user will be routed to a dedicated host for routing.

Can you please guide me on this configuration.

Is this on one host, I shall be configuring docker spawner and on the other node normal Jupyerhub server. If so how the configuration will be define. Also do I need to define Clustering at Unix OS level also?

Please suggest on this.

Regards,
Sudipta.

[MinRK]

If you are already using Docker, Docker Swarm may be a good choice, which does its own distributing of containers across machines. You can generally use DockerSpawner unmodified with Swarm.

-MinRK

--
You received this message because you are subscribed to the Google Groups "Project Jupyter" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jupyter+u...@googlegroups.com.
To post to this group, send email to jup...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jupyter/68fa42fe-bdc8-421c-b0ca-05baa6fbd1de%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Sudipto Goswami]

Thanks for your response.

The request over here based on server load, the user's request for spawning the notebook's will be routed to a server which is less busy. 

Consider the scenario like this:

We are having two nodes (A and B). So if we install Jupyterhub along with the dependencies we will be having two specific URL's one will be pointing to node A and the other to node B. Now additionally we add docker engines on both the nodes (A and B). We modify the Jupyterhub config on both the nodes to point to the docker i.e. c.JupyterHub.spawner_class = 'dockerspawner.DockerSpawner'.

Still this configuration will be having two sets of Jupyetrhub URL (Node A and Node B).

The doubt over here:

1. How do the environment figure out on the clustering part?

2. Where do I update the configurations on this.

We are planning to use Jupyterhub on larger scale at our organization. We already have configured on a single node environment. Now we are planning to scale out. Can you kindly help us on this.

--
You received this message because you are subscribed to a topic in the Google Groups "Project Jupyter" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jupyter/IBjkeMVLlGE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jupyter+u...@googlegroups.com.

To post to this group, send email to jup...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jupyter/CAHNn8BX-xris90vFObRKytjgRzB8Me_eOb4RZpyAD%3DBca2qHfQ%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Thanks and Regards,
Sudipta Goswami.