Possibility to separate the JupyterLab Web and Server

[Song Liu]

Hi,

Just like the Google Colab user experience, is that possible to separate the JupyterLab Web and Backend ?

That is the notebook viewing and editing functionality could be provided by the JupyterLab web service just with minimum resource consumption.

When running the Python code it could send it to a remote JupyterLab backend (Server and Kernel).

Thanks,

Song

[Jason Grout]

Absolutely. We wrote each part of JupyterLab as a separate component, and it should totally be possible to write just a notebook viewer and editor with JupyterLab components.

Jason

--
You received this message because you are subscribed to the Google Groups "Project Jupyter" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jupyter+u...@googlegroups.com.
To post to this group, send email to jup...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jupyter/d9872648-8909-4ad8-84f7-853ae8c81460%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Song Liu]

Thanks, could you kindly clarify which JupyterLab components could be used to compose a notebook viewer & editor ?

On Saturday, April 27, 2019 at 10:06:22 PM UTC+8, Jason Grout wrote:

Absolutely. We wrote each part of JupyterLab as a separate component, and it should totally be possible to write just a notebook viewer and editor with JupyterLab components.

Jason

On Sat, Apr 27, 2019 at 8:50 AM Song Liu <song...@gmail.com> wrote:

Hi,

Just like the Google Colab user experience, is that possible to separate the JupyterLab Web and Backend ?

That is the notebook viewing and editing functionality could be provided by the JupyterLab web service just with minimum resource consumption.

When running the Python code it could send it to a remote JupyterLab backend (Server and Kernel).

Thanks,

Song

--
You received this message because you are subscribed to the Google Groups "Project Jupyter" group.

To unsubscribe from this group and stop receiving emails from it, send an email to jup...@googlegroups.com.

[Jason Grout]

I would look at the JupyterLab notebook example. That example does require a backend, but does get you started in seeing the sorts of components needed to render/edit a notebook:

https://github.com/jupyterlab/jupyterlab/tree/master/examples/notebook

Thanks,

Jason

To unsubscribe from this group and stop receiving emails from it, send an email to jupyter+u...@googlegroups.com.

To post to this group, send email to jup...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jupyter/75cefc72-3b75-481e-8c47-d766703ec39e%40googlegroups.com.

[Song Liu]

So the official `JupyerLab Notebook` implementation is the https://github.com/jupyterlab/jupyterlab/tree/master/jupyterlab/jupyterlab ?

and the package "packages/notebook" is used ?

Is there any code structure documentation for JupyterLab ?

[Song Liu]

And the https://github.com/jupyter/notebook will be used for JupyterLab as it is without any modification ?

or what's the relationship between jupyter/notebook and jupyterlab/jupyterlab project ?

Many thanks :)

[Song Liu]

It seems that examples/notebook from JupyterLab project it is still launching a Notebook App Server which would provide the Web and Backend functions.

So that actually I am looking for a deployment separation between "static (html,js)" and the backend server, where:

- the web server provides the viewing and editing (+ save to storage) function

- the backend server is created on demand for running

[Roland Weber]

On Sunday, April 28, 2019 at 8:23:13 PM UTC+2, Song Liu wrote:

So that actually I am looking for a deployment separation between "static (html,js)" and the backend server, where:

- the web server provides the viewing and editing (+ save to storage) function

- the backend server is created on demand for running

There's Jupyter Kernel Gateway and Jupyter Enterprise Gateway for running kernels remotely. The nb2kg component ties a notebook server to the remote kernels. I don't know if there's something similar for JupyterLab already, or if nb2kg works there as well.

You should keep in mind though that notebook extensions will not work in this scenario. The extensions that users install are put on the backend server running the kernels, but the browser will try to fetch the static files of the extensions from the web server.

cheers,

  Roland

[Song Liu]

Thanks for sharing.

https://github.com/jupyter/enhancement-proposals/blob/master/jupyter-server/jupyter-server.md#separate-jupyter_server-repository

This separation is to be easy to interact with different frontend (the dashboards, nteract, JupyterLab etc.)

But my requirement is for the editing and running separation, the gateway looks like a solution.

[Jason Grout]

The jupyterlab examples/notebook example does use a notebook server backend (like I mentioned). It provides a starting point for you to see the sorts of components needed to build a front-end only solution for viewing/editing notebooks. Someone will still need to write a front-end thing like you are asking about.

Thanks,

Jason

--

You received this message because you are subscribed to the Google Groups "Project Jupyter" group.

To unsubscribe from this group and stop receiving emails from it, send an email to jupyter+u...@googlegroups.com.

To post to this group, send email to jup...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jupyter/283939d7-fdc1-4179-bbda-af9b7f36098c%40googlegroups.com.

[Jason Grout]

On Sun, Apr 28, 2019 at 11:08 AM Song Liu <song...@gmail.com> wrote:

And the https://github.com/jupyter/notebook will be used for JupyterLab as it is without any modification ?

or what's the relationship between jupyter/notebook and jupyterlab/jupyterlab project ?

jupyter/notebook is the "classic notebook" web application that has been around for a long time now. It consists of both the backend server and the frontend javascript.

jupyter/jupyterlab is a complete reimplementation of the frontend (it does not use the jupyter/notebook javascript), but still uses the jupyter/notebook backend server.

Jason

[Brian Granger]

What you are describing is possible, but there are a set of
constraints and trade-offs that may not give the experience you want.
Colab faces these same constraints and tradeoffs.

* JupyterLab/classic Notebook have a range of different service APIs -
kernels, contents (file/dirs), terminals, kernel specs (kernel
metadata), etc. The standard way of operating Jupyter is to deploy all
of these services in the same server, and to also serve the frontend
JS/HTML/CSS from that same server. In the abstraction, the
architecture certainly allows you to deploy different services on
different servers. However in practice, it is probably not what you
want to do. The reason is that the different services have coupling
that directly relates to the overall usability:

* Kernels and the file system are coupled, as a kernel needs to see
the same file system - to import other files sitting next to the
notebook, and to access co-located data files. So you can run the
kernel service separately, but it won't be useful for real work unless
you also figure out how to mount the same file system for that kernel.
* Terminals and the file system are coupled as users typically want to
see the real file system in the terminal.
* The frontend and kernels also have a coupling, because many kernel
libraries require frontend extensions to be installed.

These are exactly the tradeoffs made by colab, and they severely
limits its usefulness for real work. But I do think these trade offs
do make sense for some usage cases. Right now, the code base isn't
optimized for this more decoupled usage case, but we have different
efforts moving in this direction.

There is work to separate out the jupyter server and de coupled its
components here:

https://github.com/jupyter/jupyter_server

Unless I am misunderstanding your question, I don't think you are
looking to make changes to the frontend though - so I think the stock
JupyterLab or classic notebook would work fine. What you want is a
different way of factoring the server side of things. Probably the
most flexible way of refactoring the server would be to begin to use
the components of jupyter server, running in separate containers,
managed by something like docker compose with a traefik reverse proxy
in front. I would love to see explorations along these lines.

I should also note that the Jupyter enterprise gateway might help in this work:

https://github.com/jupyter/kernel_gateway

Hope this helps!

> --
> You received this message because you are subscribed to the Google Groups "Project Jupyter" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to jupyter+u...@googlegroups.com.
> To post to this group, send email to jup...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/jupyter/d9872648-8909-4ad8-84f7-853ae8c81460%40googlegroups.com.

> For more options, visit https://groups.google.com/d/optout.

--
Brian E. Granger
Associate Professor of Physics and Data Science
Cal Poly State University, San Luis Obispo
@ellisonbg on Twitter and GitHub
bgra...@calpoly.edu and elli...@gmail.com

[Song Liu]

Thanks Brian for your detailed explanation.

Yes, I don't want to invent another FrontEnd for Jupyter Notebook.

I understand the separation effort 's motivation is targeting for writing / interacting with a new Jupyter frondend easily and gracefully, is that right ?

My motivation is the resource optimization for the Jupyter Notebook user experience (just like Google Colab), that:

- notebook viewing / editing is very rapid and lightweight, but might needs the plugins to work

- notebook executing is resource consuming so the kernel would be started at this time

Based on the structure "FrondEnd --> Backend -> Kernel", there are two approaches to achieve this:

Approach 1: separate frondend and backend from "Jupyter Notebook"

I guess the Google Colab just take this approach since that it could just work with my local "classic Jupyter Notebook" by the Rest/Websocket API.

We don't want to follow this approach the main reason is we don't want to create another UI really.

Approach 2: separate the kernel and backend deployment

- a JupyterLab or classic Jupyter Notebook app process would be created for every notebook, assume this is some kind of lower resource consumption

- a remote kernel would be launched only when execute the code in Notebook, the kernel management might leverage the Kernel Gateway / Enterprise 's work

but this requires the Jupyer Notebook could be launched without kernel started, it looks like https://github.com/jupyter/notebook/issues/3170 is still under discussion.

We would prefer approach 2.

If I understand correclty, one of your suggestion is about trying to "deploying jupyter servers in separate containers + using the JupyterLab or classic notebook as it is".

I understand this requires the modification in "JupyterLab or classic notebook" to delegate api request to the jupyter servers ?

> To unsubscribe from this group and stop receiving emails from it, send an email to jup...@googlegroups.com.

[Song Liu]

Also, about the separation looks like a direction in the community, but there are not some progress for a long time.

and I see there are two jupyter server:

- jupyter/jupyter_server

- jupyterlab/jupyterlab_server

I feel that the refactor want to be done under the new group "jupyterlab", but jupyterlab is still using something from old group "jupyter". It would confuse if we want to contribute something.

Thanks,

Song

[Jason Grout]

It sounds like basically you just want, when opening a notebook, for the kernel to only be launched when you execute the first cell, instead of launching the kernel automatically when the notebook opens? We discussed this when implementing JupyterLab, but decided to stay with the behavior from classic notebook. However, we've been discussing recently changing the behavior in JupyterLab to not launch the kernel when a notebook starts up, and waiting until the first time a cell is executed.

Jason

--

You received this message because you are subscribed to the Google Groups "Project Jupyter" group.

To unsubscribe from this group and stop receiving emails from it, send an email to jupyter+u...@googlegroups.com.

To post to this group, send email to jup...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jupyter/8a8db447-6ec2-4a82-889e-091de9996b35%40googlegroups.com.

[Song Liu]

Yes, this is the kernel and Jupyter Server separation solution for my requirement.

So what is the schedule for this kernel launching behavior change ? where could access this kind of discussion ?

Many thanks.

On Tuesday, April 30, 2019 at 9:46:20 PM UTC+8, Jason Grout wrote:

It sounds like basically you just want, when opening a notebook, for the kernel to only be launched when you execute the first cell, instead of launching the kernel automatically when the notebook opens? We discussed this when implementing JupyterLab, but decided to stay with the behavior from classic notebook. However, we've been discussing recently changing the behavior in JupyterLab to not launch the kernel when a notebook starts up, and waiting until the first time a cell is executed.

Jason

On Tue, Apr 30, 2019 at 10:15 AM Song Liu <song...@gmail.com> wrote:

Also, about the separation looks like a direction in the community, but there are not some progress for a long time.

and I see there are two jupyter server:

- jupyter/jupyter_server

- jupyterlab/jupyterlab_server

I feel that the refactor want to be done under the new group "jupyterlab", but jupyterlab is still using something from old group "jupyter". It would confuse if we want to contribute something.

Thanks,

Song

--
You received this message because you are subscribed to the Google Groups "Project Jupyter" group.

To unsubscribe from this group and stop receiving emails from it, send an email to jup...@googlegroups.com.

[Jason Grout]

On Tue, Apr 30, 2019 at 4:36 PM Song Liu <song...@gmail.com> wrote:

Yes, this is the kernel and Jupyter Server separation solution for my requirement.

So what is the schedule for this kernel launching behavior change ? where could access this kind of discussion ?

The current state is:

1. We discussed this when implementing JupyterLab, and decided to go with the classic notebook behavior for compatibility.

2. We are releasing 1.0 shortly (in the next several months?), and we (most likely) wouldn't change this behavior at this point for 1.0.

3. We can take up this discussion after releasing 1.0, especially if someone volunteers to do the work for it.

Do you want to open an issue for "Launch a kernel for notebook only when needed, not at open" in the jupyterlab repo at https://github.com/jupyterlab/jupyterlab/issues/new? That will help us keep track of this request.

Thanks,

Jason

[Song Liu]

Yes, I can.

But it seems that there is already a same issue in jupyter/notebook project (https://github.com/jupyter/notebook/issues/3170), I understand jupyterlab is using the jupyter/notebook as its backend, or jupyterlab will rewrite not only the  frontend but also the backend ?

[Jason Grout]

The decision of if and when to launch a kernel is a frontend decision, not a backend decision. That's why it should also be a JupyterLab issue.

Thanks for finding that Jupyter notebook issue. If you could reference that, that would help tie in the discussion.

Thanks,

Jason

--
You received this message because you are subscribed to the Google Groups "Project Jupyter" group.

To unsubscribe from this group and stop receiving emails from it, send an email to jupyter+u...@googlegroups.com.

To post to this group, send email to jup...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jupyter/d829a562-14eb-4292-b723-36a70936d5f7%40googlegroups.com.

[Song Liu]

Hi Jason,

It seems there is already one for Jupyterlab: https://github.com/jupyterlab/jupyterlab/issues/4040, FYI.

Thanks,

Song

On Friday, May 3, 2019 at 2:08:51 AM UTC+8, Jason Grout wrote:

The decision of if and when to launch a kernel is a frontend decision, not a backend decision. That's why it should also be a JupyterLab issue.

Thanks for finding that Jupyter notebook issue. If you could reference that, that would help tie in the discussion.

Thanks,

Jason

On Thu, May 2, 2019 at 1:53 PM Song Liu <song...@gmail.com> wrote:

Yes, I can.

But it seems that there is already a same issue in jupyter/notebook project (https://github.com/jupyter/notebook/issues/3170), I understand jupyterlab is using the jupyter/notebook as its backend, or jupyterlab will rewrite not only the  frontend but also the backend ?

On Wednesday, May 1, 2019 at 6:48:29 AM UTC+8, Jason Grout wrote:

On Tue, Apr 30, 2019 at 4:36 PM Song Liu <song...@gmail.com> wrote:

Yes, this is the kernel and Jupyter Server separation solution for my requirement.

So what is the schedule for this kernel launching behavior change ? where could access this kind of discussion ?

The current state is:

1. We discussed this when implementing JupyterLab, and decided to go with the classic notebook behavior for compatibility.

2. We are releasing 1.0 shortly (in the next several months?), and we (most likely) wouldn't change this behavior at this point for 1.0.

3. We can take up this discussion after releasing 1.0, especially if someone volunteers to do the work for it.

Do you want to open an issue for "Launch a kernel for notebook only when needed, not at open" in the jupyterlab repo at https://github.com/jupyterlab/jupyterlab/issues/new? That will help us keep track of this request.

Thanks,

Jason

--
You received this message because you are subscribed to the Google Groups "Project Jupyter" group.

To unsubscribe from this group and stop receiving emails from it, send an email to jup...@googlegroups.com.