Can the notebook server shut itself down?

[Brendan Barnwell]

I'm working on writing a small wrapper for the notebook, which is essentially a single-tab browser.  I basically have it working so that I can do

jupyter notebook --browser="my_wrapper"

and it launches in my little wrapper.

The only problem is that the notebook server remains running in a console window and has to be manually closed with Ctrl-C.

What I would like is the ability to start a notebook server that would automatically shut down once the associated browser is closed.  Looking at old messages on the list, I get the impression this isn't possible.  Is that correct?

It seems like it would be possible to implement this by having JavaScript in the notebook page send keepalive messages back to the server.  These would serve as a sort of "dead man's switch" --- as long as the page remained open, the server would remain alive, but when the browser (or tab in a multi-tab browser) was closed, the cessation of keepalive messages would tell the server to shut itself down.  Is this feasible?

Failing that, I've also been looking for documentation on how to programmatically start the notebook server.  That is, supposing I'm running some Python code, how can I, from within that code, do the equivalent of "jupyter notebook --no-browser", and get the login token returned as a string so that I can redirect my wrapper to the appropriate URL?  Right now the only way I can see is to actually run jupyter using subprocess and parse the output, but that seems rather awkward.  Is there a part of the notebook that I can import as a module and do something like "server = notebook.start_server()" and then "token = server.token"?

[Roland Weber]

Hi Brendan,

regarding your fallback solution: the notebook server is Python code. You can import the packages and call the functions to start a notebook server in the current process. The 'jupyter' command itself is a Python script, so just follow the code path to see what it does when invoked with certain options. Once you get to the point where the notebook server is started, you'll know how to do that directly from your code. You'll probably have to follow the code path a bit further to see where the token is generated and stored.

hope that helps,

  Roland

[Brendan Barnwell]

On Wednesday, June 7, 2017 at 10:55:33 PM UTC-7, Roland Weber wrote:

Hi Brendan,

regarding your fallback solution: the notebook server is Python code. You can import the packages and call the functions to start a notebook server in the current process. The 'jupyter' command itself is a Python script, so just follow the code path to see what it does when invoked with certain options. Once you get to the point where the notebook server is started, you'll know how to do that directly from your code. You'll probably have to follow the code path a bit further to see where the token is generated and stored.

Yes, I realize that.  What I'm asking with that part of the question is whether there is a documented, "official" API for launching a notebook programmatically rather than from the command line.

[Thomas Kluyver]

On 8 June 2017 at 04:04, Brendan Barnwell <bren...@gmail.com> wrote:

What I would like is the ability to start a notebook server that would automatically shut down once the associated browser is closed.  Looking at old messages on the list, I get the impression this isn't possible.  Is that correct?

With a special-purpose browser that knows about the notebook server, yes - the browser can either use SIGTERM when it closes to shut down the notebook server, or when notebook 5.1 is released, it will be able to POST to /api/shutdown

https://github.com/jupyter/notebook/pull/2507

 

It seems like it would be possible to implement this by having JavaScript in the notebook page send keepalive messages back to the server.  These would serve as a sort of "dead man's switch" --- as long as the page remained open, the server would remain alive, but when the browser (or tab in a multi-tab browser) was closed, the cessation of keepalive messages would tell the server to shut itself down.  Is this feasible?

Feasible, and we already monitor connectivity to the kernel, but I don't think we'd ever do that by default - we don't want to kill the notebook server if your connection to it goes down. Even locally, 'keep alive' messages can be tricky to get right when computers go into standby.

Plus, of course, many people expect their notebook server to stay running without an open browser tab on it.

 

Failing that, I've also been looking for documentation on how to programmatically start the notebook server.  That is, supposing I'm running some Python code, how can I, from within that code, do the equivalent of "jupyter notebook --no-browser", and get the login token returned as a string so that I can redirect my wrapper to the appropriate URL?  Right now the only way I can see is to actually run jupyter using subprocess and parse the output, but that seems rather awkward.  Is there a part of the notebook that I can import as a module and do something like "server = notebook.start_server()" and then "token = server.token"?

Not quite as neat as that, but you can get info on the current user's running notebook servers by calling:

notebook.notebookapp.list_running_servers()

You should be able to find the one you've just started as a subprocess by checking the PIDs. That's easier than parsing info from stdout/stderr.

Thomas

[Brendan Barnwell]

On Thursday, June 8, 2017 at 7:24:57 AM UTC-7, takowl wrote:

On 8 June 2017 at 04:04, Brendan Barnwell <bren...@gmail.com> wrote:

What I would like is the ability to start a notebook server that would automatically shut down once the associated browser is closed.  Looking at old messages on the list, I get the impression this isn't possible.  Is that correct?

With a special-purpose browser that knows about the notebook server, yes - the browser can either use SIGTERM when it closes to shut down the notebook server, or when notebook 5.1 is released, it will be able to POST to /api/shutdown

https://github.com/jupyter/notebook/pull/2507

Oh, that sounds like it might be perfect.  Ideally I'd like to be able to do everything by communicating with the server rather than launching processes.

It seems like it would be possible to implement this by having JavaScript in the notebook page send keepalive messages back to the server.  These would serve as a sort of "dead man's switch" --- as long as the page remained open, the server would remain alive, but when the browser (or tab in a multi-tab browser) was closed, the cessation of keepalive messages would tell the server to shut itself down.  Is this feasible?

Feasible, and we already monitor connectivity to the kernel, but I don't think we'd ever do that by default - we don't want to kill the notebook server if your connection to it goes down. Even locally, 'keep alive' messages can be tricky to get right when computers go into standby.

Plus, of course, many people expect their notebook server to stay running without an open browser tab on it.

Yeah, I wasn't imagining that that would be the default, but that there would be a command-line option --dead-man-switch or something.  While I agree it wouldn't handle many cases, the situation I'm imagining is one where you're just using the notebook as if it were a regular program (not a long-running server that you connect to again and again).
 

 

Failing that, I've also been looking for documentation on how to programmatically start the notebook server.  That is, supposing I'm running some Python code, how can I, from within that code, do the equivalent of "jupyter notebook --no-browser", and get the login token returned as a string so that I can redirect my wrapper to the appropriate URL?  Right now the only way I can see is to actually run jupyter using subprocess and parse the output, but that seems rather awkward.  Is there a part of the notebook that I can import as a module and do something like "server = notebook.start_server()" and then "token = server.token"?

Not quite as neat as that, but you can get info on the current user's running notebook servers by calling:

notebook.notebookapp.list_running_servers()

You should be able to find the one you've just started as a subprocess by checking the PIDs. That's easier than parsing info from stdout/stderr.

Hmmmm, okay, that looks promising.

One thing I'm wondering is whether there's a way to get any of this information by communicating with the notebook server itself.  It would be nice if I could write my wrapper so it really acted as a browser from Jupyter's perspective, with the notebook starting it rather than the other way around.  The reason I'd feel more comfortable with this is that it would make the wrapper more independent of Jupyter, and thus (I hope) more robust to a situation where the wrapper is, for instance, running on a different version of Python than the notebook itself.

For instance, I'd like it if I could have my wrapper installed in just one Python environment (an Anaconda environment, say), but be able to run notebooks from arbitrary Python versions.  If I call list_running_servers from one Python process, is it guaranteed to show me all servers, even if they're running on different versions of Python?  Also, if the wrapper is the entry point and runs the notebook as a subprocess, then in my wrapper code I'd have to handle all the business of letting the user specify which Python they want the notebook server to run in.  But if jupyter is the entry point, then it already provides a way to specify the browser.  So if I can handle everything by communication with the server then I wouldn't have to worry about starting a separate Jupyter instance (e.g., to list servers) inside the wrapper.  In theory, Jupyter wouldn't even need to be installed in the Python that's running the wrapper.

In any case, thanks for your helpful comments.  They've given me some directions to explore.

[Thomas Kluyver]

On 8 June 2017 at 18:58, Brendan Barnwell <bren...@gmail.com> wrote:

One thing I'm wondering is whether there's a way to get any of this information by communicating with the notebook server itself.

Most stuff can be done by communicating with the server. Finding out how to communicate with it, such as what port it's running on, obviously can't. ;-)

 

  It would be nice if I could write my wrapper so it really acted as a browser from Jupyter's perspective, with the notebook starting it rather than the other way around.  The reason I'd feel more comfortable with this is that it would make the wrapper more independent of Jupyter, and thus (I hope) more robust to a situation where the wrapper is, for instance, running on a different version of Python than the notebook itself.

This is a sensible rationale. At the moment, I don't think that a browser launched by the notebook server has any specific knowledge of that, except for the URL it's asked to load.

If it's useful, I think we could add an environment variable when we launch the browser, like JUPYTER_NOTEBOOK_PORT, so that a special-purpose browser like yours could use that. You can probably get that information from the URL, though.

 

For instance, I'd like it if I could have my wrapper installed in just one Python environment (an Anaconda environment, say), but be able to run notebooks from arbitrary Python versions.  If I call list_running_servers from one Python process, is it guaranteed to show me all servers, even if they're running on different versions of Python?

Yes, it should find all notebook servers run by the current user on the current filesystem. That includes different versions of Python, conda envs and virtualenvs, but not VMs or docker containers (unless the relevant directories are part of a bind mount).

Thomas

[Brendan Barnwell]

On Friday, June 9, 2017 at 7:02:29 AM UTC-7, takowl wrote:

On 8 June 2017 at 18:58, Brendan Barnwell <bren...@gmail.com> wrote:

One thing I'm wondering is whether there's a way to get any of this information by communicating with the notebook server itself.

Most stuff can be done by communicating with the server. Finding out how to communicate with it, such as what port it's running on, obviously can't. ;-)

Heh, right!  But like, is it possible to get the server to tell me its own PID?

 

  It would be nice if I could write my wrapper so it really acted as a browser from Jupyter's perspective, with the notebook starting it rather than the other way around.  The reason I'd feel more comfortable with this is that it would make the wrapper more independent of Jupyter, and thus (I hope) more robust to a situation where the wrapper is, for instance, running on a different version of Python than the notebook itself.

This is a sensible rationale. At the moment, I don't think that a browser launched by the notebook server has any specific knowledge of that, except for the URL it's asked to load.

If it's useful, I think we could add an environment variable when we launch the browser, like JUPYTER_NOTEBOOK_PORT, so that a special-purpose browser like yours could use that. You can probably get that information from the URL, though.

As you said, the port can be obtained from the URL the browser is asked to load.  It would be nice, though, to have some kind of API that could be used to get at information about the server itself (rather than the kernels).  It seems like the main information missing is the PID of the server process itself.  I can see how this is not useful in many cases, though, since if the client is connecting from another computer it can't have much use for the PID.  And maybe the new /shutdown API is the only thing I really want to do with the PID anyway.  Any idea when 5.1 with that feature will be out?

[Brendan Barnwell]

On Thursday, June 8, 2017 at 7:24:57 AM UTC-7, takowl wrote:

On 8 June 2017 at 04:04, Brendan Barnwell <bren...@gmail.com> wrote:

What I would like is the ability to start a notebook server that would automatically shut down once the associated browser is closed.  Looking at old messages on the list, I get the impression this isn't possible.  Is that correct?

With a special-purpose browser that knows about the notebook server, yes - the browser can either use SIGTERM when it closes to shut down the notebook server, or when notebook 5.1 is released, it will be able to POST to /api/shutdown

https://github.com/jupyter/notebook/pull/2507

In trying to get this to work before the /api/shutdown call is in, I've hit a snag.  I thought that I could use list_running_servers and check the auth token against the one passed in the URL to match the server process with the one that launched the browser.  But I can't, because the token passed to the browser is just the one-time token, which isn't stored in the server info that's returned from list_running_servers.  So the information that would allow me to tell which server process ran the browser is, ironically, printed to the console, but not passed to the browser.  Would it be possible for the "regular" token to be passed to the browser as well as the one-time token?  Or is there some other way for the browser to identify the notebook server process that launched it?

I must say I've never fully understood the purpose of the one-time token.  Is it just so that if the user closes the notebook tab in their browser, they can then just navigate to the server URL without having to supply a token again?

[Brendan Barnwell]

Aaaand looking at it more it looks like maybe I will have to wait for the /api/shutdown thing.  Surprisingly, until that change, there seems to be no reliable way to shut down the notebook server at all (at least on Windows) except physically pressing Ctrl-C in the console window.  This is because on Windows there's (apparently) no such thing as SIGINT, and SIGTERM just kills without raising any interrupt.  So I can kill the notebook process that way, but it doesn't exit cleanly; for instance, it leaves its nbserver file behind, so that "jupyter notebook list" forever after will show that old server unless I manually delete the file.  Quite strange that this apparently hasn't been an issue for anyone until now.  Anyway I guess I'll just wait with bated breath for the 5.1 release.

[Thomas Kluyver]

On 11 June 2017 at 07:55, Brendan Barnwell <bren...@gmail.com> wrote:

Anyway I guess I'll just wait with bated breath for the 5.1 release.

We'll try to make that fairly soon, but there's still a number of open issues targeted for 5.1:
https://github.com/jupyter/notebook/milestone/18