jupyter behind nginx - what am I doing wrong ?

[Thierry Parmentelat]

As a follow-up to previous message that explains the objective that I am pursuing..

For now I spent some time musing with the idea that I could work my own way around this by putting together:

. a very light django app that would deal with the 'copy on demand' and other 'create users on demand' business, as well as of course spwaning dockers

. a nginx reverse proxy to route to that django app or to the relevant docker instances

This is depicted in the attached sketch, where

# One notebook opening request 

1. edx sends URL over https like

    https://nbhosting.inria.fr/ipythonExercice/flotbioinfo/w1/en-w1-s07-c1-walking.ipynb/mary

2. nginx proxies to the django app that creates if needed

  * Unix id, home dir for student mary

  * copy of that notebook in the student's homedir

  * docker instance: public port 8000 is bound to internal port 8888 (all docker instances run on 8888)

3. django then answers with a HTTP redirect like

    https://nbhosting.inria.fr/8000/w1/en-w1-s07-c1-walking.ipynb?token=flotbioinfo-x-mary 

 

4. browser complies with redirect, talks to nginx again

5. nginx figures it needs to proxy to local port 8000 (Set-cookie `docker_port=8000`)

6. jupyter serves raw notebook data that contains further hrefs to `/static/` and `/api/` and the like

7. browser asks for hrefs like `/static/` with cookie `docker_port=8000`

8. nginx forwards to right docker

9. answer to `/static/`  and similar

-----

I have gotten this to almost work, in that "internal" hrefs like `/static/` and similar get properly handled; at least I do not get any complaint from the browser about these any more, and indeed I get a logo to show up..

However I am not quite there yet as I am getting a large 'Error loading notebook` message (see second attachement) and I cannot quite make what the javascript console errors are about.

Any hint on what I can be doing wrong here ? 

---

And many thanks in any case to everyone for this beautiful tool suite, notebooks are awesome !

[Thierry Parmentelat]

Let me just add that the docker image that I use is 

# docker images

REPOSITORY                         TAG                 IMAGE ID            CREATED             SIZE

docker.io/jupyter/scipy-notebook   latest              070cb62f629f        13 days ago         4.158 GB

and the jupyter version is 

# docker exec flotbioinfo-x-mary jupyter --version

4.2.1

[MinRK]

My guess is that the notebook server needs to be passed a `NotebookApp.base_url` that corresponds to the prefix of forwarded requests. It's a bit bizarre that the request is failing with `200: OK`, that's very strange indeed, but suggests to me that the API requests, which will look like `/api/contents/path/to/notebook.ipynb` might not be routed correctly. Figuring out exactly what's handling that request and what the response is is where I would start.

-Min

--
You received this message because you are subscribed to the Google Groups "Project Jupyter" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jupyter+unsubscribe@googlegroups.com.
To post to this group, send email to jup...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jupyter/a1f003f6-7e69-46f2-9c8f-250884bbd88a%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Thierry Parmentelat]

On Friday, January 20, 2017 at 11:36:48 PM UTC+1, Min RK wrote:

My guess is that the notebook server needs to be passed a `NotebookApp.base_url` that corresponds to the prefix of forwarded requests. It's a bit bizarre that the request is failing with `200: OK`, that's very strange indeed, but suggests to me that the API requests, which will look like `/api/contents/path/to/notebook.ipynb` might not be routed correctly. Figuring out exactly what's handling that request and what the response is is where I would start.

Thanks Benjamin for your hint

The way I was trying to get the /api/ calls routed correctly was with a cookie, that nginx would set and read; my understanding is that this would achieve the same result as setting `NotebookApp.base_url`, and my first attempts at setting this variable seem to confirm this conjecture, as I obtain the same result.

In an attempt to provide more context, here's the log of the jupyter instance for the same session

Apparently the /api/ calls do end up where I expect them to; however I am finding the `/login/` business a bit puzzling

my take on authentication, at this early stage at least, is to set the authentication token myself (simply as the docker name `flotbioinfo-x-mary` for now) by spawning jupyter with `--NotebookApp.token=$jupyter_token` and then pass this along to the redirected URL;  

I am not exactly sure if this is supposed to be passed all along or not, could it be where I am screwing it up ?

-- Thanks again in any case for picking it up, I am kind of stuck here :)

----

[D 11:25:18.346 NotebookApp] Accepting token-authenticated connection from 172.17.0.1

[D 11:25:18.348 NotebookApp] Using contents: services/contents

[D 11:25:18.350 NotebookApp] 304 GET /notebooks/w1/en-w1-s07-c1-walking.ipynb?token=flotbioinfo-x-mary (172.17.0.1) 4.79ms

[D 11:25:18.448 NotebookApp] 304 GET /static/components/MathJax/MathJax.js?config=TeX-AMS_HTML-full,Safe&delayStartupUntil=configured (172.17.0.1) 1.21ms

[D 11:25:18.467 NotebookApp] 304 GET /custom/custom.css (172.17.0.1) 1.13ms

[D 11:25:18.579 NotebookApp] 304 GET /static/style/style.min.css.map (172.17.0.1) 1.18ms

[D 11:25:18.614 NotebookApp] 304 GET /static/components/MathJax/config/TeX-AMS_HTML-full.js?rev=2.6.0 (172.17.0.1) 1.17ms

[D 11:25:18.761 NotebookApp] 304 GET /static/notebook/js/main.min.js.map (172.17.0.1) 1.16ms

[D 11:25:18.813 NotebookApp] 304 GET /custom/custom.js?v=20170121111124 (172.17.0.1) 1.35ms

[D 11:25:18.857 NotebookApp] 304 GET /static/components/MathJax/config/Safe.js?rev=2.6.0 (172.17.0.1) 1.19ms

[I 11:25:18.913 NotebookApp] 302 GET /api/config/notebook?_=1484997918667 (172.17.0.1) 1.60ms

[I 11:25:18.940 NotebookApp] 302 GET /api/config/common?_=1484997918668 (172.17.0.1) 1.46ms

[I 11:25:18.961 NotebookApp] 302 GET /api/kernelspecs (172.17.0.1) 1.36ms

[I 11:25:18.967 NotebookApp] 302 GET /api/contents/w1/en-w1-s07-c1-walking.ipynb?type=notebook&_=1484997918669 (172.17.0.1) 1.41ms

[D 11:25:18.996 NotebookApp] 304 GET /static/components/MathJax/jax/output/HTML-CSS/fonts/STIX-Web/fontdata.js?rev=2.6.0 (172.17.0.1) 1.19ms

[D 11:25:19.021 NotebookApp] 304 GET /static/components/MathJax/extensions/Safe.js?rev=2.6.0 (172.17.0.1) 1.10ms

[D 11:25:19.029 NotebookApp] Using contents: services/contents

[D 11:25:19.030 NotebookApp] 200 GET /login?next=%2Fapi%2Fconfig%2Fnotebook%3F_%3D1484997918667 (172.17.0.1) 2.65ms

[D 11:25:19.053 NotebookApp] Using contents: services/contents

[D 11:25:19.055 NotebookApp] 200 GET /login?next=%2Fapi%2Fconfig%2Fcommon%3F_%3D1484997918668 (172.17.0.1) 2.81ms

[D 11:25:19.068 NotebookApp] Using contents: services/contents

[D 11:25:19.069 NotebookApp] 200 GET /login?next=%2Fapi%2Fkernelspecs (172.17.0.1) 2.67ms

[D 11:25:19.077 NotebookApp] Using contents: services/contents

[D 11:25:19.079 NotebookApp] 200 GET /login?next=%2Fapi%2Fcontents%2Fw1%2Fen-w1-s07-c1-walking.ipynb%3Ftype%3Dnotebook%26_%3D1484997918669 (172.17.0.1) 2.60ms

[Thierry Parmentelat]

 As an update on this issue, in case others run into similar issues, I could solve this by filling in a missing Host http header

And later on in order to get the notebook to connect to its kernel I had to apply this recipe still in the nginx conf

http://nginx.org/en/docs/http/websocket.html

not quite there yet, but definitely doing progress: students can interact with notebook and save their own copy in the host filesystem