Jupyter Kernel Spec + N times refreshing for N Kernels + Slow start up #3135

[Prabhu Kasinathan]

Hi Dev,

I found some strange scenario on Notebook. I'm running jupyter notebook 4.1.0 version on Python 3.5.2 :: Anaconda 4.1.1 (64-bit) platform.

Issue: If I enable N kernels, when I refresh Jupyter home page, then it's getting kernel spec N times and hence page refresh is too slow. Say for example: If I enable 10 custom kernels, when I refresh jupyter home page, it's taking 10+ seconds. When I see log, most of the time is taking in refreshing kernels again and again for 10 times as "Found kernel <> in /usr/local/share/jupyter/kernels". In logs, it show 100 times that kernel as found.

Version:

[22:48]:[root@myhost:~]# python -V
Python 3.5.2 :: Anaconda 4.1.1 (64-bit)
[22:48]:[root@myhost:~]# jupyter --version
4.1.0

Starting Jupyter with default Python Kernel:

[22:48]:[root@myhost:~]# jupyter notebook --no-browser --ip 0.0.0.0 --port 9000 --debug

Log: With only one Kernel by default:

[D 22:40:45.377 NotebookApp] 200 GET /api/sessions?_=1513320006484 (10.196.177.146) 0.85ms
[D 22:40:45.378 NotebookApp] 200 GET /api/terminals?_=1513320006485 (10.196.177.146) 0.42ms
[D 22:40:45.459 NotebookApp] 200 GET /api/contents?type=directory&_=1513320006486 (10.196.177.146) 1.18ms
[D 22:40:49.684 NotebookApp] Using contents: services/contents
[D 22:40:49.685 NotebookApp] 304 GET /tree (10.196.177.146) 1.46ms
[D 22:40:49.789 NotebookApp] 304 GET /custom/custom.css (10.196.177.146) 0.52ms
[D 22:40:50.111 NotebookApp] 304 GET /custom/custom.js?v=20171214222020 (10.196.177.146) 0.84ms
[D 22:40:50.112 NotebookApp] 200 GET /static/base/images/favicon.ico?v=30780f272ab4aac64aa073a841546240 (10.196.177.146) 0.64ms
[D 22:40:50.221 NotebookApp] 200 GET /api/config/common?_=1513320049961 (10.196.177.146) 2.28ms
[D 22:40:50.223 NotebookApp] 200 GET /api/config/tree?_=1513320049960 (10.196.177.146) 1.93ms
[D 22:40:50.225 NotebookApp] Native kernel (python3) available from /home/anaconda3/lib/python3.5/site-packages/ipykernel/resources
[D 22:40:50.745 NotebookApp] 304 GET /api/kernelspecs (10.196.177.146) 520.39ms
[D 22:40:50.746 NotebookApp] 200 GET /api/terminals?_=1513320049962 (10.196.177.146) 0.71ms
[D 22:40:50.747 NotebookApp] 200 GET /api/sessions?_=1513320049963 (10.196.177.146) 0.62ms
[D 22:40:50.748 NotebookApp] 200 GET /api/terminals?_=1513320049964 (10.196.177.146) 0.42ms
[D 22:40:50.993 NotebookApp] 200 GET /api/contents?type=directory&_=1513320049965 (10.196.177.146) 1.23ms

Enabling 1st custom kernel

[22:41]:[root@myhost:~]# jupyter-kernelspec install $SPARKMAGIC_PATH/sparkmagic/sparkmagic/kernels/sparkkernel
[InstallKernelSpec] Installed kernelspec sparkkernel in /usr/local/share/jupyter/kernels/sparkkernel
[22:41]:[root@myhost:~]# 

Log: After enabling 1st custom kernel, when I refresh home page

[D 22:43:48.936 NotebookApp] 200 GET /api/sessions?_=1513320206761 (10.196.177.146) 0.88ms
[D 22:43:48.938 NotebookApp] 200 GET /api/terminals?_=1513320206762 (10.196.177.146) 0.43ms
[D 22:43:49.295 NotebookApp] 200 GET /api/contents?type=directory&_=1513320206763 (10.196.177.146) 1.17ms
[D 22:44:05.654 NotebookApp] Using contents: services/contents
[D 22:44:05.655 NotebookApp] 304 GET /tree (10.196.177.146) 1.56ms
[D 22:44:05.769 NotebookApp] 304 GET /custom/custom.css (10.196.177.146) 0.55ms
[D 22:44:06.154 NotebookApp] 304 GET /custom/custom.js?v=20171214222020 (10.196.177.146) 0.50ms
[D 22:44:06.160 NotebookApp] 200 GET /static/base/images/favicon.ico?v=30780f272ab4aac64aa073a841546240 (10.196.177.146) 0.61ms
[D 22:44:06.252 NotebookApp] 200 GET /api/config/tree?_=1513320245977 (10.196.177.146) 2.08ms
[D 22:44:06.255 NotebookApp] 200 GET /api/config/common?_=1513320245978 (10.196.177.146) 2.57ms
[D 22:44:06.256 NotebookApp] Found kernel sparkkernel in /usr/local/share/jupyter/kernels
[D 22:44:06.256 NotebookApp] Native kernel (python3) available from /home/anaconda3/lib/python3.5/site-packages/ipykernel/resources
[D 22:44:06.768 NotebookApp] Found kernel sparkkernel in /usr/local/share/jupyter/kernels
[D 22:44:06.768 NotebookApp] Native kernel (python3) available from /home/anaconda3/lib/python3.5/site-packages/ipykernel/resources
[D 22:44:07.277 NotebookApp] 304 GET /api/kernelspecs (10.196.177.146) 1021.39ms
[D 22:44:07.279 NotebookApp] 200 GET /api/terminals?_=1513320245979 (10.196.177.146) 0.71ms
[D 22:44:07.280 NotebookApp] 200 GET /api/terminals?_=1513320245981 (10.196.177.146) 0.42ms
[D 22:44:07.281 NotebookApp] 200 GET /api/sessions?_=1513320245980 (10.196.177.146) 0.60ms
[D 22:44:07.602 NotebookApp] 200 GET /api/contents?type=directory&_=1513320245982 (10.196.177.146) 1.32ms

Enabling 2nd custom kernel

[22:41]:[root@myhost:~]# jupyter-kernelspec install $SPARKMAGIC_PATH/sparkmagic/sparkmagic/kernels/pysparkkernel
[InstallKernelSpec] Installed kernelspec pysparkkernel in /usr/local/share/jupyter/kernels/pysparkkernel
[22:45]:[root@myhost:~]# 

Log: After enabling 2nd custom kernel, when I refresh home page

[D 22:45:37.584 NotebookApp] 200 GET /api/sessions?_=1513320245983 (10.196.177.146) 0.90ms
[D 22:45:37.585 NotebookApp] 200 GET /api/terminals?_=1513320245984 (10.196.177.146) 0.42ms
[D 22:45:37.668 NotebookApp] 200 GET /api/contents?type=directory&_=1513320245985 (10.196.177.146) 1.18ms
[D 22:45:38.691 NotebookApp] Using contents: services/contents
[D 22:45:38.692 NotebookApp] 304 GET /tree (10.196.177.146) 1.51ms
[D 22:45:38.812 NotebookApp] 304 GET /custom/custom.css (10.196.177.146) 0.55ms
[D 22:45:39.134 NotebookApp] 304 GET /custom/custom.js?v=20171214222020 (10.196.177.146) 0.50ms
[D 22:45:39.150 NotebookApp] 200 GET /static/base/images/favicon.ico?v=30780f272ab4aac64aa073a841546240 (10.196.177.146) 0.61ms
[D 22:45:39.231 NotebookApp] 200 GET /api/config/tree?_=1513320338963 (10.196.177.146) 2.07ms
[D 22:45:39.233 NotebookApp] 200 GET /api/config/common?_=1513320338964 (10.196.177.146) 1.73ms
[D 22:45:39.235 NotebookApp] Found kernel pysparkkernel in /usr/local/share/jupyter/kernels
[D 22:45:39.235 NotebookApp] Found kernel sparkkernel in /usr/local/share/jupyter/kernels
[D 22:45:39.235 NotebookApp] Native kernel (python3) available from /home/anaconda3/lib/python3.5/site-packages/ipykernel/resources
[D 22:45:39.752 NotebookApp] Found kernel pysparkkernel in /usr/local/share/jupyter/kernels
[D 22:45:39.752 NotebookApp] Found kernel sparkkernel in /usr/local/share/jupyter/kernels
[D 22:45:39.752 NotebookApp] Native kernel (python3) available from /home/anaconda3/lib/python3.5/site-packages/ipykernel/resources
[D 22:45:40.271 NotebookApp] Found kernel pysparkkernel in /usr/local/share/jupyter/kernels
[D 22:45:40.271 NotebookApp] Found kernel sparkkernel in /usr/local/share/jupyter/kernels
[D 22:45:40.271 NotebookApp] Native kernel (python3) available from /home/anaconda3/lib/python3.5/site-packages/ipykernel/resources
[D 22:45:40.786 NotebookApp] 200 GET /api/kernelspecs (10.196.177.146) 1551.01ms
[D 22:45:40.787 NotebookApp] 200 GET /api/terminals?_=1513320338965 (10.196.177.146) 0.73ms
[D 22:45:40.788 NotebookApp] 200 GET /api/terminals?_=1513320338967 (10.196.177.146) 0.53ms
[D 22:45:40.789 NotebookApp] 200 GET /api/sessions?_=1513320338966 (10.196.177.146) 0.65ms
[D 22:45:41.125 NotebookApp] 200 GET /api/contents?type=directory&_=1513320338968 (10.196.177.146) 1.21ms

In above logs, you can see I'm seeing Found Kernel 9 times, if I have 3 kernels enabled.

[D 22:45:39.235 NotebookApp] Found kernel pysparkkernel in /usr/local/share/jupyter/kernels
[D 22:45:39.235 NotebookApp] Found kernel sparkkernel in /usr/local/share/jupyter/kernels
[D 22:45:39.235 NotebookApp] Native kernel (python3) available from /home/anaconda3/lib/python3.5/site-packages/ipykernel/resources
[D 22:45:39.752 NotebookApp] Found kernel pysparkkernel in /usr/local/share/jupyter/kernels
[D 22:45:39.752 NotebookApp] Found kernel sparkkernel in /usr/local/share/jupyter/kernels
[D 22:45:39.752 NotebookApp] Native kernel (python3) available from /home/anaconda3/lib/python3.5/site-packages/ipykernel/resources
[D 22:45:40.271 NotebookApp] Found kernel pysparkkernel in /usr/local/share/jupyter/kernels
[D 22:45:40.271 NotebookApp] Found kernel sparkkernel in /usr/local/share/jupyter/kernels
[D 22:45:40.271 NotebookApp] Native kernel (python3) available from /home/anaconda3/lib/python3.5/site-packages/ipykernel/resources

Please let me know your comments to fix this issue. It's really slowing down our Production instances performance when we enable 10+ kernels.

Issue: https://github.com/jupyter/notebook/issues/3135

Thanks!

[Thomas Kluyver]

I'll investigate and post updates on the issue.

It's best to post the issue in just one place - we get notifications for issues, so you don't need to send it to the mailing list as well. Posting in two places can split discussion and lead to duplicated effort. Thanks!

--
You received this message because you are subscribed to the Google Groups "Project Jupyter" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jupyter+unsubscribe@googlegroups.com.
To post to this group, send email to jup...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jupyter/f3f3eb17-685b-47cd-9ff1-a6d1fe3ff432%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.