Notebook stuck on "N?" with a worksheet

[Jori Mäntysalo]

Could someone try SageNB with http://www.sis.uta.fi/~jm58660/Broken.sws

For me that stucks when I try to evaluate the first calculation cell, the
one with content

#Katso komennon N helppi, erityisesti se, miten desimaalien määrää voidaan säädellä
N?

It is somehow related to some phrases or characters or something in text
cells before it, as the problem faded away when I remove those. Because of
that I did not clear the example from Finnish texts.

--
Jori Mäntysalo

[Jori Mäntysalo]

On Fri, 7 Oct 2016, Jori Mäntysalo wrote:

> Could someone try SageNB with http://www.sis.uta.fi/~jm58660/Broken.sws

More about this: the computation works, but getting output back to the
browser window does not work. So if I reload the page, it seems to be OK.
But with other worksheets everything works without need to reload.

--
Jori Mäntysalo

[kcrisman]

I see something similar when I try the example you posted here.  But doing Escape does give the output to me.  It also only seems to happen the first time; if I try to evaluate it again ...

But weirdly, when I try to type it in myself, or even cut and paste and then later replace the last line with N?, it works.

#Katso komennon N helppi, erityisesti se, miten desimaalien määrää voidaan säädellä

N?

whereas keeping the one you had, even with fewer characters,

#Katso komennon N helppi, erityisesti se, miten desimaalien m

N?

has the hang.  This is really hard to reproduce, but please do let us know if you find something more specific; I feel like maybe it has to do with the combination of introspection and the character set, but that seems odd, and sometimes I even get it without introspection.

[Jori Mäntysalo]

Anybody?

One -- and only one -- of my users if having really big problems with
cells not responding. Somehow computation seems to work, but the result is
not got back to worksheet. Pressing refresh helps sometimes, not always.

How to debug this?

--
Jori Mäntysalo

---------- Forwarded message ----------
Date: Fri, 7 Oct 2016 12:23:56 +0300 (EEST)
From: Jori Mantysalo <jm5...@uta.fi>
To: sage-...@googlegroups.com
Subject: Notebook stuck on "N?" with a worksheet

Could someone try SageNB with http://www.sis.uta.fi/~jm58660/Broken.sws

For me that stucks when I try to evaluate the first calculation cell, the one
with content

#Katso komennon N helppi, erityisesti se, miten desimaalien määrää voidaan
säädellä
N?

[kcrisman]

On Friday, October 21, 2016 at 2:38:14 AM UTC-4, Jori Mäntysalo wrote:

Anybody?

One -- and only one -- of my users if having really big problems with
cells not responding. Somehow computation seems to work, but the result is
not got back to worksheet. Pressing refresh helps sometimes, not always.

How to debug this?

That is a hard question, if it isn't due to some specific repeatable thing.  You can use the developer tools in your browser to try to track what responses are coming, but if the sagenb log isn't showing anything unusual, I'm not sure what else to say. 

However, a workaround is to put the text into true text cells, rather than as comments in the compute cell.  That is probably better anyway, since you can then style it.

[Volker Braun]

IMHO the best solution is to stop using SageNB asap and use "sage --notebook=jupyter"

[Jori Mäntysalo]

On Sat, 22 Oct 2016, Volker Braun wrote:

> IMHO the best solution is to stop using SageNB asap and use "sage
> --notebook=jupyter"

Does it have multiuser support, so that A can't see worksheet for B with
some file(...) -command?

* * *

A quick test: make new worksheet, write "1+2", press Shift+Enter. Got "3".
Press arrow up, correct text to "1+3", again Shift+Enter and arrow up. And
now how to correct the line to "1+4" without mouse?

This is 64-bit Ubuntu, Firefox from repository, all up-to-date.

(OK, <tab> can do that, but is not quick and easy.)

--
Jori Mäntysalo

[Volker Braun]

Yes, with jupyterhub you can have isolation with unix users. That is, there is a login screen and people can sign in with their normal unix username/password.

[Jori Mäntysalo]

On Sat, 22 Oct 2016, Volker Braun wrote:

> Yes, with jupyterhub you can have isolation with unix users. That is,
> there is a login screen and people can sign in with their normal unix
> username/password.

Hmm... OK. Thinking aloud:

As Sage may eat much CPU and/or memory, I want it to be on it's own
server. That server should connect to our LDAP, and have normal Unix
accounts. Also we need some local accounts for courses. Managing users is
then not possible from Sage UI, but with normal Linux commands. But that's
not a problem for us.

Normal shell logins should be disabled for almost all users, but some
researcher might use Sage from command line too.

I must check if sharing worksheets works, see if I can run plain Python2,
R and Octave too from UI.

What else? Check how much memory every worksheet eats (no, we don't have
memory ballooning in our virtualization environment) and so.

(Besides that, University of Tampere, Technical University of Tampere and
University of Applied Sciences of Tampere will be united from 1.1.2018.
So check out what other two wants and be ready for that too. Maple to
same server maybe?)

--
Jori Mäntysalo

[Jeroen Demeyer]

On 2016-10-23 08:47, Jori Mäntysalo wrote:
> I must check if sharing worksheets works

As far as I know, Jupyter(hub) does not have such a feature since it
completely relies on the OS for access checks. A notebook is just an
ordinary file owned by some Unix user.

[Jori Mäntysalo]

On Sun, 23 Oct 2016, Jeroen Demeyer wrote:

>> I must check if sharing worksheets works
>
> As far as I know, Jupyter(hub) does not have such a feature since it
> completely relies on the OS for access checks. A notebook is just an
> ordinary file owned by some Unix user.

OK, then I am not going to try this (yet).

How others teach Sage? Here I make a Sage account of the form
"Coursecode-Year", and the teacher asks students to share worksheet with
that account.

--
Jori Mäntysalo

[Luca De Feo]

> As far as I know, Jupyter(hub) does not have such a feature since it
> completely relies on the OS for access checks.

That's not correct. JupyterHub supports arbitrary authenticators:

https://jupyterhub.readthedocs.io/en/latest/authenticators.html

In particular, there is an LDAP authenticator:

https://github.com/jupyterhub/ldapauthenticator

If I understands Jori's loud thoughts, he wants to mix an LDAP
authenticator with the default PAM authenticator. It shouldn't be too
hard to write a class that tries them in some order, although I'm not
sure that's the best solution.

> A notebook is just an ordinary file owned by some Unix user.

That is pretty much configurable too :

https://jupyterhub.readthedocs.io/en/latest/spawners.html

I think the hardest thing to achieve with jupyterhub is notebook
sharing, at least in a fashion similar to the classic notebook. There
is a concept of *admin account* in JupyterHub (disjoint from Unix
admins). Admins can run the Jupyter notebook as any other user, so
they can see and run all the students' notebooks.

> How others teach Sage? Here I make a Sage account of the form
> "Coursecode-Year", and the teacher asks students to share worksheet
> with that account.

For the first year this time, I set up a jupyterhub with a PAM
authenticator. I create a unix user account for each student (and
teacher). This is fine with us, because we want them to also have ssh
access (and it's only ~10 students).

They can use Python, Sage, and Julia via the Jupyter notebook, and can
also run any of those and much more from a shell. Added bonuses:

- There is an in-browser shell in JupyterHub. Very practical for those
students who don't know how to run ssh on their home computers.

- JupyterHub file view lets the students upload and download any kind
of file from the server, not only noteboks. So this makes a very
simple "cloud host" for them to have all their work available from
anywhere (e.g., the C files they write in the system programming
class).

When I want to share a notebook with the students, I share a static
.ipynb file via gist, and visualize it via nbviewer. e.g.:

http://nbviewer.jupyter.org/gist/defeo/4adb727f7608dfd2c75eb7699d670fb7

When I want to see the student's work, I ask them to send me the
.ipynb by mail, or directly access their notebooks using my admin
account.

Works like a charm.

I'll be looking at custom authentication soon.

Luca

[Dima Pasechnik]

there are people who hack SMC into being installable and runnable on local machines or other (non/GCE clouds). See e.g.

https://groups.google.com/d/msg/sage-cloud/og3tmsXK-K0/snB-7v2_CgAJ 

--
Jori Mäntysalo

[William Stein]

Or use our official docker image

https://github.com/sagemathinc/smc/blob/master/src/dev/docker/README.md

Our docker image is not perfect, secure, or totally robust yet, but it
is easy to use.

William

>>
>> --
>> Jori Mäntysalo
>
> --
> You received this message because you are subscribed to the Google Groups
> "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sage-devel+...@googlegroups.com.
> To post to this group, send email to sage-...@googlegroups.com.
> Visit this group at https://groups.google.com/group/sage-devel.
> For more options, visit https://groups.google.com/d/optout.



--
William (http://wstein.org)

[Jori Mäntysalo]

On Sun, 23 Oct 2016, Luca De Feo wrote:

>> As far as I know, Jupyter(hub) does not have such a feature since it
>> completely relies on the OS for access checks.
>
> That's not correct. JupyterHub supports arbitrary authenticators:
>
> https://jupyterhub.readthedocs.io/en/latest/authenticators.html

Can you give me some kind of Hello world! -example of Jupyter? ./sage
--notebook=jupyter will open a server to port 8888 --- how to change the
port or, more general, is there some help file? First I would like to see
a way to have two local user using Sage and not seeing each other files.

--
Jori Mäntysalo

[Dima Pasechnik]

I mentioned this here: https://github.com/OpenDreamKit/OpenDreamKit/issues/94#issuecomment-255683103 

--
Jori Mäntysalo

[Volker Braun]

I have a setup helper for jupyterhup at 

https://github.com/vbraun/run-jupyter-hub

It might be out of date, haven't tried it with the newest version of jupyterhub. But maybe its a starting point for you...

[Luca De Feo]

> Can you give me some kind of Hello world! -example of Jupyter? ./sage
> --notebook=jupyter will open a server to port 8888 --- how to change the
> port or, more general, is there some help file? First I would like to see a
> way to have two local user using Sage and not seeing each other files.

I am not sure you realize that JupyterHub is not shipped with Sage.
Running ./sage --notebook=jupyter will run the Jupyter server, NOT
JupyterHub. This is a single-user setting: the Jupyter server runs as
your user, and can acces any file your user can access (the file
explorer is rooted in your working dir, but you can go anywhere in the
file system using Python).

JupyterHub is a different beast: it is a server that waits for
connections and spawns Jupyter servers under different userids. So
this is similar to running ./sage --notebook=jupyter as two different
users in their respective homedirs.

If you want to try out JupyterHub, you will have to install it
separately, and link the Sage kernel to it. I find the JupyterHub docs
pretty well written and easy to follow for a Hello world test

https://jupyterhub.readthedocs.io/en/latest/getting-started.html

You can test them in a virtualenv for an easy throwaway setup.
Volker's scripts may be another good starting point, although I
haven't tested them. Alternatively, there is a docker container here :

https://hub.docker.com/r/jupyterhub/jupyterhub/

which is also good for testing.

I agree with Dima that a tutorial for setting up JupyterHub in a
SageNB-like way would be much welcome. I have the feeling that the
Jupyter/JupyterHub/SageMath integration is still a bit of a moving
target ATM, but we should try to write this sooner than later.

Luca

[Jori Mäntysalo]

On Mon, 24 Oct 2016, Volker Braun wrote:

> I have a setup helper for jupyterhup at 
>
> https://github.com/vbraun/run-jupyter-hub

Thanks. Stopped with ffi.h missing, now installed libffi-dev and trying
again...

On Tue, 25 Oct 2016, Luca De Feo wrote:

>> Can you give me some kind of Hello world! -example of Jupyter?
>

> I am not sure you realize that JupyterHub is not shipped with Sage.

That explains quite much. No, I did not realize that.

> If you want to try out JupyterHub, you will have to install it
> separately, and link the Sage kernel to it. I find the JupyterHub docs
> pretty well written and easy to follow for a Hello world test
>
> https://jupyterhub.readthedocs.io/en/latest/getting-started.html

Got it installed and shown up in my browser. Good start. It rejected login
with wrong password, and with right password gave "Spawner failed to start
[status=1]". So next I guess I should somehow register Sage as a
"kernel"(?) for jupyterhub.

* * *

I tested on Ubuntu 16.04 LTS. I did

apt-get install npm nodejs-legacy
apt-get install python3-pip

npm install -g configurable-http-proxy

export LC_ALL="en_US.UTF-8"
export LC_CTYPE="en_US.UTF-8"
dpkg-reconfigure locales

pip3 install jupyterhub

Noticed that it used IPv6, got a headache when googling, found the
instruction to put GRUB_CMDLINE_LINUX_DEFAULT="ipv6.disable=1 quiet" on
/etc/default/grub, said update-grub and got the login prompt.

--
Jori Mäntysalo

[Jori Mäntysalo]

On Tue, 25 Oct 2016, Jori Mantysalo wrote:

>> https://jupyterhub.readthedocs.io/en/latest/getting-started.html
>
> Got it installed and shown up in my browser. Good start. It rejected login
> with wrong password, and with right password gave "Spawner failed to start
> [status=1]". So next I guess I should somehow register Sage as a "kernel"(?)
> for jupyterhub.

More progress. After

pip3 install jupyter
cd /usr/local/share/jupyter
ln -s /opt/sage-7.4/local/share/jupyter/kernels
ln -s /opt/sage-7.4/local/share/jupyter/nbextensions/

it almost work. Now when I modified

/opt/sage-7.4/local/share/jupyter/kernels/sagemath/kernel.json

by adding

"env": {"SAGE_ROOT": "/opt/sage-7.4/"}

to the dict I got this to work!

I used jupyterhub --no-ssl --port=80

Next move to https, then try ldap, and then combine ldap and local
accounts. And last look for worksheet sharing.

Thanks for help!

--
Jori Mäntysalo

[Jori Mäntysalo]

On Sun, 23 Oct 2016, Luca De Feo wrote:

> When I want to share a notebook with the students, I share a static
> .ipynb file via gist, and visualize it via nbviewer. e.g.:
>
> http://nbviewer.jupyter.org/gist/defeo/4adb727f7608dfd2c75eb7699d670fb7
>
> When I want to see the student's work, I ask them to send me the
> .ipynb by mail, or directly access their notebooks using my admin
> account.
>
> Works like a charm.

OK, sounds possible to do. But document sharing between researchers still
does not work. (But to be honest, only sharing so far here has been me
replying to request like "Can you test this hypothesis?". Maybe some
day...)

--
Jori Mäntysalo

[Luca De Feo]

Here's a quite popular service to share Jupyter notebooks together
with a preset environment (e.g., a SageMath kernel) :

http://mybinder.org/

Quite a different workflow compared to sharing a notebook on a SageNB
server, though.

Luca