Worksheets of users get stopped

[Andrey Novoseltsev]

Hello,

I have a 5.4.beta0 notebook server which uses a different user account
for worksheet processes. When I sign out, these worksheet processes
terminate. Using the same account for server and worker seems to keep
old behaviour - worksheet processes continue running until the next
connection. Is it a bug or design decision? (And can anyone confirm
that this happens or I just messed up something?)

Thank you!
Andrey

[Dima Pasechnik]

IMHO it's best to start the notebook server using screen.

At least this looks like a workaround for problems you describe.

 

Thank you!
Andrey

[Andrey Novoseltsev]

On Sep 13, 3:55 am, Dima Pasechnik <dimp...@gmail.com> wrote:
> IMHO it's best to start the notebook server using screen.
> At least this looks like a workaround for problems you describe.

The server process is started using screen and it works just fine. But
when a user opens a worksheet, a process running this worksheet is
launched and the worksheet on the home screen is shown as running. If
I now sign out and sign in back, the worksheet is not running anymore,
neither in description nor in the list of processes on the system.
That is, you cannot leave your session and come back to it later. This
did not happen to me before.

[Andrey Novoseltsev]

OK, it seems to be unrelated to different server/worker accounts. I
have not seen this problem before because I was using notebook folders
created by previous versions. If I try to pass a new folder to
notebook() command, all worksheets are stopped on signing out. This is
a major bug that has to be fixed before 5.4 is out, I think! I always
tell new users how great it is that you can move to another computer
and pick up right where you have left, now it is not possible.

[Jason Grout]

Interesting---I was not aware of this 'feature'! I don't know if it is
a bug, a feature, or just an incomplete design.

Thanks,

Jason

[Keshav Kini]

IMO that shouldn't be default behavior. If the user purposely signs out,
his worksheet's process should be killed, otherwise surely you would
have way too many processes running even if nobody was using the server!

-Keshav

----
Join us in #sagemath on irc.freenode.net !

[Andrey Novoseltsev]

On Sep 13, 11:21 am, Keshav Kini <keshav.k...@gmail.com> wrote:
> IMO that shouldn't be default behavior. If the user purposely signs out,
> his worksheet's process should be killed, otherwise surely you would
> have way too many processes running even if nobody was using the server!

I strongly believe that the old behaviour should be back because:

1) This WAS the default behaviour which was very easy to discover (you
sign in and see your worksheets as "running" before you do anything),
so it is a regression to randomly drop it without any warning/
deprecation period.
2) The problem of too many processes is solved by setting a timeout
period for inactive worksheets, this is a very convenient solution and
it worked for many years (since the start of the notebook?).
3) It allows running long computations (days, weeks, ...) through
notebook interface without remembering to keep connected/not signing
out.
4) Computations that take a few minutes are way more convenient to
access from several computers if you don't need to rerun everything
each time when you connect to the server.
5) I didn't investigate it, but how will closing of a window/loss of
connection/crash of a browser be handled? Everything will get killed
as well loosing all the work done?..

There were problems associated with having the same worksheet open on
different computers when they sleep/hibernate, so it seems to be good
to at least close a window on one computer when you move to another.
If this required rerunning all computations of a session on another
computer, it would be a major show stopper for me and I imagine at
least some other people... I don't know how to fix it, but I certainly
would greatly appreciate if someone took care of it.

Thank you,
Andrey

[Keshav Kini]

Andrey Novoseltsev <novo...@gmail.com> writes:
> 3) It allows running long computations (days, weeks, ...) through
> notebook interface without remembering to keep connected/not signing
> out.

This is a very good point...

[Jason Grout]

Those are all pretty persuasive arguments.

I just checked on sagenb.org (which is using an *old* notebook
directory), and it seems to stop worksheets when you log out. I checked
on my current personal notebook (which IIRC is using a freshly created
notebook directory) and it still stops worksheets when you log out. So
I'm not sure that it's the creation date of the notebook that
differentiates the behavior.

I suppose this would be a good job for git bisect...

Thanks,

Jason

[Marshall Hampton]

I totally agree with Andrey. I use a lot of different computer & sage
server combinations, and I often rely on the fact that the worksheet
processes do not stop unless "Save and Quit" is clicked.

-Marshall Hampton

[Jeroen Demeyer]

I don't know what should be done with this issue. I strongly prefer not
to revert to the previous sagenb-0.9.2, because there are a lot of good
things in sagenb-0.10.x.

I was planning anyway to make a sage-5.4.rc1, hopefully the final rc for
this version. But I don't want to wait too long for this, maybe at most
a week or so. Otherwise people complain that Sage releases are too slow...

Jeroen.

[kcrisman]

On Monday, October 1, 2012 3:07:21 PM UTC-4, Jeroen Demeyer wrote:

I don't know what should be done with this issue.  I strongly prefer not
to revert to the previous sagenb-0.9.2, because there are a lot of good
things in sagenb-0.10.x.

Speaking of which, did the "automatic static" issue ever get resolved?  I feel like #13121 was merged because there was an implicit commitment to at the very least implement Jason's workaround.  I said there, about whether it was ready to merge:

"As long as Jason and Jonathan can commit to figuring out what to do about this issue, then yes. One would hope that the 5.4 beta sequence would be long enough to accommodate this. The Jmol upgrade is otherwise really worth the effort." 

But perhaps this was a little too optimistic?  I agree that reverting to the previous notebook would be very unfortunate.  But there isn't any more mention of that issue on that ticket.

[Andrey Novoseltsev]

I am a bit confused with the general workflow for the new notebook:
what am I supposed to do with bugs found in Sage betas? So far I've
reported them on this list:

- It is not possible to stay connected to several notebook servers.
- Worksheets get stopped on signing out (not always...).
- Jmol requires a lot of virtual memory to generate static plots.
- Jmol requires different permissions for generating its own plots on
cell folders.
- Jmol shows "Make interactive" button in published worksheets but it
does not work.

I guess I should also open issues on github, but how are they tracked
within Sage? Should there be "shadow tickets" for them? That seems
like unnecessary duplication.

Andrey

[Jonathan Gutow]

What am I committing too?  

I still have not seen a ticket that lists what exactly people would prefer.   I understand than some of you preferred the old behavior.  But you must remember that the old behavior lead to crashes and hangs.  The change to a default static plot was implemented to solve reported reproducible bugs.   Without a complete rewrite of interacts we will not get reliable behavior out of them except on a fast connection with a fast user machine.  That said, I think some kind of a switch allowing for Jmol to launch live in interacts and on worksheets would be OK.  It will just be a case of user beware, in the sense that having Jmol live in interacts will lead to random hangs.  

I need someone who is working on the whole worksheet to decide what variable to put that switch in.  Once the switch variable and a checkbox for it is in it should not be hard to check it to decide whether to launch Jmol live or the static image. 

As to memory constraints for Jmol on the server.  It is using no more memory than the applet (512 -1024 MB) would use on the user's machine.  If you are trying to run jobs on the server with less memory than you expect the user to have, I think you are probably being overly optimistic.  In general a sage process needs access to a lot of memory because of all the things it might need to run. Jmol is not the only thing that uses a significant amount of memory.  Jmol would be much slower if it tried to work in a smaller amount of RAM.  Nice images are costly both in terms of memory and processing.

Jonathan

--
You received this message because you are subscribed to the Google Groups "sage-notebook" group.
Visit this group at http://groups.google.com/group/sage-notebook?hl=en-US.
 
 

                         Dr. Jonathan H. Gutow

Chemistry Department                                gu...@uwosh.edu

UW-Oshkosh                                          Office: 920-424-1326

800 Algoma Boulevard                                FAX:920-424-2042

Oshkosh, WI 54901

                http://www.uwosh.edu/facstaff/gutow

[Jason Grout]

On 9/13/12 1:15 AM, Andrey Novoseltsev wrote:
> Using the same account for server and worker seems to keep
> old behaviour - worksheet processes continue running until the next
> connection.

Can you confirm that you logged out and logged back in and the old
behavior was preserved? It works if you just close the window, but I
can't get the old behavior to happen even if the same account is used
for the server and worker.

I just realized why this has never bothered me. I've never signed out
when I want a computation to keep running. If I just close the browser
window, the worksheets keep running just fine.

Thanks,

Jason

[kcrisman]

I just realized why this has never bothered me.  I've never signed out
when I want a computation to keep running.  If I just close the browser
window, the worksheets keep running just fine.

Of course!  I always assumed that signing out would kill stuff (though I should have known not, based on lab experiences with students).   

[Andrey Novoseltsev]

On Oct 2, 12:15 pm, Jason Grout <jason-s...@creativetrax.com> wrote:
> On 9/13/12 1:15 AM, Andrey Novoseltsev wrote:
>
> > Using the same account for server and worker seems to keep
> > old behaviour - worksheet processes continue running until the next
> > connection.
>
> Can you confirm that you logged out and logged back in and the old
> behavior was preserved?  It works if you just close the window, but I
> can't get the old behavior to happen even if the same account is used
> for the server and worker.

I've reported above that it is unrelated to different accounts. It
seems to depend on whether the notebook folder was created by an old
version or not - I didn't investigate it in detail as it is not really
important, newly created notebook servers have this problem
consistently.

[Andrey Novoseltsev]

I have never assumed so. I have a server running and then I open a
window to interact with it, so signing out means that I stop
interacting, but the server continues to run and I expect that
computations will continue. I don't think that I am alone in this
interpretation.

Relying on closing the browser without signing out is bad and has
security problems - the browser may stay signed in for the next
session meaning that someone else can get access to the notebook
account if it is a public computer.

Note also that one may need to switch users - e.g. I am working as
Instructor, a student asks me to reset his/her password, which
requires me to login as admin. I cannot do so without signing out
first which would kill all my running worksheets.

[Jason Grout]

On 10/2/12 3:42 PM, Andrey Novoseltsev wrote:
> I didn't investigate it in detail as it is not really
> important

I was hoping that if we could find a case where the old behavior still
happened, it would make it a lot easier to track down where the problem is.

Thanks,

Jason

[Jason Grout]

I agree with your reasoning. I was just pointing out why I never
noticed the problem.

Thanks,

Jason

[Andrey Novoseltsev]

Well, 5.2 also has this problem as I've just checked - if I create a
new notebook folder, worksheets get stopped on signing out. It is a
bit weird that it does not happen (or does not always happen) with
"precreated" folders. So it seems like a regression of the switch to
the flask notebook, not just 0.10.x. I don't have older versions
ready, but I have never experienced such behaviour with prior versions
and I often rely on continuing execution. I thought that someone who
knows what happens/gets executed when the user presses "Sign out"
could fix it easily, but I may be wrong, of course.

Thank you,
Andrey

[Jason Grout]

Quite a bit changed in the architecture when we went to flask, so it may
not be so easy. However, looking at the code, it *may* be as easy as
commenting out these three lines:

https://github.com/sagemath/sagenb/blob/master/sagenb/notebook/notebook.py#L1510

Would you like to test that? Right now, I'm not sure if there are other
ramifications of deleting that code, but at least that would give us
something to go on.

Thanks,

Jason

[Jason Grout]

Indeed, it looks like previously the code just popped the username off
the session without quitting worksheets:
https://github.com/sagemath/sagenb/commit/b0043f0c7aa949c1f787cc03a13e4dc8089f2bae#L0R74

Thanks,

Jason