Coming SageMathCell upgrade - please test!

[Andrey Novoseltsev]

Hello and sorry for cross-posting, I'm trying to maximize visibility.

After quite a few changes I have a version of SageMathCell running Sage-7.1 and using Jupyter etc. shipped with Sage, rather than a custom old branch of IPython.

User-visible changes should be minimal, but there are some for script interaction that will probably require changes to Android app, WebWorks, and what else.

There may be also new bugs introduced in the process. I hope that most of the stuff is done correctly and I will do some more testing/fixing/documenting over the next month or so, but it would be nice if other people tried it with their favourite code, interacts, web-pages, etc. To help with this I've pushed all changed to master, so those who have their own installations should be using the new version, and I have also set up a test instance on our Madrid server:
http://cosmos.mat.uam.es:8888/
for comparison you can also use old version at
http://cosmos.mat.uam.es/

Thank you!
Andrey

[Kwankyu Lee]

The webpage from the Madrid server opens but contains no Sage cell. My web browser is Chrome on Mac. Accessing from Asia might be a cause.

[mmarco]

Same Here.

[mmarco]

Forgot to mention, I am using chromium in Linux.

[Bruno Grenet]

Works fine on Firefox 45.0 in Linux Mint.

> --
> You received this message because you are subscribed to the Google
> Groups "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to sage-devel+...@googlegroups.com
> <mailto:sage-devel+...@googlegroups.com>.
> To post to this group, send email to sage-...@googlegroups.com
> <mailto:sage-...@googlegroups.com>.
> Visit this group at https://groups.google.com/group/sage-devel.
> For more options, visit https://groups.google.com/d/optout.

[Dima Pasechnik]

works on firefox, but does not give a window to evaluate on Chrome, 49.0.2623.87 (64-bit) (both browsers the same machine, on Ubuntu 14.04)

[Thierry]

Hi,

it works almost well on iceweasel (Firefox 38.7.1) on Debian jessie
(stable). I see however two issues.

First, autocompletion does not work, when i click on <TAB>, the cursor
jumps on "Evaluate" button. I tried on the older version and it works, so
it is a regression.

Second, when trying a plot3d (type "dodecahedron()"), i got an image, but
when i click to animate it, i got an error message:

Error connecting to server:
http://your.server.here/jsmol.php?call=getRawDataFromDatabase&database=_&query=http%3A%2F%2Fcosmos.mat.uam.es%3A8888%2Fkernel%2F7af3de4b-ec65-44a7-8f53-f279879ffa9d%2Ffiles%2F.%2FtmpQwiiT9.jmol%2Fscene.zip&encoding=base64

It seems that you have to replace your.server.here somewhere in the conf.

Actually, i am noticing that the same error appears on the current
version, i am not using sagecell, so i did not notice that failure before
(this test is part of my "human testsuite" for releasing Sage Debian Live).

Ciao and thanks for maintaining sagecell,
Thierry

> --
> You received this message because you are subscribed to the Google Groups "sage-devel" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
> To post to this group, send email to sage-...@googlegroups.com.

[Eric Gourgoulhon]

Hi,

Le lundi 18 avril 2016 10:58:48 UTC+2, Thierry (sage-googlesucks@xxx) a écrit :

Hi,

it works almost well on iceweasel (Firefox 38.7.1) on Debian jessie
(stable). I see however two issues.

First, autocompletion does not work, when i click on <TAB>, the cursor
jumps on "Evaluate" button. I tried on the older version and it works, so
it is a regression.

I confirm; same problem here with Firefox 43.0.4 / Ubuntu 15.10.

Second, when trying a plot3d (type "dodecahedron()"), i got an image, but
when i click to animate it, i got an error message:

Error connecting to server:
http://your.server.here/jsmol.php?call=getRawDataFromDatabase&database=_&query=http%3A%2F%2Fcosmos.mat.uam.es%3A8888%2Fkernel%2F7af3de4b-ec65-44a7-8f53-f279879ffa9d%2Ffiles%2F.%2FtmpQwiiT9.jmol%2Fscene.zip&encoding=base64

It seems that you have to replace your.server.here somewhere in the conf.

 

dodecahedron() works well for me with Firefox 43.0.4 / Ubuntu 15.10.

Best wishes,

Eric.
 

[paulmasson]

On Monday, April 18, 2016 at 1:58:48 AM UTC-7, Thierry (sage-googlesucks@xxx) wrote:

Hi,

it works almost well on iceweasel (Firefox 38.7.1) on Debian jessie
(stable). I see however two issues.

First, autocompletion does not work, when i click on <TAB>, the cursor
jumps on "Evaluate" button. I tried on the older version and it works, so
it is a regression.

Second, when trying a plot3d (type "dodecahedron()"), i got an image, but
when i click to animate it, i got an error message:

Error connecting to server:
http://your.server.here/jsmol.php?call=getRawDataFromDatabase&database=_&query=http%3A%2F%2Fcosmos.mat.uam.es%3A8888%2Fkernel%2F7af3de4b-ec65-44a7-8f53-f279879ffa9d%2Ffiles%2F.%2FtmpQwiiT9.jmol%2Fscene.zip&encoding=base64

For Safari 9.1 / OSX El Capitan and iOS 9.3.1 this message is preceded by another:

JSmolCore.js: synchronous binary file transfer is requested but not available

 The error messages do not appear for Firefox 45.0.2 / Windows 10 and the dodecahedron is functional.

[Andrey Novoseltsev]

To summarize what was discovered so far:

1) Chrome does not work: I unfortunately tend to forget that different
browsers/OS should be tested - I was working on Firefox under Debian,
will try to fix is next weekend. Location should not matter when you
use the name of the server directly (sagecell.sagemath.org does
resolve to Madrid for certain locations).
2) TAB completion does not work: also didn't test as I rarely write
code in cells, will try to fix next weekend as well.
3) 3d plots have issues: these are likely to be the same as in the old
version - it will be a high priority to improve the situation after
the switch to Jupyter is done.

Any issues with interacts or embedding?

[paulmasson]

Testing a page with eight embedded cells in Firefox 45.0.2 / Windows 10. Assuming that <script> tags should be updated to

<script src="http://cosmos.mat.uam.es:8888/static/jquery.min.js"></script>
<script src="http://cosmos.mat.uam.es:8888/static/embedded_sagecell.js"></script>

This error message is generated by both jQuery and embedded_sagecell.js:

Using //@ to indicate sourceMappingURL pragmas is deprecated. Use //# instead

Don't know if that is fatal, but also getting a CORS error for each embedded cell:

Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at http://cosmos.mat.uam.es:8888/kernel?notebook=E4912ABA357D4BA8A6F0F925CB38B525&timeout=0&accepted_tos=true. (Reason: CORS header 'Access-Control-Allow-Origin' missing).

This is followed by the succinct message:

API request failed (404): error

Then the embedded cells stay blank with the evaluation graphic spinning and spinning.

[Andrey Novoseltsev]

Update:

1) Chrome does not work because of executing
sagecell.makeSagecell(...)
before
<script src="{{ static_url('embedded_sagecell.js') }}"></script>
finishes while Firefox waits till the script is processed. The issue
goes away if I wrap makeSagecell call in require(...), but I have no
idea how to make it work without, which is necessary for supporting
old pages. Any pointers?... I can't say I am a big fan of RequireJS,
but it felt natural to use it because Jupyter uses it. Should I
perhaps use something else instead/in addition?

2) TAB-completion should be fixed.
3) No progress on 3D yet.
4) Didn't look into CORS problems with embedding yet.

[Volker Braun]

sagecell.makeSagecell is already polling to see whether it is loaded; thats of course totally fugly but should work. You just have to make sure that it is defined. If its only defined once a requirejs module loads then you don't have control over when. Instead move the

window.sagecell = window.sagecell || {};

sagecell.makeSagecell = function (args, k) {

...

}

outside of requirejs to always define the entrypoint into sagecell.

[Andrey Novoseltsev]

Made it work in Chrome via
https://github.com/sagemath/sagecell/commit/95d2b406026ea7efc73a91ef693a1dd9fb5946c6
(my attempts to create all wrappers using a loop were not successful)

Made some progress on CORS but there are still issues.

> --
> You received this message because you are subscribed to the Google Groups

> "sage-notebook" group.

> To unsubscribe from this group and stop receiving emails from it, send an

> email to sage-noteboo...@googlegroups.com.
> Visit this group at https://groups.google.com/group/sage-notebook.

[Andrey Novoseltsev]

OK, embedding seems to work for Chrome and Firefox, so reported
regressions are fixed.

I am planning to switch main servers in 3 weeks. (Probably 7.2 will be
out by then, so I'll also see how easy it is to upgrade with the new
setup.)

[paulmasson]

Andrey, the test server is not completing evaluations. Just get the spinning GIF.

Also, I'm curious as to why you're making modifications in the master branch of the sagecell repository. I happened to be reading the JavaScript source code right when you split it into smaller files yesterday, and it was a bit disconcerting. Shouldn't you be making changes of that sort in a development branch and merging with the master branch when stable? If someone grabs a copy of the master branch when you're making significant changes, the results will be confusing in the very least.

[Andrey Novoseltsev]

Which browser OS are you using? I just checked now and have the same
issue with Firefox on Windows. Chrome on Windows seems to work (I even
have working 3d plot with JSmol, which is a bit unexpected!).
Yesterday both Firefox and Chrome worked under Linux.

Regarding changes - I was using develop branch but it got so different
from "old master" that looking/asking about it was pointless. I also
consider the current state more or less stable and requiring only some
bug fixes - moving files was not anticipated. And I really feel the
need to see how it works on production servers preferably with the
"default" setup for branches. Last but not least - I was not aware of
anyone actually looking at the code and trying to do something with it
(Volker helped with require.js and I imagine it was convenient that
going to github was showing relevant files right away). Anyway, as you
can see last commits are quite small apart from splitting a file, so
hopefully I won't cause much more trouble and thank your for testing!

> --
> You received this message because you are subscribed to a topic in the
> Google Groups "sage-devel" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/sage-devel/jPLfIbt048Q/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to

> sage-devel+...@googlegroups.com.
> To post to this group, send email to sage-...@googlegroups.com.

> Visit this group at https://groups.google.com/group/sage-devel.

[paulmasson]

Firefox on Windows 10. Firefox on OS X El Capitan has the same problem.

Chrome and Safari on OS X are both functional.

[Andrey Novoseltsev]

On Monday, 2 May 2016 20:18:06 UTC-6, paulmasson wrote:

Andrey, the test server is not completing evaluations. Just get the spinning GIF.

 Turns out I've changed some logic when fixing Chrome and Firefox got affected but not others. Now things work for me in Firefox and Chrome under Linux and Windows...

[Andrey Novoseltsev]

The test server
http://cosmos.mat.uam.es:8888/
is now running 7.2.rc2 with upgraded ipykernel which greatly helped
with stability.

If no major issues are discovered by the next weekend (which will be
long for me and I will have a bit more time), I will roll this version
to all public servers and proceed fixing problems from there.

One problem is how 3d plots work (or rather don't), but given their
current state I don't consider it a blocker.

Thank you!
Andrey

[paulmasson]

I just tested a page with several embedded plots and three numerical WKB calculations. The plots work fine but the WKB calculations produce the error message:

TypeError: unable to simplify to float approximation

This error does not occur for the servers at cosmos.mat.uam.es or sagecell.sagemath.org. I can post all the code and the complete error message if you want them.

On a related note, I've been experiencing evaluation problems with sagecell.sagemath.org over the last couple weeks. Embedded cells do not always evaluate, but just have the spinning indicator with no output. I don't think it's a timeout issue, since that would change the cell border from green to black which isn't happening, and the cells evaluate one at a time just fine.

Is this an error that should go away with the new version? Thanks.

[Andrey Novoseltsev]

On Sunday, 15 May 2016 17:18:37 UTC-6, paulmasson wrote:

I just tested a page with several embedded plots and three numerical WKB calculations. The plots work fine but the WKB calculations produce the error message:

TypeError: unable to simplify to float approximation

This error does not occur for the servers at cosmos.mat.uam.es or sagecell.sagemath.org. I can post all the code and the complete error message if you want them.

Please give the code, it is almost impossible for me to resolve issues that I can't reproduce.
 

On a related note, I've been experiencing evaluation problems with sagecell.sagemath.org over the last couple weeks. Embedded cells do not always evaluate, but just have the spinning indicator with no output. I don't think it's a timeout issue, since that would change the cell border from green to black which isn't happening, and the cells evaluate one at a time just fine.

Is this an error that should go away with the new version? Thanks.

One may hope and we will see ;-) One of the problems was that stdout/stderr of worker processes could get full and then the process locks, but since there are many workers it takes some time for the health check script to notice issues and restart the server. Another one was a race condition in buffer access. Also, what is sagecell.sagemath.org for you, i.e. what IP does it resolve to - cosmos or GCE?
 

[paulmasson]

sagecell.sagemath.org is 130.211.179.63 for me in San Francisco.

Link to code on production server.

Link to code on test server for error message.

[kcrisman]

I'm not sure what you mean - they seemed to work fine for me this spring on the "normal" Sage cell server?  Whereas this test server I can't even get "sphere()" to appear.  That seems like a fairly large regression.  Or is it because my IP is not in Europe? 

[Andrey Novoseltsev]

On Sunday, 15 May 2016 19:29:56 UTC-6, paulmasson wrote:

sagecell.sagemath.org is 130.211.179.63 for me in San Francisco.

Link to code on production server.

Link to code on test server for error message.

If I remove @interact (which invokes different code in each frontend including SageMathCell), I still get the same error message and traceback shows only Sage files:

http://cosmos.mat.uam.es:8888/?z=eJzFVdtu2zAMfS_QfyCwh8gNU1-SdhfA_7D3ICkUR3EFyJIry62zrx-l-BL0MnRbsQkILIvU4TkkQ-_FAQQDjqAh-nZ5AWCFa60GBs2DdayWcRZBDNmWpTGP4ApKXlXcv8yXJ9N0kHo70_PUG6Itg-yKx4zP_dvlxeXFnoI9MOjwlXh817Au2nJYQEYognHU4ZLijYMckgHg0OqUCOcsQ5ZgmkRRAPJQvTkj7o47kTdCicIZy9bkiBkucYU3eIuf8Qt-3SC581a53EeYQABKZXZcgT8-HfQUAujgFEjnlKaBbbAG2T5Rg9snKLjWxkHdOtBtJawsuLqT2onSUpC9tERRHUFqqKUoxJNshBdROGn0gGE5nTYgrDUWZq3mOyXAGWhkVSt5OPr9QRnugNe1NZ2suL8-G-5rIfbehxRLLeD70d0bPUZpqBjEsDlWO6NkMZ0PInxerSjpiDLf9XXz65FbNmtn0fBOsignL2X2zUTFb3HKFPpOWIRERutkM4CMDcgWabQNzpTivht9hz2wbkTZklO8GglMixoRqEHp6UmFqtz6y1za4x2X7Lk5i5dECaKXmrO_07x4LjooRo_5O6IDUIfnffanyhfvlr78YOlU77fL_b_Frj62t39Z5X_dzbrdl4LoXydJSvOARoEVlXkUUBtFg4UGhycmdUkHBDH-85_4o782TiYGaxjYrYns9c1V0Bl6ehGikPR-VsAGvZM3zYMJk9GYAVknpARfAVieAM7uD_FGlxX0ud1MWmtlXONJ05MFBQgHWTbyh8jXK8wI1Ja7wihjcwp8fYMp4Xnvu5P4PE3wZQnGdaykzkn6V6Qt73LaQXQeej7EPi9tnJ3G3ZgymHbvjv2mjBQTSu6Ug-bePLGeTuDmf-HTmASf8BHNop8AOBqd&lang=sage

so it seems to me that the problem is with this code not working in Sage 7.2, while 6.9.beta6 (running in current public cell servers) was OK. My interacts with piecewise functions also got a bit broken (still work, but show weird warnings). In general, unfortunately, it is quite likely that a bunch of involved interacts will break due to a jump in Sage version. But there is not much I can do about it and waiting longer will make only worse. On the bright side, the academic year is more or less over.

[paulmasson]

Andrey, I've confirmed that the error comes from the piecewise function and not the interacts. Looks like a bug in Sage 7.2, not in the test server.

[Andrey Novoseltsev]

OK, I've pressed the button - http://sagecell.sagemath.org is now
running the latest version of Sage 7.2, together with a "significantly
touched up" version of the SageMathCell code itself.

Please report any new (or old) errors that you notice - I will try to
fix them tomorrow (Saturday) afternoon (MST) and on Sunday/Monday. If
something is horribly wrong and I can't resolve it by Monday evening,
I'll pull back to the old version.

Thank you,
Andrey

[paulmasson]

Andrey, I have a bunch of simple interacts in Github pages of this sort

@interact

def _( n=slider(0,10,step_size=1) ):

  show( plot( bessel_J(n,x), x, 0, n+20, figsize=[4,2], color=(0,.5,1) ) )

that randomly fail to interact. The server returns an initial image but the interact doesn't function. The only error message I'm seeing in the JavaScript console is

The connection to wss://sagecell.sagemath.org/sockjs/699/p0wm4ywe/websocket was interrupted while the page was loading.

Perhaps you're already working on a solution, but thought I should let you know.

[paulmasson]

Not a bug in Sage 7.2, just lots of new behavior for piecewise.

[Andrey Novoseltsev]

On Sunday, 22 May 2016 14:01:26 UTC-6, paulmasson wrote:

Andrey, I have a bunch of simple interacts in Github pages of this sort

@interact

def _( n=slider(0,10,step_size=1) ):

  show( plot( bessel_J(n,x), x, 0, n+20, figsize=[4,2], color=(0,.5,1) ) )

that randomly fail to interact. The server returns an initial image but the interact doesn't function. The only error message I'm seeing in the JavaScript console is

The connection to wss://sagecell.sagemath.org/sockjs/699/p0wm4ywe/websocket was interrupted while the page was loading.

Perhaps you're already working on a solution, but thought I should let you know.

Thanks for letting me know - I didn't know this is happening and so far can't reproduce it with this example. How frequent is it? I have pushed a "forced update" a couple hours ago that was not waiting for current users to stop. That may have caused it but it would be over in a few minutes. Do you still have these problems now?

[paulmasson]

Still getting the problem. Occurs on about half of page loads right now. Here's a link to an index page for the repository:

http://paulmasson.github.io/sagemath-docs/functions.html

The interacts are on pages for anything with an index, so try the Bessel or elliptic functions. Not all of the links on this page lead to existing pages yet.

[Andrey Novoseltsev]

On Sunday, 22 May 2016 15:27:26 UTC-6, paulmasson wrote:

Still getting the problem. Occurs on about half of page loads right now. Here's a link to an index page for the repository:

http://paulmasson.github.io/sagemath-docs/functions.html

The interacts are on pages for anything with an index, so try the Bessel or elliptic functions. Not all of the links on this page lead to existing pages yet.

OK, so for Bessel page it seems that on reloading the page and doing nothing, some of the interacts look fine but are disabled. Will look into it. For now I am trying to make warnings more sensible. (Not show stuff about deprecated guts of IPython and hopefully the actual user line that caused it.)
 

[Andrey Novoseltsev]

On Sunday, 22 May 2016 15:27:26 UTC-6, paulmasson wrote:

Still getting the problem. Occurs on about half of page loads right now. Here's a link to an index page for the repository:

http://paulmasson.github.io/sagemath-docs/functions.html

The interacts are on pages for anything with an index, so try the Bessel or elliptic functions. Not all of the links on this page lead to existing pages yet.

This page seems to be good for testing:
http://paulmasson.github.io/sagemath-docs/functions/elliptic_f.html
I was able to make a local copy of it and reproduce the error, but only when connecting to a remote server and it seems that interact consistently fades "on the count of 5". Perhaps because of simultaneous load of several auto-eval cells some timeout is exceeded. Will keep working on it.

[kcrisman]

Please report any new (or old) errors that you notice - I will try to
fix them tomorrow (Saturday) afternoon (MST) and on Sunday/Monday. If
something is horribly wrong and I can't resolve it by Monday evening,
I'll pull back to the old version.

http://ask.sagemath.org/question/33555/problem-with-encoding-german-characters/ could be related, or so the poster seems to imply - though probably not bad enough to go back before 7.2.

[Andrey Novoseltsev]

On Thursday, 26 May 2016 10:34:07 UTC-6, kcrisman wrote:

http://ask.sagemath.org/question/33555/problem-with-encoding-german-characters/ could be related, or so the poster seems to imply - though probably not bad enough to go back before 7.2.

Thanks a lot for pointing it out - this is the worst problem I am currently aware of and it is definitely new. Would be solved by a switch to Python3, I guess ;-)

Overall things seem to be good enough that I don't plan a reversal anymore.

[Ivan Andrus]

Over the past few days I’ve gotten several complaints about the iOS sage app being broken.  Unfortunately, I missed the message that you were upgrading the cell server, so I didn’t know where to look immediately.

The method that the app uses is to load a webpage with an embedded sage cell in it, with some options other stuff to communicate back to the app.  The template webpage is 

https://bitbucket.org/gvol/sage-ios-app/src/c688df944ae283e1bb080ebabaec18dec584b2fa/template.html?fileviewer=file-view-default

Perhaps you will be able to look at it and notice that I’m doing something bad.  

-Ivan

[Andrey Novoseltsev]

This should be fixed now - I had to encode user code when writing to a file for correct warning messages.

[paulmasson]

Andrey, any idea when my embedded interacts will start working again? Thanks.

[Andrey Novoseltsev]

On Thursday, 2 June 2016 16:39:53 UTC-6, paulmasson wrote:

Andrey, any idea when my embedded interacts will start working again? Thanks.

Hopefully by Monday. I still don't know what exactly is happening, but I didn't work much on it since last weekend was short on time in general and I was looking into CodeMirror problems.

[Andrey Novoseltsev]

On Thursday, 2 June 2016 16:39:53 UTC-6, paulmasson wrote:

Andrey, any idea when my embedded interacts will start working again? Thanks.

Should be working now, please test. There was a true check (undefined === undefined), I think I'd prefer AttributeError:
https://github.com/sagemath/sagecell/commit/94616895636418000c45478e080eb1f2ca7a50fc

[Paul Masson]

All interacts appear to be working again. Thanks!

Sent from my iPad

--

[Peter Luschny]

---------------------------------------------------------------------------

ImportError                               Traceback (most recent call last)

<ipython-input-1-2ef73eaf42ab> in <module>()

----> 1 load('http://....')

      2 load('http://....')

      3 

/home/sc_serv/sage/src/sage/structure/sage_object.pyx in sage.structure.sage_object.load (/home/sc_serv/sage/src/build/cythonized/sage/structure/sage_object.c:11408)()

   1016 

   1017     if sage.repl.load.is_loadable_filename(filename):

-> 1018         sage.repl.load.load(filename, globals())

   1019         return

   1020 

/home/sc_serv/sage/local/lib/python2.7/site-packages/sage/repl/load.pyc in load(filename, globals, attach)

    252             raise NotImplementedError("you can't attach a URL")

    253         from sage.misc.remote_file import get_remote_file

--> 254         filename = get_remote_file(filename, verbose=False)

    255 

    256     from sage.repl.attach import load_attach_path

/home/sc_serv/sage/local/lib/python2.7/site-packages/sage/misc/remote_file.pyc in get_remote_file(filename, verbose)

     34 

     35     # import compatible with py2 and py3

---> 36     from six.moves.urllib.request import urlretrieve

     37 

     38     global cur

ImportError: No module named moves.urllib.request

[Andrey Novoseltsev]

This is due to the bug fixed by http://trac.sagemath.org/ticket/20797

Also, I'll try to keep further discussion of SageMathCell at https://groups.google.com/forum/#!forum/sage-cell