* Certain features of SageNB are missing: live documentation,
sharing/publishing of worksheets.
* It clutters the file system with .ipynb files.
IMHO the Jupyter/IPython notebook is a clear winner, no real contest.Nobody prevents you form using SageNB for existing worksheets or the live documentation; But if you start a new worksheet today then you really should be using Jupyter.
On Friday, December 18, 2015 at 11:35:53 AM UTC+1, Jeroen Demeyer wrote:* Certain features of SageNB are missing: live documentation,
sharing/publishing of worksheets.IMHO sharing is already better, you can just upload the ipynb file anywhere (including http://gist.github.com or any other pastebin) and then share it using the http://nbviewer.ipython.org service. For example:
Notebooks in sagenb are basically individual files; it's just that
they are stored as a directory tree in the filesystem.
https://github.com/sagemathinc/smc/blob/master/src/smc_pyutil/smc_pyutil/sws2sagews.py
It will be much easier for our users to actually *use* a converter if
we delay changing the defaults until after such a converter is
written.
different notebook. We could possibly alleviate the problem of lack of information by somehow guessing if this is the first time this user starts up sage ipython notebook
If the default changes, then most users will assume they are supposed to change with it (and we should only do this once we think users should change, e.g., when interacts work)
At the very least there should be a method to:
- start the classic notebook
On Monday, December 21, 2015 at 5:16:44 AM UTC+1, Nils Bruin wrote:different notebook. We could possibly alleviate the problem of lack of information by somehow guessing if this is the first time this user starts up sage ipython notebookThats inconsistent UX, now you open different notebooks on different computers / accounts. Unless you mean: print help to the terminal. The ticket does that already.
We could make the UX more consistent by "preinstalling" a migration ipy notebook.
In fact, that notebook could include code to open and read the sagenb worksheets (jupyter kernels don't run chrooted by default, right? so their code can still access HOME/.sagenb)
Regardless of how "useful" or awesome the Jupyter notebook is and how many people in scientific computing are using it, there is a substantial ecosystem now designed around the Sage notebook proper
And we are not removing it for all the legacy reasons that you listed. This thread is about what format new notebooks are supposed to be. Anybody who prefers SageNB can still use SageNB
But the fact is: Its unmaintained/-able, and the longer we wait the more painful it'll be to switch. How is waiting another year going to help? Because by then there will be another book explaining how to use SageNB? Do you think the book author is going to be happy that we never made it clear that SageNB is on the way out? SageNB is an evolutionary dead end, and the only question is how deep into that hole do we want to go.
And we are not removing it for all the legacy reasons that you listed. This thread is about what format new notebooks are supposed to be. Anybody who prefers SageNB can still use SageNBBut how easy will that be? It already sounds like sagenb notebooks are not really portable to Jupyter.But the fact is: Its unmaintained/-able, and the longer we wait the more painful it'll be to switch. How is waiting another year going to help? Because by then there will be another book explaining how to use SageNB? Do you think the book author is going to be happy that we never made it clear that SageNB is on the way out? SageNB is an evolutionary dead end, and the only question is how deep into that hole do we want to go.As William said, the answer is to either find a way to make them port, or to find a way to gracefully detect that someone has existing sagenb worksheets (perhaps slightly more gracefully than just checking DOT_SAGENB since that doesn't mean they actually did anything) and either ask to upgrade to Jupyter worksheets, if that's possible, or make it very easy to change a default setting.
Could there be an environment variable DEFAULT_SAGE_NOTEBOOK that one could have incredibly explicit instructions for how to use? (Would making an alias work in this context? Not in the sage: notebook() situation, I guess.)But it has to be idiot-proof. Imagine the following conversation at the Sage table:Customer (yes they are customers!): Will I ever have any problems with upgrading?Sage rep: Yep, we will make it so all your old stuff is never upgradeable with essentially no warning other than some cryptic command line thing you don't even see because you only look at your browser!Customer: I didn't hear you right, I think? You mean, "we'll support legacy stuff so long it will make Microsoft look good." I mean, I don't want to still be using .doc in 2020.Sage rep: Nope, we're sort of the opposite; we won't have any 'compatibility mode' nor any incentive to upgrade your old documents; you either go to the new one - oh, by the way, it doesn't do everything the old one does yet - or you just stick with the old kind forever.Yup, I see that as a real winner. And of course this conversation can happen again whenever the next new hit comes out in five years, which it no doubt will.Writing up a very accessible (perhaps even linked to in the warning) translation guide to getting started in Jupyter would be good too. As much as there might be a bigger ecosystem for them, that is not a very compelling argument to me; otherwise we should all be using RStudio and shiny, and well basically just using R for everything. How many of those zillions of easy-to-search sample worksheets are about something like calculating elliptic curve invariants or a nice demo for a graph theory course? Probably just a tiny proportion of the ones about data science or scientific computing.On a separate note, I'm wondering how changing the default will impact those running servers. Which is a higher knowledge crowd but also one who might not be reading sage-devel (in fact likely isn't). Basically, I'm asking about any worst-case scenarios when someone upgrades from (say) 6.7 to 7.2 at the end of a school year and either doesn't read the 'release notes', doesn't even know there are release notes, or our release notes are so minimal as to be useless, or something. (I assume that commercial software has that problem of non-reading too.)
--
How many of those zillions of easy-to-search sample worksheets are about something like calculating elliptic curve invariants or a nice demo for a graph theory course?
--
You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
To post to this group, send email to sage-...@googlegroups.com.
Visit this group at https://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.
You still haven't understood that this is about how to create new notebooks. We have the legacy support covered by shipping the old notebook. Also, I don't recall Microsoft bundling a free copy of word 2007 with word 2010, you are comparing apples and oranges.
I'm adding a Jupyter notebook extension to the sagenb exporter (https://github.com/vbraun/ExportSageNB) to hook into the GUI. So my proposal would be that the initial window will be something like this:
Should the Jupyter notebook be the default notebook for the next Sage
7.0 release? I don't really have an opinion on the matter.
Pros:
* Nice tracebacks!
* The Jupyter notebook is a mature well-maintained project, unlike
SageNB. It is widely used in the "scientific Python" community.
* Availability of other Jupyter kernels besides Sage.
Cons:
* Less compatible with Sage: Sage interacts don't work, some graphics
don't work.
* Certain features of SageNB are missing: live documentation,
sharing/publishing of worksheets.
* It clutters the file system with .ipynb files.
This looks like you are on the correct track. I would just urge you to indicate what features of Sagenb are not available yet.
I think things are close to the point that the only significant thing will be interacts. If plotly is installed, I think good interactive 2D and 3D graphs will be available.
- "other language" cells : %maxima and %r (possibly other interpreters such as %octave) do not work as expected. Both %maxima and %r open new instances of their respective interpreters and enter an REPL that cannot be exited (even with an explicit "quit();" or "q('no')". In other words, they never return
On Sunday, January 3, 2016 at 2:41:51 PM UTC+1, Emmanuel Charpentier wrote:
- "other language" cells : %maxima and %r (possibly other interpreters such as %octave) do not work as expected. Both %maxima and %r open new instances of their respective interpreters and enter an REPL that cannot be exited (even with an explicit "quit();" or "q('no')". In other words, they never return
Interactive line magics obviously can't work in the browser like on the command line
The lack of appropriate cell magics (%%maxima) is just an existing bug on the commandline.
Of course if you care about it then send in a patch.
On the plus side, the jupyter notebook comes with an official R kernel available as Sage optional package: sage -r r_jupyter.
In case you are curious, in SageMathCloud worksheets a line mode is
%foo <stuff on the line>
and a cell mode is
%foo(optional, arguments)
<rest of the cell>
[ BTW : that's not really a bug, but rather a design conflict : the original %mode functions were designed to switch (for an indefinite scope) the behaviour of the REPL.This was transposed in the Sage notebook as what amounts to the equivalent of Jupyter's cell magics (scope defined as the current cell). Whereas line magics are, as far as I understand, Jupyter-specific... We can't be consistent across notebooks without redefining our "mode switch" magics as "cell magics", and rename them with "%%"... ]
On Sunday, January 3, 2016 at 5:36:30 PM UTC+1, Emmanuel Charpentier wrote:[ BTW : that's not really a bug, but rather a design conflict : the original %mode functions were designed to switch (for an indefinite scope) the behaviour of the REPL.This was transposed in the Sage notebook as what amounts to the equivalent of Jupyter's cell magics (scope defined as the current cell). Whereas line magics are, as far as I understand, Jupyter-specific... We can't be consistent across notebooks without redefining our "mode switch" magics as "cell magics", and rename them with "%%"... ]IMHO we should just switch the interface line magics to work like other line magics in Jupyter/IPython, and also SMC: Take the rest of the line as string and evaluate
Example for IPython default magic:sage: %system ls Makefile['Makefile']Example for how we currently handle line magics on the commandline (BAD: we just forget about the argument)sage: %gap 1+1;--> Switching to Gap <--gap: ^C--> Exiting back to Sage <--Expected behavoir of %gap 1+1; is to be likesage: gap('1+1;')2On the commandline ONLY line magics without arguments could still switch the interface
Note that the "coding problems" are not that small : ensuring that r(...), %r, %R and %%R "speak" to the same instance of R does not seem trivial
> As stated, that would break the compatibility with SMC magics that William
> just declared intangible (and always "cell magic").
Just to be clear: I don't care whether or not whatever design you
guys are coming up with in this thread is compatible with SMC Sage
worksheets. SMC provides both Sage worksheets and Jupyter notebooks
on an equal footing; each has its pros and cons, and will continue to
be fully supported (likely with a converter between them, that at
least converts what can be converted).
Thank you Jonathan for clarifying what I thought was obvious in what I was saying.> As stated, that would break the compatibility with SMC magics that William
> just declared intangible (and always "cell magic").
Just to be clear: I don't care whether or not whatever design you
guys are coming up with in this thread is compatible with SMC Sage
worksheets. SMC provides both Sage worksheets and Jupyter notebooks
on an equal footing; each has its pros and cons, and will continue to
be fully supported (likely with a converter between them, that at
least converts what can be converted).
I would be interested in why SMC provides Sage worksheets if Jupyter is so *clearly* the most superior option. [Rhetorical question]
And what of the long-term in Sage itself - would an eventual "SMC personal edition" become the default? [Not rhetorical but probably too far in the future to speculate]
--
You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
To post to this group, send email to sage-...@googlegroups.com.
Visit this group at https://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.
> As stated, that would break the compatibility with SMC magics that William
> just declared intangible (and always "cell magic").
Just to be clear: I don't care whether or not whatever design you
guys are coming up with in this thread is compatible with SMC Sage
worksheets. SMC provides both Sage worksheets and Jupyter notebooks
on an equal footing; each has its pros and cons, and will continue to
be fully supported (likely with a converter between them, that at
least converts what can be converted).
I would be interested in why SMC provides Sage worksheets if Jupyter is so *clearly* the most superior option. [Rhetorical question]Weird question, given that I literally just wrote "Sage worksheets and Jupyter notebooks on an equal footing; each has its pros and cons, and will continue to be fully supported". There are many ways in which Sage worksheets are superior to Jupyter notebooks, and some very nontrivial subtle new things I've been working on the last month that will lead to more...
And what of the long-term in Sage itself - would an eventual "SMC personal edition" become the default? [Not rhetorical but probably too far in the future to speculate]No clue. There's no legal or technical reason it couldn't happen... This year it likely won't as SMC is still changing way too much.
I'd also be more than happy to ship the personal SMC edition with sage when its ready; But sticking with the essentially unsupported SageNB for 1+years just to wait&see is not a sane plan. Even then, jupyter notebooks are a forward-safe choice so we have nothing to gain from waiting while people write new SageNB notebooks.As for the Jupyter wishlist, proper output capture would also be nice. Right now only the Python-internal stdout is captured, but for examplesage: cython(r'printf("test\n")')testyields no output in Jupyter.There are at least two different multi-user Jupyter versions that are of interest; the authenticated (via unix account, much better than SageNB) jupyterhub and the anonymous https://tmpnb.org (try it now if you haven't seen it)
If you start Jupyter in ~/foo then you cannot open notebooks in ~/.sage/jupyter. Thats just how it is.On Monday, December 21, 2015 at 1:12:57 AM UTC+1, William wrote:I clearly don't agree with that, since that's never been the case
before, as I mentioned. I'll not press this any further.
One example of a subtle feature in Sage (notebook and worksheets) not
in Jupyter, which I was just reminded of, is output limiting. In Sage
there are numerous rules/options to deal with people doing stuff like:
while True:
print "hi!"
... which is exactly what students will tend to do by accident...
Jupyter doesn't deal with this, but it might not be too hard to
implement in theory. One of the main problems is figuring out what
the arbitrary rate limiting defaults "should" be; it's arbitrary, and
depends a lot on whether everything is local, over the web, etc. so
getting a bunch of people to agree is hard, which might mean they will
never implement anything.
---------- Forwarded message ----------
From: Jonathan Frederic <jon.f...@gmail.com>
Date: Tue, Jan 5, 2016 at 11:42 AM
Subject: Re: [sage-devel] Re: Jupyter notebook by default?
To: Jason Grout <grout...@gmail.com>
Cc: sage-devel <sage-...@googlegroups.com>
Jason,Thanks for pulling me in on this.William,
I agree, getting a bunch of people to agree on stuff can seem impossible. However, you mention Sage offers a couple options to mitigate output overflows, can you point me to those options? The Jupyter Notebook should provide multiple options too - this will also make it easier for everyone to agree.Also, in you experience, which of these options work the best?
I was thinking initially of doing something simple, like hard limiting data/time, then printing an error if that's exceeded. In the Jupyter Notebook, we have to worry about- Too many messages sent on the websocket- The notebook json file growing too large and consequently becoming unopenable- Too much data being appended to the DOM, crashing the browserThanks!-Jon
On Tue, Jan 5, 2016 at 10:19 AM, Jason Grout <grout...@gmail.com> wrote:
(cross-posting to ipython-dev)Jon,At the recent San Francisco meetings, we talked about this. What do you think about:1. keeping track of the size of the io messages sent from any specific kernel execution2. When the total size of io reaches some specific size (user-configurable), transmitting a special "throwing away output, but here's how to save the output to a file if you want in the future, or how to increase the limit" message3. keep a running buffer of the last bit of output attempted to be sent, and send it when the execution finishes (so basically a ring buffer that overwrites the oldest message)This:* allows small output through* provides an explanatory message* provides the last bit of output as wellOne thing to figure out: a limit on size of output that is text may not be appropriate for output that is images, etc.
Jeroen.
I know this is now hijacking the thread... but if we are on those lines, when it comes to "too much output", making sure that this is something that can allow people to easily see and compare an "old" "too much output" to a "new" "too much output" after evaluating a cell/command a second/several time/s would be very helpful - we've had numerous requests along those lines. Doubtless this is already in the works though - nice to hear from you, Jason! :)
--
I'm not sure exactly what you mean here. Can you give an example?
IMHO output capture into a web browser isn't really different from the scrollback buffer of a terminal. We obviously enjoy the infinite scrollback but do not want an unbounded drawing surface in the terminal (= dom nodes in the web browser). And certainly nobody wants a piece of their output discarded in a long-running computation.The technical implementation is virtual scrolling, this is what the terminal does and this is how the browser should do it, too.
Jon mentioned that there are a few levels for large output to cause problems. The lowest bar is putting the output on the page, which is by far the easiest to hit, causing an unresponsive browser. This is the level that can be addressed by virtual scrolling / truncating output in UI. Fortunately, it’s also the easiest one to implement.
If we truncate instead of virtual-scroll, then we have a choice for whether truncated output is included in the document or not, which alleviates the problem of opening notebooks that have a problematic amount of output. But it’s putting that on the page that’s ~always the problem, not loading the notebook JSON itself, so I’m somewhat less concerned about that.
The next level where it can cause problems is the output coming over the network in the first place. We can throttle this in the notebook server, as was implemented for 4.2 months ago. Again, this moves the bar for when output causes trouble, but isn’t a complete solution. Dumping truncated output to a file is complicated a bit by the separations we have in place, but it should be doable.
-MinRK
If we truncate instead of virtual-scroll, then we have a choice for whether truncated output is included in the document or not, which alleviates the problem of opening notebooks that have a problematic amount of output
On Wednesday, January 6, 2016 at 11:55:36 AM UTC+1, Min RK wrote:If we truncate instead of virtual-scroll, then we have a choice for whether truncated output is included in the document or not, which alleviates the problem of opening notebooks that have a problematic amount of output
There is no fundamental problem with large amounts of output (really, any content), and there is essentially only a single way to do it right:The view (dom) needs only a fixed number of dom nodes for a virtual scroll.The in-browser view model can lazily load the current scroll position, with a suitable cache. Fixed amount of browser JS memory.The server can just mmap the output file, or alternatively seek around in the file. With a suitable index. Fixed amount of server-side memory.
The kernel has to block if the notebook server can't append output fast enough, thats normal flow control just like in a pipe. Fixed memory usage in the kernel.
--
You received this message because you are subscribed to a topic in the Google Groups "sage-devel" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/sage-devel/8erxWppKxXM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to sage-devel+...@googlegroups.com.
Files aren't used for output. The filesystem should only be involved, if at all, in the exceptional case of output overflow.
Thanks Jason for cross-posting.
Since the issue of funding was brought up, I think supporting projects like this is exactly the sort of thing we should be doing with the funding we have, whether the work sits on the Jupyter or Sage side (I assume there will be both). It’s a bit tricky to keep track of all the points in an email thread, but if we could aggregate the things that are blockers and the things that would be nice, especially changes you need from Jupyter, we should be able to start ticking boxes.
A summary of what I’ve seen so far:
Some comments:
Re: language cells, I assume it’s referring to things like %%bash
, %%R
, and %%cython
. While these look similar, there is a significant difference in how they are implemented. For instance, the R magic (provided by rpy2) runs an R interpreter in-memory, and talks to it, capturing output, etc.. Where many of these magics, such as bash, ruby, perl, come from is some “script magic” machinery in IPython, which populates the default magics with shortcuts to running a script in a given interpreter. They are essentially shortcuts to cat <cell> | <interpreter>
. It’s not a fundamental limitation, or anything dire like that. If sage has an implementation of running code in a persistent alternate interpreter, then it should not be much work to represent that in magics, since cell magics are any Python functions called with two string arguments (the rest of the line and the cell), and can be defined at any time, for instance:
def mymagic(line, cell):
do_stuff_with(cell)
get_ipython().register_magic_function(mymagic, 'cell')
Re: output capturing, Thomas Kluyver and I were at CERN last month working on the Cling kernel, and one of the things we did was C-level capturing of output. Now that we have that working, integrating it into the IPython kernel should not be much work, and if it’s really important, libraries can use the same technique themselves without waiting for IPython to catch up.
Interacts are perhaps the hardest piece. I think it should be doable to get sage’s own interacts working in the notebook, rather than forcing people to adopt the much more basic interact provided by the IPython widgets.
I can’t speak to the UI transition part of the problem whenever you change defaults, which is a big challenge, but I think we can at least mitigate most of the things on the Jupyter side that are getting in your way.
-MinRK
_______________________________________________
IPython-dev mailing list
IPyth...@scipy.org
https://mail.scipy.org/mailman/listinfo/ipython-dev
These all sound like awesome ideas to me for moving more *robustly* to a sws -> ipynb transition! Would attached files work properly (or at all)? I'm only asking out of ignorance.
--
You received this message because you are subscribed to a topic in the Google Groups "sage-devel" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/sage-devel/8erxWppKxXM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to sage-devel+...@googlegroups.com.
Interactive line magics obviously can't work in the browser like on the command lineThe lack of appropriate cell magics (%%maxima) is just an existing bug on the commandline.
Should the Jupyter notebook be the default notebook for the next Sage
7.0 release? I don't really have an opinion on the matter.
Pros:
* Nice tracebacks!
* The Jupyter notebook is a mature well-maintained project, unlike
SageNB. It is widely used in the "scientific Python" community.
* Availability of other Jupyter kernels besides Sage.
Cons:
* Less compatible with Sage: Sage interacts don't work, some graphics
don't work.
* Certain features of SageNB are missing: live documentation,
sharing/publishing of worksheets.
* It clutters the file system with .ipynb files.
Don't cares:
* It's just a default choice, both options remain available.
--
You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
What if we used something like ProseMirror for the markdown editor? http://prosemirror.net/. Would that help? Right now we use Codemirror.
For what it is worth, SageMathCloud has a buttons/lists, etc. for
editing markdown in Sage worksheets, and also a realtime preview
markdown editor for md files. Two screenshots attached. I also fully
implemented a realtime WYSIWYG editor for html and markdown a year
ago, but decided it wasn't up to my standards (getting realtime sync
to fully work well was surprisingly challenging), so I
disabled/removed it.
Oh yeah, I forgot that I get very confused trying to make text cells in SMC too :( presumably for the same reason that I don't want to type %md - but I don't use SMC much (yet).
--
You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
To post to this group, send email to sage-...@googlegroups.com.
Visit this group at https://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.
I surprisingly don't remember even one user request for such tinymce style wysiwyg editing for SMC. And our users are constantly asking for the functionality they feel they really need... I'm very surprised by this lack of demand. However maybe they don't know what they want.
--
A bit more robust in escaping:
--
Nice! That's even better that it works out-of-the-box!