Proposal: move SageNB back to Sage

[Jeroen Demeyer]

Hello all,

I propose to make SageNB no longer a separate package but to move it
back into the Sage git tree. For purposes of installation and use of
SageNB, it will still be a separate Python package, but the sources will
be in $SAGE_ROOT/src/sagenb instead of a separate git repo. The changes
to the Sage build system to support this move will be minimal.

The reason is that SageNB is truly in maintenance mode currently. Making
new SageNB releases regularly to fix things is a burden for the SageNB
release manager Karl-Dieter Crisman. On #14840 [1], he said "the sooner
sagenb gets back in Sage proper, the better!"

The original reason to split SageNB from Sage was to enable quick
development. That worked for a while, but now that development has
stalled, this reason no longer applies. A secondary reason was to make
SageNB truly independent from Sage, but that also never happened. So I
see no reason to keep SageNB split from Sage currently.

I know this is a controversial proposal, but it will certainly be easier
to maintain SageNB this way. I also want to stress that this is
orthogonal to any future deprecation or removal of SageNB: we can still
do that from the Sage git tree.

Jeroen.



[1] http://trac.sagemath.org/ticket/14840#comment:58

[Sébastien Labbé]

I propose to make SageNB no longer a separate package but to move it
back into the Sage git tree.

+1

I agree.

[Kwankyu Lee]

+1

[Dima Pasechnik]

+1

[mmarco]

+1

[Erik Bray]

-1

Any problems related to this are due to deeper problems with how Sage,
its dependencies, and its dependents is developed, and less to do with
any philosophical problem with them being separate.

I say focus on fixing the real problems, not the symptoms. And
besides, how much longer do you plan to want to develop sagenb rather
than keep it in maintenance mode?

[Dima Pasechnik]

It is already in (non)maintenance mode. The problem is in (non). By folding it back one would make sure that it still works.

Oh, and by the way, it's a good example of a failed attempt to spin Sage code off its main codebase.

[Erik Bray]

Yeah, but the question is why did it fail, and does it need to continue to fail?

[Dima Pasechnik]

it's legacy code (see my post https://groups.google.com/d/msg/sage-devel/HAHulLZkKzw/-A40pl2GAgAJ)

noone can really understand, yet it is needed to support a lot of legacy things.

Ideally it should have been spun off and become installable as a standalone python package, but this has not happened.

[Dima Pasechnik]

Who does have commit access to https://github.com/sagemath/sagenb ? 

(not me).

Karl is obviously overwhelmed with other things.

If I had this access I could have reviewed at least some of these tickets

(we still would want to keep upstream on github, right?)

Dima

On Friday, April 15, 2016 at 2:26:15 PM UTC+1, Jonathan Gutow wrote:

+1

Things really have stalled.  My fixes to the code to handle properly putting the notebook behind a proxy since the server mechanism in SageNB is not robust have languished for over a year.

Jonathan

                        Dr. Jonathan H. Gutow
Chemistry Department                                 gu...@uwosh.edu
UW-Oshkosh                                                Office:920-424-1326
800 Algoma Boulevard                                 FAX:920-424-2042
Oshkosh, WI 54901
                http://www.uwosh.edu/facstaff/gutow/

--
You received this message because you are subscribed to the Google Groups "sage-notebook" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-noteboo...@googlegroups.com.
Visit this group at https://groups.google.com/group/sage-notebook.
For more options, visit https://groups.google.com/d/optout.

[William Stein]

On Fri, Apr 15, 2016 at 7:05 AM, Dima Pasechnik <dim...@gmail.com> wrote:
> Who does have commit access to https://github.com/sagemath/sagenb ?
> (not me).
> Karl is obviously overwhelmed with other things.
> If I had this access I could have reviewed at least some of these tickets
> (we still would want to keep upstream on github, right?)

I've added you. There are 9 people who can commit (some long missing)
who can commit, including you, which I think you can see here:
https://github.com/orgs/sagemath/teams/sage-notebook

Anybody who wants to be added, say so and I (or maybe Dima) can add them.

William

--
William (http://wstein.org)

[William Stein]

-1 to moving the sagenb code back into the sage git repo. The day I
removed sagenb from the sage git repo was, for me, a very happy day.

But +1 for having people who care have the ability to push to the repo
on github and generally take over maintenance.

--
William (http://wstein.org)

[Jeroen Demeyer]

On 2016-04-15 16:15, William Stein wrote:
> But +1 for having people who care have the ability to push to the repo
> on github and generally take over maintenance.

Any volunteers? If there is indeed somebody who is seriously willing to
manage SageNB, that would be a good solution. However, by lack of such a
person, I think the best solution is to use the resources of Sage to
manage SageNB.

[Jeroen Demeyer]

On 2016-04-15 14:12, Erik Bray wrote:
> I say focus on fixing the real problems, not the symptoms.

I don't care. I want *any* fix (a real fix or a symptom-fix) and the
status-quo is not a fix.

[John H Palmieri]

+1.

For those who disagree, please recognize that the current situation is unmanageable: there are interdependencies between sage and sagenb, so certain changes in sage require changes in sagenb. Getting those changes done in sagenb is difficult, because sagenb is not really actively maintained. So if you disagree, please suggest a concrete alternative, because (as Jeroen says) the status quo is not working.

  John

[Vincent Delecroix]

+1 (both for Jeroen proposition and John intervention)

Indeed the -1 from William is just about a general philosophy that he
will be of no help implementing.

I agree with Erik remark but holding this move makes it just worse. And
doing it will not prevent us from analyzing why we failed keeping Sagenb
leaving outside of Sage.

[William Stein]

On Fri, Apr 15, 2016 at 9:21 AM, Vincent Delecroix

<20100.d...@gmail.com> wrote:
> +1 (both for Jeroen proposition and John intervention)
>
> Indeed the -1 from William is just about a general philosophy that he will
> be of no help implementing.

I would like to hear what Volker thinks. I wonder if his proposal
might even to be to consider making sagenb an optional package.

> --
> You received this message because you are subscribed to the Google Groups

> "sage-devel" group.

> To unsubscribe from this group and stop receiving emails from it, send an

> email to sage-devel+...@googlegroups.com.
> To post to this group, send email to sage-...@googlegroups.com.
> Visit this group at https://groups.google.com/group/sage-devel.

[Dima Pasechnik]

I can try to make it into a proper Python package, instead of going forward with #14840.

It seems that this is not such a big deal, it's a packaging problem, basically.

Give me a month or so...

 

[Dima Pasechnik]

On Friday, April 15, 2016 at 5:06:34 PM UTC+1, John H Palmieri wrote:

+1.

For those who disagree, please recognize that the current situation is unmanageable: there are interdependencies between sage and sagenb, so certain changes in sage require changes in sagenb. Getting those changes done in sagenb is difficult, because sagenb is not really actively maintained.

Could you perhaps point out at these?

Are there any issues/pull requests on sagenb related to such changes?

[Jeroen Demeyer]

On 2016-04-15 19:11, Dima Pasechnik wrote:
> I can try to make it into a proper Python package, instead of going
> forward with #14840.

Why do you think that #14840 does not "make it a proper Python package"???

[Dima Pasechnik]

#14840 introduces a dozen extra standard packages in Sage.

Packages that are only needed by sagenb, often old versions of

otherwise maintained Python packages.

#14840 does make sagenb a proper Sage package, but not a proper Python package.

[Jeroen Demeyer]

On 2016-04-15 21:31, Dima Pasechnik wrote:
> not a proper Python package.

Why not? You haven't answered this question.

[Dima Pasechnik]

Does sagenb run without Sage now?

 

[Jeroen Demeyer]

On 2016-04-15 21:53, Dima Pasechnik wrote:
> Why not? You haven't answered this question.
>
> Does sagenb run without Sage now?

No, it needs Sage.

Python packages can have dependencies, so the fact that it needs Sage
does not imply that "it's not a proper Python package".

[Francois Bissey]

> On 16/04/2016, at 07:56, Jeroen Demeyer <jdem...@cage.ugent.be> wrote:
>
> Python packages can have dependencies, so the fact that it needs Sage does not imply that "it's not a proper Python package".

+1

sageNB has effectively been an optional package in sage-on-gentoo until I discovered
the problem with sphinxify. Most of the stuff in the sageNB tarball would be optional too.

François

[John H Palmieri]

On Friday, April 15, 2016 at 10:40:25 AM UTC-7, Dima Pasechnik wrote:

On Friday, April 15, 2016 at 5:06:34 PM UTC+1, John H Palmieri wrote:

+1.

For those who disagree, please recognize that the current situation is unmanageable: there are interdependencies between sage and sagenb, so certain changes in sage require changes in sagenb. Getting those changes done in sagenb is difficult, because sagenb is not really actively maintained.

Could you perhaps point out at these?

Are there any issues/pull requests on sagenb related to such changes?

I don't know if there are any right now, but I think that many of the recent changes in sagenb have come because there were changes in the Sage library. Maybe "difficult" is not the right word, but this adds an artificial layer of complexity: instead of making a few possibly trivial changes in the relevant sagenb files within Sage, for example when the location of the built documentation moved, it takes a pull request, someone to handle that request, someone to put together a new sagenb release, etc. It often (almost always?) ends up being the same people working on the Sage trac ticket and the sagenb pull request, so we're adding some inefficiencies to the system. If we could deal with the issues with just a trac ticket, that would be better.
 

[Jeroen Demeyer]

On 2016-04-15 22:06, John H Palmieri wrote:
> I don't know if there are any right now, but I think that many of the
> recent changes in sagenb have come because there were changes in the
> Sage library. Maybe "difficult" is not the right word, but this adds an
> artificial layer of complexity: instead of making a few possibly trivial
> changes in the relevant sagenb files within Sage, for example when the
> location of the built documentation moved, it takes a pull request,
> someone to handle that request, someone to put together a new sagenb
> release, etc. It often (almost always?) ends up being the same people
> working on the Sage trac ticket and the sagenb pull request, so we're
> adding some inefficiencies to the system. If we could deal with the
> issues with just a trac ticket, that would be better.

+1, this is an excellent summary of the problem.

[Dima Pasechnik]

On Friday, April 15, 2016 at 8:56:54 PM UTC+1, Jeroen Demeyer wrote:

On 2016-04-15 21:53, Dima Pasechnik wrote:
>     Why not? You haven't answered this question.
>
> Does sagenb run without Sage now?
No, it needs Sage

Python packages can have dependencies, so the fact that it needs Sage
does not imply that "it's not a proper Python package".

it does not quite work outside Sage tree. IMHO merely having a setup.py does not make a piece of Sage code a Python package... Well, this looks like an academic discussion to me :)

[Volker Braun]

Is there really going to be much activity on SageNB in the future? I appreciate that you fixed the packaging and dependency nightmare, but it seems that we are now (i.e. after #14840) at the point where we are likely to just wait and eventually remove SageNB. I always found it frustrating to do any kind of change on SageNB; there isn't really a good testsuite so you basically have to check by hand that you don't break stuff. And none of the core developers use SageNB so we don't really spot breakage that well either. More than anything else, we can maintain Sage easily because we have a pretty awesome testsuite.

Having to do two separate commits, one to Sage and one to SageNB, isn't really that much of a deal in comparison. Of course then waiting indefinitely for a new SageNB release *IS* a big problem. So as a minimal proposal, how about we just take over the SageNB github repo and make releases whenever we need them. Which really isn't all that often.

[William Stein]

+1 -- I agree 100% with everything above. If anybody wants added to
the repo, let me know.

William

--
William (http://wstein.org)

[Dima Pasechnik]

On Friday, April 15, 2016 at 10:59:52 PM UTC+1, Volker Braun wrote:

Is there really going to be much activity on SageNB in the future? I appreciate that you fixed the packaging and dependency nightmare, but it seems that we are now (i.e. after #14840) at the point where we are likely to just wait and eventually remove SageNB. I always found it frustrating to do any kind of change on SageNB; there isn't really a good testsuite so you basically have to check by hand that you don't break stuff. And none of the core developers use SageNB so we don't really spot breakage that well either. More than anything else, we can maintain Sage easily because we have a pretty awesome testsuite.

Having to do two separate commits, one to Sage and one to SageNB, isn't really that much of a deal in comparison. Of course then waiting indefinitely for a new SageNB release *IS* a big problem. So as a minimal proposal, how about we just take over the SageNB github repo and make releases whenever we need them. Which really isn't all that often.

Sounds quite meaningful to me. 

What I dislike about #14840 is that it creates 14 new standard packages (not needed for anything except sagenb), instead of just running 'sage --pip'. We already install stuff by 'sage --pip', e.g. the ssl support needed for sagenb and jupyter; and because we cannot make this support into packages (because we must, just must distribute tarballs...) it creates all sorts of errors and confusion. Becoming less prudish about source tarballs would only benefit users and simplify source code...

[Volker Braun]

On Saturday, April 16, 2016 at 12:32:44 AM UTC+2, Dima Pasechnik wrote:

What I dislike about #14840 is that it creates 14 new standard packages (not needed for anything except sagenb), instead of just running 'sage --pip'. We already install stuff by 'sage --pip', e.g. the ssl support needed for sagenb and jupyter; and because we cannot make this support into packages (because we must, just must distribute tarballs...) it creates all sorts of errors and confusion. Becoming less prudish about source tarballs would only benefit users and simplify source code...

As I said on the ticket, I'm happy to change our policy for what goes into a "source tarball" but we should only do that at a major version.

Also if OpenSSL would come around with their announced license change then that would be great (pip depends on openssl).

[kcrisman]

> indefinitely for a new SageNB release *IS* a big problem. So as a minimal
> proposal, how about we just take over the SageNB github repo and make
> releases whenever we need them. Which really isn't all that often.

+1  -- I agree 100% with everything above.  If anybody wants added to
the repo, let me know.

Thanks to all for the action on this.  I only took over the sagenb repo management reluctantly, because no one else who still cared was willing to do it.  I do think there is still some valuable work on it - particularly by J. Miguel Farto (see https://github.com/migeruhito/sagenb/commits/mynb still under active development Mar 9) and Jonathan Gutow on proxy issues.  But those people perhaps were not as close to the "core" project and so I took responsibility for sagenb itself.  But I am happy for anyone else who wants to sign off their name on merges to take responsibility (as well as for breakages!), as I have been completely overwhelmed with work and realistically cannot do more than I've been doing with micro-releases of pull requests, which I do try to test rather diligently.

[Jeroen Demeyer]

On 2016-04-15 23:59, Volker Braun wrote:
> So as a
> minimal proposal, how about we just take over the SageNB github repo and
> make releases whenever we need them. Which really isn't all that often.

Isn't that exactly the current situation? As long as Sage and SageNB are
in a separate git repo, we still need somebody to accept pull requests
and make releases.

[Jeroen Demeyer]

On 2016-04-15 22:46, Dima Pasechnik wrote:
> it does not quite work outside Sage tree.

Of course not. SageNB depends on Sage.

[Volker Braun]

On Saturday, April 16, 2016 at 5:57:32 AM UTC+2, Jeroen Demeyer wrote:

Isn't that exactly the current situation? As long as Sage and SageNB are
in a separate git repo, we still need somebody to accept pull requests
and make releases.

Yes, but I'm proposing that anybody on the SageNB github team (including you) can make the release whenever a Sage ticket requires it. 

[Jason Grout]

For what it's worth, +1 from me as well to Volker's proposal. That makes a lot of sense.

Jason

[kcrisman]

Isn't that exactly the current situation? As long as Sage and SageNB are
in a separate git repo, we still need somebody to accept pull requests
and make releases.

Yes, but I'm proposing that anybody on the SageNB github team (including you) can make the release whenever a Sage ticket requires it. 

Yes, as long as their name is on such commits this seems very reasonable; it's exactly the situation that obtained before sagenb was separated.

Postscript not directly related, but still relevant for the future:

Last night I was thinking about another issue - the eventual proposed complete removal of sagenb from Sage (which presumably would have to be quite some time, at least until we know the exporter works pretty comprehensively (on my docket to check after May 15, though permit me to be skeptical after all the work that proved even sws2rst and sws2tex were quite fallible until a lot of extra work was done)).  And what I was thinking about was https://groups.google.com/forum/#!msg/sage-devel/8erxWppKxXM/lCtFZY5KBgAJ - namely that the TinyMCE editor was one of the best ideas ever for sagenb, warts and all, and that so far neither SMC nor Jupyter notebooks seem to have a real replacement for this. (I don't know if the exporter deals with such content, it very well may.)

I've done an awful lot of professional development for college educators over the past six-seven years, not all Sage-related.  And one thing I know is that while there will always be bleeding-edge enthusiasts, getting rank and file to use something that is made by developers, for developers, is a very hard task.  Easier just to keep that Mma or Maple site license, our physics people have it anyway... or easier just to use Word (yes) to make exams and worksheets.  Ok, more realistically people under 50 are probably using LaTeX :) but you get my point.

That sort of person, who has limited time even if they are excited about innovation (and this would be even more true at the secondary level) will most likely not want to go to the trouble to learn markdown.  I have already witnessed enough workshops incorporating this and seen how people with PhDs (including myself) struggle - not because it's not inherently simpler than (say) TeX, but because it's different and unfamiliar, and you've only got an hour to learn a heck of a lot more than md.  Not to mention it's far less flexible than WYSIWYG editors like TinyMCE.

So I just want to reiterate that until such time as Jupyter has something like this (not necessarily TinyMCE, there could be many such solutions), one can't really consider it a replacement.  It would just be better in some ways, worse in others.  And that would mean that completely removing sagenb would be a bit overenthusiastic.

(In the far future where no one learns TeX and WYSIWYG doesn't exist, or when everyone has a github account and uses md all the time, this argument is void - but I'm not holding my breath.)

Okay, postscript done :)

[Jeroen Demeyer]

On 2016-04-17 03:27, kcrisman wrote:
> Last night I was thinking about another issue - the eventual proposed
> complete removal of sagenb from Sage

Just a small addition: even if we switch to Jupyter by default, that
doesn't mean that we will drop SageNB immediately. We should still ship
it as non-default notebook for a few years.

[Erik Bray]

On Fri, Apr 15, 2016 at 4:15 PM, William Stein <wst...@gmail.com> wrote:
> -1 to moving the sagenb code back into the sage git repo. The day I
> removed sagenb from the sage git repo was, for me, a very happy day.

>
> But +1 for having people who care have the ability to push to the repo
> on github and generally take over maintenance.

Indeed. If the problem is a lack of maintenance support I don't see
why being in a separate repo somehow makes that harder.

On the other hand, if there are API issues between sage and sagenb
then that needs to be addressed. They don't need to be in the same
repos to address that though (otherwise it's not an API).

Erik

> On Fri, Apr 15, 2016 at 7:13 AM, William Stein <wst...@gmail.com> wrote:
>> On Fri, Apr 15, 2016 at 7:05 AM, Dima Pasechnik <dim...@gmail.com> wrote:
>>> Who does have commit access to https://github.com/sagemath/sagenb ?
>>> (not me).
>>> Karl is obviously overwhelmed with other things.
>>> If I had this access I could have reviewed at least some of these tickets
>>> (we still would want to keep upstream on github, right?)
>>
>> I've added you. There are 9 people who can commit (some long missing)
>> who can commit, including you, which I think you can see here:
>> https://github.com/orgs/sagemath/teams/sage-notebook
>>
>> Anybody who wants to be added, say so and I (or maybe Dima) can add them.
>>
>> William
>>
>>>
>>> Dima
>>>
>>> On Friday, April 15, 2016 at 2:26:15 PM UTC+1, Jonathan Gutow wrote:
>>>>
>>>> +1
>>>>
>>>> Things really have stalled. My fixes to the code to handle properly
>>>> putting the notebook behind a proxy since the server mechanism in SageNB is
>>>> not robust have languished for over a year.
>>>>
>>>> Jonathan
>>>> Dr. Jonathan H. Gutow
>>>> Chemistry Department gu...@uwosh.edu
>>>> UW-Oshkosh
>>>> Office:920-424-1326
>>>> 800 Algoma Boulevard FAX:920-424-2042
>>>> Oshkosh, WI 54901
>>>> http://www.uwosh.edu/facstaff/gutow/
>>>>
>>>> On Apr 15, 2016, at 4:38 AM, mmarco <mma...@unizar.es> wrote:
>>>>
>>>> +1
>>>>
>>>> El viernes, 15 de abril de 2016, 10:44:22 (UTC+2), Jeroen Demeyer
>>>> escribió:

>>>>>
>>>>> Hello all,
>>>>>
>>>>> I propose to make SageNB no longer a separate package but to move it
>>>>> back into the Sage git tree. For purposes of installation and use of
>>>>> SageNB, it will still be a separate Python package, but the sources will
>>>>> be in $SAGE_ROOT/src/sagenb instead of a separate git repo. The changes
>>>>> to the Sage build system to support this move will be minimal.
>>>>>
>>>>> The reason is that SageNB is truly in maintenance mode currently. Making
>>>>> new SageNB releases regularly to fix things is a burden for the SageNB
>>>>> release manager Karl-Dieter Crisman. On #14840 [1], he said "the sooner
>>>>> sagenb gets back in Sage proper, the better!"
>>>>>
>>>>> The original reason to split SageNB from Sage was to enable quick
>>>>> development. That worked for a while, but now that development has
>>>>> stalled, this reason no longer applies. A secondary reason was to make
>>>>> SageNB truly independent from Sage, but that also never happened. So I
>>>>> see no reason to keep SageNB split from Sage currently.
>>>>>
>>>>> I know this is a controversial proposal, but it will certainly be easier
>>>>> to maintain SageNB this way. I also want to stress that this is
>>>>> orthogonal to any future deprecation or removal of SageNB: we can still
>>>>> do that from the Sage git tree.
>>>>>
>>>>> Jeroen.
>>>>>
>>>>>
>>>>>
>>>>> [1] http://trac.sagemath.org/ticket/14840#comment:58
>>>>
>>>>

>>>> --
>>>> You received this message because you are subscribed to the Google Groups

>>>> "sage-notebook" group.

>>>> To unsubscribe from this group and stop receiving emails from it, send an

>>>> email to sage-noteboo...@googlegroups.com.
>>>> Visit this group at https://groups.google.com/group/sage-notebook.

>>>> For more options, visit https://groups.google.com/d/optout.
>>>>
>>>>
>>> --

>>> You received this message because you are subscribed to the Google Groups

>>> "sage-notebook" group.

>>> To unsubscribe from this group and stop receiving emails from it, send an

>>> email to sage-noteboo...@googlegroups.com.
>>> Visit this group at https://groups.google.com/group/sage-notebook.

>>> For more options, visit https://groups.google.com/d/optout.
>>
>>
>>
>> --
>> William (http://wstein.org)
>
>
>

> --
> William (http://wstein.org)

[Erik Bray]

Okay,but that's totally not unusual in software development. I work
in about a dozen different repositories on a given day and it doesn't
make a difference.

Clearly we just need more help on maintaining sagenb. I need to check
with Nicolas if this is within the purview of ODK (I'm not sure that
it is given that part of ODK seems to be aimed toward moving away from
sagenb). But it's clear that sagenb still needs to be maintained.

Shoving it back into sage is a backwards move though, and in the long
term only makes the task of packaging and distributing sage more
difficult. To me it feels like self-sabotage.

Best,
Erik

[Jack Dyson]

On Friday, April 15, 2016 at 10:44:21 AM UTC+2, Jeroen Demeyer wrote:
> Hello all,
>
> I propose to make SageNB no longer a separate package but to move it
> back into the Sage git tree. For purposes of installation and use of
> SageNB, it will still be a separate Python package, but the sources will
> be in $SAGE_ROOT/src/sagenb instead of a separate git repo. The changes
> to the Sage build system to support this move will be minimal.
>
> The reason is that SageNB is truly in maintenance mode currently. Making
> new SageNB releases regularly to fix things is a burden for the SageNB
> release manager Karl-Dieter Crisman. On #14840 [1], he said "the sooner
> sagenb gets back in Sage proper, the better!"
>
> The original reason to split SageNB from Sage was to enable quick
> development. That worked for a while, but now that development has
> stalled, this reason no longer applies. A secondary reason was to make
> SageNB truly independent from Sage, but that also never happened. So I
> see no reason to keep SageNB split from Sage currently.
>
> I know this is a controversial proposal, but it will certainly be easier
> to maintain SageNB this way. I also want to stress that this is
> orthogonal to any future deprecation or removal of SageNB: we can still
> do that from the Sage git tree.
>
> Jeroen.
>
>
>
> [1] http://trac.sagemath.org/ticket/14840#comment:58

Hi everyone,

With the greatest respect, I disagree strongly - chopping and changing the notebook this way leads to a lot of instability in the code and confusion for anyone who wants to get into developing on sagenb projects.

Added to the fact that none of us would have time to document changes in detail causes new contributions stagnate, which wastes effort and randomizes progress.

Actually the functionality of the current notebook is good, the look and ui is very dated and as many are aware, a bit on the unpolished side.

Remembering Samuel Ainsworth's really good work a few year's back, I would like to ask why was that system not developed and integrated into the local sagemath distributable?

From the trials he conducted in 2012 it ran well on Sage 5.3 and was in my opinion a decent step forward. I'll post this to a separate question as I wanted to explore the possibilities of getting that up and running again, even if only for private use here.

It doesn't seem to connect to sage 7.3 so I wanted to see if anyone knew why ?

Very best to all
Jack

[Dima Pasechnik]

I understand that opinions on usability of https://github.com/sagemath/sagenb/tree/newui

diverge.  (and with the breakneck speed javascript

frameworks are developed, one may ask whether something written in 2012 is still a great idea)

You ask why sagenb is not developed further.

1) the sagenb's design is really dated, and jupyter notebook seems a better (and much better

supported) alternative. 

2) Another actively developed alternative is SMCs notebook, which can be run

locally.

[Jan Groenewald]

Hi

On 23 November 2016 at 22:05, Dima Pasechnik <dim...@gmail.com> wrote:

I understand that opinions on usability of https://github.com/sagemath/sagenb/tree/newui

diverge.  (and with the breakneck speed javascript

frameworks are developed, one may ask whether something written in 2012 is still a great idea)

You ask why sagenb is not developed further.

1) the sagenb's design is really dated, and jupyter notebook seems a better (and much better

supported) alternative. 

2) Another actively developed alternative is SMCs notebook, which can be run

locally. 

In Africa we have connectivity and power issues, which requires a local notebook.

Would jupyter be a possible replacement for sagenb if sagenb is discontinued from active development?  Is it already on a stand-alone sagemath install?

Regards,

Jan

--

  .~.
  /V\     Jan Groenewald
 /( )\    www.aims.ac.za
 ^^-^^ 

[Dima Pasechnik]

On Wednesday, November 23, 2016 at 8:11:27 PM UTC, Jan Groenewald wrote:

Hi

On 23 November 2016 at 22:05, Dima Pasechnik <dim...@gmail.com> wrote:

I understand that opinions on usability of https://github.com/sagemath/sagenb/tree/newui

diverge.  (and with the breakneck speed javascript

frameworks are developed, one may ask whether something written in 2012 is still a great idea)

You ask why sagenb is not developed further.

1) the sagenb's design is really dated, and jupyter notebook seems a better (and much better

supported) alternative. 

2) Another actively developed alternative is SMCs notebook, which can be run

locally. 

In Africa we have connectivity and power issues, which requires a local notebook.

Would jupyter be a possible replacement for sagenb if sagenb is discontinued from active development?

yes, it is already there:

Start sage as follows:

sage --notebook='jupyter'

 

  Is it already on a stand-alone sagemath install?

yes. For instance, even a docker container.

[William Stein]

On Wed, Nov 23, 2016 at 12:11 PM, Jan Groenewald <j...@aims.ac.za> wrote:
>
> Hi
>
> On 23 November 2016 at 22:05, Dima Pasechnik <dim...@gmail.com> wrote:
>>
>>
>> I understand that opinions on usability of https://github.com/sagemath/sagenb/tree/newui
>> diverge. (and with the breakneck speed javascript
>> frameworks are developed, one may ask whether something written in 2012 is still a great idea)
>>
>> You ask why sagenb is not developed further.
>> 1) the sagenb's design is really dated, and jupyter notebook seems a better (and much better
>> supported) alternative.
>> 2) Another actively developed alternative is SMCs notebook, which can be run
>> locally.
>
>
> In Africa we have connectivity and power issues, which requires a local notebook.

SMC **can be run locally**:

1. Install docker.

2. Type this (all one line)

docker run --name=smc -v ~/smc:/projects -p 80:80 -p 443:443
sagemathinc/sagemathcloud

Now you're running SMC locally (visit http://localhost). See
https://github.com/sagemathinc/smc/blob/master/src/dev/docker/README.md



--
William (http://wstein.org)

[Jan Groenewald]

Hi

We don't want to install docker. We want to install a debian package.

Regards,

Jan

[William Stein]

OK, then it's not possible to use SMC, since we do not provide it as a
Debian package. Sorry,

-- William

[Nicolas M. Thiery]

On Wed, Nov 23, 2016 at 10:11:02PM +0200, Jan Groenewald wrote:
> Would jupyter be a possible replacement for sagenb if sagenb is
> discontinued from active development? Is it already on a stand-alone
> sagemath install?

I have been using Jupyter locally for my teaching since two years, and
am very happy with it. The only missing features have been migration
tool from sagenb and interacts; both are on their way.

Kudos to all who worked toward migrating to Jupyter so that eventually
the Sage community won't have to maintain sagenb anymore and be able
to focus on the core of Sage.

Cheers,
Nicolas
--
Nicolas M. Thiéry "Isil" <nth...@users.sf.net>
http://Nicolas.Thiery.name/

[Dima Pasechnik]

On Wednesday, November 23, 2016 at 9:29:44 PM UTC, Jan Groenewald wrote:

Hi

On 23 November 2016 at 22:59, William Stein <wst...@gmail.com> wrote:

On Wed, Nov 23, 2016 at 12:11 PM, Jan Groenewald <j...@aims.ac.za> wrote:
>
> Hi
>
> On 23 November 2016 at 22:05, Dima Pasechnik <dim...@gmail.com> wrote:
>>
>>
>> I understand that opinions on usability of https://github.com/sagemath/sagenb/tree/newui
>> diverge.  (and with the breakneck speed javascript
>> frameworks are developed, one may ask whether something written in 2012 is still a great idea)
>>
>> You ask why sagenb is not developed further.
>> 1) the sagenb's design is really dated, and jupyter notebook seems a better (and much better
>> supported) alternative.
>> 2) Another actively developed alternative is SMCs notebook, which can be run
>> locally.
>
>
> In Africa we have connectivity and power issues, which requires a local notebook.


SMC **can be run locally**:

1. Install docker.

2. Type this (all one line)

    docker run --name=smc -v ~/smc:/projects -p 80:80 -p 443:443
sagemathinc/sagemathcloud

Now you're running SMC locally (visit http://localhost).  See
   https://github.com/sagemathinc/smc/blob/master/src/dev/docker/README.md

We don't want to install docker. We want to install a debian package.

but sure, do it!

https://packages.debian.org/sid/docker.io

[Jack Dyson]

Hi Dima,

Thankyou for your quick reply - I appreciate what you said about development of sagenb as a whole, and actually I do see the point.

Logically therefore, as you indicate, iPython is a good alternative. Unfortunately, it is not fully compatible with sagemath, for example R doesn't access it properly in all respects or the 3D graphics formatting was of last time I checked.

The current sagenb is excellent on both those things even if the structure is dated: I want to just make clear it does work extremely well and that's what a user spends 80% of their time doing.

It is great that SMC's notebook is actually available in some way: and here I feel that a decision needs to be made:

therefore one of these should answer well:

1)  "phase out" sagenb completely and integrate SMC into the sage distributable quickly so that it is the default option - as was hinted at a few years ago. It was in fact stated then that developing sagenb was a "waste of developer resources"

2) dump jmol and make iPython the default notebook and address its remaining incompatibilities with packages (which would need a rewrite of the sagemath API so it is no longer view model dependent I suppose)

3) create a new independent team around sagenb : decide what we need and continue Sam's work in whatever framework we want, the functional model of the code made independent from the implementation

The corollary is clear: not all sagemath users want to specialize to SMC for various reasons, and the sagemath userbase (universities for example) as a distributable is going to be under threat without a viable modern local interface just like Maxima was years back before wxmaxima.

I think William wrote something below about installing a docker and getting SMC up, very welcome indeed : I'll give that a go for starters.

Best from here,

Jack

[William Stein]

For a more local interface, also consider the possibility of building
on nteract:

https://github.com/nteract/nteract

It's basically an Electron app rewrite from scratch of Jupyter.

[Jack Dyson]

Right cheers for that William! will give it a go, hopefully we'll have it as standard at some point soon.

Best from here,

J

[William Stein]

On Wed, Nov 23, 2016 at 3:02 PM, Jack Dyson <jackd...@gmail.com> wrote:
> Right cheers for that William! will give it a go, hopefully we'll have it as
> standard at some point soon.

If you do want to work on that, some thoughts:

- There is a "painful" dependency on RethinkDB, which is a C++
program that takes a while to compile. In a few months I predict this
will be replaced by either sqlite or postgreSQL, which are both much
less painful dependencies...

Actually, I don't have much in the way of other thoughts, but don't
hesitate to ask.

[Jack Dyson]

Thanks I will! enough there to think about already !
J
PS nteract looks cool

> You received this message because you are subscribed to a topic in the Google Groups "sage-notebook" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/sage-notebook/t11JSxxCgpw/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to sage-noteboo...@googlegroups.com.

[Eric Gourgoulhon]

Hi Nicolas,

Le mercredi 23 novembre 2016 22:46:34 UTC+1, Nicolas M. Thiéry a écrit :

I have been using Jupyter locally for my teaching since two years, and
am very happy with it. The only missing features have been migration
tool from sagenb and interacts; both are on their way.

Actually, since version 7.3, Sage is shipped with a migration tool sagenb --> Jupyter, developed by Volker (cf.  #19877).
It suffices to run
sage -n export --list
to get the list of sagenb files; then

sage -n export --ipynb=output.ipynb admin:10

converts the sagenb listed as admin:10 to the Jupyter notebook output.ipynb

I've used it to migrate all the worksheets at
http://sagemanifolds.obspm.fr/examples.html
and it works very well !

Best regards,

Eric.

[Jeroen Demeyer]

On 2016-11-23 22:46, Nicolas M. Thiery wrote:
> I have been using Jupyter locally for my teaching since two years, and
> am very happy with it. The only missing features have been migration
> tool from sagenb and interacts; both are on their way.

For interacts, see https://trac.sagemath.org/ticket/21267 which is 95%
done, but it does require a non-released version of ipywidgets.

Jeroen.

[Nicolas M. Thiery]

On Wed, Nov 23, 2016 at 03:54:18PM -0800, Eric Gourgoulhon wrote:
> Actually, since version 7.3, Sage is shipped with a migration tool
> sagenb --> Jupyter, developed by Volker (cf. #19877).

Indeed! Thanks you (and Jeroen as well) for making my mentions more explicit.

For a smooth migration path, we probably still need a few extra
goodies, like having a "graphical" wizard to run the converter upon
starting the Jupyter notebook, and also the ability to import a
separate sws file in the Jupyter notebook.

Jack: if you encounter further issues with Sage in the Jupyter
notebook, please create trac tickets, and post a link on [1] (or do we
have an overview trac ticket about the migration to Jupyter?)

Cheers,
Nicolas

[1] https://github.com/OpenDreamKit/OpenDreamKit/issues/94

[Jack Dyson]

Hi William,
installed the docker version - exciting stuff - just brilliant!
thanks. We need this "sagewide" asap really. Be looking into it as I
understand more.
small question when you have a moment:
docker starts correctly, everything works-I have SMC locally but I
can't get in sage dedicated modes like R or shell. Some of them work
like maxima, python, gap. If I create a terminal in my project, I can
start R. The code is there - just not accessible from a sage file in
the project: any idea ?
best,
J

PS : an example:

%r
1+1

Error in lines 1-1
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/smc_sagews/sage_server.py",
line 968, in execute
exec compile(block+'\n', '', 'single') in namespace, locals
File "", line 1, in <module>
File "/usr/local/lib/python2.7/dist-packages/smc_sagews/sage_server.py",
line 1009, in execute_with_code_decorators
code = code_decorator(code)
File "/usr/local/lib/python2.7/dist-packages/smc_sagews/sage_salvus.py",
line 2117, in r
r.jupyter_kernel = jupyter("ir")
File "/usr/local/lib/python2.7/dist-packages/smc_sagews/sage_jupyter.py",
line 31, in __call__
return _jkmagic(kernel_name, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/smc_sagews/sage_jupyter.py",
line 111, in _jkmagic
km, kc = jupyter_client.manager.start_new_kernel(kernel_name = kernel_name)
File "/usr/lib/sagemath/local/lib/python2.7/site-packages/jupyter_client/manager.py",
line 429, in start_new_kernel
km.start_kernel(**kwargs)
File "/usr/lib/sagemath/local/lib/python2.7/site-packages/jupyter_client/manager.py",
line 230, in start_kernel
kernel_cmd = self.format_kernel_cmd(extra_arguments=extra_arguments)
File "/usr/lib/sagemath/local/lib/python2.7/site-packages/jupyter_client/manager.py",
line 170, in format_kernel_cmd
cmd = self.kernel_spec.argv + extra_arguments
File "/usr/lib/sagemath/local/lib/python2.7/site-packages/jupyter_client/manager.py",
line 82, in kernel_spec
self._kernel_spec =
self.kernel_spec_manager.get_kernel_spec(self.kernel_name)
File "/usr/lib/sagemath/local/lib/python2.7/site-packages/jupyter_client/kernelspec.py",
line 175, in get_kernel_spec
raise NoSuchKernel(kernel_name)
NoSuchKernel: No such kernel named ir

[Harald Schilly]

On Thu, Nov 24, 2016 at 10:38 AM, Jack Dyson <jackd...@gmail.com> wrote:

any idea ?

​The error is: NoSuchKernel: No such kernel named ir

​

That means, that the "ir" kernel for R (and many others) aren't available in that image. I mean, you're running a local version of smc with some minimal subset of the necessary stuff on the back-end -- that's it. In total there is much much more, but that's outside the scope of smc. ​

Here is a most likely working copy of that kernel:

https://github.com/sagemathinc/smc/tree/master/src/smc-build/jupyter-kernels/ir

-- h

[Emmanuel Charpentier]

A few random thoughts :

1) the Jupyter notebook is an interface used by a *lot* of projects. Jupyter kernels do exist for a large number of computing tools. It is therefore well-developed and actively maintained.

2) In contrast, the Sage notebook, while quite advanced *for its time*, has remained a Sage-only interface. Yes, you can use a number of other tools with i, *as long as they are known by Sage*.
This simultaneously enhances and limits its utility. For example, one can use the Sage notebook for Maxima (and even use Maxima, R or gap cells in a notebook mainly containing Sage code). But only with Sege-suported version of Maxima, R or gap. Don't even think of having Lisp or (heavens !) Fortran cells...
And, yes, this need exists : a "Fortran engine" has been added a couple years ago to an R tool offering services quite related to SageTeX (writing \LaTeX or mrakdown text interspersed with code). For some needs, Fortran has not yet been replaced...

3) The same is true, with both aggravation and mitigation, for SMC and its related tools. I like the idea of a \LaTeX editor that supports sage (or other) snippets. But, again, *only* with the tools integrated in Sage...
In contrast, emacs+SageTeX(+R+knitr) give me a better service...

4) Jupyter kernels *do* look at what is done in Sage, and some of the Sage notebook advantages are ported to other kernels. I wouldn't be *too* surprised to  see, for example, a %%Maxima cell magic appear in the IRkernel that provides R notebooks.

5) The use of standard tools allow to alleviate some of the current inadequacies of Sage or its notebook :
For example, whereas *Sage* interacts are not yet ported to Sage's Jupyter notebook, one can use Python's widgets to get interactive graphics in Sage Jupyter notebooks.
Similarly, while the current Jupyter notebook won't display the graphs produced by R, this can be worked around by displaying images exported from R using the general-purpose IPython tools.

So, we have a choice between   :

• a general-purpose tool that is maintained by a large community, which can be customized to our needs, but needs (a non-trivial amount of) manpower for this customization,
• an ancient tool, well-suited to our past needs, but difficult to maintain and single-purpose,
• a newer tool, extremely well-suited to our present perceived needs, but in practice unmaintainable outside SageMath Inc., as far as I can see... and also single-purpose.
  

I'm afraid that history shows that this isn't even a choice : the multi-purpose nature of Jupyter gives it a (probably unbeatable) competitive advantage over its competitors. The SMC, however, will probably succeed for its initial goal (serving a large group of users using exclusively Sage and its federated tools via the Web), for which it seems probably unbeatable ; but for people needing a local installation, or needing an interface to tools not (yet) integrated with Sage, Jupyter is simply better.

So I think that the best use we can make of our limited manpower is to concentrate on the Jupyter notebook and, to some extend, the SMC. Once all the abilities of the current SageNB can realistically be said to be ported to the Jupyter notebook, we should deprecate the SageNB as soon as it is possible.

BTW, since the Jupyter notebook has been realistically usable with Sage, I spend most of my (quite limited) Sage time in the Jupyter notebook, most of the rest in emacs (for plodding paper-like material), and the rest of the rest in console (to reproduce a problem and ensure that it is not an interface problem before posting in sage-support or sage-devel). It seems that I "voted with my feet) : I haven't opened the Sage notebook in more than a year, save to cut'paste older code ... usually to the Jupyter notebook.

HTH,

--
Emmanuel Charpentier

[Jack Dyson]

ok cheers Harald !

[William Stein]

On Thu, Nov 24, 2016 at 1:38 AM, Jack Dyson <jackd...@gmail.com> wrote:
> Hi William,
> installed the docker version - exciting stuff - just brilliant!
> thanks. We need this "sagewide" asap really. Be looking into it as I
> understand more.
> small question when you have a moment:
> docker starts correctly, everything works-I have SMC locally but I
> can't get in sage dedicated modes like R or shell. Some of them work
> like maxima, python, gap. If I create a terminal in my project, I can
> start R. The code is there - just not accessible from a sage file in
> the project: any idea ?

Unfortunately, we haven't added the code to support those Jupyter
kernels to the docker image, which is why it doesn't work yet. I've
made an issue for this:

https://github.com/sagemathinc/smc/issues/1304

[William Stein]

Emmanuel Charpentier wrote:
> 2) In contrast, the Sage notebook, while quite advanced *for its time*, has
> remained a Sage-only interface. Yes, you can use a number of other tools
> with i, *as long as they are known by Sage*.
> This simultaneously enhances and limits its utility. For example, one can
> use the Sage notebook for Maxima (and even use Maxima, R or gap cells in a
> notebook mainly containing Sage code). But only with Sege-suported version
> of Maxima, R or gap. Don't even think of having Lisp or (heavens !) Fortran
> cells...

Sagenb has support for "Fortran cells" since when Josh Kantor and I
added it in 2007... This support integrated with f2py and numpy, to
make fortran functions available automatically. Sagenb also has
%lisp cells.

> 3) The same is true, with both aggravation and mitigation, for SMC and its
> related tools. I like the idea of a \LaTeX editor that supports sage (or
> other) snippets. But, again, *only* with the tools integrated in Sage...

This is wrong. 1) SMC Sage worksheets support using all Jupyter
kernels via the jupyter('kernel name') command, and (2) SMC also
embeds Jupyter notebooks as one of the editor types.

> In contrast, emacs+SageTeX(+R+knitr) give me a better service...

SMC is mainly a collaborative remote website, whereas emacs + ... is a
local application -- apples to oranges.

> Similarly, while the current Jupyter notebook won't display the graphs
> produced by R,

Jupyter notebooks using the R kernel have been able to display graphs
produced by R for years, and do so pretty nicely...

https://cloud.sagemath.com/projects/4a5f0542-5873-4eed-a85c-a18c706e8bcd/files/support/2016-11-24-r-graph.ipynb

> The SMC, however, will probably succeed for
> its initial goal (serving a large group of users using exclusively Sage and
> its federated tools via the Web),

That is not our goal. A significant proportion of users of SMC don't
use Sage at all -- they use Latex or Jupyter notebooks or terminals...
SMC just happens to make it easy to use Sage, among other things.

> for which it seems probably unbeatable ;
> but for people needing a local installation, or needing an interface to
> tools not (yet) integrated with Sage, Jupyter is simply better.

SMC is to Jupyter like Sage is to GAP or Pari. Just as Sage doesn't
compete with GAP, SMC isn't competing with Jupyter notebooks.


--
William (http://wstein.org)

[kcrisman]

Just listening in - curious what the proposed move would do for sagenb server installs?  Is there a realistic porting option for those?  I assume that authentication etc. would not just port at all to either SMC or Jupyterhub, assuming the latter is even usable (I don't know the answer to that).