Project: add hundreds of contributors to sage

[Paul-Olivier Dehaye]

Again, in the big wave of emails, this one also got misdirected:

Hi everyone,

 

I am looking for people who want to help me, in one way or another, bring

hundreds of new first time contributors to sage. If I do not find enough

partners, I will look for other more suitable python-based projects.

 

The idea would be quite simple: teach python programming around some

mathematics (such as combinatorics) and simultaneously produce code that

would be useful for research and worth including in sage. Two catches:

students are given individual problems to work on, and the course is taught

on Coursera. Motivation for the students would come in various ways:

internships, for instance. Quality of the code would be ensured by

peer-testing the programs.

 

If you do not know what Coursera or MOOCs are, you are welcome to take my

upcoming course

https://class.coursera.org/massiveteaching-001

 

If you are interested to play with a MOOC platform yourself, you might want

first to watch the videostream of the 2pm-3pm slot of this conference I am

co-organising on Tuesday:

tinyurl.com/openedx-zurich

as it will help you assess the technical challenges.

 

I am looking at a start date for the course of around October-November, and

to bring the discussion off the mailing list (to private) so as to keep an

element of surprise for the students.

 

Let me know!

 

Paul

Paul-Olivier Dehaye

SNF Professor of Mathematics

University of Zurich

skype: lokami_lokami (preferred)

phone: +41 76 407 57 96

chat: paulo...@gmail.com

twitter: podehaye

freenode irc: pdehaye

[William Stein]

An obstruction to getting code contributions from "hundreds of new
developers" into Sage is the Sage development process via trac, as
documented in the Sage Developers guide. I have 40 students in my
class this quarter, and have encouraged them all to make contributions
to Sage via the standard trac process. Maybe 2 have succeeded, and
when I point them out the documentation and look at it myself, I
wince. This is not to say that the process is bad -- in fact, for
people making major contributions over time, our current system is
pretty reasonable overall, and in some ways quite good. But for
somebody new to software development who wants to make a few small
contributions, e.g., add an example to a docstring, it is not
definitely optimal.

One possible approach for your course would be to do everything
through pull requests on github, which are ridiculously easy to make.
Then we (i.e., some expert sage developers) would aggregate some of
those contributions into a smaller number of trac tickets, which would
get included in Sage in the traditional way. This approach would be
nice since it wouldn't require changing anything about Sage
development or infrastructure, but would allow us to try out new
approaches, which might later influence the Sage development process.

It would be interesting to think about "low hanging fruit" sage
development projects. There's a *ton* of obvious things one might
want to implement for Sage... which are now already done well. E.g.,
I had students in my class tell me what they wanted to do their final
projects on, and in the majority of cases I had to point out that what
they wanted to do was already done in Sage well (or at least done many
times over well in Python). They often still did the projects, but
making it clear that it was for-fun toy code. However, one example
is applying transforms to a 2d plot in Sage -- that seems not done,
though we have 3d transforms in Sage. I know trac is supposed to
have "beginning tickets", but when I send beginners to trac, they
invariably claim to find nothing to do. So a big list of low hanging
fruit that would make sense to implement in Sage would be useful.
Again, an example is:

- implement functions like rotate, translate, etc., for 2d
graphics objects, like the corresponding 3d functions.

That said, Sage is getting mature enough at the easy things that I
wouldn't be surprised if somebody responds to this email and says --
oh, that's easy, via converting the plot to matplotlib, or
something...

Of course when I think about more advanced mathematics, Sage has a
million obvious gaps to fill, and is extremely immature with a huge
amount of low hanging fruit. But this is all grad student level
stuff.

-- William

>
> Let me know!
>
> Paul
>
> Paul-Olivier Dehaye
> SNF Professor of Mathematics
> University of Zurich
> skype: lokami_lokami (preferred)
> phone: +41 76 407 57 96
> chat: paulo...@gmail.com
> twitter: podehaye
> freenode irc: pdehaye
>

> --
> You received this message because you are subscribed to the Google Groups
> "sage-combinat-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sage-combinat-d...@googlegroups.com.
> To post to this group, send email to sage-comb...@googlegroups.com.
> Visit this group at http://groups.google.com/group/sage-combinat-devel.
> For more options, visit https://groups.google.com/d/optout.



--
William Stein
Professor of Mathematics
University of Washington
http://wstein.org

[rjf]

On Wednesday, May 28, 2014 3:31:08 PM UTC-7, Paul-Olivier Dehaye wrote:

Again, in the big wave of emails, this one also got misdirected:

Hi everyone,

 

I am looking for people who want to help me, in one way or another, bring

hundreds of new first time contributors to sage. If I do not find enough

partners, I will look for other more suitable python-based projects.

 

The idea would be quite simple: teach python programming around some

mathematics (such as combinatorics) and simultaneously produce code that

would be useful for research and worth including in sage. Two catches:

students are given individual problems to work on, and the course is taught

on Coursera. Motivation for the students would come in various ways:

internships, for instance. Quality of the code would be ensured by

peer-testing the programs.

William Stein has already responded to the major issues regarding the

Sage development process, but I would just like to comment on this

particular aspect of peer-testing.  Having two or more people who have

just learned python and do not know much mathematics "peer review"

code does not lead to much of an ensured  level of quality.  

Certainly there are other clumps of python aggregating code that are not

as daunting as Sage.  Numpy and Sympy come to mind,  but I doubt

that they would really relish a MOOC's-worth of naive contributions, when

it is pretty much guaranteed that a very high percentage would, under

careful scrutiny, be duplicative, erroneous, poorly coded, or all three.

It's a nice thought to get many hands to write code free.  But impractical,

in spite of Eric Raymond's "Cathedral and Bazaar" essay.  "All bugs are shallow

with enough eyes"  (or whatever the wording is...)  is perhaps plausible if the system

is itself shallow (like linux).  

Where I differ with Raymond is I think there are not enough eyes on the planet to make some

bugs shallow in a "deep" system-- one that does (say) sophisticated symbolic mathematics.

If extra eyes were all that were necessary, there would be no long-standing mathematical

conjectures.

[Paul-Olivier Dehaye]

Thanks for the thoughtful replies. It's a fine line between being critical of the idea and dismissive of the students. Please everyone limit yourselves to criticizing the idea, as students might come to this thread later.

I don't think one should dismiss the students. Look at the Mathworks competition (another thing MATLAB does!), as it is described in Nielsen's book "Reinventing Dicovery". There is a history there of microcontributions on code leading to optimised code. 

There is also two assumptions in your emails: 

1) that they will all have just learned python: the course might be just the right blend of mathematics and CS so that some participants actually a background in python. On top, one can modulate the difficulty progressively to make sure to attract some students who actually have a strong python background already (in MOOCs, there are always some experts lurking)

2) that I would let them choose the topic. Not so. For the specifics of how the course will be run, I need to bring the discussion out of the mailing list.

As William points out, small contributions are important (example in docstring) and the process is currently suboptimal. 

I would add that other small contributions could be important, such as semantic information coming from professional mathematicians who have just learned utter basics of python, to have a mere sense of how the decorator they have just added will affect the method itself. For this, existing annotation tools suffice.

Paul

Paul-Olivier Dehaye

SNF Professor of Mathematics

University of Zurich

skype: lokami_lokami (preferred)

phone: +41 76 407 57 96

chat: paulo...@gmail.com

twitter: podehaye

freenode irc: pdehaye

[kcrisman]

Hi!  Thanks again for your thoughtful comments.  I see two different issues arising in this thread.

1) Your desire to have a MOOC teaching Python programming around some mathematics, which might end up contributing to Sage.  (Or sympy, or numpy, or Gambit, or ...)  That sounds awesome.  

I think the hardest part of doing this is not specifically the ability or knowledge of students, but the realities of getting everyone up to speed on any larger system and contributing without huge amounts of involvement by the instructor/mentor/peer leader/whatever.  I am cautiously optimistic that there is plenty to do in Sage, but without some direction it will not end up being useful.  For a great example, see Vincent Knight's GSOC project - there are lots of students who could implement some basic game theory stuff by the end of a course like this, but to have it actually play well with the rest of Sage, classes, and fit in with (say) 50 other MOOCers' contributions would tax even the most patient overseer.  Which is why having it be a GSOC project is a very good idea.

And here is one place I agree with rjf - Raymond is very insightful, but does not have a monopoly on insights regarding open source or software development.  If you don't know calculus and Lisp, you can't help with large bits of Maxima (hence Sage), period, no matter how talented otherwise.

For me personally, I would have to see a syllabus (however archaic that may seem) and the "typical" intro level for the students, though such a thing probably doesn't exist.  Given that MOOC dropout rates are pretty high, and that it will probably be hard to figure out ahead of time what a given student brings to the table, one would need a very large collection of things to look at indeed.  Unless one had a far smaller set of carefully selected projects, and then let a few dozen MOOCers at each of them, and then perhaps as a final project had them decide which of these was best... that's just a brainstorm, though.

Having given my caveats, I want to reiterate that it sounds awesome, and IS doable, but probably with more work than one thinks.  And I'll put my money where my mouth is and volunteer to chat personally at some point this summer regarding some possible projects - though again, it may be surprising just how involved even very simple things end up getting "in production", as I've experienced with some students I've worked with on contributing to Sage.

2) There is a higher burden to development than needs be.  This is always true to some extent, of course, but especially for documentation this doesn't have to be the case.  William summarized this well.  

In particular, the git switch was supposedly going to make this sort of contribution easier, but as far as I can tell, it is no easier, and maybe harder, than with the previously workflow.  See a postscript for a rant about this.  Anyway, this is something that will be an issue for contributing to any longer-term project, I think, in nontrivial ways.

- kcrisman

[Robert Bradshaw]

On Thu, May 29, 2014 at 8:49 AM, kcrisman <kcri...@gmail.com> wrote:
> Hi! Thanks again for your thoughtful comments. I see two different issues
> arising in this thread.
>
> 1) Your desire to have a MOOC teaching Python programming around some
> mathematics, which might end up contributing to Sage. (Or sympy, or numpy,
> or Gambit, or ...) That sounds awesome.

+1

Though I wouldn't expect everyone to end up with a workable
contribution, it's not a stretch to expect at least a handful of
ready-to-incorporate improvements.

> 2) There is a higher burden to development than needs be. This is always
> true to some extent, of course, but especially for documentation this
> doesn't have to be the case. William summarized this well.

There is so much bureaucracy. I just keep thinking back to the "22
easy steps to contribute to Sage."

> In particular, the git switch was supposedly going to make this sort of
> contribution easier, but as far as I can tell, it is no easier, and maybe
> harder, than with the previously workflow. See a postscript for a rant
> about this.

I think that's because we took our trac-based (and patch-based)
workflow and just plugged git in. I don't know if we want to open this
discussion now, but it's certainly worth revisiting (starting with
defining what our goals are rather than what our tools are). But now
that we're on git a pull request is 99% of what a ticket is
(specifically, an annotated point to a particular git branch/commit
that needs review) and we should seriously consider accepting that
format for contributions.

> Anyway, this is something that will be an issue for
> contributing to any longer-term project, I think, in nontrivial ways.
>
> - kcrisman
>
>

> --
> You received this message because you are subscribed to the Google Groups

> "sage-devel" group.

> To unsubscribe from this group and stop receiving emails from it, send an

> email to sage-devel+...@googlegroups.com.
> To post to this group, send email to sage-...@googlegroups.com.
> Visit this group at http://groups.google.com/group/sage-devel.

[kcrisman]

Please excuse the following rant.  As usual, it is ill-informed, and if some of the points are due to ignorance, I have no problem with that being pointed out.  But from reading the lists, I'm not the only one experiencing difficulty with this sort of thing.  At the very least I think it shows it's not that easy to do everything right now.

<rant>

Git workflow.

The goal was to reduce work for some developers and make things more modular, but in fact what happens is that people are basing their "branches" on all kinds of different starting points, forcing constant recompilation for even the most trivial changes.  I'm told "ccache" will help with this, but it's still not in the developer guide, and I'm not sure it will help with [s]pkgs anyway, which also change frequently.

In addition, a web-based interface just like github pull requests was promised, but I don't see it.  Since we don't use the github version to allow people to just "edit" (see e.g. https://github.com/grant/sage/commit/27c97d376a956a05c6e4abd97102f80195bda4c0 for one request that apparently has been ignored), we are actually in a *worse* place than we were with patches, where at least those could be applied more or less independently of the state of Sage.  

As another example, in attempting to review one patch which relies upon the new Maxima update: where that is, one cannot learn easily from git log, since 

$ git log | grep maxima | less

$ git log | grep Maxima | less

both do not tell me that in fact, even though Trac says "closed", #13973 apparently is not yet in a merged release, and so I just wasted a lot of time figuring that out.  Since "Merged in" is no longer updated.  You would think that I could learn this by doing 

$ git log | grep 13973

but since Trac #s are no longer in commit messages, that isn't foolproof either.  And indeed, the standard output of "git log" is kind of scary, because while some entries look like this, roughly as in the old one

commit 3ea4f9bfc3c1acb6cb9eaa8e7a8ffa41779a6181

Author: XYZ

Date:   Thu May 22 08:37:36 2014 +0200

    16199: documentation cosmetics

others look like this

commit e9352062ad7fedbe00ad62d397672203824e0267

Merge: cc54f6e 80e319d

Author: XYZ

Date:   Thu May 22 08:01:19 2014 +0200

    Merge branch 'develop' into t/16199/improve_docs__add_doctests_in_power_seri

    

    * develop: (606 commits)

      Updated Sage version to 6.3.beta1

      trac #16237: Typos

      Use lazy_import instead

      Use point collections more directly.

      trac #15036: fix late import of is_Matrix to avoid startup problems

      Fixes to normal_form_pc to work with point collections properly.

      Fixed failing doctests.

      trac #16277: A typo

      minor fixes

      Trac 15099: numeric evaluation of zetaderiv

      trac 16211: Review reviewer patch.

      Added a category parameter to Module

      change raise syntax to python3 style

and still others look like this

commit 9a4b0b83fa06a113c485f22f37af02c24ef35e90

Merge: f4cbb5f b8fea1c

Author: Release Manager <rel...@sagemath.org>

Date:   Wed May 21 16:40:31 2014 +0100

    Trac #15921: work around Maxima fpprintprec bug and other ARM-specific probl

    

    [https://groups.google.com/d/msg/sage-devel/oRpkswzpK38/rNVbVN2RyEcJ

    Maxima uses CL FORMAT function wrongly], resulting in outputting wrong

    number of digits for floats (one extra), and

    contradicting its own manual on fpprintprec. In particular it outputs

    too many digits on ix86 and ix86_64, which got in Sage's doctests. As a

    result, doctests fail on ARM.

    

    The fixes are to convert the results into `RealField(prec)`, with

    appropriate `prec`

    (at least 54, or sometimes more). This ticket also fixes ARM-specific

    numerical noise stemming

    from various other upstream problems, such as `eglibc` loss of precision

    in `lgamma`.

I don't know why the description of a ticket is ending up in the log.  We want a description of what actually was done, and the description in the ticket is often full of brainstorming etc.

Oh, and this all assumes that one knows about pipes and grep.  Perhaps "git trac" deals with these kinds of situations, which is great, but one shouldn't really have to use git or the command line at all to find out some of this information.  Particularly not as a beginner.

Anyway, in order to use git properly to do trivial change, one still really has to learn at least as much as before with hg_sage or hg or sage -dev or whatever the flavor of the month is.  For instance, for this one I guess I need to first do #13973, and then somehow add #13712 on top of it - which sounds like a more advanced operation of "merging", and we are many times warned not to do unnecessary merges.  A hint from Travis S. and some Google helped me eventually figure out how to do this but it's certainly no easier than what I would have done before.

</rant>

Okay, done ranting, and foolish stuff it probably was.  At first I thought this wasn't all a regression, but by the end of writing this I'm not so sure.  I appreciate Robert's input on this a lot, though.  I actually think that using Github per se would be worse, definitely, because you have to interact with all their stuff; I find helping with sagenb very tiresome for that reason.  So here are a few concrete proposals. Obviously I am not capable of doing them myself, or I would, and clearly #3 is hard.

1) Put "Merged in" back.

2) Fix just how much crazy stuff shows up in "git log", presumably by merging things differently or with different messages or something.

3) Wherever the web interface idea went to, bring it back.  Or create a workflow where github pull requests create a ticket, or something.  (I do like that we have "issues" disabled, since otherwise that would keep things in too many places.)

4) Get http://trac.sagemath.org/ticket/14372 in.

[Travis Scrimshaw]

   I never really have had a problem with the new workflow (in fact, I actually prefer it to the old one). However I had a good command of git coming into this and read the "git the hard way". So my 2 cents would be to have developers spend time learning git properly instead of using the dev scripts (I think this is a useful skill beyond Sage development). However I do see your point that this is an extra barrier to Sage development since IMO one generally needs to know more about git than about Hg in order to development.

1) Put "Merged in" back.

+1 I'd also suggest not closing/merging tickets until shortly before releasing the next beta.

 

2) Fix just how much crazy stuff shows up in "git log", presumably by merging things differently or with different messages or something.

I don't think there's much we can do about this considering how git workflows are suppose to work and since the develop does have everything merged in.

4) Get http://trac.sagemath.org/ticket/14372 in.

+1 

Best,

Travis

[William Stein]

On Thu, May 29, 2014 at 10:01 AM, kcrisman <kcri...@gmail.com> wrote:
> Please excuse the following rant. As usual, it is ill-informed, and if some

I appreciate it, and I'm glad we're having this discussion. (It's a
rant, but it isn't a flame. Yeah!)

> of the points are due to ignorance, I have no problem with that being
> pointed out. But from reading the lists, I'm not the only one experiencing
> difficulty with this sort of thing. At the very least I think it shows it's
> not that easy to do everything right now.
>
> <rant>
> Git workflow.
>
> The goal was to reduce work for some developers and make things more
> modular, but in fact what happens is that people are basing their "branches"
> on all kinds of different starting points, forcing constant recompilation
> for even the most trivial changes. I'm told "ccache" will help with this,
> but it's still not in the developer guide, and I'm not sure it will help
> with [s]pkgs anyway, which also change frequently.

There were arguments last year by various people that our patch-based
workflow was bad, at least as opposed to using branches. There was
little in the way of arguments back for patches. I personally wasn't
really paying attention then, and I frankly didn't know which was
better, having only had experience with patches. I'm glad that
you're presenting some of the arguments below in favor of the value of
a more patches-based workflow.

I recently listened to an interview with the release manager for the
stable linux kernel. Linux is of course a big project that uses git
-- in fact, as we all know, Linus invented git for Linux development.
They use *patches* rather than branches. In fact, their workflow is
email + patches. He argues in the interview (in response to
questions, etc.) that this is far more efficient than the branch
model.
Evidently, git has extremely good support for sending patches via
email and applying them from email (at least in a local mail spool).
Another project I'm involved in called bup
(https://github.com/bup/bup) does the same thing -- all code
contributions are patches to the mailing list, so I've seen this work.

I'm not arguing for replacing trac by patches to a mailing list! I'm
just pointing out that some people's assertions that using patches
instead of branches is not properly using git can't be right, since
the Linux kernel devs use of git is just as "proper" as anybody
else's. Basically, I'm trying to argue that there is merit to your
rant.

It depends. If one wanted to add a docstring in github, a person
could do this in a minute:

(0) get account on github

(1) browser to a file, e.g.,
https://github.com/sagemath/sage/blob/master/src/sage/calculus/calculus.py

(2) click edit, which in 1 click (and 1 second) creates a fork of the
repo, and opens that file in an editor in your browser.

(3) make changes and click the big green button "Propose file change"
at the bottom of the screen.

Github has solved some real technical problems regarding lowering the
barrier to entry for people to contribute to projects, which we
haven't solved, and I think we shouldn't just ignore/dismiss this...

Of course, the above is only reasonable for certain very limited types
of contributions. For us, we would want to be able to use the
modified version of Sage online as well, so one could test/develop
code, then "Propose change". If only there were a good way to use
Sage online with a code editor...

> that reason. So here are a few concrete proposals. Obviously I am not
> capable of doing them myself, or I would, and clearly #3 is hard.
>
> 1) Put "Merged in" back.
> 2) Fix just how much crazy stuff shows up in "git log", presumably by
> merging things differently or with different messages or something.
> 3) Wherever the web interface idea went to, bring it back. Or create a
> workflow where github pull requests create a ticket, or something. (I do
> like that we have "issues" disabled, since otherwise that would keep things
> in too many places.)
> 4) Get http://trac.sagemath.org/ticket/14372 in.
>

> --
> You received this message because you are subscribed to the Google Groups
> "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sage-devel+...@googlegroups.com.
> To post to this group, send email to sage-...@googlegroups.com.
> Visit this group at http://groups.google.com/group/sage-devel.
> For more options, visit https://groups.google.com/d/optout.

--
William Stein
Professor of Mathematics
University of Washington
http://wstein.org

[Volker Braun]

On Thursday, May 29, 2014 6:01:24 PM UTC+1, kcrisman wrote:

As another example, in attempting to review one patch which relies upon the new Maxima update

The git branch contains the entire code, so automatically has all requirements. You don't need to know where the maxima update comes from to use the branch.

 

: where that is, one cannot learn easily from git log, since 

$ git log | grep maxima | less

$ git log | grep Maxima | less

The git log is not a plain text file, its a directed acyclic graph. There is much more useful information in it than any possible linearization. More complicated than mailing patches? Sure. Why? Because mailing patches around doesn't work at that scale.

 

$ git log | grep 13973

$ git log build/pkgs/maxima     # see which commits actually touched maxima

commit 33417fa92057fc7ef1cf52e301b9b73a2d5c2a83

Author: Peter Bruin <P.B...@warwick.ac.uk>

Date:   Fri May 16 11:33:00 2014 +0100

    Trac 13973: upgrade Maxima to 5.33.0

Although Peter thought that the commit was for #13973 when he wrote it, that need not be the ticket that actually made use of the commit and pulled it into Sage. We check:

# git trac find 33417fa92057fc7ef1cf52e301b9b73a2d5c2a83

$ git trac find 33417fa92057fc7ef1cf52e301b9b73a2d5c2a83

Commit has been merged, but not into a released version.

commit 6220027f174e01ae252f277acb3a75de7053239c

Merge: a3c4cf3 a130eed

Author: Release Manager <rel...@sagemath.org>

Date:   Sun May 25 11:20:54 2014 +0100

    Trac #13973: Upgrade Maxima to 5.33.0

    

    This is a continuation of #13364, and aims at including more upstream

    bug fixes, which e.g. fix an

    [http://permalink.gmane.org/gmane.comp.mathematics.sage.support/30644

    issue reported on sage-support],

    which was reported as Maxima bug

    [https://sourceforge.net/p/maxima/bugs/2535 2535], and marked there as

    closed in post-5.29.1.

I don't know why the description of a ticket is ending up in the log.  We want a description of what actually was done, and the description in the ticket is often full of brainstorming etc.

Its up to you (the ticket author/reviewer) to make the description fit the ticket.

 For instance, for this one I guess I need to first do #13973, and then somehow add #13712 on top of it

No. For reviewing #13712 (say) you only need to get that branch. Batteries are already included. If it doesn't work then that is a negative review.

[William Stein]

On Thu, May 29, 2014 at 10:49 AM, Volker Braun <vbrau...@gmail.com> wrote:
> On Thursday, May 29, 2014 6:01:24 PM UTC+1, kcrisman wrote:
>>
>> As another example, in attempting to review one patch which relies upon
>> the new Maxima update
>
>
> The git branch contains the entire code, so automatically has all
> requirements. You don't need to know where the maxima update comes from to
> use the branch.
>
>>
>> : where that is, one cannot learn easily from git log, since
>> $ git log | grep maxima | less
>> $ git log | grep Maxima | less
>
>
> The git log is not a plain text file, its a directed acyclic graph. There is
> much more useful information in it than any possible linearization. More
> complicated than mailing patches? Sure. Why? Because mailing patches around
> doesn't work at that scale.

The argument that mailing patches "doesn't work at that scale" isn't
convincing, since Linux kernel development is much bigger than Sage
development, and they mail patches around [1]. There must be better
arguments.... Just to be clear -- I'm definitely not arguing that we
should switch to mailing patches around, or even that we should switch
back to using patches; also me and many other people are *very, very*
impressed and pleased with how you (Volker) are doing release
management using the current system!

[1] https://www.kernel.org/doc/Documentation/SubmittingPatches

> --
> You received this message because you are subscribed to the Google Groups
> "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sage-devel+...@googlegroups.com.
> To post to this group, send email to sage-...@googlegroups.com.
> Visit this group at http://groups.google.com/group/sage-devel.
> For more options, visit https://groups.google.com/d/optout.



--
William Stein

Professor of Mathematics
University of Washington
http://wstein.org

[Volker Braun]

On Thursday, May 29, 2014 7:00:52 PM UTC+1, William wrote:

The argument that mailing patches "doesn't work at that scale" isn't
convincing, since Linux kernel development is much bigger than Sage
development, and they mail patches around

That is not really true. They do mail patches around, but only at one very specific point in the lifetime of a changeset. The entire model is geared towards making everything as easy as possible for Linus. You present your change in a way that is as easy as possible for one of the inner circle to look at, and then forward to Linus. They don't collaborate with you, the only feedback is that your code is merged or refused (likely before it ever reaches Linus). Since it is a one way road its of course easier to mail a patch. 

The way I see it that does make it more difficult for the developers outside of the inner circle, they will have to resolve conflicts in their own branches whenever a new version appears. But that is parallelizable and developers are experienced with the tools, so its not a problem for the kernel.

[Simon King]

Hi Volker,

On 2014-05-29, Volker Braun <vbrau...@gmail.com> wrote:
>> : where that is, one cannot learn easily from git log, since
>> $ git log | grep maxima | less
>> $ git log | grep Maxima | less
>>
>
> The git log is not a plain text file, its a directed acyclic graph. There
> is much more useful information in it than any possible linearization.

If a simple question such as "what is the ticket number for the latest Maxima
upgrade" is not easy to answer from the git log, then the information is
*not* useful. The information exists, but is not useful. That's an
important practical difference. Anyway, you have shown in your mail that
it *is* possible to get the information in this case.

Maxima is an spkg. Let's instead look at the Sage library. We can use "git
blame" similar to "hg annotate" to get the revision in which a
particular line of code was changed. This information can be used to
search the corresponding commit message in the log.

Before switching to git, we had the policy (enforced by commit hooks, if
I recall correctly) that the commit message mentions the ticket number.
I think this was very helpful. But now---because you keep saying that a
commit does not belong to a specific ticket---the log often does not
mention the ticket number. Some people mention the ticket number, others
don't.

Hence, when I wonder about a specific line of code and want to look up
the discussion that took place on trac and resulted in this line of
code: How can I easily find that discussion?

Admittedly, one could argue that people shouldn't be so lazy and put the
ticket number in. But then git causes a meta-problem: Some people
advocating git go around and tell people that a commit does not belong
to a ticket. Hence, people are not putting the ticket number into the
commit message. And on the side of trac, there can not be an automatism
that inserts the ticket number if it is missing, because "Changing a commit
message is changing history"---which is something that I technically
understand (the commit's sha1 hash depending on the commit message),
but still consider it a flaw.

> More
> complicated than mailing patches? Sure. Why? Because mailing patches around
> doesn't work at that scale.

Yes, it only works on the tiny scale of Linux development, as William pointed
out... :-D

>> I don't know why the description of a ticket is ending up in the log. We
>> want a description of what actually was done, and the description in the
>> ticket is often full of brainstorming etc.
>>
>
> Its up to you (the ticket author/reviewer) to make the description fit the
> ticket.

The main complaint was that the entire ticket description is ending up in the
log. Don't you think that a ticket description has a different purpose
than a commit message?

Cheers,
Simon

[Volker Braun]

On Thursday, May 29, 2014 7:26:35 PM UTC+1, Simon King wrote:

Before switching to git, we had the policy (enforced by commit hooks, if
I recall correctly) that the commit message mentions the ticket number.

No, that was Jeroen manually (ok, with a script) telling you days/weeks/months later that the commit message is wrong and needs to be fixed.

code: How can I easily find that discussion?

Pretty easy, I would say: "git trac find <sha1>". We can add an option to open the trac ticket in a browser window if its too slow for you ;-)

(the commit's sha1 hash depending on the commit message),

Commits are immutable, and enforcing that is the primary feature of every version control system. In mercurial, too. There are tools for editing the history, but using them always comes at a price.

Really, you have two options:

* The branching model we are using where all individual commits end up "public"

* The kernel model where every developer is responsible for cleaning up their history until it is perfect, and only then share the code.

The second places a much higher burden on the individual developer, and on his direct collaborators who have to follow the rebases. The advantage is that the history looks beautiful, and it is more efficient for the guy at the top. 

The main complaint was that the entire ticket description is ending up in the 
log. Don't you think that a ticket description has a different purpose 
than a commit message? 

These are different commits. Every developer's commit contains a message to summarize what the commit is about. The release manager's merge commit doesn't have any intention behind it beyond the obvious "merge that branch". So the commit message is up for grabs. I think its a good idea to put some of the ticket metadata there, for example this is then still available if you are not currently connected to the internet. 

[Simon King]

Hi Volker,

On 2014-05-29, Volker Braun <vbrau...@gmail.com> wrote:

> code: How can I easily find that discussion?
>>
>
> Pretty easy, I would say: "git trac find <sha1>".

Wow! That is in fact very useful. Didn't know about this. So far, I was
mainly using the dev scripts, and "git trac" only once, I think.

Cheers,
Simon

[John Cremona]

Can someone point me to instructions for how to install (and use) "git trac"?

John

>
> Cheers,
> Simon

[Dima Pasechnik]

On 2014-05-29, John Cremona <john.c...@gmail.com> wrote:
> On 29 May 2014 20:22, Simon King <simon...@uni-jena.de> wrote:
>> Hi Volker,
>>
>> On 2014-05-29, Volker Braun <vbrau...@gmail.com> wrote:
>>> code: How can I easily find that discussion?
>>>>
>>>
>>> Pretty easy, I would say: "git trac find <sha1>".
>>
>> Wow! That is in fact very useful. Didn't know about this. So far, I was
>> mainly using the dev scripts, and "git trac" only once, I think.
>
> Can someone point me to instructions for how to install (and use) "git trac"?

https://github.com/sagemath/git-trac-command

[Volker Braun]

On Thursday, May 29, 2014 8:29:04 PM UTC+1, John Cremona wrote:

Can someone point me to instructions for how to install (and use) "git trac"? 

The developer guide on a sufficiently recent (6.2+) Sage version. 

On a related note, the docs on the web page are still from 6.1.1 

[John Cremona]

Thanks to both -- now installed in two machines and trying out.

John

[leif]

Robert Bradshaw wrote:
> [...] now that we're on git a pull request is 99% of what a ticket is

I'm happy that this (still) is *not* the case (for most tickets at least).

> (specifically, an annotated point to a particular git branch/commit
> that needs review) and we should seriously consider accepting that
> format for contributions.

I personally don't like trac very much (although it's become better over
time), but IMHO all discussion / review of a contribution should
(finally) happen in a central place, which currently is trac for Sage
development.

(During the switch to git, trac and ordinary reviewing effectively got
bypassed; fortunately that stopped after the switch.)


-leif

--
() The ASCII Ribbon Campaign
/\ Help Cure HTML E-Mail

[leif]

kcrisman wrote:
> <rant>
> Git workflow.
>
> The goal was to reduce work for some developers and make things more
> modular, but in fact what happens is that people are basing their
> "branches" on all kinds of different starting points, forcing constant

> recompilation for even the most trivial changes. [...] </rant>


Nobody ever promised it would make *reviewers'* life easier.

[Robert Bradshaw]

On Thu, May 29, 2014 at 2:22 PM, leif <not.r...@online.de> wrote:
> kcrisman wrote:
>>
>> <rant>
>> Git workflow.
>>
>> The goal was to reduce work for some developers and make things more
>> modular, but in fact what happens is that people are basing their
>> "branches" on all kinds of different starting points, forcing constant
>> recompilation for even the most trivial changes. [...] </rant>
>
> Nobody ever promised it would make *reviewers'* life easier.

Making the reviewer's life easier should be a primary goal as
well--look at the number of unreviewed tickets sitting on trac right
now.

Previously, we still had people basing their "branches" on all kinds
of different starting points, but it was all implicit, so it was
harder to look at and test exactly the code someone else intended to
be contributing.

If I understand your complaint correctly, the problem is that
re-compiling after switching branches is too expensive.
http://trac.sagemath.org/ticket/14372 needs to be resolved. I would
note that for most trivial changes there's no need to download and
build/test it yourself--the patchbot should do this for you. Of course
sometimes you want to try out (and contribute back!) different corner
cases, bigger examples, etc. but that's not essential for many
reviews.

It would be very cool if there was a way to have multiple builds of
Sage co-exist cheaply. Imagine a service where you could ssh into
trac1234.sagemath.org and be presented with a sage prompt with ticket
#1234 built and ready to play with. Or a notebook. How cheap could it
be to have a SMC project for every open ticket? And then you could
take some of your examples and insanely easily (e.g. with the click of
a button, or maybe a copy-paste) add them to a docstring. That's my
vision of how to make life easier for reviewers on top of what the
patchbot already does.

- Robert

[Robert Bradshaw]

Exactly. I'd like to see us better integrate this workflow rather than
try to replicate it. For more complex changes, one can easily
"migrate" from this workflow to a local copy with git. It'd be cool to
make it just as easy to migrate a branch on github (or any git hosting
service) to a SMC project (and back).

Personally, I find line comments and being able easily look at
different diffs very useful as well compared to using trac, but I
agree that it's good to have issues consolidated at one place.

[Robert Bradshaw]

On Thu, May 29, 2014 at 10:29 AM, Travis Scrimshaw <tsc...@ucdavis.edu> wrote:
> I never really have had a problem with the new workflow (in fact, I
> actually prefer it to the old one). However I had a good command of git
> coming into this and read the "git the hard way". So my 2 cents would be to
> have developers spend time learning git properly instead of using the dev
> scripts (I think this is a useful skill beyond Sage development).

Yes, yes, yes. "git trac" fits well here to plug in only the
sage-specific bits.

> However I
> do see your point that this is an extra barrier to Sage development since
> IMO one generally needs to know more about git than about Hg in order to
> development.
>
>
>> 1) Put "Merged in" back.
>
>
> +1 I'd also suggest not closing/merging tickets until shortly before
> releasing the next beta.

Why?

>> 2) Fix just how much crazy stuff shows up in "git log", presumably by
>> merging things differently or with different messages or something.
>
> I don't think there's much we can do about this considering how git
> workflows are suppose to work and since the develop does have everything
> merged in.

On this note, I think the ticket description is the perfect thing to
put in the merge commit. People should be more aware of that, and
craft (sometimes as the ticket evolves) good descriptions that
describe exactly what the ticket is about.

- Robert

[kcrisman]

Well, I am thankful that my rant led to some pretty substantive discussion.  Here let me summarize some thoughts.

A) +1 to having something where a github-like editing thing works.  I've used this at least once with matplotlib.  My beef with github is orthogonal to the web interface piece.  If there is a good workflow where we can direct contributors to make smaller edits on the github mirror and they'll be taken care of, that's great.

B) I didn't realize that the "release manager" commits were separate from the "author" ones.  That is strange to me, but I guess it's there.  Nonetheless, I think that the vast majority of ticket descriptions are (or at least start as) fairly half-baked notions of what could be added or fixed, and are entirely inappropriate for a commit message to siphon through in "git log".  Especially since they are so long compared to most commit messages (and not for good reasons).

C) I haven't heard any -1's to "Merged in" being resuscitated, so maybe that should really happen.

[More thoughts: Robert B: I think Travis' point about doing it near the release time is so that people know where to start from.  My whole trouble earlier today ended up being because, since it said closed, I assumed the Maxima upgrade HAD to be in the latest beta - which it wasn't.  And several "git pull" attempts in my develop branch did nothing, since develop is still at 6.3.beta2 as of this writing (on github at least).  As it turned out, it wasn't based on the right ticket, so I guess that could have been "needs work", but it *says* "Depends on #13973" so in the old workflow that should have been enough (install #13973, then install this one). I feel like this is closely related.]

D) Patchbot?  What patchbot?  At least the colorings aren't working right now, it seems.  But I would love to trust that again and not have to rebuild trivial things.

E) Please everyone let's put Trac #s back in the commit messages, then, if the immutability is a barrier there.  Some of the most recent ones are completely mystifying.  And then it will be possible to figure out where the new code IS in the tree, as opposed to just the merges which can be patchbombs.

[Nathann Cohen]

If I understand your complaint correctly, the problem is that
re-compiling after switching branches is too expensive.

You *NEVER* need to recompile everything when switching branches. There is a trick : you should never checkout a branch but instead pull it (merge it) with your current version of Sage.

This way, you never have to checkout a branch of Sage which is based on an old release.

This is the "gtmp" function I use for almost everything (and in particular for review). It deletes then create a "tmp" branch in git containing the remove branch you are interested in (assuming that the local branch 'd' points toward the release that you use, i.e. develop in my case)

function gtmp

{

    cd ~/sage

    git checkout d

    git branch -D tmp

    git checkout -b tmp d &&

    git pull trac "$1"

}

I only type "gtmp public/branch_name" and I can review/test a branch, no recompilation involved as only the files changes in the branch have been modified.

Nathann

[kcrisman]

On Friday, May 30, 2014 2:19:24 AM UTC-4, Nathann Cohen wrote:

If I understand your complaint correctly, the problem is that
re-compiling after switching branches is too expensive.

You *NEVER* need to recompile everything when switching branches. There is a trick : you should never checkout a branch but instead pull it (merge it) with your current version of Sage.

Hmm, I was kind of going on the "avoid unnecessary merges" thing.  Also I am scared about merge conflicts, which I do not know how to handle because I don't have a "merge editor", or maybe I do and don't know it.  But this is helpful to know, thanks.

This is the "gtmp" function I use for almost everything (and in particular for review). It deletes then create a "tmp" branch in git containing the remove branch you are interested in (assuming that the local branch 'd' points toward the release that you use, i.e. develop in my case)

Volker, is there some similar functionality in "git trac"?

 

function gtmp

{

    cd ~/sage

    git checkout d

    git branch -D tmp

    git checkout -b tmp d &&

    git pull trac "$1"

}

I only type "gtmp public/branch_name" and I can review/test a branch, no recompilation involved as only the files changes in the branch have been modified.

Well, if there are .pyx files or pkg changes... but point taken.  Thanks!

[Volker Braun]

On Friday, May 30, 2014 1:18:43 PM UTC+1, kcrisman wrote:

Hmm, I was kind of going on the "avoid unnecessary merges" thing.

You can of course do whatever you want on your computer as long as you don't upload it...

 Also I am scared about merge conflicts, which I do not know how to handle because I don't have a "merge editor"

You don't need anything special, imho the standard git procedure of adding conflict markers and editing that file is perfectly fine (and what I use most of the time even though I have some special merge editors). 

Volker, is there some similar functionality in "git trac"?

No (it doesn't really do anything different as far as communicating with trac goes), but we could easily add it...

[kcrisman]

Hmm, I was kind of going on the "avoid unnecessary merges" thing.

You can of course do whatever you want on your computer as long as you don't upload it...

Right, but the problem is that currently I am uploading it ;-)

[Paul-Olivier Dehaye]

Just got a Github Education account today for my lab, which led me to look a bit more at their site. This video is relevant:

https://education.github.com/stories#UCBerkeley

> If extra eyes were all that were necessary, there would be no long-standing mathematical conjectures.

What is needed is both extra eyes and a community welcoming of new ideas. That's what the polymath projects do, and they have been quite successful so far, proving results that had vexed Fields medalists.

Paul

Paul-Olivier Dehaye

SNF Assistant Professor of Mathematics

University of Zurich

skype: lokami_lokami (preferred)

phone: +41 76 407 57 96

chat: paulo...@gmail.com

twitter: podehaye

freenode irc: pdehaye

On Thu, May 29, 2014 at 11:34 AM, Paul-Olivier Dehaye <paul-oliv...@math.uzh.ch> wrote:

Thanks for the thoughtful replies. It's a fine line between being critical of the idea and dismissive of the students. Please everyone limit yourselves to criticizing the idea, as students might come to this thread later.

I don't think one should dismiss the students. Look at the Mathworks competition (another thing MATLAB does!), as it is described in Nielsen's book "Reinventing Dicovery". There is a history there of microcontributions on code leading to optimised code. 

There is also two assumptions in your emails: 

1) that they will all have just learned python: the course might be just the right blend of mathematics and CS so that some participants actually a background in python. On top, one can modulate the difficulty progressively to make sure to attract some students who actually have a strong python background already (in MOOCs, there are always some experts lurking)

2) that I would let them choose the topic. Not so. For the specifics of how the course will be run, I need to bring the discussion out of the mailing list.

As William points out, small contributions are important (example in docstring) and the process is currently suboptimal. 

I would add that other small contributions could be important, such as semantic information coming from professional mathematicians who have just learned utter basics of python, to have a mere sense of how the decorator they have just added will affect the method itself. For this, existing annotation tools suffice.

Paul

Paul-Olivier Dehaye

SNF Professor of Mathematics

University of Zurich

skype: lokami_lokami (preferred)

phone: +41 76 407 57 96

chat: paulo...@gmail.com

twitter: podehaye

freenode irc: pdehaye

On Thu, May 29, 2014 at 3:40 AM, rjf <fat...@gmail.com> wrote:

On Wednesday, May 28, 2014 3:31:08 PM UTC-7, Paul-Olivier Dehaye wrote:

Again, in the big wave of emails, this one also got misdirected:

Hi everyone,

 

I am looking for people who want to help me, in one way or another, bring

hundreds of new first time contributors to sage. If I do not find enough

partners, I will look for other more suitable python-based projects.

 

The idea would be quite simple: teach python programming around some

mathematics (such as combinatorics) and simultaneously produce code that

would be useful for research and worth including in sage. Two catches:

students are given individual problems to work on, and the course is taught

on Coursera. Motivation for the students would come in various ways:

internships, for instance. Quality of the code would be ensured by

peer-testing the programs.

William Stein has already responded to the major issues regarding the

Sage development process, but I would just like to comment on this

particular aspect of peer-testing.  Having two or more people who have

just learned python and do not know much mathematics "peer review"

code does not lead to much of an ensured  level of quality.  

Certainly there are other clumps of python aggregating code that are not

as daunting as Sage.  Numpy and Sympy come to mind,  but I doubt

that they would really relish a MOOC's-worth of naive contributions, when

it is pretty much guaranteed that a very high percentage would, under

careful scrutiny, be duplicative, erroneous, poorly coded, or all three.

It's a nice thought to get many hands to write code free.  But impractical,

in spite of Eric Raymond's "Cathedral and Bazaar" essay.  "All bugs are shallow

with enough eyes"  (or whatever the wording is...)  is perhaps plausible if the system

is itself shallow (like linux).  

Where I differ with Raymond is I think there are not enough eyes on the planet to make some

bugs shallow in a "deep" system-- one that does (say) sophisticated symbolic mathematics.

If extra eyes were all that were necessary, there would be no long-standing mathematical

conjectures.

 

If you do not know what Coursera or MOOCs are, you are welcome to take my

upcoming course

https://class.coursera.org/massiveteaching-001

 

If you are interested to play with a MOOC platform yourself, you might want

first to watch the videostream of the 2pm-3pm slot of this conference I am

co-organising on Tuesday:

tinyurl.com/openedx-zurich

as it will help you assess the technical challenges.

 

I am looking at a start date for the course of around October-November, and

to bring the discussion off the mailing list (to private) so as to keep an

element of surprise for the students.

 

Let me know!

 

Paul

Paul-Olivier Dehaye

SNF Professor of Mathematics

University of Zurich

skype: lokami_lokami (preferred)

phone: +41 76 407 57 96

chat: paulo...@gmail.com

twitter: podehaye

freenode irc: pdehaye

[Dima Pasechnik]

On 2014-05-30, Nathann Cohen <nathan...@gmail.com> wrote:
>
>>
>> If I understand your complaint correctly, the problem is that
>> re-compiling after switching branches is too expensive.
>>
>
> You *NEVER* need to recompile everything when switching branches. There is
> a trick : you should never checkout a branch but instead pull it (merge it)
> with your current version of Sage.

+1

On the other hand, as long as the docbuild is broken, and as long as
we care about docs being bildable, one often has to recompile
everything, as for rebuilding docs nothing short of "make distclean &&
make" works.

[Dima Pasechnik]

On 2014-05-30, kcrisman <kcri...@gmail.com> wrote:
>
>
> On Friday, May 30, 2014 2:19:24 AM UTC-4, Nathann Cohen wrote:
>>
>> If I understand your complaint correctly, the problem is that
>>> re-compiling after switching branches is too expensive.
>>>
>>
>> You *NEVER* need to recompile everything when switching branches. There is
>> a trick : you should never checkout a branch but instead pull it (merge it)
>> with your current version of Sage.
>>
>>
> Hmm, I was kind of going on the "avoid unnecessary merges" thing. Also I
> am scared about merge conflicts, which I do not know how to handle because
> I don't have a "merge editor", or maybe I do and don't know it. But this
> is helpful to know, thanks.
>
>
>> This is the "gtmp" function I use for almost everything (and in particular
>> for review). It deletes then create a "tmp" branch in git containing the
>> remove branch you are interested in (assuming that the local branch 'd'
>> points toward the release that you use, i.e. develop in my case)
>>
>>
> Volker, is there some similar functionality in "git trac"?

I asked for essentially this functionality to be in git trac, see
https://github.com/sagemath/git-trac-command/issues/13

[Volker Braun]

On Saturday, May 31, 2014 1:42:44 AM UTC+1, Dima Pasechnik wrote:

everything, as for rebuilding docs nothing short of "make distclean &&
make" works.

sage -sync-build && sage -b && make doc-clean && make doc

almost always works (there has been maybe one case where it didn't, and that was pretty f-ed up).

 

[Volker Braun]

On Friday, May 30, 2014 5:11:08 PM UTC+1, kcrisman wrote:

Right, but the problem is that currently I am uploading it ;-)

Which is exactly why it ought to be difficult to make wrong-way merges in a branch, so that you can't accidentally/misguidedly upload without understanding enough of git to know that it is wrong.

[Volker Braun]

I've implemented a version of this now. From the README:

* Review tickets with minimal recompiling. This assumes that you are

  currently on the "develop" branch, that is, the latest beta. Just

  checking out an older ticket would most likely reset the Sage tree

  to an older version, so you would have to compile older versions of

  packages to make it work. Instead, you can create an anonymous

  ("detached HEAD") merge of the ticket and the develop branch::

      $ git trac try 12345

  This will only touch files that are really modified by the

  ticket. In particular, if only Python files are changed by the

  ticket (which is true for most tickets) then you just have to run

  `sage -b` to rebuild the Sage library. When you are finished

  reviewing, just checkout a named branch. For example::

      $ git checkout develop

     

  If you want to edit the ticket branch (that is, add additional

  commits) you cannot use `git trac try`. You must use `git trac

  checkout` to get the actual ticket branch as a starting point.

[Nathann Cohen]

Yo !

> If you want to edit the ticket branch (that is, add additional
> commits) you cannot use `git trac try`. You must use `git trac
> checkout` to get the actual ticket branch as a starting point.

Why can't you add commits after a merge ? I do this all the time O_o

Nathann

[Volker Braun]

There is nothing wrong with merges 

* if you need them

* if the original branch is the first parent. 

Doing it the wrong way makes the history more difficult to understand, and we shouldn't teach or facilitate that antipattern.

[Nathann Cohen]

> There is nothing wrong with merges
>
> * if you need them
> * if the original branch is the first parent.
>
> Doing it the wrong way makes the history more difficult to understand, and
> we shouldn't teach or facilitate that antipattern.

That's my question from last november :
https://groups.google.com/d/msg/sage-git/Nokq-kVfONc/e9RMBUvl7ygJ

that's your answer from then :
https://groups.google.com/d/msg/sage-git/Nokq-kVfONc/sDoYyZOQHY0J

By the way, given that with this trick I can avoi recompiling Sage for
hours, I still think that it is the right way to work.

Nathann

[Nathann Cohen]

By the way I improved this gtmp method since, so that it prepares a
commit message which makes the history clearer (at least to a human)

function gtmp
{
cd ~/sage
git checkout d

rel="$(git log --oneline d ^d~1 | sed s/.*\ //g)"
tn="$(echo "$1" | grep -o "[0-9]*")"
echo "==========================="
echo "trac #$tn: Merged with $rel"
echo "==========================="

git branch -D tmp
git checkout -b tmp d &&
git pull trac "$1"
}

Example :

~/sage$ gtmp u/ncohen/16347
Switched to branch 'd'
Your branch is up-to-date with 'trac/develop'.
===========================
trac #16347: Merged with 6.3.beta2
===========================
Deleted branch tmp (was 27f6c9a).
Switched to a new branch 'tmp'
From trac.sagemath.org:sage
* branch u/ncohen/16347 -> FETCH_HEAD
....

And then I copy/paste this commit message... As I found no easy way to
make it the default commit message :-)

Nathann

[leif]

Did someone meanwhile track down what exactly went wrong / broke
docbuilding (such that it hangs)?

(I also deleted all temporary folders, $DOT_SAGE, and the Cython cache,
ran 'sage -ba', reinstalled Sphinx and later even Python and all
packages that depend on it... to no avail B) .)

[Volker Braun]

And which part of it being a common assumption among git users that the first merge commit points back to the branch did you not understand?

[Volker Braun]

On Saturday, May 31, 2014 3:14:41 PM UTC+1, leif wrote:

Did someone meanwhile track down what exactly went wrong / broke
docbuilding (such that it hangs)?

I can't reproduce it, works for me.

[Dima Pasechnik]

Volker,
feel free to borrow my apparently totally f-ed up laptop, as on it
this just doesn't work for months, when I am
back from Philly, and have a look...

Cheers,
Dima

[john_perry_usm]

I hope no one minds my resurrecting this, but after having spent a lot of time waiting for Sage to rebuild itself 3 or 4 times, I just discovered this hint, and would like to suggest it make its way onto http://www.sagemath.org/doc/developer/trac.html#section-review-patches, and all other similar pages in the developer guide.

If that's already in the works, then never mind me; just carry on...

john perry

[kcrisman]

I hope no one minds my resurrecting this, but after having spent a lot of time waiting for Sage to rebuild itself 3 or 4 times, I just discovered this hint, and would like to suggest it make its way onto http://www.sagemath.org/doc/developer/trac.html#section-review-patches, and all other similar pages in the developer guide.

If that's already in the works, then never mind me; just carry on...

john perry

On Saturday, May 31, 2014 8:33:14 AM UTC-5, Volker Braun wrote:

I've implemented a version of this now. From the README:

* Review tickets with minimal recompiling. This assumes that you are

  currently on the "develop" branch, that is, the latest beta. Just

  checking out an older ticket would most likely reset the Sage tree

  to an older version, so you would have to compile older versions of

  packages to make it work. Instead, you can create an anonymous

  ("detached HEAD") merge of the ticket and the develop branch::

      $ git trac try 12345

Volker, is this in the current git-trac? 

[Volker Braun]

yes it is...

[kcrisman]

On Wednesday, August 27, 2014 8:33:24 AM UTC-4, Volker Braun wrote:

yes it is...

But it's not even in http://www.sagemath.org/doc/developer/git_trac.html

Presumably the development of git trac should be pretty tightly connected to that document.  We keep telling people to read it, after all.

[Volker Braun]

You don't need to know about that particular subcommand to develop, its just a bandaid until we have a better build system. There are plenty other git concepts and subcommands that can definitely be useful in certain circumstances, but aren't really necessary to work on a ticket. 

[Robert Bradshaw]

Replying to an old thread I realized I'd starred...

On Thu, May 29, 2014 at 6:48 PM, kcrisman <kcri...@gmail.com> wrote:
> Well, I am thankful that my rant led to some pretty substantive discussion.
> Here let me summarize some thoughts.
>
> A) +1 to having something where a github-like editing thing works. I've
> used this at least once with matplotlib. My beef with github is orthogonal
> to the web interface piece. If there is a good workflow where we can direct
> contributors to make smaller edits on the github mirror and they'll be taken
> care of, that's great.

All pull requests on github turn into tickets, so this easy web
interface flow works now.

> B) I didn't realize that the "release manager" commits were separate from
> the "author" ones. That is strange to me, but I guess it's there.
> Nonetheless, I think that the vast majority of ticket descriptions are (or
> at least start as) fairly half-baked notions of what could be added or
> fixed, and are entirely inappropriate for a commit message to siphon through
> in "git log". Especially since they are so long compared to most commit
> messages (and not for good reasons).

This is an awareness issue--ticket descriptions should be well written
descriptions of what's in the ticket, and now there's further
motivation to do so as they're also stored in the git log when the
ticket in question is closed.

> C) I haven't heard any -1's to "Merged in" being resuscitated, so maybe that
> should really happen.
>
> [More thoughts: Robert B: I think Travis' point about doing it near the
> release time is so that people know where to start from. My whole trouble
> earlier today ended up being because, since it said closed, I assumed the
> Maxima upgrade HAD to be in the latest beta - which it wasn't. And several
> "git pull" attempts in my develop branch did nothing, since develop is still
> at 6.3.beta2 as of this writing (on github at least). As it turned out, it
> wasn't based on the right ticket, so I guess that could have been "needs
> work", but it *says* "Depends on #13973" so in the old workflow that should
> have been enough (install #13973, then install this one). I feel like this
> is closely related.]

You can still have "depends on." The nice thing is that if you merge a
depends on that's already in it's a no-op.

> D) Patchbot? What patchbot? At least the colorings aren't working right
> now, it seems. But I would love to trust that again and not have to rebuild
> trivial things.

Just checked, it's still up. There may eventually be a replacement,
but if you want tickets to get updated faster, run a patchbot client
on your own machine. (You can even configure it to test your
tickets/favorite category first.)

> E) Please everyone let's put Trac #s back in the commit messages, then, if
> the immutability is a barrier there. Some of the most recent ones are
> completely mystifying. And then it will be possible to figure out where the
> new code IS in the tree, as opposed to just the merges which can be
> patchbombs.
>

> --
> You received this message because you are subscribed to the Google Groups
> "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sage-devel+...@googlegroups.com.
> To post to this group, send email to sage-...@googlegroups.com.
> Visit this group at http://groups.google.com/group/sage-devel.
> For more options, visit https://groups.google.com/d/optout.