Possible patchbot(s) problem ?

[Emmanuel Charpentier]

I have set Trac#24969 to "needs_review" yesterday ;this morning, I saw that no patchbot tried to check it. I kicked it.

To date, no sign of activity.

Is that normal ? I was expecting to see this ticket tested by at least a couple of parchbots... OTOH, I have no idea of the current queue(s) for patchbot attention, nor on the scheduling policies.

FWIW :

• This is your routine vanilla upgrade to R (3.4.4 was released yesterday).
  
• The ticket has a pointer to the upstream source tarball
  
• The branch has the "right" content.
  
• On my machine, passes ptestlong with no failures, and sort-of passes its own test suite.
  

--

Emmanuel Charpentier

[Thierry]

Hi,


On Fri, Mar 16, 2018 at 02:35:01AM -0700, Emmanuel Charpentier wrote:
> I have set Trac#24969 <https://trac.sagemath.org/ticket/24969> to

> "needs_review" yesterday ;this morning, I saw that no patchbot tried to
> check it. I kicked it.
>
> To date, no sign of activity.
>
> Is that normal ? I was expecting to see this ticket tested by at least a
> couple of parchbots... OTOH, I have no idea of the current queue(s) for
> patchbot attention, nor on the scheduling policies.

I do not know if this should be considered as "normal", but the
explanation is that no patchbot succeeds to test the 8.2.beta8 because of
a failing doctest in src/sage/interacts/test_jupyter.rst see
https://patchbot.sagemath.org/ticket/0/

Hence no patchbot can test any ticket, otherwise they will report an error
on each ticket due to that unrelated failing doctest.

This will be fixed in the next beta release, see #24918 (imho, this
patchblot lock is a sufficient condition to trigger an new beta, even with
very few tickets merged).

Ciao,
Thierry



> FWIW :
>
> - This is your routine vanilla upgrade to R (3.4.4 was released
> yesterday).
> - The ticket has a pointer to the upstream source tarball
> - The branch has the "right" content.
> - On my machine, passes ptestlong with no failures, and sort-of passes

> its own test suite.
>
>
> --
> Emmanuel Charpentier
>

> --
> You received this message because you are subscribed to the Google Groups "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
> To post to this group, send email to sage-...@googlegroups.com.
> Visit this group at https://groups.google.com/group/sage-devel.
> For more options, visit https://groups.google.com/d/optout.

[Emmanuel Charpentier]

Thank you, Thierry. ow I understand...

A possible coping mechanism would be to have a 8.2.beta8.1 release containing *exactly* what is needed to allow the patchbots to work (possibly disabling the Jupyter test...), *then* restart from there.

--

Emmanuel Charpentier

[Frédéric Chapoton]

you must fill the Author field..

[David Loeffler]

Hence no patchbot can test any ticket, otherwise they will report an error
on each ticket due to that unrelated failing doctest.

This is not quite true. The correct statement is that no patchbot can *start* testing tickets. When the patchbot is started, it tests whether the latest Sage beta ("ticket 0") passes doctests, and aborts if not. But running patchbots automatically upgrade themselves to each new beta, and do not repeat the "ticket 0" check after upgrade. So any patchbot that hasn't been rebooted since 8.2.beta8 came out will still be running along merrily, and reporting failures on every single ticket it tests, which is pretty useless if you ask me.

IMHO, the patchbot should be reconfigured so that it re-tests ticket 0 after every upgrade, and if that test fails, the patchbot should stop -- and ideally auto-generate an email (to its maintainer, or perhaps even to the sage-release list) reporting the problem.

Best, David

--

You received this message because you are subscribed to the Google Groups "sage-devel" group.

To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscribe@googlegroups.com.

[Jeroen Demeyer]

On 2018-03-16 12:40, David Loeffler wrote:
> IMHO, the patchbot should be reconfigured so that it re-tests ticket 0
> after every upgrade, and if that test fails, the patchbot should stop

Absolutely not. I agree that the current patchbot situation is not good,
but what you propose is much worse.

[David Loeffler]

Why is this "much worse"? Surely it's better for the patchbot to stop than for it to keep going and produce completely useless and misleading output? I emphasise that I'm *not* proposing that it *silently* stop, but rather that it report the fact that ticket 0 fails in some sensible way, rather than spamming a whole load of unrelated tickets with spurious failure reports. This issue has been discussed on this list before, and several people agreed that the patchbot should run tests on each new beta and halt if they don't pass, with Jeroen being the only person to defend the current system. 

For an analogy: imagine I am in my university office, marking a big pile of examination scripts, and suddenly I find that I can't go on, because my desk chair has collapsed on the floor and I can no longer see the papers on my desk. What shall I do? Should I

(a) stop and report to maintenance that I need a new chair;

(b) scrawl "FAIL" across all the remaining exam scripts (because I can't see them well enough to verify that they're correct), and hope somebody notices that something is amiss and brings me a new chair?

I'm pretty sure that option (b) would get me fired rather quickly; and, of course, (b) is exactly isomorphic to the patchbot's current reaction when a new beta won't pass tests.

The signal-to-noise ratio of the patchbot is dreadful right now; for instance, on #24086 -- a rather straightforward bug-fix in the modular forms code -- the last twelve patchbot runs, on five different machines, have all reported failures, and every single one of these is totally unrelated to the ticket. At this stage the patchbot is worse than useless (strictly worse, because it creates uncertainty among ticket reviewers and thus prevents perfectly good code from getting into Sage). 

David

[Dima Pasechnik]

option (c): strike!
:-)

[Nicolas M. Thiery]

> option (c): strike!
> :-)

Wait, wait, wait. Going on strike is a French thing! We won't let
brexit get away with stealing away our dear socialist traditions!

:-)
Nicolas
--
Nicolas M. Thiéry "Isil" <nth...@users.sf.net>
http://Nicolas.Thiery.name/

[Dima Pasechnik]

On Thursday, March 22, 2018 at 10:51:16 PM UTC, Nicolas M. Thiéry wrote:

> option (c): strike!
> :-)

Wait, wait, wait. Going on strike is a French thing! We won't let
brexit get away with stealing away our dear socialist traditions!

FYI, UK universities academics have been on strike in the past 2 months for quite a bit, due to a pension fund dispute, and it's far, far from over yet...