optional package doctests badly broken

[Vincent Delecroix]

Dear all,

Probably because of #20382 a lot of optional doctests now do not pass
anymore... I found very annoying that very often tickets break
optional packages. Does anybody has ideas to make that happen less often?

Best
Vincent

[Thierry]

Hi,

Run more patchbots, with various combinations of optional packages
included.

Ciao,
Thierry

>
> Best
> Vincent
>
> --
> You received this message because you are subscribed to the Google Groups "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
> To post to this group, send email to sage-...@googlegroups.com.
> Visit this group at https://groups.google.com/group/sage-devel.
> For more options, visit https://groups.google.com/d/optout.

[John Cremona]

On 10 May 2018 at 16:20, Thierry <sage-goo...@lma.metelu.net> wrote:

Hi,

On Thu, May 10, 2018 at 04:41:39PM +0200, Vincent Delecroix wrote:
> Dear all,
>
> Probably because of #20382 a lot of optional doctests now do not pass
> anymore... I found very annoying that very often tickets break
> optional packages. Does anybody has ideas to make that happen less often?

Run more patchbots, with various combinations of optional packages
included.

sage:  2 ** sum([len(L) for L in optional_packages()])

9671406556917033397649408

 

Ciao,
Thierry

>
> Best
> Vincent
>
> --
> You received this message because you are subscribed to the Google Groups "sage-devel" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscribe@googlegroups.com.

> To post to this group, send email to sage-...@googlegroups.com.
> Visit this group at https://groups.google.com/group/sage-devel.
> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "sage-devel" group.

To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscribe@googlegroups.com.

[Thierry]

On Thu, May 10, 2018 at 04:35:36PM +0100, John Cremona wrote:
> On 10 May 2018 at 16:20, Thierry <sage-goo...@lma.metelu.net> wrote:
>
> > Hi,
> >
> > On Thu, May 10, 2018 at 04:41:39PM +0200, Vincent Delecroix wrote:
> > > Dear all,
> > >
> > > Probably because of #20382 a lot of optional doctests now do not pass
> > > anymore... I found very annoying that very often tickets break
> > > optional packages. Does anybody has ideas to make that happen less often?
> >
> > Run more patchbots, with various combinations of optional packages
> > included.
> >
>
> sage: 2 ** sum([len(L) for L in optional_packages()])
>
> 9671406556917033397649408

Indeed, but various is not all, it is clear that some optional packages
do not interct, so the set of interesting possibilities should not be so
big.

If we could maintain one patchbot with all optional packages installed for
both 64 and 32 bits, that would be a good start. Then add some somplexity
with various MILP solvers (only one is chosen when many are installed),
including commercial ones. And so on.

Ciao,
Thierry

>
>
>
>
>
> >
> > Ciao,
> > Thierry
> >
> > >
> > > Best
> > > Vincent
> > >
> > > --
> > > You received this message because you are subscribed to the Google
> > Groups "sage-devel" group.
> > > To unsubscribe from this group and stop receiving emails from it, send

> > an email to sage-devel+...@googlegroups.com.

> > > To post to this group, send email to sage-...@googlegroups.com.
> > > Visit this group at https://groups.google.com/group/sage-devel.
> > > For more options, visit https://groups.google.com/d/optout.
> >
> > --
> > You received this message because you are subscribed to the Google Groups
> > "sage-devel" group.
> > To unsubscribe from this group and stop receiving emails from it, send an

> > email to sage-devel+...@googlegroups.com.

> > To post to this group, send email to sage-...@googlegroups.com.
> > Visit this group at https://groups.google.com/group/sage-devel.
> > For more options, visit https://groups.google.com/d/optout.
> >
>
> --
> You received this message because you are subscribed to the Google Groups "sage-devel" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.

[Jeroen Demeyer]

On 2018-05-10 17:20, Thierry wrote:
> Run more patchbots, with various combinations of optional packages
> included.

-1

That would more the problem more visible at best, but it wouldn't
*solve* anything.

[Martin R]

Taken to the extreme, we should get rid of doctests altogether, since they don't solve anything :-)

Seriously, at least the FriCAS interface was broken several times because of trivial modifications.  If there would have been a patchbot running the FriCAS doctests back then, this would have been visible, as you say, and would have been fixed immediately instead of after several weeks.

Martin

[Dima Pasechnik]

this would at least prevent tickets like #20382 from getting positive reviews...

 

 

[Jeroen Demeyer]

On 2018-05-10 20:53, Dima Pasechnik wrote:
> this would at least prevent tickets like #20382 from getting positive
> reviews...

Some of the test failures were actually reported by patchbots, see for
example:

https://patchbot.sagemath.org/log/20382/Ubuntu/16.04/x86_64/4.4.0-119-generic/quasar/2018-05-01%2003:59:58

So unfortunately your statement is false.

[Jeroen Demeyer]

On 2018-05-10 20:40, 'Martin R' via sage-devel wrote:
> If there would have been a patchbot running the FriCAS doctests back then

There most likely was such a patchbot and it didn't help.

[Dima Pasechnik]

Of course, if the author/reviewer ignores patchbots, it's hopeless.

Should we introduce monetary fines for such SNAFUs? ;-)

 

[Martin R]

As far as I remember, no. But I may be mistaken.

[Simon King]

-1: When a change of SageMath results in breaking an optional package's tests,
then this is a bug in SageMath and releasing bugs should be avoided. Making a
bug visible and thus preventing it from being released does solve a problem
(for some definition of of "solving" at least).

I guess that most optional packages do not interfere with each other.
Hence, a single patchbot that installs, tests, and builds documentation
of *all* optional packages at once, would probably be very valuable.

Alternatively or in addition, we might return to the old idea that each
package has a maintainer. So, when a new release takes shape (but before
an official release takes place), the release manager may notify the
maintainers (by an automated mail) and ask them to test their package with
the current beta. If one of the maintainers reports that his/her package
got broken, it will be a critical bug or blocker for the SageMath release.

I reckon it wouldn't be much additional work for the release manager, as
the testing is assigned to the package maintainers. It would probably be
doable for the package maintainers, as it is just running one test every
now and then (we currently do not do releases every fortnight, do we?).
And after all it would be in the interest of the package maintainers if
SageMath wouldn't break their optional package.

Best regards,
Simon

[Jeroen Demeyer]

On 2018-05-11 01:51, Simon King wrote:
> Making a bug visible and thus preventing it from being released

You're making the same mistake as other people in this thread: the
implication in this sentence does not hold (or at least, to a much
smaller extent than one would hope).

[Erik Bray]

Considering that optional packages are *supposed* to work, I'm
surprised we don't have at least one buildbot that tests with all
optional packages installed, or at least the ones that don't depend on
proprietary systems (though for one buildbot even that should be
doable--but who pays for it?)

[William Stein]

Me.

I have a machine at the UW that can (in fact should) be used for this...

--
You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
To post to this group, send email to sage-...@googlegroups.com.
Visit this group at https://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

--

-- William Stein

[John Cremona]

On 11 May 2018 at 16:11, William Stein <wst...@gmail.com> wrote:

On Fri, May 11, 2018 at 7:37 AM Erik Bray <erik....@gmail.com> wrote:

On Thu, May 10, 2018 at 7:37 PM, Jeroen Demeyer <J.De...@ugent.be> wrote:
> On 2018-05-10 17:20, Thierry wrote:
>>
>> Run more patchbots, with various combinations of optional packages
>> included.
>
>
> -1
>
> That would more the problem more visible at best, but it wouldn't *solve*
> anything.

Considering that optional packages are *supposed* to work, I'm
surprised we don't have at least one buildbot that tests with all
optional packages installed, or at least the ones that don't depend on
proprietary systems (though for one buildbot even that should be
doable--but who pays for it?)

Me.

I have a machine at the UW that can (in fact should) be used for this...

The patchbot David Loeffler and I run in Warwick is on a machine which has Magma on it.

John

 

--
You received this message because you are subscribed to the Google Groups "sage-devel" group.

To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscribe@googlegroups.com.

To post to this group, send email to sage-...@googlegroups.com.
Visit this group at https://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

--

-- William Stein

--

You received this message because you are subscribed to the Google Groups "sage-devel" group.

To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscribe@googlegroups.com.

[Thierry]

On Fri, May 11, 2018 at 05:17:16PM +0100, John Cremona wrote:
> On 11 May 2018 at 16:11, William Stein <wst...@gmail.com> wrote:
>
> >
> > On Fri, May 11, 2018 at 7:37 AM Erik Bray <erik....@gmail.com> wrote:
> >
> >> On Thu, May 10, 2018 at 7:37 PM, Jeroen Demeyer <J.De...@ugent.be>
> >> wrote:
> >> > On 2018-05-10 17:20, Thierry wrote:
> >> >>
> >> >> Run more patchbots, with various combinations of optional packages
> >> >> included.
> >> >
> >> >
> >> > -1
> >> >
> >> > That would more the problem more visible at best, but it wouldn't
> >> *solve*
> >> > anything.
> >>
> >> Considering that optional packages are *supposed* to work, I'm
> >> surprised we don't have at least one buildbot that tests with all
> >> optional packages installed, or at least the ones that don't depend on
> >> proprietary systems (though for one buildbot even that should be
> >> doable--but who pays for it?)
> >

Note that regarding CPLEX and Gurobi (MILP solvers that sometimes create
doctest issues), there are free academic licenses.

Ciao,
Thierry

> >
> > Me.
> >
> > I have a machine at the UW that can (in fact should) be used for this...
> >
>
> The patchbot David Loeffler and I run in Warwick is on a machine which has
> Magma on it.
>
> John
>
>
> >
> >>
> >> --
> >> You received this message because you are subscribed to the Google Groups
> >> "sage-devel" group.
> >> To unsubscribe from this group and stop receiving emails from it, send an

> >> email to sage-devel+...@googlegroups.com.

> >> To post to this group, send email to sage-...@googlegroups.com.
> >> Visit this group at https://groups.google.com/group/sage-devel.
> >> For more options, visit https://groups.google.com/d/optout.
> >>
> > --
> > -- William Stein
> >
> > --
> > You received this message because you are subscribed to the Google Groups
> > "sage-devel" group.
> > To unsubscribe from this group and stop receiving emails from it, send an

> > email to sage-devel+...@googlegroups.com.

> > To post to this group, send email to sage-...@googlegroups.com.
> > Visit this group at https://groups.google.com/group/sage-devel.
> > For more options, visit https://groups.google.com/d/optout.
> >
>
> --

> You received this message because you are subscribed to the Google Groups "sage-devel" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.

[Erik Bray]

On Fri, May 11, 2018 at 5:11 PM, William Stein <wst...@gmail.com> wrote:
>
> On Fri, May 11, 2018 at 7:37 AM Erik Bray <erik....@gmail.com> wrote:
>>
>> On Thu, May 10, 2018 at 7:37 PM, Jeroen Demeyer <J.De...@ugent.be>
>> wrote:
>> > On 2018-05-10 17:20, Thierry wrote:
>> >>
>> >> Run more patchbots, with various combinations of optional packages
>> >> included.
>> >
>> >
>> > -1
>> >
>> > That would more the problem more visible at best, but it wouldn't
>> > *solve*
>> > anything.
>>
>> Considering that optional packages are *supposed* to work, I'm
>> surprised we don't have at least one buildbot that tests with all
>> optional packages installed, or at least the ones that don't depend on
>> proprietary systems (though for one buildbot even that should be
>> doable--but who pays for it?)
>
>
> Me.
>
> I have a machine at the UW that can (in fact should) be used for this...

Hi William--

Could you and Volker get in touch about setting this up? I would help
too if I can but I don't really have any command over Sage's buildbot
infrastructure, so it would probably be best if you two interfaced
directly on that...

Thanks,
E

[David Loeffler]

On 11 May 2018 at 17:17, John Cremona <john.c...@gmail.com> wrote:

The patchbot David Loeffler and I run in Warwick is on a machine which has Magma on it.

John

I'd be grateful if someone could explain how to configure the patchbot to run the Magma optional doctests -- I couldn't find this in the patchbot instructions.

David