Mass bug filing: dpkg-buildpackage -A

[Santiago Vila]

Greetings.

I've noticed that some packages fail to build from source when built
using "dpkg-buildpackage -A".

This is not only a violation of policy, but also prevents the package
from being uploaded in source-only form, as the architecture-independent
packages are generated by a dedicated "Arch: all" autobuilder which
does exactly "dpkg-buildpackage -A" and nothing else.

[ The problem with source-only uploads is aggravated by this bug in
ftp.debian.org:

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=798413

which apparently it's only a "normal" bug ].

As the number of bugs is going to be greater than ten, I hereby
announce my intent to report as many bugs of this type as I found.

I think those are clearly bugs, and they should be reported.
The only thing that may be unclear is the severity. I would like this
to be RC in some not too distant future (to be on par with ordinary
"dpkg-buildpackage"), but for now I think "severity: important" would
be appropriate here.

Thanks.

[Jakub Wilk]

* Santiago Vila <san...@unex.es>, 2015-11-23, 10:52:

>I've noticed that some packages fail to build from source when built
>using "dpkg-buildpackage -A".

[...]

>I think those are clearly bugs, and they should be reported. The only
>thing that may be unclear is the severity. I would like this to be RC
>in some not too distant future (to be on par with ordinary
>"dpkg-buildpackage"), but for now I think "severity: important" would
>be appropriate here.

I'd use "severity: serious", just like for a normal FTBFS.

--
Jakub Wilk

[Santiago Vila]

Ok, I was in doubt between "normal" and "important", so your email
tells me that I should probably forget about "normal".

The problem with making them "severity: serious" is that I would be
deciding on my own that those bugs are RC. Normally, the release team
decides about what bugs are RC and which ones are not.

Maybe later (but still during this release cycle) when we can be sure
that the number of packages is low enough, we can consider them to be
"severity: serious". The bugs will be already reported and it would be
just a matter of changing the severity.

Thanks.

[Thomas Goirand]

On 11/23/2015 10:52 AM, Santiago Vila wrote:
> Greetings.
>
> I've noticed that some packages fail to build from source when built
> using "dpkg-buildpackage -A".
>
> This is not only a violation of policy, but also prevents the package
> from being uploaded in source-only form, as the architecture-independent
> packages are generated by a dedicated "Arch: all" autobuilder which
> does exactly "dpkg-buildpackage -A" and nothing else.
>
> [ The problem with source-only uploads is aggravated by this bug in
> ftp.debian.org:
>
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=798413
>
> which apparently it's only a "normal" bug ].
>

Hi Santiago,

Thanks for working on this.

> As the number of bugs is going to be greater than ten

Do you have the exact figures? How many packages are affected?

Cheers,

Thomas Goirand (zigo)

[Santiago Vila]

On Mon, Nov 23, 2015 at 05:51:44PM +0100, Thomas Goirand wrote:
> > As the number of bugs is going to be greater than ten
>
> Do you have the exact figures? How many packages are affected?

I don't have exact figures yet, but I estimate about 300, more or less.

I considered the source packages generating at least one "Architecture: amd64"
package and at least one "Architecture: all" package. There are about 3100
of those in stretch/main, and the failure rate I'm getting is about 10%.

[Ben Finney]

Santiago Vila <san...@unex.es> writes:

> On Mon, Nov 23, 2015 at 11:29:19AM +0100, Jakub Wilk wrote:
> > I'd use "severity: serious", just like for a normal FTBFS.
>

> The problem with making them "severity: serious" is that I would be
> deciding on my own that those bugs are RC. Normally, the release team
> decides about what bugs are RC and which ones are not.

The release team decides what bugs they consider to be release-critical.
The severity of the bug is one criterion that they can use.

The determination of what severity to assign the bug, though, is a
function of the effects of the bug behaviour. Its value should be set
truthfully, independent of whether or not the result of a severity level
would affect the release.

You make a determination of a bug's severity by the definitions of each
severity level [0]: if a bug's behaviour best meets the definition of
“serious” the correct value is “Severity: serious”.

If “in the package maintainer's or release manager's opinion, [the bug]
makes the package unsuitable for release”, the bug is at least “serious”
severity.

If it's a “severe violation of Debian policy”, the bug is at least
“serious” severity.

Either of those are sufficient for a reporter to truthfully set the bug
report as “Severity: serious” or higher. If you determine that either of
the above definitions describe the bug you're reporting, IMO you should
truthfully set that value for the bug report.

It's still up to the release team whether the bug is critical for the
release. That's not an attribute you get to decide, and IMO you are not
making the decision for them by setting the severity of a bug report.


[0] <URL:https://www.debian.org/Bugs/Developer#severities>

--
\ “It is well to remember that the entire universe, with one |
`\ trifling exception, is composed of others.” —John Andrew Holmes |
_o__) |
Ben Finney

[Julien Cristau]

On Tue, Nov 24, 2015 at 06:14:43 +1100, Ben Finney wrote:

> If it's a “severe violation of Debian policy”, the bug is at least
> “serious” severity.
>

The release team's RC policy decides which policy violations we consider
"severe" in the sense of "gets a serious severity bug".

Cheers,
Julien

[Josh Triplett]

Santiago Vila wrote:
> I've noticed that some packages fail to build from source when built
> using "dpkg-buildpackage -A".
>
> This is not only a violation of policy, but also prevents the package
> from being uploaded in source-only form, as the architecture-independent
> packages are generated by a dedicated "Arch: all" autobuilder which
> does exactly "dpkg-buildpackage -A" and nothing else.

Thanks for checking this and for filing bugs on this.

> As the number of bugs is going to be greater than ten, I hereby
> announce my intent to report as many bugs of this type as I found.

Please do.

> I think those are clearly bugs, and they should be reported.
> The only thing that may be unclear is the severity. I would like this
> to be RC in some not too distant future (to be on par with ordinary
> "dpkg-buildpackage"), but for now I think "severity: important" would
> be appropriate here.

These bugs represent policy violations; by definition, policy violations
default to "Severity: serious".

I would suggest filing these bugs as "serious", but also explicitly
tagging the versions affected, as the bugs almost certainly affect the
versions in testing as well. As long as you tag the affected versions,
the bugs won't actually prevent migration to testing.

- Josh Triplett

[Ben Finney]

Julien Cristau <jcri...@debian.org> writes:

> On Tue, Nov 24, 2015 at 06:14:43 +1100, Ben Finney wrote:
>
> > If it's a “severe violation of Debian policy”, the bug is at least
> > “serious” severity.

This is one way that a bug can be determined “Severity: serious”, by
definition.

In other words: “severe violation of Debian policy” implies “bug is
Severity: serious”.

> The release team's RC policy decides which policy violations we
> consider "severe" in the sense of "gets a serious severity bug".

And this is another, by definition.

In other words: “release team determined the bug is RC” implies “bug is
Severity: serious”.

The reverse implication is not true: one cannot infer from “Severity:
serious” that the release team made that determination. The “Severity”
field doesn't *only* mean that, so it's not a valid inference.

By my reading of the “serious” severity definition, any of its “or”
clauses makes for sufficient justification to set “Severity: serious”
without considering the other clauses.

--
\ “If this is your first visit to the USSR, you are welcome to |
`\ it.” —hotel room, Moscow |
_o__) |
Ben Finney

[Adam D. Barratt]

On Tue, 2015-11-24 at 08:48 +1100, Ben Finney wrote:
> Julien Cristau <jcri...@debian.org> writes:
>
> > On Tue, Nov 24, 2015 at 06:14:43 +1100, Ben Finney wrote:
> >
> > > If it's a “severe violation of Debian policy”, the bug is at least
> > > “serious” severity.
>
> This is one way that a bug can be determined “Severity: serious”, by
> definition.
>
> In other words: “severe violation of Debian policy” implies “bug is
> Severity: serious”.
>
> > The release team's RC policy decides which policy violations we
> > consider "severe" in the sense of "gets a serious severity bug".
>
> And this is another, by definition.

I think you may have missed Julien's point - there is no specific
definition of "severe violation of Debian policy" that is distinct from
"is listed in the Release Team's RC policy".

Neither policy nor the BTS defines exactly what is meant by "severe
violation". Policy indicates that violations are "roughly equivalent" to
particular bug severities and https://www.debian.org/Bugs/Developer.html
similarly uses "roughly".

The BTS used to have a more precise definition, most recently by
explicitly referring to the separate RC policy - see
http://anonscm.debian.org/viewvc/webwml/webwml/english/Bugs/Developer.wml?r1=1.46&r2=1.47 and #695531.

(Whether there should be such a distinction is something on which there
are differing opinions; that's also subtly different from whether there
is one.)

Regards,

Adam

[Santiago Vila]

Indeed, this is how I thought the severities were being used. Not in
theory but in practice.

Please don't worry about severities. I will try to be "nice" and will
report them as "important", as there are a lot of affected packages.

But will use some kind of usertags so that release managers are able
to change severities easily. I would love to see this as a release
goal for stretch, but I think it's soon to decide about that.

Thanks.

[Julien Cristau]

Sounds great, thanks!

Cheers,
Julien

[Santiago Vila]

Hi.

I will put all the bugs that I report about this issue here:

https://bugs.debian.org/cgi-bin/pkgreport.cgi?tag=binary-indep;users=san...@debian.org

So far there are one that I reported 19 days ago as a "test report"
and 162 that I reported today. I'm intentionally excluding packages
having a different version in stretch and sid, since I'm only building
packages in stretch (too many FTBFS even without using -A, fortunately
Chris Lamb and my other reproducible builds friends usually report
those before they reach testing :-).

There are already quite a few fixed packages, which is great.

Will continue reporting in the next days/weeks/months, depending on my
free time.

Thanks a lot.