Bug#976811: transition: php8.0

[Ondřej Surý]

Package: release.debian.org
Severity: normal
User: release.d...@packages.debian.org
Usertags: transition

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Hi,

I would like to transition the PHP to version 8.0; it's not such a huge bump as
it was with 5.6 -> 7.0 and most of the packages that were compatible with PHP
7.4 are working just fine with PHP 8.0.

I do have most of the extensions ready for PHP 8.0 either via new upstream version
or with patches.

Ondrej

Ben file:

title = "php8.0";
is_affected = .depends ~ "phpapi-20190902" | .depends ~ "phpapi-20200930";
is_good = .depends ~ "phpapi-20200930";
is_bad = .depends ~ "phpapi-20190902";


- -- System Information:
Debian Release: 10.7
APT prefers stable-updates
APT policy: (500, 'stable-updates'), (500, 'stable')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 4.19.0-12-amd64 (SMP w/8 CPU cores)
Kernel taint flags: TAINT_OOT_MODULE, TAINT_UNSIGNED_MODULE
Locale: LANG=en_IE.UTF-8, LC_CTYPE=en_IE.UTF-8 (charmap=UTF-8), LANGUAGE=en_IE:en (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

-----BEGIN PGP SIGNATURE-----

iQKTBAEBCgB9FiEEw2Gx4wKVQ+vGJel9g3Kkd++uWcIFAl/POTZfFIAAAAAALgAo
aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldEMz
NjFCMUUzMDI5NTQzRUJDNjI1RTk3RDgzNzJBNDc3RUZBRTU5QzIACgkQg3Kkd++u
WcLQvA//ZgsLvoyZ4q/KCMJwee2MzyDtLBrdkKr435kOsBNiBgS1n68eXiEM6TsX
9B+hIYcq2XDXG600y60IiB2NJ3yQ2MptzO7J56mA5es834s9fM8yoxvqEObxld4z
YCNi/3By9MXiCOFwy2cLuohnTNLvvvPXD0vGreqWl7+nLBCPNaOQtgo5V/fo/1fA
XiTC/0fPlYyAF8I4ZRFtopc4MckOuKdQNoRuSrGNpqYmSWIFMTdnauhQFUK7l5/o
PRcL7uz3WTV7t/yA3cKZ+Q/2d8tp/dCiBNQHZ9DEtJSa/c9kYf6v2UtHSs/9Vg6O
o/Q8Ueprh1145C3kPdM0qmMM6imKKhpi0cZp3SL9OejqxWjias1I4frNbnNQl4/7
CDi914TKq3ILO6fRQ7vFXJMwl3pEogGSIR8abnDgHRiKirmwP3eA/KWfuQti81yE
PzW1VoS1YRXqJKBHQ0QaTxlin9HlzVyR5PsBdIHWXNeD/rnm9L0lloPyFeaUWeBX
PnJk+5qRHKV/DpsH4WGYF3lxyyC45GckBOmw3ZBIsD+u2rEfKd58hqSdsHvkqaBc
lKmO+lsjBjDuBVDpbr8lEmTGyRmbJZuVa+rxsMfdiLvDdeFUfj90fzqquOjxvDLu
i3pLyjmgcMPyrjfQrRd54j31rSE9zfcLHT44ImqYvlOUI4AHI4k=
=IbF+
-----END PGP SIGNATURE-----

[Ondřej Surý]

The main problem is the PHP release schedule: https://www.php.net/supported-versions.php

If we release with PHP 7.4, the upstream security support will end sooner than security support for Debian bullseye. If we release with PHP 8.0 we will have full three years of upstream support.

There’s still month before the transition freeze and we will have time to fix the downstream users after the transition is over.

I think the transition is worth the trouble. Having official security support is important especially for PHP.

Ondřej 

--

Ondřej Surý <ond...@sury.org> (He/Him)

On 11. 12. 2020, at 17:38, David Prévot <taf...@debian.org> wrote:

Hi Ondřej,

Le Tue, Dec 08, 2020 at 09:28:38AM +0100, Ondřej Surý a écrit :

I would like to transition the PHP to version 8.0;

The timing of this request makes me uneasy: php8.0 has been in Debian
for less than a week, and we are a month away from the transition
freeze.

it's not such a huge bump as it was with 5.6 -> 7.0 and

If I remember correctly, the 7.3 -> 7.4 was not a huge bump either, yet
php7.4 packages were in Debian months before the actual transition
happen, and it took months for the updated php-defaults to actually
migrates into testing.

most of the packages that were compatible with PHP

7.4 are working just fine with PHP 8.0.

That does not match my experience as a maintainer of about a hundred
packages relying on PHP. Many upstream are currently releasing updates
to fix compatibility with PHP 8.0, and many more have not yet done so.

I’m actually surprised to discover this request in the BTS without any
prior communication with teams involved in PHP related packaging.

For example, PHPUnit 8 as currently available in unstable and testing is
expected to run on PHP 8 (thanks to upstream updates less than two weeks
ago), yet “Please note that the code coverage functionality is not
available for PHPUnit 8.5 on PHP 8.” (from upstream changelog 8.5.12).
So shipping PHPUnit 8 with PHP 8 would mean having a major
fonctionnality unavailable for the whole Bullseye life cycle. PHPUnit 9
is available from experimental, yet uploading to unstable would mean
having to deal with dozens of breakage (in the FTBFS form):

https://release.debian.org/britney/pseudo-excuses-experimental.html#phpunit

PHPUnit is just one example, but it seems unrealistic to ship version 9
with Bullseye (I really abandonned this option months ago). Other
packages will break (and I suspect the number of breakage will be high).

This kind of disruptive change would hopefully be better suited early in
the release cycle rather than just before the beginning of the freeze.

That said, it would be nice to have an updated php-default in
*experimental* to help have a grasp of the possible brekages.

Regards

David

[David Prévot]

Hi Ondřej,

Le Tue, Dec 08, 2020 at 09:28:38AM +0100, Ondřej Surý a écrit :

> I would like to transition the PHP to version 8.0;

The timing of this request makes me uneasy: php8.0 has been in Debian
for less than a week, and we are a month away from the transition
freeze.

> it's not such a huge bump as it was with 5.6 -> 7.0 and

If I remember correctly, the 7.3 -> 7.4 was not a huge bump either, yet
php7.4 packages were in Debian months before the actual transition
happen, and it took months for the updated php-defaults to actually
migrates into testing.

> most of the packages that were compatible with PHP
> 7.4 are working just fine with PHP 8.0.

[Holger Levsen]

Hi Ondřej,

On Fri, Dec 11, 2020 at 06:01:31PM +0100, Ondřej Surý wrote:
> I think the transition is worth the trouble. Having official security support is important especially for PHP.

as a comment from the sideline...

> > On 11. 12. 2020, at 17:38, David Prévot <taf...@debian.org> wrote:
> > That said, it would be nice to have an updated php-default in
> > *experimental* to help have a grasp of the possible brekages.

you didn't reply to that - IMHO - sensible suggestion from David as
a/the step forward. Maybe you missed it, maybe you disagree, maybe you are
busy preparing those uploads? ;)


--
cheers,
Holger

⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ holger@(debian|reproducible-builds|layer-acht).org
⢿⡄⠘⠷⠚⠋⠀ PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C
⠈⠳⣄

If secure encryption is outlawed, only criminals will have it.

[Ondřej Surý]

Sorry, I was on my phone and missed that. Yes, I can do that.

--

--
Ondřej Surý

[Holger Levsen]

On Fri, Dec 11, 2020 at 07:43:00PM +0100, Ondřej Surý wrote:
> Sorry, I was on my phone and missed that. Yes, I can do that.

:) & thank you!

[Ondřej Surý]

And done, and uploaded.

--
Ondřej Surý <ond...@sury.org> (He/Him)

> On 11. 12. 2020, at 19:52, Holger Levsen <hol...@layer-acht.org> wrote:

[Holger Levsen]

On Fri, Dec 11, 2020 at 07:58:06PM +0100, Ondřej Surý wrote:
> And done, and uploaded.

yay, thank you!

[Ondřej Surý]

And let me restate that it’s not my intent to make anyone’s life hell and
I am willing to help with any package (as usual). I am just trying to do
the most sane thing to do security and maintainer wise.

Ondrej
--
Ondřej Surý (He/Him)
ond...@sury.org

[Holger Levsen]

On Sat, Dec 12, 2020 at 08:05:19AM +0100, Ondřej Surý wrote:
> And let me restate that it’s not my intent to make anyone’s life hell and
> I am willing to help with any package (as usual). I am just trying to do
> the most sane thing to do security and maintainer wise.

oh sure, I never expected anything else, on the contrary. many thanks for
very nicely caring about php! (and other packages too! :)

[David Prévot]

Hi,

Le Fri, Dec 11, 2020 at 12:38:01PM -0400, David Prévot a écrit :
> Le Tue, Dec 08, 2020 at 09:28:38AM +0100, Ondřej Surý a écrit :
>
> > I would like to transition the PHP to version 8.0;
>

> The timing of this request makes me uneasy […]

>
> > it's not such a huge bump as it was with 5.6 -> 7.0 and
>

> [ 7.3 -> 7.4 was not a huge bump but took months to deal with ]

A look at https://php.watch/versions/8.0 (just the size of the page,
compared to https://php.watch/versions/7.4) and a few pages linked from
there helps me disagree to your “not such a huge bump”.

> > most of the packages that were compatible with PHP
> > 7.4 are working just fine with PHP 8.0.
>
> That does not match my experience as a maintainer of about a hundred
> packages relying on PHP. Many upstream are currently releasing updates
> to fix compatibility with PHP 8.0, and many more have not yet done so.

[…]
> uploading [PHPUnit from experimenatal] to unstable would mean

> having to deal with dozens of breakage (in the FTBFS form):
>
> https://release.debian.org/britney/pseudo-excuses-experimental.html#phpunit

I spent my weekend on this list, uploaded about twenty packages that can
now at least pass their test suite with PHPUnit 8 or 9, under PHP 7 or
8, so the list is about half the size it was two days ago. I managed to
deal with some (easy) PHP 8 issues, but I’m stuck on a dozen of the
remaining packages: I’ve updated their repository, but didn’t upload
them yet (because it only fixes the PHPUnit issues, not the PHP 8 ones).

My point of view after actually working on those issues is that there is
a significant number of packages that are not working just fine with PHP
8.0, and require a fair amount of work before that. Some of them being
security sensitive, so just working around visible issues may not be of
the best interest of anyone…

> That said, it would be nice to have an updated php-default in

> *experimental* to help have a grasp of the possible breakages.

Thank you for your quick upload! As expected, the number of packages
failing on CI is significant (a lot more than the PHPUnit related ones),
and may I stress that it only shows packages that do run autopkgtests,
so we are probably missing a lot (and it may take some time to catch
them).

https://release.debian.org/britney/pseudo-excuses-experimental.html#php-defaults

I hear the argument about PHP release schedule, but this schedule is not
new (there is a lot we could have done to prepare for such a huge bump
if you had shared your plans in advance), and the PHP 7.2 to 7.3 update
before Buster freeze was really nothing compared to this PHP 7.4 to 8.0
update.

Don’t get me wrong, I love how PHP 8.0 is way faster in my environment,
but I’m really concerned about the compatibility of the code running on
it (not only as Debian packages, but also as installed by our users).

Regards

David

[Mike Gabriel]

Hi David,

On Di 29 Dez 2020 23:48:59 CET, David Prévot wrote:

> Hi Horde maintainers, Mike,

>
> Le 11/12/2020 à 12:38, David Prévot a écrit :
>
>>> most of the packages that were compatible with PHP
>>> 7.4 are working just fine with PHP 8.0.
>>

>> That does not match my experience as a maintainer of about a hundred
>> packages relying on PHP. Many upstream are currently releasing updates
>> to fix compatibility with PHP 8.0, and many more have not yet done so.

> […]

>> That said, it would be nice to have an updated php-default in

>> *experimental* to help have a grasp of the possible breakages.
>
> https://release.debian.org/britney/pseudo-excuses-experimental.html#php-defaults
>
> That happened, and it looks like almost half of the spotted CI
> issues are related to php-horde-* packages. You may want to
> investigate those issues, and eventually document the one you’re not
> able to solve in a timely manner in (bugs blocking) #976811 if
> relevant.
>
> Regards
>
> David

So, bullseye will be shipped with PHP 8.0? Well then, another
php-horde fixing round then.

I will look into the autopkgtest logs and see into fixing the issues.

Mike
--

mike gabriel aka sunweaver (Debian Developer)
mobile: +49 (1520) 1976 148
landline: +49 (4351) 486 14 27

GnuPG Fingerprint: 9BFB AEE8 6C0A A5FF BF22 0782 9AF4 6B30 2577 1B31
mail: sunw...@debian.org, http://sunweavers.net

[David Prévot]

Hi Mike,

Le 30/12/2020 à 04:03, Mike Gabriel a écrit :

> So, bullseye will be shipped with PHP 8.0?

That’s the maintainer preference. The release team may not proceed
according to their doubts during the last meeting [1]. I believe the
related issues are worth investigating anyway: easy fixex are useful to
reduce the work around the transition even if it happens after Bullseye
is released (and will be useful for our users using PHP 8.0 from
backports or the maintainer’s repository [2]). If something can’t easily
be fixed, at least being documented on (or as blocking bugs against)
#976811 may help the release team and other players involved get a grasp
of the work to do.

1:
http://meetbot.debian.net/debian-release/2020/debian-release.2020-12-23-18.58.log.html
2: https://deb.sury.org/

Regards

David

[Ondřej Surý]

I guess we are going to release with PHP 7.4 then.  Not the ideal choice, but I will do whatever I can to support it.

However, I would like to get an official statement from the release team what's their position, so we can go forward. Pretty please?

Also an advice on how to proceed next time this happens. The timing of our freeze and new PHP release coincidentally goes together. Should I start the transition with alpha/beta/rc instead of the first final release?

Ondrej

[Paul Gevers]

Hi Ondřej,

On 12-01-2021 12:40, Ondřej Surý wrote:
> I guess we are going to release with PHP 7.4 then.  Not the ideal
> choice, but I will do whatever I can to support it.

That's correct.

> However, I would like to get an official statement from the release team
> what's their position, so we can go forward. Pretty please?

The transition freeze has started, so we'll not transition to php8.0 in
bullseye.

> Also an advice on how to proceed next time this happens. The timing of
> our freeze and new PHP release coincidentally goes together. Should I
> start the transition with alpha/beta/rc instead of the first final release?

For next time I advice:

1) Most importantly: communicate your plans earlier to the Release Team
(and relevant other Teams; the fact that you surprised an active PHP
maintainer wasn't a good sign to us).

2) Working with alpha/beta/rc can start early in experimental,
especially as, like in this case, there is considerable fall out. The
fall out needs to be identified (with rebuilds and autopkgtests (see 3)
and should ideally be fixed before the transition start, but at least
bugs need to be filed, ideally with patches or update instructions.
Experimental is meant for this. In the end, it's your call as a
maintainer if you request to start the transition already with alpha,
beta or rc releases.

3) Adding a new php version to experimental apparently only really shows
it's fall out if also php-defaults is updated, I recommend to do that
early too, as the experimental pseudo excuses will also tell you how
autopkgtests are faring with that change, which is extremely useful.

Paul

[Sebastian Ramacher]

Hi Ondřej

On 2021-01-12 12:40:22, Ondřej Surý wrote:
> I guess we are going to release with PHP 7.4 then. Not the ideal choice,
> but I will do whatever I can to support it.
>
> However, I would like to get an official statement from the release team
> what's their position, so we can go forward. Pretty please?
>
> Also an advice on how to proceed next time this happens. The timing of our
> freeze and new PHP release coincidentally goes together. Should I start the
> transition with alpha/beta/rc instead of the first final release?

The timing is not ideal and there are still quite a bunch of blocking
bugs open. Given also list of autopkgtest regressions reported for the
php-defaults version in experimental, I'm not even sure if all of them
have been reported to the BTS.

I'm also CCing the security team for their input in case the have a
strong opinion on this transition.

To avoid the same situation for the bookworm freeze, I think staging the
transition early in experimental with alpha/beta/rc version with the
corresponding php-defaults change would make sense. This would help to
get most of the FTBFS bugs and autopkgtest regressiones reported and
would give the maintainers time to fix them beforehand.

Cheers and sorry for not replying earlier.

--
Sebastian Ramacher

[Moritz Mühlenhoff]

Am Thu, Jan 14, 2021 at 10:28:41AM +0100 schrieb Sebastian Ramacher:
> I'm also CCing the security team for their input in case the have a
> strong opinion on this transition.

It's fine. PHP 8 would have been great, but it is what it is.

Cheers,
Moritz

[Ondřej Surý]

Thinking about it, security-wise it might be better. Microsoft will support the security backports to EOL versions of PHP 7.x, but they announced they won’t do it for PHP 8.x, so we are (maybe) bit more covered with PHP 7.4.

Ondrej

--
Ondřej Surý <ond...@sury.org> (He/Him)

> On 15. 1. 2021, at 19:45, Moritz Mühlenhoff <j...@inutil.org> wrote:

[Moritz Mühlenhoff]

Am Fri, Jan 15, 2021 at 07:58:10PM +0100 schrieb Ondřej Surý:
> Thinking about it, security-wise it might be better. Microsoft will support the security backports to EOL versions of PHP 7.x, but they announced they won’t do it for PHP 8.x, so we are (maybe) bit more covered with PHP 7.4.

Even better :-)

Cheers,
Moritz

[Salvatore Bonaccorso]

Hi,

On Fri, Jan 15, 2021 at 07:58:10PM +0100, Ondřej Surý wrote:
> Thinking about it, security-wise it might be better. Microsoft will
> support the security backports to EOL versions of PHP 7.x, but they
> announced they won’t do it for PHP 8.x, so we are (maybe) bit more
> covered with PHP 7.4.

Ok, sounds good then.

Btw, we are currently at 7.4.11-1 in unstable, while at least updating
up to 7.4.14 would fix as well CVE-2020-7071.

If nothing speaks against it, it would be good to start bullseye with
a version most near with upstream stable released version (we would
otherwise anyway need to otherwise rebase to the next 7.4.y version in
the first DSA for php7.4).

Are you planning to update to 7.4.14 (or later) before the soft
freeze?

Regards,
Salvatore

[Thorsten Glaser]

block 976811 by 980567
thanks

On Sat, 12 Dec 2020, Ondřej Surý wrote:

> And let me restate that it’s not my intent to make anyone’s life hell and
> I am willing to help with any package (as usual). I am just trying to do
> the most sane thing to do security and maintainer wise.

You probably should have communicated this with maintainers of PHP
packages. I was taken completely by surprise (via #980567) that
PHP 8 is even available in Debian now, and back when I asked you
to provide a release candidate via experimental, so that we could
test software with it before the release, some months ago, you
denied that, so colour me surprised to get a bugreport from it now.

As things are, I won’t upgrade php-react-promise now. If I find
the time I may backport the two commits mentioned to fix the one
issue reported, but… it’s already tight. (Upgrading is completely
out of the picture, as this involves multiple packages, and there
is a dependency chain reaching as far as composer. By the way, if
the composer maintainers with to take over the reactphp packages,
that could be arranged.)

I also had to actively search for this, wondering which PHP version
we’d be having with bullseye then… and this was not obvious to find.

bye,
//mirabilos
--
Yes, I hate users and I want them to suffer.
-- Marco d'Itri on gmane.linux.debian.devel.general

[Salvatore Bonaccorso]

Ondrej, Release Team,

Right now php8.0 will be as well in bullseye. Should php8.0 be kept
out of testing until bullseye is released? Otherwise there will be
expectation that both php7.4 and php8.0 will be covered by (security)
support in bullseye if we release with php8.0 included.

Regards,
Salvatore

[Moritz Muehlenhoff]

On Sat, Feb 06, 2021 at 09:26:39PM +0100, Salvatore Bonaccorso wrote:
> Otherwise there will be
> expectation that both php7.4 and php8.0 will be covered by (security)
> support in bullseye if we release with php8.0 included.

Yeah, let's drop 8.0 then.

Cheers,
Moritz

[Ondřej Surý]

I concur, and I was planning to ask for removal after soft freeze kicks in, so it could not migrate back, but RC bug that Salvatore filled is fine too.

Cheers,
        Moritz

--

--
Ondřej Surý

[Sebastian Ramacher]

Hi Ondřej

On 2020-12-08 09:28:38 +0100, Ondřej Surý wrote:
> Package: release.debian.org
> Severity: normal
> User: release.d...@packages.debian.org
> Usertags: transition
>

> Hi,
>
> I would like to transition the PHP to version 8.0; it's not such a huge bump as
> it was with 5.6 -> 7.0 and most of the packages that were compatible with PHP
> 7.4 are working just fine with PHP 8.0.
>
> I do have most of the extensions ready for PHP 8.0 either via new upstream version
> or with patches.

bullseye was released so I think we can revisit this transition. What's
the status here?

Cheers

>
> Ondrej
>
> Ben file:
>
> title = "php8.0";
> is_affected = .depends ~ "phpapi-20190902" | .depends ~ "phpapi-20200930";
> is_good = .depends ~ "phpapi-20200930";
> is_bad = .depends ~ "phpapi-20190902";
>
>

> -- System Information:
> Debian Release: 10.7
> APT prefers stable-updates
> APT policy: (500, 'stable-updates'), (500, 'stable')
> Architecture: amd64 (x86_64)
> Foreign Architectures: i386
>
> Kernel: Linux 4.19.0-12-amd64 (SMP w/8 CPU cores)
> Kernel taint flags: TAINT_OOT_MODULE, TAINT_UNSIGNED_MODULE
> Locale: LANG=en_IE.UTF-8, LC_CTYPE=en_IE.UTF-8 (charmap=UTF-8), LANGUAGE=en_IE:en (charmap=UTF-8)
> Shell: /bin/sh linked to /bin/dash
> Init: systemd (via /run/systemd/system)
>
>

--

Sebastian Ramacher

[Ondřej Surý]

Hi Sebastian,

the PHP 8.1 RC1 was released, so I think it would be better to skip php8.0
and go directly with php8.1. I do have the base package, but I haven’t tested
any extensions yet, but I do plan to do so in upcoming weeks.

I’ll update this issue when I am ready.

Ondrej
--
Ondřej Surý (He/Him)
ond...@sury.org

[Sebastian Ramacher]

Control: forwarded -1 https://release.debian.org/transitions/html/php8.1.html

On 2021-09-05 19:26:39, Ondřej Surý wrote:
> Hi Sebastian,
>
> the PHP 8.1 RC1 was released, so I think it would be better to skip php8.0
> and go directly with php8.1. I do have the base package, but I haven’t tested
> any extensions yet, but I do plan to do so in upcoming weeks.
>
> I’ll update this issue when I am ready.

The tracker for php8.1 is now at
https://release.debian.org/transitions/html/php8.1.html

Cheers

--
Sebastian Ramacher

[David Prévot]

Hi,

Le 10/11/2021 à 05:16, Sebastian Ramacher a écrit :
> On 2021-09-05 19:26:39, Ondřej Surý wrote:
>> Hi Sebastian,
>>
>> the PHP 8.1 RC1 was released, so I think it would be better to skip php8.0

[…]

>> I’ll update this issue when I am ready.

It seems that php-defaults (85) was uploaded to unstable, pushing PHP
8.1 as default in unstable before this was even a default in experimental.

Did I miss some communication, has this transition already started?

Regards

David

[Paul Gevers]

Hi,

Not that I'm aware of, and I don't think it's a good idea to have it
started already as there is so much fall out. Can this please be
reverted and staged in experimental first? Bugs filed for all issues
with autopkgtest? Such that there's a good overview (and fixes) *before*
we transition?

Paul

[Sebastiaan Couwenberg]

swig is not included in the transition tracker, but it should support
PHP 8 from 4.2.0 onward which has not been released yet.

Kind Regards,

Bas

--
GPG Key ID: 4096R/6750F10AE88D4AF1
Fingerprint: 8182 DE41 7056 408D 6146 50D1 6750 F10A E88D 4AF1

[Ondřej Surý]

I disagree, but I uploaded reverted package. It’s after release and we have two years before next release and a year before PHP 8.2, so unstable could be a bit unstable. It’s not like the bump from php7.4 to php8.1 is that large as it was from php5.6 to php7.0.

The transition to php8.x has been announced a year ago.

Ondrej
--
Ondřej Surý <ond...@sury.org> (He/Him)

> On 19. 11. 2021, at 21:27, Paul Gevers <elb...@debian.org> wrote:
>
> Hi,

[David Prévot]

Hi Ondřej,

Le 19/11/2021 à 16:41, Ondřej Surý a écrit :
> I disagree, but I uploaded reverted package.

Thank you for your quick action. However, php-defaults 86 as just
uploaded reverted the default PHP version to 8.0, de facto starting a
transition you wanted to skip (and still making it impossible to install
and build several packages in the mean time).

Can you please actually revert the package to what it contained in
version 76 currently in testing (and stable)?

In the mean time, providing a PHP 8.1 default version in experimental
would be welcome in order to help and find regressions (we’ve already
been able to chase and fix many issues with PHP 8.0, it would be nice to
track issues that still needs to be fixed in order to help make this
transition as smooth and fast as possible).

Regards

David

[Paul Gevers]

Hi Ondřej,

On 19-11-2021 21:41, Ondřej Surý wrote:
> I disagree, but I uploaded reverted package. It’s after release and we have two years before next release and a year before PHP 8.2, so unstable could be a bit unstable. It’s not like the bump from php7.4 to php8.1 is that large as it was from php5.6 to php7.0.

Thanks for your willingness to cooperate, even though you don't agree.
And sorry the way I phrased my response yesterday. I was too surprised
and one of "my" packages got broken (see the cacti autopkgtest results).
(I knew this from your previous upload to experimental, but had
forgotten about it and I haven't seen a bug report about it yet). I
feared (without proper checking, I should have done that) that cacti
wouldn't be the only one broken without warning. (I'll handle cacti
shortly, its new upstream release should at least work with php 8.0).

> The transition to php8.x has been announced a year ago.

To the release team, yes, but how about the affected packages
maintainers? And how about *coordinating* a transition? That's where
transition bug are for [1]. If you tell us when you're ready to start a
transition, we can tell you when it fits in all the other transitions.
There are enough collisions mentioned on the transition page [1] to
warrant an ACK from the release team before going ahead.

In this particular case, we would have asked if bugs had been filed for
all reverse dependencies that are known to be broken. As mention above
at least cacti is missing such a bug report. Maybe there are more? As
mentioned by David, it would be helpful to have an upload to
experimental doing the bump there. So issues can be weed out before the
transition starts.

Paul

[1] https://wiki.debian.org/Teams/ReleaseTeam/Transitions
[2] https://release.debian.org/transitions/html/php8.1.html

[David Prévot]

Hi Ondřej,

Le 19/11/2021 à 16:41, Ondřej Surý a écrit :

> I disagree, but I uploaded reverted package.

Unfortunately, you also need to bump binary packages version. This
revert got rejected:

$ ssh coccia.debian.org cat
/srv/ftp-master.debian.org/queue/reject/php-defaults_87_all-buildd.changes.reason

Version check failed:
Your upload included the binary package libapache2-mod-php, version
2:7.4+87, for all,
however unstable already has version 2:8.1+85.
Uploads to unstable must have a higher version than present in unstable.

Regards

David

[David Prévot]

Le 22/11/2021 à 08:45, Ondřej Surý a écrit :

> Or we could stop delaying the inevitable[1] and instead of bumping
> epoch just go ahead with the transition.

You don’t need to bump epoch (especially on source package and every
binary ones) just to temporarily bump version of one binary package.

IIRC, last time such transition was pushed in an uncoordinated way like
this (7.3 -> 7.4), it took months to handle. Can we please not repeat
this painful experience for such an expected bigger transition (7.4 -> 8.1)?

The current status is holding some of us to actually work on this
transition (e.g., I’m not able to properly build and push new versions
expected to actually work with recent PHP, and have little view of the
remaining issues that could be spotted via autopkgtests in experimental,
since little or no bugs were raised on affected packages).

Regards

David

[David Prévot]

[ Ondřej, your last mail didn’t make it to the transition bug report,
neither did the previous one. FWIW, I can only see a blank one from your
“Apple Mail” MUA. ]

[ Here is a copy of the sources of your email. I reply after this copy
to try not to add more confusion. ]

Le 22/11/2021 à 10:26, Ondřej Surý a écrit :
>> On 22. 11. 2021, at 14:15, David Pr=C3=A9vot <da...@tilapin.org> =
> wrote:
>>=20
>> Le 22/11/2021 =C3=A0 08:45, Ond=C5=99ej Sur=C3=BD a =C3=A9crit :
>>=20

>> > Or we could stop delaying the inevitable[1] and instead of bumping
>> > epoch just go ahead with the transition.

>>=20
>> You don=E2=80=99t need to bump epoch (especially on source package and =
> every binary ones) just to temporarily bump version of one binary =
> package.
>>=20
>> IIRC, last time such transition was pushed in an uncoordinated way =
> like this (7.3 -> 7.4), it took months to handle. Can we please not =
> repeat this painful experience for such an expected bigger transition =
> (7.4 -> 8.1)?
>
> You asked for more time last December, a year has passed and it=E2=80=99s =
> again not good time. What would just having more time grant? We cannot =
> have bookworm released with PHP 7.4, so what will change and how much =
> more time will be =E2=80=9Cenough=E2=80=9D? We cannot hold back on PHP =
> version because Debian contains software that cannot keep up with the =
> PHP release model. People would actually appreciate having a recent PHP =
> interpreter packaged in Debian instead of software written in PHP. As =
> far as I understand most people are using stuff like composer anyway.
>
>> The current status is holding some of us to actually work on this =
> transition (e.g., I=E2=80=99m not able to properly build and push new =
> versions expected to actually work with recent PHP, and have little view =
> of the remaining issues that could be spotted via autopkgtests in =

> experimental, since little or no bugs were raised on affected packages).
>

> What transition? PHP 7.4 has been released in Debian bullseye, so I =
> cannot imaging what transition you are talking about. PHP 7.4 is =
> switching to security-only in December with only one extra year. We need =
> to switch to PHP 8.1 now and to PHP 8.2 before bookworm release.
>
> So, again - how much time would be enough if a year was not enough?
>
> Ondrej

[ It case that was not clear, I’m not a release team member. ]

Last December, I pointed that the lack of coordination to push PHP 8.0
into Bullseye just before the freeze was not optimal. IIUC, the people
in charge at the time (the release team, the security team, and the PHP
maintainers) decided to stick with PHP 7.4 for Bullseye.

I have no intent to delay the transition to PHP 8: we all know that
Bookworm won’t ship with PHP 7.

> What transition?

I really mean the php8.1 transition, the one we’re discussing in this
bug report, apologies if that was unclear.

What is currently bothering me is, since you pushed php-defaults 85 to
unstable three days ago, packages are currently uninstallable (and thus,
packages can’t currently be built) [1]. You claim to be willing to
revert the transition you started without coordination, but the two
packages you uploaded for that purpose can’t work or reach the archive.
IIRC, you also pushed the previous transition (the php7.4 one) after
uploading php-defaults to unstable by mistake. Seeing the same pattern
again makes it more difficult to assume good faith.

Regards

David

1: That is the issue that triggered my initial mail three days ago, I
hope you (PHP maintainers and release team) can work out some solution
quickly.

[David Prévot]

Hi Ondřej,

Le 22/11/2021 à 09:15, David Prévot a écrit :
> Le 22/11/2021 à 08:45, Ondřej Surý a écrit :
>

> > Or we could stop delaying the inevitable[1] and instead of bumping
> > epoch just go ahead with the transition.
>

> You don’t need to bump epoch

Please find attached a short debdiff that fixes the issue at hand
(restoring php7.4 as default in unstable), without an epoch (the ugly
2:8.1+85+really7.4+87+nmu1 version being higher than 2:8.1+85, the
binary packages won’t be rejected, and will disappear as soon as you’ll
upload a higher version (e.g. 88) without this change when the php8.1
transition will start, 2:8.1+88 being higher than
2:8.1+85+really7.4+87+nmu1).

I’ve tested that the new binary packages are now allowed to be installed
and that related packages can again be built.

I’m happy to upload it if you or the release team agree. I don’t mind if
the transition gets started right now either (even if we have no proper
php8.1 as default in experimental to get a grasp of expected issues).

Regards

David

[Paul Gevers]

Hi Ondřej,

On 23-11-2021 11:52, Ondřej Surý wrote:

>> On 22. 11. 2021, at 22:28, David Prévot <da...@tilapin.org> wrote:
>>
>> I’m happy to upload it if you or the release team agree. I don’t
>> mind if the transition gets started right now either (even if we
>> have no proper php8.1 as default in experimental to get a grasp of
>> expected issues). >

> I’ve just uploaded a version with your fix.

Thanks a lot.

> David, can we now agree on a timeframe when we start the transition?

As a Release Team member I value the input from David on this matter,
but it's not up to him to decide. That said, I'd love the PHP community
in Debian (I assume I can take Debian PHP Maintainers + Debian PHP PEAR
Maintainers for that, please correct me if I'm wrong) to align on a
transition plan and share that with us. As a Release Team member, I'd
like to see a plan (maybe I outlined it below already) where we can hope
for a reasonable short time where php-defaults in unstable and testing
are out of sync. Practically that means that most of the issues need to
be identified, and sufficient progress needs to be made, *before* the
upload to unstable, which starts the transition. We can remove some
packages where progress isn't reasonably achievable in a timely matter,
but we can't remove half the reverse dependencies of php.

Experimental is the ideal place to find that out. I does require
somebody to go through the regressions and file bug though, this doesn't
happen magically. I think David offered help there. I (Release Team
member hat *off*) am willing to help a bit too. I welcome everybody to
file issues related to switching from php7.4 to php8.1 and add a block
on this bug. That way, we can generate a good overview of where we
stand. (The cacti issue seems to have been resolved with my latest
upload; one issue down). David, I think you already said you wouldn't
mind if the transition already started now, is that the stance of the
Debian PHP PEAR Maintainers? How much of the broken reverse dependencies
are outside that teams maintenance (approximately)?

> And I would prefer if we roughly agree on a timeframe when we start
> the transition to php8.2 - I can upload the 8.2.0~beta1 as soon as it
> is ready upstream, so the ftp-masters have time, and keep uploading
> rcN versions (this will usually take few months) and start the
> transition right away when 8.2.0 is golden (December 2022). Would
> that work?
The Release Team _very much_ welcomes staging/testing these things in
experimental exactly as you propose. So by all means, upload early,
including the relevant php-defaults too. Similar as for the current
situation, if the amount of fall out has been identified, bugs filed and
(most) solutions available we can go ahead. We can't commit to an ACK
already (it really depends on the amount of open issues), but with the
right preparation I'd say it's very doable. But it also depends a bit on
the cooperation of the maintainers of your reverse dependencies and/or
NMU actions. This takes time and coordination.

Paul

[David Prévot]

Hi,

Le 23/11/2021 à 15:57, Paul Gevers a écrit :
> On 23-11-2021 11:52, Ondřej Surý wrote:
>>> On 22. 11. 2021, at 22:28, David Prévot <da...@tilapin.org> wrote:

>> I’ve just uploaded a version with your fix.
>
> Thanks a lot.

+1.

>> David, can we now agree on a timeframe when we start the transition?

> […] it's not up to him to decide.

> […] David, I think you already said you wouldn't

> mind if the transition already started now, is that the stance of the
> Debian PHP PEAR Maintainers?

AFAICT, nobody objects (the team is CCed, since a few mails). We’re also
holding on some updates because some packages already require PHP 8
(other are still after PHP 5 compatibility, the world is a mess ;).

> How much of the broken reverse dependencies
> are outside that teams maintenance (approximately)?

No clue, sorry. Not sure autopkgtest are used a lot outside of the PHP
PEAR (and Composer) and Horde teams, that doesn’t help to get a quick grasp.

>> And I would prefer if we roughly agree on a timeframe when we start
>> the transition to php8.2 - I can upload the 8.2.0~beta1 as soon as it
>> is ready upstream

[…]

> The Release Team _very much_ welcomes staging/testing these things in
> experimental exactly as you propose. So by all means, upload early,
> including the relevant php-defaults too.

[…]

> But it also depends a bit on
> the cooperation of the maintainers of your reverse dependencies and/or
> NMU actions. This takes time and coordination.

Thanks for sharing your plans in advance, happy to try and make this
timeframe work.

Regards

David

[David Prévot]

Hi,

Le 23/11/2021 à 15:57, Paul Gevers a écrit :> On 23-11-2021 11:52,
Ondřej Surý wrote:
[…]

> Experimental is the ideal place to find that out. I does require
> somebody to go through the regressions and file bug though, this doesn't
> happen magically. I think David offered help there.

I’ve checked that all [0] packages with a failing testsuite using PHP
8.1 from experimental [1] have now a bug (severity important, blocking
the current one) about this issue.

0: *except*
- php-horde* packages (Mike and team CCed);
- packages not in testing (shouldn’t be a blocker);
- packages already fixed (come on, refresh! ;).

1:
https://release.debian.org/britney/pseudo-excuses-experimental.html#php-defaults

For Horde packages, Paul already noticed many errors are “just” PHP
deprecations, so using “Restrictions: allow-stderr” in d/t/control
should at least allow to go one step further, even if we’re just hiding
issues under the carpet for now.

>> And I would prefer if we roughly agree on a timeframe when we start
>> the transition to php8.2 - I can upload the 8.2.0~beta1 as soon as it
>> is ready upstream, so the ftp-masters have time, and keep uploading
>> rcN versions (this will usually take few months) and start the
>> transition right away when 8.2.0 is golden (December 2022). Would
>> that work?

(keeping that part to inform Horde maintainers in the mean time).

Regards

David

[Ondřej Surý]

Folks,

I just want say sorry that I was so grumpy. I am doing this for too long and I am tired. Thanks for not picking fight with me.

I will try to do better, but still having a co-maintainer(s) that would help with the administrivia would help a lot. I can handle most of the technical stuff, but the things beyond are sometimes too much.

Ondrej
--
Ondřej Surý <ond...@sury.org> (He/Him)

> On 26. 11. 2021, at 15:30, David Prévot <da...@tilapin.org> wrote:
>
> Hi,

[Paul Gevers]

Hi all,

Bugs have been filed (with the exception of the horde stack, but they
are aware) of items we're aware of. Let's give the maintainers a month.
I propose we can start the php8.1 transition around Christmas 2021. Does
that work for you Ondřej?

Paul

[Paul Gevers]

Control: tags -1 confirmed
Control: severity 1000574 serious
Control: severity 1000593 serious
Control: severity 977403 serious
Control: severity 1000642 serious
Control: severity 1000654 serious
Control: severity 1000653 serious
Control: severity 1000619 serious
Control: severity 978457 serious
Control: severity 977385 serious
Control: severity 1000474 serious
Control: severity 1002020 serious
Control: severity 1000650 serious
Control: severity 1002504 serious
Control: severity 1000596 serious
Control: severity 1000571 serious
Control: severity 977373 serious

Hi Ondřej,

On 05-12-2021 22:11, Paul Gevers wrote:
> I propose we can start the php8.1 transition around Christmas 2021. Does
> that work for you Ondřej?

Assuming you were OK with this timing, I propose you go ahead now.

Paul

[Ondřej Surý]

Uploaded php 8.1.1 and php-defaults 91 switching the Debian PHP version to PHP 8.1

Happy New Year everyone!

Ondrej
--
Ondřej Surý (He/Him)
ond...@sury.org

[Paul Gevers]

Hi Ondřej, PHP PECL Maintainers,

On 31-12-2021 12:50, Ondřej Surý wrote:
> Uploaded php 8.1.1 and php-defaults 91 switching the Debian PHP version to PHP 8.1

Thanks.

Will you also upload src:php-imagick, which seems to block some rebuilds
in the current state.

I want to update the ben transition tracker file to catch these kind of
packages. Should I add a "Depends on php7.4-common" to the "bad" list?

Paul

[Paul Gevers]

Hi,

It seems I don't understand how PHP packaging works. I scheduled
rebuilds of several packages that were yesterday on the tracker page [1]
when that still only had the phpapi-* listed. Several packages can't be
build yet it seems (because they depend on packages that need new
uploads), but e.g. wikidiff2 built fine *but disappeared* from the
tracker. I looked at a build log [2] and noticed that the binary has
files in /etc/php8.1 (and not in /etc/php7.4), but doesn't tell
otherwise (in package relations) that it's meant for php8.1 and not for
php7.4. Why did that phpapi-* thing disappear? Is that intended? The
built binaries already migrated to testing, while the default there is
still php7.4. I assume we can consider php-wikidiff2 (partially?) broken
now *in testing*? Same for php-luasandbox [3], php-geos [4], and
php-excimer [5] (which also FTBFS on mipsel).

Some rebuilds failed, e.g. php-horde-lz4 [6], php-wmerrors [7], owfs [8].

I also notice that quite some php packages grew a php7.4-* package in
unstable. Is the idea that that needs to be done for php8.1 too and that
all those packages require a round trip through NEW? If that's the case,
next time please prepare those in experimental, such that we don't need
to wait for processing of NEW during the transition. If that's not the
case, please upload new versions of those packages dropping the php7.4
package.

Please enlighten me on the details of php packaging that I'm missing and
that we should be aware of for this (and future) transition(s).

Paul

[1] https://release.debian.org/transitions/html/php8.1.html
[2]
https://buildd.debian.org/status/fetch.php?pkg=wikidiff2&arch=amd64&ver=1.13.0-1%2Bb1&stamp=1640981156&raw=0
[3] https://buildd.debian.org/status/package.php?p=php-luasandbox
[4] https://buildd.debian.org/status/package.php?p=php-geos
[5] https://buildd.debian.org/status/package.php?p=php-excimer
[6] https://buildd.debian.org/status/package.php?p=php-horde-lz4
[7] https://buildd.debian.org/status/package.php?p=php-wmerrors
[8] https://buildd.debian.org/status/package.php?p=owfs

[Kunal Mehta]

Hi,

On 1/1/22 00:01, Paul Gevers wrote:
> It seems I don't understand how PHP packaging works. I scheduled
> rebuilds of several packages that were yesterday on the tracker page [1]
> when that still only had the phpapi-* listed. Several packages can't be
> build yet it seems (because they depend on packages that need new
> uploads), but e.g. wikidiff2 built fine *but disappeared* from the
> tracker. I looked at a build log [2] and noticed that the binary has
> files in /etc/php8.1 (and not in /etc/php7.4), but doesn't tell
> otherwise (in package relations) that it's meant for php8.1 and not for
> php7.4. Why did that phpapi-* thing disappear? Is that intended? The
> built binaries already migrated to testing, while the default there is
> still php7.4. I assume we can consider php-wikidiff2 (partially?) broken
> now *in testing*? Same for php-luasandbox [3], php-geos [4], and
> php-excimer [5] (which also FTBFS on mipsel).

Previously the dh_php step would add the phpapi-<date> dependency,
however this was removed[1] and the new php<version>-common dependency
is added via pkg-pecl.mk[2]. The packaging for
wikidiff2/luasandbox/excimer/wmerrors (all basically identical) only
uses the dh_php step, and not pkg-pecl.mk.

If it's intentional that using dh_php alone is no longer enough, I can
figure out how to switch those to using pkg-pecl.mk, though last I
checked it wasn't straightforward to use for packages not actually on
PECL, which is still the case for wikidiff2/wmerrors.

The php-excimer test failure on mipsel is most likely just a race
condition in the test, I'll look into it in the next few days.

> Some rebuilds failed, e.g. php-horde-lz4 [6], php-wmerrors [7], owfs [8].

I filed [3] upstream for wmerrors, will poke the author if I can't
figure out the fix myself.

[1]
https://salsa.debian.org/php-team/dh-php/-/commit/87f1657c1b26ad10b7cb919336d998b59d137313
[2]
https://salsa.debian.org/php-team/dh-php/-/commit/fdea4536adfbb28fb55ac3e3d0a247faae2b597b
[3] https://phabricator.wikimedia.org/T298421

-- Kunal

[Sebastiaan Couwenberg]

On 1/1/22 10:18, Kunal Mehta wrote:
> On 1/1/22 00:01, Paul Gevers wrote:
>> It seems I don't understand how PHP packaging works. I scheduled
>> rebuilds of several packages that were yesterday on the tracker page
>> [1] when that still only had the phpapi-* listed. Several packages
>> can't be build yet it seems (because they depend on packages that need
>> new uploads), but e.g. wikidiff2 built fine *but disappeared* from the
>> tracker. I looked at a build log [2] and noticed that the binary has
>> files in /etc/php8.1 (and not in /etc/php7.4), but doesn't tell
>> otherwise (in package relations) that it's meant for php8.1 and not
>> for php7.4. Why did that phpapi-* thing disappear? Is that intended?
>> The built binaries already migrated to testing, while the default
>> there is still php7.4. I assume we can consider php-wikidiff2
>> (partially?) broken now *in testing*? Same for php-luasandbox [3],
>> php-geos [4], and php-excimer [5] (which also FTBFS on mipsel).
>
> Previously the dh_php step would add the phpapi-<date> dependency,
> however this was removed[1] and the new php<version>-common dependency
> is added via pkg-pecl.mk[2]. The packaging for
> wikidiff2/luasandbox/excimer/wmerrors (all basically identical) only
> uses the dh_php step, and not pkg-pecl.mk.

The removal of the phpapi dependency should be reverted.

php-geos now only depends on: php-common (>= 1:7.0+33~)

But it installs files in /etc/php/8.1/mods-available and
/usr/lib/php/20210902 which are not provided by older php-common versions.

[Paul Gevers]

Hi

On 01-01-2022 14:20, Ondřej Surý wrote:
> 2. Generate all binary packages to debian/control during the build,
> but this would then require overrides (which I think doesn’t really
> scale)
If you mean (re)generating debian/control during build, that's not
allowed (albeit I can't quickly find a reference).

Paul

[Paul Gevers]

Hi,

>> Please enlighten me on the details of php packaging that I'm missing and that we should be aware of for this (and future) transition(s).

Is it true that because the api is already provided in testing that we
now have some packages working with (and pulling in) php7.4 and some
with php8.1 by default (in testing). Is that supposed to work
gracefully? Or is this transition (in unstable) now also breaking
testing in ways we don't want? Hmm, thinking a bit about this, I guess
the php configuration needs to be enabled in mods-enabled, so package
upgrades break already without admin intervention, right? And this has
been that way for upgrades between stable releases as well? So, am I
overly worried or is this all "by design"?

Paul

[Paul Gevers]

Hi,

Back from holidays.

On 01-01-2022 14:20, Ondřej Surý wrote:

>> Some rebuilds failed, e.g. php-horde-lz4 [6],
>

> The FTBFS is unrelated to PHP 8.1 transition

>
>> php-wmerrors [7], owfs [8].
>

> These two need upstream update.

I see no activity and no bug reports. Can somebody please update these
packages or file appropriate bugs?

I also see some autopkgtest regressions which have this (eg. [1, 2]):
"""
PHPUnit requires the "dom" extension.
"""
where should that get fixed?

While we're waiting for php-redis in NEW, can we have a version of
php-redis that doesn't need to pass NEW?

Looking at the regression caused by php-apcu in symfony [3], are we
missing some versioned dependency tracking somewhere (or a versioned
breaks)?

Paul

[1]
https://ci.debian.net/data/autopkgtest/testing/amd64/p/php-monolog/18158564/log.gz
[2]
https://ci.debian.net/data/autopkgtest/testing/amd64/p/php-doctrine-cache/18158566/log.gz
[3]
https://ci.debian.net/data/autopkgtest/testing/amd64/s/symfony/18158567/log.gz

[David Prévot]

Hi,

Le 08/01/2022 à 17:38, Paul Gevers a écrit :
> On 01-01-2022 14:20, Ondřej Surý wrote:

[…]

> I also see some autopkgtest regressions which have this (eg. [1, 2]):
> """
> PHPUnit requires the "dom" extension.
> """
> where should that get fixed?

There are several php7.4-* packages pulled in those logs, so it’s not
really a surprise that doesn’t end well (php-xml 2:7.4+76 is probably
not the expected version either).

> [1]
> https://ci.debian.net/data/autopkgtest/testing/amd64/p/php-monolog/18158564/log.gz
>
> [2]
> https://ci.debian.net/data/autopkgtest/testing/amd64/p/php-doctrine-cache/18158566/log.gz

Regards

David

[Paul Gevers]

Hi David,

On 08-01-2022 23:09, David Prévot wrote:
>> I also see some autopkgtest regressions which have this (eg. [1, 2]):
>> """
>> PHPUnit requires the "dom" extension.
>> """
>> where should that get fixed?
>
> There are several php7.4-* packages pulled in those logs, so it’s not
> really a surprise that doesn’t end well (php-xml 2:7.4+76 is probably
> not the expected version either).

Normally this points to an issue with the right versioned Depends or
versioned Breaks. This is important because some of these packages are
currently allowed to migrate to testing without php-defaults (were it
not for the autopgktest failure), and would break functionality in
testing. We need to find out how to fix that.

Paul

[Ondřej Surý]

Hmm, this might need same treatment as src:php-defaults. I will need to export the version-to-break somehow from the src:php-defaults. I’ll have to think about it for a while, but if you have another smart idea, please share it.

--
Ondřej Surý <ond...@sury.org> (He/Him)

> On 9. 1. 2022, at 19:38, Paul Gevers <elb...@debian.org> wrote:
>
> Hi David,

[David Prévot]

Hi,

Le 09/01/2022 à 14:37, Paul Gevers a écrit :
[…]

> On 08-01-2022 23:09, David Prévot wrote:

[…]

>>> PHPUnit requires the "dom" extension.
>>> """
>>> where should that get fixed?
>>
>> There are several php7.4-* packages pulled in those logs, so it’s not
>> really a surprise that doesn’t end well (php-xml 2:7.4+76 is probably
>> not the expected version either).
>
> Normally this points to an issue with the right versioned Depends or
> versioned Breaks.

I was actually trying to point an issue in the test environment (but let
me try in verbose mode).

> This is important because some of these packages are
> currently allowed to migrate to testing without php-defaults (were it
> not for the autopgktest failure), and would break functionality in
> testing. We need to find out how to fix that.

I assume you pointed to logs in testing where php-defaults is pulled
from unstable. Actually, php-defaults (source package, version 91, from
unstable) builds php-xml (binary package, version 2:8.1+91) that
correctly depends on php8.1-xml. According to the log (and that’s not a
surprise given the error message you pointed out), the php-xml version
actually installed in the autopkgtest environment is 2:7.4+76 (i.e. the
version from testing, built from php-defaults 76). My question is: how
is that possible? If the autopkgtest environment is supposed to pull
(the binary packages built by) php-defaults, it looks like this failed.

Regards

David

[Kunal Mehta]

Hi,

On 1/8/22 13:38, Paul Gevers wrote:
>>> php-wmerrors [7], owfs [8].
>>
>> These two need upstream update.
>
> I see no activity and no bug reports. Can somebody please update these
> packages or file appropriate bugs?

Filed #1003432 for php-wmerrors, and am working on it upstream since
it's more involved than I initially expected. I assume it's fine if it
drops out of testing for a bit.

-- Kunal

[Paul Gevers]

Hi,

On 09-01-2022 23:22, David Prévot wrote:
>> This is important because some of these packages are currently allowed to migrate to testing without php-defaults (were it not for the autopgktest failure), and would
>> break functionality in testing. We need to find out how to fix that.
>
> I assume you pointed to logs in testing where php-defaults is pulled from unstable. Actually, php-defaults (source package, version 91, from unstable) builds php-xml
> (binary package, version 2:8.1+91) that correctly depends on php8.1-xml. According to the log (and that’s not a surprise given the error message you pointed out), the
> php-xml version actually installed in the autopkgtest environment is 2:7.4+76 (i.e. the version from testing, built from php-defaults 76). My question is: how is that
> possible? If the autopkgtest environment is supposed to pull (the binary packages built by) php-defaults, it looks like this failed.

The logs I pointed at were NOT for php-defaults, but for respectively
php-amqp [1] and php-apcu [2], both of which aren't migrating because of
this [3, 4]. But if these failures are *because* php-defaults from
unstable isn't pulled in, there is a missing versioned dependency or
breaks somewhere. Ondřej suggested breaks from src:php-defaults need to
propagate to reverse (build) dependencies somehow, I think the easiest
would be if this is covered by packages that provide the phpapi (as long
as all relevant packages properly depend on that).

Paul

[1]
https://ci.debian.net/data/autopkgtest/testing/amd64/p/php-monolog/18158564/log.gz
[2]
https://ci.debian.net/data/autopkgtest/testing/amd64/p/php-doctrine-cache/18158566/log.gz

[3] https://qa.debian.org/excuses.php?package=php-amqp
[4] https://qa.debian.org/excuses.php?package=php-apcu

[Paul Gevers]

Hi Ondřej,

Found via the transition tracker [0], just in case you forgot about
them, php-geoip [1], php-pinba [2], php-propro [3] php-stomp [4] and
php-apcu-bc [5] are awaiting new uploads from you for the transition and
because of source-only uploads.

Paul

[0] https://release.debian.org/transitions/html/php8.1.html
[1] https://tracker.debian.org/pkg/php-geoip
[2] https://tracker.debian.org/pkg/php-pinba
[3] https://tracker.debian.org/pkg/php-propro
[4] https://tracker.debian.org/pkg/php-stomp
[5] https://tracker.debian.org/pkg/php-apcu-bc

[Paul Gevers]

Hi

And will you take care of sassphp too? I didn't trigger the binNMU
because of the build failure reported, but your other uploads seem to
work around the issue in dh-php.

Paul

[Paul Gevers]

Hi,

On 10-01-2022 21:13, Ondřej Surý wrote:
> I thought I filled RM bugs for all of them, but I found only #1003055 for php-apcu-bc, something must went wrong.
>
> Neither of these support PHP 8.x, and those packages should be removed.

I missed that. I'll remove them from testing already.

> I’ll refill the missing RM bugs tomorrow.

Yes, please do, to also remove them from unstable.

Paul

[Paul Gevers]

Hi,

On 10-01-2022 21:32, Paul Gevers wrote:
> Hi,
>
> On 10-01-2022 21:13, Ondřej Surý wrote:
>> I thought I filled RM bugs for all of them, but I found only #1003055
>> for php-apcu-bc, something must went wrong.
>>
>> Neither of these support PHP 8.x, and those packages should be removed.
>
> I missed that. I'll remove them from testing already.

Seems like that needs more work:

elbrus@coccia:~$ dak rm --no-action -R --suite testing php-apcu-bc
[...]
Checking reverse dependencies...
# Broken Build-Depends:
php-doctrine-cache: php-apcu-bc
symfony: php-apcu-bc

Which means those need to be updated first to not need php-apcu-bc.

Paul

[David Prévot]

Le 10/01/2022 à 16:44, Paul Gevers a écrit :
>> On 10-01-2022 21:13, Ondřej Surý wrote:
>>> I thought I filled RM bugs for all of them, but I found only #1003055
>>> for php-apcu-bc, something must went wrong.
>>>
>>> Neither of these support PHP 8.x, and those packages should be removed.

> Seems like that needs more work:
>
> elbrus@coccia:~$ dak rm --no-action -R --suite testing php-apcu-bc
> [...]
> Checking reverse dependencies...
> # Broken Build-Depends:
> php-doctrine-cache: php-apcu-bc

That on is gone.

> symfony: php-apcu-bc

There seems to be a new (unrelated?) FTBFS, so we need to figure it out
(or drop symfony from testing until then).

Regards

David

[Paul Gevers]

Hi David,

On 10-01-2022 23:43, David Prévot wrote:
> Le 10/01/2022 à 16:44, Paul Gevers a écrit :
>>> On 10-01-2022 21:13, Ondřej Surý wrote:
>>>> I thought I filled RM bugs for all of them, but I found only
>>>> #1003055 for php-apcu-bc, something must went wrong.
>>>>
>>>> Neither of these support PHP 8.x, and those packages should be removed.
>
>> Seems like that needs more work:
>>
>> elbrus@coccia:~$ dak rm --no-action -R --suite testing php-apcu-bc
>> [...]
>> Checking reverse dependencies...
>> # Broken Build-Depends:
>> php-doctrine-cache: php-apcu-bc
>
> That on is gone.

In unstable (thanks for that), but it fails to migrate to testing, due
to the 'PHPUnit requires the "dom" extension.' issue.

>> symfony: php-apcu-bc
>
> There seems to be a new (unrelated?) FTBFS, so we need to figure it out
> (or drop symfony from testing until then).

If that's OK with you/the team, I can check how much needs to be
removed.... doesn't seems like a lot of fun yet:

elbrus@coccia:~$ dak rm --no-action -R --suite testing symfony

[...]

# Broken Depends:
civicrm: civicrm-common
composer: composer
doctrine: php-doctrine-orm
pdepend: pdepend
php-async-aws-core: php-async-aws-core
php-nesbot-carbon: php-nesbot-carbon
php-proxy-manager: php-proxy-manager
php-symfony-mercure: php-symfony-mercure
php-symfony-security-acl: php-symfony-security-acl
php-tijsverkoyen-css-to-inline-styles: php-tijsverkoyen-css-to-inline-styles
php-twig: php-twig-cache-extra
php-twig-extra-bundle
php-twig-html-extra
php-twig-intl-extra
php-twig-string-extra
phpmyadmin: phpmyadmin
phpmyadmin-motranslator: php-phpmyadmin-motranslator
tweeper: tweeper

# Broken Build-Depends:
composer: php-symfony-console
php-symfony-filesystem
php-symfony-finder
php-symfony-process
doctrine: php-symfony-cache
php-symfony-console
php-symfony-yaml
libphp-swiftmailer: php-symfony-phpunit-bridge
pdepend: php-symfony-config
php-symfony-dependency-injection
php-async-aws-core: php-symfony-http-client
php-doctrine-annotations: php-symfony-cache
php-doctrine-cache: php-symfony-cache
php-doctrine-data-fixtures: php-symfony-cache
php-doctrine-dbal: php-symfony-cache
php-symfony-console
php-doctrine-persistence: php-symfony-cache
php-league-commonmark: php-symfony-finder
php-nesbot-carbon: php-symfony-translation
php-nyholm-psr7: php-symfony-error-handler
php-oscarotero-gettext: php-symfony-yaml
php-phpstan-phpdoc-parser: php-symfony-process
php-pimple: php-symfony-phpunit-bridge
php-proxy-manager: php-symfony-filesystem
php-symfony-mercure: php-symfony-http-client
php-symfony-http-foundation
php-symfony-stopwatch
php-symfony-web-link
php-symfony-polyfill: php-symfony-intl
php-symfony-var-dumper
php-symfony-security-acl: php-symfony-cache
php-symfony-phpunit-bridge
php-symfony-security-core
php-tijsverkoyen-css-to-inline-styles: php-symfony-css-selector
php-twig: php-symfony-framework-bundle
php-symfony-intl
php-symfony-mime
php-symfony-phpunit-bridge
php-symfony-string
php-symfony-twig-bundle
phpmyadmin: php-symfony-config
php-symfony-dependency-injection
php-symfony-expression-language
phpmyadmin-motranslator: php-symfony-expression-language (>= 3.2)
phpunit-diff: php-symfony-process

Paul

[Paul Gevers]

Hi,

On 11-01-2022 20:52, Paul Gevers wrote:
>> There seems to be a new (unrelated?) FTBFS, so we need to figure it
>> out (or drop symfony from testing until then).
>
> If that's OK with you/the team, I can check how much needs to be
> removed.... doesn't seems like a lot of fun yet:

I stopped (the list was still growing) when I had the list below of 184
packages to remove. I saw on reproducible-builds.org that it's in
debwait (on unstable). Worth investigating what needs to be fixed. I
think the FTBFS in testing is due to one of the early binNMU that lost
the versioned dependency (like I feared).

Paul

symfony phpunit-diff phpmyadmin-motranslator phpmyadmin php-twig
php-tijsverkoyen-css-to-inline-styles php-symfony-security-acl
php-symfony-polyfill php-symfony-mercure php-proxy-manager php-pimple
php-phpstan-phpdoc-parser php-oscarotero-gettext php-nyholm-psr7
php-nesbot-carbon php-league-commonmark php-doctrine-persistence
php-doctrine-dbal php-doctrine-data-fixtures php-doctrine-cache
php-doctrine-annotations php-async-aws-core pdepend libphp-swiftmailer
doctrine composer twig-i18n-extension shaarli phpunit
phpmyadmin-sql-parser phpmd php-zend-code php-slim php-monolog php-fxsl
php-fdomdocument php-doctrine-common php-directory-scanner
php-composer-metadata-minifier php-async-aws-sqs php-async-aws-sns
php-async-aws-sesphp-async-aws-ses awl baconqrcode dasprid-enum
google-recaptcha jsonlint mariadb-mysql-kbs php-amqplib
php-arthurhoaro-web-thumbnailer php-async-aws-ses php-codecoverage
php-codesniffer php-composer-ca-bundle php-composer-pcre
php-composer-semver php-composer-spdx-licenses
php-composer-xdebug-handler php-constant-time php-crypt-gpg php-db
php-deepcopy php-doctrine-collections php-doctrine-deprecations
php-doctrine-event-manager php-doctrine-inflector
php-doctrine-instantiator php-doctrine-lexer
php-dragonmantank-cron-expression php-email-validator php-file-iterator
php-geos php-getallheaders php-gettext php-gettext-languages
php-guzzlehttp-promises php-guzzlehttp-psr7 php-hamcrest
php-http-psr7-integration-tests php-httpful php-invoker php-json-schema
php-klogger php-league-flysystem php-league-html-to-markdown
php-league-mime-type-detection php-lorenzo-pinky php-malkusch-lock
php-masterminds-html5 php-mikey179-vfsstream php-mock
php-mock-integration php-mock-phpunit php-mockery php-net-ldap2
php-net-sieve php-netscape-bookmark-parser php-nikic-fast-route
php-nrk-predis php-opis-closure php-parsedown php-parser
php-pda-pheanstalk php-phar-io-manifest php-phar-io-version
php-phpdocumentor-reflection-common
php-phpdocumentor-reflection-docblock php-phpdocumentor-type-resolver
php-phpoption php-phpseclib php-phpseclib3 php-phpspec-prophecy
php-phpspec-prophecy-phpunit php-random-compat php-react-promise
php-sabre-vobject php-sabredav php-shellcommand php-sql-formatter
php-symfony-contracts php-text-password php-text-template php-timer
php-tokenizer php-vlucas-phpdotenv php-webmozart-assert
php-zend-eventmanager php-zend-stdlib php-zeta-base
php-zeta-console-tools phpab phpcpd phpdox phploc phpmyadmin-shapefile
phpseclib phpunit-cli-parser phpunit-code-unit
phpunit-code-unit-reverse-lookup phpunit-comparator phpunit-complexity
phpunit-environment phpunit-exporter phpunit-global-state
phpunit-lines-of-code phpunit-object-enumerator phpunit-object-reflector
phpunit-recursion-context phpunit-resource-operations phpunit-type
roundcube thrift avro-java civicrm davical gnuradio htrace
libphp-phpmailer php-cache-integration-tests php-cache-tag-interop
php-getid3 php-http-httplug php-http-message-factory php-http-promise
php-log php-markdown php-psr-cache php-psr-container
php-psr-event-dispatcher php-psr-http-client php-psr-http-factory
php-psr-http-message php-psr-link php-psr-log php-psr-simple-cache
php-pubsubhubbub-publisher php-ramsey-uuid php-zeta-unit-test
phpunit-version python-elasticsearch reactphp-promise-stream
reactphp-promise-timer roundcube-plugins-extra tcpdf

[David Prévot]

Hi Paul,

Le 11/01/2022 à 15:52, Paul Gevers a écrit :
> On 10-01-2022 23:43, David Prévot wrote:
>> Le 10/01/2022 à 16:44, Paul Gevers a écrit :
>>>> On 10-01-2022 21:13, Ondřej Surý wrote:
>>>>> I thought I filled RM bugs for all of them, but I found only
>>>>> #1003055 for php-apcu-bc, something must went wrong.
>>>>>
>>>>> Neither of these support PHP 8.x, and those packages should be
>>>>> removed.
>>
>>> Seems like that needs more work:
>>>
>>> elbrus@coccia:~$ dak rm --no-action -R --suite testing php-apcu-bc
>>> [...]
>>> Checking reverse dependencies...
>>> # Broken Build-Depends:
>>> php-doctrine-cache: php-apcu-bc
>>

>> That on[e] is gone.

>
> In unstable (thanks for that), but it fails to migrate to testing, due
> to the 'PHPUnit requires the "dom" extension.' issue.

Can’t find this issue. Did it go away by itself, or did you make any change?

>>> symfony: php-apcu-bc
>>
>> There seems to be a new (unrelated?) FTBFS, so we need to figure it
>> out (or drop symfony from testing until then).

Found a fix, version 5.4.2+dfsg-2 uploaded.

> If that's OK with you/the team, I can check how much needs to be
> removed.... doesn't seems like a lot of fun yet:

Right, that’s probably why it used to be a key package.

Regards

David

[Paul Gevers]

Hi,

On 12-01-2022 15:56, David Prévot wrote:
>> In unstable (thanks for that), but it fails to migrate to testing, due
>> to the 'PHPUnit requires the "dom" extension.' issue.
>
> Can’t find this issue. Did it go away by itself, or did you make any
> change?

https://ci.debian.net/packages/p/php-doctrine-cache/testing/amd64/ shows
that it's "fixed" by the new upload of src:php-defaults. php-common has
a whole bunch of Breaks, which cause the test to be run with:
src:php-defaults from unstable
src:php-amqp from unstable
src:php-apcu from unstable
src:php-doctrine-cache-bundle from un...
src:php-doctrine-cache from unstable
src:php-gnupg from unstable
src:php-memcache from unstable
src:php-memcached from unstable
src:php-pcov from unstable
src:php-redis from unstable
src:php-uopz from unstable
src:php-uploadprogress from unstable
src:xdebug from unstable

>>>> symfony: php-apcu-bc
>>>
>>> There seems to be a new (unrelated?) FTBFS, so we need to figure it
>>> out (or drop symfony from testing until then).
>
> Found a fix, version 5.4.2+dfsg-2 uploaded.

Thanks.

It seems like we're getting somewhere. Some packages still need a
source-only upload. And we need to weed through the regressions listed
here: https://qa.debian.org/excuses.php?package=php-defaults. Priority
may lay with the mediawiki* regression on i386: "Internal Server Error"
doesn't sound great, and other non-horde package.

I nearly reached the php-defaults set with my bug filing campaign for
autopgktest regressions, so I'll weed through it. Once all bugs are
filed, I'll ignore all deprecation warning only tests.

Paul

[Paul Gevers]

Dear horde maintainers, Mike,

On 05-12-2021 22:11, Paul Gevers wrote:

You will probably have noticed that the transition started and that I
filed a bunch of bugs against php-horde-* packages about their failing
autopkgtests. I'll stop filing more for now, to not flood your mail
boxes, but I would like a response from your side about what your plan
is with the horde packages and the php8.1 transition.

To progress the transition, I plan to "ignore" the autopkgtest failures
that are merely due to deprecation warnings, but
1) I expect these deprecation warnings to end up in journals/logs, so
they may flood them excessively
2) failing autopkgtests are (and remain) RC issues in testing but I'm
willing to tolerate it for some time if you elaborate on the way forward
and it makes sense.

I think it's necessary to either *fix* the issue, or suppress the
deprecation warnings in the horde packages. Just ignoring them in the
autopkgtests is probably the worse outcome.

Paul

[Mike Gabriel]

Hi Paul,

I plan to fix those, but I can't see light at the end of customer
project tunnels these days. So it might not be as promptly as required.

Mike
--

mike gabriel aka sunweaver (Debian Developer)
mobile: +49 (1520) 1976 148
landline: +49 (4351) 486 14 27

GnuPG Fingerprint: 9BFB AEE8 6C0A A5FF BF22 0782 9AF4 6B30 2577 1B31
mail: sunw...@debian.org, http://sunweavers.net

[Paul Gevers]

Hi,

On 12-01-2022 21:16, Paul Gevers wrote:
> Priority
> may lay with the mediawiki* regression on i386: "Internal Server Error"
> doesn't sound great, and other non-horde package.

Did anybody already take a look at this [1, 2, 3]? It's the last thing
before I'll add a hint to ignore the autopkgtest regressions.
Intermittent "Internal Server Error" sounds more like something in
php8.1 than in mediawiki* hence my reluctance to let php-defaults
migrate. What do you think?

> I nearly reached the php-defaults set with my bug filing campaign for
> autopgktest regressions, so I'll weed through it. Once all bugs are
> filed, I'll ignore all deprecation warning only tests.

I've filed the bugs, minus the mediawiki* one as that's intermittent and
I didn't reach it with my script yet (and minus a load of horde bugs,
but I had other mails about that).

Paul

[1] https://ci.debian.net/packages/m/mediawiki-skin-greystuff/testing/i386/
[2]
https://ci.debian.net/packages/m/mediawiki-extension-youtube/testing/i386/
[3] https://ci.debian.net/packages/m/mediawiki/testing/i386/

[Paul Gevers]

Hi,

One more thing.

On 16-01-2022 20:52, Paul Gevers wrote:
> It's the last thing
> before I'll add a hint to ignore the autopkgtest regressions.

I see in the excuses [1] that some php-* package become uninstallable.
Those need to be fixed. php7.4-common will be removed (of course), I
have just done a no change source only NMU upload of php-igbinary, but
php-lua needs a real upload.

Paul

[1] https://qa.debian.org/excuses.php?package=php-defaults

[Kunal Mehta]

Hi,

On 1/16/22 11:52, Paul Gevers wrote:
> On 12-01-2022 21:16, Paul Gevers wrote:
>> Priority may lay with the mediawiki* regression on i386: "Internal
>> Server Error" doesn't sound great, and other non-horde package.
>
> Did anybody already take a look at this [1, 2, 3]? It's the last thing
> before I'll add a hint to ignore the autopkgtest regressions.
> Intermittent "Internal Server Error" sounds more like something in
> php8.1 than in mediawiki* hence my reluctance to let php-defaults
> migrate. What do you think?

Sorry, didn't get a chance to look at this until now. All 3 failures are
most likely the same underlying cause. As for whether this is a
MediaWiki or PHP 8.1 issue, I'm not sure. Upstream in MW we don't run
any 32-bit CI, so it's entirely possible some 8.0 or 8.1 change broke
something we were doing.

I don't have any i386 hardware, I tried pulling down a i386 Debian
unstable Docker container on my amd64 laptop and installing the
mediawiki+php8.1 packages on that but it didn't trigger the test
failure. Do you have any other suggestions/tips on how I could debug this?

As an alternative I just committed [4] which would hopefully dump the
underlying error message upon failure. Please let me know if it's OK for
me to upload that without it being disruptive.

> [1] https://ci.debian.net/packages/m/mediawiki-skin-greystuff/testing/i386/
> [2]
> https://ci.debian.net/packages/m/mediawiki-extension-youtube/testing/i386/
> [3] https://ci.debian.net/packages/m/mediawiki/testing/i386/

[4]
https://salsa.debian.org/mediawiki-team/mediawiki/-/commit/20a94e6971be6b06d5f41338ec2811cc8572e05f

-- Kunal

[Paul Gevers]

Hi,

On 17-01-2022 06:49, Kunal Mehta wrote:
> I don't have any i386 hardware, I tried pulling down a i386 Debian
> unstable Docker container on my amd64 laptop and installing the
> mediawiki+php8.1 packages on that but it didn't trigger the test
> failure. Do you have any other suggestions/tips on how I could debug this?

The test is flaky, did you run it multiple times? And did you install
php-defaults/92? I don't know how to use Docker, I create i386 lxc for
these [1] (but I think Docker does the same), that's also how they are
created on the ci.d.n infrastructure.

> As an alternative I just committed [4] which would hopefully dump the
> underlying error message upon failure. Please let me know if it's OK for
> me to upload that without it being disruptive.

That works. Personally I would copy them to the artifacts folder instead
of dumping them in the log (to not pollute the log), but whatever. If we
raise confidence in php itself (and can blame mediawiki for these
failures), I'm all for it.

Paul

[1] sudo autopkgtest-build-lxc debian unstable i386

[Paul Gevers]

Hi Ondřej,

I checked why e.g. src:php-uploadprogress isn't migrating. I think I
spotted a packaging mistake, probably coming from the toolchain.
php-uploadprogress Depends on php7.4-uploadprogress, but that package
isn't built anymore or provided by any binary package. As the binary
isn't part of the library Section, migrating src:php-uploadprogress to
testing will make one of its own binary uninstallable. That's a
regression, so britney will not allow it.

It's a bit of a bad timing to implement toolchain changes during a
transition. Can we next time please do these kind of changes in the
toolchain separately from a transition? This php8.1 transition is
causing much more issues that I expected.

How do you plan to fix this?

Paul

[Paul Gevers]

Hi,

On 18-01-2022 19:37, Ondřej Surý wrote:
> the fix is simple, it was actually cruft from before when uploadprogress didn’t support PHP 8.1:

https://sources.debian.org/src/php-uopz/7.1.1+6.1.2-5/debian/rules/#L3
https://sources.debian.org/src/php-gnupg/1.5.1-1/debian/rules/#L3

I guess too.

Paul

[Paul Gevers]

Hi Bryce,

On 19-01-2022 10:28, Bryce Harrington wrote:

> With [4] applied, I'm seeing the following dumped on armhf:
>
> ## https://autopkgtest.ubuntu.com/packages/m/mediawiki/jammy/armhf
> cat: /var/log/mediawiki/error.log: No such file or directory
> 2022-01-19 09:16:57 autopkgtest-lxd-eeoxik autopkgtestwiki: [e33d1ec89af515673078437e] /mediawiki/index.php/Main_Page PHP Fatal Error from line 110 of /usr/share/mediawiki/includes/json/FormatJson.php: Maximum execution time of 30 seconds exceeded
> #0 [internal function]: MWExceptionHandler::handleFatalError()
> #1 {main}
> 2022-01-19 09:17:27 autopkgtest-lxd-eeoxik autopkgtestwiki: [1284cbae5a84eb5387f33003] /mediawiki/index.php/Main_Page PHP Fatal Error from line 152 of /usr/share/mediawiki/includes/HookContainer/HookContainer.php: Maximum execution time of 30 seconds exceeded
> #0 [internal function]: MWExceptionHandler::handleFatalError()
> #1 {main}
> 2022-01-19 09:17:57 autopkgtest-lxd-eeoxik autopkgtestwiki: [31731278ed6070e946af4fbe] /mediawiki/index.php/Main_Page PHP Fatal Error from line 110 of /usr/share/mediawiki/includes/json/FormatJson.php: Maximum execution time of 30 seconds exceeded
> #0 [internal function]: MWExceptionHandler::handleFatalError()
> #1 {main}
> cat: /var/log/mediawiki/fatal.log: No such file or directory

As this issue seems intermittent (as mediawiki passed at one point) I
guess you're trying to say that you think mediawiki got slower with
php8.1? Or is this timeout new with php8.1? Obviously the code that
fails to finish is mediawiki's code, so it doesn't seem to be a generic
issue with php8.1 except if the timeout happens because of something
inside php8.1. Can anybody shed a light on if it's reasonable to time
out on what mediawiki is trying to do here. And why doesn't it happen on
other architectures (in Debian, as far as I checked)?

Paul

[Paul Gevers]

Hi Ondřej,

My NMU of php-igbinary fails to migrate to testing because you added a
Breaks on << 3.2.6+2.0.8-6~ (where my NMU has version 3.2.6+2.0.8-5.1).
Can you please fix this somehow?

Paul

[Kunal Mehta]

Hi,

On 1/19/22 14:08, Bryce Harrington wrote:

> On Wed, Jan 19, 2022 at 09:58:52PM +0100, Paul Gevers wrote:
>> Hi Bryce,
>>
>> On 19-01-2022 10:28, Bryce Harrington wrote:
>>
>>> With [4] applied, I'm seeing the following dumped on armhf:
>>>
>>> ## https://autopkgtest.ubuntu.com/packages/m/mediawiki/jammy/armhf
>>> cat: /var/log/mediawiki/error.log: No such file or directory
>>> 2022-01-19 09:16:57 autopkgtest-lxd-eeoxik autopkgtestwiki: [e33d1ec89af515673078437e] /mediawiki/index.php/Main_Page PHP Fatal Error from line 110 of /usr/share/mediawiki/includes/json/FormatJson.php: Maximum execution time of 30 seconds exceeded
>>> #0 [internal function]: MWExceptionHandler::handleFatalError()
>>> #1 {main}
>>> 2022-01-19 09:17:27 autopkgtest-lxd-eeoxik autopkgtestwiki: [1284cbae5a84eb5387f33003] /mediawiki/index.php/Main_Page PHP Fatal Error from line 152 of /usr/share/mediawiki/includes/HookContainer/HookContainer.php: Maximum execution time of 30 seconds exceeded
>>> #0 [internal function]: MWExceptionHandler::handleFatalError()
>>> #1 {main}
>>> 2022-01-19 09:17:57 autopkgtest-lxd-eeoxik autopkgtestwiki: [31731278ed6070e946af4fbe] /mediawiki/index.php/Main_Page PHP Fatal Error from line 110 of /usr/share/mediawiki/includes/json/FormatJson.php: Maximum execution time of 30 seconds exceeded
>>> #0 [internal function]: MWExceptionHandler::handleFatalError()
>>> #1 {main}
>>> cat: /var/log/mediawiki/fatal.log: No such file or directory

Thanks for applying the patch and testing it!

>> As this issue seems intermittent (as mediawiki passed at one point) I guess
>> you're trying to say that you think mediawiki got slower with php8.1? Or is
>> this timeout new with php8.1?
>

> The failing cases are mostly ones with triggers for
> php-defaults/84~0ubuntu1 or newer, which marks where php8.1 is set as
> default. The ones that pass and don't specify that are thus running
> php8.0. So it looks like the pass-vs-fail correlates to php8.0-vs-8.1
> and is something specific to armhf.
>
> Unfortunately I don't have i386 builds to compare with (due to
> dependency issues), so it's just a hunch that this is the same problem
> Debian sees on i386. It'd be illuminating to doublecheck on your i386
> builds that it's also hitting this same timeout situation.

>
>> Obviously the code that fails to finish is mediawiki's code, so it
>> doesn't seem to be a generic issue with php8.1 except if the timeout
>> happens because of something inside php8.1.
>

> Line 110 that the error points to appears to be a json_encode() call.

>
>> Can anybody shed a light on if it's reasonable to time out on what
>> mediawiki is trying to do here. And why doesn't it happen on other
>> architectures (in Debian, as far as I checked)?

MediaWiki is just loading the page, which in theory should be cheap
(<1s), but since it's the first page load it will be filling empty
caches and running background tasks which could be expensive. When I
grepped for FormatJson::encode, I came up with SpecialRunJobs
(background tasks) and MessageBlobStore (caching) as code paths that
would get hit, and both have the potential to be slow/add extra time to
page loads.

> I can't tell what it may be trying to encode, but presumably it's either
> Main_Page or something used by Main_Page, which I'm guessing should only
> take a fraction of a second to encode. I suppose we could test
> increasing max_execution_time. But my suspicion is that the encoder is
> getting stuck in a loop or a recursion.

30 seconds seems a bit absurd, but given an empty opcache/jit, empty
caches, on 32-bit VMs, it's within the realm of possibility...I think
bumping max_execution_time is a good next step. Should the test add a
php.ini snippet or something?

I read through the 8.1 changelog
(<https://www.php.net/ChangeLog-8.php#PHP_8_1>) and couldn't find
anything relevant that wasn't already in PHP 8.0.8 (specifically
<https://github.com/php/php-src/commit/bf9dc534356ec3b372abe5f3030c3e362aa261ac>
looked suspect, but that was included in 8.0.7).

Just to give some expectations, I won't have time to dig into this until
the weekend. I did drop a pointer to the bug in the MediaWiki developers
IRC channel in case anyone there had suggestions.

-- Kunal

[Paul Gevers]

Hi Kunal,

Thanks for the analysis.

On 20-01-2022 10:56, Kunal Mehta wrote:
>> I can't tell what it may be trying to encode, but presumably it's either
>> Main_Page or something used by Main_Page, which I'm guessing should only
>> take a fraction of a second to encode.  I suppose we could test
>> increasing max_execution_time.  But my suspicion is that the encoder is
>> getting stuck in a loop or a recursion.
>
> 30 seconds seems a bit absurd, but given an empty opcache/jit, empty
> caches, on 32-bit VMs, it's within the realm of possibility...I think
> bumping max_execution_time is a good next step. Should the test add a
> php.ini snippet or something?

I think that would be great. But please upload to Debian as well, let's
not only have these tests in Ubuntu.

> I read through the 8.1 changelog
> (<https://www.php.net/ChangeLog-8.php#PHP_8_1>) and couldn't find
> anything relevant that wasn't already in PHP 8.0.8 (specifically
> <https://github.com/php/php-src/commit/bf9dc534356ec3b372abe5f3030c3e362aa261ac>
> looked suspect, but that was included in 8.0.7).

In Debian we're going from 7.4 to 8.1, so all intermediate upstreams are
relevant.

> Just to give some expectations, I won't have time to dig into this until
> the weekend. I did drop a pointer to the bug in the MediaWiki developers
> IRC channel in case anyone there had suggestions.

Can that php.ini addition to the test happen thought, or is even that to
much of your time? (Not blaming, just asking).

Paul