info
Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
Bug#1060202: glibc - autopkgtest flacky on arm64
0 views
Skip to first unread message
Bastian Blank
Jan 7, 2024, 8:00:04 AMJan 7
to
Package: glibc
Version: 2.37-13
Severity: important
Currently the autopkgtest on arm64 fails sometimes.
Timeout while building:
https://ci.debian.net/packages/g/glibc/unstable/arm64/41516611/
Failed test:
https://ci.debian.net/packages/g/glibc/testing/arm64/40439311/
--
There is an order of things in this universe.
-- Apollo, "Who Mourns for Adonais?" stardate 3468.1
Version: 2.37-13
Severity: important
Currently the autopkgtest on arm64 fails sometimes.
Timeout while building:
https://ci.debian.net/packages/g/glibc/unstable/arm64/41516611/
Failed test:
https://ci.debian.net/packages/g/glibc/testing/arm64/40439311/
--
There is an order of things in this universe.
-- Apollo, "Who Mourns for Adonais?" stardate 3468.1
Aurelien Jarno
Jan 7, 2024, 12:30:05 PMJan 7
to
Hi,
On 2024-01-07 13:56, Bastian Blank wrote:
> Package: glibc
> Version: 2.37-13
> Severity: important
>
> Currently the autopkgtest on arm64 fails sometimes.
>
> Timeout while building:
> https://ci.debian.net/packages/g/glibc/unstable/arm64/41516611/
There are indeed many failures with cc1 getting killed, it seems that it
started around 2024-01-02. I haven't spotted any change to the toolchain
nor kernel version.
I am not able to reproduce the issue, so it is very likely linked to
debci. Would it be possible to now why cc1 get killed? OOM? timeout?
> Failed test:
> https://ci.debian.net/packages/g/glibc/testing/arm64/40439311/
This is likely due to a too loaded host.
Regards
Aurelien
--
Aurelien Jarno GPG: 4096R/1DDD8C9B
aure...@aurel32.net http://aurel32.net
On 2024-01-07 13:56, Bastian Blank wrote:
> Package: glibc
> Version: 2.37-13
> Severity: important
>
> Currently the autopkgtest on arm64 fails sometimes.
>
> Timeout while building:
> https://ci.debian.net/packages/g/glibc/unstable/arm64/41516611/
started around 2024-01-02. I haven't spotted any change to the toolchain
nor kernel version.
I am not able to reproduce the issue, so it is very likely linked to
debci. Would it be possible to now why cc1 get killed? OOM? timeout?
> Failed test:
> https://ci.debian.net/packages/g/glibc/testing/arm64/40439311/
This is likely due to a too loaded host.
Regards
Aurelien
--
Aurelien Jarno GPG: 4096R/1DDD8C9B
aure...@aurel32.net http://aurel32.net
Aurelien Jarno
Jan 7, 2024, 7:50:04 PMJan 7
to
On 2024-01-07 21:59, Paul Gevers wrote:
> Hi,
>
Thanks that's very useful. It appears that the tgmath3-fma test got
killed after using 2611964kB. In practice it needs a few MB more, and
that's not something that has changed recently, it needs about the same
amount of memory with gcc-12 from bookworm. This test is known to take a
lot of memory to compile, and on amd64, it needs around 3.2GB.
I am still not sure why it got killed on arm64 and not on other debci
workers, there as still swap available. Actually looking at the munin
plot, it seems that the arm64 debci workers stopped using swap around
September 2023 contrary to the other architectures.
The good news is that it seems that gcc-13 needs a bit less RAM to
compile that file. We can switch to it once we get glibc 2.38 into
testing.
> Hi,
>
> On 07-01-2024 18:21, Aurelien Jarno wrote:
> > > Timeout while building:
> > > https://ci.debian.net/packages/g/glibc/unstable/arm64/41516611/
> >
> On 07-01-2024 18:21, Aurelien Jarno wrote:
> > > Timeout while building:
> > > https://ci.debian.net/packages/g/glibc/unstable/arm64/41516611/
> >
> > There are indeed many failures with cc1 getting killed, it seems that it
> > started around 2024-01-02. I haven't spotted any change to the toolchain
> > nor kernel version.
> >
> > I am not able to reproduce the issue, so it is very likely linked to
> > debci. Would it be possible to now why cc1 get killed? OOM? timeout?
>
> I extracted the attached journal piece indicating OOM.
> > started around 2024-01-02. I haven't spotted any change to the toolchain
> > nor kernel version.
> >
> > I am not able to reproduce the issue, so it is very likely linked to
> > debci. Would it be possible to now why cc1 get killed? OOM? timeout?
>
>
Thanks that's very useful. It appears that the tgmath3-fma test got
killed after using 2611964kB. In practice it needs a few MB more, and
that's not something that has changed recently, it needs about the same
amount of memory with gcc-12 from bookworm. This test is known to take a
lot of memory to compile, and on amd64, it needs around 3.2GB.
I am still not sure why it got killed on arm64 and not on other debci
workers, there as still swap available. Actually looking at the munin
plot, it seems that the arm64 debci workers stopped using swap around
September 2023 contrary to the other architectures.
The good news is that it seems that gcc-13 needs a bit less RAM to
compile that file. We can switch to it once we get glibc 2.38 into
testing.
Aurelien Jarno
Jan 8, 2024, 3:20:05 PMJan 8
to
On 2024-01-08 19:50, Paul Gevers wrote:
> Hi,
>
> Hi,
>
> On 08-01-2024 01:45, Aurelien Jarno wrote:
> > I am still not sure why it got killed on arm64 and not on other debci
> > workers, there as still swap available. Actually looking at the munin
> > plot, it seems that the arm64 debci workers stopped using swap around
> > September 2023 contrary to the other architectures.
>
> Can you tell me how you saw that? I neither spotted that, nor is not having
> > I am still not sure why it got killed on arm64 and not on other debci
> > workers, there as still swap available. Actually looking at the munin
> > plot, it seems that the arm64 debci workers stopped using swap around
> > September 2023 contrary to the other architectures.
>
> swap a conscious act, so rather a mistake.
From the log you send, I was surprised that OOM happened while none of
the swap has been used at all. So I looked at the munin link you gave me
by IRC and realized that the Swap in/out plot is stuck as 0.00 for
min/avg/max values since September 2023. This is not the case of the
workers of the other architectures which use a bit of swap from time to
time, and this have a maximum different of 0.00.
That still doesn't explain while the problem with glibc seems to have
started one week ago.
Paul Gevers
Jan 8, 2024, 3:40:04 PMJan 8
to
Hi,
On 08-01-2024 01:45, Aurelien Jarno wrote:
> I am still not sure why it got killed on arm64 and not on other debci
> workers, there as still swap available. Actually looking at the munin
> plot, it seems that the arm64 debci workers stopped using swap around
> September 2023 contrary to the other architectures.
This does seem to align with us upgrading from the bookworm kernel to
> workers, there as still swap available. Actually looking at the munin
> plot, it seems that the arm64 debci workers stopped using swap around
> September 2023 contrary to the other architectures.
the backports kernel because of bug 1050256.
> That still doesn't explain while the problem with glibc seems to
> have started one week ago.
Paul
0 new messages