💥 PANICKED: Test report for kernel 5.11.0-rc6 (mainline.kernel.org-clang)

[CKI Project]

Hello,

We ran automated tests on a recent commit from this kernel tree:

Kernel repo: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
Commit: 3aaf0a27ffc2 - Merge tag 'clang-format-for-linux-v5.11-rc7' of git://github.com/ojeda/linux

The results of these automated tests are provided below.

Overall result: FAILED (see details below)
Merge: OK
Compile: OK
Selftests compile: FAILED
Tests: PANICKED

All kernel binaries, config files, and logs are available for download here:

https://arr-cki-prod-datawarehouse-public.s3.amazonaws.com/index.html?prefix=datawarehouse-public/2021/02/02/622905

One or more kernel tests failed:

aarch64:
❌ selinux-policy: serge-testsuite

x86_64:
❌ selinux-policy: serge-testsuite
❌ LTP lite
💥 LTP

We hope that these logs can help you find the problem quickly. For the full
detail on our testing procedures, please scroll to the bottom of this message.

Please reply to this email if you have any questions about the tests that we
ran or if you have any suggestions on how to make future tests more effective.

,-. ,-.
( C ) ( K ) Continuous
`-',-.`-' Kernel
( I ) Integration
`-'
______________________________________________________________________________

Compile testing
---------------

We compiled the kernel for 3 architectures:

aarch64:
make options: make LLVM=1 -j30 INSTALL_MOD_STRIP=1 targz-pkg

ppc64le:
make options: make CC=clang -j30 INSTALL_MOD_STRIP=1 targz-pkg

x86_64:
make options: make LLVM=1 -j30 INSTALL_MOD_STRIP=1 targz-pkg


We built the following selftests:

x86_64:
net: fail
bpf: fail
install and packaging: fail

You can find the full log (build-selftests.log) in the artifact storage above.


Hardware testing
----------------
We booted each kernel and ran the following tests:

aarch64:
Host 1:
✅ Boot test
❌ selinux-policy: serge-testsuite
✅ lvm thinp sanity
✅ storage: software RAID testing
✅ stress: stress-ng
🚧 ✅ xfstests - ext4
🚧 ✅ xfstests - xfs
🚧 ✅ Storage blktests
🚧 ❌ Storage nvme - rdma
🚧 ✅ Storage nvme - tcp
🚧 ✅ Storage: swraid mdadm raid_module test
🚧 ❌ storage: iSCSI parameters

Host 2:

⚡ Internal infrastructure issues prevented one or more tests (marked
with ⚡⚡⚡) from running on this architecture.
This is not the fault of the kernel that was tested.

✅ Boot test
⚡⚡⚡ kdump - sysrq-c
🚧 ⚡⚡⚡ kdump - file-load

Host 3:

⚡ Internal infrastructure issues prevented one or more tests (marked
with ⚡⚡⚡) from running on this architecture.
This is not the fault of the kernel that was tested.

✅ Boot test
✅ ACPI table test
✅ LTP lite
⚡⚡⚡ LTP
⚡⚡⚡ Loopdev Sanity
⚡⚡⚡ Memory function: memfd_create
⚡⚡⚡ AMTU (Abstract Machine Test Utility)
⚡⚡⚡ Networking bridge: sanity
⚡⚡⚡ Networking MACsec: sanity
⚡⚡⚡ Networking socket: fuzz
⚡⚡⚡ Networking sctp-auth: sockopts test
⚡⚡⚡ Networking: igmp conformance test
⚡⚡⚡ Networking route: pmtu
⚡⚡⚡ Networking route_func - local
⚡⚡⚡ Networking route_func - forward
⚡⚡⚡ Networking TCP: keepalive test
⚡⚡⚡ Networking UDP: socket
⚡⚡⚡ Networking tunnel: geneve basic test
⚡⚡⚡ Networking tunnel: gre basic
⚡⚡⚡ L2TP basic test
⚡⚡⚡ Networking tunnel: vxlan basic
⚡⚡⚡ Networking ipsec: basic netns - transport
⚡⚡⚡ Networking ipsec: basic netns - tunnel
⚡⚡⚡ Libkcapi AF_ALG test
⚡⚡⚡ pciutils: update pci ids test
⚡⚡⚡ ALSA PCM loopback test
⚡⚡⚡ ALSA Control (mixer) Userspace Element test
⚡⚡⚡ Usex - version 1.9-29
⚡⚡⚡ storage: SCSI VPD
⚡⚡⚡ Syscalls: nrdiff
🚧 ⚡⚡⚡ CIFS Connectathon
🚧 ⚡⚡⚡ POSIX pjd-fstest suites
🚧 ⚡⚡⚡ Trinity Fuzzer - Tier1
🚧 ⚡⚡⚡ Memory function: kaslr
🚧 ⚡⚡⚡ LTP: openposix test suite
🚧 ⚡⚡⚡ Ethernet drivers sanity
🚧 ⚡⚡⚡ Networking: ipv6/Fujitsu-socketapi-test
🚧 ⚡⚡⚡ Networking firewall: basic netfilter test
🚧 ⚡⚡⚡ audit: audit testsuite test
🚧 ⚡⚡⚡ storage: dm/common
🚧 ⚡⚡⚡ trace: ftrace/tracer
🚧 ⚡⚡⚡ Libhugetlbfs - version 2.2.1
🚧 ⚡⚡⚡ kdump - kexec_boot

Host 4:

⚡ Internal infrastructure issues prevented one or more tests (marked
with ⚡⚡⚡) from running on this architecture.
This is not the fault of the kernel that was tested.

✅ Boot test
⚡⚡⚡ kdump - sysrq-c
🚧 ⚡⚡⚡ kdump - file-load

Host 5:

⚡ Internal infrastructure issues prevented one or more tests (marked
with ⚡⚡⚡) from running on this architecture.
This is not the fault of the kernel that was tested.

✅ Boot test
⚡⚡⚡ kdump - sysrq-c
🚧 ⚡⚡⚡ kdump - file-load

ppc64le:
Host 1:

⚡ Internal infrastructure issues prevented one or more tests (marked
with ⚡⚡⚡) from running on this architecture.
This is not the fault of the kernel that was tested.

⚡⚡⚡ Boot test
⚡⚡⚡ kdump - sysrq-c
🚧 ⚡⚡⚡ kdump - file-load

Host 2:

⚡ Internal infrastructure issues prevented one or more tests (marked
with ⚡⚡⚡) from running on this architecture.
This is not the fault of the kernel that was tested.

⚡⚡⚡ Boot test
⚡⚡⚡ LTP lite
⚡⚡⚡ LTP
⚡⚡⚡ Loopdev Sanity
⚡⚡⚡ Memory function: memfd_create
⚡⚡⚡ AMTU (Abstract Machine Test Utility)
⚡⚡⚡ Networking bridge: sanity
⚡⚡⚡ Networking MACsec: sanity
⚡⚡⚡ Networking socket: fuzz
⚡⚡⚡ Networking sctp-auth: sockopts test
⚡⚡⚡ Networking route: pmtu
⚡⚡⚡ Networking route_func - local
⚡⚡⚡ Networking route_func - forward
⚡⚡⚡ Networking TCP: keepalive test
⚡⚡⚡ Networking UDP: socket
⚡⚡⚡ Networking tunnel: geneve basic test
⚡⚡⚡ Networking tunnel: gre basic
⚡⚡⚡ L2TP basic test
⚡⚡⚡ Networking tunnel: vxlan basic
⚡⚡⚡ Networking ipsec: basic netns - tunnel
⚡⚡⚡ Libkcapi AF_ALG test
⚡⚡⚡ pciutils: update pci ids test
⚡⚡⚡ ALSA PCM loopback test
⚡⚡⚡ ALSA Control (mixer) Userspace Element test
⚡⚡⚡ Usex - version 1.9-29
⚡⚡⚡ Syscalls: nrdiff
🚧 ⚡⚡⚡ CIFS Connectathon
🚧 ⚡⚡⚡ POSIX pjd-fstest suites
🚧 ⚡⚡⚡ Trinity Fuzzer - Tier1
🚧 ⚡⚡⚡ Memory function: kaslr
🚧 ⚡⚡⚡ LTP: openposix test suite
🚧 ⚡⚡⚡ Ethernet drivers sanity
🚧 ⚡⚡⚡ Networking: ipv6/Fujitsu-socketapi-test
🚧 ⚡⚡⚡ Networking firewall: basic netfilter test
🚧 ⚡⚡⚡ audit: audit testsuite test
🚧 ⚡⚡⚡ storage: dm/common
🚧 ⚡⚡⚡ trace: ftrace/tracer
🚧 ⚡⚡⚡ Libhugetlbfs - version 2.2.1

Host 3:

⚡ Internal infrastructure issues prevented one or more tests (marked
with ⚡⚡⚡) from running on this architecture.
This is not the fault of the kernel that was tested.

⚡⚡⚡ Boot test
⚡⚡⚡ selinux-policy: serge-testsuite
⚡⚡⚡ lvm thinp sanity
⚡⚡⚡ storage: software RAID testing
🚧 ⚡⚡⚡ xfstests - ext4
🚧 ⚡⚡⚡ xfstests - xfs
🚧 ⚡⚡⚡ Storage blktests
🚧 ⚡⚡⚡ Storage nvme - rdma
🚧 ⚡⚡⚡ Storage nvme - tcp
🚧 ⚡⚡⚡ Storage: swraid mdadm raid_module test

Host 4:

⚡ Internal infrastructure issues prevented one or more tests (marked
with ⚡⚡⚡) from running on this architecture.
This is not the fault of the kernel that was tested.

⚡⚡⚡ Boot test
⚡⚡⚡ selinux-policy: serge-testsuite
⚡⚡⚡ lvm thinp sanity
⚡⚡⚡ storage: software RAID testing
🚧 ⚡⚡⚡ xfstests - ext4
🚧 ⚡⚡⚡ xfstests - xfs
🚧 ⚡⚡⚡ Storage blktests
🚧 ⚡⚡⚡ Storage nvme - rdma
🚧 ⚡⚡⚡ Storage nvme - tcp
🚧 ⚡⚡⚡ Storage: swraid mdadm raid_module test

Host 5:

⚡ Internal infrastructure issues prevented one or more tests (marked
with ⚡⚡⚡) from running on this architecture.
This is not the fault of the kernel that was tested.

⚡⚡⚡ Boot test
⚡⚡⚡ selinux-policy: serge-testsuite
⚡⚡⚡ lvm thinp sanity
⚡⚡⚡ storage: software RAID testing
🚧 ⚡⚡⚡ xfstests - ext4
🚧 ⚡⚡⚡ xfstests - xfs
🚧 ⚡⚡⚡ Storage blktests
🚧 ⚡⚡⚡ Storage nvme - rdma
🚧 ⚡⚡⚡ Storage nvme - tcp
🚧 ⚡⚡⚡ Storage: swraid mdadm raid_module test

x86_64:
Host 1:

⚡ Internal infrastructure issues prevented one or more tests (marked
with ⚡⚡⚡) from running on this architecture.
This is not the fault of the kernel that was tested.

✅ Boot test
❌ selinux-policy: serge-testsuite
✅ lvm thinp sanity
✅ storage: software RAID testing
✅ stress: stress-ng
🚧 ✅ xfstests - ext4
🚧 ✅ xfstests - xfs
🚧 ✅ Storage blktests
🚧 ❌ Storage nvme - rdma
🚧 ❌ Storage nvme - tcp
🚧 ⚡⚡⚡ Storage nvdimm ndctl test suite
🚧 ✅ Storage: swraid mdadm raid_module test
🚧 ❌ storage: iSCSI parameters

Host 2:
✅ Boot test
✅ kdump - sysrq-c
🚧 ✅ kdump - file-load

Host 3:
✅ Boot test
✅ ACPI table test
❌ LTP lite
💥 LTP
⚡⚡⚡ Loopdev Sanity
⚡⚡⚡ Memory function: memfd_create
⚡⚡⚡ AMTU (Abstract Machine Test Utility)
⚡⚡⚡ Networking bridge: sanity
⚡⚡⚡ Networking MACsec: sanity
⚡⚡⚡ Networking socket: fuzz
⚡⚡⚡ Networking sctp-auth: sockopts test
⚡⚡⚡ Networking: igmp conformance test
⚡⚡⚡ Networking route: pmtu
⚡⚡⚡ Networking route_func - local
⚡⚡⚡ Networking route_func - forward
⚡⚡⚡ Networking TCP: keepalive test
⚡⚡⚡ Networking UDP: socket
⚡⚡⚡ Networking tunnel: geneve basic test
⚡⚡⚡ Networking tunnel: gre basic
⚡⚡⚡ L2TP basic test
⚡⚡⚡ Networking tunnel: vxlan basic
⚡⚡⚡ Networking ipsec: basic netns - transport
⚡⚡⚡ Networking ipsec: basic netns - tunnel
⚡⚡⚡ Libkcapi AF_ALG test
⚡⚡⚡ pciutils: sanity smoke test
⚡⚡⚡ pciutils: update pci ids test
⚡⚡⚡ ALSA PCM loopback test
⚡⚡⚡ ALSA Control (mixer) Userspace Element test
⚡⚡⚡ Usex - version 1.9-29
⚡⚡⚡ storage: SCSI VPD
⚡⚡⚡ Syscalls: nrdiff
🚧 ⚡⚡⚡ CIFS Connectathon
🚧 ⚡⚡⚡ POSIX pjd-fstest suites
🚧 ⚡⚡⚡ Trinity Fuzzer - Tier1
🚧 ⚡⚡⚡ Memory function: kaslr
🚧 ⚡⚡⚡ LTP: openposix test suite
🚧 ⚡⚡⚡ Ethernet drivers sanity
🚧 ⚡⚡⚡ Networking: ipv6/Fujitsu-socketapi-test
🚧 ⚡⚡⚡ Networking firewall: basic netfilter test
🚧 ⚡⚡⚡ audit: audit testsuite test
🚧 ⚡⚡⚡ storage: dm/common
🚧 ⚡⚡⚡ trace: ftrace/tracer
🚧 ⚡⚡⚡ Libhugetlbfs - version 2.2.1
🚧 ⚡⚡⚡ Tracepoints: operational test
🚧 ⚡⚡⚡ kdump - kexec_boot

Test sources: https://gitlab.com/cki-project/kernel-tests
💚 Pull requests are welcome for new tests or improvements to existing tests!

Aborted tests
-------------
Tests that didn't complete running successfully are marked with ⚡⚡⚡.
If this was caused by an infrastructure issue, we try to mark that
explicitly in the report.

Waived tests
------------
If the test run included waived tests, they are marked with 🚧. Such tests are
executed but their results are not taken into account. Tests are waived when
their results are not reliable enough, e.g. when they're just introduced or are
being fixed.

Testing timeout
---------------
We aim to provide a report within reasonable timeframe. Tests that haven't
finished running yet are marked with ⏱.

[Nathan Chancellor]

On Wed, Feb 03, 2021 at 01:58:32AM -0000, CKI Project wrote:
>
> Hello,
>
> We ran automated tests on a recent commit from this kernel tree:
>
> Kernel repo: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
> Commit: 3aaf0a27ffc2 - Merge tag 'clang-format-for-linux-v5.11-rc7' of git://github.com/ojeda/linux
>
> The results of these automated tests are provided below.
>
> Overall result: FAILED (see details below)
> Merge: OK
> Compile: OK
> Selftests compile: FAILED
> Tests: PANICKED
>
> All kernel binaries, config files, and logs are available for download here:
>
> https://arr-cki-prod-datawarehouse-public.s3.amazonaws.com/index.html?prefix=datawarehouse-public/2021/02/02/622905
>
> One or more kernel tests failed:
>
> aarch64:
> ❌ selinux-policy: serge-testsuite
>
> x86_64:
> ❌ selinux-policy: serge-testsuite
> ❌ LTP lite

These should be the failures for this particular set of tests correct?

https://arr-cki-prod-datawarehouse-public.s3.amazonaws.com/datawarehouse-public/2021/02/02/622905/build_x86_64_redhat%3A1092370/tests/LTP_lite/9507327_x86_64_1_RHELKT1LITE.FILTERED.fail.log

I am curious, is there any way to get a side by side comparison of the
test results between GCC and clang? In other words, I would like to know
if a test is failing with both compilers versus just clang. This would
help us understand if it is a compiler specific issue or something that
broke the kernel.

> 💥 LTP

From what I can tell:

[12262.428593] general protection fault, probably for non-canonical address 0x8bc08b308b608b5: 0000 [#1] SMP PTI
[12262.428810] CPU: 1 PID: 465515 Comm: proc01 Not tainted 5.11.0-rc6 #1
[12262.428810] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2007
[12262.428810] RIP: 0010:string+0x50/0x110
[12262.428810] Code: ce 48 c1 fe 30 85 f6 0f 84 be 00 00 00 31 c0 eb 15 66 0f 1f 84 00 00 00 00 00 48 83 c0 01 39 c6 0f 84 ac 00 00 00 4c 8d 0c 07 <45> 0f b6 14 00 45 84 d2 0f 84 a2 00 00 00 49 39 d1 73 dd 45 88 11
[12262.428810] RSP: 0018:ffffb1ea012bbc40 EFLAGS: 00010246
[12262.428810] RAX: 0000000000000000 RBX: ffff9ea25e9ea03e RCX: ffff0a00ffffff04
[12262.428810] RDX: ffff9ea25e9eb000 RSI: ffffffffffffffff RDI: ffff9ea25e9ea03e
[12262.428810] RBP: ffffffff8f5c95f1 R08: 08bc08b308b608b5 R09: ffff9ea25e9ea03e
[12262.428810] R10: 0000001000000000 R11: ffffffff8e212401 R12: ffff9ea25e9eb000
[12262.428810] R13: ffffffff8f5c95f3 R14: 0000000000000002 R15: ffffb1ea012bbca0
[12262.428810] FS: 00007fab43075740(0000) GS:ffff9ea35bd00000(0000) knlGS:0000000000000000
[12262.428810] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[12262.428810] CR2: 00005584bb6dba68 CR3: 0000000111bbe000 CR4: 00000000000006e0
[12262.428810] DR0: 0000000000000001 DR1: 0000000000000000 DR2: 0000000000000000
[12262.428810] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000600
[12262.428810] Call Trace:
[12262.428810] vsnprintf+0x570/0x710
[12262.428810] seq_printf+0x6b/0x90
[12262.428810] ? lock_acquire+0x27/0x280
[12262.428810] ? mod_objcg_state+0xd2/0x160
[12262.428810] ? vsnprintf+0x32f/0x710
[12262.428810] print_name+0x46/0xc0
[12262.428810] ? seq_printf+0x6b/0x90
[12262.428810] ? __kmalloc_node+0xaf/0x340
[12262.428810] ? lock_acquire+0x27/0x280
[12262.428810] ? kvmalloc_node+0x4b/0x90
[12262.428810] lc_show+0x82/0xe0
[12262.428810] seq_read_iter+0x311/0x420
[12262.428810] proc_reg_read_iter+0x3f/0x80
[12262.428810] vfs_read+0x2c3/0x340
[12262.428810] ksys_read+0x5f/0xb0
[12262.428810] do_syscall_64+0x33/0x40
[12262.428810] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[12262.428810] RIP: 0033:0x7fab4318791d
[12262.428810] Code: 0f 1f 44 00 00 48 8b 0d 51 45 0b 00 45 31 c0 64 83 39 0b 75 d2 eb c0 e8 b1 fb ff ff 90 f3 0f 1e fa 48 39 ca 77 2b 31 c0 0f 05 <48> 3d 00 f0 ff ff 77 0b c3 66 2e 0f 1f 84 00 00 00 00 00 48 8b 15
[12262.428810] RSP: 002b:00007ffcaeb265b8 EFLAGS: 00000246 ORIG_RAX: 0000000000000000
[12262.428810] RAX: ffffffffffffffda RBX: 00007ffcaeb27730 RCX: 00007fab4318791d
[12262.428810] RDX: 0000000000000400 RSI: 000000000044b820 RDI: 0000000000000005
[12262.428810] RBP: 0000000000008000 R08: 0000000000000000 R09: 00007ffcaeb27520
[12262.428810] R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
[12262.428810] R13: 0000000000000005 R14: 0000000000000000 R15: 0000000000000000
[12262.428810] Modules linked in: snd_seq_dummy minix snd_hrtimer snd_seq snd_seq_device authenc pcrypt crypto_user sha3_generic algif_hash salsa20_generic binfmt_misc n_gsm pps_ldisc slcan ppp_synctty n_hdlc mkiss ax25 ppp_async ppp_generic slip slhc nfsv3 nfs_acl nfs nfs_ssc fscache lockd grace rds tun brd overlay exfat vfat fat xfs sctp tcp_diag udp_diag inet_diag rfkill sunrpc snd_hda_codec_generic ledtrig_audio snd_hda_intel snd_intel_dspcfg soundwire_intel soundwire_generic_allocation snd_soc_core snd_compress snd_pcm_dmaengine soundwire_cadence snd_hda_codec snd_hda_core ac97_bus snd_hwdep snd_pcm joydev snd_timer snd virtio_net soundcore net_failover pcspkr virtio_balloon failover i2c_piix4 fuse zram ip_tables x_tables cirrus drm_kms_helper cec virtio_blk drm serio_raw ata_generic pata_acpi floppy btrfs raid6_pq xor [last unloaded: can]
[12262.462039] ---[ end trace f683b95858968bee ]---

I cannot reproduce this in QEMU with a Debian rootfs with the supplied
configuration file though, proc01 -m 128 passes for me.

Cheers,
Nathan

> --
> You received this message because you are subscribed to the Google Groups "Clang Built Linux" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to clang-built-li...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/cki.C90653A6EF.V9S14CF2MY%40redhat.com.

[Rachel Sibley]

Hi Nathan,

We recently moved the mainline tree to use Fedora Rawhide as the base distro, this helps out kdump and other tests which require
updated user space for testing. As a result, there were a few tests that shouldn't have run like LTP Lite. So the log you linked
above is not related to the panic:
https://arr-cki-prod-datawarehouse-public.s3.amazonaws.com/datawarehouse-public/2021/02/02/622905/build_x86_64_redhat%3A1092370/tests/LTP_lite/9507327_x86_64_1_RHELKT1LITE.FILTERED.fail.log

This is the version of LTP we run for internal RHEL kernels and therefore isn't supported against upstream kernels. I'm working on a fix
for this now and should be resolved soon, sorry about that.

As far as the panic, it was was triggered on our upstream LTP test, I don't have a corresponding x86_64/LTP mainline(GCC) job since there is a BPF
bug which is causing our jobs to abort while updating the kernel for x86. You will see the following in the logs:

'failed to validate module [something] BTF: -22 '

Here is a regular mainline job for reference where you can it:
https://arr-cki-prod-datawarehouse-public.s3.amazonaws.com/datawarehouse-public/2021/02/02/622904/build_x86_64_redhat%3A1092352/tests/9507513_x86_64_1_console.log

As far as comparing tests for mainline run with gcc vs clang, we don't email an upstream ML for regular mainline, so nothing archived.
However, the logs are still published externally and we can always link them here for comparison.

If you like, we could start forwarding you reports for both ? Otherwise, we are working on publishing our results database we use internally,
but this will take some time.

Thanks,
Rachel

[Sedat Dilek]

Guess this is with CONFIG_DEBUG_INFO_BTF=y and LLVM/Clang?

Today, I played with above and diverse DWARF version settings.

Even recent pahole from Git in combination with up-to-date LLVM/Clang is incompatible with DWARF-v4 and DWARF-v5.

Means all my builds were broken.

Mark Wielaard says:

"I would try to avoid using clang producing DWARF5. It clearly has some
incompatibilities with dwarves/pahole. It should work if you don't set
DEBUG_INFO_DWARF5. Try GCC 11 (which defaults to -gdwarf-5) or an
earlier version (probably at least GCC 8 or higher) using -gdwarf-5
explicitly."

If this is with CONFIG_DEBUG_INFO_BTF=y you should make it unavailable when CC_IS_CLANG=y.

- Sedat -

 

To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/5877bb33-ba17-6459-410c-1c51bd193481%40redhat.com.

[Nick Desaulniers]

I'm attending BPF office hours tomorrow (morning, California time) to discuss this with the pahole maintainers.

https://lore.kernel.org/bpf/CAADnVQ+weKq=-=KB7j=0FfCE0bLanJ_ppn...@mail.gmail.com/

 

To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/CA%2BicZUWTia0CS5nrdGx7oE%2BZj_J3oC4id6U3jpVbUgqVosoFxw%40mail.gmail.com.

--

Thanks,

~Nick Desaulniers

[Sedat Dilek]

Great.

Bonne chance.

Just reported to linux-bpf ML.

- Sedat -

[1]  https://lore.kernel.org/bpf/CAEf4BzZ-uU3vkMA1RPt1f2Hb...@mail.gmail.com/T/#m2a5f11c5bb6ce66b3c59a1be2005dd40724d1d8b

[Nathan Chancellor]

On Wed, Feb 03, 2021 at 10:36:32PM +0100, Sedat Dilek wrote:
> On Wed, Feb 3, 2021 at 9:45 PM Rachel Sibley <rasi...@redhat.com> wrote:

[snip]

> > As far as the panic, it was was triggered on our upstream LTP test, I
> > don't have a corresponding x86_64/LTP mainline(GCC) job since there is a BPF
> > bug which is causing our jobs to abort while updating the kernel for x86.
> > You will see the following in the logs:
> >
> > 'failed to validate module [something] BTF: -22 '
> >
>
> Guess this is with CONFIG_DEBUG_INFO_BTF=y and LLVM/Clang?

I read that sentence as there are issues with BPF issues with GCC, not
LLVM/Clang.

CONFIG_DEBUG_INFO_BTF is already disabled for the LLVM jobs:

https://gitlab.com/cki-project/pipeline-definition/-/commit/7d138e9a3aede46f7674476fa1b3ca02a391767b#90e1e97a102a5713d6a68df323738846b425341a_1358_1369

Cheers,
Nathan

[Nathan Chancellor]

Hi Rachel,

Thanks for clarifying :)

> As far as the panic, it was was triggered on our upstream LTP test, I don't have a corresponding x86_64/LTP mainline(GCC) job since there is a BPF
> bug which is causing our jobs to abort while updating the kernel for x86. You will see the following in the logs:
>
> 'failed to validate module [something] BTF: -22 '
>
> Here is a regular mainline job for reference where you can it:
> https://arr-cki-prod-datawarehouse-public.s3.amazonaws.com/datawarehouse-public/2021/02/02/622904/build_x86_64_redhat%3A1092352/tests/9507513_x86_64_1_console.log
>
> As far as comparing tests for mainline run with gcc vs clang, we don't email an upstream ML for regular mainline, so nothing archived.
> However, the logs are still published externally and we can always link them here for comparison.
>
> If you like, we could start forwarding you reports for both ? Otherwise, we are working on publishing our results database we use internally,
> but this will take some time.
>

If just adding a link to the GCC logs in the clang reports is not too
much effort for you guys, that would be mighty helpful. I do not mind
sifting through the logs to try and compare results manually (as I just
need to compare the failing clang tests to the GCC run).

If that is not possible for various reasons, please explicitly CC me on
the GCC mainline reports (nat...@kernel.org) and I can just grab the
link myself. Our list gets enough reports as it stands so we can keep
the GCC ones off of this list.

Cheers,
Nathan

[Ondrej Mosnacek]

On Wed, Feb 3, 2021 at 11:06 PM Nathan Chancellor <nat...@kernel.org> wrote:
> On Wed, Feb 03, 2021 at 10:36:32PM +0100, Sedat Dilek wrote:
> > On Wed, Feb 3, 2021 at 9:45 PM Rachel Sibley <rasi...@redhat.com> wrote:
>
> [snip]
>
> > > As far as the panic, it was was triggered on our upstream LTP test, I
> > > don't have a corresponding x86_64/LTP mainline(GCC) job since there is a BPF
> > > bug which is causing our jobs to abort while updating the kernel for x86.
> > > You will see the following in the logs:
> > >
> > > 'failed to validate module [something] BTF: -22 '
> > >
> >
> > Guess this is with CONFIG_DEBUG_INFO_BTF=y and LLVM/Clang?
>
> I read that sentence as there are issues with BPF issues with GCC, not
> LLVM/Clang.

Yes, it's probably this issue:
https://lore.kernel.org/bpf/CAADnVQKbku+Mv++h2TKYZfFN...@mail.gmail.com/T/

>
> CONFIG_DEBUG_INFO_BTF is already disabled for the LLVM jobs:
>
> https://gitlab.com/cki-project/pipeline-definition/-/commit/7d138e9a3aede46f7674476fa1b3ca02a391767b#90e1e97a102a5713d6a68df323738846b425341a_1358_1369
>
> Cheers,
> Nathan
>

--
Ondrej Mosnacek
Software Engineer, Linux Security - SELinux kernel
Red Hat, Inc.

[CKI Project]

Hello,

We ran automated tests on a recent commit from this kernel tree:

Kernel repo: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git

Commit: 3afe9076a7c1 - Merge tag 'arm64-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux

The results of these automated tests are provided below.

Overall result: FAILED (see details below)
Merge: OK
Compile: OK
Selftests compile: FAILED
Tests: PANICKED

All kernel binaries, config files, and logs are available for download here:

https://arr-cki-prod-datawarehouse-public.s3.amazonaws.com/index.html?prefix=datawarehouse-public/2021/02/03/623040

One or more kernel tests failed:

aarch64:
❌ selinux-policy: serge-testsuite

x86_64:

❌ LTP lite
💥 LTP
❌ selinux-policy: serge-testsuite

✅ Boot test
❌ selinux-policy: serge-testsuite
✅ lvm thinp sanity
✅ storage: software RAID testing
✅ stress: stress-ng
🚧 ✅ xfstests - ext4
🚧 ✅ xfstests - xfs
🚧 ✅ Storage blktests
🚧 ❌ Storage nvme - rdma
🚧 ✅ Storage nvme - tcp
🚧 ✅ Storage: swraid mdadm raid_module test
🚧 ❌ storage: iSCSI parameters

Host 4:

⚡ Internal infrastructure issues prevented one or more tests (marked
with ⚡⚡⚡) from running on this architecture.
This is not the fault of the kernel that was tested.

⚡⚡⚡ Boot test
⚡⚡⚡ kdump - sysrq-c
🚧 ⚡⚡⚡ kdump - file-load

Host 5:

⚡ Internal infrastructure issues prevented one or more tests (marked
with ⚡⚡⚡) from running on this architecture.
This is not the fault of the kernel that was tested.

✅ Boot test
⚡⚡⚡ kdump - sysrq-c
🚧 ⚡⚡⚡ kdump - file-load

ppc64le:
Host 1:

⚡ Internal infrastructure issues prevented one or more tests (marked
with ⚡⚡⚡) from running on this architecture.
This is not the fault of the kernel that was tested.

⚡⚡⚡ Boot test

Host 2:

⚡ Internal infrastructure issues prevented one or more tests (marked
with ⚡⚡⚡) from running on this architecture.
This is not the fault of the kernel that was tested.

⚡⚡⚡ Boot test

⚡⚡⚡ selinux-policy: serge-testsuite
⚡⚡⚡ lvm thinp sanity
⚡⚡⚡ storage: software RAID testing
🚧 ⚡⚡⚡ xfstests - ext4
🚧 ⚡⚡⚡ xfstests - xfs
🚧 ⚡⚡⚡ Storage blktests
🚧 ⚡⚡⚡ Storage nvme - rdma
🚧 ⚡⚡⚡ Storage nvme - tcp
🚧 ⚡⚡⚡ Storage: swraid mdadm raid_module test

Host 3:

⚡ Internal infrastructure issues prevented one or more tests (marked
with ⚡⚡⚡) from running on this architecture.
This is not the fault of the kernel that was tested.

⚡⚡⚡ Boot test
⚡⚡⚡ kdump - sysrq-c
🚧 ⚡⚡⚡ kdump - file-load

Host 2:

⚡ Internal infrastructure issues prevented one or more tests (marked
with ⚡⚡⚡) from running on this architecture.
This is not the fault of the kernel that was tested.

✅ Boot test

❌ selinux-policy: serge-testsuite
✅ lvm thinp sanity
✅ storage: software RAID testing
✅ stress: stress-ng
🚧 ✅ xfstests - ext4
🚧 ✅ xfstests - xfs
🚧 ✅ Storage blktests
🚧 ✅ Storage nvme - rdma
🚧 ✅ Storage nvme - tcp
🚧 ⚡⚡⚡ Storage nvdimm ndctl test suite
🚧 ✅ Storage: swraid mdadm raid_module test
🚧 ❌ storage: iSCSI parameters

Host 3:

✅ Boot test
✅ kdump - sysrq-c
🚧 ✅ kdump - file-load

[CKI Project]

Hello,

We ran automated tests on a recent commit from this kernel tree:

Kernel repo: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git

Commit: 61556703b610 - Merge tag 'for-linus-5.11-rc7' of git://git.kernel.org/pub/scm/linux/kernel/git/rw/uml

The results of these automated tests are provided below.

Overall result: FAILED (see details below)
Merge: OK
Compile: OK
Selftests compile: FAILED
Tests: PANICKED

All kernel binaries, config files, and logs are available for download here:

https://arr-cki-prod-datawarehouse-public.s3.amazonaws.com/index.html?prefix=datawarehouse-public/2021/02/03/623045

❌ selinux-policy: serge-testsuite
✅ lvm thinp sanity
✅ storage: software RAID testing
✅ stress: stress-ng
🚧 ✅ xfstests - ext4
🚧 ⚡⚡⚡ xfstests - xfs
🚧 ⚡⚡⚡ Storage blktests
🚧 ⚡⚡⚡ Storage nvme - rdma
🚧 ⚡⚡⚡ Storage nvme - tcp
🚧 ⚡⚡⚡ Storage: swraid mdadm raid_module test
🚧 ⚡⚡⚡ storage: iSCSI parameters

Host 3:

⚡ Internal infrastructure issues prevented one or more tests (marked
with ⚡⚡⚡) from running on this architecture.
This is not the fault of the kernel that was tested.

✅ Boot test
⚡⚡⚡ kdump - sysrq-c
🚧 ⚡⚡⚡ kdump - file-load

Host 4:

⚡ Internal infrastructure issues prevented one or more tests (marked
with ⚡⚡⚡) from running on this architecture.
This is not the fault of the kernel that was tested.

✅ Boot test

⚡⚡⚡ kdump - sysrq-c
🚧 ⚡⚡⚡ kdump - file-load

Host 5:

⚡ Internal infrastructure issues prevented one or more tests (marked
with ⚡⚡⚡) from running on this architecture.
This is not the fault of the kernel that was tested.

✅ Boot test
⚡⚡⚡ kdump - sysrq-c
🚧 ⚡⚡⚡ kdump - file-load

ppc64le:
Host 1:

⚡ Internal infrastructure issues prevented one or more tests (marked
with ⚡⚡⚡) from running on this architecture.
This is not the fault of the kernel that was tested.

⚡⚡⚡ Boot test

⚡⚡⚡ selinux-policy: serge-testsuite
⚡⚡⚡ lvm thinp sanity
⚡⚡⚡ storage: software RAID testing
🚧 ⚡⚡⚡ xfstests - ext4
🚧 ⚡⚡⚡ xfstests - xfs
🚧 ⚡⚡⚡ Storage blktests
🚧 ⚡⚡⚡ Storage nvme - rdma
🚧 ⚡⚡⚡ Storage nvme - tcp
🚧 ⚡⚡⚡ Storage: swraid mdadm raid_module test

Host 2:

⚡ Internal infrastructure issues prevented one or more tests (marked
with ⚡⚡⚡) from running on this architecture.
This is not the fault of the kernel that was tested.

⚡⚡⚡ Boot test
⚡⚡⚡ kdump - sysrq-c
🚧 ⚡⚡⚡ kdump - file-load

Host 3:

[Sedat Dilek]

On Wed, Feb 3, 2021 at 11:06 PM Nathan Chancellor <nat...@kernel.org> wrote:
>

OK, Thanks.

Can you give me the link of your (above) response in the
ClangBuiltLinux mailing-list?

- Sedat -

[Nathan Chancellor]

https://groups.google.com/g/clang-built-linux/c/de_mNh23FOc/m/oNIZyE6fBQAJ

Cheers,
Nathan

[Sedat Dilek]

Thanks for the link.

You need a Gmail-account to be able to read the mailing-list?
Asking for offline readers.

- Sedat -

[1] https://groups.google.com/g/clang-built-linux/

[Nick Desaulniers]

We can move to vger+lore. Google groups was faster to setup, but has
many downsides. I don't have resources to pursue archiving the
existing list, and corralling the folks to set up vger and lore, but I
can point people in the right direction and would be supportive.

>
> - Sedat -
>
> [1] https://groups.google.com/g/clang-built-linux/
>

> --
> You received this message because you are subscribed to the Google Groups "Clang Built Linux" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to clang-built-li...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/CA%2BicZUVV3q3Jr8HEi%3DLmqYucOWK8b3zOEvfGsk8Mn5FS--8bnQ%40mail.gmail.com.



--
Thanks,
~Nick Desaulniers

[Nathan Chancellor]

It should not require a Gmail account. I can view that link no problem
in incognito mode in Chrome.

Cheers,
Nathan

[Sedat Dilek]

Logged-out from Gmail and I was able to read "offline".
Thanks for confirming.

- Sedat -

[Veronika Kabatova]

Hi,

not sure which email to reply so I'm going with this one...

Confirmed with BPF folks that the BTF boot issue from the gcc run is due
to the dwarves incompatibility that was already mentioned and tested a new
dwarves rawhide build which fixes it. According to them it should be
released in a few days.

> > As far as comparing tests for mainline run with gcc vs clang, we don't
> > email an upstream ML for regular mainline, so nothing archived.
> > However, the logs are still published externally and we can always link
> > them here for comparison.
> >
> > If you like, we could start forwarding you reports for both ? Otherwise, we
> > are working on publishing our results database we use internally,
> > but this will take some time.
> >
>
> If just adding a link to the GCC logs in the clang reports is not too
> much effort for you guys, that would be mighty helpful. I do not mind
> sifting through the logs to try and compare results manually (as I just
> need to compare the failing clang tests to the GCC run).
>
> If that is not possible for various reasons, please explicitly CC me on
> the GCC mainline reports (nat...@kernel.org) and I can just grab the
> link myself. Our list gets enough reports as it stands so we can keep
> the GCC ones off of this list.
>

The two test runs are separate, so the gcc run may not yet be finished
when the clang reports are ready, which makes providing links hard and
we'd have to implement some sort of half-reliable syncing mechanism.
So if you don't mind, we'd just go with the cc for now (once the dwarves
issue is resolved as the kernels just can't boot due to it).

We're hoping to have the public result database within a few months,
and potentially also fully publicly available runs (independent of the
database), both of which should make the comparison process easier.


Veronika

> Cheers,
> Nathan
>
>
>

[Miguel Ojeda]

On Thu, Feb 4, 2021 at 7:09 PM 'Nick Desaulniers' via Clang Built
Linux <clang-bu...@googlegroups.com> wrote:
>
> We can move to vger+lore. Google groups was faster to setup, but has
> many downsides. I don't have resources to pursue archiving the
> existing list, and corralling the folks to set up vger and lore, but I
> can point people in the right direction and would be supportive.

I can submit the request like I did for Rust for Linux if you want.

Cheers,
Miguel

[Sedat Dilek]

+1

I am fine with that move to vger + lore.

- Sedat -

[Nick Desaulniers]

Moving our CKI friends to BCC, which I perhaps should have done sooner.

If Nathan's cool with it, I would appreciate it. I'm proud that we
have folks that would volunteer to do so, and more than happy to owe
more beers. I look forward to buying those! Being cooped up for so
long due to the pandemic has given me cabin fever.

> >
>
> +1
>
> I am fine with that move to vger + lore.
>
> - Sedat -

--
Thanks,
~Nick Desaulniers

[Nathan Chancellor]

Yes, that is fine with me! We tried to submit a request for a
vger.kernel.org list once but I do not think that we got any response.
Maybe now that the project is bigger and more relevant, they will be
more responsive to our request.

Migrating off of Google Groups would be nice (although vger has its own
problems at times).

Cheers,
Nathan

[Miguel Ojeda]

On Thu, Feb 4, 2021 at 11:22 PM Nick Desaulniers
<ndesau...@google.com> wrote:
>
> If Nathan's cool with it, I would appreciate it. I'm proud that we
> have folks that would volunteer to do so, and more than happy to owe
> more beers. I look forward to buying those! Being cooped up for so
> long due to the pandemic has given me cabin fever.

No worries (I'll just choose a very expensive import beer ;-)

Joking aside, it is just a few emails, the guys doing the work are
others (and they are quite busy, so this will likely take weeks or
months), so we can buy them the beers.

Cheers,
Miguel

[Miguel Ojeda]

On Thu, Feb 4, 2021 at 11:35 PM Nathan Chancellor <nat...@kernel.org> wrote:
>
> Yes, that is fine with me! We tried to submit a request for a
> vger.kernel.org list once but I do not think that we got any response.
> Maybe now that the project is bigger and more relevant, they will be
> more responsive to our request.

I will give it a try then.

Cheers,
Miguel

[Nick Desaulniers]

Now would be the time to use llvm-linux@vger or linux-llvm@vger if we
want to consider a rename.

I'm not sure precisely what the previous list was:
https://lists.linuxfoundation.org/mailman/listinfo/llvmlinux. I guess
it was llvm...@lists.linuxfoundation.org, not @vger?

--
Thanks,
~Nick Desaulniers

[Miguel Ojeda]

On Thu, Feb 4, 2021 at 11:51 PM Nick Desaulniers
<ndesau...@google.com> wrote:
>
> Now would be the time to use llvm-linux@vger or linux-llvm@vger if we
> want to consider a rename.

Let me know if you prefer that (I already sent the request with the
same name as the google groups one, but these things usually take
time, so I guess you have time to change).

On one hand, linux-llvm looks best (and follows the other linux-*
lists). On the other, people know the project by clang-built-linux...
So, up to you!

Cheers,
Miguel

[CKI Project]

Hello,

We ran automated tests on a recent commit from this kernel tree:

Kernel repo: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git

Commit: 927002ed29e2 - Merge tag 'acpi-5.11-rc7' of git://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm

The results of these automated tests are provided below.

Overall result: FAILED (see details below)
Merge: OK
Compile: OK
Selftests compile: FAILED
Tests: PANICKED

All kernel binaries, config files, and logs are available for download here:

https://arr-cki-prod-datawarehouse-public.s3.amazonaws.com/index.html?prefix=datawarehouse-public/2021/02/04/623139

One or more kernel tests failed:

aarch64:
❌ selinux-policy: serge-testsuite

x86_64:

Host 2:

⚡ Internal infrastructure issues prevented one or more tests (marked
with ⚡⚡⚡) from running on this architecture.
This is not the fault of the kernel that was tested.

✅ Boot test
⚡⚡⚡ kdump - sysrq-c
🚧 ⚡⚡⚡ kdump - file-load

Host 3:

⚡ Internal infrastructure issues prevented one or more tests (marked
with ⚡⚡⚡) from running on this architecture.
This is not the fault of the kernel that was tested.

✅ Boot test

❌ selinux-policy: serge-testsuite
✅ lvm thinp sanity
✅ storage: software RAID testing
⚡⚡⚡ stress: stress-ng
🚧 ⚡⚡⚡ xfstests - ext4
🚧 ⚡⚡⚡ xfstests - xfs
🚧 ⚡⚡⚡ Storage blktests
🚧 ⚡⚡⚡ Storage nvme - rdma
🚧 ⚡⚡⚡ Storage nvme - tcp
🚧 ⚡⚡⚡ Storage: swraid mdadm raid_module test
🚧 ⚡⚡⚡ storage: iSCSI parameters

Host 4:

⚡ Internal infrastructure issues prevented one or more tests (marked
with ⚡⚡⚡) from running on this architecture.
This is not the fault of the kernel that was tested.

✅ Boot test
⚡⚡⚡ kdump - sysrq-c
🚧 ⚡⚡⚡ kdump - file-load

Host 5:

⚡ Internal infrastructure issues prevented one or more tests (marked
with ⚡⚡⚡) from running on this architecture.
This is not the fault of the kernel that was tested.

✅ Boot test
⚡⚡⚡ kdump - sysrq-c
🚧 ⚡⚡⚡ kdump - file-load

ppc64le:
Host 1:

⚡ Internal infrastructure issues prevented one or more tests (marked
with ⚡⚡⚡) from running on this architecture.
This is not the fault of the kernel that was tested.

⚡⚡⚡ Boot test

⚡⚡⚡ selinux-policy: serge-testsuite
⚡⚡⚡ lvm thinp sanity
⚡⚡⚡ storage: software RAID testing
🚧 ⚡⚡⚡ xfstests - ext4
🚧 ⚡⚡⚡ xfstests - xfs
🚧 ⚡⚡⚡ Storage blktests
🚧 ⚡⚡⚡ Storage nvme - rdma
🚧 ⚡⚡⚡ Storage nvme - tcp
🚧 ⚡⚡⚡ Storage: swraid mdadm raid_module test

✅ Boot test

✅ kdump - sysrq-c
🚧 ✅ kdump - file-load

Host 3:

⚡ Internal infrastructure issues prevented one or more tests (marked
with ⚡⚡⚡) from running on this architecture.
This is not the fault of the kernel that was tested.

✅ Boot test

❌ selinux-policy: serge-testsuite
✅ lvm thinp sanity
✅ storage: software RAID testing
✅ stress: stress-ng
🚧 ✅ xfstests - ext4
🚧 ✅ xfstests - xfs
🚧 ✅ Storage blktests
🚧 ✅ Storage nvme - rdma
🚧 ✅ Storage nvme - tcp
🚧 ⚡⚡⚡ Storage nvdimm ndctl test suite
🚧 ✅ Storage: swraid mdadm raid_module test
🚧 ❌ storage: iSCSI parameters

[CKI Project]

Hello,

We ran automated tests on a recent commit from this kernel tree:

Kernel repo: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git

Commit: 5c279c4cf206 - Revert "x86/setup: don't remove E820_TYPE_RAM for pfn 0"

The results of these automated tests are provided below.

Overall result: FAILED (see details below)
Merge: OK
Compile: OK
Selftests compile: FAILED
Tests: PANICKED

All kernel binaries, config files, and logs are available for download here:

https://arr-cki-prod-datawarehouse-public.s3.amazonaws.com/index.html?prefix=datawarehouse-public/2021/02/04/623142

One or more kernel tests failed:

⚡⚡⚡ selinux-policy: serge-testsuite

⚡⚡⚡ lvm thinp sanity
⚡⚡⚡ storage: software RAID testing
⚡⚡⚡ stress: stress-ng
🚧 ⚡⚡⚡ xfstests - ext4
🚧 ⚡⚡⚡ xfstests - xfs
🚧 ⚡⚡⚡ Storage blktests
🚧 ⚡⚡⚡ Storage nvme - rdma
🚧 ⚡⚡⚡ Storage nvme - tcp
🚧 ⚡⚡⚡ Storage: swraid mdadm raid_module test
🚧 ⚡⚡⚡ storage: iSCSI parameters

Host 2:

⚡ Internal infrastructure issues prevented one or more tests (marked
with ⚡⚡⚡) from running on this architecture.
This is not the fault of the kernel that was tested.

✅ Boot test
⚡⚡⚡ kdump - sysrq-c
🚧 ⚡⚡⚡ kdump - file-load

Host 3:

⚡ Internal infrastructure issues prevented one or more tests (marked
with ⚡⚡⚡) from running on this architecture.
This is not the fault of the kernel that was tested.

✅ Boot test

Host 4:

⚡ Internal infrastructure issues prevented one or more tests (marked
with ⚡⚡⚡) from running on this architecture.
This is not the fault of the kernel that was tested.

✅ Boot test
⚡⚡⚡ kdump - sysrq-c
🚧 ⚡⚡⚡ kdump - file-load

Host 5:

⚡ Internal infrastructure issues prevented one or more tests (marked
with ⚡⚡⚡) from running on this architecture.
This is not the fault of the kernel that was tested.

⚡⚡⚡ Boot test
⚡⚡⚡ kdump - sysrq-c
🚧 ⚡⚡⚡ kdump - file-load

ppc64le:
Host 1:

⚡ Internal infrastructure issues prevented one or more tests (marked
with ⚡⚡⚡) from running on this architecture.
This is not the fault of the kernel that was tested.

⚡⚡⚡ Boot test

⚡⚡⚡ selinux-policy: serge-testsuite
⚡⚡⚡ lvm thinp sanity
⚡⚡⚡ storage: software RAID testing
🚧 ⚡⚡⚡ xfstests - ext4
🚧 ⚡⚡⚡ xfstests - xfs
🚧 ⚡⚡⚡ Storage blktests
🚧 ⚡⚡⚡ Storage nvme - rdma
🚧 ⚡⚡⚡ Storage nvme - tcp
🚧 ⚡⚡⚡ Storage: swraid mdadm raid_module test

Host 2:

⚡ Internal infrastructure issues prevented one or more tests (marked
with ⚡⚡⚡) from running on this architecture.
This is not the fault of the kernel that was tested.

⚡⚡⚡ Boot test
⚡⚡⚡ kdump - sysrq-c
🚧 ⚡⚡⚡ kdump - file-load

Host 3:

⚡ Internal infrastructure issues prevented one or more tests (marked
with ⚡⚡⚡) from running on this architecture.
This is not the fault of the kernel that was tested.

⚡⚡⚡ Boot test

[Miguel Ojeda]

On Fri, Feb 5, 2021 at 12:03 AM Miguel Ojeda
<miguel.oje...@gmail.com> wrote:
>
> Let me know if you prefer that (I already sent the request with the
> same name as the google groups one, but these things usually take
> time, so I guess you have time to change).
>
> On one hand, linux-llvm looks best (and follows the other linux-*
> lists). On the other, people know the project by clang-built-linux...
> So, up to you!

They are fine with both linux-llvm and clang-built-linux, so it is
your call. I think the rename is a good idea if you want to give a
feeling that now there is official support for LLVM and Clang in the
kernel (and commercially supported). For Rust for Linux, if we manage
to mainline it, I was also thinking of a rename in the future to
linux-rust to reflect that too.

Also: do you want to try to automatically import the users? That means
parsing the old emails and subscribing people automatically to the new
one. It is more work for them (so let's avoid it if possible), and
they would prefer to start fresh. I think it is a good idea anyway: a
cleanup is always good and people may find it intrusive if they are
subscribed automatically, but the option is there nevertheless if you
really need it.

The archival is also requested (although that is done after the ML is
setup, by another team). It would be great if you can see if it is
possible to download the entire list as an .mbox file -- that way we
can migrate all the old emails for posterity. If Google Groups doesn't
support that, perhaps Nick can find an internal way to do it ;-)

Cheers,
Miguel

[Nick Desaulniers]

On Thu, Feb 4, 2021 at 6:16 PM Miguel Ojeda
<miguel.oje...@gmail.com> wrote:
>
> On Fri, Feb 5, 2021 at 12:03 AM Miguel Ojeda
> <miguel.oje...@gmail.com> wrote:
> >
> > Let me know if you prefer that (I already sent the request with the
> > same name as the google groups one, but these things usually take
> > time, so I guess you have time to change).
> >
> > On one hand, linux-llvm looks best (and follows the other linux-*
> > lists). On the other, people know the project by clang-built-linux...
> > So, up to you!
>
> They are fine with both linux-llvm and clang-built-linux, so it is

Yeah, let's go for it. The ML doesn't need to reflect the github name
or anything. linux-llvm@vger, please.

> your call. I think the rename is a good idea if you want to give a
> feeling that now there is official support for LLVM and Clang in the
> kernel (and commercially supported). For Rust for Linux, if we manage
> to mainline it, I was also thinking of a rename in the future to
> linux-rust to reflect that too.
>
> Also: do you want to try to automatically import the users? That means
> parsing the old emails and subscribing people automatically to the new
> one. It is more work for them (so let's avoid it if possible), and
> they would prefer to start fresh. I think it is a good idea anyway: a
> cleanup is always good and people may find it intrusive if they are
> subscribed automatically, but the option is there nevertheless if you
> really need it.
>
> The archival is also requested (although that is done after the ML is
> setup, by another team). It would be great if you can see if it is
> possible to download the entire list as an .mbox file -- that way we
> can migrate all the old emails for posterity. If Google Groups doesn't
> support that, perhaps Nick can find an internal way to do it ;-)

https://github.com/icy/google-group-crawler looks promising.
--
Thanks,
~Nick Desaulniers

[Nathan Chancellor]

On Fri, Feb 05, 2021 at 03:16:01AM +0100, Miguel Ojeda wrote:
> On Fri, Feb 5, 2021 at 12:03 AM Miguel Ojeda
> <miguel.oje...@gmail.com> wrote:
> >
> > Let me know if you prefer that (I already sent the request with the
> > same name as the google groups one, but these things usually take
> > time, so I guess you have time to change).
> >
> > On one hand, linux-llvm looks best (and follows the other linux-*
> > lists). On the other, people know the project by clang-built-linux...
> > So, up to you!
>
> They are fine with both linux-llvm and clang-built-linux, so it is
> your call. I think the rename is a good idea if you want to give a
> feeling that now there is official support for LLVM and Clang in the
> kernel (and commercially supported). For Rust for Linux, if we manage
> to mainline it, I was also thinking of a rename in the future to
> linux-rust to reflect that too.

My vote is for linux-llvm as we are focused on LLVM as a whole now,
which includes clang.

> Also: do you want to try to automatically import the users? That means
> parsing the old emails and subscribing people automatically to the new
> one. It is more work for them (so let's avoid it if possible), and
> they would prefer to start fresh. I think it is a good idea anyway: a
> cleanup is always good and people may find it intrusive if they are
> subscribed automatically, but the option is there nevertheless if you
> really need it.

No, I think that people should opt into the new list. We can post an
announcement that we are moving.

> The archival is also requested (although that is done after the ML is
> setup, by another team). It would be great if you can see if it is
> possible to download the entire list as an .mbox file -- that way we
> can migrate all the old emails for posterity. If Google Groups doesn't
> support that, perhaps Nick can find an internal way to do it ;-)

It does not look like this is possible within Google Groups but there
appear to be other ways of doing this?

https://github.com/icy/google-group-crawler
https://pypi.org/project/googlegroupexporter/

Cheers,
Nathan

[CKI Project]

Hello,

We ran automated tests on a recent commit from this kernel tree:

Kernel repo: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git

Commit: dd86e7fa07a3 - Merge tag 'pci-v5.11-fixes-2' of git://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci

The results of these automated tests are provided below.

Overall result: FAILED (see details below)
Merge: OK
Compile: OK
Selftests compile: FAILED
Tests: PANICKED

All kernel binaries, config files, and logs are available for download here:

https://arr-cki-prod-datawarehouse-public.s3.amazonaws.com/index.html?prefix=datawarehouse-public/2021/02/05/623168

One or more kernel tests failed:

aarch64:
❌ LTP

x86_64:
💥 LTP

✅ ACPI table test
✅ ACPI enabled test
❌ LTP
✅ Loopdev Sanity
✅ Memory: fork_mem
✅ Memory function: memfd_create

✅ AMTU (Abstract Machine Test Utility)
✅ Networking bridge: sanity

✅ Networking socket: fuzz
✅ Networking: igmp conformance test

✅ Networking route: pmtu
✅ Networking route_func - local
✅ Networking route_func - forward
✅ Networking TCP: keepalive test
✅ Networking UDP: socket
✅ Networking tunnel: geneve basic test
✅ Networking tunnel: gre basic
✅ L2TP basic test
✅ Networking tunnel: vxlan basic
✅ Networking ipsec: basic netns - transport
✅ Networking ipsec: basic netns - tunnel
✅ Libkcapi AF_ALG test
✅ pciutils: update pci ids test
✅ ALSA PCM loopback test
✅ ALSA Control (mixer) Userspace Element test

✅ storage: SCSI VPD
🚧 ✅ CIFS Connectathon
🚧 ✅ POSIX pjd-fstest suites
🚧 ⚡⚡⚡ Firmware test suite
🚧 ✅ jvm - jcstress tests
🚧 ✅ Memory function: kaslr
🚧 ✅ Ethernet drivers sanity
🚧 ✅ Networking firewall: basic netfilter test
🚧 ✅ audit: audit testsuite test
🚧 ✅ trace: ftrace/tracer
🚧 ✅ kdump - kexec_boot

Host 2:

⚡ Internal infrastructure issues prevented one or more tests (marked
with ⚡⚡⚡) from running on this architecture.
This is not the fault of the kernel that was tested.

✅ Boot test

✅ storage: software RAID testing

🚧 ✅ xfstests - ext4
🚧 ✅ xfstests - xfs

🚧 ✅ xfstests - btrfs
🚧 ❌ IPMI driver test
🚧 ✅ IPMItool loop stress test
🚧 ❌ selinux-policy: serge-testsuite
🚧 ✅ Storage blktests
🚧 ✅ Storage block - filesystem fio test
🚧 ⚡⚡⚡ Storage block - queue scheduler test

🚧 ⚡⚡⚡ Storage nvme - tcp
🚧 ⚡⚡⚡ Storage: swraid mdadm raid_module test

🚧 ⚡⚡⚡ stress: stress-ng

ppc64le:
Host 1:

⚡ Internal infrastructure issues prevented one or more tests (marked
with ⚡⚡⚡) from running on this architecture.
This is not the fault of the kernel that was tested.

✅ Boot test

⚡⚡⚡ LTP
⚡⚡⚡ Loopdev Sanity
⚡⚡⚡ Memory: fork_mem

⚡⚡⚡ Memory function: memfd_create
⚡⚡⚡ AMTU (Abstract Machine Test Utility)
⚡⚡⚡ Networking bridge: sanity

⚡⚡⚡ Networking socket: fuzz

⚡⚡⚡ Networking route: pmtu
⚡⚡⚡ Networking route_func - local
⚡⚡⚡ Networking route_func - forward
⚡⚡⚡ Networking TCP: keepalive test
⚡⚡⚡ Networking UDP: socket
⚡⚡⚡ Networking tunnel: geneve basic test
⚡⚡⚡ Networking tunnel: gre basic
⚡⚡⚡ L2TP basic test
⚡⚡⚡ Networking tunnel: vxlan basic
⚡⚡⚡ Networking ipsec: basic netns - tunnel
⚡⚡⚡ Libkcapi AF_ALG test
⚡⚡⚡ pciutils: update pci ids test
⚡⚡⚡ ALSA PCM loopback test
⚡⚡⚡ ALSA Control (mixer) Userspace Element test

🚧 ⚡⚡⚡ CIFS Connectathon
🚧 ⚡⚡⚡ POSIX pjd-fstest suites

🚧 ⚡⚡⚡ jvm - jcstress tests
🚧 ⚡⚡⚡ Memory function: kaslr
🚧 ⚡⚡⚡ Ethernet drivers sanity

🚧 ⚡⚡⚡ Networking firewall: basic netfilter test
🚧 ⚡⚡⚡ audit: audit testsuite test

🚧 ⚡⚡⚡ trace: ftrace/tracer

Host 2:
✅ Boot test
⏱ LTP
⏱ Loopdev Sanity
⏱ Memory: fork_mem
⏱ Memory function: memfd_create

⏱ AMTU (Abstract Machine Test Utility)
⏱ Networking bridge: sanity

⏱ Networking socket: fuzz
⏱ Networking route: pmtu

⏱ Networking route_func - local
⏱ Networking route_func - forward
⏱ Networking TCP: keepalive test
⏱ Networking UDP: socket
⏱ Networking tunnel: geneve basic test
⏱ Networking tunnel: gre basic
⏱ L2TP basic test
⏱ Networking tunnel: vxlan basic
⏱ Networking ipsec: basic netns - tunnel
⏱ Libkcapi AF_ALG test
⏱ pciutils: update pci ids test
⏱ ALSA PCM loopback test
⏱ ALSA Control (mixer) Userspace Element test

⏱ CIFS Connectathon
⏱ POSIX pjd-fstest suites
⏱ jvm - jcstress tests
⏱ Memory function: kaslr
⏱ Ethernet drivers sanity
⏱ Networking firewall: basic netfilter test
⏱ audit: audit testsuite test
⏱ trace: ftrace/tracer

x86_64:

Host 1:

⚡ Internal infrastructure issues prevented one or more tests (marked
with ⚡⚡⚡) from running on this architecture.
This is not the fault of the kernel that was tested.

✅ Boot test

✅ storage: software RAID testing
🚧 ❌ CPU: Frequency Driver Test
🚧 ✅ CPU: Idle Test
🚧 ✅ xfstests - ext4
🚧 ✅ xfstests - xfs
🚧 ✅ xfstests - btrfs
🚧 ✅ xfstests - nfsv4.2
🚧 ✅ xfstests - cifsv3.11
🚧 ❌ IPMI driver test
🚧 ✅ IPMItool loop stress test
🚧 ❌ selinux-policy: serge-testsuite
🚧 ❌ Storage blktests
🚧 ✅ Storage block - filesystem fio test
🚧 ✅ Storage block - queue scheduler test

🚧 ✅ Storage nvme - tcp
🚧 ⚡⚡⚡ Storage nvdimm ndctl test suite
🚧 ✅ Storage: swraid mdadm raid_module test

🚧 ✅ stress: stress-ng

Host 2:

✅ Boot test
✅ ACPI table test
💥 LTP

⚡⚡⚡ Loopdev Sanity
⚡⚡⚡ Memory: fork_mem

⚡⚡⚡ Memory function: memfd_create
⚡⚡⚡ AMTU (Abstract Machine Test Utility)
⚡⚡⚡ Networking bridge: sanity

⚡⚡⚡ Networking socket: fuzz

⚡⚡⚡ Networking: igmp conformance test
⚡⚡⚡ Networking route: pmtu
⚡⚡⚡ Networking route_func - local
⚡⚡⚡ Networking route_func - forward
⚡⚡⚡ Networking TCP: keepalive test
⚡⚡⚡ Networking UDP: socket
⚡⚡⚡ Networking tunnel: geneve basic test
⚡⚡⚡ Networking tunnel: gre basic
⚡⚡⚡ L2TP basic test
⚡⚡⚡ Networking tunnel: vxlan basic
⚡⚡⚡ Networking ipsec: basic netns - transport
⚡⚡⚡ Networking ipsec: basic netns - tunnel
⚡⚡⚡ Libkcapi AF_ALG test
⚡⚡⚡ pciutils: sanity smoke test
⚡⚡⚡ pciutils: update pci ids test
⚡⚡⚡ ALSA PCM loopback test
⚡⚡⚡ ALSA Control (mixer) Userspace Element test

⚡⚡⚡ storage: SCSI VPD

🚧 ⚡⚡⚡ CIFS Connectathon
🚧 ⚡⚡⚡ POSIX pjd-fstest suites

🚧 ⚡⚡⚡ Firmware test suite
🚧 ⚡⚡⚡ jvm - jcstress tests
🚧 ⚡⚡⚡ Memory function: kaslr
🚧 ⚡⚡⚡ Ethernet drivers sanity

🚧 ⚡⚡⚡ Networking firewall: basic netfilter test
🚧 ⚡⚡⚡ audit: audit testsuite test

🚧 ⚡⚡⚡ trace: ftrace/tracer

🚧 ⚡⚡⚡ kdump - kexec_boot

Host 3:

✅ Boot test
🚧 ✅ kdump - sysrq-c
🚧 ✅ kdump - file-load

[Nathan Chancellor]

On Fri, Feb 05, 2021 at 08:48:39PM -0000, CKI Project wrote:
>
> Hello,
>
> We ran automated tests on a recent commit from this kernel tree:
>
> Kernel repo: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
> Commit: dd86e7fa07a3 - Merge tag 'pci-v5.11-fixes-2' of git://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci
>
> The results of these automated tests are provided below.
>
> Overall result: FAILED (see details below)
> Merge: OK
> Compile: OK
> Selftests compile: FAILED
> Tests: PANICKED
>
> All kernel binaries, config files, and logs are available for download here:
>
> https://arr-cki-prod-datawarehouse-public.s3.amazonaws.com/index.html?prefix=datawarehouse-public/2021/02/05/623168

I don't know if I am missing something or there was a mix up somewhere
but all of the configuration files in that link show that GCC was used
for all of these builds:

https://arr-cki-prod-datawarehouse-public.s3.amazonaws.com/datawarehouse-public/2021/02/05/623168/build_aarch64_redhat%3A1095500/kernel-mainline.kernel.org-clang-aarch64-dd86e7fa07a3ec33c92c957ea7b642c4702516a0.config

CONFIG_CC_VERSION_TEXT="aarch64-linux-gnu-gcc (GCC) 10.2.1 20200826 (Red Hat Cross 10.2.1-3)"
CONFIG_CC_IS_GCC=y
CONFIG_GCC_VERSION=100201
CONFIG_LD_VERSION=235010000
CONFIG_CLANG_VERSION=0
CONFIG_LLD_VERSION=0

https://arr-cki-prod-datawarehouse-public.s3.amazonaws.com/datawarehouse-public/2021/02/05/623168/build_ppc64le_redhat%3A1095501/kernel-mainline.kernel.org-clang-ppc64le-dd86e7fa07a3ec33c92c957ea7b642c4702516a0.config

CONFIG_CC_VERSION_TEXT="powerpc64le-linux-gnu-gcc (GCC) 10.2.1 20200826 (Red Hat Cross 10.2.1-3)"
CONFIG_CC_IS_GCC=y
CONFIG_GCC_VERSION=100201
CONFIG_LD_VERSION=235010000
CONFIG_CLANG_VERSION=0
CONFIG_LLD_VERSION=0

https://arr-cki-prod-datawarehouse-public.s3.amazonaws.com/datawarehouse-public/2021/02/05/623168/build_x86_64_redhat%3A1095499/kernel-mainline.kernel.org-clang-x86_64-dd86e7fa07a3ec33c92c957ea7b642c4702516a0.config

CONFIG_CC_VERSION_TEXT="gcc (GCC) 11.0.0 20210130 (Red Hat 11.0.0-0)"
CONFIG_CC_IS_GCC=y
CONFIG_GCC_VERSION=110000
CONFIG_LD_VERSION=235010000
CONFIG_CLANG_VERSION=0
CONFIG_LLD_VERSION=0

Cheers,
Nathan

[Nathan Chancellor]

I can see by booting the binary that the kernel was compiled with clang
and linked with lld though. However, I still cannot reproduce this
crash.

root@ubuntu-m3-large-x86:~# cat /proc/version
Linux version 5.11.0-rc6 (cki@runner-3uc3rmvr-project-2-concurrent-4lc6vt) (clang version 11.1.0 (Fedora 11.1.0-0.4.rc2.fc34), LLD 11.1.0) #1 SMP Fri Feb 5 00:21:48 UTC 2021
root@ubuntu-m3-large-x86:~# lsmod
Module Size Used by
binfmt_misc 24576 1
intel_rapl_msr 20480 0
intel_rapl_common 32768 1 intel_rapl_msr
amd_energy 16384 0
crct10dif_pclmul 16384 1
crc32_pclmul 16384 0
crc32c_intel 24576 0
ghash_clmulni_intel 16384 0
snd_pcm 139264 0
snd_timer 45056 1 snd_pcm
snd 106496 2 snd_timer,snd_pcm
joydev 28672 0
soundcore 16384 1 snd
serio_raw 20480 0
pcspkr 16384 0
virtio_net 65536 0
net_failover 28672 1 virtio_net
failover 16384 1 net_failover
ata_generic 16384 0
i2c_piix4 28672 0
pata_acpi 16384 0
floppy 94208 0
qemu_fw_cfg 20480 0
bpf_preload 16384 0
ip_tables 32768 0
x_tables 53248 1 ip_tables
root@ubuntu-m3-large-x86:~# ltp/testcases/kernel/fs/proc/proc01 -m 128
proc01 0 TINFO : /proc/sys/fs/binfmt_misc/register: is write-only.
proc01 0 TINFO : /proc/sys/net/ipv6/conf/all/stable_secret: known issue: errno=EIO(5): Input/output error
proc01 0 TINFO : /proc/sys/net/ipv6/conf/default/stable_secret: known issue: errno=EIO(5): Input/output error
proc01 0 TINFO : /proc/sys/net/ipv6/conf/dummy0/stable_secret: known issue: errno=EIO(5): Input/output error
proc01 0 TINFO : /proc/sys/net/ipv6/conf/ens2/stable_secret: known issue: errno=EIO(5): Input/output error
proc01 0 TINFO : /proc/sys/net/ipv6/conf/erspan0/stable_secret: known issue: errno=EIO(5): Input/output error
proc01 0 TINFO : /proc/sys/net/ipv6/conf/gre0/stable_secret: known issue: errno=EIO(5): Input/output error
proc01 0 TINFO : /proc/sys/net/ipv6/conf/gretap0/stable_secret: known issue: errno=EIO(5): Input/output error
proc01 0 TINFO : /proc/sys/net/ipv6/conf/ifb0/stable_secret: known issue: errno=EIO(5): Input/output error
proc01 0 TINFO : /proc/sys/net/ipv6/conf/ifb1/stable_secret: known issue: errno=EIO(5): Input/output error
proc01 0 TINFO : /proc/sys/net/ipv6/conf/ip6_vti0/stable_secret: known issue: errno=EIO(5): Input/output error
proc01 0 TINFO : /proc/sys/net/ipv6/conf/ip6gre0/stable_secret: known issue: errno=EIO(5): Input/output error
proc01 0 TINFO : /proc/sys/net/ipv6/conf/ip6tnl0/stable_secret: known issue: errno=EIO(5): Input/output error
proc01 0 TINFO : /proc/sys/net/ipv6/conf/ip_vti0/stable_secret: known issue: errno=EIO(5): Input/output error
proc01 0 TINFO : /proc/sys/net/ipv6/conf/lo/stable_secret: known issue: errno=EIO(5): Input/output error
proc01 0 TINFO : /proc/sys/net/ipv6/conf/tunl0/stable_secret: known issue: errno=EIO(5): Input/output error
proc01 0 TINFO : /proc/kmsg: known issue: errno=EAGAIN/EWOULDBLOCK(11): Resource temporarily unavailable
proc01 0 TINFO : /proc/sysrq-trigger: is write-only.
proc01 0 TINFO : /proc/self/task/753/mem: known issue: errno=EIO(5): Input/output error
proc01 0 TINFO : /proc/self/task/753/clear_refs: is write-only.
proc01 0 TINFO : /proc/self/task/753/pagemap: reached maxmbytes (-m)
proc01 0 TINFO : /proc/self/mem: known issue: errno=EIO(5): Input/output error
proc01 0 TINFO : /proc/self/clear_refs: is write-only.
proc01 0 TINFO : /proc/self/pagemap: reached maxmbytes (-m)
proc01 1 TPASS : readproc() completed successfully, total read: 280264271 bytes, 3326 objs
root@ubuntu-m3-large-x86:~# ltp/testcases/kernel/fs/read_all/read_all -d /proc -q -r 3
read_all.c:446: TPASS: Finished reading files/fs/read_all/read_all -d /proc -q -r 3

Summary:
passed 1
failed 0
broken 0
skipped 0
warnings 0

Cheers,
Nathan

[CKI Project]

Hello,

We ran automated tests on a recent commit from this kernel tree:

Kernel repo: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git

Commit: 17fbcdf9f163 - Merge tag 'nfsd-5.11-3' of git://git.kernel.org/pub/scm/linux/kernel/git/cel/linux

The results of these automated tests are provided below.

Overall result: FAILED (see details below)
Merge: OK
Compile: OK
Selftests compile: FAILED
Tests: PANICKED

All kernel binaries, config files, and logs are available for download here:

https://arr-cki-prod-datawarehouse-public.s3.amazonaws.com/index.html?prefix=datawarehouse-public/2021/02/05/623222

One or more kernel tests failed:

aarch64:
❌ LTP

❌ Memory: fork_mem

✅ storage: software RAID testing
🚧 ✅ xfstests - ext4
🚧 ✅ xfstests - xfs
🚧 ✅ xfstests - btrfs
🚧 ❌ IPMI driver test
🚧 ✅ IPMItool loop stress test
🚧 ❌ selinux-policy: serge-testsuite
🚧 ✅ Storage blktests
🚧 ✅ Storage block - filesystem fio test
🚧 ⚡⚡⚡ Storage block - queue scheduler test
🚧 ⚡⚡⚡ Storage nvme - tcp
🚧 ⚡⚡⚡ Storage: swraid mdadm raid_module test
🚧 ⚡⚡⚡ stress: stress-ng

Host 2:

⚡ Internal infrastructure issues prevented one or more tests (marked
with ⚡⚡⚡) from running on this architecture.
This is not the fault of the kernel that was tested.

✅ Boot test

ppc64le:
Host 1:

[CKI Project]

Hello,

We ran automated tests on a recent commit from this kernel tree:

Kernel repo: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git

Commit: 1e0d27fce010 - Merge branch 'akpm' (patches from Andrew)

The results of these automated tests are provided below.

Overall result: FAILED (see details below)
Merge: OK
Compile: OK
Selftests compile: FAILED
Tests: PANICKED

All kernel binaries, config files, and logs are available for download here:

https://arr-cki-prod-datawarehouse-public.s3.amazonaws.com/index.html?prefix=datawarehouse-public/2021/02/05/623250

One or more kernel tests failed:

ppc64le:
❌ Networking bridge: sanity
❌ Networking route: pmtu

❌ Networking route_func - local
❌ Networking route_func - forward

❌ Networking tunnel: geneve basic test

❌ Networking tunnel: gre basic
❌ L2TP basic test
❌ Networking tunnel: vxlan basic
❌ Networking ipsec: basic netns - tunnel

aarch64:
❌ Networking bridge: sanity
❌ Networking route: pmtu

❌ Networking route_func - local
❌ Networking route_func - forward

❌ Networking tunnel: geneve basic test

❌ Networking tunnel: gre basic
❌ L2TP basic test
❌ Networking tunnel: vxlan basic
❌ Networking ipsec: basic netns - transport
❌ Networking ipsec: basic netns - tunnel

x86_64:
💥 LTP

We hope that these logs can help you find the problem quickly. For the full
detail on our testing procedures, please scroll to the bottom of this message.

Please reply to this email if you have any questions about the tests that we
ran or if you have any suggestions on how to make future tests more effective.

,-. ,-.
( C ) ( K ) Continuous
`-',-.`-' Kernel
( I ) Integration
`-'
______________________________________________________________________________

Compile testing
---------------

We compiled the kernel for 3 architectures:

aarch64:

make options: make LLVM=1 -j24 INSTALL_MOD_STRIP=1 targz-pkg

ppc64le:
make options: make CC=clang -j24 INSTALL_MOD_STRIP=1 targz-pkg

x86_64:
make options: make LLVM=1 -j24 INSTALL_MOD_STRIP=1 targz-pkg

We built the following selftests:

x86_64:
net: fail
bpf: fail
install and packaging: fail

You can find the full log (build-selftests.log) in the artifact storage above.


Hardware testing
----------------
We booted each kernel and ran the following tests:

aarch64:
Host 1:

⏱ Boot test

Host 3:

⚡⚡⚡ ACPI table test

Host 2:

⚡ Internal infrastructure issues prevented one or more tests (marked
with ⚡⚡⚡) from running on this architecture.
This is not the fault of the kernel that was tested.

✅ Boot test

✅ storage: software RAID testing
🚧 ✅ CPU: Frequency Driver Test

🚧 ✅ xfstests - ext4
🚧 ✅ xfstests - xfs
🚧 ✅ xfstests - btrfs
🚧 ✅ xfstests - nfsv4.2
🚧 ✅ xfstests - cifsv3.11
🚧 ❌ IPMI driver test
🚧 ✅ IPMItool loop stress test
🚧 ❌ selinux-policy: serge-testsuite
🚧 ✅ Storage blktests
🚧 ✅ Storage block - filesystem fio test
🚧 ⚡⚡⚡ Storage block - queue scheduler test
🚧 ⚡⚡⚡ Storage nvme - tcp
🚧 ⚡⚡⚡ Storage nvdimm ndctl test suite
🚧 ⚡⚡⚡ Storage: swraid mdadm raid_module test
🚧 ⚡⚡⚡ stress: stress-ng

Host 3:
✅ Boot test
🚧 ✅ kdump - sysrq-c
🚧 ✅ kdump - file-load

Host 4:

⚡ Internal infrastructure issues prevented one or more tests (marked
with ⚡⚡⚡) from running on this architecture.
This is not the fault of the kernel that was tested.

⚡⚡⚡ Boot test
⚡⚡⚡ ACPI table test

⚡⚡⚡ LTP

Host 5:

[Veronika Kabatova]

----- Original Message -----
> From: "Nathan Chancellor" <nat...@kernel.org>
> To: "CKI Project" <cki-p...@redhat.com>
> Cc: "Milos Malik" <mma...@redhat.com>, clang-bu...@googlegroups.com, "Memory Management" <mm...@redhat.com>,
> "Ondrej Mosnacek" <omos...@redhat.com>, skt-resul...@redhat.com, "David Arcari" <dar...@redhat.com>, "Yi
> Zhang" <yiz...@redhat.com>, "Jan Stancek" <jsta...@redhat.com>
> Sent: Saturday, February 6, 2021 3:19:28 AM
> Subject: Re: 💥 PANICKED: Test report for kernel 5.11.0-rc6 (mainline.kernel.org-clang)
>

Hi,

I'll take a look if running config file scripts with the same make
options as the kernel build overrides the gcc data in there. The base
config is grabbed from fedora and already contains the gcc strings as
the fedora configs are built with gcc.

Adding Rachel and keeping test maintainers in this thread since I
can't help with the LTP failure.

Veronika

[CKI Project]

Hello,

We ran automated tests on a recent commit from this kernel tree:

Kernel repo: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git

Commit: b75dba7f472c - Merge tag 'libnvdimm-fixes-5.11-rc7' of git://git.kernel.org/pub/scm/linux/kernel/git/nvdimm/nvdimm

The results of these automated tests are provided below.

Overall result: FAILED (see details below)
Merge: OK
Compile: OK
Selftests compile: FAILED
Tests: PANICKED

All kernel binaries, config files, and logs are available for download here:

https://arr-cki-prod-datawarehouse-public.s3.amazonaws.com/index.html?prefix=datawarehouse-public/2021/02/07/623298

One or more kernel tests failed:

ppc64le:

💥 LTP

aarch64:
❌ LTP

Host 2:

⏱ Boot test
⏱ storage: software RAID testing
⏱ xfstests - ext4
⏱ xfstests - xfs
⏱ xfstests - btrfs
⏱ IPMI driver test
⏱ IPMItool loop stress test
⏱ selinux-policy: serge-testsuite
⏱ Storage blktests
⏱ Storage block - filesystem fio test
⏱ Storage block - queue scheduler test
⏱ Storage nvme - tcp
⏱ Storage: swraid mdadm raid_module test
⏱ stress: stress-ng

✅ Boot test
🚧 ✅ kdump - sysrq-c
🚧 ✅ kdump - file-load

Host 3:

⚡ Internal infrastructure issues prevented one or more tests (marked
with ⚡⚡⚡) from running on this architecture.
This is not the fault of the kernel that was tested.

✅ Boot test

✅ storage: software RAID testing
🚧 ❌ CPU: Frequency Driver Test

🚧 ✅ CPU: Idle Test

🚧 ✅ xfstests - ext4
🚧 ✅ xfstests - xfs
🚧 ✅ xfstests - btrfs
🚧 ✅ xfstests - nfsv4.2
🚧 ✅ xfstests - cifsv3.11
🚧 ✅ IPMI driver test
🚧 ✅ IPMItool loop stress test
🚧 ❌ selinux-policy: serge-testsuite
🚧 ✅ Storage blktests
🚧 ❌ Storage block - filesystem fio test
🚧 ✅ Storage block - queue scheduler test
🚧 ✅ Storage nvme - tcp
🚧 ⚡⚡⚡ Storage nvdimm ndctl test suite
🚧 ✅ Storage: swraid mdadm raid_module test
🚧 ✅ stress: stress-ng

[Veronika Kabatova]

Update:

Fixing up the make options to be called with the config scripts works.
I'm currently checking if this helps with the previous test failures
where tests tried to build external modules with the wrong compiler, or
if that would require some more future work (I already added the clang
dependencies to the test runs).

Slightly related, the problem with DEBUG_INFO_BTF having to be disabled
due to pahole segfaulting is now gone as a fixed pahole version is in
fedora rawhide now. x86_64 compiles with the enabled option fine,
however aarch64 fails with

FAILED unresolved symbol vfs_truncate

Is this a problem you already know about? This is the newest mainline,
compilation with gcc works. I can publish the logs and configs or
whatever else if needed.


Veronika

[Nathan Chancellor]

This is not something that I am aware of. I have pretty much zero
experience with testing BPF/BTF configs. I can reproduce this with one
of your recent configs + CONFIG_DEBUG_INFO_BTF:

$ mkdir -p build/aarch64

$ curl -LSso build/aarch64/.config https://arr-cki-prod-datawarehouse-public.s3.amazonaws.com/datawarehouse-public/2021/02/07/623298/build_aarch64_redhat%3A1097392/kernel-mainline.kernel.org-clang-aarch64-b75dba7f472ca6c2dd0b8ee41f5a4b5a45539306.config

$ scripts/config --file build/aarch64/.config -e DEBUG_INFO_BTF

$ make -skj"$(nproc)" ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- LLVM=1 O=build/aarch64 olddefconfig all
...
FAILED unresolved symbol vfs_truncate
...

I will see if I can isolate some configs and report it to the BPF folks
to see if they have any ideas what is going on.

Cheers,
Nathan

[Veronika Kabatova]

----- Original Message -----
> From: "Veronika Kabatova" <vkab...@redhat.com>
> To: "Nathan Chancellor" <nat...@kernel.org>

> Cc: "Milos Malik" <mma...@redhat.com>, clang-bu...@googlegroups.com, "Memory Management" <mm...@redhat.com>,

> "Ondrej Mosnacek" <omos...@redhat.com>, skt-resul...@redhat.com, "David Arcari" <dar...@redhat.com>, "CKI
> Project" <cki-p...@redhat.com>, "Yi Zhang" <yiz...@redhat.com>, "Jan Stancek" <jsta...@redhat.com>
> Sent: Monday, February 8, 2021 12:07:11 PM

Hi, the config files are fixed now. There's one currently running
pipeline that has the old data, but all new ones after that should
have the fix in them.


Veronika