[RFC PATCH 0/2] pgo: fixes for LLVM-13 and kernel v5.14+

[Jarmo Tiitto]

Hello!
I started hacking again on the kees/for-next/clang/pgo tree
now that the __no_profile and noinstr function attributes work.

Unfortunately I noticed the current for-next/clang/pgo tree is broken
from v5.14 onwards and the instrumented kernel fails to boot.
I accumulated following two bug-fix patches that I hope to get reviewed
for the kees/for-next/clang/pgo tree.

The first patch enables processing the raw profile data with LLVM-13 tools
and the second patch fixes a issue with the kernel module loader.

Also a offnote I have plans to publish my kernel tree
on behalf of pgo/modules project. It is an redo of the module
instrumentation support that I sent RFC patch earlier this year.
I would like to get some eyes on it. ;-)

Jarmo Tiitto (2):
pgo: Update .profraw file format to version 7
pgo: Make module.lds.S linker script to merge __llvm_prf_ sections.

kernel/pgo/fs.c | 1 +
kernel/pgo/pgo.h | 4 +++-
scripts/module.lds.S | 33 +++++++++++++++++++++++++++++++++
3 files changed, 37 insertions(+), 1 deletion(-)


base-commit: a15058eaefffc37c31326b59fa08b267b2def603
--
2.34.0

[Jarmo Tiitto]

Recent LLVM-13 can't read the version 5 profile file any more
so update the format to .profraw file version 7.

This version adds binary_ids_size entry to struct llvm_prf_header
and it is output as zero.

This is enough to fix processing the profile data on LLVM-13 tools.

Signed-off-by: Jarmo Tiitto <jarmo....@gmail.com>
---

kernel/pgo/fs.c | 1 +
kernel/pgo/pgo.h | 4 +++-

2 files changed, 4 insertions(+), 1 deletion(-)

diff --git a/kernel/pgo/fs.c b/kernel/pgo/fs.c
index 3c5aa7c2a4ce..77d23f869503 100644
--- a/kernel/pgo/fs.c
+++ b/kernel/pgo/fs.c
@@ -62,6 +62,7 @@ static void prf_fill_header(void **buffer)
#endif
header->version = LLVM_VARIANT_MASK_IR_PROF | LLVM_INSTR_PROF_RAW_VERSION;
header->data_size = prf_data_count();
+ header->binary_ids_size = 0;
header->padding_bytes_before_counters = 0;
header->counters_size = prf_cnts_count();
header->padding_bytes_after_counters = 0;
diff --git a/kernel/pgo/pgo.h b/kernel/pgo/pgo.h
index 04fbf3bcde1e..45eeff9ab6c4 100644
--- a/kernel/pgo/pgo.h
+++ b/kernel/pgo/pgo.h
@@ -43,7 +43,7 @@
(u64)'R' << 8 | \
(u64)129)

-#define LLVM_INSTR_PROF_RAW_VERSION 5
+#define LLVM_INSTR_PROF_RAW_VERSION 7
#define LLVM_INSTR_PROF_DATA_ALIGNMENT 8
#define LLVM_INSTR_PROF_IPVK_FIRST 0
#define LLVM_INSTR_PROF_IPVK_LAST 1
@@ -56,6 +56,7 @@
* struct llvm_prf_header - represents the raw profile header data structure.
* @magic: the magic token for the file format.
* @version: the version of the file format.
+ * @binary_ids_size: the number of binary ids. (since LLVM_INSTR_PROF_RAW_VERSION >= 7)
* @data_size: the number of entries in the profile data section.
* @padding_bytes_before_counters: the number of padding bytes before the
* counters.
@@ -72,6 +73,7 @@
struct llvm_prf_header {
u64 magic;
u64 version;
+ u64 binary_ids_size;
u64 data_size;
u64 padding_bytes_before_counters;
u64 counters_size;
--
2.34.0

[Jarmo Tiitto]

The compiler leaves many duplicate instances of __llvm_prf_* sections
in final module objects and they confuse the kernel module loader.
Fix this by providing module.lds.S linker script that merges the split
sections like vmlinux.lds.S currently does.

Signed-off-by: Jarmo Tiitto <jarmo....@gmail.com>
---

I'm not sure the linker script is entirely correct and
it propably needs more work.
---
scripts/module.lds.S | 33 +++++++++++++++++++++++++++++++++
1 file changed, 33 insertions(+)

diff --git a/scripts/module.lds.S b/scripts/module.lds.S
index 04c5685c25cf..83ca5825c0c3 100644
--- a/scripts/module.lds.S
+++ b/scripts/module.lds.S
@@ -60,6 +60,39 @@ SECTIONS {
*(.text .text.[0-9a-zA-Z_]* .text..L.cfi*)
}
#endif
+#ifdef CONFIG_PGO_CLANG
+ /*
+ * With CONFIG_PGO_CLANG the compiler may split __llvm_prf_xxx
+ * objects into multiple sections. Merge them in final .ko object.
+ * However leave .rela__llvm_prf_data sections as-is
+ * since they are needed by the module loader.
+ */
+ __llvm_prf_data : AT(ADDR(__llvm_prf_data)) {
+ __llvm_prf_data_start = .;
+ KEEP(*(SORT(__llvm_prf_data)))
+ __llvm_prf_data_end = .;
+ }
+ __llvm_prf_cnts : AT(ADDR(__llvm_prf_cnts)) {
+ __llvm_prf_cnts_start = .;
+ KEEP(*(SORT(__llvm_prf_cnts)))
+ __llvm_prf_cnts_end = .;
+ }
+ __llvm_prf_names : AT(ADDR(__llvm_prf_names)) {
+ __llvm_prf_names_start = .;
+ KEEP(*(SORT(__llvm_prf_names)))
+ __llvm_prf_names_end = .;
+ }
+ __llvm_prf_vals : AT(ADDR(__llvm_prf_vals)) {
+ __llvm_prf_vals_start = .;
+ KEEP(*(SORT(__llvm_prf_vals)))
+ __llvm_prf_vals_end = .;
+ }
+ __llvm_prf_vnds : AT(ADDR(__llvm_prf_vnds)) {
+ __llvm_prf_vnds_start = .;
+ KEEP(*(SORT(__llvm_prf_vnds)))
+ __llvm_prf_vnds_end = .;
+ }
+#endif
}

/* bring in arch-specific sections */
--
2.34.0

[Jarmo Tiitto]

Hello!
I started hacking again on the kees/for-next/clang/pgo tree
now that the __no_profile and noinstr function attributes work.

Unfortunately I noticed the current for-next/clang/pgo tree is broken
from v5.14 onwards and the instrumented kernel fails to boot.
I accumulated following two bug-fix patches that I hope to get reviewed
for the kees/for-next/clang/pgo tree.

The first patch enables processing the raw profile data with LLVM-13 tools
and the second patch fixes a issue with the kernel module loader.

A offnote: I have plans to publish my kernel tree
on behalf of the related pgo/modules project. It is an redo of the
modules PGO patchset that I sent earlier this year.

I would like to get some eyes on it. ;-)

Jarmo

Jarmo Tiitto (2):
pgo: Update .profraw file format to version 7
pgo: Make module.lds.S linker script to merge __llvm_prf_ sections.

kernel/pgo/fs.c | 1 +
kernel/pgo/pgo.h | 4 +++-

[Jarmo Tiitto]

Recent LLVM-13 can't read the version 5 profile file any more
so update the format to .profraw file version 7.

This version adds binary_ids_size entry to struct llvm_prf_header
and it is output as zero.

This is enough to fix processing the profile data on LLVM-13 tools.

Signed-off-by: Jarmo Tiitto <jarmo....@gmail.com>
---

kernel/pgo/fs.c | 1 +
kernel/pgo/pgo.h | 4 +++-

[Jarmo Tiitto]

The compiler leaves many duplicate instances of __llvm_prf_* sections
in final module objects and they confuse the kernel module loader.
Fix this by providing module.lds.S linker script that merges the split
sections like vmlinux.lds.S currently does.

Signed-off-by: Jarmo Tiitto <jarmo....@gmail.com>
---