[PATCH 1/4] kbuild: remove LLVM=1 test from HAS_LTO_CLANG

[Masahiro Yamada]

This guarding is wrong. As Documentation/kbuild/llvm.rst notes, LLVM=1
switches the default of tools, but you can still override CC, LD, etc.
individually.

BTW, LLVM is not 1/0 flag. If LLVM is not passed in, it is empty.

Non-zero return code is all treated as failure anyway.

So, $(success,test $(LLVM) -eq 1) and $(success,test "$(LLVM)" = 1)
works equivalently in the sense that both are expanded to 'n' if LLVM
is not given. The difference is that the former internally fails due
to syntax error.

$ test ${LLVM} -eq 1
bash: test: -eq: unary operator expected
$ echo $?
2

$ test "${LLVM}" -eq 1
bash: test: : integer expression expected
$ echo $?
2

$ test "${LLVM}" = 1
echo $?
1

$ test -n "${LLVM}"
$ echo $?
1

Signed-off-by: Masahiro Yamada <masa...@kernel.org>
---

arch/Kconfig | 1 -
1 file changed, 1 deletion(-)

diff --git a/arch/Kconfig b/arch/Kconfig
index 2bb30673d8e6..2af10ebe5ed0 100644
--- a/arch/Kconfig
+++ b/arch/Kconfig
@@ -632,7 +632,6 @@ config HAS_LTO_CLANG
def_bool y
# Clang >= 11: https://github.com/ClangBuiltLinux/linux/issues/510
depends on CC_IS_CLANG && CLANG_VERSION >= 110000 && LD_IS_LLD
- depends on $(success,test $(LLVM) -eq 1)
depends on $(success,test $(LLVM_IAS) -eq 1)
depends on $(success,$(NM) --help | head -n 1 | grep -qi llvm)
depends on $(success,$(AR) --help | head -n 1 | grep -qi llvm)
--
2.27.0

[Masahiro Yamada]

Documentation/process/changes.rst defines the minimum assembler version
(binutils version), but we have never checked it in the build time.

Kbuild never invokes 'as' directly because all assembly files in the
kernel tree are *.S, hence must be preprocessed. I do not expect
raw assembly source files (*.s) would be added to the kernel tree.

Therefore, we always use $(CC) as the assembler driver, and commit
aa824e0c962b ("kbuild: remove AS variable") removed 'AS'. However,
we are still interested in the version of the assembler sitting behind.

As usual, the --version option prints the version string.

$ as --version | head -n 1
GNU assembler (GNU Binutils for Ubuntu) 2.35.1

But, we do not have $(AS). So, we can add the -Wa prefix so that
$(CC) passes --version down to the backing assembler.

$ gcc -Wa,--version | head -n 1
gcc: fatal error: no input files
compilation terminated.

OK, we need to input something to satisfy gcc.

$ gcc -Wa,--version -c -x assembler /dev/null -o /dev/null | head -n 1
GNU assembler (GNU Binutils for Ubuntu) 2.35.1

The combination of Clang and GNU assembler works in the same way:

$ clang -no-integrated-as -Wa,--version -c -x assembler /dev/null -o /dev/null | head -n 1
GNU assembler (GNU Binutils for Ubuntu) 2.35.1

Clang with the integrated assembler fails like this:

$ clang -integrated-as -Wa,--version -c -x assembler /dev/null -o /dev/null | head -n 1
clang: error: unsupported argument '--version' to option 'Wa,'

With all this in my mind, I implemented scripts/as-version.sh.

$ scripts/as-version.sh gcc
GNU 23501
$ scripts/as-version.sh clang -no-integrated-as
GNU 23501
$ scripts/as-version.sh clang -integrated-as
LLVM 0

Signed-off-by: Masahiro Yamada <masa...@kernel.org>
---

arch/Kconfig | 3 +-
init/Kconfig | 12 +++++++
scripts/Kconfig.include | 6 ++++
scripts/as-version.sh | 77 +++++++++++++++++++++++++++++++++++++++++
4 files changed, 96 insertions(+), 2 deletions(-)
create mode 100755 scripts/as-version.sh

diff --git a/arch/Kconfig b/arch/Kconfig
index 2af10ebe5ed0..d7214f4ae1f7 100644
--- a/arch/Kconfig
+++ b/arch/Kconfig
@@ -631,8 +631,7 @@ config ARCH_SUPPORTS_LTO_CLANG_THIN

config HAS_LTO_CLANG
def_bool y
# Clang >= 11: https://github.com/ClangBuiltLinux/linux/issues/510

- depends on CC_IS_CLANG && CLANG_VERSION >= 110000 && LD_IS_LLD
- depends on $(success,test $(LLVM_IAS) -eq 1)
+ depends on CC_IS_CLANG && CLANG_VERSION >= 110000 && LD_IS_LLD && AS_IS_LLVM

depends on $(success,$(NM) --help | head -n 1 | grep -qi llvm)
depends on $(success,$(AR) --help | head -n 1 | grep -qi llvm)

depends on ARCH_SUPPORTS_LTO_CLANG
diff --git a/init/Kconfig b/init/Kconfig
index 22946fe5ded9..f76e5a44e4fe 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -41,6 +41,18 @@ config CLANG_VERSION
default $(cc-version) if CC_IS_CLANG
default 0

+config AS_IS_GNU
+ def_bool $(success,test "$(as-name)" = GNU)
+
+config AS_IS_LLVM
+ def_bool $(success,test "$(as-name)" = LLVM)
+
+config AS_VERSION
+ int
+ # If it is integrated assembler, the version is the same as Clang's one.
+ default CLANG_VERSION if AS_IS_LLVM
+ default $(as-version)
+
config LD_IS_BFD
def_bool $(success,test "$(ld-name)" = BFD)

diff --git a/scripts/Kconfig.include b/scripts/Kconfig.include
index 58fdb5308725..0496efd6e117 100644
--- a/scripts/Kconfig.include
+++ b/scripts/Kconfig.include
@@ -45,6 +45,12 @@ $(error-if,$(success,test -z "$(cc-info)"),Sorry$(comma) this compiler is not su
cc-name := $(shell,set -- $(cc-info) && echo $1)
cc-version := $(shell,set -- $(cc-info) && echo $2)

+# Get the assembler name, version, and error out if it is not supported.
+as-info := $(shell,$(srctree)/scripts/as-version.sh $(CC) $(CLANG_FLAGS))
+$(error-if,$(success,test -z "$(as-info)"),Sorry$(comma) this assembler is not supported.)
+as-name := $(shell,set -- $(as-info) && echo $1)
+as-version := $(shell,set -- $(as-info) && echo $2)
+
# Get the linker name, version, and error out if it is not supported.
ld-info := $(shell,$(srctree)/scripts/ld-version.sh $(LD))
$(error-if,$(success,test -z "$(ld-info)"),Sorry$(comma) this linker is not supported.)
diff --git a/scripts/as-version.sh b/scripts/as-version.sh
new file mode 100755
index 000000000000..205d8b9fc4d4
--- /dev/null
+++ b/scripts/as-version.sh
@@ -0,0 +1,77 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Print the assembler name and its version in a 5 or 6-digit form.
+# Also, perform the minimum version check.
+# (If it is the integrated assembler, return 0 as the version, and
+# the version check is skipped.)
+
+set -e
+
+# Convert the version string x.y.z to a canonical 5 or 6-digit form.
+get_canonical_version()
+{
+ IFS=.
+ set -- $1
+
+ # If the 2nd or 3rd field is missing, fill it with a zero.
+ #
+ # The 4th field, if present, is ignored.
+ # This occurs in development snapshots as in 2.35.1.20201116
+ echo $((10000 * $1 + 100 * ${2:-0} + ${3:-0}))
+}
+
+orig_args="$@"
+
+# Get the first line of the --version output.
+IFS='
+'
+# Add 2>&1 to check both stdout and stderr.
+# If the backing assembler is binutils, we get the version string in stdout.
+# If it is clang's integrated assembler, we get the following error in stderr:
+# clang: error: unsupported argument '--version' to option 'Wa,'
+# To avoid the error message affected by locale, set LC_MESSAGES=C just in case.
+set -- $(LC_MESSAGES=C "$@" -Wno-unused-command-line-argument -Wa,--version -c -x assembler /dev/null -o /dev/null 2>&1)
+line="$1"
+
+if [ "$line" = "clang: error: unsupported argument '--version' to option 'Wa,'" ]; then
+ # For the intergrated assembler, we do not check the version here.
+ # It is the same as the clang version, and it has been already checked
+ # by scripts/cc-version.sh.
+ echo LLVM 0
+ exit 0
+fi
+
+# Split the line on spaces.
+IFS=' '
+set -- $line
+
+tool_version=$(dirname $0)/tool-version.sh
+
+if [ "$1" = GNU -a "$2" = assembler ]; then
+ shift $(($# - 1))
+ version=$1
+ min_version=$($tool_version binutils)
+ name=GNU
+else
+ echo "$orig_args: unknown assembler invoked" >&2
+ exit 1
+fi
+
+# Some distributions append a package release number, as in 2.34-4.fc32
+# Trim the hyphen and any characters that follow.
+version=${version%-*}
+
+cversion=$(get_canonical_version $version)
+min_cversion=$(get_canonical_version $min_version)
+
+if [ "$cversion" -lt "$min_cversion" ]; then
+ echo >&2 "***"
+ echo >&2 "*** Assembler is too old."
+ echo >&2 "*** Your $name assembler version: $version"
+ echo >&2 "*** Minimum $name assembler version: $min_version"
+ echo >&2 "***"
+ exit 1
+fi
+
+echo $name $cversion
--
2.27.0

[Nick Desaulniers]

+ Sami

On Wed, Mar 3, 2021 at 10:34 AM Masahiro Yamada <masa...@kernel.org> wrote:
>
> This guarding is wrong. As Documentation/kbuild/llvm.rst notes, LLVM=1
> switches the default of tools, but you can still override CC, LD, etc.
> individually.
>
> BTW, LLVM is not 1/0 flag. If LLVM is not passed in, it is empty.

Do we have the same problem with LLVM_IAS? LGTM otherwise, but wanted
to check that before signing off.

(Also, the rest of the patches in this series seem more related to
DWARFv5 cleanups; this patch seems orthogonal while those are a
visible progression).

IIRC, we needed some other LLVM utilities like llvm-nm and llvm-ar,
which are checked below. So I guess we can still support CC=clang
AR=llvm-ar NM=llvm-nm, and this check is redundant.

> depends on $(success,test $(LLVM_IAS) -eq 1)
> depends on $(success,$(NM) --help | head -n 1 | grep -qi llvm)
> depends on $(success,$(AR) --help | head -n 1 | grep -qi llvm)
> --
> 2.27.0
>

--
Thanks,
~Nick Desaulniers

[Nathan Chancellor]

Yes, there is not too much point of checking if $(LLVM) is set or not
because we already check for the other tools.

Reviewed-by: Nathan Chancellor <nat...@kernel.org>

[Nathan Chancellor]

It looks like Debian and Ubuntu do something interesting with their
clang packaging. Normally, "clang" is a symlink to "clang-#" in the same
folder. In my folder from just running "ninja install"

$ ls -l clang clang-13
lrwxrwxrwx 8 nathan 2 Mar 20:58 clang -> clang-13
.rwxr-xr-x 93M nathan 2 Mar 20:58 clang-13

The same thing is true for Arch Linux:

$ ls -l /usr/bin/{clang,clang-11}
lrwxrwxrwx 8 root 17 Feb 8:54 /usr/bin/clang -> clang-11
.rwxr-xr-x 145k root 17 Feb 8:54 /usr/bin/clang-11

As a result, this does not quite work.

$ LC_MESSAGES=C /usr/bin/clang -Wno-unused-command-line-argument -Wa,--version -c -x assembler /dev/null -o /dev/null
clang-11: error: unsupported argument '--version' to option 'Wa,'

I am not sure what the cleanest fix would be. The rest of the patch looks okay to me.

Cheers,
Nathan

[Nick Desaulniers]

On Wed, Mar 3, 2021 at 10:34 AM Masahiro Yamada <masa...@kernel.org> wrote:
>

Was this a feature request to "please implement -Wa,--version for clang?" :-P
https://github.com/ClangBuiltLinux/linux/issues/1320

Checking the precise error message is too brittle; what if it changes?
Why not check the return code a la cc-option and friends? Is checking
the return code of a subshell an issue here?

> + # For the intergrated assembler, we do not check the version here.

s/intergrated/integrated/

--
Thanks,
~Nick Desaulniers

[Masahiro Yamada]

On Thu, Mar 4, 2021 at 5:47 AM 'Nick Desaulniers' via Clang Built
Linux <clang-bu...@googlegroups.com> wrote:
>
> + Sami
>
> On Wed, Mar 3, 2021 at 10:34 AM Masahiro Yamada <masa...@kernel.org> wrote:
> >
> > This guarding is wrong. As Documentation/kbuild/llvm.rst notes, LLVM=1
> > switches the default of tools, but you can still override CC, LD, etc.
> > individually.
> >
> > BTW, LLVM is not 1/0 flag. If LLVM is not passed in, it is empty.
>
> Do we have the same problem with LLVM_IAS? LGTM otherwise, but wanted
> to check that before signing off.

3/4 will replace this LLVM_IAS check with AS_IS_LLVM.

We do not need to add a noise change.

>
> (Also, the rest of the patches in this series seem more related to
> DWARFv5 cleanups; this patch seems orthogonal while those are a
> visible progression).

Kind of orthogonal, but I am touching the same code hunk,
which would cause a merge conflict.

Yes, I think so.

>
> > depends on $(success,test $(LLVM_IAS) -eq 1)
> > depends on $(success,$(NM) --help | head -n 1 | grep -qi llvm)
> > depends on $(success,$(AR) --help | head -n 1 | grep -qi llvm)
> > --
> > 2.27.0
> >
>
>
> --
> Thanks,
> ~Nick Desaulniers
>

> --
> You received this message because you are subscribed to the Google Groups "Clang Built Linux" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to clang-built-li...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/CAKwvOdkhZGv_q9vgDdYY44OrbzmMD_E%2BGL3SyOk-jQ0kdXtMzg%40mail.gmail.com.



--
Best Regards
Masahiro Yamada

[Masahiro Yamada]

As Nathan pointed out, this is fragile.

I just tried my best to make this script work stand-alone.

With the only exit code chec,

scripts/as-version.sh false

would return LLVM, which is false-positive.
Probably this is not a big deal
since completely wrong input was blocked by
scripts/cc-version.sh


If we give up making this script stand-alone,
another idea is to check LLVM_IAS=1, which
is passed from the top Makefile.

Yet another idea is to explicitly pass
-integrated-as as CLANG_FLAGS.
Then, this script can parse the presence
of -integrated-as.


BTW, is there any guarantee that the integrated assembler
is always enabled by default? Or, is it dependent on
the configuration?

The top Makefile adds -no-integrated-as if LLVM_IAS != 1,
but adds nothing if LLVM_IAS == 1.
So, -integrated-as must be always default.

[Nathan Chancellor]

This is probably the easiest thing to do for all versions of LLVM
(adding -Wa,--version support to the integrated assembler as Nick
suggests only fixes future versions of LLVM) and goes along with the
"explicit is better than implicit" policy, which I tend to align with.
It makes easy to quickly see what assembler was used when looking at a
clang invocation as well.

> BTW, is there any guarantee that the integrated assembler
> is always enabled by default? Or, is it dependent on
> the configuration?

There is a virtual method called IsIntegratedAssemblerDefault(), which
defaults to false but is overridden to true by various targets. For all
of the architectures/targets that the kernel cares about, this appears
to be true.

https://github.com/llvm/llvm-project/blob/45f949ee469f3e59d30a88c97aa7c1139069b261/clang/lib/Driver/ToolChains/Gnu.cpp#L2738-L2774

Cheers,
Nathan

[Sami Tolvanen]

I'm fine with removing the check, but the idea here was to just make
it slightly harder for people to accidentally use a mismatched
toolchain, even though checking for LLVM=1 doesn't stop them from
doing so anyway. But yes, the only LLVM tools required in addition to
the compiler and the linker are llvm-ar and llvm-nm.

Sami

[Masahiro Yamada]

As Documentation/kbuild/llvm.rst notes, LLVM=1 switches the default of

tools, but you can still override CC, LD, etc. individually. This LLVM=1
check is unneeded because each tool is already checked separately.

"make CC=clang LD=ld.lld NM=llvm-nm AR=llvm-ar LLVM_IAS=1 menuconfig"
should be able to enable Clang LTO.

Signed-off-by: Masahiro Yamada <masa...@kernel.org>
Reviewed-by: Nathan Chancellor <nat...@kernel.org>

---

arch/Kconfig | 1 -
1 file changed, 1 deletion(-)

diff --git a/arch/Kconfig b/arch/Kconfig

index 2e7139b39e8f..ecfd3520b676 100644

--- a/arch/Kconfig
+++ b/arch/Kconfig
@@ -632,7 +632,6 @@ config HAS_LTO_CLANG
def_bool y
# Clang >= 11: https://github.com/ClangBuiltLinux/linux/issues/510
depends on CC_IS_CLANG && CLANG_VERSION >= 110000 && LD_IS_LLD
- depends on $(success,test $(LLVM) -eq 1)

depends on $(success,test $(LLVM_IAS) -eq 1)

depends on $(success,$(NM) --help | head -n 1 | grep -qi llvm)
depends on $(success,$(AR) --help | head -n 1 | grep -qi llvm)

--
2.27.0