[PATCH v2 0/2] infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1

[Nick Desaulniers]

We get constant feedback that the command line invocation of make is too
long. CROSS_COMPILE is helpful when a toolchain has a prefix of the
target triple, or is an absolute path outside of $PATH, but it's mostly
redundant for a given ARCH.

Instead, let's infer it from SRCARCH, and move some flag handling into a
new file included from the top level Makefile.

Changes v1 -> v2:
* patch 1/2 untouched.
* Fix typos in commit message as per Geert and Masahiro.
* Use SRCARCH instead of ARCH, simplifying x86 handling, as per
Masahiro. Add his sugguested by tag.
* change commit oneline from 'drop' to 'infer.'
* Add detail about explicit host --target and relationship of ARCH to
SRCARCH, as per Masahiro.

Nick Desaulniers (2):
Makefile: move initial clang flag handling into scripts/Makefile.clang
Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1

Documentation/kbuild/llvm.rst | 5 ++++
MAINTAINERS | 1 +
Makefile | 15 +-----------
scripts/Makefile.clang | 44 +++++++++++++++++++++++++++++++++++
4 files changed, 51 insertions(+), 14 deletions(-)
create mode 100644 scripts/Makefile.clang


base-commit: a0e781a2a35a8dd4e6a38571998d59c6b0e32cd8
--
2.32.0.93.g670b81a890-goog

[Nick Desaulniers]

With some of the changes we'd like to make to CROSS_COMPILE, the initial
block of clang flag handling which controls things like the target triple,
whether or not to use the integrated assembler and how to find GAS,
and erroring on unknown warnings is becoming unwieldy. Move it into its
own file under scripts/.

Signed-off-by: Nick Desaulniers <ndesau...@google.com>
---
MAINTAINERS | 1 +
Makefile | 15 +--------------
scripts/Makefile.clang | 14 ++++++++++++++
3 files changed, 16 insertions(+), 14 deletions(-)
create mode 100644 scripts/Makefile.clang

diff --git a/MAINTAINERS b/MAINTAINERS
index 81e1edeceae4..9c1205c258c7 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -4433,6 +4433,7 @@ B: https://github.com/ClangBuiltLinux/linux/issues
C: irc://chat.freenode.net/clangbuiltlinux
F: Documentation/kbuild/llvm.rst
F: include/linux/compiler-clang.h
+F: scripts/Makefile.clang
F: scripts/clang-tools/
K: \b(?i:clang|llvm)\b

diff --git a/Makefile b/Makefile
index cbab0dc53065..010e3a4e770b 100644
--- a/Makefile
+++ b/Makefile
@@ -586,20 +586,7 @@ endif
CC_VERSION_TEXT = $(subst $(pound),,$(shell $(CC) --version 2>/dev/null | head -n 1))

ifneq ($(findstring clang,$(CC_VERSION_TEXT)),)
-ifneq ($(CROSS_COMPILE),)
-CLANG_FLAGS += --target=$(notdir $(CROSS_COMPILE:%-=%))
-endif
-ifeq ($(LLVM_IAS),1)
-CLANG_FLAGS += -integrated-as
-else
-CLANG_FLAGS += -no-integrated-as
-GCC_TOOLCHAIN_DIR := $(dir $(shell which $(CROSS_COMPILE)elfedit))
-CLANG_FLAGS += --prefix=$(GCC_TOOLCHAIN_DIR)$(notdir $(CROSS_COMPILE))
-endif
-CLANG_FLAGS += -Werror=unknown-warning-option
-KBUILD_CFLAGS += $(CLANG_FLAGS)
-KBUILD_AFLAGS += $(CLANG_FLAGS)
-export CLANG_FLAGS
+include $(srctree)/scripts/Makefile.clang
endif

# Include this also for config targets because some architectures need
diff --git a/scripts/Makefile.clang b/scripts/Makefile.clang
new file mode 100644
index 000000000000..297932e973d4
--- /dev/null
+++ b/scripts/Makefile.clang
@@ -0,0 +1,14 @@
+ifneq ($(CROSS_COMPILE),)
+CLANG_FLAGS += --target=$(notdir $(CROSS_COMPILE:%-=%))
+endif
+ifeq ($(LLVM_IAS),1)
+CLANG_FLAGS += -integrated-as
+else
+CLANG_FLAGS += -no-integrated-as
+GCC_TOOLCHAIN_DIR := $(dir $(shell which $(CROSS_COMPILE)elfedit))
+CLANG_FLAGS += --prefix=$(GCC_TOOLCHAIN_DIR)$(notdir $(CROSS_COMPILE))
+endif
+CLANG_FLAGS += -Werror=unknown-warning-option
+KBUILD_CFLAGS += $(CLANG_FLAGS)
+KBUILD_AFLAGS += $(CLANG_FLAGS)
+export CLANG_FLAGS
--
2.32.0.93.g670b81a890-goog

[Nick Desaulniers]

We get constant feedback that the command line invocation of make is too
long. CROSS_COMPILE is helpful when a toolchain has a prefix of the
target triple, or is an absolute path outside of $PATH, but it's mostly

redundant for a given SRCARCH. SRCARCH itself is derived from ARCH
(normalized for a few different targets).

If CROSS_COMPILE is not set, simply set --target= for CLANG_FLAGS,
KBUILD_CFLAGS, and KBUILD_AFLAGS based on $SRCARCH.

Previously, we'd cross compile via:
$ ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- make LLVM=1 LLVM_IAS=1
Now:
$ ARCH=arm64 make LLVM=1 LLVM_IAS=1

For native builds (not involving cross compilation) we now explicitly
specify a target triple rather than rely on the implicit host triple.

Link: https://github.com/ClangBuiltLinux/linux/issues/1399
Suggested-by: Arnd Bergmann <ar...@kernel.org>
Suggested-by: Nathan Chancellor <nat...@kernel.org>
Suggested-by: Masahiro Yamada <masa...@kernel.org>

Signed-off-by: Nick Desaulniers <ndesau...@google.com>
---

Changes v1 -> v2:

* Fix typos in commit message as per Geert and Masahiro.
* Use SRCARCH instead of ARCH, simplifying x86 handling, as per
Masahiro. Add his sugguested by tag.
* change commit oneline from 'drop' to 'infer.'
* Add detail about explicit host --target and relationship of ARCH to
SRCARCH, as per Masahiro.

Changes RFC -> v1:
* Rebase onto linux-kbuild/for-next
* Keep full target triples since missing the gnueabi suffix messes up
32b ARM. Drop Fangrui's sugguested by tag. Update commit message to
drop references to arm64.
* Flush out TODOS.
* Add note about -EL/-EB, -m32/-m64.
* Add note to Documentation/.

Documentation/kbuild/llvm.rst | 5 +++++
scripts/Makefile.clang | 34 ++++++++++++++++++++++++++++++++--
2 files changed, 37 insertions(+), 2 deletions(-)

diff --git a/Documentation/kbuild/llvm.rst b/Documentation/kbuild/llvm.rst
index b18401d2ba82..80c63dd9a6d1 100644
--- a/Documentation/kbuild/llvm.rst
+++ b/Documentation/kbuild/llvm.rst
@@ -46,6 +46,11 @@ example: ::

clang --target=aarch64-linux-gnu foo.c

+When both ``LLVM=1`` and ``LLVM_IAS=1`` are used, ``CROSS_COMPILE`` becomes
+unnecessary and can be inferred from ``ARCH``. Example: ::
+
+ ARCH=arm64 make LLVM=1 LLVM_IAS=1
+
LLVM Utilities
--------------

diff --git a/scripts/Makefile.clang b/scripts/Makefile.clang
index 297932e973d4..956603f56724 100644
--- a/scripts/Makefile.clang
+++ b/scripts/Makefile.clang
@@ -1,6 +1,36 @@
-ifneq ($(CROSS_COMPILE),)
+# Individual arch/{arch}/Makfiles should use -EL/-EB to set intended endianness
+# and -m32/-m64 to set word size based on Kconfigs instead of relying on the
+# target triple.
+ifeq ($(CROSS_COMPILE),)
+ifneq ($(LLVM),)
+ifeq ($(LLVM_IAS),1)
+ifeq ($(SRCARCH),arm)
+CLANG_FLAGS += --target=arm-linux-gnueabi
+else ifeq ($(SRCARCH),arm64)
+CLANG_FLAGS += --target=aarch64-linux-gnu
+else ifeq ($(SRCARCH),hexagon)
+CLANG_FLAGS += --target=hexagon-linux-gnu
+else ifeq ($(SRCARCH),m68k)
+CLANG_FLAGS += --target=m68k-linux-gnu
+else ifeq ($(SRCARCH),mips)
+CLANG_FLAGS += --target=mipsel-linux-gnu
+else ifeq ($(SRCARCH),powerpc)
+CLANG_FLAGS += --target=powerpc64le-linux-gnu
+else ifeq ($(SRCARCH),riscv)
+CLANG_FLAGS += --target=riscv64-linux-gnu
+else ifeq ($(SRCARCH),s390)
+CLANG_FLAGS += --target=s390x-linux-gnu
+else ifeq ($(SRCARCH),x86)
+CLANG_FLAGS += --target=x86_64-linux-gnu
+else
+$(error Specify CROSS_COMPILE or add '--target=' option to scripts/Makefile.clang)
+endif # SRCARCH
+endif # LLVM_IAS
+endif # LLVM
+else

CLANG_FLAGS += --target=$(notdir $(CROSS_COMPILE:%-=%))
-endif

+endif # CROSS_COMPILE
+
ifeq ($(LLVM_IAS),1)
CLANG_FLAGS += -integrated-as
else
--
2.32.0.93.g670b81a890-goog

[Nathan Chancellor]

On 7/8/2021 4:25 PM, 'Nick Desaulniers' via Clang Built Linux wrote:
> With some of the changes we'd like to make to CROSS_COMPILE, the initial
> block of clang flag handling which controls things like the target triple,
> whether or not to use the integrated assembler and how to find GAS,
> and erroring on unknown warnings is becoming unwieldy. Move it into its
> own file under scripts/.
>
> Signed-off-by: Nick Desaulniers <ndesau...@google.com>

Reviewed-by: Nathan Chancellor <nat...@kernel.org>

[Nathan Chancellor]

On 7/8/2021 4:25 PM, 'Nick Desaulniers' via Clang Built Linux wrote:

> We get constant feedback that the command line invocation of make is too
> long. CROSS_COMPILE is helpful when a toolchain has a prefix of the
> target triple, or is an absolute path outside of $PATH, but it's mostly
> redundant for a given SRCARCH. SRCARCH itself is derived from ARCH
> (normalized for a few different targets).
>
> If CROSS_COMPILE is not set, simply set --target= for CLANG_FLAGS,
> KBUILD_CFLAGS, and KBUILD_AFLAGS based on $SRCARCH.
>
> Previously, we'd cross compile via:
> $ ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- make LLVM=1 LLVM_IAS=1
> Now:
> $ ARCH=arm64 make LLVM=1 LLVM_IAS=1
>
> For native builds (not involving cross compilation) we now explicitly
> specify a target triple rather than rely on the implicit host triple.
>
> Link: https://github.com/ClangBuiltLinux/linux/issues/1399
> Suggested-by: Arnd Bergmann <ar...@kernel.org>
> Suggested-by: Nathan Chancellor <nat...@kernel.org>
> Suggested-by: Masahiro Yamada <masa...@kernel.org>
> Signed-off-by: Nick Desaulniers <ndesau...@google.com>

I tested arm, arm64, i386, riscv, and x86_64 and verified that the error
fired for an unsupported arch. Consider this:

Reviewed-by: Nathan Chancellor <nat...@kernel.org>
Tested-by: Nathan Chancellor <nat...@kernel.org>

[Masahiro Yamada]

Do you need to check $(LLVM) ?


LLVM=1 is a convenient switch to change all the
defaults, but yet you can flip each tool individually.

Instead of LLVM=1, you still should be able to
get the equivalent setups by:


make CC=clang LD=ld.lld AR=llvm-ar OBJCOPY=llvm-objcopy ...


The --target option is passed to only
KBUILD_CFLAGS and KBUILD_AFLAGS.

So, when we talk about --target=,
we only care about whether $(CC) is Clang.
Not caring about $(AR), $(LD), or $(OBJCOPY).


scripts/Makefile.clang is already guarded by:

ifneq ($(findstring clang,$(CC_VERSION_TEXT)),

> --
> You received this message because you are subscribed to the Google Groups "Clang Built Linux" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to clang-built-li...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/20210708232522.3118208-3-ndesaulniers%40google.com.



--
Best Regards
Masahiro Yamada

[Nathan Chancellor]

$ make ARCH=arm64 CC=clang LLVM_IAS=1

will use the right compiler and assembler but none of the other binary
tools because '--prefix=' will not be set so CROSS_COMPILE needs to be
specified still, which defeats the purpose of this whole change. This
patch is designed to work for the "normal" case of saying "I want to use
all of the LLVM tools", not "I want to use clang by itself".

[Linus Torvalds]

On Tue, Jul 20, 2021 at 1:05 AM Masahiro Yamada <masa...@kernel.org> wrote:
>
> LLVM=1 is a convenient switch to change all the
> defaults, but yet you can flip each tool individually.

Actually, I'd argue that "LLVM=1" is *not* a convenient switch.
Neither are the individual other command line settings.

When clang was the odd man out, and special, it all made sense.
Changing the path to CC was similar to changing the path to AWK. And
that's obviously why we have what we have.

But clang has become a primary compiler for some kernel communities,
and I think it might be time to just re-visit that entirely.

In particular, I think we should just make it a Kconfig option. I hate
the command flag stuff so much, that my clang tree literally has this
patch in it:

-CC = $(CROSS_COMPILE)gcc
+CC = $(CROSS_COMPILE)clang

so that I can just do the same "make -j128" in both my gcc tree and my
clang tree.

But each build tree already has its own .config file, so it would be a
lot more convenient if that was how the compiler was chosen, and then
"make oldconfig" would just DTRT.

We do most of the other heavy lifting in this area in Kconfig anyway,
why not add that compiler choice?

Obviously it would be gated by the tests to see which compilers are
_installed_ (and that they are valid versions), so that it doesn't ask
stupid things ("do you want gcc or clang" when only one of them is
installed and/or viable).

Hmm? So then any "LLVM=1" thing would be about the "make config"
stage, not the actual build stage.

(It has annoyed me for years that if you want to cross-compile, you
first have to do "make ARCH=xyz config" and then remember to do "make
ARCH=xyz" for the build too, but I cross-compile so seldom that I've
never really cared).

Let the flame wars^H^Hpolite discussions ensue..

Linus

[Arnd Bergmann]

On Tue, Jul 20, 2021 at 7:43 PM Linus Torvalds
<torv...@linux-foundation.org> wrote:
> On Tue, Jul 20, 2021 at 1:05 AM Masahiro Yamada <masa...@kernel.org> wrote:
>
> We do most of the other heavy lifting in this area in Kconfig anyway,
> why not add that compiler choice?
>
> Obviously it would be gated by the tests to see which compilers are
> _installed_ (and that they are valid versions), so that it doesn't ask
> stupid things ("do you want gcc or clang" when only one of them is
> installed and/or viable).

I don't see a good way of making Kconfig options both select the
compiler and defining variables based on the compiler, since that
would mean teaching Kconfig about re-evaluating all compiler
dependent settings whenever the first option changes.

I do have another idea that I think would work though.

> Hmm? So then any "LLVM=1" thing would be about the "make config"
> stage, not the actual build stage.
>
> (It has annoyed me for years that if you want to cross-compile, you
> first have to do "make ARCH=xyz config" and then remember to do "make
> ARCH=xyz" for the build too, but I cross-compile so seldom that I've
> never really cared).

The best thing that I have come up with is a pre-configure step, where
an object tree gets seeded with a makefile fragment that gets included
for any 'make' invocation. This would set 'ARCH=', 'CROSS_COMPILE',
'CC=' and possibly any other option that gets passed to 'make' as
a variable and has to exist before calling 'make *config'.

This is probably the easiest part here, and it lets you set up multiple
object directories in which you can then do

make O=obj-x86 defconfig vmlinux modules -skj32
make O=obj-x86-clang
make O=obj-arm64 randconfig
...

without ever having to type the additional CC/CROSS_COMPILE/LLVM
variables again.

One step further is the script that automatically finds a working toolchain
for a given architecture and sets up that object directory accordingly,
or even downloads a working cross-toolchain from kernel.org when
asked to do that.

Arnd

[Nick Desaulniers]

On Tue, Jul 20, 2021 at 12:58 PM Arnd Bergmann <ar...@kernel.org> wrote:
>
> On Tue, Jul 20, 2021 at 7:43 PM Linus Torvalds
> <torv...@linux-foundation.org> wrote:
> > On Tue, Jul 20, 2021 at 1:05 AM Masahiro Yamada <masa...@kernel.org> wrote:
> >
> > We do most of the other heavy lifting in this area in Kconfig anyway,
> > why not add that compiler choice?
> >
> > Obviously it would be gated by the tests to see which compilers are
> > _installed_ (and that they are valid versions), so that it doesn't ask
> > stupid things ("do you want gcc or clang" when only one of them is
> > installed and/or viable).
>
> I don't see a good way of making Kconfig options both select the
> compiler and defining variables based on the compiler, since that
> would mean teaching Kconfig about re-evaluating all compiler
> dependent settings whenever the first option changes.

Right, there are many Kconfigs that are predicated on knowing whether
the user is building with gcc vs clang. We probably could note the
users preference in Kconfig, but we would need to guarantee that we
evaluate that ASAP and before most other Kconfigs were checked.
--
Thanks,
~Nick Desaulniers

[Nick Desaulniers]

On Tue, Jul 20, 2021 at 10:43 AM Linus Torvalds
<torv...@linux-foundation.org> wrote:
>
> On Tue, Jul 20, 2021 at 1:05 AM Masahiro Yamada <masa...@kernel.org> wrote:
> >
> > LLVM=1 is a convenient switch to change all the
> > defaults, but yet you can flip each tool individually.
>
> Actually, I'd argue that "LLVM=1" is *not* a convenient switch.

Compared to the old way of CC=clang LD=ld.lld OBJCOPY=.... it certainly is.

> Neither are the individual other command line settings.

Agreed, but we needed flexibility until we could get all of the
command line tools working for each architecture. They're still
useful when there's a regression and we need to fall back. So I
wouldn't be in favor of removing them (not that that's been proposed).

> When clang was the odd man out, and special, it all made sense.
> Changing the path to CC was similar to changing the path to AWK. And
> that's obviously why we have what we have.
>
> But clang has become a primary compiler for some kernel communities,
> and I think it might be time to just re-visit that entirely.

:^)

> In particular, I think we should just make it a Kconfig option. I hate
> the command flag stuff so much, that my clang tree literally has this
> patch in it:
>
> -CC = $(CROSS_COMPILE)gcc
> +CC = $(CROSS_COMPILE)clang
>
> so that I can just do the same "make -j128" in both my gcc tree and my
> clang tree.

So you haven't been using LLD... :( (imagine using more than one
thread to link, and being faster than ld.gold) If anything you should
be hard coding LLVM=1 in that tree. Also, please be careful you don't
accidentally commit that! 0:-)

> But each build tree already has its own .config file, so it would be a
> lot more convenient if that was how the compiler was chosen, and then
> "make oldconfig" would just DTRT.
>
> We do most of the other heavy lifting in this area in Kconfig anyway,
> why not add that compiler choice?
>
> Obviously it would be gated by the tests to see which compilers are
> _installed_ (and that they are valid versions), so that it doesn't ask
> stupid things ("do you want gcc or clang" when only one of them is
> installed and/or viable).
>
> Hmm? So then any "LLVM=1" thing would be about the "make config"
> stage, not the actual build stage.
>
> (It has annoyed me for years that if you want to cross-compile, you
> first have to do "make ARCH=xyz config" and then remember to do "make
> ARCH=xyz" for the build too, but I cross-compile so seldom that I've
> never really cared).
>
> Let the flame wars^H^Hpolite discussions ensue..

I agree with you. Overall the command line invocation of make when
cross compiling, or when using LLVM is too long. You even call out
LLVM=1 and ARCH separately. Each one of these had good reasons to
exist for years.

But I disagree that all needs to be sorted out together, or right now.
And I'd much rather tackle them separately, one by one, than try to
completely rewrite how we cross compile the kernel today.

Right now, we have:
$ ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- make LLVM=1 LLVM_IAS=1 -j72

This series is concerned with just CROSS_COMPILE (and just for LLVM=1).
Next I plan to default on LLVM_IAS=1 for all architectures we support,
minus ppc and s390 where we still have some assembler bugs.
Your/Arnd's ideas about LLVM=1 or not via Kconfig, or pre-Kconfig is a
good idea for eliminating LLVM=1.
Then that just leaves ARCH.
Arnd's idea about helping you install a toolchain from kernel.org is
one I support, but orthogonal to the above somewhat. Do you allow
someone to have a config that denotes intent to build with clang then
prompt if they don't have clang installed to download it? Or do you
prevent someone from selecting building with clang because it's not in
the $PATH?
Your/Arnd's idea about detecting which toolchains are installed is one
I support, but orthogonal to the above somewhat. (For that, I'm
curious for our build servers if that means having to put tools in
certain locations; I prefer we reference $PATH when possible. Or if
.configs can no longer be shared if tools are in different locations.
But perhaps that's a non-issue). I'm also curious how many stat calls
we'll need to test/probe/find these, and how we prioritize which tools
are selected when there's more than one version installed.

I encourage us to make steps in the right direction; but I think this
series is ready to go for at least one of the command line variables.
I don't think we need to wait for some probing machinery to eliminate
CROSS_COMPILE when LLVM=1; and if we ever get such machinery we can
revisit whether that helps this case at all.
--
Thanks,
~Nick Desaulniers

[Linus Torvalds]

On Tue, Jul 20, 2021 at 1:52 PM Nick Desaulniers
<ndesau...@google.com> wrote:
>
> > -CC = $(CROSS_COMPILE)gcc
> > +CC = $(CROSS_COMPILE)clang
>

> So you haven't been using LLD... :( (imagine using more than one
> thread to link, and being faster than ld.gold)

I started out just building my own clang - nothing else.

And now that I'm using the distro clang, the linker isn't even _close_
to a performance issue. Since clang is a big, slow, bloated pig.

I posted profiles and performance numbers of clang being three times
slower than gcc, and it was all just clang itself, not the linker. All
due to insanely expensive startup costs.

I suspect a lot of clang users build bloated C++ code where the time
to compile a single object file is so big that the startup costs don't
even show up. But for the (fairly) lean C kernel header files that
don't bring in millions of lines of headers for some template library,
startup costs are a big deal.

So honestly, I don't care at all about "imagine being faster than ld.gold".

I can only _dream_ of clang itself not being 3x slower than gcc.

Linus

[Nick Desaulniers]

On Tue, Jul 20, 2021 at 2:11 PM Linus Torvalds
<torv...@linux-foundation.org> wrote:
>
> On Tue, Jul 20, 2021 at 1:52 PM Nick Desaulniers
> <ndesau...@google.com> wrote:
> >
> > > -CC = $(CROSS_COMPILE)gcc
> > > +CC = $(CROSS_COMPILE)clang
> >
> > So you haven't been using LLD... :( (imagine using more than one
> > thread to link, and being faster than ld.gold)
>
> I started out just building my own clang - nothing else.
>
> And now that I'm using the distro clang, the linker isn't even _close_
> to a performance issue. Since clang is a big, slow, bloated pig.
>
> I posted profiles and performance numbers of clang being three times
> slower than gcc, and it was all just clang itself, not the linker. All
> due to insanely expensive startup costs.

Yeah, it's unfortunate that package maintainers for your distro chose
to distribute clang built in such a manner. But in response to that
thread:
1. I planned a whole conference around how folks distribute LLVM
(particularly how the configurations are done, and important ones to
use):
https://lists.llvm.org/pipermail/llvm-dev/2021-June/150861.html
We have a pretty good set of CFPs lined up (and I need to go chase
folks from Apple and Qualcomm that are working on proposals...)
https://github.com/ClangBuiltLinux/llvm-distributors-conf-2021/issues

2. I brainstormed a whole set of ways to make Clang even faster.
https://github.com/ClangBuiltLinux/tc-build/issues
Most promising I think would be building musl with LTO and statically
linking that into the toolchain. I don't have a ton of time to pursue
that, but I will continue to bang on that drum in front of management.
It's just very hard to fund improving build times for users on
thinkpads when the corporate world has moved on to distributed and
cached building in the cloud.

3. Pretty sure we've resolved that config with your distro's packager,
and made other changes to upstream LLVM to default on some flags that
help a lot when those unfortunate configs are used.
https://reviews.llvm.org/D102090

> I suspect a lot of clang users build bloated C++ code where the time
> to compile a single object file is so big that the startup costs don't
> even show up. But for the (fairly) lean C kernel header files that
> don't bring in millions of lines of headers for some template library,
> startup costs are a big deal.
>
> So honestly, I don't care at all about "imagine being faster than ld.gold".
>
> I can only _dream_ of clang itself not being 3x slower than gcc.

Let's be precise here, because quotes (even when imprecise) lead to PR
cycles. Clang startup time for NULL input when testing command line
flags as part of a build configure step, when clang is misconfigured
is slower than GCC. Clang is not generally 3x slower than GCC.
--
Thanks,
~Nick Desaulniers

[Nick Desaulniers]

On Tue, Jul 20, 2021 at 10:43 AM Linus Torvalds
<torv...@linux-foundation.org> wrote:
>

I will concede that "why do I need to respecify all of my command line
args/env vars to make when I already did so for a configure step?" is
a reasonable question to ask, and may be worth pursuing.

There's still some that we can eliminate for BOTH configure and actual
build, like CROSS_COMPILE. :^)
--
Thanks,
~Nick Desaulniers

[Linus Torvalds]

On Tue, Jul 20, 2021 at 2:29 PM Nick Desaulniers
<ndesau...@google.com> wrote:
>
> There's still some that we can eliminate for BOTH configure and actual
> build, like CROSS_COMPILE. :^)

I have this (perhaps odd) model where I use separate trees for
different things, which is something git obviously is partly designed
for.

So when I create a source tree, I usually create it for one particular
use - whether it be for one particular configuration (ie "build with
clang" or "set up a tree for a 32-bit cross compile" or whatever), or
whether it be for some particular development issue ("long-term branch
for testing asm goto with inputs").

Yeah, sometimes I use multiple git branches in the same tree for
things - but I tend to do that mainly for short-term things where I
don't expect to keep working on it over time.

End result: I find 'make' command line flags to be really really
annoying, because then I have to match them to whatever tree I'm in.

I'd much rather have a local per-tree configuration that just sticks
around. It's why ".config" works fairly well for me.

But there are other things that stick around. In particular, I have
considered simply using git config variables for that.

Something like this in the main Makefile:

ARCH=$(shell git config build.arch)
CC=$(shell git config --default=gcc build.cc)

would actually be optimal for what _I_ do. Then for my clang build tree, I'd do

git config build.cc clang

and it would stick in that tree. If you don't set the option, it would use gcc.

Maybe that would be an acceptable model these days? We've used git
long enough that non-git worries aren't a big deal any more, and I
guess people could use the old-fashioned

make CC=clang

if they don't use git.

We could get rid of all the CROSS_COMPILE etc garbage that way,
because you'd just set it in the .git/config file. Those are really
convenient and easy to edit.

Would people hate that kind of model?

Linus

[Linus Torvalds]

On Tue, Jul 20, 2021 at 2:54 PM Linus Torvalds
<torv...@linux-foundation.org> wrote:
>
> But there are other things that stick around. In particular, I have
> considered simply using git config variables for that.
>
> Something like this in the main Makefile:

A slightly more fleshed-out patch to do this just to see if somebody
wants to work on something like this.

It really would seem to make it very easy to set up any random build
environment, and not have to remember it once it's done.

There are probably better ways to do this, I normally try to avoid
doing Makefile magic, since others have taken it over. Masahiro?

Linus

[Linus Torvalds]

On Tue, Jul 20, 2021 at 4:19 PM Linus Torvalds
<torv...@linux-foundation.org> wrote:
>
> There are probably better ways to do this

Oh. In particular, it probably should use a '?=' in order to allow
overriding, and not expanding that $(shell git config..) thing every
time $(CC) is used.

Or something. I didn't really think about those things very much.
Think of that patch - as so many of my patches - as a starting point
and a "how about something like this".

Linus

[Masahiro Yamada]

I disagree.

LLVM=1 is a shorthand.



make LLVM=1 LLVM_IAS=1

should be equivalent to:

make CC=clang LD=ld.lld AR=llvm-ar NM=llvm-nm STRIP=llvm-strip \
OBJCOPY=llvm-objcopy OBJDUMP=llvm-objdump READELF=llvm-readelf \
HOSTCC=clang HOSTCXX=clang++ HOSTAR=llvm-ar HOSTLD=ld.lld \
LLVM_IAS=1



We do not care about the origin of CC=clang,
whether it came from LLVM=1 or every tool was explicitly,
individually specified.



ifneq ($(LLVM),) is a garbage code
that checks a pointless thing.

[Masahiro Yamada]

There is no need to add a hook to include such makefile fragment(s).

Quite opposite, you can put your Makefile (in a different filename)
that includes the top Makefile.


I think this is what people are already doing:


GNU Make looks for 'GNUmakefile', 'makefile', and 'Makefile'
in this order.


So, you can put 'GNUmakefile' with your favorite setups.


$ cat GNUmakefile
ARCH=arm64
CROSS_COMPILE=aarch64-linux-gnu-
CC=clang
include Makefile



Then, you can simply do "make defconfig; make"
for building arm64 with clang.



If you want to have multiple setups, you can put
custom Makefiles, and select by the -f option.


For example,



$ cat Makefile-arm64
ARCH=arm64
CROSS_COMPILE=aarch64-linux-gnu-
include Makefile


$ cat Makefile-arm64-llvm
ARCH=arm64
CROSS_COMPILE=aarch64-linux-gnu-
LLVM=1
include Makefile


$ cat Makefile-arm
ARCH=arm
CROSS_COMPILE=arm-linux-gnueabihf-
include Makefile


Then,

$ make -f Makefile-arm64 defconfig all -j24

$ make -f Makefile-arm64-llvm defconfig all -j24

$ make -f Makefile-arm defconfig all -j24


You can use TAB-completion for the 'Makefile-*' part,
so you can save many typings than you do
make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- ...



It is just a tip to ease your life without any change to upstream.
Maybe, some people might be doing that.

>
> This is probably the easiest part here, and it lets you set up multiple
> object directories in which you can then do
>
> make O=obj-x86 defconfig vmlinux modules -skj32
> make O=obj-x86-clang
> make O=obj-arm64 randconfig
> ...
>
> without ever having to type the additional CC/CROSS_COMPILE/LLVM
> variables again.
>
> One step further is the script that automatically finds a working toolchain
> for a given architecture and sets up that object directory accordingly,
> or even downloads a working cross-toolchain from kernel.org when
> asked to do that.

I saw Intel's 0-day bot attached a shell script to download and
set up cross compilers.
(make.cross)

[Masahiro Yamada]

On Wed, Jul 21, 2021 at 2:43 AM Linus Torvalds
<torv...@linux-foundation.org> wrote:
>
> On Tue, Jul 20, 2021 at 1:05 AM Masahiro Yamada <masa...@kernel.org> wrote:
> >
> > LLVM=1 is a convenient switch to change all the
> > defaults, but yet you can flip each tool individually.
>
> Actually, I'd argue that "LLVM=1" is *not* a convenient switch.
> Neither are the individual other command line settings.
>
> When clang was the odd man out, and special, it all made sense.
> Changing the path to CC was similar to changing the path to AWK. And
> that's obviously why we have what we have.
>
> But clang has become a primary compiler for some kernel communities,
> and I think it might be time to just re-visit that entirely.
>
> In particular, I think we should just make it a Kconfig option. I hate
> the command flag stuff so much, that my clang tree literally has this
> patch in it:
>
> -CC = $(CROSS_COMPILE)gcc
> +CC = $(CROSS_COMPILE)clang
>
> so that I can just do the same "make -j128" in both my gcc tree and my
> clang tree.
>
> But each build tree already has its own .config file, so it would be a
> lot more convenient if that was how the compiler was chosen, and then
> "make oldconfig" would just DTRT.
>
> We do most of the other heavy lifting in this area in Kconfig anyway,
> why not add that compiler choice?

This does not work in the current design.

When we moved compiler tests to Kconfig from Makefile at v4.18,
I discussed a lot about the implementation.
Especially 'static' vs 'dynamic'.
(mostly Ulf Magnusson actively gave me feed backs)


static model
Kconfig is given with a fixed set of toolchains.
All macros such as cc-option are expanded in the early parse stage
before proceeding to the configuration stage.


dynamic model
Every time a user toggles a CONFIG option, it feeds back to
cc-option because CONFIG options (such as CONFIG_64BIT)
might affect cc-option. In this model, CONFIG_CROSS_COMPILE
works well.




Of course, the dynamic model is much more complex,
and I could not find a sane syntax to achieve it.

So, I implemented the static model; in this model,
$(cc-option ) are all constant values.

So, you cannot do:


choice
prompt "Compiler suite"

config GCC
"GCC and GNU binutils"

config LLVM
"LLVM (clang and llvm tools)"

endchoice



To achieve this, we need the dynamic model.

Every time a user switches between CONFIG_GCC and CONFIG_LLVM,
all the cc-options must be re-evaluated.
This does not work in the current implementation.






BTW, keeping the command line interface
such as CROSS_COMPILE=... CC=... LD=...
is useful when the kernel is built from umbrella
projects such as Buildroot, OpenEmbedded, etc.

Such projects build compilers by themselves,
and feed the exact compiler paths to the kernel.

Too long command line is an annoyance when
we build the kernel manually.

>
> Obviously it would be gated by the tests to see which compilers are
> _installed_ (and that they are valid versions), so that it doesn't ask
> stupid things ("do you want gcc or clang" when only one of them is
> installed and/or viable).
>
> Hmm? So then any "LLVM=1" thing would be about the "make config"
> stage, not the actual build stage.
>
> (It has annoyed me for years that if you want to cross-compile, you
> first have to do "make ARCH=xyz config" and then remember to do "make
> ARCH=xyz" for the build too, but I cross-compile so seldom that I've
> never really cared).
>
> Let the flame wars^H^Hpolite discussions ensue..
>
> Linus

[Christoph Hellwig]

On Tue, Jul 20, 2021 at 10:42:57AM -0700, Linus Torvalds wrote:
> (It has annoyed me for years that if you want to cross-compile, you
> first have to do "make ARCH=xyz config" and then remember to do "make
> ARCH=xyz" for the build too, but I cross-compile so seldom that I've
> never really cared).

Same here. I'd so much love to just have ARCH an CROSS_COMPILE in my
.config for cross-builds. Oterwise I keep forgetting them, or keep
forgetting to unset CROSS_COMPILE and will just waist a lot of time
either way.

The fact the the .config files even mention what architecture they are
for in a comment but don't make use of that information is just hostile
to users.

[Christoph Hellwig]

On Tue, Jul 20, 2021 at 02:54:35PM -0700, Linus Torvalds wrote:
> But there are other things that stick around. In particular, I have
> considered simply using git config variables for that.
>
> Something like this in the main Makefile:
>
> ARCH=$(shell git config build.arch)
> CC=$(shell git config --default=gcc build.cc)
>
> would actually be optimal for what _I_ do. Then for my clang build tree, I'd do
>
> git config build.cc clang
>
> and it would stick in that tree. If you don't set the option, it would use gcc.
>
> Maybe that would be an acceptable model these days? We've used git
> long enough that non-git worries aren't a big deal any more, and I
> guess people could use the old-fashioned

I still this only papers around the fact that the architecture is
fundamentally part of the kernel config and should come from .config,
and the toolchain probably as well.

I for one do have a few different tree for different projects, but
need to try cross compiles in each of them. And unfortunately sometimes
with clang (nothing against clang itself [1], just that having to deal
with multiple toolchains is a pain)

[1] well, except for the sometimes idiotic diagnostics..

[Masahiro Yamada]

I also like to make the integrated assembler our default.

We can add LLVM_DISABLE_IAS=1 to
replace LLVM_IAS=1.

> Then that just leaves ARCH.
> Arnd's idea about helping you install a toolchain from kernel.org is
> one I support, but orthogonal to the above somewhat. Do you allow
> someone to have a config that denotes intent to build with clang then
> prompt if they don't have clang installed to download it? Or do you
> prevent someone from selecting building with clang because it's not in
> the $PATH?
> Your/Arnd's idea about detecting which toolchains are installed is one
> I support, but orthogonal to the above somewhat. (For that, I'm
> curious for our build servers if that means having to put tools in
> certain locations; I prefer we reference $PATH when possible. Or if
> .configs can no longer be shared if tools are in different locations.
> But perhaps that's a non-issue). I'm also curious how many stat calls
> we'll need to test/probe/find these, and how we prioritize which tools
> are selected when there's more than one version installed.
>
> I encourage us to make steps in the right direction; but I think this
> series is ready to go for at least one of the command line variables.
> I don't think we need to wait for some probing machinery to eliminate
> CROSS_COMPILE when LLVM=1; and if we ever get such machinery we can
> revisit whether that helps this case at all.
> --
> Thanks,
> ~Nick Desaulniers

[Masahiro Yamada]

The patch would uglify the Makefile.

If git is not installed, $(shell git config ...)
will spit an error message, so '2>/dev/null' is needed.

Actually, I believe we should be able to build the kernel
from a source tarball without relying on git in any way.

In fact, we rely on git in some places, for example,
scripts/setlocalversion can add commit hash to the release
string, but I want to minimize the git-dependent code.








> Linus

>
> --
> You received this message because you are subscribed to the Google Groups "Clang Built Linux" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to clang-built-li...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/CAHk-%3DwhS1zBU%3DuhDTk2M1LwkqG7AKLsR0_%2BXG%2BsaY_s2FHZr-A%40mail.gmail.com.

[Masahiro Yamada]

On Wed, Jul 21, 2021 at 1:53 PM Christoph Hellwig <h...@infradead.org> wrote:
>
> On Tue, Jul 20, 2021 at 02:54:35PM -0700, Linus Torvalds wrote:
> > But there are other things that stick around. In particular, I have
> > considered simply using git config variables for that.
> >
> > Something like this in the main Makefile:
> >
> > ARCH=$(shell git config build.arch)
> > CC=$(shell git config --default=gcc build.cc)
> >
> > would actually be optimal for what _I_ do. Then for my clang build tree, I'd do
> >
> > git config build.cc clang
> >
> > and it would stick in that tree. If you don't set the option, it would use gcc.
> >
> > Maybe that would be an acceptable model these days? We've used git
> > long enough that non-git worries aren't a big deal any more, and I
> > guess people could use the old-fashioned
>
> I still this only papers around the fact that the architecture is
> fundamentally part of the kernel config

In fact, I am the person who did it in U-boot several years ago.

In U-Boot Kconfig, the architecture is selected by a Kconfig choice:

https://github.com/u-boot/u-boot/blob/v2021.07/arch/Kconfig#L22

(but, if you look at Makefiles and Kconfigs of U-Boot,
you will notice they are uglier than the kernel's ones.)



Theoretically, yes, it is possible to make ARCH as part of the .config
in the kernel as well, but we need careful consideration to make
it robust, clean, and efficient.

In the current model, the Kconfig stage would be even slower
because Kconfig would end up with a bunch of unrelated cc-option
from other architectures.

For example, arch/s390/Kconfig has so many cc-option.
If we allow Kconfig to choose "ARCH', all Kconfig files
of all architectures must be parsed.
If you intend to build the x86 kernel, you might think that
parsing the expensive arch/s390/Kconfig is pointless.

> and should come from .config,
> and the toolchain probably as well.
>
> I for one do have a few different tree for different projects, but
> need to try cross compiles in each of them. And unfortunately sometimes
> with clang (nothing against clang itself [1], just that having to deal
> with multiple toolchains is a pain)
>
> [1] well, except for the sometimes idiotic diagnostics..

[Geert Uytterhoeven]

Hi Yamada-san,

Exactly. I only have a few source repositories, but lots of build directories
(I never build in a repo directory). Each build directory has a GNUmakefile:

build/linux-riscv-starlight$ cat GNUmakefile
MAKEARGS = ARCH=riscv CROSS_COMPILE=riscv64-linux-gnu-

include ~/linux/default.mk
build/linux-riscv-starlight$ cat ~/linux/default.mk
MAKEFLAGS += --no-print-directory

.PHONY: all $(MAKECMDGOALS)

all := $(filter-out all Makefile,$(MAKECMDGOALS))

all:
@$(MAKE) $(MAKEARGS) $(all) -f Makefile

Makefile:;

$(all): all
@:

%/: all
@:
build/linux-riscv-starlight$

Perhaps we could have a "make setup ARCH=foo CROSS_COMPILE=bar" target,
which creates such a GNUmakefile?

P.S. I put the extra logic in ~/linux/default.mk, so I don't have to update all
GNUmakefiles when I want to make a change to the main logic.

For build dirs where I want to track a specific config, I have a slightly
different version:

build/linux-m68k-allmodconfig-sun3$ cat GNUmakefile
MAKEARGS = ARCH=m68k KCONFIG_ALLCONFIG=1
DEFCONFIG = allmodconfig

include ~/linux/override-oldconfig.mk
build/linux-m68k-allmodconfig-sun3$ cat ~/linux/override-oldconfig.mk
MAKEFLAGS += --no-print-directory

.PHONY: all oldconfig realoldconfig $(MAKECMDGOALS)

all := $(filter-out all oldconfig realoldconfig
Makefile,$(MAKECMDGOALS))

all:
@$(MAKE) $(MAKEARGS) $(all) -f Makefile

# Replace oldconfig by $(DEFCONFIG)
oldconfig:
@$(MAKE) $(MAKEARGS) $(DEFCONFIG) -f Makefile

realoldconfig:
@$(MAKE) $(MAKEARGS) oldconfig -f Makefile

Makefile:;

$(all): all
@:

%/: all
@:
build/linux-m68k-allmodconfig-sun3$

That way I can always just type "make oldconfig", and it will do what
I want.

Gr{oetje,eeting}s,

Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- ge...@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds

[Masahiro Yamada]

No, I do not think so.

Your GNUmakefile is customized in your way.
Other people implement GNUmakefile in their own way.

Upstream is not a place to merge everybody's local tools.

> --
> You received this message because you are subscribed to the Google Groups "Clang Built Linux" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to clang-built-li...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/CAMuHMdXJBqrpzaSNDJgic14ESiHV6cCcb_5E-st6iniXdmm9_g%40mail.gmail.com.

[Eric W. Biederman]

Very weird.

I just tested this and it does not work.
I did this:

$ cat GNUmakefile
ARCH = alpha
CROSS_COMPILE = $(arch-prefix alpha)
include Makefile

In one of my build directories and the main makefile simply does not see
the value of ARCH or CROSS_COMPILE I set. I have confirmed that my
GNUmakefile is being read, because everything breaks if I remove the
include line.

Does anyone have any ideas?

Something so we don't have to specify all of these variables on the make
command line would be nice.

Eric

[Geert Uytterhoeven]

Hi Eric,

Just including the main Makefile does not work.
That's why I went with the more convoluted solution in
https://lore.kernel.org/linux-kbuild/CAMuHMdXJBqrpzaSNDJgic14E...@mail.gmail.com/

Please try the attached, which combines everything above in a single
file, and which works for me. Note that "$(arch-prefix alpha)" didn't
work for me (it resolved to "gcc"?), so I used "alpha-linux-gnu-"
instead.

Good luck!

[Arnd Bergmann]

On Tue, Jul 27, 2021 at 9:07 AM Geert Uytterhoeven <ge...@linux-m68k.org> wrote:
> On Mon, Jul 26, 2021 at 10:27 PM Eric W. Biederman <ebie...@xmission.com> wrote:
> > Masahiro Yamada <masa...@kernel.org> writes:
> > > On Wed, Jul 21, 2021 at 4:58 AM Arnd Bergmann <ar...@kernel.org> wrote:
> >
> > Does anyone have any ideas?
> >
> > Something so we don't have to specify all of these variables on the make
> > command line would be nice.
>
> Just including the main Makefile does not work.
> That's why I went with the more convoluted solution in
> https://lore.kernel.org/linux-kbuild/CAMuHMdXJBqrpzaSNDJgic14E...@mail.gmail.com/

This is roughly what I use as well, but it does have the downside that
it confuses
the inner 'make' when you build multiple targets in parallel, e.g.

make -skj30 kernel/ mm/ init/

works with the normal Makefile, but fails spectacularly with my nested
GNUmakefile because it starts multiple sub-processes that each try to
build the same preparation files (I did not try your version).

What I had in mind was to use something like (but not exactly)

-include $(O)/Makefile.cross

to conditionally include another file from the top-level Makefile before
doing anything else.

Arnd

[Geert Uytterhoeven]

Hi Arnd,

On Tue, Jul 27, 2021 at 9:49 AM Arnd Bergmann <ar...@kernel.org> wrote:
> On Tue, Jul 27, 2021 at 9:07 AM Geert Uytterhoeven <ge...@linux-m68k.org> wrote:
> > On Mon, Jul 26, 2021 at 10:27 PM Eric W. Biederman <ebie...@xmission.com> wrote:
> > > Masahiro Yamada <masa...@kernel.org> writes:
> > > > On Wed, Jul 21, 2021 at 4:58 AM Arnd Bergmann <ar...@kernel.org> wrote:
> > >
> > > Does anyone have any ideas?
> > >
> > > Something so we don't have to specify all of these variables on the make
> > > command line would be nice.
> >
> > Just including the main Makefile does not work.
> > That's why I went with the more convoluted solution in
> > https://lore.kernel.org/linux-kbuild/CAMuHMdXJBqrpzaSNDJgic14E...@mail.gmail.com/
>
> This is roughly what I use as well, but it does have the downside that
> it confuses
> the inner 'make' when you build multiple targets in parallel, e.g.
>
> make -skj30 kernel/ mm/ init/
>
> works with the normal Makefile, but fails spectacularly with my nested
> GNUmakefile because it starts multiple sub-processes that each try to
> build the same preparation files (I did not try your version).

Seems to work fine with mine...

[Arnd Bergmann]

On Tue, Jul 27, 2021 at 9:55 AM Geert Uytterhoeven <ge...@linux-m68k.org> wrote:
> On Tue, Jul 27, 2021 at 9:49 AM Arnd Bergmann <ar...@kernel.org> wrote:
> > make -skj30 kernel/ mm/ init/
> >
> > works with the normal Makefile, but fails spectacularly with my nested
> > GNUmakefile because it starts multiple sub-processes that each try to
> > build the same preparation files (I did not try your version).
>
> Seems to work fine with mine...
>

Ok, nevermind. I tried your version and it works for me too. The next
time I get annoyed by this, I'll figure out what I did wrong then;-)

Arnd

[Masahiro Yamada]

Worked for me.

Could you tell me the exact steps you did?


This is my case:

My kernel source tree is located at $HOME/ref/linux
alpha tool chains are located at $HOME/tools/alpha-10.1.0/bin



I tried a simple GNUmakefile with 3 lines.

You can see 'make' is building the alpha kernel


Please see below:




masahiro@grover:~/ref/linux$ cat GNUmakefile
ARCH = alpha
CROSS_COMPILE = $(HOME)/tools/alpha-10.1.0/bin/alpha-linux-
include Makefile
masahiro@grover:~/ref/linux$ make mrproper
CLEAN arch/alpha/kernel
CLEAN certs
CLEAN drivers/tty/vt
CLEAN drivers/video/logo
CLEAN kernel
CLEAN lib
CLEAN security/selinux
CLEAN usr/include
CLEAN usr
CLEAN vmlinux.symvers modules-only.symvers modules.builtin
modules.builtin.modinfo
CLEAN scripts/basic
CLEAN scripts/dtc
CLEAN scripts/genksyms
CLEAN scripts/kconfig
CLEAN scripts/mod
CLEAN scripts/selinux/genheaders
CLEAN scripts/selinux/mdp
CLEAN scripts
CLEAN include/config include/generated
arch/alpha/include/generated .config .config.old .version
Module.symvers
masahiro@grover:~/ref/linux$ make defconfig
HOSTCC scripts/basic/fixdep
HOSTCC scripts/kconfig/conf.o
HOSTCC scripts/kconfig/confdata.o
HOSTCC scripts/kconfig/expr.o
LEX scripts/kconfig/lexer.lex.c
YACC scripts/kconfig/parser.tab.[ch]
HOSTCC scripts/kconfig/lexer.lex.o
HOSTCC scripts/kconfig/menu.o
HOSTCC scripts/kconfig/parser.tab.o
HOSTCC scripts/kconfig/preprocess.o
HOSTCC scripts/kconfig/symbol.o
HOSTCC scripts/kconfig/util.o
HOSTLD scripts/kconfig/conf
*** Default configuration is based on 'defconfig'
#
# configuration written to .config
#
masahiro@grover:~/ref/linux$ make
SYSHDR arch/alpha/include/generated/uapi/asm/unistd_32.h
SYSTBL arch/alpha/include/generated/asm/syscall_table.h
HOSTCC scripts/kallsyms
WRAP arch/alpha/include/generated/uapi/asm/bpf_perf_event.h
WRAP arch/alpha/include/generated/uapi/asm/ipcbuf.h
WRAP arch/alpha/include/generated/uapi/asm/msgbuf.h
WRAP arch/alpha/include/generated/uapi/asm/poll.h
WRAP arch/alpha/include/generated/uapi/asm/sembuf.h
WRAP arch/alpha/include/generated/uapi/asm/shmbuf.h
WRAP arch/alpha/include/generated/asm/export.h
WRAP arch/alpha/include/generated/asm/kvm_para.h
WRAP arch/alpha/include/generated/asm/mcs_spinlock.h
WRAP arch/alpha/include/generated/asm/compat.h
WRAP arch/alpha/include/generated/asm/current.h
WRAP arch/alpha/include/generated/asm/exec.h
WRAP arch/alpha/include/generated/asm/fb.h
WRAP arch/alpha/include/generated/asm/irq_work.h
WRAP arch/alpha/include/generated/asm/kmap_size.h
WRAP arch/alpha/include/generated/asm/kprobes.h
WRAP arch/alpha/include/generated/asm/local64.h
WRAP arch/alpha/include/generated/asm/mmiowb.h
WRAP arch/alpha/include/generated/asm/module.lds.h
WRAP arch/alpha/include/generated/asm/msi.h
WRAP arch/alpha/include/generated/asm/preempt.h
WRAP arch/alpha/include/generated/asm/sections.h
WRAP arch/alpha/include/generated/asm/simd.h
WRAP arch/alpha/include/generated/asm/softirq_stack.h
WRAP arch/alpha/include/generated/asm/trace_clock.h
WRAP arch/alpha/include/generated/asm/unaligned.h
WRAP arch/alpha/include/generated/asm/vermagic.h
UPD include/config/kernel.release
UPD include/generated/uapi/linux/version.h
UPD include/generated/utsrelease.h
CC scripts/mod/empty.o
HOSTCC scripts/mod/mk_elfconfig
MKELF scripts/mod/elfconfig.h
HOSTCC scripts/mod/modpost.o
CC scripts/mod/devicetable-offsets.s
UPD scripts/mod/devicetable-offsets.h
HOSTCC scripts/mod/file2alias.o
HOSTCC scripts/mod/sumversion.o
HOSTLD scripts/mod/modpost
CC kernel/bounds.s
UPD include/generated/bounds.h
UPD include/generated/timeconst.h
CC arch/alpha/kernel/asm-offsets.s
UPD include/generated/asm-offsets.h
CALL scripts/checksyscalls.sh
<stdin>:1515:2: warning: #warning syscall clone3 not implemented [-Wcpp]
CALL scripts/atomic/check-atomics.sh
CC init/main.o
CHK include/generated/compile.h
UPD include/generated/compile.h
CC init/version.o
CC init/do_mounts.o
CC init/noinitramfs.o
CC init/calibrate.o
CC init/init_task.o
AR init/built-in.a
AR usr/built-in.a
AS arch/alpha/kernel/head.o
LDS arch/alpha/kernel/vmlinux.lds
AS arch/alpha/kernel/entry.o
CC arch/alpha/kernel/traps.o
CC arch/alpha/kernel/process.o
CC arch/alpha/kernel/osf_sys.o
CC arch/alpha/kernel/irq.o
CC arch/alpha/kernel/irq_alpha.o
CC arch/alpha/kernel/signal.o
CC arch/alpha/kernel/setup.o
CC arch/alpha/kernel/ptrace.o
CC arch/alpha/kernel/time.o
AS arch/alpha/kernel/systbls.o
CC arch/alpha/kernel/err_common.o
CC arch/alpha/kernel/io.o
CC arch/alpha/kernel/bugs.o
CC arch/alpha/kernel/console.o
CC arch/alpha/kernel/pci.o
CC arch/alpha/kernel/pci_iommu.o
CC arch/alpha/kernel/pci-sysfs.o
CC arch/alpha/kernel/module.o
CC arch/alpha/kernel/rtc.o
CC arch/alpha/kernel/core_apecs.o
CC arch/alpha/kernel/core_cia.o
CC arch/alpha/kernel/core_irongate.o
CC arch/alpha/kernel/core_lca.o
CC arch/alpha/kernel/core_mcpcia.o
CC arch/alpha/kernel/core_polaris.o
CC arch/alpha/kernel/core_t2.o
CC arch/alpha/kernel/core_tsunami.o
CC arch/alpha/kernel/sys_alcor.o
CC arch/alpha/kernel/sys_cabriolet.o
CC arch/alpha/kernel/sys_dp264.o
CC arch/alpha/kernel/sys_eb64p.o
CC arch/alpha/kernel/sys_eiger.o
CC arch/alpha/kernel/sys_jensen.o
CC arch/alpha/kernel/sys_miata.o
CC arch/alpha/kernel/sys_mikasa.o
CC arch/alpha/kernel/sys_nautilus.o
CC arch/alpha/kernel/sys_noritake.o
CC arch/alpha/kernel/sys_rawhide.o
CC arch/alpha/kernel/sys_ruffian.o
CC arch/alpha/kernel/sys_rx164.o
CC arch/alpha/kernel/sys_sable.o
CC arch/alpha/kernel/sys_sio.o
CC arch/alpha/kernel/sys_sx164.o
CC arch/alpha/kernel/sys_takara.o
CC arch/alpha/kernel/irq_pyxis.o
CC arch/alpha/kernel/irq_i8259.o
CC arch/alpha/kernel/irq_srm.o
CC arch/alpha/kernel/err_ev6.o
CC arch/alpha/kernel/es1888.o
CC arch/alpha/kernel/smc37c669.o
CC arch/alpha/kernel/smc37c93x.o
CC arch/alpha/kernel/pc873xx.o
CC arch/alpha/kernel/gct.o
CC arch/alpha/kernel/srmcons.o
AR arch/alpha/kernel/built-in.a
CC [M] arch/alpha/kernel/srm_env.o
CC arch/alpha/mm/init.o
CC arch/alpha/mm/fault.o
AR arch/alpha/mm/built-in.a
CC arch/alpha/math-emu/math.o
^Cmake[2]: *** Deleting file 'arch/alpha/math-emu/math.o'
make[2]: *** [scripts/Makefile.build:271: arch/alpha/math-emu/math.o] Interrupt
make[1]: *** [scripts/Makefile.build:514: arch/alpha/math-emu] Interrupt
make: *** [Makefile:1842: arch/alpha] Interrupt

[Eric W. Biederman]

Interesting. That appears to work if I don't specify a build directory.
Once I specify a build directory with O= it does not work.

When I am working on a change that affects multiple architectures
I really want a build directory that is not my source tree so I can
test small changes on multiple architectures without needing to rebuild
everything.

Eric

[Masahiro Yamada]

On Tue, Jul 27, 2021 at 11:16 PM Eric W. Biederman

Ah, OK.
When you specify O= option, Kbuild will
invoke sub-make.
So you need to add 'export' to the
variables.

Please see the following modified version.





masahiro@grover:~/ref/linux$ rm -rf build-dir/
masahiro@grover:~/ref/linux$ cat GNUmakefile
export ARCH = alpha
export CROSS_COMPILE = $(HOME)/tools/alpha-10.1.0/bin/alpha-linux-
include Makefile
masahiro@grover:~/ref/linux$ make O=build-dir defconfig
make[1]: Entering directory '/home/masahiro/ref/linux/build-dir'
GEN Makefile

HOSTCC scripts/basic/fixdep
HOSTCC scripts/kconfig/conf.o
HOSTCC scripts/kconfig/confdata.o
HOSTCC scripts/kconfig/expr.o
LEX scripts/kconfig/lexer.lex.c
YACC scripts/kconfig/parser.tab.[ch]
HOSTCC scripts/kconfig/lexer.lex.o
HOSTCC scripts/kconfig/menu.o
HOSTCC scripts/kconfig/parser.tab.o
HOSTCC scripts/kconfig/preprocess.o
HOSTCC scripts/kconfig/symbol.o
HOSTCC scripts/kconfig/util.o
HOSTLD scripts/kconfig/conf
*** Default configuration is based on 'defconfig'
#
# configuration written to .config
#

make[1]: Leaving directory '/home/masahiro/ref/linux/build-dir'
masahiro@grover:~/ref/linux$ make O=build-dir
make[1]: Entering directory '/home/masahiro/ref/linux/build-dir'
GEN Makefile

CALL ../scripts/checksyscalls.sh

<stdin>:1515:2: warning: #warning syscall clone3 not implemented [-Wcpp]

CALL ../scripts/atomic/check-atomics.sh

AS arch/alpha/math-emu/qrnnd.o
AR arch/alpha/math-emu/built-in.a
AR arch/alpha/built-in.a
CC kernel/sched/core.o
CC kernel/sched/loadavg.o
CC kernel/sched/clock.o
CC kernel/sched/cputime.o
CC kernel/sched/idle.o
CC kernel/sched/fair.o
CC kernel/sched/rt.o
CC kernel/sched/deadline.o
CC kernel/sched/wait.o
CC kernel/sched/wait_bit.o
CC kernel/sched/swait.o
CC kernel/sched/completion.o
CC kernel/sched/debug.o
CC kernel/sched/membarrier.o
AR kernel/sched/built-in.a
CC kernel/locking/mutex.o
CC kernel/locking/semaphore.o
CC kernel/locking/rwsem.o
CC kernel/locking/percpu-rwsem.o
CC kernel/locking/rtmutex.o
AR kernel/locking/built-in.a
CC kernel/power/qos.o
CC kernel/power/poweroff.o
AR kernel/power/built-in.a
CC kernel/printk/printk.o
CC kernel/printk/printk_safe.o
CC kernel/printk/printk_ringbuffer.o
AR kernel/printk/built-in.a
CC kernel/irq/irqdesc.o
CC kernel/irq/handle.o
CC kernel/irq/manage.o
CC kernel/irq/spurious.o
CC kernel/irq/resend.o
CC kernel/irq/chip.o
CC kernel/irq/dummychip.o
CC kernel/irq/devres.o
CC kernel/irq/autoprobe.o
CC kernel/irq/proc.o
AR kernel/irq/built-in.a
CC kernel/rcu/update.o
CC kernel/rcu/sync.o
CC kernel/rcu/srcutiny.o
CC kernel/rcu/tiny.o
AR kernel/rcu/built-in.a
AR kernel/livepatch/built-in.a
CC kernel/dma/mapping.o
CC kernel/dma/direct.o
CC kernel/dma/ops_helpers.o
CC kernel/dma/dummy.o
AR kernel/dma/built-in.a
AR kernel/entry/built-in.a
CC kernel/time/time.o
CC kernel/time/timer.o
CC kernel/time/hrtimer.o
CC kernel/time/timekeeping.o
CC kernel/time/ntp.o
CC kernel/time/clocksource.o
CC kernel/time/jiffies.o
CC kernel/time/timer_list.o
CC kernel/time/timeconv.o
CC kernel/time/timecounter.o
CC kernel/time/alarmtimer.o
CC kernel/time/posix-timers.o
CC kernel/time/posix-cpu-timers.o
CC kernel/time/posix-clock.o
CC kernel/time/itimer.o
CC kernel/time/clockevents.o
CC kernel/time/tick-common.o
AR kernel/time/built-in.a
CC kernel/bpf/core.o
AR kernel/bpf/built-in.a

[Eric W. Biederman]

Yes. That does seem to work and be simpler.

Is there any chance we can get the command that generates the Makefile
in the build directory to include variables like ARCH and CROSS_COMPILE
in the Makefile as well? Just so they don't need to be set by hand to
the same value every time?

Eric

[Nick Desaulniers]

Masahiro,
Nathan is correct. Test for yourself; if you apply these two patches,
then apply:

diff --git a/scripts/Makefile.clang b/scripts/Makefile.clang
index 956603f56724..a1b46811bdc6 100644
--- a/scripts/Makefile.clang
+++ b/scripts/Makefile.clang
@@ -2,7 +2,6 @@

# and -m32/-m64 to set word size based on Kconfigs instead of relying on the

# target triple.
ifeq ($(CROSS_COMPILE),)
-ifneq ($(LLVM),)
ifeq ($(LLVM_IAS),1)
ifeq ($(SRCARCH),arm)
CLANG_FLAGS += --target=arm-linux-gnueabi
@@ -26,7 +25,6 @@ else

$(error Specify CROSS_COMPILE or add '--target=' option to
scripts/Makefile.clang)

endif # SRCARCH
endif # LLVM_IAS
-endif # LLVM

else
CLANG_FLAGS += --target=$(notdir $(CROSS_COMPILE:%-=%))

endif # CROSS_COMPILE

Then build as Nathan specified:
$ ARCH=arm64 make CC=clang LLVM_IAS=1 -j72 defconfig all
...
arch/arm64/Makefile:25: ld does not support --fix-cortex-a53-843419;
kernel may be susceptible to erratum
...
LD arch/arm64/kernel/vdso/vdso.so.dbg
ld: unrecognised emulation mode: aarch64linux
Supported emulations: elf_x86_64 elf32_x86_64 elf_i386 elf_iamcu
elf_l1om elf_k1om i386pep i386pe
make[1]: *** [arch/arm64/kernel/vdso/Makefile:56:
arch/arm64/kernel/vdso/vdso.so.dbg] Error 1
make: *** [arch/arm64/Makefile:193: vdso_prepare] Error 2

Nathan referred to --prefix, but in this failure, because
CROSS_COMPILE was never set, the top level Makefile set LD to:
452 LD = $(CROSS_COMPILE)ld
in this case `ld` in my path was my host x86 linker, which is not
correct for a cross compilation of arm64 target.

Perhaps we can somehow support "implicit CROSS_COMPILE" with just
CC=clang, and not LLVM=1, but I think it would be inflexible to
hardcode such target triple prefixes. What if someone has
arm-linux-gnueabi-as but not arm-linux-gnueabihf-as installed? That's
the point of CROSS_COMPILE in my opinion to provide such flexibility
at the cost of additional command line verbosity.

For the common case of LLVM=1 though, this series is a simplification.
If users want to specify CC=clang, then they MUST use CROSS_COMPILE
when cross compiling.

Please review the current approach and see if there's more I can
improve in a v3; otherwise I still think this series is good to go.
--
Thanks,
~Nick Desaulniers

[Masahiro Yamada]

I am OK to document this small tip
in Documentation/kbuild/ if it is useful,
but am reluctant to add any code to
generate custom Makefile or GNUmakefile.

It is highly user-dependent after all.

If we merged the code to set up ARCH=...
and CROSS_COMPILE=...,
somebody would send a patch
"I need LLVM=1 in my GNUmakefile", then
another person, "I want CC=clang", and so on.

You can edit it by yourself.

[Masahiro Yamada]

On Thu, Jul 29, 2021 at 4:00 AM 'Nick Desaulniers' via Clang Built

Yes, LD is set to GNU ld, which is only for x86.

$ ARCH=arm64 make CC=clang LLVM_IAS=1 LD=ld.lld \
OBJCOPY=llvm-objcopy STRIP=llvm-strip \
-j72 defconfig all

worked for me.



It is true that the most common use-case would be
LLVM=1 LLVM_IAS=1, but as I said, there is no good
reason to prevent this feature from working when
CC, LD, etc. are specified individually.

> --
> You received this message because you are subscribed to the Google Groups "Clang Built Linux" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to clang-built-li...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/CAKwvOdkENUWd7OgJO%3DdNiYjH6D1aJ0puBgs4W7uuYO9xQiAiNg%40mail.gmail.com.