[PATCH bpf-next v3] samples/bpf: Update build procedure for manually compiling LLVM and Clang

131 views
Skip to first unread message

Tiezhu Yang

unread,
Jan 21, 2021, 12:28:00 AM1/21/21
to Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko, Martin KaFai Lau, Song Liu, Yonghong Song, John Fastabend, KP Singh, Nathan Chancellor, Nick Desaulniers, net...@vger.kernel.org, b...@vger.kernel.org, clang-bu...@googlegroups.com, linux-...@vger.kernel.org, Xuefeng Li, Fangrui Song
The current LLVM and Clang build procedure in samples/bpf/README.rst is
out of date. See below that the links are not accessible any more.

$ git clone http://llvm.org/git/llvm.git
Cloning into 'llvm'...
fatal: unable to access 'http://llvm.org/git/llvm.git/': Maximum (20) redirects followed
$ git clone --depth 1 http://llvm.org/git/clang.git
Cloning into 'clang'...
fatal: unable to access 'http://llvm.org/git/clang.git/': Maximum (20) redirects followed

The LLVM community has adopted new ways to build the compiler. There are
different ways to build LLVM and Clang, the Clang Getting Started page [1]
has one way. As Yonghong said, it is better to copy the build procedure
in Documentation/bpf/bpf_devel_QA.rst to keep consistent.

I verified the procedure and it is proved to be feasible, so we should
update README.rst to reflect the reality. At the same time, update the
related comment in Makefile.

Additionally, as Fangrui said, the dir llvm-project/llvm/build/install is
not used, BUILD_SHARED_LIBS=OFF is the default option [2], so also change
Documentation/bpf/bpf_devel_QA.rst together.

[1] https://clang.llvm.org/get_started.html
[2] https://www.llvm.org/docs/CMake.html

Signed-off-by: Tiezhu Yang <yangt...@loongson.cn>
Acked-by: Yonghong Song <y...@fb.com>
---

v2: Update the commit message suggested by Yonghong,
thank you very much.

v3: Remove the default option BUILD_SHARED_LIBS=OFF
and just mkdir llvm-project/llvm/build suggested
by Fangrui.

Documentation/bpf/bpf_devel_QA.rst | 3 +--
samples/bpf/Makefile | 2 +-
samples/bpf/README.rst | 16 +++++++++-------
3 files changed, 11 insertions(+), 10 deletions(-)

diff --git a/Documentation/bpf/bpf_devel_QA.rst b/Documentation/bpf/bpf_devel_QA.rst
index 5b613d2..18788bb 100644
--- a/Documentation/bpf/bpf_devel_QA.rst
+++ b/Documentation/bpf/bpf_devel_QA.rst
@@ -506,11 +506,10 @@ that set up, proceed with building the latest LLVM and clang version
from the git repositories::

$ git clone https://github.com/llvm/llvm-project.git
- $ mkdir -p llvm-project/llvm/build/install
+ $ mkdir -p llvm-project/llvm/build
$ cd llvm-project/llvm/build
$ cmake .. -G "Ninja" -DLLVM_TARGETS_TO_BUILD="BPF;X86" \
-DLLVM_ENABLE_PROJECTS="clang" \
- -DBUILD_SHARED_LIBS=OFF \
-DCMAKE_BUILD_TYPE=Release \
-DLLVM_BUILD_RUNTIME=OFF
$ ninja
diff --git a/samples/bpf/Makefile b/samples/bpf/Makefile
index 26fc96c..d061446 100644
--- a/samples/bpf/Makefile
+++ b/samples/bpf/Makefile
@@ -208,7 +208,7 @@ TPROGLDLIBS_xdpsock += -pthread -lcap
TPROGLDLIBS_xsk_fwd += -pthread

# Allows pointing LLC/CLANG to a LLVM backend with bpf support, redefine on cmdline:
-# make M=samples/bpf/ LLC=~/git/llvm/build/bin/llc CLANG=~/git/llvm/build/bin/clang
+# make M=samples/bpf LLC=~/git/llvm-project/llvm/build/bin/llc CLANG=~/git/llvm-project/llvm/build/bin/clang
LLC ?= llc
CLANG ?= clang
OPT ?= opt
diff --git a/samples/bpf/README.rst b/samples/bpf/README.rst
index dd34b2d..23006cb 100644
--- a/samples/bpf/README.rst
+++ b/samples/bpf/README.rst
@@ -65,17 +65,19 @@ To generate a smaller llc binary one can use::
Quick sniplet for manually compiling LLVM and clang
(build dependencies are cmake and gcc-c++)::

- $ git clone http://llvm.org/git/llvm.git
- $ cd llvm/tools
- $ git clone --depth 1 http://llvm.org/git/clang.git
- $ cd ..; mkdir build; cd build
- $ cmake .. -DLLVM_TARGETS_TO_BUILD="BPF;X86"
- $ make -j $(getconf _NPROCESSORS_ONLN)
+ $ git clone https://github.com/llvm/llvm-project.git
+ $ mkdir -p llvm-project/llvm/build
+ $ cd llvm-project/llvm/build
+ $ cmake .. -G "Ninja" -DLLVM_TARGETS_TO_BUILD="BPF;X86" \
+ -DLLVM_ENABLE_PROJECTS="clang" \
+ -DCMAKE_BUILD_TYPE=Release \
+ -DLLVM_BUILD_RUNTIME=OFF
+ $ ninja

It is also possible to point make to the newly compiled 'llc' or
'clang' command via redefining LLC or CLANG on the make command line::

- make M=samples/bpf LLC=~/git/llvm/build/bin/llc CLANG=~/git/llvm/build/bin/clang
+ make M=samples/bpf LLC=~/git/llvm-project/llvm/build/bin/llc CLANG=~/git/llvm-project/llvm/build/bin/clang

Cross compiling samples
-----------------------
--
2.1.0

Nathan Chancellor

unread,
Jan 21, 2021, 12:36:31 AM1/21/21
to Tiezhu Yang, Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko, Martin KaFai Lau, Song Liu, Yonghong Song, John Fastabend, KP Singh, Nick Desaulniers, net...@vger.kernel.org, b...@vger.kernel.org, clang-bu...@googlegroups.com, linux-...@vger.kernel.org, Xuefeng Li, Fangrui Song
On Thu, Jan 21, 2021 at 01:27:35PM +0800, Tiezhu Yang wrote:
> The current LLVM and Clang build procedure in samples/bpf/README.rst is
> out of date. See below that the links are not accessible any more.
>
> $ git clone http://llvm.org/git/llvm.git
> Cloning into 'llvm'...
> fatal: unable to access 'http://llvm.org/git/llvm.git/': Maximum (20) redirects followed
> $ git clone --depth 1 http://llvm.org/git/clang.git
> Cloning into 'clang'...
> fatal: unable to access 'http://llvm.org/git/clang.git/': Maximum (20) redirects followed
>
> The LLVM community has adopted new ways to build the compiler. There are
> different ways to build LLVM and Clang, the Clang Getting Started page [1]
> has one way. As Yonghong said, it is better to copy the build procedure
> in Documentation/bpf/bpf_devel_QA.rst to keep consistent.
>
> I verified the procedure and it is proved to be feasible, so we should
> update README.rst to reflect the reality. At the same time, update the
> related comment in Makefile.
>
> Additionally, as Fangrui said, the dir llvm-project/llvm/build/install is
> not used, BUILD_SHARED_LIBS=OFF is the default option [2], so also change
> Documentation/bpf/bpf_devel_QA.rst together.
>
> [1] https://clang.llvm.org/get_started.html
> [2] https://www.llvm.org/docs/CMake.html
>
> Signed-off-by: Tiezhu Yang <yangt...@loongson.cn>
> Acked-by: Yonghong Song <y...@fb.com>

Reviewed-by: Nathan Chancellor <natecha...@gmail.com>

Small comment below.
Technically, ninja is now a build dependency as well, it might be worth
mentioning that here (usually the package is ninja or ninja-build).

Regardless of whether that is addressed or not (because it is small),
feel free to carry forward my tag in any future revisions unless they
drastically change.

Andrii Nakryiko

unread,
Jan 21, 2021, 3:08:43 AM1/21/21
to Nathan Chancellor, Tiezhu Yang, Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko, Martin KaFai Lau, Song Liu, Yonghong Song, John Fastabend, KP Singh, Nick Desaulniers, Networking, bpf, clang-built-linux, open list, Xuefeng Li, Fangrui Song
it's possible to generate Makefile by passing `-g "Unix Makefiles"`,
which would avoid dependency on ninja, no?

Sedat Dilek

unread,
Jan 21, 2021, 3:55:32 AM1/21/21
to Andrii Nakryiko, Nathan Chancellor, Tiezhu Yang, Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko, Martin KaFai Lau, Song Liu, Yonghong Song, John Fastabend, KP Singh, Nick Desaulniers, Networking, bpf, clang-built-linux, open list, Xuefeng Li, Fangrui Song
AFAICS, cmake is now the default and "Unix Makefiles" deprecated with
newer versions of LLVM/Clang.

- Sedat -
> --
> You received this message because you are subscribed to the Google Groups "Clang Built Linux" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to clang-built-li...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/CAEf4BzbZxuy8LRmngRzLZ3VTnrDw%3DRf70Ghkbu1a5%2BfNpQud5Q%40mail.gmail.com.

Sedat Dilek

unread,
Jan 21, 2021, 5:54:39 AM1/21/21
to Andrii Nakryiko, Nathan Chancellor, Tiezhu Yang, Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko, Martin KaFai Lau, Song Liu, Yonghong Song, John Fastabend, KP Singh, Nick Desaulniers, Networking, bpf, clang-built-linux, open list, Xuefeng Li, Fangrui Song
Hmm, I mixed it up...
This is about the cmake-generator (GNU/make was deprecated/abandoned).
Cannot say I use "ninja" - it's fast.

Just as a hint for a fast build of LLVM/Clang:
Use tc-build together with stage1-only (build and install) options.

- Sedat -

[1] https://github.com/ClangBuiltLinux/tc-build

Nathan Chancellor

unread,
Jan 21, 2021, 9:23:07 AM1/21/21
to Andrii Nakryiko, Tiezhu Yang, Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko, Martin KaFai Lau, Song Liu, Yonghong Song, John Fastabend, KP Singh, Nick Desaulniers, Networking, bpf, clang-built-linux, open list, Xuefeng Li, Fangrui Song
Yes, although I am fairly certain that building with ninja is quicker so
I would recommend keeping it. One small extra dependency never killed
anyone plus ninja is becoming more common nowadays :)

Yonghong Song

unread,
Jan 21, 2021, 10:01:31 AM1/21/21
to Nathan Chancellor, Andrii Nakryiko, Tiezhu Yang, Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko, Martin KaFai Lau, Song Liu, John Fastabend, KP Singh, Nick Desaulniers, Networking, bpf, clang-built-linux, open list, Xuefeng Li, Fangrui Song
Agree. Let us use 'ninja' here, which is widely used in llvm community
for llvm-project build compared to other alternatives.

Tiezhu Yang

unread,
Jan 21, 2021, 8:40:07 PM1/21/21
to Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko, Martin KaFai Lau, Song Liu, Yonghong Song, John Fastabend, KP Singh, Nathan Chancellor, Nick Desaulniers, net...@vger.kernel.org, b...@vger.kernel.org, clang-bu...@googlegroups.com, linux-...@vger.kernel.org, Xuefeng Li, Fangrui Song
The current LLVM and Clang build procedure in samples/bpf/README.rst is
out of date. See below that the links are not accessible any more.

$ git clone http://llvm.org/git/llvm.git
Cloning into 'llvm'...
fatal: unable to access 'http://llvm.org/git/llvm.git/': Maximum (20) redirects followed
$ git clone --depth 1 http://llvm.org/git/clang.git
Cloning into 'clang'...
fatal: unable to access 'http://llvm.org/git/clang.git/': Maximum (20) redirects followed

The LLVM community has adopted new ways to build the compiler. There are
different ways to build LLVM and Clang, the Clang Getting Started page [1]
has one way. As Yonghong said, it is better to copy the build procedure
in Documentation/bpf/bpf_devel_QA.rst to keep consistent.

I verified the procedure and it is proved to be feasible, so we should
update README.rst to reflect the reality. At the same time, update the
related comment in Makefile.

Additionally, as Fangrui said, the dir llvm-project/llvm/build/install is
not used, BUILD_SHARED_LIBS=OFF is the default option [2], so also change
Documentation/bpf/bpf_devel_QA.rst together.

At last, we recommend that developers who want the fastest incremental
builds use the Ninja build system [1], you can find it in your system's
package manager, usually the package is ninja or ninja-build [3], so add
ninja to build dependencies suggested by Nathan.

[1] https://clang.llvm.org/get_started.html
[2] https://www.llvm.org/docs/CMake.html
[3] https://github.com/ninja-build/ninja/wiki/Pre-built-Ninja-packages

Signed-off-by: Tiezhu Yang <yangt...@loongson.cn>
Acked-by: Yonghong Song <y...@fb.com>
Reviewed-by: Nathan Chancellor <natecha...@gmail.com>
---

v2: Update the commit message suggested by Yonghong,
thank you very much.

v3: Remove the default option BUILD_SHARED_LIBS=OFF
and just mkdir llvm-project/llvm/build suggested
by Fangrui.

v4: Add some description about ninja suggested by Nathan.

Documentation/bpf/bpf_devel_QA.rst | 11 +++++++----
samples/bpf/Makefile | 2 +-
samples/bpf/README.rst | 22 ++++++++++++++--------
3 files changed, 22 insertions(+), 13 deletions(-)

diff --git a/Documentation/bpf/bpf_devel_QA.rst b/Documentation/bpf/bpf_devel_QA.rst
index 5b613d2..2ed89ab 100644
--- a/Documentation/bpf/bpf_devel_QA.rst
+++ b/Documentation/bpf/bpf_devel_QA.rst
@@ -501,16 +501,19 @@ All LLVM releases can be found at: http://releases.llvm.org/

Q: Got it, so how do I build LLVM manually anyway?
--------------------------------------------------
-A: You need cmake and gcc-c++ as build requisites for LLVM. Once you have
-that set up, proceed with building the latest LLVM and clang version
+A: We recommend that developers who want the fastest incremental builds
+use the Ninja build system, you can find it in your system's package
+manager, usually the package is ninja or ninja-build.
+
+You need ninja, cmake and gcc-c++ as build requisites for LLVM. Once you
+have that set up, proceed with building the latest LLVM and clang version
from the git repositories::

$ git clone https://github.com/llvm/llvm-project.git
- $ mkdir -p llvm-project/llvm/build/install
+ $ mkdir -p llvm-project/llvm/build
$ cd llvm-project/llvm/build
$ cmake .. -G "Ninja" -DLLVM_TARGETS_TO_BUILD="BPF;X86" \
-DLLVM_ENABLE_PROJECTS="clang" \
- -DBUILD_SHARED_LIBS=OFF \
-DCMAKE_BUILD_TYPE=Release \
-DLLVM_BUILD_RUNTIME=OFF
$ ninja
diff --git a/samples/bpf/Makefile b/samples/bpf/Makefile
index 26fc96c..d061446 100644
--- a/samples/bpf/Makefile
+++ b/samples/bpf/Makefile
@@ -208,7 +208,7 @@ TPROGLDLIBS_xdpsock += -pthread -lcap
TPROGLDLIBS_xsk_fwd += -pthread

# Allows pointing LLC/CLANG to a LLVM backend with bpf support, redefine on cmdline:
-# make M=samples/bpf/ LLC=~/git/llvm/build/bin/llc CLANG=~/git/llvm/build/bin/clang
+# make M=samples/bpf LLC=~/git/llvm-project/llvm/build/bin/llc CLANG=~/git/llvm-project/llvm/build/bin/clang
LLC ?= llc
CLANG ?= clang
OPT ?= opt
diff --git a/samples/bpf/README.rst b/samples/bpf/README.rst
index dd34b2d..60c6494 100644
--- a/samples/bpf/README.rst
+++ b/samples/bpf/README.rst
@@ -62,20 +62,26 @@ To generate a smaller llc binary one can use::

-DLLVM_TARGETS_TO_BUILD="BPF"

+We recommend that developers who want the fastest incremental builds
+use the Ninja build system, you can find it in your system's package
+manager, usually the package is ninja or ninja-build.
+
Quick sniplet for manually compiling LLVM and clang
-(build dependencies are cmake and gcc-c++)::
+(build dependencies are ninja, cmake and gcc-c++)::

patchwork-b...@kernel.org

unread,
Jan 22, 2021, 6:10:11 PM1/22/21
to Tiezhu Yang, a...@kernel.org, dan...@iogearbox.net, and...@kernel.org, ka...@fb.com, songliu...@fb.com, y...@fb.com, john.fa...@gmail.com, kps...@kernel.org, natecha...@gmail.com, ndesau...@google.com, net...@vger.kernel.org, b...@vger.kernel.org, clang-bu...@googlegroups.com, linux-...@vger.kernel.org, lixu...@loongson.cn, mas...@google.com
Hello:

This patch was applied to bpf/bpf-next.git (refs/heads/master):

On Fri, 22 Jan 2021 09:39:44 +0800 you wrote:
> The current LLVM and Clang build procedure in samples/bpf/README.rst is
> out of date. See below that the links are not accessible any more.
>
> $ git clone http://llvm.org/git/llvm.git
> Cloning into 'llvm'...
> fatal: unable to access 'http://llvm.org/git/llvm.git/': Maximum (20) redirects followed
> $ git clone --depth 1 http://llvm.org/git/clang.git
> Cloning into 'clang'...
> fatal: unable to access 'http://llvm.org/git/clang.git/': Maximum (20) redirects followed
>
> [...]

Here is the summary with links:
- [bpf-next,v4] samples/bpf: Update build procedure for manually compiling LLVM and Clang
https://git.kernel.org/bpf/bpf-next/c/628add78b07a

You are awesome, thank you!
--
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html


Reply all
Reply to author
Forward
0 new messages