[PATCH bpf] samples/bpf: Update README.rst and Makefile for manually compiling LLVM and clang

[Tiezhu Yang]

In the current samples/bpf/README.rst, the url of llvm and clang git
may be out of date, they are unable to access:

$ git clone http://llvm.org/git/llvm.git
Cloning into 'llvm'...
fatal: unable to access 'http://llvm.org/git/llvm.git/': Maximum (20) redirects followed
$ git clone --depth 1 http://llvm.org/git/clang.git
Cloning into 'clang'...
fatal: unable to access 'http://llvm.org/git/clang.git/': Maximum (20) redirects followed

There are different ways to build llvm/clang, I find the Clang Getting
Started page [1] has one way, as Yonghong said, it is better to just
copy the build procedure in Documentation/bpf/bpf_devel_QA.rst to keep
consistent.

I verified the procedure and it is proved to be feasible, so we should
update README.rst to reflect the reality. At the same time, update the
related comment in Makefile.

[1] https://clang.llvm.org/get_started.html

Signed-off-by: Tiezhu Yang <yangt...@loongson.cn>
---
samples/bpf/Makefile | 2 +-
samples/bpf/README.rst | 17 ++++++++++-------
2 files changed, 11 insertions(+), 8 deletions(-)

diff --git a/samples/bpf/Makefile b/samples/bpf/Makefile
index 26fc96c..d061446 100644
--- a/samples/bpf/Makefile
+++ b/samples/bpf/Makefile
@@ -208,7 +208,7 @@ TPROGLDLIBS_xdpsock += -pthread -lcap
TPROGLDLIBS_xsk_fwd += -pthread

# Allows pointing LLC/CLANG to a LLVM backend with bpf support, redefine on cmdline:
-# make M=samples/bpf/ LLC=~/git/llvm/build/bin/llc CLANG=~/git/llvm/build/bin/clang
+# make M=samples/bpf LLC=~/git/llvm-project/llvm/build/bin/llc CLANG=~/git/llvm-project/llvm/build/bin/clang
LLC ?= llc
CLANG ?= clang
OPT ?= opt
diff --git a/samples/bpf/README.rst b/samples/bpf/README.rst
index dd34b2d..d1be438 100644
--- a/samples/bpf/README.rst
+++ b/samples/bpf/README.rst
@@ -65,17 +65,20 @@ To generate a smaller llc binary one can use::
Quick sniplet for manually compiling LLVM and clang
(build dependencies are cmake and gcc-c++)::

- $ git clone http://llvm.org/git/llvm.git
- $ cd llvm/tools
- $ git clone --depth 1 http://llvm.org/git/clang.git
- $ cd ..; mkdir build; cd build
- $ cmake .. -DLLVM_TARGETS_TO_BUILD="BPF;X86"
- $ make -j $(getconf _NPROCESSORS_ONLN)
+ $ git clone https://github.com/llvm/llvm-project.git
+ $ mkdir -p llvm-project/llvm/build/install
+ $ cd llvm-project/llvm/build
+ $ cmake .. -G "Ninja" -DLLVM_TARGETS_TO_BUILD="BPF;X86" \
+ -DLLVM_ENABLE_PROJECTS="clang" \
+ -DBUILD_SHARED_LIBS=OFF \
+ -DCMAKE_BUILD_TYPE=Release \
+ -DLLVM_BUILD_RUNTIME=OFF
+ $ ninja

It is also possible to point make to the newly compiled 'llc' or
'clang' command via redefining LLC or CLANG on the make command line::

- make M=samples/bpf LLC=~/git/llvm/build/bin/llc CLANG=~/git/llvm/build/bin/clang
+ make M=samples/bpf LLC=~/git/llvm-project/llvm/build/bin/llc CLANG=~/git/llvm-project/llvm/build/bin/clang

Cross compiling samples
-----------------------
--
2.1.0

[Yonghong Song]

On 1/18/21 7:53 PM, Tiezhu Yang wrote:
> In the current samples/bpf/README.rst, the url of llvm and clang git
> may be out of date, they are unable to access:

Let us just rephrase the above more clearly, something like below.

The current clang/llvm build procedure in samples/bpf/README.rst is
out of date. See below that the links are not accessible any more.

>
> $ git clone http://llvm.org/git/llvm.git
> Cloning into 'llvm'...
> fatal: unable to access 'http://llvm.org/git/llvm.git/ ': Maximum (20) redirects followed
> $ git clone --depth 1 http://llvm.org/git/clang.git
> Cloning into 'clang'...
> fatal: unable to access 'http://llvm.org/git/clang.git/ ': Maximum (20) redirects followed
>

The llvm community has adopted new ways to build the compiler.
[followed by your descriptions below]

> There are different ways to build llvm/clang, I find the Clang Getting
> Started page [1] has one way, as Yonghong said, it is better to just
> copy the build procedure in Documentation/bpf/bpf_devel_QA.rst to keep
> consistent.
>
> I verified the procedure and it is proved to be feasible, so we should
> update README.rst to reflect the reality. At the same time, update the
> related comment in Makefile.
>
> [1] https://clang.llvm.org/get_started.html
>
> Signed-off-by: Tiezhu Yang <yangt...@loongson.cn>

Ack with minor nits in the above. Also, this is a documentation update.
I think it is okay to target the patch to bpf-next instead of bpf.

Acked-by: Yonghong Song <y...@fb.com>

[Tiezhu Yang]

The current llvm/clang build procedure in samples/bpf/README.rst is

out of date. See below that the links are not accessible any more.

$ git clone http://llvm.org/git/llvm.git
Cloning into 'llvm'...
fatal: unable to access 'http://llvm.org/git/llvm.git/': Maximum (20) redirects followed
$ git clone --depth 1 http://llvm.org/git/clang.git
Cloning into 'clang'...
fatal: unable to access 'http://llvm.org/git/clang.git/': Maximum (20) redirects followed

The llvm community has adopted new ways to build the compiler. There are
different ways to build llvm/clang, the Clang Getting Started page [1] has
one way. As Yonghong said, it is better to just copy the build procedure

in Documentation/bpf/bpf_devel_QA.rst to keep consistent.

I verified the procedure and it is proved to be feasible, so we should
update README.rst to reflect the reality. At the same time, update the
related comment in Makefile.

[1] https://clang.llvm.org/get_started.html

Signed-off-by: Tiezhu Yang <yangt...@loongson.cn>

Acked-by: Yonghong Song <y...@fb.com>
---

v2: Update the commit message suggested by Yonghong,
thank you very much.

[Fangrui Song]

llvm-project/llvm/build/install is not used.

>+ $ cd llvm-project/llvm/build
>+ $ cmake .. -G "Ninja" -DLLVM_TARGETS_TO_BUILD="BPF;X86" \
>+ -DLLVM_ENABLE_PROJECTS="clang" \
>+ -DBUILD_SHARED_LIBS=OFF \

-DBUILD_SHARED_LIBS=OFF is the default. It can be omitted.

>+ -DCMAKE_BUILD_TYPE=Release \
>+ -DLLVM_BUILD_RUNTIME=OFF

-DLLVM_BUILD_RUNTIME=OFF can be omitted if none of
compiler-rt/libc++/libc++abi is built.

>+ $ ninja
>
> It is also possible to point make to the newly compiled 'llc' or
> 'clang' command via redefining LLC or CLANG on the make command line::
>
>- make M=samples/bpf LLC=~/git/llvm/build/bin/llc CLANG=~/git/llvm/build/bin/clang
>+ make M=samples/bpf LLC=~/git/llvm-project/llvm/build/bin/llc CLANG=~/git/llvm-project/llvm/build/bin/clang
>
> Cross compiling samples
> -----------------------
>--
>2.1.0
>

>--
>You received this message because you are subscribed to the Google Groups "Clang Built Linux" group.
>To unsubscribe from this group and stop receiving emails from it, send an email to clang-built-li...@googlegroups.com.
>To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/1611042978-21473-1-git-send-email-yangtiezhu%40loongson.cn.

[Tiezhu Yang]

Yes, just mkdir -p llvm-project/llvm/build is OK.

>
>> + $ cd llvm-project/llvm/build
>> + $ cmake .. -G "Ninja" -DLLVM_TARGETS_TO_BUILD="BPF;X86" \
>> + -DLLVM_ENABLE_PROJECTS="clang" \
>> + -DBUILD_SHARED_LIBS=OFF \
>
> -DBUILD_SHARED_LIBS=OFF is the default. It can be omitted.

I search the related doc [1] [2], remove this option is OK for me.

BUILD_SHARED_LIBS:BOOL

Flag indicating if each LLVM component (e.g. Support) is built as a
shared library (ON) or as a static library (OFF). Its default value is OFF.

[1] https://www.llvm.org/docs/CMake.html
[2] https://cmake.org/cmake/help/latest/variable/BUILD_SHARED_LIBS.html

>
>> + -DCMAKE_BUILD_TYPE=Release \
>> + -DLLVM_BUILD_RUNTIME=OFF
>
> -DLLVM_BUILD_RUNTIME=OFF can be omitted if none of
> compiler-rt/libc++/libc++abi is built.

I am not very sure about it because the default value of
LLVM_BUILD_RUNTIME is ON? [3]

option(LLVM_BUILD_RUNTIME
"Build the LLVM runtime libraries." ON)

[3] https://github.com/llvm/llvm-project/blob/main/llvm/CMakeLists.txt

If anyone has any more suggestions, please let me know.
I will send v3 after waiting for other feedback.

By the way, Documentation/bpf/bpf_devel_QA.rst maybe need a separate
patch to remove some cmake options?

Thanks,
Tiezhu

[Nick Desaulniers]

On Mon, Jan 18, 2021 at 11:56 PM Tiezhu Yang <yangt...@loongson.cn> wrote:
>
> The current llvm/clang build procedure in samples/bpf/README.rst is
> out of date. See below that the links are not accessible any more.
>
> $ git clone http://llvm.org/git/llvm.git
> Cloning into 'llvm'...
> fatal: unable to access 'http://llvm.org/git/llvm.git/': Maximum (20) redirects followed
> $ git clone --depth 1 http://llvm.org/git/clang.git
> Cloning into 'clang'...
> fatal: unable to access 'http://llvm.org/git/clang.git/': Maximum (20) redirects followed
>
> The llvm community has adopted new ways to build the compiler. There are
> different ways to build llvm/clang, the Clang Getting Started page [1] has
> one way. As Yonghong said, it is better to just copy the build procedure
> in Documentation/bpf/bpf_devel_QA.rst to keep consistent.
>
> I verified the procedure and it is proved to be feasible, so we should
> update README.rst to reflect the reality. At the same time, update the
> related comment in Makefile.
>
> [1] https://clang.llvm.org/get_started.html

There's also https://www.kernel.org/doc/html/latest/kbuild/llvm.html#getting-llvm
(could cross link in rst/sphinx).

Is the BPF target not yet on by default? I frown upon disabling other backends.

> - $ make -j $(getconf _NPROCESSORS_ONLN)
> + $ git clone https://github.com/llvm/llvm-project.git
> + $ mkdir -p llvm-project/llvm/build/install
> + $ cd llvm-project/llvm/build
> + $ cmake .. -G "Ninja" -DLLVM_TARGETS_TO_BUILD="BPF;X86" \
> + -DLLVM_ENABLE_PROJECTS="clang" \
> + -DBUILD_SHARED_LIBS=OFF \
> + -DCMAKE_BUILD_TYPE=Release \
> + -DLLVM_BUILD_RUNTIME=OFF
> + $ ninja
>
> It is also possible to point make to the newly compiled 'llc' or
> 'clang' command via redefining LLC or CLANG on the make command line::
>
> - make M=samples/bpf LLC=~/git/llvm/build/bin/llc CLANG=~/git/llvm/build/bin/clang
> + make M=samples/bpf LLC=~/git/llvm-project/llvm/build/bin/llc CLANG=~/git/llvm-project/llvm/build/bin/clang
>
> Cross compiling samples
> -----------------------
> --
> 2.1.0
>

> --
> You received this message because you are subscribed to the Google Groups "Clang Built Linux" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to clang-built-li...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/1611042978-21473-1-git-send-email-yangtiezhu%40loongson.cn.

--
Thanks,
~Nick Desaulniers

[Yonghong Song]

Please submit updated this patch and Documentation/bpf/bpf_devel_QA.rst
together. This way, it is easy to cross check. Thanks.

>
> Thanks,
> Tiezhu
>
[...]

[Tiezhu Yang]

The default value includes BPF.

> I frown upon disabling other backends.

I think the initial aim of the build procedure in samples/bpf/README.rst [1]
is a simple and quick start guide:

[ To generate a smaller llc binary one can use:

-DLLVM_TARGETS_TO_BUILD="BPF" ]

[1] https://github.com/torvalds/linux/blob/master/samples/bpf/README.rst