building envoy from source

1,626 views

Skip to first unread message

Nikos Skalis

unread,

Aug 31, 2018, 9:01:47 AM8/31/18

to envoy-users

Hi,

I am trying to follow the guidelines described here https://github.com/envoyproxy/envoy/blob/master/ci/prebuilt/BUILD

and build envoy from source (no Docker).

I have installed bazel:

$ bazel version
WARNING: --batch mode is deprecated. Please instead explicitly shut down your Bazel server using the command "bazel shutdown". 
Build label: 0.16.1- (@non-git) 
Build target: bazel-out/k8-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar 
Build time: Mon Aug 13 16:42:29 2018 (1534178549) 
Build timestamp: 1534178549 
Build timestamp as int: 1534178549

I have setup a newer version for gcc:

# gcc --version
gcc (GCC) 7.3.1 20180303 (Red Hat 7.3.1-5) 
Copyright (C) 2017 Free Software Foundation, Inc. 
This is free software; see the source for copying conditions.  There is NO 
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

But I do not know how to install (step #2 in the above link):

These pre-built third party dependencies. Meaning https://github.com/envoyproxy/envoy/tree/master/ci/build_container/build_recipes
These Bazel native dependencies. Meaning https://github.com/envoyproxy/envoy/blob/master/bazel/repository_locations.bzl

Would you be so kind and advise how to complete step #2 ?

nikos....@gmail.com

unread,

Sep 3, 2018, 4:12:02 AM9/3/18

to envoy-users

Hi,

the envoy project directory looks like this:

$ ls -salt
total 176
 0 drwxr-xr-x.  3 app app    37 Aug 31 13:06 ..
 4 drwxrwxr-x. 16 app app  4096 Aug  4 02:28 .
 0 drwxrwxr-x.  2 app app    23 Aug  4 02:28 .circleci
 4 -rwxrwxr-x.  1 app app   259 Aug  4 02:28 .clang-format
 0 drwxrwxr-x.  2 app app    22 Aug  4 02:28 .github
 4 -rw-rw-r--.  1 app app   217 Aug  4 02:28 .gitignore
 4 -rw-rw-r--.  1 app app    61 Aug  4 02:28 BUILD
 4 -rw-rw-r--.  1 app app   433 Aug  4 02:28 CODEOWNERS
 4 -rw-rw-r--.  1 app app   139 Aug  4 02:28 CODE_OF_CONDUCT.md
12 -rw-rw-r--.  1 app app 10993 Aug  4 02:28 CONTRIBUTING.md
 4 -rw-rw-r--.  1 app app  1421 Aug  4 02:28 DCO
 8 -rw-rw-r--.  1 app app  4221 Aug  4 02:28 DEPRECATED.md
 4 -rw-rw-r--.  1 app app  2189 Aug  4 02:28 DEVELOPER.md
12 -rw-rw-r--.  1 app app  8202 Aug  4 02:28 GOVERNANCE.md
 4 -rw-rw-r--.  1 app app  1718 Aug  4 02:28 ISSUE_TEMPLATE.md
12 -rw-rw-r--.  1 app app 11358 Aug  4 02:28 LICENSE
 4 -rw-rw-r--.  1 app app   108 Aug  4 02:28 NOTICE
 4 -rw-rw-r--.  1 app app  2923 Aug  4 02:28 OWNERS.md
 4 -rw-rw-r--.  1 app app  2211 Aug  4 02:28 PULL_REQUEST_TEMPLATE.md
 8 -rw-rw-r--.  1 app app  4844 Aug  4 02:28 README.md
 8 -rw-rw-r--.  1 app app  6907 Aug  4 02:28 REPO_LAYOUT.md
16 -rw-rw-r--.  1 app app 13200 Aug  4 02:28 SECURITY_RELEASE_PROCESS.md
12 -rw-rw-r--.  1 app app 12085 Aug  4 02:28 STYLE.md
 4 -rw-rw-r--.  1 app app     6 Aug  4 02:28 VERSION
 4 -rw-rw-r--.  1 app app   514 Aug  4 02:28 WORKSPACE
 4 drwxrwxr-x.  9 app app  4096 Aug  4 02:28 api
 4 drwxrwxr-x.  3 app app  4096 Aug  4 02:28 bazel
 4 drwxrwxr-x.  4 app app  4096 Aug  4 02:28 ci
 4 drwxrwxr-x.  4 app app  4096 Aug  4 02:28 configs
 4 drwxrwxr-x.  3 app app  4096 Aug  4 02:28 docs
 4 drwxrwxr-x.  8 app app  4096 Aug  4 02:28 examples
 0 drwxrwxr-x.  3 app app    18 Aug  4 02:28 include
 0 drwxrwxr-x.  2 app app    41 Aug  4 02:28 restarter
 0 drwxrwxr-x.  7 app app    70 Aug  4 02:28 source
 0 drwxrwxr-x.  3 app app    50 Aug  4 02:28 support
 4 drwxrwxr-x. 15 app app  4096 Aug  4 02:28 test
 4 drwxrwxr-x.  6 app app  4096 Aug  4 02:28 tools

and there are 2 bazel workspaces like:

$ find . -iname WORKSPACE
./WORKSPACE 
./ci/WORKSPACE

I guess I have to build the bazel native dependencies from the top directory, but i do not know how the specify the target BUILD file.

Could you please advise on that ?

When it comes to the 3rd party dependecies, could you please confirm that the ./ci/WORKSPACE has to be used ?

Then I have to specify the target BUILD file as well, as per my previosu question.

nikos....@gmail.com

unread,

Sep 5, 2018, 4:29:52 AM9/5/18

to envoy-users

That is a very basic step in order to start using envoy, and wish that was documented.

Could you please advise ? On how to build envoy from source ?

On Friday, August 31, 2018 at 3:01:47 PM UTC+2, Nikos Skalis wrote:

Dan Noé

unread,

Sep 5, 2018, 9:38:19 AM9/5/18

to nikos....@gmail.com, Envoy Users

The build step ("blaze build //source/exe:envoy") described in the link should work on its own. It will pull down the external dependencies and build them the first time, then use the built external dependencies for future invocations. If something is going wrong at that step most likely an external dependency is failing to build, or it is failing to download the external dependency sources.

If you want to use separately built dependencies because it makes sense for your production environment you can hack up the WORKSPACE file to point to them, but you shouldn't need to do this to get started.

Check out #envoy-users or #envoy-dev on Slack for more help from the community.

--
You received this message because you are subscribed to the Google Groups "envoy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to envoy-users...@googlegroups.com.
To post to this group, send email to envoy...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/envoy-users/db173597-cc35-406a-a641-b0ce95bb999c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Nikos Skalis

unread,

Sep 5, 2018, 10:00:17 AM9/5/18

to envoy-users

Hi,

Thanks for the advice, will ask in #envoy-dev (i have already asked in #envoy-users) but no luck so far.

This is where it fails:

$ bazel build //source/exe:envoy
WARNING: detected http_proxy set in env, setting no_proxy for localhost. 
Starting local Bazel server and connecting to it... 

INFO: Repository rule 'io_bazel_rules_go' returned: {"remote": "https://github.com/bazelbuild/rules_go", "commit": "a4586a158a1c992756421b99b706832a1c1f6ded", "shallow_since": "", "init_submodules": False, "verbose": False, "strip_prefix": "", "patches": [], "patch_tool": "patch", "patch_args": ["-p0"], "patch_cmds": [], "name": "io_bazel_rules_go"} 

INFO: Repository rule 'com_lyft_protoc_gen_validate' returned: {"remote": "https://github.com/lyft/protoc-gen-validate.git", "commit": "9f600c2cd2d7031fdc8e25e1c9f5ad81c8cab4fe", "shallow_since": "", "init_submodules": False, "verbose": False, "strip_prefix": "", "patches": [], "patch_tool": "patch", "patch_args": ["-p0"], "patch_cmds": [], "name": "com_lyft_protoc_gen_validate"} 

External dependency cache directory /home/app/.cache/bazel/_bazel_app/93cf2aaa5a193ee1dca34ebc2235b4f8/external/envoy_deps_cache_219e66025b43ee3f6e6b9d918b4a2752 

make: Entering directory `/home/app/.cache/bazel/_bazel_app/93cf2aaa5a193ee1dca34ebc2235b4f8/external/envoy_deps' 

cat: /home/app/.cache/bazel/_bazel_app/93cf2aaa5a193ee1dca34ebc2235b4f8/external/envoy_deps_cache_219e66025b43ee3f6e6b9d918b4a2752/cares.dep.env: No such file or directory 

cat: /home/app/.cache/bazel/_bazel_app/93cf2aaa5a193ee1dca34ebc2235b4f8/external/envoy_deps_cache_219e66025b43ee3f6e6b9d918b4a2752/benchmark.dep.env: No such file or directory 

cat: /home/app/.cache/bazel/_bazel_app/93cf2aaa5a193ee1dca34ebc2235b4f8/external/envoy_deps_cache_219e66025b43ee3f6e6b9d918b4a2752/libevent.dep.env: No such file or directory 

cat: /home/app/.cache/bazel/_bazel_app/93cf2aaa5a193ee1dca34ebc2235b4f8/external/envoy_deps_cache_219e66025b43ee3f6e6b9d918b4a2752/gperftools.dep.env: No such file or directory 

cat: /home/app/.cache/bazel/_bazel_app/93cf2aaa5a193ee1dca34ebc2235b4f8/external/envoy_deps_cache_219e66025b43ee3f6e6b9d918b4a2752/nghttp2.dep.env: No such file or directory 

cat: /home/app/.cache/bazel/_bazel_app/93cf2aaa5a193ee1dca34ebc2235b4f8/external/envoy_deps_cache_219e66025b43ee3f6e6b9d918b4a2752/luajit.dep.env: No such file or directory 

cat: /home/app/.cache/bazel/_bazel_app/93cf2aaa5a193ee1dca34ebc2235b4f8/external/envoy_deps_cache_219e66025b43ee3f6e6b9d918b4a2752/yaml-cpp.dep.env: No such file or directory 

cat: /home/app/.cache/bazel/_bazel_app/93cf2aaa5a193ee1dca34ebc2235b4f8/external/envoy_deps_cache_219e66025b43ee3f6e6b9d918b4a2752/zlib.dep.env: No such file or directory 

Building in /home/app/.cache/bazel/_bazel_app/93cf2aaa5a193ee1dca34ebc2235b4f8/external/envoy_deps_cache_219e66025b43ee3f6e6b9d918b4a2752/cares.dep.build, logs at /home/app/.cache/bazel/_bazel_app/93cf2aaa5a193ee1dca34ebc2235b4f8/external/envoy_deps_cache_219e66025b43ee3f6e6b9d918b4a2752/cares.dep.log 

Building in /home/app/.cache/bazel/_bazel_app/93cf2aaa5a193ee1dca34ebc2235b4f8/external/envoy_deps_cache_219e66025b43ee3f6e6b9d918b4a2752/benchmark.dep.build, logs at /home/app/.cache/bazel/_bazel_app/93cf2aaa5a193ee1dca34ebc2235b4f8/external/envoy_deps_cache_219e66025b43ee3f6e6b9d918b4a2752/benchmark.dep.log 

Building in /home/app/.cache/bazel/_bazel_app/93cf2aaa5a193ee1dca34ebc2235b4f8/external/envoy_deps_cache_219e66025b43ee3f6e6b9d918b4a2752/gperftools.dep.build, logs at /home/app/.cache/bazel/_bazel_app/93cf2aaa5a193ee1dca34ebc2235b4f8/external/envoy_deps_cache_219e66025b43ee3f6e6b9d918b4a2752/gperftools.dep.log 

Building in /home/app/.cache/bazel/_bazel_app/93cf2aaa5a193ee1dca34ebc2235b4f8/external/envoy_deps_cache_219e66025b43ee3f6e6b9d918b4a2752/nghttp2.dep.build, logs at /home/app/.cache/bazel/_bazel_app/93cf2aaa5a193ee1dca34ebc2235b4f8/external/envoy_deps_cache_219e66025b43ee3f6e6b9d918b4a2752/nghttp2.dep.log 

Building in /home/app/.cache/bazel/_bazel_app/93cf2aaa5a193ee1dca34ebc2235b4f8/external/envoy_deps_cache_219e66025b43ee3f6e6b9d918b4a2752/luajit.dep.build, logs at /home/app/.cache/bazel/_bazel_app/93cf2aaa5a193ee1dca34ebc2235b4f8/external/envoy_deps_cache_219e66025b43ee3f6e6b9d918b4a2752/luajit.dep.log 

Building in /home/app/.cache/bazel/_bazel_app/93cf2aaa5a193ee1dca34ebc2235b4f8/external/envoy_deps_cache_219e66025b43ee3f6e6b9d918b4a2752/yaml-cpp.dep.build, logs at /home/app/.cache/bazel/_bazel_app/93cf2aaa5a193ee1dca34ebc2235b4f8/external/envoy_deps_cache_219e66025b43ee3f6e6b9d918b4a2752/yaml-cpp.dep.log 

Building in /home/app/.cache/bazel/_bazel_app/93cf2aaa5a193ee1dca34ebc2235b4f8/external/envoy_deps_cache_219e66025b43ee3f6e6b9d918b4a2752/zlib.dep.build, logs at /home/app/.cache/bazel/_bazel_app/93cf2aaa5a193ee1dca34ebc2235b4f8/external/envoy_deps_cache_219e66025b43ee3f6e6b9d918b4a2752/zlib.dep.log 

Building in /home/app/.cache/bazel/_bazel_app/93cf2aaa5a193ee1dca34ebc2235b4f8/external/envoy_deps_cache_219e66025b43ee3f6e6b9d918b4a2752/libevent.dep.build, logs at /home/app/.cache/bazel/_bazel_app/93cf2aaa5a193ee1dca34ebc2235b4f8/external/envoy_deps_cache_219e66025b43ee3f6e6b9d918b4a2752/libevent.dep.log 

INFO: Repository rule 'com_google_absl' returned: {"remote": "https://github.com/abseil/abseil-cpp", "commit": "92020a042c0cd46979db9f6f0cb32783dc07765e", "shallow_since": "", "init_submodules": False, "verbose": False, "strip_prefix": "", "patches": [], "patch_tool": "patch", "patch_args": ["-p0"], "patch_cmds": [], "name": "com_google_absl"} 

INFO: Repository rule 'com_github_google_jwt_verify' returned: {"remote": "https://github.com/google/jwt_verify_lib", "commit": "4eb9e96485b71e00d43acc7207501caafb085b4a", "shallow_since": "", "init_submodules": False, "verbose": False, "strip_prefix": "", "patches": [], "patch_tool": "patch", "patch_args": ["-p0"], "patch_cmds": [], "name": "com_github_google_jwt_verify"} 

INFO: Repository rule 'com_github_circonus_labs_libcircllhist' returned: {"remote": "https://github.com/circonus-labs/libcircllhist", "commit": "476687ac9cc636fc92ac3070246d757ae6854547", "shallow_since": "", "init_submodules": False, "verbose": False, "strip_prefix": "", "patches": [], "patch_tool": "patch", "patch_args": ["-p0"], "patch_cmds": [], "build_file": Label("@envoy//bazel/external:libcircllhist.BUILD"), "build_file_content": "", "workspace_file_content": "", "name": "com_github_circonus_labs_libcircllhist"} 

INFO: Repository rule 'grpc_httpjson_transcoding' returned: {"remote": "https://github.com/grpc-ecosystem/grpc-httpjson-transcoding", "commit": "05a15e4ecd0244a981fdf0348a76658def62fa9c", "shallow_since": "", "init_submodules": False, "verbose": False, "strip_prefix": "", "patches": [], "patch_tool": "patch", "patch_args": ["-p0"], "patch_cmds": [], "name": "grpc_httpjson_transcoding"} 

INFO: Repository rule 'com_github_bombela_backward' returned: {"remote": "https://github.com/bombela/backward-cpp", "commit": "44ae9609e860e3428cd057f7052e505b4819eb84", "shallow_since": "", "init_submodules": False, "verbose": False, "strip_prefix": "", "patches": [], "patch_tool": "patch", "patch_args": ["-p0"], "patch_cmds": [], "build_file": Label("@envoy//bazel/external:backward.BUILD"), "build_file_content": "", "workspace_file_content": "", "name": "com_github_bombela_backward"} 

INFO: Repository rule 'com_github_cyan4973_xxhash' returned: {"remote": "https://github.com/Cyan4973/xxHash", "commit": "7cc9639699f64b750c0b82333dced9ea77e8436e", "shallow_since": "", "init_submodules": False, "verbose": False, "strip_prefix": "", "patches": [], "patch_tool": "patch", "patch_args": ["-p0"], "patch_cmds": [], "build_file": Label("@envoy//bazel/external:xxhash.BUILD"), "build_file_content": "", "workspace_file_content": "", "name": "com_github_cyan4973_xxhash"} 

INFO: Repository rule 'io_opentracing_cpp' returned: {"remote": "https://github.com/opentracing/opentracing-cpp", "commit": "3b36b084a4d7fffc196eac83203cf24dfb8696b3", "shallow_since": "", "init_submodules": False, "verbose": False, "strip_prefix": "", "patches": [], "patch_tool": "patch", "patch_args": ["-p0"], "patch_cmds": [], "name": "io_opentracing_cpp"} 

INFO: Repository rule 'com_github_eile_tclap' returned: {"remote": "https://github.com/eile/tclap", "commit": "3627d9402e529770df9b0edf2aa8c0e0d6c6bb41", "shallow_since": "", "init_submodules": False, "verbose": False, "strip_prefix": "", "patches": [], "patch_tool": "patch", "patch_args": ["-p0"], "patch_cmds": [], "build_file": Label("@envoy//bazel/external:tclap.BUILD"), "build_file_content": "", "workspace_file_content": "", "name": "com_github_eile_tclap"} 

INFO: Repository rule 'com_lightstep_tracer_cpp' returned: {"remote": "https://github.com/lightstep/lightstep-tracer-cpp", "commit": "ae6a6bba65f8c4d438a6a3ac855751ca8f52e1dc", "shallow_since": "", "init_submodules": False, "verbose": False, "strip_prefix": "", "patches": [], "patch_tool": "patch", "patch_args": ["-p0"], "patch_cmds": [], "name": "com_lightstep_tracer_cpp"} 

INFO: Repository rule 'com_github_nodejs_http_parser' returned: {"remote": "https://github.com/nodejs/http-parser", "commit": "cf69c8eda9fe79e4682598a7b3d39338dea319a3", "shallow_since": "", "init_submodules": False, "verbose": False, "strip_prefix": "", "patches": [], "patch_tool": "patch", "patch_args": ["-p0"], "patch_cmds": [], "build_file": Label("@envoy//bazel/external:http-parser.BUILD"), "build_file_content": "", "workspace_file_content": "", "name": "com_github_nodejs_http_parser"} 

INFO: SHA256 (https://github.com/googleapis/googleapis/archive/d642131a6e6582fc226caf9893cb7fe7885b3411.tar.gz) = 16f5b2e8bf1e747a32f9a62e211f8f33c94645492e9bbd72458061d9a9de1f63 

INFO: SHA256 (https://codeload.github.com/bazelbuild/buildtools/zip/799e530642bac55de7e76728fa0c3161484899f6) = ea23bbec9e86205b71ef647e1755ae0ec400aa76aeb5d13913d3fc3a37afbb5f 

INFO: SHA256 (https://github.com/gogo/protobuf/archive/1adfc126b41513cc696b209667c8656ea7aac67c.tar.gz) = c84230a4ec87e1bbb3283d6f5bdd0da7e8f43b2784dec6ab37488f130bdafe37 

Successful build of /home/app/.cache/bazel/_bazel_app/93cf2aaa5a193ee1dca34ebc2235b4f8/external/envoy_deps_cache_219e66025b43ee3f6e6b9d918b4a2752/zlib.dep 

INFO: SHA256 (https://codeload.github.com/golang/tools/zip/3d92dd60033c312e3ae7cac319c792271cf67e37) = e21212ccbd08e265ba2a6abec94abe506d0675267ee89abc7574902f2b53e884 

INFO: Repository rule 'com_github_tencent_rapidjson' returned: {"remote": "https://github.com/tencent/rapidjson", "commit": "f54b0e47a08782a6131cc3d60f94d038fa6e0a51", "shallow_since": "", "init_submodules": False, "verbose": False, "strip_prefix": "", "patches": [], "patch_tool": "patch", "patch_args": ["-p0"], "patch_cmds": [], "build_file": Label("@envoy//bazel/external:rapidjson.BUILD"), "build_file_content": "", "workspace_file_content": "", "name": "com_github_tencent_rapidjson"} 

Successful build of /home/app/.cache/bazel/_bazel_app/93cf2aaa5a193ee1dca34ebc2235b4f8/external/envoy_deps_cache_219e66025b43ee3f6e6b9d918b4a2752/luajit.dep 

Successful build of /home/app/.cache/bazel/_bazel_app/93cf2aaa5a193ee1dca34ebc2235b4f8/external/envoy_deps_cache_219e66025b43ee3f6e6b9d918b4a2752/nghttp2.dep 

Successful build of /home/app/.cache/bazel/_bazel_app/93cf2aaa5a193ee1dca34ebc2235b4f8/external/envoy_deps_cache_219e66025b43ee3f6e6b9d918b4a2752/libevent.dep 

INFO: Repository rule 'boringssl' returned: {"remote": "https://github.com/google/boringssl", "commit": "2a52ce799382c87cd3119f3b44fbbebf97061ab6", "shallow_since": "", "init_submodules": False, "verbose": False, "strip_prefix": "", "patches": [], "patch_tool": "patch", "patch_args": ["-p0"], "patch_cmds": [], "name": "boringssl"} 

Successful build of /home/app/.cache/bazel/_bazel_app/93cf2aaa5a193ee1dca34ebc2235b4f8/external/envoy_deps_cache_219e66025b43ee3f6e6b9d918b4a2752/benchmark.dep 

ERROR: /home/app/.cache/bazel/_bazel_app/93cf2aaa5a193ee1dca34ebc2235b4f8/external/io_bazel_rules_go/proto/BUILD.bazel:21:1: no such package '@org_golang_x_net//context': failed to fetch org_golang_x_net: 2018/09/05 07:57:16 unrecognized import path "golang.org/x/net" 

 and referenced by '@io_bazel_rules_go//proto:go_grpc' 

ERROR: Analysis of target '//source/exe:envoy' failed; build aborted: no such package '@org_golang_x_net//context': failed to fetch org_golang_x_net: 2018/09/05 07:57:16 unrecognized import path "golang.org/x/net" 

INFO: Elapsed time: 104.778s 

INFO: 0 processes. 

FAILED: Build did NOT complete successfully (248 packages loaded)