twister2 compile error at victor

[Ahmet Uyar]

Hi guys,

When I tried to build twister2 at victor, I got below error. It can not compile the protocol-buffer. 

Ahmet

./compile-unpack.sh
WARNING: Output base '/N/u/auyar/.cache/bazel/_bazel_auyar/7cbb9edc52362fd45f0b3afc33551284' is on NFS. This may lead to surprising failures and undetermined behavior.
Starting local Bazel server and connecting to it...
INFO: Analyzed target //scripts/package:tarpkgs (394 packages loaded, 4111 targets configured).
INFO: Found 1 target...
ERROR: /N/u/auyar/.cache/bazel/_bazel_auyar/7cbb9edc52362fd45f0b3afc33551284/external/com_google_protobuf/BUILD:643:36: Executing genrule @com_google_protobuf//:gen_well_known_protos_java failed (Exit 1)
bazel-out/host/bin/external/com_google_protobuf/protoc: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by bazel-out/host/bin/external/com_google_protobuf/protoc)
bazel-out/host/bin/external/com_google_protobuf/protoc: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by bazel-out/host/bin/external/com_google_protobuf/protoc)
bazel-out/host/bin/external/com_google_protobuf/protoc: /lib64/libstdc++.so.6: version `CXXABI_1.3.8' not found (required by bazel-out/host/bin/external/com_google_protobuf/protoc)
Target //scripts/package:tarpkgs failed to build
Use --verbose_failures to see the command lines of failed build steps.
ERROR: /N/u/auyar/twister2/twister2/proto/BUILD:53:14 Executing genrule @com_google_protobuf//:gen_well_known_protos_java failed (Exit 1)
INFO: Elapsed time: 16.210s, Critical Path: 6.64s
INFO: 0 processes.
FAILED: Build did NOT complete successfully
build was not successful. Exiting...

[Supun Kamburugamuve]

Hi Ahmet,

I got the same error with the same bazel version. Will check more onto it.

Best,

Supun..

--
You received this message because you are subscribed to the Google Groups "Twister2" group.
To unsubscribe from this group and stop receiving emails from it, send an email to twister2+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/twister2/CAPBRfYfakdBg3PEn%3D4trgsoivhYQfTmVyzP9jbveqEkwvfJEhw%40mail.gmail.com.

--

Supun Kamburugamuve, PhD

Digital Science Center, Indiana University

Member, Apache Software Foundation; http://www.apache.org
E-mail: supun@apache.org;  Mobile: +1 812 219 2563

[Ahmet Uyar]

Hi Supun,

There is a workaround for a similar problem at: 

https://github.com/tensorflow/tensorflow/issues/40860

However, I could not get it to work with twister2 yet. 

Ahmet

[Ahmet Uyar]

Hi Supun and all,

When I modify below line under "protoc_lib" both for "cc_library" and "cc_binary" at the file: bazel-twister2/external/com_google_protobuf/BUILD

  linkopts = LINK_OPTS + ["-Wl,-rpath,/N/u/auyar/gcc-10.2.0/lib64", "-L/N/u/auyar/gcc-10.2.0/lib64"],

I do not get the previous error. However, I got a new error. 

I attached cmake logs. 

It seems that for compiling Cylon, it is using older versions of cc and c++. I installed gcc 10.2 and it is in the path. However, it is still using cc and c++ from /usr/bin/cc. they are (GCC) 4.8.5. 

I added following envs to .bashrc but that does not work either: 

export CC="/N/u/auyar/gcc-10.2.0/bin/gcc"
export CXX="/N/u/auyar/gcc-10.2.0/bin/c++"

Do you have any suggestions? thanks, 

Ahmet

[Supun Kamburugamuve]

Hi Ahmet, 

It seems the problem comes from the rule that invokes the protoc command to generate the files. Somehow it doesn't take environmental variables. 

I tried to specify a LD_LIBRARY_PATH environment variable using the following command. 

bazel build --config=ubuntu --verbose_failures -s --action_env=LD_LIBRARY_PATH

It seems all the other rules are using this environment variable while protoc is not using it. 

Here is how it is not using it.

/N/u/skamburu/twister2/tempest/twister2/third_party/java/bazel/BUILD:10:14: Generating Java (Immutable) proto_library //third_party/java/bazel:extra_actions_proto failed (Exit 1): protoc failed: error executing command
  (cd /N/u/skamburu/.cache/bazel/_bazel_skamburu/05980b3046bc342787e70bcef42716fa/execroot/__main__ && \
  exec env - \
    PATH=/bin:/usr/bin:/usr/local/bin \
  bazel-out/host/bin/external/com_google_protobuf/protoc '--java_out=bazel-out/host/bin/third_party/java/bazel/extra_actions_proto-speed-src.jar' '-Ithird_party/java/bazel/extra_actions_base.proto=third_party/java/bazel/extra_actions_base.proto' --direct_dependencies third_party/java/bazel/extra_actions_base.proto '--direct_dependencies_violation_msg=%s is imported, but //third_party/java/bazel:extra_actions_proto doesn'\''t directly depend on a proto_library that '\''srcs'\'' it.' third_party/java/bazel/extra_actions_base.proto)
Execution platform: @local_config_platform//:host

Will try to dig into the issue a little more. 

[Supun Kamburugamuve]

Hi Chathura and Ahmet,

I was able to progress a little further using the following command and modifications.

I changed the following two lines in .bazelrc file

build --noincompatible_strict_action_env
run --noincompatible_strict_action_env

Then used the following command

[skamburu@t-login1 twister2]$ bazel build --config=ubuntu --verbose_failures -s --action_env=LD_LIBRARY_PATH scripts/package:tarpkgs 

Now it fails at cylon

[100%] Built target Arrow
-- Arrow installed here: /tmp/cylon.0z655H/build/arrow/install
-- Arrow library: /tmp/cylon.0z655H/build/arrow/install/lib64/libarrow.so
-- FlatBuffers installed here: /tmp/cylon.0z655H/build/arrow/build/flatbuffers_ep-prefix/src/flatbuffers_ep-install
-- Apache Arrow found in /tmp/cylon.0z655H/build/arrow/install/include
-- Found PythonLibs: /usr/lib64/libpython3.6m.so (found version "3.6.8")
-- Looking for pthread.h
-- Looking for pthread.h - not found
-- Configuring incomplete, errors occurred!
See also "/tmp/cylon.0z655H/build/CMakeFiles/CMakeOutput.log".
See also "/tmp/cylon.0z655H/build/CMakeFiles/CMakeError.log".

Target //scripts/package:tarpkgs failed to build

INFO: Elapsed time: 257.919s, Critical Path: 256.37s
INFO: 0 processes.

[Ahmet Uyar]

Hi Supun,

how you checked CMakeOutput.log and CMakeError.log? Are they similar to mine? 

In my case, it is using the cc version that is installed on the machine. Although, I have a newer version of gcc installed on my path. 

Ahmet

[Ahmet Uyar]

Hi Supun,

When I create a link from gcc to cc on gcc/bin directory, then it uses the correct cc and compiles. 

Ahmet

[Pulasthi Supun Wickramasinghe]

Hi Ahmet, 

can you share what you did to resolve this issue? I am getting the same in victor.what do you mean by creating a link, a symbolic link?

Best Regards.

Pulasthi

To view this discussion on the web visit https://groups.google.com/d/msgid/twister2/CAPBRfYdE1yuOQbTXooQSRMQO%2Bzz6w%3DAU8uKtGu6_2xfz87DdZg%40mail.gmail.com.

--

Pulasthi S. Wickramasinghe

PhD Candidate  | Research Assistant

School of Informatics and Computing | Digital Science Center

Indiana University, Bloomington

cell: 224-386-9035

[Ahmet Uyar]

Hi Pulasthi,

I installed gcc-10.2.0 on my home directory. I added it to my path. However, somehow an older version of cc that is installed by default is being used when twister2 is compiled. So, to force the build tool to use gcc, I created a symbolic link from gcc to cc under /N/u/auyar/gcc-10.2.0/bin

now, when I execute "which cc" on the command line, I get:

~/gcc-10.2.0/bin/cc

In addition, I modified the content of the following build file as it is explained in one of the previous email in this thread: bazel-twister2/external/com_google_protobuf/BUILD

Ahmet

[Pulasthi Supun Wickramasinghe]

HI Ahmet

That worked, i did not make any changes in the google-protobuf/BUILD file just linked gcc to cc as you mentioned and the built completed

Best Regards,

Pulasthi