Upgrade to rules_nodejs version >=2.0.0 / build for arm64

[dominik...@sit.fraunhofer.de]

Hi,

I am trying to build Onos for arm64, but rules_nodejs does not have support for arm64 in version 1.6.0. Would it be possible to upgrade it to version >=2.0.0, where support for arm64 was introduced? I tried it myself, but the uprade breaks the loading of rules which i do not know how to fix.

Or is there something else i need which I am missing?

Greetings,

Dominik

[Eric Tang]

Hi,

You may try below version of Bazel for that:

https://github.com/bazelbuild/bazel/archive/release-4.0.0rc4.zip

Regards,

Eric

--
You received this message because you are subscribed to the Google Groups "ONOS Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to onos-dev+u...@onosproject.org.
To view this discussion on the web visit https://groups.google.com/a/onosproject.org/d/msgid/onos-dev/928446ed-bcc6-4097-b536-49a8ab272f96n%40onosproject.org.

[Sean Condon]

To use a different version of Bazel, you should change the .bazelversion file in the root folder.

Use Bazelisk to retrieve the specified version automatically.

To view this discussion on the web visit https://groups.google.com/a/onosproject.org/d/msgid/onos-dev/CANjJQeZAupJASaEM4-M4y-EdDVZMO9jPkv6wifsDamh33%3D1H4g%40mail.gmail.com.

--

Sean Condon

Member of Technical Staff

Open Networking Foundation

se...@opennetworking.org

[dominik...@sit.fraunhofer.de]

The newer version fails with an internal error:

FATAL: bazel crashed due to an internal error. Printing stack trace:
java.lang.RuntimeException: Unrecoverable error while evaluating node 'REPOSITORY_DIRECTORY:@com_google_protobuf' (requested by nodes 'REPOSITORY:@com_google_protobuf')

...

Caused by: net.starlark.java.eval.Starlark$UncheckedEvalException: java.lang.NullPointerException (Starlark stack: [_http_archive_impl@/root/.cache/bazel/_bazel_root/548916f1c2f50851181f393ea08eaba4/external/bazel_tools/tools/build_defs/repo/http.bzl:111:45, download_and_extract@<builtin>])

But anyway, i dont think this would solve the problem.

Bazel itself is working fine (in a self-compiled 3.0.0 version), the problem is that the ruleset rules_nodejs does not define rules for arm64. Thus it falls back to amd64 and then fails to execute the amd64 version of nodejs:

root@dbf2f487f5da:~/onos# bazel build onos
Extracting Bazel installation...
Starting local Bazel server and connecting to it...
INFO: Call stack for the definition of repository 'npm' which is a yarn_install (rule definition at /root/.cache/bazel/_bazel_root/548916f1c2f50851181f393ea08eaba4/external/build_bazel_rules_nodejs/internal/npm_install/npm_install.bzl:434:16):
 - <builtin>
 - /root/.cache/bazel/_bazel_root/548916f1c2f50851181f393ea08eaba4/external/build_bazel_rules_nodejs/index.bzl:85:5
 - /root/onos/WORKSPACE:170:1
ERROR: An error occurred during the fetch of repository 'npm':
   pre_process_package_json.js failed:
STDOUT:

STDERR:
/root/.cache/bazel/_bazel_root/548916f1c2f50851181f393ea08eaba4/external/nodejs_linux_amd64/bin/node: line 19: /root/.cache/bazel/_bazel_root/548916f1c2f50851181f393ea08eaba4/external/nodejs_linux_amd64/bin/nodejs/bin/node: No such file or directory
ERROR: no such package '@npm//': pre_process_package_json.js failed:
STDOUT:

STDERR:
/root/.cache/bazel/_bazel_root/548916f1c2f50851181f393ea08eaba4/external/nodejs_linux_amd64/bin/node: line 19: /root/.cache/bazel/_bazel_root/548916f1c2f50851181f393ea08eaba4/external/nodejs_linux_amd64/bin/nodejs/bin/node: No such file or directory
INFO: Elapsed time: 44.171s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded)

Note: The node executable actually exists, it just cant execute it.

I tried fixing this by changing rules_nodejs to version 2.0.0 (WORKSPACE line 135), adding a line for arm64 in the node_repositories (line 162) and changing the version numbers of all necessary packages to 2.0.0 (web/gui2/package.json). Then the build seems to use the correct nodejs version, but fails due to a cycle in the workspace file. This is probably due to the changes to load() mentioned in the release notes of version 2.0.0: https://github.com/bazelbuild/rules_nodejs/releases/tag/2.0.0
Thus i was asking if it would be possible to upgrade to rules_nodejs version 2.0.0 or higher.

[Eric Tang]

I haven’t tried that, you may try not to run as root.

BTW, bazel expunge may be required to clear cache.

This has rules_nodejs version 2.x: https://github.com/bazelbuild/bazel/blob/release-4.0.0rc4/WORKSPACE

[Sean Condon]

Dominik - I have no problem with trying the newer version of rules_nodejs - but this is something you should be able to try out locally on your own machine, without needing to commit any code.

Also in my experience, any version of NodeJS above 10.16 will cause problems with the legacy ONOS GUI (GUI1). I think you would have to update the list in "node_repositories" on line 164 or WORKSPACE too to support Arm. 

There was a lot of churn in earlier releases of rules_nodejs that caused incompatibilities, but there's only one way to really know - try it out.

I agree with Eric, you should be building as a non root user, or the build will break.

In any case the compilation of ONOS on Arm has not been successfully done before to my knowledge - you may encounter many difficulties, but it is great that you are tackling it.

Best regards, Sean

[Eric Tang]

Regarding the internal error of 4.0.0-rc4, it seems to be fixed in
below version:
https://releases.bazel.build/4.0.0/rc6/

Good luck.

>>>>>> <https://groups.google.com/a/onosproject.org/d/msgid/onos-dev/928446ed-bcc6-4097-b536-49a8ab272f96n%40onosproject.org?utm_medium=email&utm_source=footer>
>>>>>> .

>>>>>>
>>>>> --
>>>>> You received this message because you are subscribed to the Google
>>>>> Groups "ONOS Developers" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>>> an email to onos-dev+u...@onosproject.org.
>>>>>
>>>> To view this discussion on the web visit

>>>>> https://groups.google.com/a/onosproject.org/d/msgid/onos-dev/CANjJQeZAupJASaEM4-M4y-EdDVZMO9jPkv6wifsDamh33%3D1H4g%40mail.gmail.com
>>>>> <https://groups.google.com/a/onosproject.org/d/msgid/onos-dev/CANjJQeZAupJASaEM4-M4y-EdDVZMO9jPkv6wifsDamh33%3D1H4g%40mail.gmail.com?utm_medium=email&utm_source=footer>
>>>>> .

[dominik...@sit.fraunhofer.de]

Okay, thanks for the tips. And yeah, I already updated the "node_repositories" list in the WORKSPACE.

I am now currently porting onos to rules_nodejs 2.3.1 on x64 (because i thought it would be easier to patch on x64 and then port to arm64). Currently I am using bazel 4.0.0.rc6 and onos 2.5.0, but from my experience bazel 3.0.0 and rules_nodejs 2.0.0 showed the same behaviour here.
It seems like my changes are working mostly, I just got some small problems left where I don't know how to solve them (this is the first time for me working with bazel / angular).  It would be nice if you could help me with those, I appended patches below so you can try it yourself. Also it seems like the build would work if I could build without the GUI. Is it possible to build without it? My target systen is an embedded system, so i dont need a GUI.

These were my steps:
I changed the WORKSPACE, web/gui2/package.json, .bazelversion and .bazelrc for the new versions.
Changes as decribed here: https://github.com/bazelbuild/rules_nodejs/wiki#migrating-load-statements-from-npm_bazel_-to-npmbazel
https://github.com/bazelbuild/rules_nodejs/wiki#ivy
The latter failed on the web/gui2/src/main/webapp/BUILD.bazel file due to a missing asset parameter in the ng_module node, I fixed this manually. Also I changed the tools/build/bazel/angular_workspace.bzl because the ts_setup_workspace target is not defined anymore in the new version.
The last step was to change multiple load statements because those would not find the files, e.g. from "@npm_angular_bazel//:" to "@npm//@angular/bazel:". This can be done with "find . -type f -exec sed -i 's,@npm_\(.*\)_\(.*\)//:,@npm//@\1/\2:,g' {} +".

You can recreate these steps by applying onos.patch to the onos root. I also appended a zip file with a Dockerfile if you want to try it out instead of directly patching.

When I build this, i get two errors in the cache:

• ERROR: /home/user/onos/web/gui2/src/main/webapp/BUILD.bazel:66:16: every rule of type terser_minified implicitly depends upon the target '@npm//@bazel/terser:terser_config.default.json', but this target could not be found because of: no such target '@npm//@bazel/terser:terser_config.default.json': target 'terser_config.default.json' not declared in package '@bazel/terser'; however, a source file of this name exists.  (Perhaps add 'exports_files(["terser_config.default.json"])' to @bazel/terser/BUILD?) defined by /home/user/.cache/bazel/_bazel_user/c42cbfd2cdf6b9981f69ddf5581a02a8/external/npm/@bazel/terser/BUILD.bazel
• ERROR: /home/user/onos/web/gui2/src/main/webapp/app/BUILD.bazel:27:1: error loading package 'web/gui2-fw-lib': in /home/user/.cache/bazel/_bazel_user/c42cbfd2cdf6b9981f69ddf5581a02a8/external/npm/@angular/bazel/index.bzl: Label '@npm//src:ng_module.bzl' is invalid because 'src' is not a package; perhaps you meant to put the colon here: '@npm//:src/ng_module.bzl'? and referenced by '//web/gui2/src/main/webapp/app:app'
  

The first one is easily solvable by adding the suggestion. The second error (and multiple others with the same underlying problem) are also solvable by changing the load statements. You can apply both fixes by applying cache.patch to the cache. But it seems like those errors would vanish if bazel would use a new version of those packages, but i don't know how to do this. After patching i get another error:

• ERROR: /home/user/onos/web/gui2-fw-lib/BUILD.bazel:31:11: //web/gui2-fw-lib:gui2-fw-lib: no such attribute 'entry_point' in 'ts_library' rule

This is due to migrating from angulars ng_module to the equivalent ts_library (because ng_module is deprecated, as mentioned in one of the links above). I guess ts_library does not have the entrypoint used here.

So, to summarize: Can you either help me with those problems or tell me how to build without the GUI?

Thank you and have a nice weekend,

Dominik

[Eric Tang]

I'm able to build onos-2.5 on amd64 with a few hacks on .bazelrc using 4.0.0-rc6. I let others in the know to assist.

[dominik...@sit.fraunhofer.de]

I managed to build onos-2.5 using my patch with rules_nodejs 2.3.1 and bazel 3.0.0 using the minimal profile on amd64 (the GUI still has errors in it, the rest should work i think). I will try building it on arm64 now.

[dominik...@sit.fraunhofer.de]

The arm64 build was also successful, the only thing I needed to change was using the Amazon Corretto JDK instead of the default JDK. The default JDK was throwing a segmentation fault.

[Eric Tang]

Awesome.

[dominik...@sit.fraunhofer.de]

I am currently not able to run onos because two libraries are not available for arm64:

2020-12-16T16:03:06,772 | ERROR | activator-1-thread-2 | BootFeaturesInstaller            | 11 - org.apache.karaf.features.core - 4.2.9 | Error installing boot features

org.apache.karaf.features.internal.util.MultiException: Error restarting bundles:

    Unable to resolve io.netty.tcnative-boringssl-static [141](R 141.0): missing requirement [io.netty.tcnative-boringssl-static [141](R 141.0)] osgi.native; ...

    Unable to resolve org.knowhowlab.osgi.sigar [162](R 162.0): missing requirement [org.knowhowlab.osgi.sigar [162](R 162.0)] osgi.native;

But it seems like karaf wants to use a jar for both cases, which should be executable on arm64. How can i tell Karaf that those jars are executable on arm64? At least netty-tcnative is definitely cross-platform: https://netty.io/wiki/forked-tomcat-native.html#artifacts

I also compiled and installed sigar manually for arm64, but karaf does not detect that.

[dominik...@sit.fraunhofer.de]

Is there any way to use local jar files to satisfy dependencies of onos (in deps.json)? My issue is that one jar dependency does not contain code for arm64, but that should be easily fixable if i could modify it. It seems like onos-lib-gen and java_import_external don't work with local files, but is there any other way to pass jar files to onos? I could modify the onos-lib-gen to accept local files, but i dont know what command I need to use instead of the java_import_external to import local files.

Additionally i tried to add a maven plugin (this one: https://github.com/trustin/os-maven-plugin) to the tools/build/pom.xml, but bazel seems to ignore my changes (it also accepts the file if it contains syntax errors). Is there anything I need to do to propagate the changes to the build?

[Eric Tang]

Not sure if a similar local hack for maven repository would work: https://groups.google.com/a/opennetworking.org/g/trellis-dev/c/NqPea1Aqee8/m/1j3C02cJAAAJ

There is another way suggested which I haven't tried: https://groups.google.com/a/opennetworking.org/g/trellis-dev/c/NqPea1Aqee8/m/nkYfxPEIAAAJ

[Sean Condon]

Dominik

I took some of your onos.patch (above) and made it in to a real ONOS patch.

Upgraded to bazel 3.7.2 and rules_nodejs 2.3.2 | https://gerrit.onosproject.org/c/onos/+/24245

Can you check that this meets the need you had for building on arm64?

Best regards, Sean

To view this discussion on the web visit https://groups.google.com/a/onosproject.org/d/msgid/onos-dev/CANjJQeYb%3DYSS_yvE96gXuZnn4Kf_5t1h4HGcOjksa93n5n050w%40mail.gmail.com.

[Lorych, Dominik]

Hi Sean,

thanks for the work, the patch is working. The GUI still does not work, but that seems to be due to my local phantomjs installation.

But the problems with the incompatible jar's persists. netty-tcnative-boringssl-static supports arm64 in a newer version (2.0.35), but it does not work because on my system the architecture is reported as aarch64, while the jar expects aarch_64. There is a maven plugin which fixes this (https://github.com/trustin/os-maven-plugin), but bazel seems to ignore my changes to multiple pom.xml's and the template.pom (even changes with syntax errors are getting built). Is there anything i need to do to propagate the changes to the build?

Best regards,

Dominik

---

Von: Sean Condon [se...@opennetworking.org]
Gesendet: Montag, 4. Januar 2021 17:21
An: Eric Tang
Cc: Lorych, Dominik; ONOS Developers
Betreff: Re: [onos-dev] Upgrade to rules_nodejs version >=2.0.0 / build for arm64