building against glibc v 2.12

[Andrey Ponomarev]

Hi,

yarn cluster nodes I deploy to doesn't support glibc v. 2.14. 

Thus, to run onnxruntime on 'em I need to rebuild with glibc v. 2.12. 

I tried to do that according the instructions given on javacpp-presets wiki page but with centos 6  image, while centos 7 is based on glibc 2.14, and failed many times with:

```

[ERROR] Failed to execute goal org.bytedeco:javacpp:1.5.4:build (javacpp-cppbuild-install) on project onnx: Execution javacpp-cppbuild-install of goal org.bytedeco:javacpp:1.5.4:build failed: Process exited with an error: 1 -> [Help 1]

org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.bytedeco:javacpp:1.5.4:build (javacpp-cppbuild-install) on project onnx: Execution javacpp-cppbuild-install of goal org.bytedeco:javacpp:1.5.4:build failed: Process exited with an error: 1

at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)

at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)

at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)

at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)

at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)

at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)

at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)

at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)

at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)

at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)

at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)

at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)

at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:498)

at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)

at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)

at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)

at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)

Caused by: org.apache.maven.plugin.PluginExecutionException: Execution javacpp-cppbuild-install of goal org.bytedeco:javacpp:1.5.4:build failed: Process exited with an error: 1

at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:145)

at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)

... 20 more

Caused by: java.lang.RuntimeException: Process exited with an error: 1

at org.bytedeco.javacpp.tools.Builder.build(Builder.java:1060)

at org.bytedeco.javacpp.tools.BuildMojo.execute(BuildMojo.java:417)

at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)

... 21 more

```

 

Would you have a suggestion on the issue ? 

Is it at all possible ?

[Samuel Audet]

ONNX and ONNX Runtime's build systems need Python 3. What does `python3 --version` returns?

[Andrey Ponomarev]

Great, many thanks. I'll try and report back ASAP

```

[root@1c1042523b8c onnx]# python --version

Python 2.6.6

[root@1c1042523b8c onnx]# python3 --version

bash: python3: command not found

```

среда, 21 октября 2020 г. в 05:03:26 UTC+3, Samuel Audet:

[Andrey Ponomarev]

I played around with dependencies (devtoolset-7,8,9, rh-python36, etc.) and applied some memory adjustments (MacOs Docker Desktop has default memory limitations, which could cause oom and killing of building proccess by os).

Anyway, the result is the same.

```

[ERROR]

[ERROR] Re-run Maven using the -X switch to enable full debug logging.

[ERROR]

[ERROR] For more information about the errors and possible solutions, please read the following articles:

[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException

[root@780bd863dca2 onnx]# python3 --version

Python 3.6.9

```

среда, 21 октября 2020 г. в 05:03:26 UTC+3, Samuel Audet:

ONNX and ONNX Runtime's build systems need Python 3. What does `python3 --version` returns?

[Samuel Audet]

rh-python36 is not available for CentOS 6. You'll need to install it some other way

[Andrey Ponomarev]

Well, but somehow it's installed. And actually I tried with `yum install python3` as well:

```

[root@780bd863dca2 onnx]# rpm -qa | grep rh-python36

rh-python36-python-setuptools-36.5.0-1.el6.noarch

rh-python36-python-virtualenv-15.1.0-2.el6.noarch

rh-python36-python-libs-3.6.9-2.el6.x86_64

rh-python36-python-pip-9.0.1-2.el6.noarch

rh-python36-python-devel-3.6.9-2.el6.x86_64

rh-python36-2.0-1.el6.x86_64

rh-python36-runtime-2.0-1.el6.x86_64

rh-python36-python-3.6.9-2.el6.x86_64

[root@780bd863dca2 onnx]# yum list installed rh-python36

Loaded plugins: fastestmirror, ovl

Loading mirror speeds from cached hostfile

 * base: mirror.tversu.ru

 * centos-sclo-rh: mirror.tversu.ru

 * centos-sclo-sclo: mirror.tversu.ru

 * epel: mirror.yandex.ru

 * extras: mirror.tversu.ru

 * updates: mirror.tversu.ru

Installed Packages

rh-python36.x86_64                                                                                 2.0-1.el6                                                                                 @centos-sclo-rh

[root@780bd863dca2 onnx]# cat /etc/redhat-release

CentOS release 6.10 (Final)

[root@780bd863dca2 onnx]#

[root@780bd863dca2 onnx]# python3 --version

Python 3.6.9

[root@780bd863dca2 onnx]#

```

среда, 21 октября 2020 г. в 10:43:06 UTC+3, Samuel Audet:

[Samuel Audet]

Ah, yes, you're right. Python 3 has been available for CentOS 6 for a
while now. Hum, it builds fine on CentOS 6 for me actually, so I guess
I'll just update the build accordingly on Travis CI and you should be
able to use the snapshots for this: http://bytedeco.org/builds/

[Samuel Audet]

Ok, the snapshots are up, give it a try: http://bytedeco.org/builds/

If you'd still like to build from source, I had to make a very small
change to the build script for ONNX to get it working on CentOS 6:
https://github.com/bytedeco/javacpp-presets/commit/d3e2a832acc924f6685261d76f11c7d291a915e9#diff-d6ac4243385b1e1a23f13d5a366f46804cf31f038e7c7ce55decb435bb318f77

[Andrey Ponomarev]

Many thanks for that. 

But now  libOpenCL.so.1 is lacking:

```

....jar/org/bytedeco/dnnl/linux-x86_64/libjnidnnl.so: libOpenCL.so.1: cannot open shared object file: No such file or directory 

```

onnx and dnnl presets are used now. 

Could you recommend a way round ?

пятница, 23 октября 2020 г. в 04:19:20 UTC+3, Samuel Audet:

[Samuel Audet]

I see, it's easy enough to install on CentOS though.
Run `yum install ocl-icd-devel` as shown in the CI script:
https://github.com/bytedeco/javacpp-presets/blob/master/ci/install-travis.sh

[Samuel Audet]

I've now added presets for OpenCL, so the libraries don't need to be installed on the system anymore. Please give a try with onnxruntime-1.6.0-1.5.5-SNAPSHOT! Thanks

[Paula Schultz]

Tried using 1.6.0-1.5.5-SNAPSHOT , got a problem :

[error] Caused by: java.lang.UnsatisfiedLinkError: /Users/d.khalmatov/.javacpp/cache/dnnl-1.7-1.5.5-20201215.141603-44-macosx-x86_64.jar/org/bytedeco/dnnl/macosx-x86_64/libjnidnnl.dylib: dlopen(/Users/d.khalmatov/.javacpp/cache/dnnl-1.7-1.5.5-20201215.141603-44-macosx-x86_64.jar/org/bytedeco/dnnl/macosx-x86_64/libjnidnnl.dylib, 1): Library not loaded: @rpath/libOpenCL.1.dylib

[error]   Referenced from: /Users/d.khalmatov/.javacpp/cache/dnnl-1.7-1.5.5-20201215.141603-44-macosx-x86_64.jar/org/bytedeco/dnnl/macosx-x86_64/libjnidnnl.dylib

[Samuel Audet]

Hi, Paula,

Sorry about that, it just looks like the last build for Mac messed up.
I've reran it and it seems fine now. Please try again by downloading the
latest binaries with mvn -U ... Thanks!

Samuel

[Paula Schultz]

Now it works as intended. Thank you!