aparapi-2012-12-02 - Failed to load aparapi native library aparapi_x86_64 or possibly failed to locate opencl native library

[MikOfClassX]

Hi there,

it seems that he new aparapi versions no longer run on Win7/NVidia, or at least they are unable to locate the native libs.

Here's the output for mandel sample:

C:\Users\Mik\Desktop\aparapi-2012-12-02\samples\mandel>java  -Djava.library.path=../..  -Dcom.amd.aparapi.executionMode=  -Dcom.amd.aparapi.enableProfiling=false  -Dcom.amd.aparapi.enableShowGeneratedOpenCL=true  -classpath ../../aparapi.jar;mandel.jar  com.amd.aparapi.sample.mandel.Main2D range= 2D(global:768x768 local:(derived)16x16)

Check your environment. Failed to load aparapi native library aparapi_x86_64 or possibly failed to locate opencl native library (opencl.dll/opencl.so). Ensure that both are in your PATH (windows) or in LD_LIBRARY_PATH (linux). Execution mode=JTP FPS = 17


Running the mandel aparapi-2011-10-13 sample I have no problems and apparently it runs on the GPU at 50 fps.

Cheers,

Mik
--

[gfrost]

Mik

Of course this is possibly not helpful, but I just downloaded the latest binary, unzipped and executed the sample mandel and it worked fine. 

jar xvf aparapi-2012-12-02.zip

cd aparapi-2012-12-02

cd samples

cd mandel 

mandel

Can you try to replicate these steps.  Also check and make sure you don't have another copy of aparapi_x86_64.dll in your path somewhere.   This is most likely the issue .

Gary

[MikOfClassX]

Same result:

C:\Users\Mik\Desktop\aparapi-2012-12-02\samples\mandel>java  -Djava.library.path

=../..  -Dcom.amd.aparapi.executionMode=  -Dcom.amd.aparapi.logLevel=OFF -Dcom.a
md.aparapi.enableVerboseJNI=false  -Dcom.amd.aparapi.enableProfiling=false  -Dco
m.amd.aparapi.enableShowGeneratedOpenCL=true  -Dcom.amd.aparapi.enableVerboseJNI
OpenCLResourceTracking=false  -Dcom.amd.aparapi.dumpFlags=true  -Dcom.amd.aparap
i.enableInstructionDecodeViewer=false  -classpath ../../aparapi.jar;mandel.jar
com.amd.aparapi.sample.mandel.Main
com.amd.aparapi.executionMode{GPU|CPU|JTP|SEQ}=
com.amd.aparapi.logLevel{OFF|FINEST|FINER|FINE|WARNING|SEVERE|ALL}=OFF
com.amd.aparapi.enableProfiling{true|false}=false
com.amd.aparapi.enableProfilingCSV{true|false}=false
com.amd.aparapi.enableVerboseJNI{true|false}=false
com.amd.aparapi.enableVerboseJNIOpenCLResourceTracking{true|false}=false
com.amd.aparapi.enableShowGeneratedOpenCL{true|false}=true
com.amd.aparapi.enableExecutionModeReporting{true|false}=false
com.amd.aparapi.enableInstructionDecodeViewer{true|false}=false
com.amd.aparapi.instructionListenerClassName{<class name which extends com.amd.a
parapi.Config.InstructionListener>}=null

Check your environment. Failed to load aparapi native library aparapi_x86_64 or
possibly failed to locate opencl native library (opencl.dll/opencl.so). Ensure t
hat both are in your PATH (windows) or in LD_LIBRARY_PATH (linux).
Execution mode=JTP

FPS = 19

[Chris Wilkes]

The mandel.sh file in that directory has this:
 -Djava.library.path=../../com.amd.aparapi.jni/dist
it looks like you just have ../.. which doesn't have the JNI files in it.

[MikOfClassX]

Gary,

Ok, Found the problem. it works with the 32bit jvm. fails with the 64bit jvm.

Did you test it ?

Mik

[gfrost]

Weird. 

Yes.  In fact I only have the 64 bit JVM on my Win 7 machine. But it is correct that the 64 bit dll will only load on the 64 bit JVM. 

Try this. 

Modify the mandel.bat file to report the java version and to turn on FINE level logging - here is mine. 

-8<- mandel.bat -

java -version

java ^

 -Djava.library.path=../.. ^

 -Dcom.amd.aparapi.executionMode=%1 ^

 -Dcom.amd.aparapi.logLevel=FINE^

 -Dcom.amd.aparapi.enableVerboseJNI=false ^

 -Dcom.amd.aparapi.enableProfiling=false ^

 -Dcom.amd.aparapi.enableShowGeneratedOpenCL=false ^

 -Dcom.amd.aparapi.enableVerboseJNIOpenCLResourceTracking=false ^

 -Dcom.amd.aparapi.dumpFlags=false ^

 -Dcom.amd.aparapi.enableInstructionDecodeViewer=false ^

 -classpath ../../aparapi.jar;mandel.jar ^

 com.amd.aparapi.sample.mandel.Main

->8-

  

Then run using 

> mandel >log 2>&1

You can exit mandel app as soon as you see the window (it will fall back to JTP for you). 

At the top of your log you should see something like this.  

 -8<- log

c:\Users\gfrost\aparapi\trunk\dist_windows_x86_64\samples\mandel>java -version 

java version "1.8.0-ea"

Java(TM) SE Runtime Environment (build 1.8.0-ea-b45)

Java HotSpot(TM) 64-Bit Server VM (build 24.0-b14, mixed mode)

c:\Users\gfrost\aparapi\trunk\dist_windows_x86_64\samples\mandel>java  -Djava.library.path=../..  -Dcom.amd.aparapi.executionMode=  -Dcom.amd.aparapi.logLevel=FINE -Dcom.amd.aparapi.enableVerboseJNI=true  -Dcom.amd.aparapi.enableProfiling=false  -Dcom.amd.aparapi.enableShowGeneratedOpenCL=true  -Dcom.amd.aparapi.enableVerboseJNIOpenCLResourceTracking=false  -Dcom.amd.aparapi.dumpFlags=true  -Dcom.amd.aparapi.enableInstructionDecodeViewer=false  -classpath ../../aparapi.jar;mandel.jar  com.amd.aparapi.sample.mandel.Main 

com.amd.aparapi.executionMode{GPU|CPU|JTP|SEQ}=

com.amd.aparapi.logLevel{OFF|FINEST|FINER|FINE|WARNING|SEVERE|ALL}=FINE

com.amd.aparapi.enableProfiling{true|false}=false

com.amd.aparapi.enableProfilingCSV{true|false}=false

com.amd.aparapi.enableVerboseJNI{true|false}=false

com.amd.aparapi.enableVerboseJNIOpenCLResourceTracking{true|false}=false

com.amd.aparapi.enableShowGeneratedOpenCL{true|false}=false

com.amd.aparapi.enableExecutionModeReporting{true|false}=false

com.amd.aparapi.enableInstructionDecodeViewer{true|false}=false

com.amd.aparapi.instructionListenerClassName{<class name which extends com.amd.aparapi.Config.InstructionListener>}=null

Dec 05, 2012 2:35:51 PM com.amd.aparapi.OpenCLJNI <clinit>

FINE: arch = amd64

Dec 05, 2012 2:35:51 PM com.amd.aparapi.OpenCLJNI <clinit>

FINE: attempting to load aparapi shared lib aparapi_x86_64

On Wednesday, December 5, 2012 1:55:37 AM UTC-6, MikOfClassX wrote:

->8-

Note that the Java VM arch is 64 bit and this matches the value of arch reported by Aparpi, also that Aparapi reports the name of the shared lib it is looking for is indeed aparapi_x86_64.

Can you recreate this?

Also are you sure your OpenCL dll is 64 bit ;) 

If your OpenCL.dll is 32 bit then the 64 bit Aparapi dll will not link with it, and this will be reported as a failure to load aparapi.dll

Perhaps your OpenCL.dll is 32 bit? Which would mean that only the 32 bit aparapi dll can load it, so you will need to use a 32 bit JVM?

Gary

[gfrost]

Chris

I think that Mik was referring to the path from the samples subdir of the binary distribution.  This differs from the path from the samples dir in the SVN repo.

When we use 'ant dist' to create a binary distribution, the ant script actually hacks the sample .bash and .bat files to modify the paths to fit in with the layout in the binary zip file. 

Yes it is confusing.

I am still trying to work out why I did this :)

Gary 

[MikOfClassX]

Gary,

I have created a pair of logs: one is the dos output as requested, the other is a ProcMon dump of the mandel application.

If you look for "opencl.dll" and "aparapi_x86_64.dll" you see that both are apparently located and loaded correctly from the appropriate locations.
OpenCL is loaded from "C:\Windows\System32\OpenCL.dll" as expected under win7 x64 os.

So.. still confused.

I have enclosed an NVidia system info, just in case.

Cheers,

Mik
--

[Ryan R. LaMothe]

This must be a problem specific to your hardware or environment.

I just unzipped the 12/02 release of Aparapi, executed Mandel.bat and everything executed in GPU mode without a problem.

Interestingly enough, Mandel2D.bat is failing in GPU mode due to "invalid workgroup size".

I am using an AMD W9000 Windows 7 64-bit so it shouldn't be a wimpy hardware problem :)

Have you tried typing 'echo %PATH%' in a cmd window to make sure all of your path entries are valid?

[gfrost]

I have another idea. 

Try downloading dependency walker and  opening aparapi_x86_64.dll 

http://www.dependencywalker.com/

If there is a dependency issue, dependency walker will be able to determine which library or symbol within a library is missing. 

The puzzling thing for me, is that this happened between the latest two binaries. 

If it was between the June 6th and Nov 14th version, it could be an OpenCL 1.2 vs 1.1 build issue. 

In fact the changes between the last two releases are so slight, I can't imagine they are problematic here. 

You might consider swapping (in turn) the Nov 14th and Dec 2 aparapi.jar and/or aparapi_x86_64.dll.  

Gary

[gfrost]

Oh wait I just re-read your initial posting.  You were successful with the Nov 2011 binary.  That was definitely prior to OpenCL 1.2. So I think this is the issue. 

So your OpenCL  is 1.1 and the binaries are now built against 1.2. Sadly OpenCL distros are not backward compatible.  I know for sure that one API we were using in OpenCL 1.1 was removed in OpenCL 1.2. 

Hopefully you can just pick up an OpenCL 1.2 runtime (from NVidia?) or you can build your own dll (in the com.amd.aparapi.jni src) using your (NVidia I guess) OpenCL headers. Either of these should solve this. 

I know this is a pain, but unlike Java. OpenCL does not attempt binary backward compatibility. IMHO this is a very bad idea.

Actually, using dependencywalker you will be able to diagnose this.  My guess is that dependency walker will highlight that aparapi_x86_64.dll is failing to load because of an unresolved symbol clEnqueueMarkerWithWaitList().

The source in question is in aparapi.cpp

#ifdef CL_VERSION_1_2

         status = clEnqueueMarkerWithWaitList(jniContext->commandQueue, 0, NULL, &firstEvent);

#else

         // this was deprecated in 1.1

         status = clEnqueueMarker(jniContext->commandQueue, &firstEvent);

#endif

The method clEnqueueMarker() was removed in 1.2. and replaced by a method of a different signature.

Gary

[MikOfClassX]

Hi Gary,

thanks for your time behind this issue. This is bad news for me...

I already checked aparapy with my (old friend) dependecyWalker and it seems there are no signs of error.

But indeed I checked the entry points in opencl.dll and is seems we are missing clEnqueueMarkerWithWaitList. So this may be the cause of the problems.

But... this does not solve the mistery, as the x86 aparapi mandelbrot is up and running using OpenCL 1.1.

Another (stupid?) question: is it's just matter of a single function, why do not check the CL_DEVICE_VERSION at runtime and load the appropriate
clEnqueueMarkerWithWaitList / clEnqueueMarker on the fly at dll initialization.

Cheers!

Mik

[MikOfClassX]

Hi Gary,

still no news about this win/x64 issue ?

I can confirm that the x86 version is running on GPU, while the x64 falls back to CPU mode. This also happens with the latest NVidia drivers 310.70
Is there anyone with the same issue ?
Did anyone check the latest aparapi on a Win7/x64/nvidia system ?

Cheers,

Mik
--

[gfrost]

Mik

If you are in a position to compile from SVN then you might consider applying the patch (attached) and then rebuilding. It removes the OpenCL 1.2 method with seems to be failing to load. 

Assuming you checkout code to <AparapiRoot>

Apply the patch to <AparapiRoot>/com.amd.aparapi.jni/src/cpp/aparapi.cpp

Then 

  $ cd <AparapiRoot>

  $ ant clean dist

You should end up with a dist_windows_x86_65 dir with a useable jar and dll. 

I cannot explain why the 32 bit version is loading OK.

Gary

[gfrost]

Mik

I applied the patch above and built a new binary based on the r904 trunk with the patch above applied. 

http://code.google.com/p/aparapi/downloads/list

Let me know if this helps. 

Gary

[MikOfClassX]

Gary,

the patch you applied now makes the x64 fly at GPU speeds.... but the x86 version is now falling down to CPU (JTP).

So the behaviour is the opposite as the previous version. And the x86 version is now telling "failed to load....".

Mik

[Ryan R. LaMothe]

Just out of curiosity, if you have a working build environment, can you build and execute the aparapi-issues-71-44 branch?

I'm interested to see if the bug exists in that branch as well.

[MikOfClassX]

Hi,

sadly I did not setup the build environment for Aparapi..

Mik

[Ryan R. LaMothe]

Setting up a build environment is very easy...are you interested in trying? It would help us debug your problem.

[MikOfClassX]

Sure. I'll do my best in the next days.

I hope I'll have no problems or clashes between aparapi build env and the huge number of existing build envs that are currently in my machine..
It would be nice ho have a ready-to-use VirtualBox VM so that I can keep my system "clean".

Mik