Ok, I think I have the root cause.
The issue here is that Java, by default, can end up having the 32-bit library paths in the java.library.path system property and there is a bug in JNR's Platform.Linux:
@Override
public String locateLibrary(final String libName, List<String> libraryPath) {
FilenameFilter filter = new FilenameFilter() {
Pattern p = Pattern.compile("lib" + libName + "\\.so\\.[0-9]+$");
String exact = "lib" + libName + ".so";
public boolean accept(File dir, String name) {
return p.matcher(name).matches() || exact.equals(name);
}
};
List<File> matches = new LinkedList<File>();
for (String path : libraryPath) {
File[] files = new File(path).listFiles(filter);
if (files != null && files.length > 0) {
matches.addAll(Arrays.asList(files));
}
}
//
// Search through the results and return the highest numbered version
// i.e. libc.so.6 is preferred over libc.so.5
//
int version = 0;
String bestMatch = null;
for (File file : matches) {
String path = file.getAbsolutePath();
if (bestMatch == null && path.endsWith(".so")) {
bestMatch = path;
version = 0;
} else {
String num = path.substring(path.lastIndexOf(".so.") + 4);
try {
if (Integer.parseInt(num) >= version) {
bestMatch = path;
}
} catch (NumberFormatException e) {
} // Just skip if not a number
}
}
return bestMatch != null ? bestMatch : mapLibraryName(libName);
}
The bug being that it will return the last match is all the matches have the same version (I suspect it is that (Integer.parseInt(num) >= version) should be (Integer.parseInt(num) > version) but I am not currently sure, as I have yet to dig into this fully)
So what you need to ensure is that the library paths (if they include a mix of 64bit and 32bit libraries) are a palindrome, now the search path is a combination of four system properties in order:
- jnr.ffi.library.path
- jaffl.library.path
- jna.library.path
- java.library.path
So what you want to do is see what the combined path for all of those is... and then (since you are here because there is a 32bit lib on that path) reverse the path and append it to java.library.path
A quicker fix is to just remove the 32-bit folders from java.library.path
By way of example, if I go to the System Information page for my CentOS 6.5 slave, I see that it says:
java.library.path /usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
So I can either set the JVM options for that slave to include -Djava.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib:/lib:/lib64:/usr/lib64:/usr/java/packages/lib/amd64 (which is the palindrome trick) or I can simply remove the 32-bit directories from -Djava.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64
Either of these fixes the issue