Bluecove native library loading, Fedora RPM packaging

[F. Kooman]

Hi,

I'm working on packaging BlueCove for Fedora. I'm running into a few
issues with the native library loading.

Files on x86_64:
/usr/lib64/bluecove/bluecove-2.1.0.jar
/usr/lib64/bluecove/libbluecove_x64.so

Files on i386:
/usr/lib/bluecove/bluecove-2.1.0.jar
/usr/lib/bluecove/libbluecove.so

So, the library is removed from the JAR file and put in the bluecove
directory. As it is a "private" library no other software needs it to
link with it as "required" by the Fedora packaging guidelines [2].

How would I specify the library path? With a system property? But how
can I set that? I read in [1] that you can create a file
"bluecove.native.path" inside the JAR with the correct library path in
it? This doesn't seem to work...

In this case it also doesn't make sense any more to have different
names for the libraries as they can't conflict with each other anyway?

Furthermore Fedora also builds on ppc and ppc64 for which there is
nothing for the library loader so far. Of course I can patch that in
as well, but maybe there is a better solution :)

Any thoughts? Thanks,

François

[1] http://code.google.com/p/bluecove/source/browse/trunk/bluecove/src/main/java/com/intel/bluetooth/BlueCoveConfigProperties.java
[2] https://fedoraproject.org/wiki/Packaging/Java

[Vlad Skarzhevskyy]

Hi François
Thanks for your work.

1. Why jars on 64 and 32 platforms are different?

2. For fedora ideal package context would be:
/usr/lib/bluecove/bluecove-2.1.0.jar
/usr/lib/bluecove/bluecove-emu-2.1.0.jar
/usr/lib/bluecove/bluecove-gpl.2.1.0.jar
Files on i386:
/usr/lib/bluecove/2.1.0/libbluecove.so
Files on x86_64:
/usr/lib64/bluecove/2.1.0/libbluecove.so


The jar bluecove-gpl.2.1.0.jar should have no native code inside!

This Should work.
-Dbluecove.native.path=/usr/lib64/bluecove/libbluecove.so

I can modify library loader so the path would be a real path that
(users used to have:) like this
/usr/lib/bluecove/:/usr/lib64/bluecove/

As to ppc and ppc64 I don't have this computers to test the bluecove.
Probably it will compile and work. Send the suggested changes Please!
BTW does path looks like this: /usr/libppc/ and /usr/libppc64/ on ppc ?

--
Vlad

[Vlad Skarzhevskyy]

Ups my mistake Path is path Not file name
-Dbluecove.native.path=/usr/lib64/bluecove

I will add support for path separators now. So this will work in 2.1.1
-Dbluecove.native.path=/my/lib64/bluecove:/my/lib/bluecove

Also the default path on Linux would be:
Files on i386:
/usr/lib/bluecove/${version}/
Files on x86_64:
/usr/lib64/bluecove/${version}/

This is is SVN for version 2.1.1



And I'd prefere on 64bit platform package to place the file here:
/usr/lib64/bluecove/2.1.0/libbluecove_x64.so

See then name of the file as you originally wrote.
Any suggestions are welcome!

--
Vlad

[F. Kooman]

On Jan 16, 5:15 pm, Vlad Skarzhevskyy <skarzhevs...@gmail.com> wrote:
> 1.  Why jars on 64 and 32 platforms are different?

The JARs for 64 bit and 32 bit are the same as far as I know in my
idea? Fedora Packaging prefers that JARs which use JNI are placed in /
usr/lib(64)/$name/ together with their native library, but the JAR
files will actually be the same.

> 2. For fedora ideal package context would be:
> /usr/lib/bluecove/bluecove-2.1.0.jar
> /usr/lib/bluecove/bluecove-emu-2.1.0.jar
> /usr/lib/bluecove/bluecove-gpl.2.1.0.jar
> Files on i386:
> /usr/lib/bluecove/2.1.0/libbluecove.so
> Files on x86_64:
> /usr/lib64/bluecove/2.1.0/libbluecove.so

Well, this doesn't keep stuff together. For Fedora it would make more
sense then to put the JAR files in /usr/share/java where the other
JARs are (that don't use JNI) but this isn't preferred according to
the packaging guidelines. Of course those aren't holy, but there are
some good reasons for it.

>  The jar bluecove-gpl.2.1.0.jar should have no native code inside!

Well, that is what happens when I use the Ant buildscript and it is
also described in the docs online. It puts the native lib also inside
the JAR file itself.

> This Should work.
> -Dbluecove.native.path=/usr/lib64/bluecove/libbluecove.so

Where would I specify this -D flag? and at build time? At runtime?

> I can modify library loader so the path would be a real path that
> (users used to have:) like this
> /usr/lib/bluecove/:/usr/lib64/bluecove/

This would be then hard coded in the native lib loader?

> As to ppc and ppc64 I don't have this computers to test the bluecove.
> Probably it will  compile and work. Send the suggested changes Please!
> BTW does path looks like this:  /usr/libppc/ and /usr/libppc64/ on ppc ?

I guess it's just lib and lib64 as well, but I'll get to that once
i386 and x86_64 work :)

Thanks!
François

[F. Kooman]

On Jan 16, 5:46 pm, Vlad Skarzhevskyy <skarzhevs...@gmail.com> wrote:
> Also the default path on Linux would be:
> Files on i386:
> /usr/lib/bluecove/${version}/
> Files on x86_64:
> /usr/lib64/bluecove/${version}/

Why version number in the path? That seems unnecessary as the native
lib is always rebuild together with the JAR file.

> Any suggestions are welcome!

I'd like to put everything in bluecove-2.1.0.jar (also the GPL
components) as to make it possible to just have to include one JAR
file in your project's classpath and have it work. I realise that is
not quite compatible with your ideas as to have a separate gpl JAR
file, but from the developer's point of view it seems easier.

Let me know what you think, thanks for your response!

François

[Mina Shokry]

Hi Francois,

we can not put everything in one jar because of different licenses.
main bluecove jar is apache licensed and linux module is gpl licensed!

the problem is legal not technical!
if you could find a legal solution, it will be very nice.

[F. Kooman]

On Jan 17, 2:23 pm, Mina Shokry <minasho...@gmail.com> wrote:
> the problem is legal not technical!
> if you could find a legal solution, it will be very nice.

From: http://www.gnu.org/licenses/gpl-faq.html#GPLRequireSourcePostedPublic
======
If a library is released under the GPL (not the LGPL), does that mean
that any program which uses it has to be under the GPL or a GPL-
compatible license?

Yes, because the program as it is actually run includes the
library.
======

From: http://www.gnu.org/licenses/license-list.html#GPLCompatibleLicenses
======
Apache License, Version 2.0
This is a free software license, compatible with version 3 of the
GPL.

Please note that this license is not compatible with GPL version
2, because it has some requirements that are not in the older version.
These include certain patent termination and indemnification
provisions.
======

BlueCove is Apache License 2.0, BlueZ is GPLv2+ (so GPLv2 or GPLv3),
BlueCove-gpl is GPLv3+. So they are compatible which means that you
can use BlueZ in BlueCove, I guess bluecove-gpl could even be Apache
2.0 licensed?

Why would it not be possible to put differently licensed files inside
one JAR file? For the record: IANAL.

[Vlad Skarzhevskyy]

This is my points.
- There should be minimum two jars in RPM package, because this is
the way library distributed now!
Don't forget that there are additional modules:
bluecove-emu, bluecove-emu-gui, bluecove-bluez
Also in next version I would add bluecove-stack

- I suggested /usr/lib/bluecove/${version}/ because native library
cab be loaded only once.
If it is not compatible it can't be unloaded so proper version is loaded.
User may have different versions of BlueCove on his computer. One
from package one from webstart, one from maven repository .....

- Build puts the native lib inside the JAR. This can be changed!
There is no point to have native libs inside RPM package twice. One in
jar another in /usr/lib/bluecove

- I have no preferences where to put jar in the system
/usr/share/java or /usr/lib(64)/$name/
I think that it would be nice to have then located together side by side.

Regards,
Vlad

PS
We need to clarify "BlueCove-gpl" GPLv2 or GPLv3. I was thinking
that we should keep the same license as BlueZ. The links on out
website are incorrect. We never specified the exact version of GPL we
used.

[F. Kooman]

On Jan 17, 9:02 pm, Vlad Skarzhevskyy <skarzhevs...@gmail.com> wrote:
> This is my points.
> - There should be minimum two jars in  RPM package,  because this is
> the way library distributed now!

I have done this now, actually gpl is now a subpackage in bluecove
spec file. This can still be split up in multiple "real" packages if
necessary, it may make sense to do that seen the amount of different
modules :)

>    Don't forget that there are additional modules:
>     bluecove-emu, bluecove-emu-gui, bluecove-bluez

bluecove-bluez is what exactly? A backend that doesn't use BlueZ
itself directly, only through dbus? Or is it only for device
discovery / using existing trusts/bindings? Will it eventually replace
bluecove-gpl or is it an addon for bluecove-gpl?

>    Also in next version I would add bluecove-stack

What is the purpose of this one?

> - I suggested /usr/lib/bluecove/${version}/  because native library
> cab be loaded only once.
>    If it is not compatible it can't be unloaded so proper version is loaded.
>    User may have different versions of BlueCove on his computer. One
> from package one from webstart, one from maven repository .....

I'm not sure I understand correctly why it's needed, but for now I
have done this, the problem now is that "SNAPSHOT" is included in the
version, so the layout of the (bluecove-gpl) package on x86_64 is like
this:

drwxr-xr-x /usr/lib64/bluecove
drwxr-xr-x /usr/lib64/bluecove/2.1.1-SNAPSHOT
-rw-r--r-- /usr/lib64/bluecove/2.1.1-SNAPSHOT/bluecove-gpl-2.1.1.jar
-rwxr-xr-x /usr/lib64/bluecove/2.1.1-SNAPSHOT/libbluecove_x64.so
drwxr-xr-x /usr/share/doc/bluecove-gpl-2.1.1
-rw-r--r-- /usr/share/doc/bluecove-gpl-2.1.1/AUTHORS.txt
-rw-r--r-- /usr/share/doc/bluecove-gpl-2.1.1/LICENSE.txt

> - Build puts the native lib inside the JAR.  This can be changed!
> There is no point to have native libs inside RPM package twice. One in
> jar another in  /usr/lib/bluecove

I was doing that already in the spec file, but that's a bit ugly. It
makes more sense to modify the build.xml file indeed, or make it
somehow optional...

> - I have no preferences where to put jar in the system
> /usr/share/java or /usr/lib(64)/$name/
> I think that it would be nice to have then located together side by side.

I put now the platform independent jar (bluecove) in /usr/share/java
and the bluecove-gpl in /usr/lib<arch>/bluecove/version/ for now.

>  We need to clarify  "BlueCove-gpl" GPLv2 or GPLv3.  I was thinking
> that we should keep the same license as BlueZ. The links on out
> website are incorrect.  We never specified the exact version of GPL we
> used.

Well, I just checked the text in the source files of the projects and
this is what came up. So they are actually specifically licensed under
what I said above, so it seems to be clear :)

Some other (minor) issues that came up while packaging:
- README.txt in bluecove has DOS line endings (fix with: sed --in-
place 's/\r$//' README.txt)
- AUTHORS.txt in bluecove-gpl has DOS line endings (fix with: sed --in-
place 's/\r$//' AUTHORS.txt)
- AUTHORS.txt in bluecove-gpl has wrong permissions (executable, fix
with chmod 0644 AUTHORS.txt)
- device.txt is not UTF-8 (fix with: iconv --from=ISO-8859-1 --
to=UTF-8 device.txt > device.txt.new ; mv device.txt.new device.txt)
- changelog.txt is not UTF-8 (fix with: iconv --from=ISO-8859-1 --
to=UTF-8 changelog.txt > changelog.txt.new ; mv changelog.txt.new
changelog.txt)

Also, the native library is not linked against libc and doesn't have a
soname, apparently this is not really nice as it generates some errors
in the build process of the package. I fixed this by removing the
line:
<arg value="-nodefaultlibs"/>

from bluecove-gpl build.xml and adding the line:
<arg value="-Wl,-soname,${library_name}"/>

Not sure whether ${library_name} is appropriate as a soname, but it
seems to work.

I'm using the SVN snapshot r2706 now.

To make it complete, here the layout of the bluecove package:

drwxr-xr-x /usr/share/doc/bluecove-2.1.1
-rw-r--r-- /usr/share/doc/bluecove-2.1.1/AUTHORS.txt
-rw-r--r-- /usr/share/doc/bluecove-2.1.1/LICENSE.txt
-rw-r--r-- /usr/share/doc/bluecove-2.1.1/README.txt
-rw-r--r-- /usr/share/doc/bluecove-2.1.1/changelog.txt
-rw-r--r-- /usr/share/doc/bluecove-2.1.1/device.txt
-rw-r--r-- /usr/share/doc/bluecove-2.1.1/stacks.txt
-rw-r--r-- /usr/share/doc/bluecove-2.1.1/todo.txt
-rw-r--r-- /usr/share/java/bluecove-2.1.1.jar

Thanks,
François

[Vlad Skarzhevskyy]

The "2.1.1-SNAPSHOT" is not a problem it is a feature!
Now the library is in development! So for version 2.1.1 only
SNAPSHOTs are available.
Once we made a release of 2.1.1 (some time this spring) it would have
a version number without "-SNAPSHOT"

I would suggest to make a packages for 2.1.0 version. Using some
selected files from current svn!

BTW bluecove.jar is also platform defendant and is using native code
on windows and Mac OSX.

If you taking changelog.txt from svn the keep in mind that it is not maintained.
better get the text from this URL
http://code.google.com/p/bluecove/wiki/Changelog

bluecove-bluez was intended to rplace bluecove-gpl!
It was fasted to make bluecove-gpl module then proper implementation
'bluecove-bluez' using D-Bus
bluecove-bluez is still work in progress.

""BlueCove-gpl" GPLv2 or GPLv3."
I'm managing the headers and I probably made a mistake! As I see we
can't link to GPLv2 (bluez) and say that we are GPLv3. So its a
mess....

--
Vlad

[Vlad Skarzhevskyy]

build.xml bluecove-gpl has been updated to allow building jar for RPM
packages without .so.

Run like this: ant -Dbluecove.native.resources.skip=true jar

Keep in mind that if you already have any .so in directory
src/main/resources then they would be added.
Directory src/main/resources is not cleaned during build because on
build server (cruisecontrol) I copy files from different platforms to
one aggregated build. I may need to review this approach.

[Vlad Skarzhevskyy]

Hi François
I need more information understanding why -nodefaultlibs should be remove.
I added an option to build to supply custom linker options:

ant -Dbluecove.native.linker.options=""
linking .o -> libbluecove_x64.so ()
ldd libbluecove_x64.so
linux-vdso.so.1 => (0x00007fffa55ff000)
libbluetooth.so => /usr/lib64/libbluetooth.so (0x000000000031f000)
libc.so.6 => /lib64/libc.so.6 (0x0000000000537000)
/lib64/ld-linux-x86-64.so.2 (0x00000034d1e00000)

and

ant -Dbluecove.native.linker.options=-nodefaultlibs
linking .o -> libbluecove_x64.so (-nodefaultlibs)
ldd libbluecove_x64.so
linux-vdso.so.1 => (0x00007fffcd9ff000)
libbluetooth.so => /usr/lib64/libbluetooth.so (0x000000000031f000)
libc.so.6 => /lib64/libc.so.6 (0x0000000000537000)
/lib64/ld-linux-x86-64.so.2 (0x00000034d1e00000)

Looks like the same results!

As far as I remember I added -nodefaultlibs when our library code was
in C++ ....

On Sat, Jan 17, 2009 at 4:32 PM, F. Kooman <fko...@tuxed.net> wrote:

> Also, the native library is not linked against libc and doesn't have a
> soname, apparently this is not really nice as it generates some errors
> in the build process of the package. I fixed this by removing the
> line:
> <arg value="-nodefaultlibs"/>
>
> from bluecove-gpl build.xml and adding the line:
> <arg value="-Wl,-soname,${library_name}"/>
>
>

> Thanks,
> François

--
Vlad