bndtools native lib resolving problem

[Arvid]

Hello,

 

I have a problem to implement JOGL!  

I think, the problem based on the natives, because all tests inside of my bundle with the classes 

that are referencing the JOGL-classes are running! I read a lot of posts and tutorials about this topic. 

But nothing helps me, because I cannot resolve the JOGL classes outside oft he bundle.

I reference the bundles from io7m. (com.io7m.bundles.org.jogamp.jogl/jocl/joal; version=2.3.2)

They are probably OSGI bundles. They have native libs e.g. in a „natives/windows-amd64/“ path.

(built with "https://github.com/io7m/org.jogamp" project)

First, I got:

Unable to resolve <<INITIAL>> version=null:

   missing requirement com.xy.application

->  Unable to resolve com.xy.application version=1.0.0.201711241047:

   missing requirement  com.jogamp.opengl.awt; version=[2.3.0,3.0.0)

->  Unable to resolve com.io7m.bundles.org.jogamp.jogl version=2.3.2:

   missing requirement |

   (&(osgi.native.osname=Linux)(osgi.native.processor=x86-64))

   (&(osgi.native.osname=Linux)(osgi.native.processor=x86-64))

…

After I played with the backlist and and the „-runsystemcapabilities: ${native_capability}“ command, I get (every time???):

Unable to resolve <<INITIAL>> version=null:

   missing requirement  com.xy.application

->  Unable to resolve com.xy.application version=1.0.0.201711241047:

   missing requirement com.jogamp.common.util]

 

BUT! My big problem is, that the resolving process is a big black box for me!!!

(That's the reason why I post it in the bndtools forum and not in a jogl forum)

How can I get a better look on that what happens at the resolving process?

Are the „activate“  methods of the bundles called by resolving?

So that I can include something like my own native lib resolver?

Can someone helps me???

Kind regards

Arvid

 

[Peter Kriens]

Did you read http://enroute.osgi.org/appnotes/resolving.html ?

Kid regards,

Peter Kriens

--
You received this message because you are subscribed to the Google Groups "bndtools-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bndtools-user...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Timothy Ward]

It sounds as though you correctly identified that you need to have a native capability when resolving to support native code (basically you’re telling it to use your local system’s native attributes). This means that the resolver can validate that the native code will work on your system.

The next step is complaining that no bundle provides com.jogamp.common.util which I also can’t find on google. There must be a bundle somewhere which has it, and if you add that to your repository then it should be able to resolve.

Regards,

Tim

[Mark Raynsford]

On 2017-11-28T09:56:33 +0000

Timothy Ward <tim....@paremus.com> wrote:

> It sounds as though you correctly identified that you need to have a native capability when resolving to support native code (basically you’re telling it to use your local system’s native attributes). This means that the resolver can validate that the native code will work on your system.
>
> The next step is complaining that no bundle provides com.jogamp.common.util which I also can’t find on google. There must be a bundle somewhere which has it, and if you add that to your repository then it should be able to resolve.
>

Hello!

I'm the author of these bundles, although I admit I haven't ended up
using them very much and they've only had minimal testing (I check
whether or not https://github.com/io7m/org.jogamp.example runs and
assume the bundles work if it does).

The com.jogamp.common.util package is part of GlueGen[0] but due to
classloading issues with JOGL (lots of assumptions in the code about
how ClassLoaders are arranged), the GlueGen jar contents are included
directly by each bundle.

I'm at a loss as to why the package would be missing...

[0] http://jogamp.org/gluegen/www/

--
Mark Raynsford | http://www.io7m.com

[Arvid]

First of all thank you for your answers!

@ Peter: No, I didn't know this site. I haven't found it with google :-(

 

I have to admit that I have little experience with OSGI and even less with classloading issues and incorporating native libraries.

My background is that I want to bring a monolithic Eclipse RCP project into a new (more modern) form with OSGI (web frontend instead of RCP).

The division into services has actually worked well, apart from some understanding problems on my part.

The package "com.jogamp.common.util" was referenced by one test resolver, without it comes again the same error with the missing natives!

I have also used the linked JOGLExample class for a mini application and used it with the io7m bundles as in my big project.

Here is the error (same):

Unable to resolve <<INITIAL>> version=null:

   missing requirement com.perzeption.causetree.jogltest.application 

->  Unable to resolve com.perzeption.causetree.jogltest.application version=1.0.0.201711292042:

   missing requirement com.jogamp.opengl; version=[2.3.0,3.0.0) 

->  Unable to resolve com.io7m.bundles.org.jogamp.jogl version=2.3.2:

   missing requirement |

   (&(osgi.native.osname=Linux)(osgi.native.processor=x86-64))

   (&(osgi.native.osname=Linux)(osgi.native.processor=x86-64))

   (&(osgi.native.osname=Linux) ...

   

   

What I know or mine to know:

1) Mutually referencing native libraries are a bigger problem in Linux as in Windows. I think that's not the case here. Correct?

2) Normally, the native files are unpacked into a temporary directory via a gluegen process. Correct?

3) There is a problem with OSGI! This native files are not found, because each bundle has its own class loader. Basically?

4) In my understanding, the io7m bundles are full fledged osgi bundles, Correct? 

5) What about:  "-runsystemcapabilities: ${native_capability}" Is this correct?

   Is this macro automatically set by the system or is it just a variable that I have to define myself?

   

And one last question: Is the whole story with OSGI (bndtools) and JOGL in itself a problematic thing, 

or am I just having an understanding problem. How did others solve this successfully?

I hope we can further narrow the problem together!

Kind regards, Arvid

[Paul F Fraser]

Hi Arvid,

No idea if it is related to your problem with JOGL, but a working example of an OSGi bundle using native libs can be found at
https://github.com/knowhowlab/org.knowhowlab.osgi.sigar

Paul Fraser

[Timothy Ward]

Hi Arvid,

->  Unable to resolve com.io7m.bundles.org.jogamp.jogl version=2.3.2:

   missing requirement |

   (&(osgi.native.osname=Linux)(osgi.native.processor=x86-64))

   (&(osgi.native.osname=Linux)(osgi.native.processor=x86-64))

   (&(osgi.native.osname=Linux) …

As this message is truncated I can’t tell you which it’s failing. What it shows so far is that the bundle requires Linux x86_64, so if you’re not running on Linux with an x86_64 processor then it won’t work.

What I know or mine to know:

1) Mutually referencing native libraries are a bigger problem in Linux as in Windows. I think that's not the case here. Correct?

I’m not an expert at native code, but the native libraries here are supplied in the OSGi bundle you’re deploying, and should be the ones that you end up using. There may be issues if you need to link to other libraries that aren’t in the bundle.

2) Normally, the native files are unpacked into a temporary directory via a gluegen process. Correct?

In OSGi this is an opaque process. All that you know is that the OSGi framework will make the appropriate binaries available to your bundle based on the platform that  you’re running on and the Bundle-NativeCode header

3) There is a problem with OSGI! This native files are not found, because each bundle has its own class loader. Basically?

Native code has been working in OSGi for a long time and for a lot of people. The main thing to be aware of is that only the bundle providing the native code has access to it. You should then expose the features that you want from this bundle using OSGi services.

4) In my understanding, the io7m bundles are full fledged osgi bundles, Correct? 

They contain enough metadata to make it clear that the author intended them to be full-fledged OSGi bundles. Unfortunately that’s not a guarantee that they are tested in OSGi, or have been designed with OSGi in mind.

5) What about:  "-runsystemcapabilities: ${native_capability}" Is this correct?

   Is this macro automatically set by the system or is it just a variable that I have to define myself?

This macro automatically adds the capabilities of the *current* system when resolving. That means that if you’re running on Linux it will resolve for Linux, if you’re on Windows it will resolve for Windows etc.

And one last question: Is the whole story with OSGI (bndtools) and JOGL in itself a problematic thing, 

or am I just having an understanding problem. How did others solve this successfully?

I’m certain that this is a tractable problem. It may simply be that you’re the first person to try doing this in a real system.

Tim

[Mark Raynsford]

On 2017-11-29T13:43:54 -0800

Arvid <arvid.g...@gmail.com> wrote:
>
> 1) Mutually referencing native libraries are a bigger problem in Linux as
> in Windows. I think that's not the case here. Correct?
>
> 2) Normally, the native files are unpacked into a temporary directory via a
> gluegen process. Correct?
>
> 3) There is a problem with OSGI! This native files are not found, because
> each bundle has its own class loader. Basically?
>
> 4) In my understanding, the io7m bundles are full fledged osgi bundles,
> Correct?
>
> 5) What about: "-runsystemcapabilities: ${native_capability}" Is this
> correct?
> Is this macro automatically set by the system or is it just a variable
> that I have to define myself?
>
> And one last question: Is the whole story with OSGI (bndtools) and JOGL in
> itself a problematic thing,
> or am I just having an understanding problem. How did others solve this
> successfully?

To try to clarify things:

JOGL and GlueGen make assumptions about classloaders (such as "I can
retrieve the parent of any classloader and assume that the result is
the system classloader and can resolve any class") that don't hold
under OSGi implementations. We got around this problem by packing all
of the classes and native libraries into a single bundle. That way,
there's effectively only one classloader involved for anything JOGL
cares about.

The org.jogamp.* bundles are intended to be fully-fledged OSGi bundles,
and I've tested them against the org.jogamp.example [0] code to test
whether they work or not. I've received positive results on Linux, but
I've not tested on other platforms. It's possible that there's a
problem with the bundle metadata that causes the bundles not to work on
other platforms.

[0] https://github.com/io7m/org.jogamp.example

[Arvid]

Hello,

I'm so happy! I have it!

You all helped me understand the problem better!!!

But after Mark's post, I quickly came up with the problem:

I work with Windows 10 and the system property "os.name" is also "Windows 10"! 

I have read about a similar case in a forum but didn't pay attention to it at this time.

I changed it, and it works! 

Thank you very much!!!

Kind regards

Arvid

 

[Mark Raynsford]

On 2017-12-01T08:50:32 -0800

Arvid <arvid.g...@gmail.com> wrote:

> Hello,
>
> I'm so happy! I have it!
>
> You all helped me understand the problem better!!!
>
> But after Mark's post, I quickly came up with the problem:
> I work with Windows 10 and the system property "os.name" is also "Windows
> 10"!
> I have read about a similar case in a forum but didn't pay attention to it
> at this time.

That sounds like a bug in the bundle metadata.

https://github.com/io7m/org.jogamp/blob/master/org.jogamp.jogl/pom.xml#L377

What is the correct way to indicate that a native library works on "any
version of Windows"?

[Mark Raynsford]

On 2017-12-01T17:47:40 +0000

Mark Raynsford <list+bndt...@io7m.com> wrote:
>
> What is the correct way to indicate that a native library works on "any
> version of Windows"?

It looks like I want "Win32" according to table 4.3 in the spec.

[Arvid]

Hello Mark,

I think, it need an extra entry for every windows version! 

It looks like it is a known problem, because the value comes from Windows!(?)

Nevertheless, many thanks for your bundles!!!

Kind regards

Arvid

[Mark Raynsford]

On 2017-12-04T02:16:19 -0800

Arvid <arvid.g...@gmail.com> wrote:

> Hello Mark,
>
> I think, it need an extra entry for every windows version!
> It looks like it is a known problem, because the value comes from
> Windows!(?)

If you have a look at page 101 of the OSGi Core spec, it looks as
though it's safe to use "Win32" as all of the Windows versions except
for "WindowsServer2008" alias to "Win32".

[Arvid]

So if I understand that correctly, then e.g. "Win32/x86_64" would be a correct setting for Windows 10 / 64bit!?

[Mark Raynsford]

On 2017-12-05T03:14:22 -0800

Arvid <arvid.g...@gmail.com> wrote:

> So if I understand that correctly, then e.g. "Win32/x86_64" would be a
> correct setting for Windows 10 / 64bit!?

Er, to be clear: This is something I need to fix in the bundles. You
should never have to set a custom osname yourself. I will get to it
soon.

Win32/x86_64 should be correct.

[Arvid]

I know :-)

It was just to make sure I got it right.

[Arvid]

I open a new topic at the JOGL-Forum (http://jogamp.org/forum.html // Aviatrix3D JOGL Problem)!