java.lang.NoClassDefFoundError: org.osgi.service.cm.ConfigurationAdmin not found by org.apache.felix.webconsole [11]

[St Clair Clarke]

Hello,
I am trying to use the Felix Webconsole as instructed in the bnd
tutorial for components at http://www.aqute.biz/Bnd/Components.

The console browser displays allright on Jetty port 8080.

However, when I clicked the "Configuration" tab I get the following
error: Configuration Printer failed: java.lang.NoClassDefFoundError:
org.osgi.service.cm.ConfigurationAdmin not found by
org.apache.felix.webconsole [11]

It also says that: Apache Felix Declarative Service not installed when
the "Declarative Service tab is clicked.

I attempt to resolve the issues by adding the
org.apache.felix.configadmin to the Run requirements of the
org.example.impls bundle in the tutorial but the error remains
unresolved.

Any help would be appreciated.

Thanks

[Neil Bartlett]

This kind of thing is an issue to raise on the Felix list. However I happen to know the answer so I'll answer it here.

There is a bug in the current released version of WebConsole. The manifest is invalid, and it has an optional dependency on package org.osgi.service.cm, but that import should not be optional.

An as aside... in proper OSGi usage we should never see errors like NoClassDefFound or ClassNotFoundException, but using optional or dynamic imports means we can see such errors.

Anyway the solution is to install the OSGi Compendium bundle (osgi.cmpn). This contains the API for ConfigurationAdmin (plus lots of other APIs). Incidentally, adding the org.apache.felix.configadmin bundle didn't work because that is an *implementation* bundle; the resolution error shows that its an API that is missing.

Regards
Neil

[Felix Meschberger]

Hi,

Am 15.12.2011 um 22:14 schrieb Neil Bartlett:

> This kind of thing is an issue to raise on the Felix list. However I happen to know the answer so I'll answer it here.
>
> There is a bug in the current released version of WebConsole. The manifest is invalid, and it has an optional dependency on package org.osgi.service.cm, but that import should not be optional.

The idea of it being optional is that we don't require this for the Web Console proper. The bug is that we don't cope correctly with the API missing.

>
> An as aside... in proper OSGi usage we should never see errors like NoClassDefFound or ClassNotFoundException, but using optional or dynamic imports means we can see such errors.
>
> Anyway the solution is to install the OSGi Compendium bundle (osgi.cmpn). This contains the API for ConfigurationAdmin (plus lots of other APIs). Incidentally, adding the org.apache.felix.configadmin bundle didn't work because that is an *implementation* bundle; the resolution error shows that its an API that is missing.

The Felix Configuration Admin bundle exports the CM API itself. The web console has to be refreshed after the installation for export package to be properly wired, though.

Regards
Felix

[Neil Bartlett]

Thanks for the clarification Felix (I didn't know you were on this list... thanks for joining!)

Wouldn't it be better to use an export-and-import of the org.osgi.service.cm package in Web Console, rather than use an optional import?

Neil

[Felix Meschberger]

Hi,

Am 16.12.2011 um 12:04 schrieb Neil Bartlett:

> Thanks for the clarification Felix (I didn't know you were on this list... thanks for joining!)
>
> Wouldn't it be better to use an export-and-import of the org.osgi.service.cm package in Web Console, rather than use an optional import?

I have thought about that, but I am afraid of version collisions...

Consider a framework where WC is active with an old cm version and then CM is installed providing and requiring a more recent version. WC will happily ignore the Configuration Admin service ;-)

This would be ok for WC, but other consumers might get angry, e.g. DS.

Regards
Felix

[Chris Brind]

I thought that was one of the benefits of osgi? Ie being able to
handle multiple versions of the same package (and bundles?) at the
same time.

Are you worried the other consumer isn't configured correctly to pick
up the version they need?

That's their problem, not yours, right?

Cheers
Chris

Sent from the future using a wormhole and iPhone 6.2

[Neil Bartlett]

Well... yes and no.

There are certain cases where you want to make a provider (or in this case, an optional consumer) of a service work whether or not the API bundle is installed. It's just much more convenient to have a single standalone bundle that you can drop in anywhere.

So long as your bundle both exports AND imports the API package, then it doesn't have any negative effects. When the API bundle is installed, your bundle will import it and the export will be shadowed, so there will still only be one "visible" export of that package. Check out section 3.5.6 of the core spec ("Exporting and Importing a Package") for more details.

Cheers
Neil

On Friday, 16 December 2011 at 11:06, Ferry Huberts wrote:

> On 12/16/2011 12:04 PM, Neil Bartlett wrote:
> > Thanks for the clarification Felix (I didn't know you were on this list... thanks for joining!)
> >
> > Wouldn't it be better to use an export-and-import of the org.osgi.service.cm package in Web Console, rather than use an optional import?
>

> I thought that's what API bundles are for....????
>
> If everybody starts including the APIs in their bundles: :-(
>
>
> Ferry

[Ferry Huberts]

On 12/16/2011 12:12 PM, Neil Bartlett wrote:
> Well... yes and no.
>
> There are certain cases where you want to make a provider (or in this case, an optional consumer) of a service work whether or not the API bundle is installed. It's just much more convenient to have a single standalone bundle that you can drop in anywhere.
>
> So long as your bundle both exports AND imports the API package, then it doesn't have any negative effects. When the API bundle is installed, your bundle will import it and the export will be shadowed, so there will still only be one "visible" export of that package. Check out section 3.5.6 of the core spec ("Exporting and Importing a Package") for more details.

I know, but it obfuscates the dependencies. Apparently WC really does
depend on the API...
What's wrong with a _mandatory_ dependency on an _API_ bundle?

> Cheers
> Neil
>
>
> On Friday, 16 December 2011 at 11:06, Ferry Huberts wrote:
>
>> On 12/16/2011 12:04 PM, Neil Bartlett wrote:
>>> Thanks for the clarification Felix (I didn't know you were on this list... thanks for joining!)
>>>
>>> Wouldn't it be better to use an export-and-import of the org.osgi.service.cm package in Web Console, rather than use an optional import?
>> I thought that's what API bundles are for....????
>>
>> If everybody starts including the APIs in their bundles: :-(
>>
>>
>> Ferry

--
Ferry Huberts

[Neil Bartlett]

Chris,

Multiple versions of packages is *possible* in OSGi, but it's still something that should be avoided wherever possible. The reason is you split the class space, and you get services that can only be seen by a subset of consumers due to compatibility checking on the APIs. However that's not what's happening when you both export and import a package in your bundle. The OSGi framework picks one or the other: it imports the package (i.e. from the API bundle, osgi.cmpn) when it's available; otherwise it exports your "copy" of the package.

This is a common pattern with bundles that provide services. Not so much with bundles that consume services, because usually if the service API is not available then there can't be any providers, so it doesn't matter if your consumer fails to resolve. In the case of the Web Console it is an *optional* consumer of the ConfigAdmin service, so it may be useful to do this.

Then again maybe Web Console should be split into multiple bundles with better internal coherency... but then it would be far less convenient to use. So it's a trade-off.

Regards
Neil

On Friday, 16 December 2011 at 11:11, Chris Brind wrote:

> I thought that was one of the benefits of osgi? Ie being able to
> handle multiple versions of the same package (and bundles?) at the
> same time.
>
> Are you worried the other consumer isn't configured correctly to pick
> up the version they need?
>
> That's their problem, not yours, right?
>
> Cheers
> Chris
>
> Sent from the future using a wormhole and iPhone 6.2
>

[Felix Meschberger]

Hi,

Am 16.12.2011 um 12:11 schrieb Chris Brind:

> I thought that was one of the benefits of osgi? Ie being able to
> handle multiple versions of the same package (and bundles?) at the
> same time.

OSGi can. But users maybe not.

>
> Are you worried the other consumer isn't configured correctly to pick
> up the version they need?
>
> That's their problem, not yours, right?

Yes, and no (at the end, I am not living in an ivory tower and also have to support customers ;-) )

Consider this:

1. install and start WebConsole exporting and re-importing cm v 1.2 (because that's
all the WebConsole needs
2. install and start DS. This imports CM 1.2 from the Web Console bundle, happy.
3. install and start CM exporting and re-importing CM 1.3, becaut that's
implemented.

With step 3 it becomes nasty: Now the CM API is available twice. And all imports after step 3 will bind the CM 1.3 (highest version available). But DS and WebConsole will not be bound there and would have to be re-wired (PackageAdmin.refreshPackages in pre-R4.3).

A consequence of DS and WebConsole not being wired correctly is that WC cannot manage configurations and DS will never get configurations because the ConfigurationAdmin service will not be supplied.

Ok, after a framework restart, everything will also be wired fine, but then ...

Regards
Felix

[Felix Meschberger]

Hi,

Am 16.12.2011 um 12:20 schrieb Neil Bartlett:

> Then again maybe Web Console should be split into multiple bundles with better internal coherency... but then it would be far less convenient to use. So it's a trade-off.

We started doing exactly that and trading less convenience use against better dependency handling.

We all should use deployment helpers like OBR, anyway. So "less convenience" should be becoming relative ;-)

[Marcel Offermans]

On Dec 16, 2011, at 12:22 PM, Felix Meschberger wrote:

Am 16.12.2011 um 12:11 schrieb Chris Brind:

I thought that was one of the benefits of osgi? Ie being able to

handle multiple versions of the same package (and bundles?) at the

same time.

OSGi can. But users maybe not.

I'm sure we're all aware that we actually have a FAQ on this issue:

http://felix.apache.org/site/apache-felix-osgi-faq.html#ApacheFelixOSGiFAQ-Shouldabundleimportitsownexportedpackages%253F

Depending on the outcome of this discussion, we might want to add a few nuances to it?

Are you worried the other consumer isn't configured correctly to pick

up the version they need?

That's their problem, not yours, right?

Yes, and no (at the end, I am not living in an ivory tower and also have to support customers ;-) )

Consider this:

 1. install and start WebConsole exporting and re-importing cm v 1.2 (because that's
     all the WebConsole needs
 2. install and start DS. This imports CM 1.2 from the Web Console bundle, happy.
 3. install and start CM exporting and re-importing CM 1.3, becaut that's
    implemented.

With step 3 it becomes nasty: Now the CM API is available twice. And all imports after step 3 will bind the CM 1.3 (highest version available).

Step 3 introduces two CM's, so you will have a "split class space".

But DS and WebConsole will not be bound there and would have to be re-wired (PackageAdmin.refreshPackages in pre-R4.3).

You can agrue about what DS should do, but let's keep the focus on WebConsole here for a moment.

The real problem is that WebConsole was not designed to actually work with more than one class space simultaneously. So from here on you can start argueing in two directions:

a) WebConsole should start handling more class spaces, using calls like getAllServiceReferences to track multiple CM services and probably providing a UI for each one it found (or a drop down list of class spaces to choose from).

b) WebConsole should somehow be able to be installed multiple times, once for each class space. That would require more work as (up to 4.3) you cannot install a bundle multiple times.

Maybe there's a third, which is to accept the limitation and not fix it at all.

I would personally prefer an option to do a) but if you really need that will depend on your use case.

A consequence of DS and WebConsole not being wired correctly is that WC cannot manage configurations and DS will never get configurations because the ConfigurationAdmin service will not be supplied.

Yes, it simply lives in a different class space, and cannot handle multiple ones. At best, by re-resolving just WebConsole, you can convince it to "move" to a different class space.

Ok, after a framework restart, everything will also be wired fine, but then ...

... we should strive to write code that never requires a restart. :)

Greetings, Marcel

[Marcel Offermans]

On Dec 16, 2011, at 12:22 PM, Felix Meschberger wrote:

> Consider this:
>
> 1. install and start WebConsole exporting and re-importing cm v 1.2 (because that's
> all the WebConsole needs
> 2. install and start DS. This imports CM 1.2 from the Web Console bundle, happy.
> 3. install and start CM exporting and re-importing CM 1.3, becaut that's
> implemented.
>
> With step 3 it becomes nasty: Now the CM API is available twice. And all imports after step 3 will bind the CM 1.3 (highest version available).

As a side note:

The CM API is currently at version 1.3 which suggests that all versions of the API starting with 1.0 have been "backward compatible" upgrades of this API.

You can argue that the CM implementation could also have defined its Import-Package for the CM API as [1.0, 1.4) which would have avoided a split in the class space altogether.

Of course I am assuming that the changes to the API have indeed been backward compatible and that there's nothing in the 1.3 implementation that somehow makes it impossible for that implementation to work with an older API.

Greetings, Marcel

[Felix Meschberger]

Hi,

Am 16.12.2011 um 13:01 schrieb Marcel Offermans:

On Dec 16, 2011, at 12:22 PM, Felix Meschberger wrote:

Am 16.12.2011 um 12:11 schrieb Chris Brind:

I thought that was one of the benefits of osgi? Ie being able to

handle multiple versions of the same package (and bundles?) at the

same time.

OSGi can. But users maybe not.

I'm sure we're all aware that we actually have a FAQ on this issue:

http://felix.apache.org/site/apache-felix-osgi-faq.html#ApacheFelixOSGiFAQ-Shouldabundleimportitsownexportedpackages%253F

Depending on the outcome of this discussion, we might want to add a few nuances to it?

Are you worried the other consumer isn't configured correctly to pick

up the version they need?

That's their problem, not yours, right?

Yes, and no (at the end, I am not living in an ivory tower and also have to support customers ;-) )

Consider this:

 1. install and start WebConsole exporting and re-importing cm v 1.2 (because that's
     all the WebConsole needs
 2. install and start DS. This imports CM 1.2 from the Web Console bundle, happy.
 3. install and start CM exporting and re-importing CM 1.3, becaut that's
    implemented.

With step 3 it becomes nasty: Now the CM API is available twice. And all imports after step 3 will bind the CM 1.3 (highest version available).

Step 3 introduces two CM's, so you will have a "split class space".

Which is not what we generally want (subsystems guys might disagree ;-) )

But DS and WebConsole will not be bound there and would have to be re-wired (PackageAdmin.refreshPackages in pre-R4.3).

You can agrue about what DS should do, but let's keep the focus on WebConsole here for a moment.

The real problem is that WebConsole was not designed to actually work with more than one class space simultaneously. So from here on you can start argueing in two directions:

a) WebConsole should start handling more class spaces, using calls like getAllServiceReferences to track multiple CM services and probably providing a UI for each one it found (or a drop down list of class spaces to choose from).

We still either have to wire to the correct packages or use reflection. Neither is very promising. Other than having two instances of the CM plugin, each bound to a different CM API provider....

b) WebConsole should somehow be able to be installed multiple times, once for each class space. That would require more work as (up to 4.3) you cannot install a bundle multiple times.

Maybe there's a third, which is to accept the limitation and not fix it at all.

I would personally prefer an option to do a) but if you really need that will depend on your use case.

A consequence of DS and WebConsole not being wired correctly is that WC cannot manage configurations and DS will never get configurations because the ConfigurationAdmin service will not be supplied.

Yes, it simply lives in a different class space, and cannot handle multiple ones. At best, by re-resolving just WebConsole, you can convince it to "move" to a different class space.

Yes, which is a manual action; which is why I would like to prevent it from being required and thus prevent split class spaces; particularly unexpected and undesired ones as in this case.

Ok, after a framework restart, everything will also be wired fine, but then ...

... we should strive to write code that never requires a restart. :)

That's what I was hinting at ;-)

Regards

Felix

Greetings, Marcel

[Felix Meschberger]

Hi,

Am 16.12.2011 um 13:10 schrieb Marcel Offermans:

> On Dec 16, 2011, at 12:22 PM, Felix Meschberger wrote:
>
>> Consider this:
>>
>> 1. install and start WebConsole exporting and re-importing cm v 1.2 (because that's
>> all the WebConsole needs
>> 2. install and start DS. This imports CM 1.2 from the Web Console bundle, happy.
>> 3. install and start CM exporting and re-importing CM 1.3, becaut that's
>> implemented.
>>
>> With step 3 it becomes nasty: Now the CM API is available twice. And all imports after step 3 will bind the CM 1.3 (highest version available).
>
> As a side note:
>
> The CM API is currently at version 1.3 which suggests that all versions of the API starting with 1.0 have been "backward compatible" upgrades of this API.
>
> You can argue that the CM implementation could also have defined its Import-Package for the CM API as [1.0, 1.4) which would have avoided a split in the class space altogether.

No, because the 1.4 CM Implementation will typically implement 1.4 and thus import [1.4,1.5) ...

>
> Of course I am assuming that the changes to the API have indeed been backward compatible and that there's nothing in the 1.3 implementation that somehow makes it impossible for that implementation to work with an older API.

Its only backwards compatible with consumers ... and this now enters the semantice versioning space ;-)

Regards
Felix

[Peter Kriens]

There is a HUGE difference between a provider and consumer of an API. A provider of an API is very closely bound to the API since each responsibility of the contract must be provided by it. In the semantic versioning, this is reflected that a provider must import with a fixed minor number, i.e. [1.2,1.3)

A client has an easier life because it cannot use functionality that it is not aware of, this allows APIs to evolved and remain backward compatible with consumers. Lets say that you add additional semantics to a method in an API, the provider must than understand this new meaning but the consumer can be oblivious since it does not use these new semantics.

Therefore, a provider of an API should export and import the API packages but consumers should NEVER do this. If a consumer exports an API it shoots himself in the foot, if it is chosen as the exporter it might no longer be able to talk to a perfectly capable provider.

So I think Felix is correct, optional import is better for a consumer.

Kind regards,

Peter Kriens

[Marcel Offermans]

Let me illustrate my point with a little example:

We start with:

package foo; // version 1.0 of our API
interface Foo {
void doWork();
}

package foo; // version 1.1 of our API with a backward compatible change
interface Foo {
void doWork();
double getProgress();
}

Let's now look at this from a provider perspective, starting with our first implementation, made when 1.0 of the API was released:

class FooImpl implements Foo {
void doWork() { /* do something */ }
}

This bundle might package the API in version 1.0 and have the following headers:

Import-Package: foo;version="[1.0, 1.1)"
Export-Package: foo;version="1.0"

Now the 1.1 API is released and I am creating a new implementation:

class FooImpl implements Foo {
void doWork() { /* do something */ }
double getProgress() { /* return progress */ }
}

You now say (if I interpret your response correctly) that we now must use the following headers:

Import-Package: foo;version="[1.1, 1.2)"
Export-Package: foo;version="1.1"

And I'm argueing that we can just as well use these headers:

Import-Package: foo;version="[1.0, 1.2)"
Export-Package: foo;version="1.1"

Now I did not actually go over all changes to the CM API over time, but if all changes were backward compatible, there is a good chance that the latest implementation will still correctly implement the first API as well.

Sure, in this example, if everybody actually ends up being wired to the foo package in version 1.0, nobody will be able to take advantage of the newly added "getProgress" method, but then again, probably nobody was using it either (if all consumers indicated they were importing [1.0, 2.0) anyway).

Greetings, Marcel

[Felix Meschberger]

Hi,

Am 16.12.2011 um 13:58 schrieb Marcel Offermans:

> You now say (if I interpret your response correctly) that we now must use the following headers:
>
> Import-Package: foo;version="[1.1, 1.2)"
> Export-Package: foo;version="1.1"
>

Yes, you should do that. The impl implements 1.1.

> And I'm argueing that we can just as well use these headers:
>
> Import-Package: foo;version="[1.0, 1.2)"
> Export-Package: foo;version="1.1"
>
> Now I did not actually go over all changes to the CM API over time, but if all changes were backward compatible, there is a good chance that the latest implementation will still correctly implement the first API as well.

This kills some of your clients requiring 1.1 API !

I do not exactly know what a framework is doing with the export here .. But either the package is still exported and usable but the class will not be compatible or it is not exported (I assume this is the case actually) and the client will not have its dependencies resolved and thus fails -- even though the API would "theoretically" be available.

I woud definitely never advise to do something like that.

Regards
Felix

[Marcel Offermans]

On Dec 16, 2011, at 19:47 PM, Felix Meschberger wrote:

> Am 16.12.2011 um 13:58 schrieb Marcel Offermans:
>
>> You now say (if I interpret your response correctly) that we now must use the following headers:
>>
>> Import-Package: foo;version="[1.1, 1.2)"
>> Export-Package: foo;version="1.1"
>>
>
> Yes, you should do that. The impl implements 1.1.
>
>> And I'm argueing that we can just as well use these headers:
>>
>> Import-Package: foo;version="[1.0, 1.2)"
>> Export-Package: foo;version="1.1"
>>
>> Now I did not actually go over all changes to the CM API over time, but if all changes were backward compatible, there is a good chance that the latest implementation will still correctly implement the first API as well.
>
> This kills some of your clients requiring 1.1 API !

True, but that is not the scenario we were discussing here.

If you have clients requiring the 1.1 API, and you started out with an already resolved 1.0 API, you cannot prevent a split.

If you go back to the original scenario we were discussing, by using my suggestion you can actually prevent a split.

> I do not exactly know what a framework is doing with the export here .. But either the package is still exported and usable but the class will not be compatible or it is not exported (I assume this is the case actually) and the client will not have its dependencies resolved and thus fails -- even though the API would "theoretically" be available.

In this case, the framework is doing nothing with the export in the sense that it is not being used (wired to anybody) at all. That's the whole point of the "import what you export" substitutability. Even though you ship your bundle with a copy of some API, that copy might not be used at runtime.

> I woud definitely never advise to do something like that.

Fair enough. If someone favors a single class space over using the latest and greatest API at the expense of a split, I would, in fact this thread has got me thinking about what semantic versioning should actually propose here, because one advice could definitely be to extend the import range if you can indeed compile your 1.1 implementation to the 1.0 API and there were no semantic changes that could not be picked up by the compiler.