Hamcrest packaging and OSGi

[Ketan Padegaonkar]

Hi,

I'm the maintainer of the hamcrest OSGi bundles at eclipse.org, and
wanted to provide inputs to help resolve some of the pain points with
hamcrest.

Here's the situation:
Hamcrest (v1.1) provides a few jars (or bundles in OSGi land) for the
sake of naming them the OSGi way:
- org.hamcrest.core
- org.hamcrest.integration
- org.hamcrest.library

There are others but I'll leave them out of this email for now.

The general convention in the OSGi land to wire up bundles is to use the
'Import-Package' header which basically auto-wires bundles based on
which bundle exports the package.

Things get really funny when two or more bundles export the same
package. This is happening in the case of all the above bundles
containing classes in the org.hamcrest.core namespace therefore causing
confusion and complexities for downstream OSGi packagers and consumers
who use hamcrest.

I understand that this is rather late in the 1.2 development cycle, but
would the development team want to consider making a change so that
org.hamcrest.library would for e.g. export org.hamcrest.core as
org.hamcrest.library instead.

What would be preferred is to rename everything org.hamcrest.* in
org.hamcrest.library as org.hamcrest.library.*. But I believe this would
completely break all clients and hamcrest 1.2 would no longer be
backward compatible.

Thoughts, brickbats ?

-- Ketan
http://eclipse.org/swtbot -- SWT/Eclipse UI Testing

[Ketan Padegaonkar]

To give some context to the conversation:

https://bugs.eclipse.org/bugs/show_bug.cgi?id=268881#c0
https://bugs.eclipse.org/bugs/show_bug.cgi?id=268881#c5
https://bugs.eclipse.org/bugs/show_bug.cgi?id=268881#c8

-- Ketan
http://eclipse.org/swtbot -- SWT/Eclipse UI Testing

[Nat Pryce]

It sounds like a pretty serious limitation (bug even?) in OSGi.
Obviously breaking backward compatibility and so breaking all existing
client code that relies upon Hamcrest is not an acceptable option.
Could you raise this as a bug with the OSGi developers and get them to
fix it?

Alternatively, you could write packaging scripts that repackage the
Hamcrest classes into bundles that work within OSGi's constraints.
This is how Hamcrest packaging is managed by other downstream
distribution mechanisms.

--Nat

2009/4/29 Ketan Padegaonkar <ketanpad...@gmail.com>:

--
http://www.natpryce.com

[Steve Freeman]

I'm not sure I totally understand what the problem is. I think you're
saying that osgi is treating each hamcrest jar as a bundle and that
there are problems with the same package in multiple bundles.

Hamcrest isn't very large, could you put the whole thing into one
bundle?

S.

Steve Freeman
Winner of the Agile Alliance Gordon Pask award 2006

http://www.m3p.co.uk

M3P Limited.
Registered office. 2 Church Street, Burnham, Bucks, SL1 7HZ.
Company registered in England & Wales. Number 03689627

[Ketan Padegaonkar]

Hi,

This is not a bug, but perhaps a limitation. I'll discuss some potential
workarounds in later part of this email.

The case in point here is JUnit 4.x which depends on, and packages
hamcrest-core-1.1. Bundles that request org.hamcrest.core could get
wired to the junit bundle which is not what that bundle expected.

To make hamcrest more modular, we've split hamcrest using something
called as split packages[1] in the OSGi land.

What this lets you do is to declare that there are potentially more than
one bundles that can export the package com.foo.bar.

In case of hamcrest, hamcrest-integration and hamcrest-library need
org.hamcrest(from hamcrest-core, perhaps even other bundles) as a
dependency, and also export org.hamcrest from within themselves.

The added complexity of these cyclic kind of dependencies makes
packaging quite difficult.

The workarounds are to:
a. package hamcrest as a single bundle -- this has the caveat that the
bundle won't load up since it needs dependencies like qdox, easymock and
jmock
b. package hamcrest in a way so that different bundles use completely
different namespaces -- this means that the OSGi bundles are not drop in
replacements, and will need changes to the import statements in the java
code.

Option b. seems to be a nicer way to package hamcrest provided that
upstream also packages it the same way, and ensures that bundles from
multiple vendors are compatible -- this will break all compatibility for
consumers who're using hamcrest and we may need to figure out ways to
get around this by deprecating stuff.

I'm of course open to any other ideas that you may have, and be more
than happy to help in this effort.

[1] -
http://eclipsesource.com/blogs/2008/08/22/tip-split-packages-and-visibility/

-- Ketan

[Nat]

On Apr 30, 5:33 am, Ketan Padegaonkar <ketanpadegaon...@gmail.com>
wrote:

> The case in point here is JUnit 4.x which depends on, and packages
> hamcrest-core-1.1. Bundles that request org.hamcrest.core could get
> wired to the junit bundle which is not what that bundle expected.

So the problem seems to be with the OSGi packaging of JUnit.

> In case of hamcrest, hamcrest-integration and hamcrest-library need
> org.hamcrest(from hamcrest-core, perhaps even other bundles) as a
> dependency, and also export org.hamcrest from within themselves.
>
> The added complexity of these cyclic kind of dependencies makes
> packaging quite difficult.

There are no cyclic dependencies between these JARs except in the
model that OSGi is trying to force upon them. Hamcrest does not seal
the org.hamcrest package. Unless a package is sealed (by a JAR
manifest) there is no relationship between JARs and packages --
classes in the package can reside in more than one JAR. See
http://java.sun.com/docs/books/tutorial/deployment/jar/sealman.html
for more info. If OSGi does not support this, then OSGi is broken.
Raise a bug with OSGi and help them fix it.

> The workarounds are to:
> a. package hamcrest as a single bundle -- this has the caveat that the
> bundle won't load up since it needs dependencies like qdox, easymock and
> jmock

That does not sound like a workaround.

> b. package hamcrest in a way so that different bundles use completely
> different namespaces -- this means that the OSGi bundles are not drop in
> replacements, and will need changes to the import statements in the java
> code.
>
> Option b. seems to be a nicer way to package hamcrest provided that
> upstream also packages it the same way, and ensures that bundles from
> multiple vendors are compatible -- this will break all compatibility for
> consumers who're using hamcrest and we may need to figure out ways to
> get around this by deprecating stuff.

That is definitely NOT a nicer way to package hamcrest then. Lots of
projects rely on hamcrest. Changing its package structure is not
acceptable.

> I'm of course open to any other ideas that you may have, and be more
> than happy to help in this effort.

I suggest fixing OSGi since it falls over when faced with a basic
feature of the Java language.

--Nat

[Nat Pryce]

2009/4/30 Nat <nat....@gmail.com>:

>
> On Apr 30, 5:33 am, Ketan Padegaonkar <ketanpadegaon...@gmail.com>
> wrote:
>> The case in point here is JUnit 4.x which depends on, and packages
>> hamcrest-core-1.1. Bundles that request org.hamcrest.core could get
>> wired to the junit bundle which is not what that bundle expected.
>
> So the problem seems to be with the OSGi packaging of JUnit.

Sorry for the grumpy tone of my last mail.

JUnit distribute Hamcrest as part of junit-4.x.jar just as a
convenience, so that programmers can put the JAR on their classpath
and not worry about other dependencies. OSGi packaging is definitely
outside that use case, so it sounds like packaging junit-4.x.jar as an
OSGi bundle is the wrong thing to do.

JUnit also distribute junit-deps-4.x.jar. This does not contain
Hamcrest in it. It sounds like the problem can be addressed by
building the JUnit OSGi bundle from junit-deps-4.x.jar and making that
depend on the hamcrest OSGi bundles.

--Nat

[ste...@kestle.net]

Ok - just for context, I raised http://code.google.com/p/hamcrest/issues/detail?id=53
(wrong login, but hey).

This OSGi "problem" has very little to do with hamcrest.
* Junit (as has been stated) has no business creating an OSGi bundle
with org.hamcrest in it. Period.
If that's what you want, fix junit (and any mock frameworks that may
conviniently import it). Also, it's far more important to get Hamcrest
into OSGi in the way that makes sense for it than Junit - how many
people deploy junit into running code? It's certainly smaller than
those that want hamcrest.
* Class dependencies: In my experience (with our partially modified
OSGi/equinox), there's no issue with dependent classes. If you want
to import other packages/bundles, and they're not there, you'll get
the normal ClassPath issues when you attempt to use the classes. Where
problems arise is when you:
a) declare services (which hamcrest does not, as a utility package)
b) another bundle relies on a static reference to the declarative
service (which would stop it from loading).
Can you let me know what OSGi platform requires every possible class
to be included on startup so I can steer clear of it please?

I wouldn't see a problem with all the (split vs combined) bundles
having their OSGi declarations set, and leave the choice of what's
used to the user of said bundles.

[David Saff]

On Sun, May 3, 2009 at 10:30 PM, ste...@kestle.net <ste...@kestle.net> wrote:
>
> Ok - just for context, I raised http://code.google.com/p/hamcrest/issues/detail?id=53
> (wrong login, but hey).
>
> This OSGi "problem" has very little to do with hamcrest.
> * Junit (as has been stated) has no business creating an OSGi bundle
> with org.hamcrest in it. Period.

"JUnit" is not in the business of creating OSGi bundles at all. Let
me know if this issue needs clarification.

David Saff

[Nat Pryce]

Neither is Hamcrest.

Whoever us responsible for creating the Junit and hamcrest OSGi
bundles need to get together and sort this out. And raise issues with
the OSGi project if it turns out they are hampered by limitations of
OSGi itself.

The teams that package and distribute the two libraries by other means
have not had this much trouble -- APT, RPM, even Maven. Why don't you
OSGi folks look at what they've done.

--Nat

--
http://www.natpryce.com

[Ketan Padegaonkar]

Wanted to drop in a note to mention that we packaged it using a middle
ground[1]. This is not an issue with junit, but the way we packaged
junit available from upstream without putting too much thought into
the fact that there are other parts to hamcrest :)

With hamcrest split packages required by me (I'll put up some content
on the interesting usage of hamcrest page) things got a bit complex
because a bundle cannot be an exporter and an importer of a package at
the same time when the it gets resolved.

What makes OSGi special (read complex) is the fact that it packages
java packages which are versioned, and have dependencies on either
other bundles or packages, this is not the case with jars packaged as
RPMs or APT, since it is still just a plain simple jar that's added to
the amalgamation of classpath.

As of now the issue is closed and we'll continue to package hamcrest
the way we're currently packaging (split packages + one full package
that exports them all), of course there's a full hamcrest package
available from the springsource repository if anybody is interested.

[1] - https://bugs.eclipse.org/bugs/show_bug.cgi?id=271348#c8

-- Ketan

--
Ketan Padegaonkar
I blog... therefore I am... http://ketan.padegaonkar.name
http://swtbot.org/ - a functional testing tool for SWT/Eclipse