Dependency quality: xpp3_min + xmlpull vs xpp3 (RPM, OSGi, JBoss modules, Jigsaw, ...)

[Geoffrey De Smet]

Hi guys,

If I look at the maven dependency tree of XStream, I get this (same for
1.4.9):

[INFO] +- com.thoughtworks.xstream:xstream:jar:1.4.7:compile
[INFO] | +- xmlpull:xmlpull:jar:1.1.3.1:compile
[INFO] | \- xpp3:xpp3_min:jar:1.1.4c:compile

xstream is high quality, active, well maintained jar with a maven build,
sources in Git.
xmlpull is a inactive project with a ANT build (or worse), sources in CVS.
xmlpull contains just 4 classes.

xmlpull is a PITA.
The lack of a maven or gradle build makes it hard to build from source
for RPM's.
The lack of manifest metadata makes it a risk (= an exception) for OSGi,
JBoss modules, Java 9 (Jigsaw), ...
For example, in Java 9 it will never be in the module path in it's
current state.


I would like to propose 3 ways to improve the situation (to Jorg and the
community):

Proposal A) Replace the xmlpull and xpp3_min dependency with the xpp3
dependency.
The xpp3 dependency embeds the 4 xmlpull classes.

Proposal B) Embed the 4 xmlpull and the 3 xpp3_min classes in xstream.
To avoid split packages (OSGi req), they should probably be
subnamespaced into
package com.thoughtworks.xstream.org.xmlpull...

Proposal C) Revive xmlpull: migrate it to github and mavenize it.



Legal implications (I am not a lawyer, this is not legal advice)
"The XmlPull API is in public domain in hope that it will be embraced by
Java developers (conformance tests are under LGPL license to minimize
risk of incompatible implementations)."
So embedding the non-test classes is ok (just keep the original legal
header),
it will not affect xstream's license.
But we can't copy the tests as that would impact the xstream license -
unless we get the original authors to agree on re-license the tests
under a BSD license.

What do you think?


wkr,
Geoffrey De Smet
OptaPlanner

[Jörg Schaible]

Hi Geoffry

Geoffrey De Smet wrote:

> Hi guys,
>
> If I look at the maven dependency tree of XStream, I get this (same for
> 1.4.9):
>
> [INFO] +- com.thoughtworks.xstream:xstream:jar:1.4.7:compile
> [INFO] | +- xmlpull:xmlpull:jar:1.1.3.1:compile
> [INFO] | \- xpp3:xpp3_min:jar:1.1.4c:compile
>
> xstream is high quality, active, well maintained jar with a maven build,
> sources in Git.
> xmlpull is a inactive project with a ANT build (or worse), sources in CVS.
> xmlpull contains just 4 classes.

That's the official implementation.

> xmlpull is a PITA.
> The lack of a maven or gradle build makes it hard to build from source
> for RPM's.

It does not have to change, it just runs.

BTW: Ever tried to find the source repository for Xpp3?

> The lack of manifest metadata makes it a risk (= an exception) for OSGi,
> JBoss modules, Java 9 (Jigsaw), ...
> For example, in Java 9 it will never be in the module path in it's
> current state.

We'll see, what we have to do about it with Jigsaw.

> I would like to propose 3 ways to improve the situation (to Jorg and the
> community):
>
> Proposal A) Replace the xmlpull and xpp3_min dependency with the xpp3
> dependency.
> The xpp3 dependency embeds the 4 xmlpull classes.

No. Xpp3 is one possible implementation, kXML2 is the other one and a
derived one is e.g. default on Android. One reason for selecting xpp3_min as
default is exactly because the XmlPullParser API is *not* embedded. XStream
already tried to switch to kXML2 years ago and failed badly:

http://x-stream.github.io/jira/659/

In XStream 1.3.x Xpp3 was the only supported XmlPullParser, this was
explicitly changed for XStream 1.4.

> Proposal B) Embed the 4 xmlpull and the 3 xpp3_min classes in xstream.
> To avoid split packages (OSGi req), they should probably be
> subnamespaced into
> package com.thoughtworks.xstream.org.xmlpull...

Never. It is on purpose that XStream uses the factory of the XPP API. It is
the official method to select between the different XPP implementations.
Embedding the API under a different package does not make sense. In Android
you simply exclude both dependencies and use nevertheless the XPP factory.

> Proposal C) Revive xmlpull: migrate it to github and mavenize it.

I don't think you will ever get something into Maven central using
xmlpull:xmlpull. And using a different G:A will make the situation worse,
because you'll add another library containing classes with the same name.

> Legal implications (I am not a lawyer, this is not legal advice)
> "The XmlPull API is in public domain in hope that it will be embraced by
> Java developers (conformance tests are under LGPL license to minimize
> risk of incompatible implementations)."
> So embedding the non-test classes is ok (just keep the original legal
> header),
> it will not affect xstream's license.
> But we can't copy the tests as that would impact the xstream license -
> unless we get the original authors to agree on re-license the tests
> under a BSD license.
>
> What do you think?

If you don't like the XStream default, why don't you simply exclude the two
dependencies and add the one you like to use (e.g. xpp3)? Or, if you only
don't like the XPP API, then simply use directly the Xpp3Driver and exclude
xmlpull.

Cheers,
Jörg

[Geoffrey De Smet]

> BTW: Ever tried to find the source repository for Xpp3?

Ow god :)
And we (= optaplanner) have that thing in our dependency tree. I feel
dirty now ;)

> If you don't like the XStream default, why don't you simply exclude the two
> dependencies and add the one you like to use (e.g. xpp3)?

Because we're an (open source) library.
If someone depends on us and on another project that depends on xstream,
I fear trouble.

In any case, xpp3 is bloated (unlike xpp3_min).
Even worse, it's an uber jar (a jar that merges the content of other jars).
- So it contains split packages, which is a blocker for OSGi support
(yes OSGi sucks, but we need to support it anyway).
- So it's potential trouble in maven/gradle land (if a dep tree has both
xpp3 and xpp3_min in it's classpath).

>> Proposal B) Embed the 4 xmlpull and the 3 xpp3_min classes in xstream.

> Never. It is on purpose that XStream uses the factory of the XPP API.

Agreed that proposal A) and B) are not in the best interest of XStream
parser's independent nature.

>> Proposal C) Revive xmlpull: migrate it to github and mavenize it.
> I don't think you will ever get something into Maven central using
> xmlpull:xmlpull.

Challenge accepted :)

If I can in touch with the original authors, and they don't want their
projects to die,
I am sure we can get this mess sorted out.

Basically:
- migrate the sources to github
- mavenize the build
- release the next hotfix version.

> And using a different G:A will make the situation worse,
> because you'll add another library containing classes with the same name.

Agreed. The only option in that case is to fork xmlpull and xpp3, give
them a new name and each class a new namespace, to avoid a classpath mess.

Wkr,
Geoffrey

[Jörg Schaible]

Hi Geoffry,

Geoffrey De Smet wrote:

>> BTW: Ever tried to find the source repository for Xpp3?
> Ow god :)
> And we (= optaplanner) have that thing in our dependency tree. I feel
> dirty now ;)

Hehe. AFAICS the current code base seems located here:
https://sourceforge.net/p/ogce/svn/HEAD/tree/extreme-repos/

They have even newer releases, but unfortunately there is no longer any
xpp3_min artifact.

>> If you don't like the XStream default, why don't you simply exclude the
>> two dependencies and add the one you like to use (e.g. xpp3)?
> Because we're an (open source) library.
> If someone depends on us and on another project that depends on xstream,
> I fear trouble.

Why? You can simply select the driver of your own choice (e.g. StAX
integrated in the JDK) and exclude xpp3/xmlpull for your own artifacts.

> In any case, xpp3 is bloated (unlike xpp3_min).
> Even worse, it's an uber jar (a jar that merges the content of other
> jars). - So it contains split packages, which is a blocker for OSGi
> support (yes OSGi sucks, but we need to support it anyway).
> - So it's potential trouble in maven/gradle land (if a dep tree has both
> xpp3 and xpp3_min in it's classpath).

That's the reason for xpp3_min usage

>>> Proposal B) Embed the 4 xmlpull and the 3 xpp3_min classes in xstream.
>> Never. It is on purpose that XStream uses the factory of the XPP API.
> Agreed that proposal A) and B) are not in the best interest of XStream
> parser's independent nature.
>
>>> Proposal C) Revive xmlpull: migrate it to github and mavenize it.
>> I don't think you will ever get something into Maven central using
>> xmlpull:xmlpull.
> Challenge accepted :)
>
> If I can in touch with the original authors, and they don't want their
> projects to die,
> I am sure we can get this mess sorted out.

AFAICS, xmlpull and xpp3 were both developed by the same team.

> Basically:
> - migrate the sources to github
> - mavenize the build
> - release the next hotfix version.
>> And using a different G:A will make the situation worse,
>> because you'll add another library containing classes with the same name.
> Agreed. The only option in that case is to fork xmlpull and xpp3, give
> them a new name and each class a new namespace, to avoid a classpath mess.

Or use StAX and drop the xpp3/xmlpull dependency.

Cheers,
Jörg

[Geoffrey De Smet]

Update:
The original XMLpull and XPP3 authors (Stefan and Aleksander) are ok
with moving their code to GitHub.
The current plan is to migrate it from CVS to GitHub (probably by
Aleksander) and then to mavenize it (probably by me).

Once those libs have good metadata and they are buildable from source,
there should be no more reason to avoid them, au contraire
(their code itself has stand the test of time, which means a lot).

[Jörg Schaible]

Hi Geoffrey,

Geoffrey De Smet wrote:

> Update:
> The original XMLpull and XPP3 authors (Stefan and Aleksander) are ok
> with moving their code to GitHub.

OK, fine.

> The current plan is to migrate it from CVS to GitHub (probably by
> Aleksander) and then to mavenize it (probably by me).

Hope we'll have a xpp3_min again.

> Once those libs have good metadata and they are buildable from source,
> there should be no more reason to avoid them, au contraire
> (their code itself has stand the test of time, which means a lot).

Maybe I can then raise an issue for a parsing error with Xpp3. I have it in
my bag for years but no place for a report ;-)

Cheers,
Jörg

[Geoffrey De Smet]

The new blessed git repo for XPP3 is:
https://github.com/aslom/xpp3

The new blessed git repo for XMLPull is:
https://github.com/aslom/xmlpull-api-v1

Note that the old blessed CVS repo's are offline for years.


I 've send in a PR to mavenize XMLPull:
https://github.com/aslom/xmlpull-api-v1/pull/1
Once, that's merged (+1 votes welcome!), I 'll also send in a PR to
mavenize XPP3.


HTH

[Jörg Schaible]

Hi Geoffrey,

Geoffrey De Smet wrote:

> The new blessed git repo for XPP3 is:
> https://github.com/aslom/xpp3
>
> The new blessed git repo for XMLPull is:
> https://github.com/aslom/xmlpull-api-v1
>
> Note that the old blessed CVS repo's are offline for years.
>
>
> I 've send in a PR to mavenize XMLPull:
> https://github.com/aslom/xmlpull-api-v1/pull/1
> Once, that's merged (+1 votes welcome!), I 'll also send in a PR to
> mavenize XPP3.

I already created the first issue ;-)

Cheers,
Jörg

[Geoffrey De Smet]

Great :) I doubt if aslom has the intention to fixes issues and do
releases.
But if I can convince him to give me management rights,
I 'll mavenize it and push forward to do a new release (although I won't
fix issues myself, I 'll get PR's merged and fixes released).

[Jörg Schaible]

Hi Geoffrey,

Geoffrey De Smet wrote:

> Great :) I doubt if aslom has the intention to fixes issues and do
> releases.

Nevertheless, it is better to have an open issue there than to keep this for
another 3 years in my inbox :D

> But if I can convince him to give me management rights,
> I 'll mavenize it and push forward to do a new release (although I won't
> fix issues myself, I 'll get PR's merged and fixes released).

Fine.

Cheers,
Jörg

[Geoffrey De Smet]

On 18/05/16 10:45, Jörg Schaible wrote:
> Hi Geoffrey,
>
> Geoffrey De Smet wrote:
>
>> Great :) I doubt if aslom has the intention to fixes issues and do
>> releases.
> Nevertheless, it is better to have an open issue there than to keep this for
> another 3 years in my inbox :D

agreed :)