Need to understand basics of mirroring Maven artifacts in p2 repo

[dkarr]

I work on an Eclipse plugin application that needs to get some artifacts (only four) that I would normally get from a Maven repository.  I'm now understanding that that's basically not possible.  I have to get those Maven artifacts transformed into OSGi bundles in a p2 repository.

I work with a team that maintains a Nexus repo for Maven artifacts, and the sysadmin is working on setting this "p2 mirror" up for me.  I've used Nexus as a user, but I don't administrate it.  I'd just like to understand some of the things that need to happen here.

He first did a simple step of creating a simple p2 mirror to our existing Maven artifacts and had me test that.  Unsurprisingly (now), when I specified that in my target platform in Eclipse, it couldn't see any artifacts.  I'm guessing that's because none of the artifacts there were OSGi bundles.

The sysadmin found an article titled "BLOG-199 Nexus OSGi Experimental Features - P2 Repository Plugin".  This seems to have some useful information, although I found that several of the links and URLs in this article are out of date or invalid.  About the only one that worked was the link to the "nexus-p2-repository-plugin" in the public Sonatype nexus repository.  The URL for the public Sonatype p2 Maven Central mirror was invalid, along with the various github links.

I believe what we eventually have to achieve is to have a repo with both "plain" Maven artifacts and OSGi bundled versions of those same artifacts, with slightly different names, of course.  The article referred to earlier mentions the "Nexus Bundle Maker" plugin and the "nexus-p2-repository-plugin", which seem to be the same thing.  It appears this is what does the transformation of a plain artifact to an OSGi bundle.

It seems likely to me that this article is old, and what it describes as experimental is likely already integrated into the Nexus instance that we use, so we likely don't have to install any "experimental" code.  Can someone confirm that?

[Shahim Essaid]

Hi David,

I'm not part of the Nexus team but I thought I can share what I know in case it is useful.

There is no need to create two artifacts. You just need to deploy Maven artifacts that have the necessary OSGi/Eclipse metadata and Nexus will recognize this make the same artifacts available in the generated P2 repository. How you do this depends on your build. I use Tycho to take care of this but there are other options. The P2 integration is now included as part of the Nexus release and you don't need any additional plugins. If you are working with an older Nexus version you might need to install the two plugins. It's been some time since I looked at the documentation but the last time I read it it described all the steps needed to enable a P2 repository based on an already hosted/mirrored Maven repository but I usually had to lookup the documentation for installing the P2 plugins elsewhere before the plugins were part of the release.

Keep in mind that in Eclipse you will not see plain Eclipse/OSGi bundles when you point at such a P2 repository. Eclipse's views only show you P2 features. You either have to add features to your released bundles (I use Tycho for my builds and this makes it really easy to do), or the other workaround is to mirror the P2 repository (as needed) to your local machine and then use the "folder" option to define target platforms. When you use the "folder" option you will see the individual bundles and you won't need features. But relying on features is the better option if you are mostly developing the Eclipse way and want to take advantage of the Eclipse UIs and the P2 APIs.  There are a couple of pages on the Eclipse wiki that show how to use the command line to mirror P2 repositories and the Tycho plugins also have Maven goals to do this. I usually use the Tycho goals if I need this but I try to use features as much as possible.

Best,

Shahim

--
You received this message because you are subscribed to the Google Groups "Nexus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nexus-users...@glists.sonatype.com.
To post to this group, send email to nexus...@glists.sonatype.com.
To view this discussion on the web visit https://groups.google.com/a/glists.sonatype.com/d/msgid/nexus-users/893b3fc6-285b-494b-8ac7-63508fefefb2%40glists.sonatype.com.
For more options, visit https://groups.google.com/a/glists.sonatype.com/d/optout.

[Manfred Moser]

Just for reference in terms of installation... the P2 chapter of the
Nexus Repository Manager documentation can be found at
http://books.sonatype.com/nexus-book/reference/p2.html

Manfred

> https://groups.google.com/a/glists.sonatype.com/d/msgid/nexus-users/CALEEv-S3rUrLahuW7gQhvg3JjSg%2BqqC9LnH%2BGwWULS5NpXHPEA%40mail.gmail.com.

[David M. Karr]

On 03/08/2016 09:28 AM, Shahim Essaid wrote:

Hi David,

I'm not part of the Nexus team but I thought I can share what I know in case it is useful.

There is no need to create two artifacts. You just need to deploy Maven artifacts that have the necessary OSGi/Eclipse metadata and Nexus will recognize this make the same artifacts available in the generated P2 repository. How you do this depends on your build. I use Tycho to take care of this but there are other options. The P2 integration is now included as part of the Nexus release and you don't need any additional plugins. If you are working with an older Nexus version you might need to install the two plugins. It's been some time since I looked at the documentation but the last time I read it it described all the steps needed to enable a P2 repository based on an already hosted/mirrored Maven repository but I usually had to lookup the documentation for installing the P2 plugins elsewhere before the plugins were part of the release.

Keep in mind that in Eclipse you will not see plain Eclipse/OSGi bundles when you point at such a P2 repository. Eclipse's views only show you P2 features. You either have to add features to your released bundles (I use Tycho for my builds and this makes it really easy to do), or the other workaround is to mirror the P2 repository (as needed) to your local machine and then use the "folder" option to define target platforms. When you use the "folder" option you will see the individual bundles and you won't need features. But relying on features is the better option if you are mostly developing the Eclipse way and want to take advantage of the Eclipse UIs and the P2 APIs.  There are a couple of pages on the Eclipse wiki that show how to use the command line to mirror P2 repositories and the Tycho plugins also have Maven goals to do this. I usually use the Tycho goals if I need this but I try to use features as much as possible.

I understand the basic process of adding the manifest to make something an OSGi bundle, but I don't know how to handle this last point about features.

I need to mirror artifacts from Maven Central and other non-Eclipse builds as p2 artifacts, and I need that available on our Nexus server.

I'll read that chapter on p2 support in Nexus that another poster cited.

[David M. Karr]

On 03/08/2016 09:35 AM, Manfred Moser wrote:
> Just for reference in terms of installation... the P2 chapter of the
> Nexus Repository Manager documentation can be found at
> http://books.sonatype.com/nexus-book/reference/p2.html

I notice that this seems to correspond to version 2.12, even though the
link doesn't have a version in it, and that isn't the latest version,
which is 3.0. Why is it that the link without specifying a version
isn't the latest version? I also notice that the 3.0 doc doesn't have a
chapter referencing p2.

[Manfred Moser]

There is currently no P2 support in 3.x available yet.

Manfred

[David M. Karr]

On 03/08/2016 10:17 AM, Manfred Moser wrote:
> There is currently no P2 support in 3.x available yet.

Interesting. The 2.x version had it, and now 3.x does not? Could you
provide some additional background on that? Why was it not part of
3.x? You say it's not available "yet". Is it planned? So if someone
is running Nexus (Pro) 3.x today, this simply won't work?

[Brian Fox]

3.0 Does not have feature parity with 2.x. We've been focusing on the most popular requests, particularly ones not available in 2.x. P2 is pretty infrequently used based on the data we've collected so there isn't a specific timeframe for that implementation yet. It doesn't mean it won't be done, it's just not firmly scheduled.

To view this discussion on the web visit https://groups.google.com/a/glists.sonatype.com/d/msgid/nexus-users/56DF1868.4060603%40gmail.com.

[Shahim Essaid]

I noticed that I didn't reply to the list so I'm re-posting a couple of replies in case it helps others.

On Tue, Mar 8, 2016 at 9:45 AM, David M. Karr <davidmic...@gmail.com> wrote:

On 03/08/2016 09:28 AM, Shahim Essaid wrote:

Hi David,

I'm not part of the Nexus team but I thought I can share what I know in case it is useful.

There is no need to create two artifacts. You just need to deploy Maven artifacts that have the necessary OSGi/Eclipse metadata and Nexus will recognize this make the same artifacts available in the generated P2 repository. How you do this depends on your build. I use Tycho to take care of this but there are other options. The P2 integration is now included as part of the Nexus release and you don't need any additional plugins. If you are working with an older Nexus version you might need to install the two plugins. It's been some time since I looked at the documentation but the last time I read it it described all the steps needed to enable a P2 repository based on an already hosted/mirrored Maven repository but I usually had to lookup the documentation for installing the P2 plugins elsewhere before the plugins were part of the release.

Keep in mind that in Eclipse you will not see plain Eclipse/OSGi bundles when you point at such a P2 repository. Eclipse's views only show you P2 features. You either have to add features to your released bundles (I use Tycho for my builds and this makes it really easy to do), or the other workaround is to mirror the P2 repository (as needed) to your local machine and then use the "folder" option to define target platforms. When you use the "folder" option you will see the individual bundles and you won't need features. But relying on features is the better option if you are mostly developing the Eclipse way and want to take advantage of the Eclipse UIs and the P2 APIs.  There are a couple of pages on the Eclipse wiki that show how to use the command line to mirror P2 repositories and the Tycho plugins also have Maven goals to do this. I usually use the Tycho goals if I need this but I try to use features as much as possible.

I understand the basic process of adding the manifest to make something an OSGi bundle, but I don't know how to handle this last point about features.

I need to mirror artifacts from Maven Central and other non-Eclipse builds as p2 artifacts, and I need that available on our Nexus server.

Sure, that should be easily doable, I use it all the time. Create Maven mirrors on your Nexus for the repositories you need, enable P2 on these Maven mirrors, and then run some build to populate your mirror with the needed Maven artifacts and you will see the corresponding P2 artifacts created in the generated P2 repository under /.meta/p2    Look at the metadata files in that location to check if it is working as expected. After you start seeing the artifacts in the artifacts.xml and content.xml (i.e. the Nexus/P2 setup is working) you need to decide on how to use them from Eclipse, your builds, your target platforms, etc.  This is where you might want to have Eclipse features.
 

[Shahim Essaid]

On Tue, Mar 8, 2016 at 10:00 AM, David M. Karr <davidmic...@gmail.com> wrote:

On 03/08/2016 09:53 AM, Shahim Essaid wrote:

On Tue, Mar 8, 2016 at 9:45 AM, David M. Karr <davidmic...@gmail.com> wrote:

On 03/08/2016 09:28 AM, Shahim Essaid wrote:

Hi David,

I'm not part of the Nexus team but I thought I can share what I know in case it is useful.

There is no need to create two artifacts. You just need to deploy Maven artifacts that have the necessary OSGi/Eclipse metadata and Nexus will recognize this make the same artifacts available in the generated P2 repository. How you do this depends on your build. I use Tycho to take care of this but there are other options. The P2 integration is now included as part of the Nexus release and you don't need any additional plugins. If you are working with an older Nexus version you might need to install the two plugins. It's been some time since I looked at the documentation but the last time I read it it described all the steps needed to enable a P2 repository based on an already hosted/mirrored Maven repository but I usually had to lookup the documentation for installing the P2 plugins elsewhere before the plugins were part of the release.

Keep in mind that in Eclipse you will not see plain Eclipse/OSGi bundles when you point at such a P2 repository. Eclipse's views only show you P2 features. You either have to add features to your released bundles (I use Tycho for my builds and this makes it really easy to do), or the other workaround is to mirror the P2 repository (as needed) to your local machine and then use the "folder" option to define target platforms. When you use the "folder" option you will see the individual bundles and you won't need features. But relying on features is the better option if you are mostly developing the Eclipse way and want to take advantage of the Eclipse UIs and the P2 APIs.  There are a couple of pages on the Eclipse wiki that show how to use the command line to mirror P2 repositories and the Tycho plugins also have Maven goals to do this. I usually use the Tycho goals if I need this but I try to use features as much as possible.

I understand the basic process of adding the manifest to make something an OSGi bundle, but I don't know how to handle this last point about features.

I need to mirror artifacts from Maven Central and other non-Eclipse builds as p2 artifacts, and I need that available on our Nexus server.

Sure, that should be easily doable, I use it all the time. Create Maven mirrors on your Nexus for the repositories you need, enable P2 on these Maven mirrors, and then run some build to populate your mirror with the needed Maven artifacts and you will see the corresponding P2 artifacts created in the generated P2 repository under /.meta/p2    Look at the metadata files in that location to check if it is working as expected. After you start seeing the artifacts in the artifacts.xml and content.xml (i.e. the Nexus/P2 setup is working) you need to decide on how to use them from Eclipse, your builds, your target platforms, etc.  This is where you might want to have Eclipse features.

So you're saying that by simply publishing a Maven artifact to a Maven repo served by Nexus that is "p2 enabled", it will transparently add a manifest to that jar that exports and imports all classes?

No. The Maven artifact has to already have all the metadata. Nexus will read that metadata, aggregate it, etc. to build and update the generated P2 repository for that specific Maven repository whether it is hosted, mirrored, grouped, etc.
 

I assume that this transformation only happens when you publish something to this "p2 enabled" repo.  It won't happen magically for any existing artifacts already in the repo when the p2 flag is turned on?

Same comment. The artifacts in the Nexus Maven repository have to have the metadata that was somehow generated during the Maven build. Many Maven Central artifacts already have this metadata and Nexus can read and generate the P2 repository from it.
 

Is there any reason NOT to simply "p2 enable" our existing Maven repositories that our plain Maven builds are using, so that Eclipse builds can use the same artifacts?  What are the consequences of doing that?

I'm starting to get the feeling that you're saying that configuring "features" is really just an enhancement to make it easier to specify the artifacts you need.  If so, I don't see the need for that here.

 Yes. Features are only needed or useful if you are using tools that need them. Features are just another layer for defining modularity, dependencies, versioning, etc.