Consistent Component release process

[Kevin Sutter]

Hi,
As each of the components have started to create their RCn releases and, in some cases, the final releases, we've all been copying the perform_release scripts from the microprofile-config repo.  For the most part, this process has been working out fairly well.  Thanks to Ondro and Emily for paving the way for us.

But, we have also discovered that these scripts are starting to diverge, along with the maven plugins and configuration in our pom.xml files.  I'd like to figure out a proper answer for all of our components.  Of course, there may be some minor tweaks that are unique for a given component.  But, the basic processing should all be the same.  When I did a little comparison, I found pom plugin usage and configuration differences across several components.

To that end, I took the Config instructions and tried to generalize them at the project level:
https://wiki.eclipse.org/MicroProfile/SpecRelease/Release

But, I need some assistance with generalizing the scripts and required pom.xml updates.

Thoughts or suggestions on how to proceed?
Thanks, Kevin

[John D. Ament]

I had taken a todo at some point to figure out why we needed the extra steps that we do for repository syncing.  I just created https://bugs.eclipse.org/bugs/show_bug.cgi?id=521965 to try to tackle part of this.

Basically, if we could use the Eclipse Nexus instance to manage our staged releases, then we would never actually need either script, independent developers would be able to just run the release locally (as mvn releease:prepare release:perform) and assuming they have write access to the nexus instance, it would create an automatic staging area that we can review against, and once ready either drop it (as not being released) or release it (meaning we sync it to maven central).

John

[John D. Ament]

Ok, so the nexus answer is no.

But an eclipse policy question (since I don't know and I'm not sure who to ask).  Do we have to put the RC's on the Eclipse repository?  Or is it viable to stage them on the OSS repos from Sonatype?

John

[Kevin Sutter]

I think the use of the Eclipse repo has been an open question...  I don't think it's a firm requirement.  But, we need to decide whether it's useful to populate the Eclipse repo as well as the OSS repo.  Ondro had looked into this previously.

--  Kevin

--
You received this message because you are subscribed to a topic in the Google Groups "Eclipse MicroProfile" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/microprofile/X8xwuHrPWcM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to microprofile+unsubscribe@googlegroups.com.
To post to this group, send email to microp...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/338e6766-d42a-4569-b0da-594e24c5eb53%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[ondrej....@gmail.com]

I think it's really up to us to decide between Eclipse and Sonatype repos.

 

So far, eclipse was used for RC because it can be automated with Jenkins. Son type requires running man deploy locally, but is used for final versions because everybody wants final versions in maven central.

 

Ideally, we would only release to Sonatype with Jenkins jobs, but there are technical obstacles in doing so.

 

 

 

Ondrej Mihalyi

To unsubscribe from this group and all its topics, send an email to microprofile...@googlegroups.com.

To post to this group, send email to microp...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/338e6766-d42a-4569-b0da-594e24c5eb53%40googlegroups.com.

[John D. Ament]

But that would be as simple as getting eclipse webmaster to add a custom settings.xml file for us, right?

To unsubscribe from this group and all its topics, send an email to microprofile+unsubscribe@googlegroups.com.

To post to this group, send email to microp...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/338e6766-d42a-4569-b0da-594e24c5eb53%40googlegroups.com.

[John D. Ament]

So I asked Eclipse the question about creating a settings.xml file.  Sounds like they may already do this in other places, so they're running with it.  We'll be able to push to an OSS staging area.

so with that said, we could just write jenkins jobs that do the work.  Thoughts?

John

[sst...@redhat.com]

That is certainly much preferred in my opinion.

To unsubscribe from this group and all its topics, send an email to microprofile...@googlegroups.com.

To post to this group, send email to microp...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/338e6766-d42a-4569-b0da-594e24c5eb53%40googlegroups.com.

[Kevin Sutter]

+1

[John D. Ament]

It looks like Eclipse can accommodate everything needed to make this automated via jenkins.

- Setup the OSS account

- Setup the jenkins config for settings.xml

- Create a user that has write access to our repos to push tags/branches/commits

- Configure jenkins to have our account

I asked them to hold on it for now, not to cause confusion with our current in flight releases.  I figure once 1.2 is wrapped, we can ask them to do the cut over and we can setup the necessary changes in jenkins to work.

Any concerns?  The jenkins jobs would be changed to:

- Checkout master (or maybe some other branch)

- Perform the maven release steps

- Push the released artifacts to Sonatype OSS

- Push the relevant commits to the MP repos

- Archive the spec documents

We could consider other steps like auto closing the repo if we want.

John

[Emily Jiang]

Sounds good! When you say archive the spec documents, do you mean the spec archive will be uploaded to Sonatype OSS as well?

Emily

[Ondro Mihályi]

Thank you, John, you're awesome!

I never thought Eclipse would do that much for us but they are really amazing.

We can archive the spec documents in the eclipse download area. I already ensured this can be done from a Jenkins job.

The last thing to sort out is to have a GPG key to sign maven artifacts (requirement of the Sonatype maven repo). The best would be if Eclipse generated a private/public key pair for us on the Jenkins server so that a Jenkins job has access to them. However, I'm not sure whether the private key can be stored there safely.

--Ondro

[John D. Ament]

Yes, the expectation is that they'll have the necessary keys in place to both sign the packages and push changes to the repo.  

I suspect pushing the specs to the download site make sense.  Which job did you do this in?

[Ondro Mihályi]

I've asked Eclipse webmasters to allow Jenkins to upload to the download area and tested it with a simple Jenkins job https://ci.eclipse.org/microprofile/job/test%20filesystem/

You can see the uploaded file here: http://download.eclipse.org/microprofile/testfile

My vision is that every feature would have a Jenkins job, which would deploy released artifacts and upload PDFs into the Eclipse Download area.

We can also have a Jenkins job to update the page at http://download.eclipse.org/microprofile/ with information about every released feature (e.g. something like I did for Config 1.0 with the maven site plugin - https://ondrejm.github.io/microprofile-config/index.html)

--Ondro

[Heiko Braun]

I am late to the game. Can someone briefly summarise the decisions taken? Do we have a common release procedure or do all spec roll their own for now? 

[Emily Jiang]

Heiko, 

All specs roll their own for now. More experiments are being done by John and Ondro.

Emily

[John D. Ament]

In general its looking like:

- Both snapshots and release builds get pushed Sonatype OSS

- Setup a jenkins job that executes the release process, so that its push button

This would all be tested after MP 1.2 is shipped.

John

[Emily Jiang]

John,

Am I right to assume the release builds including the spec pdf files? If yes, the spec pdfs can be downloaded from Sonatype OSS, which will be awesome. 

Emily

--

You received this message because you are subscribed to a topic in the Google Groups "Eclipse MicroProfile" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/microprofile/X8xwuHrPWcM/unsubscribe.

To unsubscribe from this group and all its topics, send an email to microprofile+unsubscribe@googlegroups.com.

To post to this group, send email to microp...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/1857bd18-a9f3-4b7f-9872-0139b25d34c5%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Thanks
Emily
=================
Emily Jiang
eji...@apache.org

[John D. Ament]

No, Ondrej indicated they could grab them from the eclipse.org downloads section.

[Ondro Mihályi]

I haven't yet proved that PDFs can be uploaded to Sonatype as maven artifacts with a reasonable effort. Therefore, the next step is to make them available at download.eclipse.org, which we know can be done with a Jenkins job.

Later, if possible, we can release PDFs as maven artifacts, but that won't happen for MP 1.2

--Ondro

On Tuesday, September 12, 2017 at 8:12:50 PM UTC+2, John D. Ament wrote:

No, Ondrej indicated they could grab them from the eclipse.org downloads section.

On Tuesday, September 12, 2017 at 1:56:09 PM UTC-4, Emily Jiang wrote:

John,

Am I right to assume the release builds including the spec pdf files? If yes, the spec pdfs can be downloaded from Sonatype OSS, which will be awesome. 

Emily

On Tue, Sep 12, 2017 at 6:45 PM, John D. Ament <john.d...@gmail.com> wrote:

In general its looking like:

- Both snapshots and release builds get pushed Sonatype OSS

- Setup a jenkins job that executes the release process, so that its push button

This would all be tested after MP 1.2 is shipped.

John

On Tuesday, September 12, 2017 at 9:03:49 AM UTC-4, Heiko Braun wrote:

I am late to the game. Can someone briefly summarise the decisions taken? Do we have a common release procedure or do all spec roll their own for now? 

--
You received this message because you are subscribed to a topic in the Google Groups "Eclipse MicroProfile" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/microprofile/X8xwuHrPWcM/unsubscribe.

To unsubscribe from this group and all its topics, send an email to microprofile...@googlegroups.com.

To post to this group, send email to microp...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/1857bd18-a9f3-4b7f-9872-0139b25d34c5%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Emily Jiang]

Thanks John and Ondro!

Emily

[Kevin Sutter]

I would like us to qualify the generation of the spec documents with the release designation.  Currently, mosat of the spec names used are generic...

microprofile-fault-tolerance-spec.pdf

But, it would be better to have it qualified like the api and tck artifacts...

microprofile-fault-tolerance-spec-1.0-RC3.pdf

(I know that FT already made this change, but we should recommend this update across the components.)

Thanks,

Kevin

To unsubscribe from this group and all its topics, send an email to microprofile+unsubscribe@googlegroups.com.

To post to this group, send email to microp...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/7531aacc-50b2-4fa2-9c01-eef6d00b1a32%40googlegroups.com.

[sst...@redhat.com]

I'm not seeing how to configure this. I did just pull down the fault tolerance repo and built the spec pdf and it does not have the version encoded into the name in my build.

I do see an outputFile config option described in https://github.com/asciidoctor/asciidoctor-maven-plugin#configuration-options, but I can't seem to get that to work. 

[sst...@redhat.com]

It looks like the outputFile config option was recently added and post the current 1.5.5 release.

[Kevin Sutter]

Scott...  I had suggested it to Emily and I noticed it was up on her RC3 release for FT...  I just assumed she had modified her script, but maybe it was a manual change.  Sorry for the confusion.

--  Kevin

To unsubscribe from this group and all its topics, send an email to microprofile+unsubscribe@googlegroups.com.

To post to this group, send email to microp...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/b4933078-8d69-4f15-9a94-1023ac1edde8%40googlegroups.com.

[Emily Jiang]

Yes, it was a manual change and upload.

Emily

[sst...@redhat.com]

Ok, I'll do that for the github release.

I'm also going to see if I can do a PR for the asciidoc plugin to make the new outputFile config option easier to use and currently it is a fully qualified path rather than something relative to target/generated-docs as I would expect.

[John D. Ament]

All,

Just as a heads up, I've given the go ahead to Eclipse Webmaster to pick this back up (I'm more available to do the jenkins work + not blocking the 1.2 release).  It sounds like they'll be picking this back up.

John