Maven release process

[Thomas Ferris Nicolaisen]

Hi,

I noticed that you have a rather tedious release process [1]. It
includes manual tagging and deployment, etc.

Many projects use the maven-release-plugin for this. Have you
considered it?

The biggest challenge is getting the released artifacts into a public
maven repo automatically. You could start struggling with the Maven
Wagon SCM plugin to achieve this in your Subversion repo, but I see
that there is a repository available for Google's maven projects [2].
Maybe you could use this?

If you're interested, I could suggest a patch to your pom.xml that
would support this setup. It might require some restructuring in your
project though (locate project in gson/trunk).

This would effectively remove steps 3-5 plus 8-16, and replace with a
two-step mvn release:prepare, mvn release:perform.

(It would also let you enjoy the benefits of Maven's SNAPSHOT
mechanism in your development process.)

So, worth the effort?

[1] http://sites.google.com/site/gson/release-process
[2] http://code.google.com/p/google-maven-repository/wiki/GuidelinesForGoogleMavenProjects

[inde...@gmail.com]

Hi Thomas,

Indeed, our lack of knowledge of Maven continues to frustrate us. Your
patch would be very welcome!

Thanks
Inder

> [2]http://code.google.com/p/google-maven-repository/wiki/GuidelinesForGo...

[Thomas Ferris Nicolaisen]

Hi Inder, that's great.

First step is to request to use Sonatype's open source maven
repository. As you are the one who is going to make releases (and need
credentials to upload artifacts into the maven repository), it's best
if you do this step. You need to follow the instructions described
under step 3 (you'll have to sign-up first if you don't already have
an account):

https://docs.sonatype.org/display/Repository/Sonatype+OSS+Maven+Repository+Usage+Guide

It should only take you a few minutes. After that, Sonatype will
prepare the repository for you. Please post back with the url to the
ticket you create.

I'll fix up the patch and write some instructions until then.

-Thomas

[Thomas Ferris Nicolaisen]

I think rather than just dumping the patched pom, we can make a bit
more fluent transition.

The first step is to change the gson trunk project into being a
SNAPSHOT version.

[version X]-SNAPSHOT basically means getting-closer-to-releasing
version X. It is a *snapshot* in time on the road to release.

Since gson is currently developing what will be released as version
1.7, so for now, it should be 1.7-SNAPSHOT.

When the time comes for a release, we will
* remove SNAPSHOT from the version
* Copy the codebase into tags/gson-1.7
* set the new version to be 1.8-SNAPSHOT. And so life continues.

The above steps are exactly what the maven-release-plugin does in the
prepare-goal. We'll get back to this when we actually are ready to
make a release later on.

For now, we want to be able to build and deploy 1.7-SNAPSHOT, so that
other projects can try out our latest-greatest gson. Maybe they are
anxiously waiting for the 1.7 release, and would like to download a
snapshot right now, without having to check out and build the project
themselves.

The best way to do this is to *deploy* gson-1.7-SNAPSHOT into the
SonaType snapshot-repository here:

https://oss.sonatype.org/content/repositories/snapshots/

In order to deploy, we need to have
1) Permission to deploy to the repository
2) Configured gson's pom to deploy snapshots into the repository

We achieve (1) by filing a request at SonaType's Jira (hopefully
already done, and we're waiting for them to create a user-account).
When you receive the credentials, add them into your own personal
~/.m2/settings.xml:

<settings>
...
<servers>
<server>
<id>sonatype-nexus-snapshots</id>
<username>your-jira-id</username>
<password>your-jira-pwd</password>
</server>
<server>
<id>sonatype-nexus-staging</id>
<username>your-jira-id</username>
<password>your-jira-pwd</password>
</server>
</servers>
...
</settings>


The easiest way to achieve step (2), is to add SonaType's standard
open-source parent as the gson pom's parent:

...
<groupId>com.google.code.gson</groupId>
<artifactId>gson</artifactId>
<packaging>jar</packaging>
<version>1.7</version>
<parent>
<groupId>org.sonatype.oss</groupId>
<artifactId>oss-parent</artifactId>
<version>5</version>
</parent>
...

Once this is in place, it should be possible to deploy gson snapshots
into the SonaType repository like this:

mvn clean deploy

Once this works, we can continue towards getting a real release
working. Let me know how far you get, and please post back with any
problems.

-Thomas

On Nov 29, 10:26 am, Thomas Ferris Nicolaisen <tfn...@gmail.com>
wrote:

> Hi Inder, that's great.
>
> First step is to request to use Sonatype's open source maven
> repository. As you are the one who is going to make releases (and need
> credentials to upload artifacts into the maven repository), it's best
> if you do this step. You need to follow the instructions described
> under step 3 (you'll have to sign-up first if you don't already have
> an account):
>

> https://docs.sonatype.org/display/Repository/Sonatype+OSS+Maven+Repos...

[Thomas Ferris Nicolaisen]

Just to bump this topic, Google Guice and Sitebricks are now using mentioned released strategy. Here is Sitebrick's account request:

https://issues.sonatype.org/browse/OSSRH-95

As a result, you can for instance see that Google Guice 3.0 RC1 is already available in the repo, one day after release:

https://oss.sonatype.org/index.html#nexus-search;quick~guice

.. and soon it will be synced into the maven central repo, I believe.

I believe Stuart McCulloch (mcculls) assisted them with the process.

Here's Guice's latest pom. Notice the 3.0-SNAPSHOT version, as well as the "sonatype-grid" profile:

http://code.google.com/p/google-guice/source/browse/trunk/pom.xml?r=1453

[Inderjeet Singh]

Hi Thomas, 

I am really sorry that I didn't get around to resolving this earlier. However, now I do have some time to spend on it and would appreciate your help. 

I have created the ticket at sonatatype: 

https://issues.sonatype.org/browse/OSSRH-1166

Thanks

Inder

--
You received this message because you are subscribed to the Google Groups "google-gson" group.
To post to this group, send email to googl...@googlegroups.com.
To unsubscribe from this group, send email to google-gson...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-gson?hl=en.

[Thomas Ferris Nicolaisen]

No worries, we all get a little busy around the holidays :)

Now we have to wait until they've created your account for the repository. After that we'll continue. Not sure how long it takes.. 

[Thomas Ferris Nicolaisen]

Oh, it appears they have already done it. Paste from jira:

Juven Xu added a comment - 01/11/11 12:55 AM

Configuration has been prepared, now you can:

• Deploy snapshot artifacts into repository https://oss.sonatype.org/content/repositories/snapshots
• Deploy release artifacts into the staging repository https://oss.sonatype.org/service/local/staging/deploy/maven2
• Promote staged artifacts into repository 'Releases'
• Download snapshot and release artifacts from group https://oss.sonatype.org/content/groups/public
• Download snapshot, release and staged artifacts from staging group https://oss.sonatype.org/content/groups/staging

So, going back to the guide, the next step is to deploy some snapshots. But first you need to satisfy point 5. Prerequisites, including getting GPG signing working.

When you've got that, it's time to go to work on the pom.xml. First make sure that all the sections they require are included, and add the oss-parent, and so on. The guide is pretty straight forward on this.

Then add the sonatype repos, and the credentials of your freshly created account to your settings.xml before you try a snapshot deployment (make sure you change version in the pom to a -SNAPSHOT first):

 $ mvn clean deploy

If this works, we're nearly there. Next thing will get the maven-release-plugin working (which might be a bit of fiddling).

[Simone Tripodi]

Hi all guys,
just to let you know that Google-Doclava[1] is available as well in
public mvn repo, I (proudly) helped the team to finalize the pom and
deploy the artifact on sonatype's repo, we wrote a resume guide on[2].
Just let me know is there's need of more effort, I would be more than
pleased to provide you my contribution.
All the best, have a nice day!
Simo

[1] http://code.google.com/p/doclava/
[2] http://code.google.com/p/doclava/wiki/ProcessRelease

http://people.apache.org/~simonetripodi/
http://www.99soft.org/

[Inderjeet Singh]

Thanks Thomas and Simone, 

I have setup my PGP keys, encrypted passwords in settings.xml and was able to run mvn clean deploy. 

I also updated our pom.xml per the guidelines. Can you do a quick review? It is available at: 

http://code.google.com/p/google-gson/source/browse/trunk/gson/pom.xml

Also, what do I need to do to get the maven-release-plugin? Is there a way to test it out without actually cutting a new release? 

Thanks

Inder

[Thomas Ferris Nicolaisen]

Hi Inder, thanks for the fast feedback/progress!

You don't have to do anything special to acquire the maven-release-plugin, it's included in maven core just like the compiler-plugin, or deploy-plugin.

The pom.xml looks excellent.

There is one thing that *might* make doing a release easier: having a trunk/tags structure for the project you want to release like this:

gson/trunk/

gson/tags/

It might be possible to work around this by configuring an alternative tagBase in the maven-release-plugin:

    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-release-plugin</artifactId>
        <configuration>
          <tagBase>https://google-gson.googlecode.com/svn/tags/</tagBase>
        </configuration>
      </plugin>
    </plugins>

In any case, you can "test" a release by doing a dry-run:

$ mvn release:prepare -DdryRun=true

This doesn't do much, but shows you the interaction of the release process, letting you choose versions for release, but just go with defaults: it should be version 1.7. It will create some temporary files in your directory. These will have to be cleaned up before you try a real release:

Note: Before you release, make sure you have command-line write-access to the svn-url where the tag will be created. Remember that you can't have any local changes in the project when releasing.

$ mvn release:clean

$ mvn release:prepare

The prepare-goal does a lot: 

1. full build
   
2. changes release version (1.7) and scmUrls (tags/..) in the pom.xml
   
3. commits 
   
4. creates the tag
   
5. changes the pom.xml to next version (1.8-SNAPSHOT), commits again
   

In my experience, this process sometimes fails (especially the first time you do it on a machine), and you're left in a mess. If that happens, keep cool. Try doing a mvn release:rollback - and if that doesn't work, just bang the project back into being 1.7-SNAPSHOT again (mind the <version> section and that <scm> points at trunk again). Remove temporary release files and commit. Remove the svn-tag if it was at all created. Find out what was wrong, fix it, and try again.

If it does work, congratulations! The worst part is over. Now the build from the newly created tag will have to be deployed to the maven repository. We use the second goal in the maven-release-plugin for this:

$ mvn release:perform

Since your snapshot deployment is already working, this should work fine too.

After deploying, you should be able to find the artifacts by searching for them here: https://oss.sonatype.org/index.html#nexus-search;quick~gson (the 1.7-SNAPSHOT is already in there).

Now comes the part I'm quite unfamiliar with: The release is now staged. Before it is promoted to the real repository, you have to log into Nexus and to some clicking. There are some instructions here:

https://docs.sonatype.org/display/Repository/Sonatype+OSS+Maven+Repository+Usage+Guide#SonatypeOSSMavenRepositoryUsageGuide-8.ReleaseIt

... but Juxen also wrote in Jira that you can "Promote staged artifacts into repository 'Releases'", which sounds a bit different. Maybe Simone can help more with this part.

[Simone Tripodi]

Hi all and sorry I come back a little late,
excellent guide Thomas, very useful!!! I don't have anything more to
add, I can just suggest to have a look at [1] to better understand
what 'staging' and 'releases' repositories are.
Have a nice day!
Simo

[1] http://www.sonatype.com/books/nexus-book/reference/staging-sect-managing-staging.html

http://people.apache.org/~simonetripodi/
http://www.99soft.org/

[inde...@gmail.com]

Hi Thomas and Simone,

I was able to run mvn release:prepare -DdryRun=true successfully and
also verified that the snapshot is available in sonatype repository.
I guess I will be able to test the full process only when we cut Gson
1.7 so I will ask again on this thread when we get there.

If I run mvn release:clean and mvn release:prepare from a machine and
then run mvn:rollback would that kill the release without publishing
it to any repository?

I would also setup a similar deployment process for our other open-
source project: http://code.google.com/p/greaze/
That project has several deployment jars. Would we create a single
Jira repository for them?

Thank you very much for your help!
Inder

On Jan 12, 11:13 pm, Simone Tripodi <simonetrip...@apache.org> wrote:
> Hi all and sorry I come back a little late,
> excellent guide Thomas, very useful!!! I don't have anything more to
> add, I can just suggest to have a look at [1] to better understand
> what 'staging' and 'releases' repositories are.
> Have a nice day!
> Simo
>

> [1]http://www.sonatype.com/books/nexus-book/reference/staging-sect-manag...
>
> http://people.apache.org/~simonetripodi/http://www.99soft.org/

> >https://docs.sonatype.org/display/Repository/Sonatype+OSS+Maven+Repos...

[Thomas Ferris Nicolaisen]

Hi Inder,

I was able to run mvn release:prepare -DdryRun=true successfully and
also verified that the snapshot is available in sonatype repository.
I guess I will be able to test the full process only when we cut Gson
1.7 so I will ask again on this thread when we get there.

If you want to test the release process before that, you can release a preliminary version, you can enter 1.6.1, or 1.7-RC1 or something like that when running mvn release:prepare.

If I run mvn release:clean and mvn release:prepare from a machine and
then run mvn:rollback would that kill the release without publishing
it to any repository?

 

Yes. Uploading to repository is done after preparing, in the release:perform step. release:rollback will revert the versions in the pom.xml and remove the tags in svn (I think).

 

I would also setup a similar deployment process for our other open-
source project: http://code.google.com/p/greaze/
That project has several deployment jars. Would we create a single
Jira repository for them?

I'm not sure, but the safest thing to do is probably to just create a task in Jira and ask SonaType. 

[Simone Tripodi]

Hi all guys,
I'm very sorry to come always late, please apologize!!!
What I'd suggest you to make things easier, being these projects part
of the same organization, is trying to obtain the deploy grant for a
common groupId, like 'com.google.XXX', so you are free to deploy all
the modules you need without opening new jira issues on Sonatype for
every new project.
Moreover, in that way you can think about a more general parent pom,
so you don't need to copy'n'paste the same stuff from a project to
another. Just let me know if you need help on this, I already did the
same in the MyBatis[1] project.
HTH, have a nice day!!!
Simo

[1] http://code.google.com/p/mybatis

http://people.apache.org/~simonetripodi/
http://www.99soft.org/

[Carlos Sanchez]

Hey guys,

This is Carlos from the Maven project.

I'd like to get gson 1.6 into the central, for current users

Either you can upload it using a webdav client or you can give me permission to do it myself (just say in this mailing list that it's ok)

For the former, please take the files from
http://google-gson.googlecode.com/svn/mavenrepo/com/google/code/gson/gson/1.6/
remove the repositories section from the pom and regenerate it's md5 and sha1
gpg sign all jar and pom files
gpg2 --armor --output $file.asc --detach-sig $file

then using webdav just copy them into
https://oss.sonatype.org/service/local/staging/deploy/maven2/com/google/code/gson/gson/1.6/
and promote the release from Nexus web interface

Thanks!

[Carlos Sanchez]

Actually make that 1.6 and 1.5, I need both of them

[Jerome Lacoste]

On Wednesday, February 2, 2011 7:41:10 PM UTC+1, Carlos Sanchez wrote:

Actually make that 1.6 and 1.5, I need both of them

+1 for that request. I need 1.6 as well :)

Thanks,

Jerome Lacoste

[inde...@gmail.com]

Carlos: Please go ahead with updating Gson versions 1.6 and 1.5 in
central.

Sorry, I was away on vacation and didn't see this email until now.

[Carlos Sanchez]

Thanks, gson 1.5 and 1.6 are now up in the Maven central repo

http://repo1.maven.org/maven2/com/google/code/gson/gson/

[inde...@gmail.com]

Here is a problem I encountered in using this Sonatype automatic
deployment process while releasing Gson 1.7 and 1.7.1:

mvn release:prepare and mvn release:perform would not upload the GnuPG
signatures to the Sonatype repository. This forced me to use mvn
gpg:sign-and-deploy-file manually for each of the artifacts. Both of
those commands did prompt me for my gpg password but didn't generate
the .asc files.

Can someone help me figure out how to solve this? Here is a link to
our pom.xml: http://code.google.com/p/google-gson/source/browse/trunk/gson/pom.xml

Thanks
Inder

[Thomas Ferris Nicolaisen]

Hi Inder,

Maybe you can post the exact output or error message you got when trying to upload?

Otherwise, I have no experience with gpg-signed artifacts, so someone else will have to fill in there.