[JIRA] [subversion] (JENKINS-18907) Add feature to expose the latest revision number of a checkout

[kevin.phillips@caris.com (JIRA)]

Kevin Phillips created JENKINS-18907 Add feature to expose the latest revision number of a checkout Issue Type: Improvement Affects Versions: current Assignee: Unassigned Components: subversion Created: 24/Jul/13 2:35 PM Description: Currently the SVN plugin only provides a single SVN revision number for any given URL (ignoring for the moment the fact you can provide multiple URLs in one configuration). Further, this one SVN revision number is the revision of the "last modification" to the repository URL provided in the configuration. This version number is sufficient in certain use cases but not in every use case. One example of this is when a given SVN URL uses externals to non-static revisions of other sources. In this case, the SVN plugin incorrectly reports the last change for the root URL and ignores the revisions of all externals. Ideally this plugin would either provide the option or expose as a secondary Jenkins property the HEAD or latest checkout / repository revision instead of / as well as the last changed revision. This would allow jobs configured with a single SVN URL with external dependencies to use the latest checkout revision as the unique ID for the checkout. Environment: Win7 x64 Jenkins v1.590.1 SVN plugin v1.48 Project: Jenkins Priority: Major Reporter: Kevin Phillips

This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira

[kutzi@gmx.de (JIRA)]

kutzi commented on JENKINS-18907

Add feature to expose the latest revision number of a checkout

IMO the plugin works as designed as AFAIK it returns pretty much the same, what svn info would return as 'Last Changed Rev' Externals are a different beast.

[kutzi@gmx.de (JIRA)]

kutzi commented on JENKINS-18907

Add feature to expose the latest revision number of a checkout

So to summarize/rephrase your request: you want to have the revisions of the SVN externals as environment variables?

[kevin.phillips@caris.com (JIRA)]

Kevin Phillips updated JENKINS-18907

Add feature to expose the latest revision number of a checkout

Change By: Kevin Phillips (24/Jul/13 2:56 PM) Description:

Currently the SVN plugin only provides a single SVN revision number for any given URL (ignoring for the moment the fact you can provide multiple URLs in one configuration).

Further,  ad-hoc tests reveal that  this one SVN revision number  is  represents  the  revision of the  "last  modification  changed revision "  to  for  the  repository URL  provided  in the configuration  URL .  This version number is sufficient in   In  certain  use  cases  but not in every use case. One example  it would be useful to also have access to the "latest" revision as well - ie: the revision  of  this is when  the repository after the most recent checkout or update operation. I suspect such an enhancement could be made in  a  given SVN URL uses externals to  non- static revisions of other sources breaking way as well .  In this case  For example ,  perhaps  the SVN plugin  incorrectly reports  could provide an option - either globally in  the  last change for  main Jenkins configuration area, or per-job in  the  root URL and ignores the revisions  SCM configuration section  of  all externals. Ideally this plugin would either provide  each job - to have  the  option or expose as a secondary  SVN_REVISION  Jenkins property  defined by either  the  HEAD  last changed revision  or  the  latest  checkout / repository  revision  instead of / ,  as  well as the last changed revision  desired .  This  Alternatively, perhaps a new Jenkins property could be introduced, say SVN_LATEST_REVISION, which  would  allow jobs configured with  contain the appropriate value. NOTE: I suspect that this enhancement could be used as  a  single SVN URL with external dependencies  workaround  to  use  some (but not all) of  the  latest checkout revision as  workflows being discussed on JENKINS-1241. Also given  the  unique ID for  fact that either solution I suggested would be non-breaking it could conceivably be implemented and released on all currently supported versions of  the  checkout  plugin .

[kevin.phillips@caris.com (JIRA)]

Kevin Phillips commented on JENKINS-18907

Add feature to expose the latest revision number of a checkout

Thanks for the prompt response. I just re-phrased the description to be a bit more clear. Let me know if it makes more sense now.

[kutzi@gmx.de (JIRA)]

kutzi commented on JENKINS-18907

Add feature to expose the latest revision number of a checkout

BTW: what other example than externals do you see when a single revision number is not sufficient?

[kevin.phillips@caris.com (JIRA)]

Kevin Phillips commented on JENKINS-18907

Add feature to expose the latest revision number of a checkout

Potential Use Case #1: Suppose you have an SVN module that uses one or more SVN externals to pull in dependent projects. For the sake of simplicity lets further assume that all said externals use the same repository as the root module. Finally, let also assume that you are trying to keep up to date with the bleeding edge version of each of those externals and to do so you are using the head revisions of the target branches of each of them. In this case, when you do a checkout or update you are not only updating your root folder / module to its latest revision, you are also updating all of your externals to their latest revisions. Now since this is a mixed revision checkout getting a single unique identifier for the whole checkout is problematic. However, in this particular use case you can approximate it by using the "latest revision" of the local working folder (primarily as a side effect to the fact that all externals use the same root repo and SVN revision numbers are globally unique across any given repo).

[kevin.phillips@caris.com (JIRA)]

Kevin Phillips commented on JENKINS-18907

Add feature to expose the latest revision number of a checkout

Potential Use Case #2: Suppose a site has a single SVN repository for all of their code modules and this repository is shared across many teams and projects. In such a case there are typically commits occurring frequently enough that CI builds - even those triggered my non SCM changes - are likely to get a new "latest revision" every time they do an SVN update. In such a case it is feasible to use the SVN revision number as a universal unique identifier for any particular build of any particular module. For convenience it is handy to propagate this revision number throughout the system, using it as the unique identifier for all CI artifacts (builds, tests, releases, packages, etc.) Now, in order for such a configuration to rely on the SVN revision number being unique (at least in most cases) they need to leverage the "latest" revision number of a working folder rather than the "last changed" revision - particularly for those cases when a build is manually triggered (manual triggers would do an SVN update even if there were no changes to that particular module, and the resulting "latest revision" would be updated accordingly). In such a configuration using the last change revision can not always ensure uniqueness in the build number, however so long as the frequency of modifications to the central repository is high enough (which in a large system it typically is) the "latest" revision can effectively be used.

[kevin.phillips@caris.com (JIRA)]

Kevin Phillips commented on JENKINS-18907

Add feature to expose the latest revision number of a checkout

Potential Use Case #3: Suppose again that you have a single SVN repository and you have a Jenkins configuration with 2 jobs, one dependent on the other, both stored in this single repository. Next lets suppose you have the Parameterized Trigger Plugin installed and configured so that downstream job A triggers upstream job B, passing along the Subversion revision number - to ensure the triggered instance of Job B uses the same revision as job A. Now, lets suppose you are working on a change that requires modification to both module A and B. Since both modules are in independent folders (albeit in the same repo) the commits to module A and B will generate two unique revision numbers (say revision 123 and 124 respectively). So when job A triggers (ie: via an SCM polling operation) it will first do an update to module A and set the SVN revision number. Now because the Jenkins plugin uses the last change revision it will set the revision of this execution to 123. Once complete job A will trigger job B passing revision 123 - and job B will fail... because it needs revision 124 to be successful. In this case, if Jenkins were to use the "latest revision" rather than the "last change revision", then job A would have performed its update and pulled down the latest version of itself (ie: v123) but the latest revision would actually be v124 (assuming that both changes were committed before job A started it's build - which is often the case so long as you have a reasonable "quiet period" configured for the job). Then, if this more appropriate version number were propagated to job B then it would have succeeded as expected.

[kevin.phillips@caris.com (JIRA)]

Kevin Phillips commented on JENKINS-18907

Add feature to expose the latest revision number of a checkout

In response to this earlier comment, while I think it may be helpful in certain cases to expose the revisions of any / all externals to Jenkins builds, to help limit the scope of this particular enhancement request (and possibly to keep the resulting changes as generic and generally applicable as possible) I think it will suffice to simply expose the "latest" SVN revision for a checkout to the build environment as mentioned in the description.

[kevin.phillips@caris.com (JIRA)]

Kevin Phillips commented on JENKINS-18907

Add feature to expose the latest revision number of a checkout

Also, in response to this other earlier comment, while I do agree the plugin is working as intended, the use cases I describe in my earlier comments illustrate the need for accessing the latest revision instead of / as well as the last changed revision. These revisions serve different purposes for different use cases - which is why the SVN tools expose them both to the user as need. To relate this request to functionality in the standard SVN client toolset, currently the SVN plugin exposes the revision generated by the command "svnversion . -c" but I would also like the ability to get the results of "svnversion ." as well.

[kevin.phillips@caris.com (JIRA)]

Kevin Phillips edited a comment on JENKINS-18907

Add feature to expose the latest revision number of a checkout

Also, in response to this other earlier comment, while I do agree the plugin is working as intended, the use cases I describe in my earlier comments illustrate the need for accessing the latest revision instead of / as well as the last changed revision. These revisions serve different purposes for different use cases - which is why the SVN tools expose them both to the user as needed.

To relate this request to functionality in the standard SVN client toolset, currently the SVN plugin exposes the revision generated by the command "svnversion . -c" but I would also like the ability to get the results of "svnversion ." as well.

This message is automatically generated by JIRA.

[kutzi@gmx.de (JIRA)]

kutzi commented on JENKINS-18907

Add feature to expose the latest revision number of a checkout

I've some reservations about use case 1 (IMO externals should better be avoided if possible), but 2 and 3 sound indeed realistic. I guess the way to go is to add another set of Environment Variables (e.g. SVN_CURRENT_REVISION_n) containing the 'current' revision. According to most documentation I could find, this seems to be the common name for this: current revision - in contrast to the latest-changed revision

[kutzi@gmx.de (JIRA)]

kutzi commented on JENKINS-18907

Add feature to expose the latest revision number of a checkout

Feel free to create a pull request for the improvement

[kutzi@gmx.de (JIRA)]

kutzi updated JENKINS-18907 Expose svn 'current' revision in addition to the 'last-changed' revision Change By: kutzi (29/Jul/13 8:31 AM) Summary: Add feature Expose svn 'current' revision in addition  to  expose  the  latest  'last-changed'  revision  number of a checkout

[kevin.phillips@caris.com (JIRA)]

Kevin Phillips commented on JENKINS-18907

Expose svn 'current' revision in addition to the 'last-changed' revision

Your proposal indeed sounds reasonable. I was actually considering putting in a pull request. I am not that familiar with that process as I've never done that before but I had considered figuring that out at some point.

[kevin.phillips@caris.com (JIRA)]

Kevin Phillips commented on JENKINS-18907 Expose svn 'current' revision in addition to the 'last-changed' revision

I had one somewhat-unrelated question. You mention that externals should be avoided if possible - do you know of another way of managing complex sets of dependencies between subsystems without using externals? I am working in a C++ shop so we don't have the luxury of advanced dependency management tools like Maven, Nexus and such..... Externals seem like the only practical solution for this in our case. Also, just so you know, all three of the "potential use cases" I outlined above we actually do use (or have tried) at our site... so they aren't really "potential" in that regard. And option #1 is our most common usage pattern at the moment.

[kevin.phillips@caris.com (JIRA)]

Kevin Phillips edited a comment on JENKINS-18907

Expose svn 'current' revision in addition to the 'last-changed' revision

I had one somewhat-unrelated question. You mention that externals should be avoided if possible - do you know of another way of managing complex sets of dependencies between subsystems without using externals? I am working in a C++ shop so we don't have the luxury of advanced dependency management tools like Maven, Nexus and such..... Externals seem like the only practical solution for this in our case.

Also, just so you know, all three of the "potential use cases" I outlined above we actually do use (or have tried) at our site... so they aren't really "potential" in that regard. And use case #1 is our most common usage pattern at the moment.

[kutzi@gmx.de (JIRA)]

kutzi commented on JENKINS-18907

Expose svn 'current' revision in addition to the 'last-changed' revision

Basically, the workflow I would propose is this: Fork the Jenkins repository at Github Clone your forked repository Create a new branch for the change Do the changes Send out the pull request See also https://wiki.jenkins-ci.org/display/JENKINS/Governance+Document#GovernanceDocument-pullrequest

[kutzi@gmx.de (JIRA)]

kutzi commented on JENKINS-18907 Expose svn 'current' revision in addition to the 'last-changed' revision

"do you know of another way of managing complex sets of dependencies between subsystems without using externals? I am working in a C++ shop so we don't have the luxury of advanced dependency management tools like Maven, Nexus and such" Dependency management would have been my recommendation... - It's vastly superior to externals IMO I don't know much about C++ development, so I cannot give you any advise how to do dep management there.

[kevin.phillips@caris.com (JIRA)]

Kevin Phillips commented on JENKINS-18907

Expose svn 'current' revision in addition to the 'last-changed' revision

I figured it was a long shot, but that is the story of my life Dependency management in C++ is a complex beast and I haven't found any good tools (and very few not-so-good tools) to do the job. Externals handle the basics code dependencies for us, and we use Visual Studio to handle some of the project / module dependencies but VS doesn't scale well to very-large-scale cross-team projects unfortunately (and it isn't cross platform either which sucks). I just mention all this because even if use case #1 should be avoided, I think it is a very real and important use case that isn't all that uncommon. I wouldn't discount it out-of-hand.