Maintaining a unique Build Number across multiple build servers

[Robert Kruck]

 Is it possible to preserve the integrity of build numbers (NO DUPLICATES and build numbers in order) while building in multiple Jenkins build servers?

If this capability exists in Jenkins, what Jenkins plugins are required, and what versions of Jenkins itself, and of the required Jenkins plugins, are needed?

[Cuong Tran]

It doesn't exist.

[Michael Neale]

Not sure if it may help solve a similar aim - but https://wiki.jenkins-ci.org/display/JENKINS/Version+Number+Plugin exists (might be others)

[Robert Kruck]

On Monday, September 19, 2016 at 2:54:15 PM UTC-5, Robert Kruck wrote:

 Is it possible to preserve the integrity of build numbers (NO DUPLICATES and build numbers in order) while building in multiple Jenkins build servers?

If this capability exists in Jenkins, what Jenkins plugins are required, and what versions of Jenkins itself, and of the required Jenkins plugins, are needed?

Would the simplest solution to maintain build number integrity across multiple Jenkins servers be to:

a) Create a versioned ini file (in our case, a StarTeam file in a *.ini format), containing the latest 5-digit build number.

b) The above StarTeam file would be visible to all our build scripts, no matter what Jenkins server they were located in.

c) When a build script is under way, it would do an EXCLUSIVE checkout of the StarTeam build number ini file. (Perhaps a wait time could be added to this checkout step, in case the above file was checked-out & locked at the time by another build script).

d) The build script increments the build number in the build number ini file using WriteINI, and uses this incremented build number for its build.

e) Finally, the build script checks in the StarTeam build number ini file, so it is unlocked / correctly updated / and visible to other build scripts in other Jenkins servers.

1) Obtaining the build number via a versioned *.ini file would seem to have no problems for an interactive, non-Jenkins build.

2) Would obtaining the build number via a versioned *.ini file, rather than a build number generated by Jenkins, be practical in a build done through Jenkins?

3) Is the above "versioned build number *.ini file" concept reasonable, or would use of Jenkins plugin(s) & Windows Environment variables be the best way to reliably increment, then communicate, build numbers across multiple Jenkins servers?

[Michael Lasevich]

There are many approaches to this problem, and I am not sure what your setup or requirements are, but here is some information that may be helpful.

* It is important to understand that any jenkins build has two ids - the actual immutable build_id - which is always consecutive for a job and specific to that job alone, and the build display name which is what you see in the UI. You can set the build display name to anything you want from within your job. By default it is # (pound symbol) followed by build_id.

* You are usually better off having one build server with many slaves than parallel build servers that are independent of each other. This way as long as you are building the same job, it does not matter what slave you are building on, your build ids will be consecutive and unique.

* If you do need multiple jobs on same master (even if builds are performed on any slave), you can write some code that will issue numbers from same pool. You can back that pool by anything, the simplest thing being writing a file to disk on master. I have implemented this many times via writing my own plugin, but these days it is far easier to do this via a pipelines shared library (if you are using pipelines, and you should). The simplest thing to do is to have a function that takes some id, and for that id it issues you next number by incrementing a synchronized counter somewhere. Of course this will still keep you on a single Jenkins master...

* If you want to do this across multiple masters as you indicated (and I would strongly urge you to reconsider) - you can use the above with some sort of a shared storage resource that either supports atomic changes or locking - for example you can use a shared MySQL DB. You just have a table you lock and increment a counter in while locked (there are many other approaches too)

* If you do not care for keeping a shared counter somewhere and do not mind longer build ids, a cheap and easy way to go is to assign each jenkins server an id number, then use a build name consisting of timestamp followed by server id, followed by whatever you want to guarantee it is unique within a single server. Your build ids will be long, but will always be in a sequential order - even if not consecutive.

* All that said, I would also reconsider overall value of consecutive build numbers - in the end you probably only care about what branch it was done from and which commit it was done from. If you use a SCM that provides you with a sequential commit id (SVN is awesome for this) - just use the commit id as your build indicator. If you are using git or any other non-sequential commit SCM - you can just include the hash of the SCM on build and to get sequence, you can cheat a little by also counting commits in the log. Last bit is not exactly great as git allows you to rewrite your commit history, but if you can lock it down, it is reasonably functional indicator.

Good luck,

-M

On Monday, September 19, 2016 at 12:54:15 PM UTC-7, Robert Kruck wrote: