Git branching organization on Example42 modules

[Alessandro Franceschi]

Hallo all,

I write here to have some suggestions on how to manage with git the different versions of the Example42 modules.
Some of the topics here are related to the 'StdMod: Git, Branching, Versioning, Backwards Compatibility, Repos' thread but I prefer to open a new thread, strictly related on how future Example42 git repos are going to be organised, that can remain for future reference: various things are changing and some confusion may arise.

I've just made some commits on GitHub with on these "collection" repos (the ones that link different Puppet modules as git submodules):

https://github.com/example42/puppet-modules - Is the main modules repo, I've just removed all the "old" 1.x modules (I'd not suggest them to anyone now) and the intention is to gather here as submodules both 2.x and 3.x modules pointing always to the master branch and so (read later) switching from 2.x to 3.x when a module is ready

https://github.com/example42/puppet-modules-nextgen - Contains only 2.x modules. It's the one I'm currently using in my production environments.

https://github.com/example42/puppet-modules-stdmod - Contains only 3.x modules (Consider this as "my" selection of stdmod modules, which might contain both modules from example42  and others at the desired state)

As written in the mentioned thread we may have different collections where different authors link modules at different commits and manage test and code status at their own will.

We may have also an "official" stdmod collection (like this one https://github.com/stdmod/puppet-modules ) but I don't intend to discuss here how to organise things in Github's stdmod.

These are the "containers" or "collection" repos, which are more or less a set of submodules to different modules at different branches/commits.

Most ex42 actual modules repos, like https://github.com/example42/puppet-tomcat have now just one branch, master, where all commits and pull requests are accepted.

To organise the development in a more controlled way and to manage 2.x and 3.x versions I'm considering this approach for each individual module:

- Introduce a branch called 2 (or 2.x) to manage nextgen modules ( https://github.com/example42/puppet-modules-nextgen submodules will always point to branch 2 (now they point to master) )

- Introduce a branch called 3 (or 3.x) to manage stdmod modules ( https://github.com/example42/puppet-modules-stdmod submodules will always point to branch 3 )

- Have master pointing always to the latest "stable" version of the module ( https://github.com/example42/puppet-modules will point to master branches, and therefore switch from time to time from 2. to 3.x)
- Eventually introduce a devel branch where to test commits that have to be merged on master

I'm not a git ninja and not an experienced and disciplined developer so I'm bit worried at the idea of managing correctly branches and keeping them aligned.
In particular I wonder if there's a sane way, with hooks or whatever, to keep bidirectionally synced the master branch with the "currently active" release branch.

The rationale is that I have a 2.x only module and want master to be synced always to branch 2
Then I can start to work on 3 and when I think it's ready I want master to be synced to branch 3
This syncing should be bidirectional: commits to master should be merged in the currently "active" branch, for example 2, and commits to branch 2, if it's active, should be merged automatically on master.
When I want to switch master to 3 and start syncing commits from master to 3 and vice versa I should just edit/touch a single file/thing.

Does this approach make sense for you?
Would it make sense and be usable for end users?
Do you have some hooks/samples/suggestions on how to do that?

Should this be done via GitHub, in git local repos or both?

Any suggestion or comment definitively welcomed

al

[Freeaqingme]

Hi Al,

> https://github.com/example42/puppet-modules-stdmod - Contains only 3.x modules (Consider this as "my" selection of stdmod modules, which might contain both modules from example42 and others at the desired state)

I personally don't use the collection repos myself. And given that they are your personal selection of modules, I'd say you really should do whatever you want with it. It does touch lightly on an other topic I'm unsure about though. In the past Example42 was by many regarded as a github organization or as a stable puppet framework. Now we're setting up the framework under StdMod, Example42 will become more and more your personal playground I think(?).

> To organise the development in a more controlled way and to manage 2.x and 3.x versions I'm considering this approach for each individual module:
> - Introduce a branch called 2 (or 2.x) to manage nextgen modules ( https://github.com/example42/puppet-modules-nextgen submodules will always point to branch 2 (now they point to master) )
> - Introduce a branch called 3 (or 3.x) to manage stdmod modules ( https://github.com/example42/puppet-modules-stdmod submodules will always point to branch 3 )
> - Have master pointing always to the latest "stable" version of the module ( https://github.com/example42/puppet-modules will point to master branches, and therefore switch from time to time from 2. to 3.x)
> - Eventually introduce a devel branch where to test commits that have to be merged on master

I think something along these lines really is a good idea. From my experience as a software developer, I think I can say that this model is most commonly used: http://nvie.com/posts/a-successful-git-branching-model/  (forget about hotfixes for now, they're just there to make stuff difficult, and hardly apply to example42). The model is based on two main concepts: The master branch always points to the latest stable release and the develop branch always points to the unstable thing that may brake things. In case of Example42 I'd do the following:

- Introduce a branch called 2.$Y ($Y is replaced by the next available minor version), put in there what's currently known as 'nextgen' (lets start calling that version 2 while we're on it ;)).Maintain full backwards compatibility on this branch.

-- If there are any major new functions that should be part of the version two release, we create a branch 2.$Y ($Y is replaced by the next available minor version) where that can be developed.

- Develop the next major version on the branch named "develop" (I wouldnt use 'devel', only RHEL folks use that ;)).

-- Once we're ready to release the next major version (version 3), we create a branch 3.x, and we start developing on that.

- Every release is tagged (naming scheme "v$X.$Y.$Z").

-- Whenever we tag something, the master branch is pointed to the most recently tagged commit.

> I'm not a git ninja and not an experienced and disciplined developer so I'm bit worried at the idea of managing correctly branches and keeping them aligned.

I'd suggest writing a script that runs all spec tests, creates the tag, pushes the tag to github, updates the ref of the master branch, and automatically uploads the latest version to Puppet Forge (and maybe update those collection repos?). We could write such a script once, and simply execute it for all releases we want to create.

> The rationale is that I have a 2.x only module and want master to be synced always to branch 2
> Then I can start to work on 3 and when I think it's ready I want master to be synced to branch 3
> This syncing should be bidirectional: commits to master should be merged in the currently "active" branch, for example 2, and commits to branch 2, if it's active, should be merged automatically on master.
> When I want to switch master to 3 and start syncing commits from master to 3 and vice versa I should just edit/touch a single file/thing.

I think these issues have been addressed by the scheme above.Sticking to the scheme should work for end users, as it's commonly used and easily recognized. 

Regards,

Dolf

-- Freeaqingme

[Alessandro Franceschi]

Il giorno mercoledì 16 ottobre 2013 14:54:12 UTC+2, Freeaqingme ha scritto:

Hi Al,

> https://github.com/example42/puppet-modules-stdmod - Contains only 3.x modules (Consider this as "my" selection of stdmod modules, which might contain both modules from example42 and others at the desired state)

I personally don't use the collection repos myself. And given that they are your personal selection of modules, I'd say you really should do whatever you want with it. It does touch lightly on an other topic I'm unsure about though. In the past Example42 was by many regarded as a github organization or as a stable puppet framework. Now we're setting up the framework under StdMod, Example42 will become more and more your personal playground I think(?).

Well, it will become, as it is now, the modules set I propose to my customers or use for my projects, with my opinionated additions to make, at least my, life easier.

> To organise the development in a more controlled way and to manage 2.x and 3.x versions I'm considering this approach for each individual module:
> - Introduce a branch called 2 (or 2.x) to manage nextgen modules ( https://github.com/example42/puppet-modules-nextgen submodules will always point to branch 2 (now they point to master) )
> - Introduce a branch called 3 (or 3.x) to manage stdmod modules ( https://github.com/example42/puppet-modules-stdmod submodules will always point to branch 3 )
> - Have master pointing always to the latest "stable" version of the module ( https://github.com/example42/puppet-modules will point to master branches, and therefore switch from time to time from 2. to 3.x)
> - Eventually introduce a devel branch where to test commits that have to be merged on master

I think something along these lines really is a good idea. From my experience as a software developer, I think I can say that this model is most commonly used: http://nvie.com/posts/a-successful-git-branching-model/  (forget about hotfixes for now, they're just there to make stuff difficult, and hardly apply to example42). The model is based on two main concepts: The master branch always points to the latest stable release and the develop branch always points to the unstable thing that may brake things. In case of Example42 I'd do the following:

- Introduce a branch called 2.$Y ($Y is replaced by the next available minor version), put in there what's currently known as 'nextgen' (lets start calling that version 2 while we're on it ;)).Maintain full backwards compatibility on this branch.

-- If there are any major new functions that should be part of the version two release, we create a branch 2.$Y ($Y is replaced by the next available minor version) where that can be developed.

- Develop the next major version on the branch named "develop" (I wouldnt use 'devel', only RHEL folks use that ;)).

-- Once we're ready to release the next major version (version 3), we create a branch 3.x, and we start developing on that.

- Every release is tagged (naming scheme "v$X.$Y.$Z").

-- Whenever we tag something, the master branch is pointed to the most recently tagged commit.

Good. Will do this.

 

> I'm not a git ninja and not an experienced and disciplined developer so I'm bit worried at the idea of managing correctly branches and keeping them aligned.

I'd suggest writing a script that runs all spec tests, creates the tag, pushes the tag to github, updates the ref of the master branch, and automatically uploads the latest version to Puppet Forge (and maybe update those collection repos?). We could write such a script once, and simply execute it for all releases we want to create.

I'm currently using this:
https://github.com/example42/Example42-tools/blob/master/publish.sh
which does more or less those tasks except managing branches, master and the collection repos.

Any addition, which could be reused also for stdmod is welcomed.

 

> The rationale is that I have a 2.x only module and want master to be synced always to branch 2
> Then I can start to work on 3 and when I think it's ready I want master to be synced to branch 3
> This syncing should be bidirectional: commits to master should be merged in the currently "active" branch, for example 2, and commits to branch 2, if it's active, should be merged automatically on master.
> When I want to switch master to 3 and start syncing commits from master to 3 and vice versa I should just edit/touch a single file/thing.

I think these issues have been addressed by the scheme above.Sticking to the scheme should work for end users, as it's commonly used and easily recognized. 

Regards,

Thank yoiu for the feedback. Quite useful.

al