Build on Pre-Commit

[San Martino]

Hello,

I know this has been asked already, but we would really like to reject any attempt to commit files which would break a project in trunk. The decision to reject the committed files is based on the result of the building process of the project the files being committed belong to. I know that during the pre-commit phase the repository cannot be accessed concurrently, but this is not a problem for us since our buildings are really fast and we can tolerate any delay involved.

Are there any tools to achieve what we want? Note that it will be necessary ro recompile the whole project unit, not just the single files being committed.

Thanks.

[Stefan Sperling]

It sounds like you need a working copy for building.
I don't think there is a way to construct a useful working copy
from a transaction.

You'd have to use some fragile construct such as a working copy
of HEAD with local changes which are pulled from the transacion
with the help of svnlook. This is inherently racy, because
transactions run concurrently and HEAD could change during your
test build.

Essentially, doing what you have in mind really isn't supposed to be done.

Instead, you could forbid direct commits to trunk, and only merge changes
from branches that have been vetted. Have you considered that approach?
Everything is transactional, and you always have a working trunk build.
Your pre-commit hook could reject attempted commits to trunk which do
not tweak svn:mergeinfo, and possibly even verify that the merge is
merging changes from an approved branch by checking the mergeinfo and
comparing it to a list of branches which may merge into trunk.

[San Martino]

We already use branches, but we use/justify them for developping long features or bug-fixes which require some time and are not urgent.

In many cases, however, we must urgently fix small, short and important bugs and release everything immediatly.. We act on trunk directly for this and for this reason we shoule be sure that trunk is always ok. Opening a branch for each fix instead of using a validation on pre-commit makes the whole procedure more heavy does not guarantee that trunk is always in a "correct" state in any case. While searching on internet I found two "Pre-Commit CI" :

http://www.pmease.com/features/continuous-integration.html#proof-build
http://www.jetbrains.com/teamcity/features/delayed_commit.html

I have not read the documentation, but it appears they are near to what we want

What do you think about them?

2013/5/24 Stefan Sperling <st...@elego.de>

[Stefan Sperling]

On Fri, May 24, 2013 at 12:05:19PM +0200, San Martino wrote:
> We already use branches, but we use/justify them for developping long
> features or bug-fixes which require some time and are not urgent.
> In many cases, however, we must urgently fix small, short and important
> bugs and release everything immediatly.. We act on trunk directly for this
> and for this reason we shoule be sure that trunk is always ok. Opening a
> branch for each fix instead of using a validation on pre-commit makes the
> whole procedure more heavy does not guarantee that trunk is always in a
> "correct" state in any case.

The overhead isn't very high:

svn checkout $TRUNK_URL my-fix
cd my-fix

edit files, test, build locally
svn copy ^/trunk ^/branches/my-fix-branch
svn switch ^/branches/my-fix-branch
svn commit -m "commit my fix to a branch"

my-fix-branch is tested / test fails
edit files, test, build locally
svn commit -m "fix the fix"

my-fix-branch is tested / approved
svn switch ^/trunk
svn merge --reintegrate ^/branches/my-fix-branch
svn commit -m "merge my-fix-branch to trunk"

In this example, the extra steps are a copy, two switches, and one merge.
Also, intermediate states of the fix are saved into the repository.

Conflicts during the reintegrate merge are unlikely because the
branch is short-lived. With your suggested approach, conflicts
aren't any less likely and would need to be dealt with during
updates of the trunk working copy.

BTW, with a Subversion 1.8 client, the step:

svn merge --reintegrate ^/branches/my-fix-branch

becomes just:

svn merge ^/branches/my-fix-branch

> While searching on internet I found two
> "Pre-Commit CI" :
>
> http://www.pmease.com/features/continuous-integration.html#proof-build
> http://www.jetbrains.com/teamcity/features/delayed_commit.html
>
> I have not read the documentation, but it appears they are near to what we
> want
> What do you think about them?

Sounds like tools that run a build based on a working copy with
some local changes applied to that working copy.

If your continuous integration software supports building a revision
plus local changes (e.g. in the form of patches submitted to the CI
software by the developer somehow), you could ask your developers to
perform such a test build before commit.

However, if your tooling isn't too complex to fit onto a single
developer machine, then I wonder where the advantage is over
having the developer run a local build before a commit.
Which generally is the expected behaviour of committers, anyway.

[Philip Martin]

San Martino <sanm...@gmail.com> writes:

> I know that
> during the pre-commit phase the repository cannot be accessed concurrently,

That's not right. The pre-commit runs before taking the repository lock
so other access, including other commits, can occur while the pre-commit
is running. And that means that the pre-commit cannot guarantee
anything about files that are not part of the commit.

--
Certified & Supported Apache Subversion Downloads:
http://www.wandisco.com/subversion/download

[Les Mikesell]

On Fri, May 24, 2013 at 5:05 AM, San Martino <sanm...@gmail.com> wrote:
> We already use branches, but we use/justify them for developping long
> features or bug-fixes which require some time and are not urgent.
> In many cases, however, we must urgently fix small, short and important bugs
> and release everything immediatly.. We act on trunk directly for this and
> for this reason we shoule be sure that trunk is always ok. Opening a branch
> for each fix instead of using a validation on pre-commit makes the whole
> procedure more heavy does not guarantee that trunk is always in a "correct"
> state in any case. While searching on internet I found two "Pre-Commit CI" :
>
> http://www.pmease.com/features/continuous-integration.html#proof-build
> http://www.jetbrains.com/teamcity/features/delayed_commit.html
>
> I have not read the documentation, but it appears they are near to what we
> want
> What do you think about them?

Have you considered using release branches and accepting the
possibility of a 'dirty' trunk? Projects that have to support
multiple release versions concurrently are pretty much forced to do it
that way, but it is not particularly cumbersome even if you only have
one revision deployed at a time now. And the trunk seems like the
right place to have untested development work happening.

--
Les Mikesell
lesmi...@gmail.com

[Bob Archer]

I don't think there is ever a 100% certainty that a commit won't break a build even if that commit was ok on another branch.

What is the problem you are trying to solve by doing this? One major point of continuous integration is that your builds break early, giving instant feedback so the dev can fix this issue.

You can "move" the problem to a branch. All commits go to a branch and after the branch builds your build server merges that branch to trunk (or whatever) but that may still break the trunk build.

This is best dealt with procedures. Do your devs have the same build.bat file (or whatever) that the build server does? They should... this way they can...

1. Make code changes test locally.
2. Update their working copy, run same build/test local that the build server runs.
3. If step two is successful commit to the server.

BOb