Using the current Branch in Git
Loic Matthey
I'm just starting to play with Sumatra, looks very good for now!
I'm using a Git repository, and using different branches for different
releases (and also when writing new features).
If I understand correctly, running Sumatra now assumes that the branch
is "master". Would it be possible to choose which one is active? For
example, using the current branch in the local repository.
Also, what does the "store-diff" option, as it seemed to be a way to
get around using commited code?
Thanks a lot
Loic
Andrew Davison
On Jul 2, 10:19 pm, Loic Matthey <azhag...@gmail.com> wrote:
> Hello,
>
> I'm just starting to play with Sumatra, looks very good for now!
>
> I'm using a Git repository, and using different branches for different
> releases (and also when writing new features).
>
> If I understand correctly, running Sumatra now assumes that the branch
> is "master".
> Would it be possible to choose which one is active? For
> example, using the current branch in the local repository.
seems like it should be straightforward to implement. Since I don't
use git much myself, can anyone else take a look at
http://neuralensemble.org/trac/sumatra/browser/src/versioncontrol/_git.py
and let me know what you think would be required?
> Also, what does the "store-diff" option, as it seemed to be a way to
> get around using commited code?
to have to commit every time you run a computation (especially if
using Subversion over a slow network), so if you use the "store-diff"
option Sumatra will store the diff for you. It is intended that you
still commit frequently, so that the stored diff is only small.
> Thanks a lot
> Loic
Cheers,
Andrew
Bartosz Telenczuk
>
> Not at the moment, but using the current branch rather than "master"
> seems like it should be straightforward to implement. Since I don't
> use git much myself, can anyone else take a look at
> http://neuralensemble.org/trac/sumatra/browser/src/versioncontrol/_git.py
> and let me know what you think would be required?
I have looked at the code and things are a bit complicated. "latest" version is supposed to be the last commit in the repository (at least this is the way it is implemented in mercurial and svn). However, in git there is no ordering on commits coming from different branches (unlike the changeset in mercurial) and it is not possible to decide which commit was the last. Therefore, now it is reasonably assumed that it is the last commit from the "master" branch.
In order to run simulations from different branches, there are two possibilities:
1) take a commit from the current branch (using "git merge origin" first to work in the remote changes). This may be a bit confusing since the user may checkout an older commit for testing and forget to switch it back before running a simulation.
2) use a command line argument to define the branch. This is theoretically possible with smt run -v 'branch' option, but now fails due to an assertion error (records.py:63 http://neuralensemble.org/trac/sumatra/browser/src/records.py#L63 - version number i.e. commit hash is not the same as branch identifier). However, it can be easily fixed by replacing the branch name with a respective commit hash.
Let me know what you think. If you decide on one of the above options, I may try to implement it.
Bartosz
Michele Mattioni
I'm using sumatra 0.1 still here (big legacy database and writing up
prevented me to upgrade) so I don't know if applies to new release but
let's say I have the repo
master
topic1
and they differ, to run the simulation on topic1 I have only to do
git co topic1
smt run default.param
and will automatically run the code on topic one, the commit will be
automatically recorded and it's easy to track it to topic 1 looking
into the log, or with gitg/gitk any visual git client which shows the
branches.
I used in the paste, and it's very useful.
To use the latest code of your current branch you just need to use
HEAD, which always points to the tip of your current branch.
Have a look here:
https://github.com/mattions/sumatra_on_github/blob/master/src/versioncontrol/_git.py#L62
My two cents
Best,
Michele.
Bartosz Telenczuk
>
> To use the latest code of your current branch you just need to use
> HEAD, which always points to the tip of your current branch.
>
> Have a look here:
> https://github.com/mattions/sumatra_on_github/blob/master/src/versioncontrol/_git.py#L62
It makes sense to me. Unfortunately, it is not consistent with the mercurial behaviour, which takes the "tip" of the repository. I am not an expert on mercurial, but I guess this is forced by mercurial "branches" implementation.
Cheers,
Bartek
> My two cents
>
> Best,
> Michele.
>
> On Sat, Jul 16, 2011 at 1:34 AM, Bartosz Telenczuk
> <b.tel...@biologie.hu-berlin.de> wrote:
>> Hi all,
>>
>>>
>>> Not at the moment, but using the current branch rather than "master"
>>> seems like it should be straightforward to implement. Since I don't
>>> use git much myself, can anyone else take a look at
>>> http://neuralensemble.org/trac/sumatra/browser/src/versioncontrol/_git.py
>>> and let me know what you think would be required?
>>
>> I have looked at the code and things are a bit complicated. "latest" version is supposed to be the last commit in the repository (at least this is the way it is implemented in mercurial and svn). However, in git there is no ordering on commits coming from different branches (unlike the changeset in mercurial) and it is not possible to decide which commit was the last. Therefore, now it is reasonably assumed that it is the last commit from the "master" branch.
>>
>> In order to run simulations from different branches, there are two possibilities:
>>
>> 1) take a commit from the current branch (using "git merge origin" first to work in the remote changes). This may be a bit confusing since the user may checkout an older commit for testing and forget to switch it back before running a simulation.
>>
>> 2) use a command line argument to define the branch. This is theoretically possible with smt run -v 'branch' option, but now fails due to an assertion error (records.py:63 http://neuralensemble.org/trac/sumatra/browser/src/records.py#L63 - version number i.e. commit hash is not the same as branch identifier). However, it can be easily fixed by replacing the branch name with a respective commit hash.
>>
>> Let me know what you think. If you decide on one of the above options, I may try to implement it.
>>
>> Bartosz
Bartosz Telenczuk
Institute for Theoretical Biology
Humboldt University of Berlin, Germany
Phone: +4930/2093-8838
Homepage: http://neuroscience.telenczuk.pl
Michele Mattioni
On Wed, Jul 20, 2011 at 12:29 PM, Bartosz Telenczuk
<b.tel...@biologie.hu-berlin.de> wrote:
> Hi Michele,
>
>>
>> To use the latest code of your current branch you just need to use
>> HEAD, which always points to the tip of your current branch.
>>
>> Have a look here:
>> https://github.com/mattions/sumatra_on_github/blob/master/src/versioncontrol/_git.py#L62
>
> It makes sense to me. Unfortunately, it is not consistent with the mercurial behaviour, which takes the "tip" of the repository. I am not an expert on mercurial, but I guess this is forced by mercurial "branches" implementation.
>
If I've got correctly in mercurial you are encouraged to have
different folder for different branches, and switch between them using
a classic cd. Therefore when you run sumatra, always the 'tip' of the
branch-folder is picked, which should be your latest code (disclaimer:
I used mercurial a bit, but I gave up pretty quickly and my knowledge
of it is super limited. Feel free to integrate correct)
In a git repo the situation is different, because several branches can
co-exist in the same folder, holding code as different as you like. To
switch between them
git checkout name_branch
is used.
There is not tip of the repo as such in a git repo. What actually
exists is HEAD, a symlink to the latest commit in the active branch.
Therefore running the HEAD commit would be 'consistent' with mercurial
approach, given the fact that the active branch has already been
picked by the user.
I have originally written the git port from sumatra and I'm using it
this way. I'm still running 0.1 for legacy reasons, and I can run
simulations from any branch I switch to, with any arbitrary name,
using the latest code in that branch, with the method I've proposed.
If you think is reasonable, and you are happy like that, I think it
would be wise to re-implement the HEAD checkout and leave it to the
user to switch to the desired branch, before running the classic
smt run default.param
Cheers,
Michele.
Bartosz Telenczuk
Thanks for implementing the git support. I have been using sumatra together with git for some time and I absolutely love it.
I think that we both agree that running the simulation from the HEAD is the way to go. Should we then go on with HEAD? Would you like to apply changes yourself? Otherwise I can do it.
Cheers,
Bartek
Bartosz Telenczuk
Bartosz Telenczuk
I found that the change from HEAD to master was introduced in commit '4dfbedeff0ee'
@Andrew: What was the motivation to use 'master' branch instead of HEAD?
Yours,
Bartek
On 20.07.2011, at 14:08, Michele Mattioni wrote:
Bartosz Telenczuk
Michele Mattioni
sorry for the misspelled name. My bad.
On Wed, Jul 20, 2011 at 1:28 PM, Bartosz Telenczuk
<b.tel...@biologie.hu-berlin.de> wrote:
> Hi Michele,
>
> Thanks for implementing the git support. I have been using sumatra together with git for some time and I absolutely love it.
>
> I think that we both agree that running the simulation from the HEAD is the way to go. Should we then go on with HEAD? Would you like to apply changes yourself? Otherwise I can do it.
>
I don't have an up-to-date copy of sumatra, so if you have it and it's
easy for you just go ahead.
I'm planning to catch up with the latest smt version ASAP I submit the thesis.
Cheers,
Michele.
Bartosz Telenczuk
> sorry for the misspelled name. My bad.
No problem.
> I don't have an up-to-date copy of sumatra, so if you have it and it's
> easy for you just go ahead.
> I'm planning to catch up with the latest smt version ASAP I submit the thesis.
Ok, I will do it. Good luck with your thesis. BTW I have just found your Neuronvisio software, cool stuff, I have to try it out.
Cheers,
Bartek
Andrew Davison
> I found that the change from HEAD to master was introduced in commit
> '4dfbedeff0ee'
>
> https://neuralensemble.org/trac/sumatra/browser/src/versioncontrol/_git.py?rev=140%3A06b533919a3c
> #L62
>
> @Andrew: What was the motivation to use 'master' branch instead of
> HEAD?
I'm afraid I can't remember.
One possible issue is that "smt repeat" should checkout the branch and
version that were used for the original simulation, run the
simulation, then change back to the starting branch/version. The
current GitRepository/GitWorkingCopy implementation does not handle
branch switching, so it is a little more complex than just replacing
'master' by 'HEAD'.
Michele Mattioni
On Wed, Jul 20, 2011 at 2:02 PM, Andrew Davison
<andrew....@unic.cnrs-gif.fr> wrote:
> On 20 juil. 11, at 14:44, Bartosz Telenczuk wrote:
>>
>> I found that the change from HEAD to master was introduced in commit
>> '4dfbedeff0ee'
>>
>>
>> https://neuralensemble.org/trac/sumatra/browser/src/versioncontrol/_git.py?rev=140%3A06b533919a3c#L62
>>
>> @Andrew: What was the motivation to use 'master' branch instead of HEAD?
>
> I'm afraid I can't remember.
>
> One possible issue is that "smt repeat" should checkout the branch and
> version that were used for the original simulation, run the simulation, then
> change back to the starting branch/version. The current
> GitRepository/GitWorkingCopy implementation does not handle branch
> switching, so it is a little more complex than just replacing 'master' by
> 'HEAD'.
>
if the repeat use the unique commit recorded during the run, there
should be no problem, because the branch switching is handled
automatically.
I can't figure out if this is what is happening during the repeat.
What is the commit used for the checkout here:
http://neuralensemble.org/trac/sumatra/browser/src/commands.py#L423 ?
Bartosz Telenczuk
>
> if the repeat use the unique commit recorded during the run, there
> should be no problem, because the branch switching is handled
> automatically.
I do not understand how the branch is switched automatically. If you checkout a commit with git you get a "detached HEAD" that is not associated with any branch. Therefore if you checkout HEAD again, you won't get the branch you were originally at, but this "detached HEAD".
>
> I can't figure out if this is what is happening during the repeat.
> What is the commit used for the checkout here:
> http://neuralensemble.org/trac/sumatra/browser/src/commands.py#L423 ?
The function will ultimately call Project.update_code (http://neuralensemble.org/trac/sumatra/browser/src/projects.py?rev=270%3Aa1ab3cb47706#L155) which in turn calls either GitWorkingCopy.use_latest_version or GitWorkingCopy.use_version. The latter creates a "detached HEAD" and leaves the working copy there. When simulation is ran next time, it will use the detached HEAD instead of the branch we have been previously working on.
I think that Andrew is right - we have to take care of branch switching. Unfortunately, this would require that we store the original branch name, but as far as I understand this is not easy to implement without changing the Project code.
Another possibility is to make GitWorkingCopy.use_version to clone a new working copy in a temporary directory, from which we may then run the simulation.
Finally, we might also use some git mastery to test whether we are in a detached HEAD and then use the git-reflog (i.e. git log -g -2 HEAD) to check the name of the original branch.
Any ideas?
Cheers,
Bartosz
Michele Mattioni
<b.tel...@biologie.hu-berlin.de> wrote:
>
>>
>> if the repeat use the unique commit recorded during the run, there
>> should be no problem, because the branch switching is handled
>> automatically.
>
> I do not understand how the branch is switched automatically. If you checkout a commit with git you get a "detached HEAD" that is not associated with any branch. Therefore if you checkout HEAD again, you won't get the branch you were originally at, but this "detached HEAD".
>
>>
>> I can't figure out if this is what is happening during the repeat.
>> What is the commit used for the checkout here:
>> http://neuralensemble.org/trac/sumatra/browser/src/commands.py#L423 ?
>
> The function will ultimately call Project.update_code (http://neuralensemble.org/trac/sumatra/browser/src/projects.py?rev=270%3Aa1ab3cb47706#L155) which in turn calls either GitWorkingCopy.use_latest_version or GitWorkingCopy.use_version. The latter creates a "detached HEAD" and leaves the working copy there. When simulation is ran next time, it will use the detached HEAD instead of the branch we have been previously working on.
>
> I think that Andrew is right - we have to take care of branch switching. Unfortunately, this would require that we store the original branch name, but as far as I understand this is not easy to implement without changing the Project code.
>
> Another possibility is to make GitWorkingCopy.use_version to clone a new working copy in a temporary directory, from which we may then run the simulation.
>
This sounds the more plausible, but just gut feeling...
> Finally, we might also use some git mastery to test whether we are in a detached HEAD and then use the git-reflog (i.e. git log -g -2 HEAD) to check the name of the original branch.
>
I have no clue! ;)
> Any ideas?
>
got no easy solution.
If I'll get an idea, I'll let you know.
For the time being reverting to HEAD gives at least the possibility to
run different branch with no problem. But yes, it leaves the repeat in
a broken state. (Does it work now? Did it ever worked? got to admit I
never tried.)
Cheers,
Michele.
Bartosz Telenczuk
>
> For the time being reverting to HEAD gives at least the possibility to
> run different branch with no problem. But yes, it leaves the repeat in
> a broken state. (Does it work now? Did it ever worked? got to admit I
> never tried.)
Some users might not be happy. If we are going to do it, we should do it the right way. It seems that we will need some unittests before we can even start. I will work on them in the next few days...
If you have any good ideas how to fix it, please let us know.
Cheers,
Bartek
Andrew Davison
On 20 juil. 11, at 17:17, Bartosz Telenczuk wrote:
>> For the time being reverting to HEAD gives at least the possibility
>> to
>> run different branch with no problem. But yes, it leaves the repeat
>> in
>> a broken state. (Does it work now? Did it ever worked? got to admit I
>> never tried.)
Yes, it works as long as you stay within the 'master' branch.
> Some users might not be happy. If we are going to do it, we should
> do it the right way. It seems that we will need some unittests
> before we can even start. I will work on them in the next few days...
There are some unit tests already (see http://neuralensemble.org/trac/sumatra/browser/test/unittests/test_versioncontrol.py#L226)
but it is always good to have more, or to improve the existing ones.
On 20 juil. 11, at 15:15, Michele Mattioni wrote:
> What is the commit used for the checkout here:
> http://neuralensemble.org/trac/sumatra/browser/src/commands.py#L423 ?
This is a terminology problem. "checkout()" here is in the Subversion
sense, which is equivalent to "clone" for Git/Mercurial.
Bartosz Telenczuk
> There are some unit tests already (see http://neuralensemble.org/trac/sumatra/browser/test/unittests/test_versioncontrol.py#L226) but it is always good to have more, or to improve the existing ones.
Perfect. We may add two unittests:
- one to check whether after smt repeat we stay in the same HEAD (now will fail if not in master)
- another one to test whether the simulation was ran from the current branch (now will fail if the current branch different from master)
For this, we will need a new branch in the example repository.
Cheers,
Bartek
>
>
> On 20 juil. 11, at 15:15, Michele Mattioni wrote:
>> What is the commit used for the checkout here:
>> http://neuralensemble.org/trac/sumatra/browser/src/commands.py#L423 ?
>
> This is a terminology problem. "checkout()" here is in the Subversion sense, which is equivalent to "clone" for Git/Mercurial.
>
>
>
>
Bartosz Telenczuk
Bartosz Telenczuk
As promised, I implemented the unittests to test git branches support. I have added two test to TestGitWorkingCopy:
test__use_current_branch - checks whether the sumatra uses the code in the current branch
test__latest_version_after_repeat - checks whether the branch is switched back to the original after checking out old revisions
I also updated the implementation of git classes to include support for custom branches (other than "master"). Currently, the current branch is used for running simulation. However after smt repeat the repository "forgets" the original branch and folds back to the "master" branch. Therefore, test__use_current_branch passes and test__latest_version_after_repeat fails.
The solution is a sort of compromise between simplicity and functionality - it allows for multiple branches, but does not store the original branch name before repeat, which would require more coding. The latter could still be implemented using git reflog's, but the implementation would be more complex and it would overload my free weekend resources.
You can checkout the code for testing from bitbucket:
https://bitbucket.org/btel/sumatra_devel
Let me know what you think. If you are happy with it I will push the code to the main sumatra repository.
Yours,
Bartosz
Bartosz Telenczuk
Andrew Davison
On Jul 26, 12:04 pm, Bartosz Telenczuk <b.telenc...@biologie.hu-
> As promised, I implemented the unittests to test git branches support. I have added two test to TestGitWorkingCopy:
>
> test__use_current_branch - checks whether the sumatra uses the code in the current branch
>
> test__latest_version_after_repeat - checks whether the branch is switched back to the original after checking out old revisions
>
> I also updated the implementation of git classes to include support for custom branches (other than "master"). Currently, the current branch is used for running simulation. However after smt repeat the repository "forgets" the original branch and folds back to the "master" branch. Therefore, test__use_current_branch passes and test__latest_version_after_repeat fails.
>
> The solution is a sort of compromise between simplicity and functionality - it allows for multiple branches, but does not store the original branch name before repeat, which would require more coding. The latter could still be implemented using git reflog's, but the implementation would be more complex and it would overload my free weekend resources.
>
> You can checkout the code for testing from bitbucket:
>
> https://bitbucket.org/btel/sumatra_devel
>
> Let me know what you think. If you are happy with it I will push the code to the main sumatra repository.
would be great if Loic (who started this thread) and Michele could
give Bartosz's branch a try, and check it works for them.
Cheers,
Andrew
JT Bates
That's great Bartosz. Since I don't use git in my day-to-day work, it
would be great if Loic (who started this thread) and Michele could
give Bartosz's branch a try, and check it works for them.
Has there been any further progress implementing this feature?
Michele Mattioni
didn't have time to test it, 'cause didn't upgrade yet to the latest
to not loose the DB. Need to close the project first.
I guess Bartosz has more exp on this.
Cheers,
Michele.
Bartosz Telenczuk
Recently, I have not been very actively involved in sumatra development. Apparently, this changeset was not merged into the main repository. I guess this is because of lack of user testing.
You can try it out from my sumatra repo at bitbucket: https://bitbucket.org/btel/sumatra_devel . I merged all recent additions to sumatra, so it is at the most recent development version + my commits.
Let me know if it works for you.
Cheers,
Bartosz
JT Bates
Let me know if it works for you.
Bartosz Telenczuk
> Let me know if it works for you.
>
> I haven't tested extensively but it seems to be working correctly.
Cool. Please report if you come across any problem. If we have some more people testing it, we might finally merge it with main sumatra tree.
Cheers,
Bartosz