extending branch state for wstool info

[tkruse]

Hi,

based on a feature request, I am trying to extend the data displayed by wstool info. I would like some opinions on problematic issues.

The general idea is for wstool info to point to dirtiness in your workspace. Dirtiness could mean for git working on a different branch than the one specified in the .rosinstall, working against a different remote, or the branch tracking a differently named branch, or working against a different remote than 'origin'. [1]

In short, wstool info could report something like this:

 Localname S SCM  Version (Spec)                
 --------- - ---- --------------                
 repo0       git  master
 repo1     C git  master
 repo2       git  <detached>  (-)
 repo3       git  foo  (bar)
 repo4     V git  baz < remote2/bar  (foo)

repo0 is on master, should be on master, tracking origin/master, is up-to-date
repo1 has diverged with its remote, hence a 'C'.
repo2 is not on a branch, nor does the .rosinstall declare one.
repo3 is on branch foo, but ought to be on branch bar.
repo4 is on branch baz tracking remote2/bar, but should follow foo.

I would hope this new information would make it easier to check for workspace sanity at a glance.

I currently have 2 problems to decide.

1. When to refresh remote data (i.e. git fetch)? Fetching takes a comparatively long time, currently wstool info only uses local information. Should this be an extra command (wstool fetch), or an option (wstool info --fetch), or done always with wstool info (maybe with opt-out option --no-fetch)?

I tend towards a --fetch option.


2. I try to offer similar support for mercurial, but Mercurial has the competing concepts of named branches and bookmarks. Supporting both is a mess, and to support only one I have to make a choice. Bitbucket as an example seems to treat named branches as the favourite way to go, but apparently has some kind of smudgy way to also show bookmarks as branches [2, 3].
The mercurial branch command however prints this information: "(branches are permanent and global, did you want a bookmark?)".
My guess is that some people might use named branches for commit sequences never to be merged like '2.0', '2.1', "indigo-devel', ... and booksmarks for commit sequences to be merged soon like 'fix35', 'featureABC'. However bitbucket support for merge requests based on bookmarks is poor. [4]
So the alternatives are to support: named branches only, bookmarks only, a mix of named branches and bookmarks (display both).

I tend towards supporting named branches only (just because they have a longer tradition). I don't even know if there are enough Mercurial users in the ROS world for this to matter either way.


Opinions?


cheers,
  Thibault



[1]: https://github.com/vcstools/wstool/pull/35
[2]: https://answers.atlassian.com/questions/119488/what-are-branches-in-bitbucket-and-how-to-manage-them
[3]: https://bitbucket.org/site/master/issues/3182/mercurial-bookmarks-should-show-up-in-the
[4]: https://stackoverflow.com/questions/21726865/how-to-create-pull-request-from-mercurial-bookmark-on-bitbucket

[Jack O'Quin]

+1 seems like a good idea. 

Specifically:

1. I tend to prefer the ``wstool info --fetch`` approach, too.

2. I hardly ever use Mercurial, so no opinion about it.

--

 joq

[Thomas Denewiler]

1. For git I prefer to opt-in for this feature, so ``wstool info --fetch``. There are many times when wstool can be useful even without a connection to the main git remotes. Also, if users have written scripts that include wstool outputs then the opt-in behavior will not force them to (potentially) modify working scripts.

2.  I do not have a preference for Hg.

I would like some clarification. Is this proposal intending to modify display of the hashes for git repos? That is a feature that I really like to have available, so I hope that the wstool report you show is just a snippet and not the entire output.

Thanks,

[William Woodall]

I also prefer having an explicit `--fetch` option.

I'm not super familiar with HG's workflow to know which is preferable. Matching as closely as possible to BitBucket seems as good an idea as any.

--
You received this message because you are subscribed to the Google Groups "ROS Buildsystem Special Interest Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ros-sig-buildsy...@googlegroups.com.
To post to this group, send email to ros-sig-b...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ros-sig-buildsystem/14706c65-1e0d-499c-b1d3-d65cdf211f65%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

William Woodall

ROS Development Team

wil...@osrfoundation.org

http://wjwwood.io/

[tkruse]

On Sunday, July 26, 2015 at 9:53:24 PM UTC+2, Thomas Denewiler wrote:

I would like some clarification. Is this proposal intending to modify display of the hashes for git repos? That is a feature that I really like to have available, so I hope that the wstool report you show is just a snippet and not the entire output.

Thanks,

Hi Thomas,

sorry, the shown snippet was shortened to show the changes only, the rest of the output (hashes, urls) will remain as it was.

regards,
  Thibault

[tkruse]

As an example, here is the real output of the current prototype:

$ wstool info --fetch
workspace: /home/kruset/work/python/wstool/ws2

 Localname S SCM  Version (Spec) UID  (Spec)  URI  (Spec) [http(s)://...]
 --------- - ---- -------------- -----------  ---------------------------
 vcstools   git  master  (-)    cf774a270811 g...@github.com:vcstools/vcstools.git
 ccv-code   svn  trunk          -r19         svn://svn.code.sf.net/p/ccv/code/
 javahg     hg   default  (-)   6bf643f82436 bitbucket.org/aragost/javahg
 tomdroid   bzr                 324          lp:tomdroid


Regarding --fetch, actually, I believe this can be parallelized much more than using something like the number of CPUs. Since the parallel threads do mostly network / IO, CPU activity should be of no concern. Actually the same goes for the ROS installation instructions (http://wiki.ros.org/jade/Installation/Source)

while it currently says:
$ wstool init -j8 

I see no reason why this could not be changed to say 50, (unless I am missing something).
The same would go for the diff and status commands, I think.

I should still keep --fetch, since some fetches can be really slow.



There is another small problem to consider. The change would print 'C' as status flag if local and remote have changed, basically to indicate the user should sync with the remote. The problem is whether to use the remote 'origin' for that, or the remote the current branch is tracking.
The update function also currently uses 'origin' always, which we could change as well.

Conceptually, the .rosinstall files specifies what the local checkouts should be like, the URL given is to be the git remote 'origin', and it would be valid semantics to only support that remote.

However if the user went to his local repository and added a remote and tracked a branch on that remote, it is unclear conceptually what wstool should do. In the worst case 'origin' might not have the current branch, or a branch with the same name on 'origin' might be unrelated to the tracked branch on the alternative remote.

For wstool info, since we start displaying non-'origin' remotes, I could live with evaluating the status flag "C" against the tracked remote rather than origin.
For wstool update, I am not sure (in particular since now this would change historically established behavior, though undocumented).

Since I am not sure how many people will ever use branches tracking other remotes and notice the "C", the decision is probably not that dramatic.

Thoughts?

[Tully Foote]

Since we don't have the ability to instrument which remote we track I think making all the different functions consistently use 'origin' would be the least confusing for the user. If they manually interact with another remote it won't bother the tool. And if they want to change which one the tool is interacting with they can change the url of origin.

Tully

--

You received this message because you are subscribed to the Google Groups "ROS Buildsystem Special Interest Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ros-sig-buildsy...@googlegroups.com.
To post to this group, send email to ros-sig-b...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ros-sig-buildsystem/1c13f5af-2050-4d4f-97b4-df7a970b0d08%40googlegroups.com.

[Kentaro Wada]

I think showing tracking branch at version column has no problem and not confusing for users.

And in my opinion, C flag should mean different version between current branch and tracking branch of specified branch not that of current branch.

So In this case, there is no need to change wstool update, and see not origin remote only in function which is used for version column in wstool info.

[tkruse]

Hm, this get's a bit difficult, not sure which place is best to discuss this.

The current branch can have a tracking branch on a different remote.
The "specified branch" is just a label, it may not have a local branch, and even if, it might not have a tracking branch.

So "master" could be the specified version, but no local master branch might exist (yet).

So let us assume that the specified version is "master", and the current branch is "devel", tracking "remote2/devel".
And let us assume that the local repository also has a master branch, that tracks "remote2/master".

So the user is working on branch "devel" right now. Then the question is whether the user is most interested in divergence between
CO - "devel" and "origin/devel"  (current branch vs. same name branch on origin)
CT - "devel" and "remote2/devel" (current branch vs. same name branch on tracked remote)
CSO - "devel" and "origin/master" (current branch vs. specified branch on origin)
ST - "master" and "remote2/master" (specified branch vs. specified branch on tracked remote)
SO - "master" and "origin/master" (specified branch vs. specified branch on origin)


Again, for D and E, the specified branch might not exist locally in other scenarios.
Given there are 5 reasonable alternatives (not sure if I missed a useful one), it is difficult to estimate which alternatives will surprise what percentage of users. However, "C" could be used for more than one of those alternatives (using OR operant).

It is also possible that there is no specified branch (which implies checkout will used the default configured branch, but later updates stay on any current branch).

In that case we have these possibilities, considering divergence between:
CO - "devel" and "origin/devel"  (current branch vs. same name branch on origin)
CT - "devel" and "remote2/devel" (current branch vs. same name branch on tracked remote)
I - ignore divergences, as nothing is specified

I would like the status flag "C" to appear only in situations that are interesting to the user. So if the user is working on branch "devel", but "master" is specified, then it's a normal situation that there is a divergence, so the user does not need an additional "C" flag to remind him he's on a different branch. I believe that eliminates alternative CSO.

Similarly, if the user is working on devel, it is not so interesting what happens between the master branches on local and either remote. They may diverge often, the user works on "devel" because the "devel" branch has features he currently cares about. So even if "master" is the specified version, I believe a C there will not often tell the user something he is interested in, so I am not in favor of ST and SO.

Also CSO, ST and SO are only viable if there is a specified version, and I think it would not be nice to have different behaviors of the C flag depending on whether a version is specified or not (note also that the specified version can also be a tag or an SHA). Also these alternatives require a user to take into account even more potential causes for a "C" than CS and CO.

So that's why I believe only CO and CS are realistic candidates for the 'C' flag. The user works on "devel" and he wants to see at a glance whether there is something someone else did on "devel", or something he has to publish on "devel" for other people to see.

Like Tully, I am somewhat inclined towards saying an "origin"-centered workflow is the default that wstool strives to support (meaning CO is best). However, in most such cases users would be tracking branches on "origin", so by supporting a different tracked remote, we do not break the workflow of most users.

There is a realistic situation of multiple users without push permissions on 'origin' collaborating on a feature. Say the people of Lab XYZ want to completely change move_base, but the move_base maintainers do not trust Lab XYZ, so the people of Lab XYZ prepare their pull requests collaboratively on their own fork. In this case they might want to have 'origin' point to the one true move_base, but have local branches tracking their fork. I believe the alternative CS supports this use-case without interrupting the 'origin'-centered workflow that much.


We can also try to validate this with other scenarios. The user has a specified branch of indigo-devel, but is working on jade-devel. Is he interested in divergences to jade-devel, or to divergences to indigo-devel? I believe only to jade-devel.

So Kentaro, I believe what you were suggesting was alternative ST. Maybe you can present the workflow of your team (how you name your branches and remotes), so that we might understand why you mostly want to see the status flag "C" in divergent situations like that.

[Kentaro Wada]

Okay, now I'm thinking difference between current-branch and current-tracking-branch is better.

Anyway, I will write my use case below >>

In my team, we specify master for some repositories and does not specify for others.

And each member does not push to origin but their own remote (ex. remote2), and usually checkout other branch (ex. devel) from master.

when version is specified

the tracking remote of devel is remote2/devel, and what I'm interested in is difference between 

• devel <-> origin/master ( as notification for pull )
• devel <-> remote2/devel ( as notification for pull or push )

It's a situation of CSO and CT.

to show two situation, two flags may be needed.

when version is not specified

Even if this situation, I am interested in the current_branch I am working on in each repository and its tracking_branch.

And this feature is coming as tkruse said at first post and the display will be like below:

Localname S SCM Version (Spec) --------- - ---- --------------
 repo git devel < remote2/devel (-)


in this situation, my request will be achieved by coming feature. 

Also, if difference between

• devel <-> remote2/devel ( as notification for pull or push)

is displayed, it's better (this is CT status) and C flag is enough.

[tkruse]

On Tuesday, July 28, 2015 at 4:10:32 PM UTC+2, Kentaro Wada wrote:

the tracking remote of devel is remote2/devel, and what I'm interested in is difference between 

• devel <-> origin/master ( as notification for pull )
• devel <-> remote2/devel ( as notification for pull or push )

In the first situation, when origin/master is not the same as local devel, then you probably have to do a merge, i.e. git merge origin/master. Or a rebase. Either way, the result of that operation would again be a divergence, so you could not get rid of the "C" that way, it would appear permanently. That does not seem useful.

Maybe you would care if there as a divergence between

master <-> origin/master
devel <-> remote2/devel

In that case you could checkout master, merge/rebase to origin/master, checkout devel, merge/rebase with master, and push, and then the C would be gone (master in sync with origin/master, devel in sync with remote2/devel). However, it is possible (and faster) to merge/rebase with origin/master without updating the local master, so that's also not too useful.

So I believe CO and CT remain the most valid scenarios, CO for working on long-lived branches like jade-devel, and CT for working on collaborative feature branches prior to pull requests. I believe those cases will usually be disjunct, if the current branch tracks a branch on a different remote, it is likely this is a feature branch having no same-name branch on origin, and vice versa.

So we may first check whether the current branch has a branch on 'origin', and if so evaluate the C flag against that one, or else check if the branch tracks a branch on another remote, and if so compute the C flag against that. Using the tracked remote branch as a fallback does not invalidate the general preference for 'origin', so I could live with that.

Kentaro, would you agree that where you have a tracked branch on another remote, you will likely not have that branch on origin?

[Kentaro Wada]

> Kentaro, would you agree that where you have a tracked branch on another remote, you will likely not have that branch on origin?

Yes I agree with you.

> So we may first check whether the current branch has a branch on 'origin', and if so evaluate the C flag against that one, or else check if the branch tracks a branch on another remote, and if so compute the C flag against that. Using the tracked remote branch as a fallback does not invalidate the general preference for 'origin', so I could live with that.

Having read your suggestion, I think this is a good condition for C flag.

--

和田 健太郎 / Kentaro Wada

http://wkentaro.com

[tkruse]

Tully, since you preferred an origin-only solution before, would you agree to a solution using a tracked branch remote2/mybranch as a fallback to evaluating the 'C' flag when origin/mybranch does not exist?

This would support the "collaborative feature branch on a fork" workflow, without any impact on people working on branches existing also on origin.

[Tully Foote]

I haven't been able to follow this super closely, but as long as it is visible to the user that should be fine.

Tully