rebase?

[Joshua N Pritikin]

With respect to GIT usage, I understand why you always merge at the end
of a branch. However, I'm surprise by the frequency of merges between
branches. Why don't you occasionally rebase issue or feature branches on
top of develop HEAD before you merge back to HEAD? With so many merges,
I find it hard to follow development history.

--
Joshua N. Pritikin
Department of Psychology
University of Virginia
485 McCormick Rd, Gilmer Hall Room 102
Charlottesville, VA 22904
http://people.virginia.edu/~jnp3bc

[Ben Goodrich]

Paranoia about loosing commits.

[Daniel Lee]

Hi Joshua,

I saw you made another comment regarding git history.

What's the value of following development history? Perhaps if we had an actual use case, we'd be more careful.

Right now, we've made it easy on the developers to move forward rather than add another bit of complicated git process to our process. I find walking through pull requests to be sufficient for browsing history, whenever I need to review what's happened. (Of course, that's relying on GitHub functionality, but eh... we're tied to GitHub at this point.)

Daniel

--
You received this message because you are subscribed to the Google Groups "stan development mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to stan-dev+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Joshua N Pritikin]

On Wed, Mar 11, 2015 at 03:07:26PM -0400, Daniel Lee wrote:
> I saw you made another comment regarding git history.
> What's the value of following development history? Perhaps if we had an
> actual use case, we'd be more careful.

I like to use tig, http://jonas.nitro.dk/tig/

> Right now, we've made it easy on the developers to move forward rather
> than add another bit of complicated git process to our process.

Why do you think merge is somehow easier than rebase? I mean, you have
to resolve conflicts one way or another. I expect that merge and rebase
are an equal amount of work (i.e., to resolve conflicts).

As I mentioned in the previous email, a merge should actually indicate
that some branch was conceptually merged. Mis-notating rebases as merges
makes history confusing.

> I find walking through pull requests to be sufficient for browsing
> history, whenever I need to review what's happened. (Of course,
> that's relying on GitHub functionality, but eh... we're tied to
> GitHub at this point.)

I guess that's why you aren't aghast at the history tangle. ;-)

[Daniel Lee]

On Wed, Mar 11, 2015 at 3:20 PM, Joshua N Pritikin <jpri...@pobox.com> wrote:

On Wed, Mar 11, 2015 at 03:07:26PM -0400, Daniel Lee wrote:
>    I saw you made another comment regarding git history.
>    What's the value of following development history? Perhaps if we had an
>    actual use case, we'd be more careful.

I like to use tig, http://jonas.nitro.dk/tig/

I see. But what does this gain you?

 

>    Right now, we've made it easy on the developers to move forward rather
>    than add another bit of complicated git process to our process.

Why do you think merge is somehow easier than rebase? I mean, you have
to resolve conflicts one way or another. I expect that merge and rebase
are an equal amount of work (i.e., to resolve conflicts).

It doesn't look too different, but I just read through this section:
http://git-scm.com/book/en/v2/Git-Branching-Rebasing
and we've kind of gone with the "record of what actually happened" approach.

(That chapter wasn't as filled in when I first read the book)

As I mentioned in the previous email, a merge should actually indicate
that some branch was conceptually merged. Mis-notating rebases as merges
makes history confusing.

What about it makes it confusing? Perhaps you can share a screenshot of tig?

Even if it were confusing in tig, I wouldn't suggest a process that makes it difficult for the developers based on a tool that's not standard.

>    I find walking through pull requests to be sufficient for browsing
>    history, whenever I need to review what's happened. (Of course,
>    that's relying on GitHub functionality, but eh... we're tied to
>    GitHub at this point.)

I guess that's why you aren't aghast at the history tangle. ;-)

Exactly. History is easy on GitHub, which you should also have access to and without the need to install an additional tool. If you could show that the history looks much different on GitHub, I think that's a compelling argument to change our process if it's not going to be a burden. If not, then I'd like table this discussion to some time in the future.

>    On Fri, Jan 16, 2015 at 7:54 PM, Ben Goodrich

>    <[1]goodri...@gmail.com> wrote:
>
>    Paranoia about loosing commits.
>    On Friday, January 16, 2015 at 4:00:55 PM UTC-5, Joshua Pritikin wrote:
>
>      With respect to GIT usage, I understand why you always merge at the
>      end
>      of a branch. However, I'm surprise by the frequency of merges
>      between
>      branches. Why don't you occasionally rebase issue or feature
>      branches on
>      top of develop HEAD before you merge back to HEAD? With so many
>      merges,
>      I find it hard to follow development history.

--
Joshua N. Pritikin
Department of Psychology
University of Virginia
485 McCormick Rd, Gilmer Hall Room 102
Charlottesville, VA 22904
http://people.virginia.edu/~jnp3bc

[Joshua N Pritikin]

On Wed, Mar 11, 2015 at 03:31:48PM -0400, Daniel Lee wrote:
> On Wed, Mar 11, 2015 at 3:20 PM, Joshua N Pritikin
> <[1]jpri...@pobox.com> wrote:
>
> On Wed, Mar 11, 2015 at 03:07:26PM -0400, Daniel Lee wrote:
> > I saw you made another comment regarding git history.
> > What's the value of following development history? Perhaps if
> we had an
> > actual use case, we'd be more careful.

> I like to use tig, [2]http://jonas.nitro.dk/tig/

>
> I see. But what does this gain you?

It is a way to examine history. That understanding history can be
helpful in understanding a project should be self evident? A clear
history will help attract more collaborators.

> > Right now, we've made it easy on the developers to move forward
> rather
> > than add another bit of complicated git process to our process.
> Why do you think merge is somehow easier than rebase? I mean, you
> have
> to resolve conflicts one way or another. I expect that merge and
> rebase
> are an equal amount of work (i.e., to resolve conflicts).
>
> It doesn't look too different, but I just read through this section:

> [3]http://git-scm.com/book/en/v2/Git-Branching-Rebasing

Yeah

> and we've kind of gone with the "record of what actually happened"
> approach.

Not really. What I advocate is a record of what actually happened.
Specifically, when you have a topic branch, you can rebase against
develop before pulling it with a merge. You need to resolve conflicts
anyway so it makes no difference whether you resolve the rebase
conflicts or the merge conflicts.

> As I mentioned in the previous email, a merge should actually
> indicate
> that some branch was conceptually merged. Mis-notating rebases as
> merges
> makes history confusing.
>
> What about it makes it confusing? Perhaps you can share a screenshot of
> tig?

Sure, attached.

I suggest that merges should mostly go in one direction, toward the
'develop' branch. That is, topical branches should rebase on top of
'develop' just prior to being merged to 'develop'. Some exceptions to
this rule are no problem, but the current situation looks more like an
undisciplined free-for-all.

> Even if it were confusing in tig, I wouldn't suggest a process that
> makes it difficult for the developers based on a tool that's not
> standard.

I'm not sure what is non-standard about tig. Another tool, gitk, is as
old as GIT and shows essentially the same output.

> > I find walking through pull requests to be sufficient for
> browsing
> > history, whenever I need to review what's happened. (Of course,
> > that's relying on GitHub functionality, but eh... we're tied to
> > GitHub at this point.)
> I guess that's why you aren't aghast at the history tangle. ;-)
>
> Exactly. History is easy on GitHub, which you should also have access
> to and without the need to install an additional tool.

Wait, are you talking about this?

https://github.com/stan-dev/stan/commits/develop

This view of history only shows the first parent?

> If you could show that the history looks much different on GitHub,
> I think that's a compelling argument to change our process if it's
> not going to be a burden. If not, then I'd like table this
> discussion to some time in the future.

[Daniel Lee]

Can you show a history of a project that does all that you suggest? Just so I can see a difference?

I'm assuming the left-most vertical line is the `develop` branch, and if so, this looks fine to me. I guess I've just adopted the distributed nature and I realize people work on things while other things happen. It's interesting, because this view actually tells you how many active branches are being worked on.

The more I look at it, this sort of depth is the right history I'd want to maintain instead of flattening it out and losing the idea of concurrency, but that's just me. I'll wait for another screenshot to stand behind that last statement.

On Wed, Mar 11, 2015 at 4:24 PM, Joshua N Pritikin <jpri...@pobox.com> wrote:

On Wed, Mar 11, 2015 at 03:31:48PM -0400, Daniel Lee wrote:
>    On Wed, Mar 11, 2015 at 3:20 PM, Joshua N Pritikin
>    <[1]jpri...@pobox.com> wrote:
>
>      On Wed, Mar 11, 2015 at 03:07:26PM -0400, Daniel Lee wrote:
>      >    I saw you made another comment regarding git history.
>      >    What's the value of following development history? Perhaps if
>      we had an
>      >    actual use case, we'd be more careful.
>      I like to use tig, [2]http://jonas.nitro.dk/tig/
>
>    I see. But what does this gain you?

It is a way to examine history. That understanding history can be
helpful in understanding a project should be self evident? A clear
history will help attract more collaborators.

Sure, but the way we're developing, we move forward, we don't really dwell on the past. Now, if we had some obligation to support old versions, this would make sense, but we don't. It simplifies life if we keep moving forward.

>      >    Right now, we've made it easy on the developers to move forward
>      rather
>      >    than add another bit of complicated git process to our process.
>      Why do you think merge is somehow easier than rebase? I mean, you
>      have
>      to resolve conflicts one way or another. I expect that merge and
>      rebase
>      are an equal amount of work (i.e., to resolve conflicts).
>
>    It doesn't look too different, but I just read through this section:
>    [3]http://git-scm.com/book/en/v2/Git-Branching-Rebasing

Yeah

>    and we've kind of gone with the "record of what actually happened"
>    approach.

Not really. What I advocate is a record of what actually happened.
Specifically, when you have a topic branch, you can rebase against
develop before pulling it with a merge. You need to resolve conflicts
anyway so it makes no difference whether you resolve the rebase
conflicts or the merge conflicts.

After considering what you've suggested, I don't think what you're suggesting is a record of what happened. But, that's splitting hairs. Let's see an example and see if it's more useful the other way. I'm pragmatic. If it does work better, I'll go with it, if it's not hard to do.

 

>      As I mentioned in the previous email, a merge should actually
>      indicate
>      that some branch was conceptually merged. Mis-notating rebases as
>      merges
>      makes history confusing.
>
>    What about it makes it confusing? Perhaps you can share a screenshot of
>    tig?

Sure, attached.

I suggest that merges should mostly go in one direction, toward the
'develop' branch. That is, topical branches should rebase on top of
'develop' just prior to being merged to 'develop'. Some exceptions to
this rule are no problem, but the current situation looks more like an
undisciplined free-for-all.

Not undisciplined free-for-all. You just happen to be seeing a part of history where we did a whole bunch of work and it broke things. We've put off refactoring the autodiff / math lib for multiple years because it would have caused this pain no matter what.

I saw a (very) small lull in our development path, so I jumped at the opportunity, grabbed Rob, and we knocked it out.

Anyway, take a look maybe a few months ago. It should be a lot cleaner and easier to follow. We just had to get through this pull request or it'll be another 2 years before we're able to accomplish it. Even though we did it as fast as we could, it did have some merge problems. I couldn't see an easy way around it.

>    Even if it were confusing in tig, I wouldn't suggest a process that
>    makes it difficult for the developers based on a tool that's not
>    standard.

I'm not sure what is non-standard about tig. Another tool, gitk, is as
old as GIT and shows essentially the same output.

If I type 'tig' from the command-line, nothing comes up. That's my bar.

 

>      >    I find walking through pull requests to be sufficient for
>      browsing
>      >    history, whenever I need to review what's happened. (Of course,
>      >    that's relying on GitHub functionality, but eh... we're tied to
>      >    GitHub at this point.)
>      I guess that's why you aren't aghast at the history tangle. ;-)
>
>    Exactly. History is easy on GitHub, which you should also have access
>    to and without the need to install an additional tool.

Wait, are you talking about this?

  https://github.com/stan-dev/stan/commits/develop

This view of history only shows the first parent?

Yup. Easy enough to pick out the pull requests, no? I don't care what happened during the pull request. Just that pull requests were merged.

 

>    If you could show that the history looks much different on GitHub,
>    I think that's a compelling argument to change our process if it's
>    not going to be a burden. If not, then I'd like table this
>    discussion to some time in the future.

--
Joshua N. Pritikin
Department of Psychology
University of Virginia
485 McCormick Rd, Gilmer Hall Room 102
Charlottesville, VA 22904
http://people.virginia.edu/~jnp3bc

[Joshua N Pritikin]

On Wed, Mar 11, 2015 at 04:52:20PM -0400, Daniel Lee wrote:
> Can you show a history of a project that does all that you suggest?
> Just so I can see a difference?
> I'm assuming the left-most vertical line is the `develop` branch, and
> if so, this looks fine to me. I guess I've just adopted the distributed
> nature and I realize people work on things while other things happen.
> It's interesting, because this view actually tells you how many active
> branches are being worked on.
> The more I look at it, this sort of depth is the right history I'd want
> to maintain instead of flattening it out and losing the idea of
> concurrency, but that's just me.

Ah, you may misunderstand which part of history I am complaining about.
I am only complaining about the merges that say,

Merge branch 'develop' of https://github.com/stan-dev/stan into develop

In my opinion, that should be a rebase. It is perfectly reasonable to
have parallel active branches under development and to merge them to
'develop' when they are ready. I am only complaining about merges in the
other direction.

More concretely, when you are getting ready to merge your branch, I
advocate,

git fetch origin
git rebase origin/develop

Instead of

git pull origin

Or, I suppose you could do,

git pull --rebase origin

but I find the 2 step fetch/rebase sequence easier to understand.

> I'll wait for another screenshot to stand behind that last
> statement.

See attached. This is part of the OpenMx history. OpenMx has fewer
active developers than Stan, but the thing to notice here is that when
merges occur, the fanout of branches is usually reduced (see merges
circled in red).

> On Wed, Mar 11, 2015 at 4:24 PM, Joshua N Pritikin

> <[1]jpri...@pobox.com> wrote:
>
> On Wed, Mar 11, 2015 at 03:31:48PM -0400, Daniel Lee wrote:
> > On Wed, Mar 11, 2015 at 3:20 PM, Joshua N Pritikin
> > <[1][2]jpri...@pobox.com> wrote:
> >
> > On Wed, Mar 11, 2015 at 03:07:26PM -0400, Daniel Lee wrote:
> > > I saw you made another comment regarding git history.
> > > What's the value of following development history?
> Perhaps if
> > we had an
> > > actual use case, we'd be more careful.

> > I like to use tig, [2][3]http://jonas.nitro.dk/tig/

> >
> > I see. But what does this gain you?
> It is a way to examine history. That understanding history can be
> helpful in understanding a project should be self evident? A clear
> history will help attract more collaborators.
>
> Sure, but the way we're developing, we move forward, we don't really
> dwell on the past. Now, if we had some obligation to support old
> versions, this would make sense, but we don't. It simplifies life if we
> keep moving forward.
>
> > > Right now, we've made it easy on the developers to move
> forward
> > rather
> > > than add another bit of complicated git process to our
> process.
> > Why do you think merge is somehow easier than rebase? I mean,
> you
> > have
> > to resolve conflicts one way or another. I expect that merge
> and
> > rebase
> > are an equal amount of work (i.e., to resolve conflicts).
> >
> > It doesn't look too different, but I just read through this
> section:

> > [3][4]http://git-scm.com/book/en/v2/Git-Branching-Rebasing

> [5]https://github.com/stan-dev/stan/commits/develop

[Bob Carpenter]

Ben --- do you want to chime in here? I think you're
the one who knows the most about this.

I, for one, am happy to do whatever others think
best. The merge bubbles are mildly annoying, but
I tend not to think of them. I usually use git log
to look at history when doing the release notes.

I believe we discussed some way to mitigate the merge
bubbles way back when we were learning Git and decided
to keep them for reasons I don't remember and didn't
understand at the time. Is it possible to get rid of
them by fast forwarding instead of rebasing and is that
safer?

I believe the danger of rebasing I was remembering was
about rebasing all the commits in a branch so that it
looked like a single commit when merged. I'm also OK doing
that if there's no danger of losing work. Very rarely,
if ever, are intermediate commits on branches worth saving.

- Bob

> --
> You received this message because you are subscribed to the Google Groups "stan development mailing list" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to stan-dev+u...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

> <Screenshot from 2015-03-12 09:24:05.png>

[Joshua N Pritikin]

On Fri, Mar 13, 2015 at 12:36:58AM +1100, Bob Carpenter wrote:
> I believe the danger of rebasing I was remembering was
> about rebasing all the commits in a branch so that it
> looked like a single commit when merged.

No, that is completely orthogonal. Rebase does not squash commits. Every
commit is carefully preserved.

> Very rarely, if ever, are intermediate commits on branches worth
> saving.

Well, maybe, but I think it can be useful to see how a branch evolved.
In any case, whether you squash all the commits down to a single commit
is completely orthogonal to what I am talking about.

> Is it possible to get rid of them by fast forwarding instead of
> rebasing and is that safer?

I have just given instructions in the previous email on how to avoid
creating these conflusing merges. If you don't understand then let's go
through it more slowly ... ?

[Bob Carpenter]

> On Mar 13, 2015, at 12:44 AM, Joshua N Pritikin <jpri...@pobox.com> wrote:
>
> On Fri, Mar 13, 2015 at 12:36:58AM +1100, Bob Carpenter wrote:
>> I believe the danger of rebasing I was remembering was
>> about rebasing all the commits in a branch so that it
>> looked like a single commit when merged.
>
> No, that is completely orthogonal. Rebase does not squash commits. Every
> commit is carefully preserved.
>

>> Is it possible to get rid of them by fast forwarding instead of
>> rebasing and is that safer?
>
> I have just given instructions in the previous email on how to avoid
> creating these conflusing merges. If you don't understand then let's go
> through it more slowly ... ?

:-)

It wasn't that I didn't understand your instructions, just
that I was wondering how safe it was, since we'd talked ages ago
about the dangers of rebasing. I thought there may be an alternative
safer version.

If it's OK with Ben and Daniel to do it this way, it's OK by me.

I can then rewrite our Wiki dev guide if we do change process. It'd
be good for me to do some more reading up on Git now that I've used
it for awhile, but it's a lot like directions, and I have a terrible
sense of direction.

- Bob

[Daniel Lee]

Joshua, quick question: if someone already did a:

git merge develop

how would they go about fixing that to look like a git rebase? Is that easy to do?

[Joshua N Pritikin]

On Fri, Mar 13, 2015 at 12:51:48AM +1100, Bob Carpenter wrote:
> It wasn't that I didn't understand your instructions, just
> that I was wondering how safe it was, since we'd talked ages ago
> about the dangers of rebasing.

Rebasing can transform merges into regular commits, but I don't
anticipate that you will run into that situation when you are only
rebasing topical branches.

You can always make a local clone of your repo and try it out to see if
there is any collateral damage. I suspect that you will be pleasantly
surprised.

[Joshua N Pritikin]

On Thu, Mar 12, 2015 at 09:55:51AM -0400, Daniel Lee wrote:
> Joshua, quick question: if someone already did a:
> git merge develop
> how would they go about fixing that to look like a git rebase? Is that
> easy to do?

Assuming your working tree is clean ('git status' shows nothing), then
you can do,

git reset --hard HEAD^ # move your branch to the previous commit

The merge should be gone now and you can rebase at your leisure,

git rebase develop # or origin/develop

[Daniel Lee]

On Thu, Mar 12, 2015 at 10:17 AM, Joshua N Pritikin <jpri...@pobox.com> wrote:

On Thu, Mar 12, 2015 at 09:55:51AM -0400, Daniel Lee wrote:
>    Joshua, quick question: if someone already did a:
>    git merge develop
>    how would they go about fixing that to look like a git rebase? Is that
>    easy to do?

Assuming your working tree is clean ('git status' shows nothing), then
you can do,

  git reset --hard HEAD^  # move your branch to the previous commit

The merge should be gone now and you can rebase at your leisure,

  git rebase develop   # or origin/develop

I take that answer as a no. That is definitely not going to work in general, as in if you've already merged in the past.

Daniel

[Joshua N Pritikin]

Oh, I may have misunderstood Bob's question. I thought he was talking
about a merge that just happened, but maybe he was talking about all the
merges in history going back months and months. If you are talking about
long-term history, I wouldn't touch that. History should generally be
considered immutable.

[Daniel Lee]

I was not thinking about changing the develop branch. I was thinking about a local branch.

Those instructions don't work if you've made multiple commits past a merge. I was hoping you had an easy way to fix a local branch.

[Joshua N Pritikin]

On Thu, Mar 12, 2015 at 11:30:31AM -0400, Daniel Lee wrote:
> I was not thinking about changing the develop branch. I was thinking
> about a local branch.
>
> Those instructions don't work if you've made multiple commits past a
> merge. I was hoping you had an easy way to fix a local branch.

I don't think there is an "easy" way. Certainly it can be done, but I
wouldn't urge you to go experiment unless you are confident in your GIT
skills. That said, you can rebase against develop even when there are
intervening merge commits. I haven't tried this recently, but I believe
those merges will disappear and may be replaced by empty commits or
commits with change fragments. Any fragments can be cleaned up with
interactive rebase. Various tutorials for interactive rebase are
available online, such as,

http://andreabergia.com/git-rebase-interactive/

However, I will be entirely happy if merges from develop to topical
branches are created less frequently in the future. Changing the past is
often not worth the effort.

[Marcus Brubaker]

Rebasing is fine if it's done on branches that haven't been pushed at all.  Once they've been pushed, they're public in a sense and other people can end up relying on their history.  Rebasing at this stage will break that history and require --force to push.  The issue really is that we often push our branches early.  Personally, I think this is good habit, as it backs up the branch.  It is also the most convenient way to share branches across multiple machines.  E.G., I use around three different machines for Stan development sometimes (office, laptop, server, etc).

One solution is to have a separate personal repository where no one can depend on your branch and you can break history at will.  Then, only when you're ready to merge and run testing, do you push the branch to the main Stan repo.  That said, I personally don't think it's worth the trouble for what is inherently a cosmetic issue.

Cheers,

Marcus

--
You received this message because you are subscribed to the Google Groups "stan development mailing list" group.

To unsubscribe from this group and stop receiving emails from it, send an email to stan-dev+unsubscribe@googlegroups.com.