Page on developing Vim

[Bram Moolenaar]

I have asked Christian Brabandt to write down how he creates and
maintains patches for Vim. You can read it here:

http://www.vim.org/develop.php

I hope this is useful. If you have suggestions to improve this page,
please discuss here.

--
A radioactive cat has eighteen half-lives.

/// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net \\\
/// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\ an exciting new programming language -- http://www.Zimbu.org ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///

[Ben Fritz]

On Saturday, March 29, 2014 10:10:03 AM UTC-5, Bram Moolenaar wrote:
> I have asked Christian Brabandt to write down how he creates and
> maintains patches for Vim. You can read it here:
>
> http://www.vim.org/develop.php
>
> I hope this is useful. If you have suggestions to improve this page,
> please discuss here.
>

Nice! I have a couple suggestions:

First, this part I think has a typo:

> So you save your work, refresh the current patch and fix that small bug
> with a new patch:
>
> ~/code/vim $ hg qrefresh
> popping my_fancy_feature
> patch queue now empty

Shouldn't that be qpop instead of (or in addition to) qrefresh?

Secondly, I disagree that "It is dangerous to pull changes from the central
vim repository, while there are still patches applied." Pulling with patches
applied always works just fine, the mq patches act just like real Mecurial
changesets. What you don't want to do, is update after a pull with the
patches still applied, because then you need to back up to a different
changeset to unapply the patches. But even update isn't "dangerous". What
would be bad is trying to merge the upstream changes into your mq patches
using Mercurial merge commands.

I think "hg pull" without the "-u" flag, so that you can at least preview
the incoming changes, would actually be a good thing. Just make sure to
"hg qpop -a" before doing an "hg update".

Finally, it is OK to use mq in a repository others can pull from, if you use
the relatively new "phases" feature to hide your mq changesets from anybody
doing a "pull". You do this by enabling the "secret" option in mq, to make
any changesets created by the extension hidden from others on pull or push:

http://mercurial.selenic.com/wiki/MqExtension#Ensure_patch_queue_changesets_use_secret_Phase

[Christian Brabandt]

On So, 30 Mär 2014, Ben Fritz wrote:
> On Saturday, March 29, 2014 10:10:03 AM UTC-5, Bram Moolenaar wrote:
> > I have asked Christian Brabandt to write down how he creates and
> > maintains patches for Vim. You can read it here:
> >
> > http://www.vim.org/develop.php
> >
> > I hope this is useful. If you have suggestions to improve this page,
> > please discuss here.
> >
>
> Nice! I have a couple suggestions:
>
> First, this part I think has a typo:
>
> > So you save your work, refresh the current patch and fix that small bug
> > with a new patch:
> >
> > ~/code/vim $ hg qrefresh
> > popping my_fancy_feature
> > patch queue now empty
>
> Shouldn't that be qpop instead of (or in addition to) qrefresh?

Yes the qpop is missing. It should be:
~/code/vim $ hg qrefresh
~/code/vim $ hg qpop

popping my_fancy_feature
patch queue now empty

> Secondly, I disagree that "It is dangerous to pull changes from the central
> vim repository, while there are still patches applied." Pulling with patches
> applied always works just fine, the mq patches act just like real Mecurial
> changesets. What you don't want to do, is update after a pull with the
> patches still applied, because then you need to back up to a different
> changeset to unapply the patches. But even update isn't "dangerous". What
> would be bad is trying to merge the upstream changes into your mq patches
> using Mercurial merge commands.

Yes, that's what I meant. So, to how about this:

It is dangerous to pull changes from the central vim repository and
update your working copy at the same time (-u flag), while there are
still patches applied. Instead, make sure to pop all patches, update the
repository and push your patches again:

> I think "hg pull" without the "-u" flag, so that you can at least preview
> the incoming changes, would actually be a good thing. Just make sure to
> "hg qpop -a" before doing an "hg update".
>
> Finally, it is OK to use mq in a repository others can pull from, if you use
> the relatively new "phases" feature to hide your mq changesets from anybody
> doing a "pull". You do this by enabling the "secret" option in mq, to make
> any changesets created by the extension hidden from others on pull or push:
>
> http://mercurial.selenic.com/wiki/MqExtension#Ensure_patch_queue_changesets_use_secret_Phase

I saw this on that webpage. But I didn't want to confuse by introducing
yet another option of which I don't if it is available at all clients.
So how about this:

Don't use MQ in a repository anyone might pull from (unless you hide
your mq changesets away using the <a
href='http://mercurial.selenic.com/wiki/MqExtension#Ensure_patch_queue_changesets_use_secret_Phase'>secret
phase</a>)

Best,
Christian

[Danek Duvall]

Christian Brabandt wrote:

> > Secondly, I disagree that "It is dangerous to pull changes from the central
> > vim repository, while there are still patches applied." Pulling with patches
> > applied always works just fine, the mq patches act just like real Mecurial
> > changesets. What you don't want to do, is update after a pull with the
> > patches still applied, because then you need to back up to a different
> > changeset to unapply the patches. But even update isn't "dangerous". What
> > would be bad is trying to merge the upstream changes into your mq patches
> > using Mercurial merge commands.
>
> Yes, that's what I meant. So, to how about this:
>
> It is dangerous to pull changes from the central vim repository and
> update your working copy at the same time (-u flag), while there are
> still patches applied. Instead, make sure to pop all patches, update the
> repository and push your patches again:

You may be better off enabling the rebase extension and running "hg pull
--rebase", since that will enable mercurial's merging mechanics, rather
than the qpop/qpush pair, which will (if there are conflicts) leave patch
reject files lying around, which you then need to merge manually.

It's also not at all dangerous to pull -u when there are patches applied;
the only thing that will happen is that mercurial will complain that it
won't update across heads and leave you where you are. You can then
qpop/qpush or rebase your queue.

Danek

[Bram Moolenaar]

Thanks for the changes, I'll update the page.

--
hundred-and-one symptoms of being an internet addict:
10. And even your night dreams are in HTML.

[Ben Fritz]

On Monday, March 31, 2014 3:15:52 PM UTC-5, Christian Brabandt wrote:
> On So, 30 Mär 2014, Ben Fritz wrote:
> > I disagree that "It is dangerous to pull changes from the central
> > vim repository, while there are still patches applied." Pulling with patches
> > applied always works just fine, the mq patches act just like real Mecurial
> > changesets. What you don't want to do, is update after a pull with the
> > patches still applied, because then you need to back up to a different
> > changeset to unapply the patches. But even update isn't "dangerous". What
> > would be bad is trying to merge the upstream changes into your mq patches
> > using Mercurial merge commands.
>
> Yes, that's what I meant. So, to how about this:
>
> It is dangerous to pull changes from the central vim repository and
> update your working copy at the same time (-u flag), while there are
> still patches applied. Instead, make sure to pop all patches, update the
> repository and push your patches again:
>

That works, though as update may work it might be confusing to get the
mq changesets moved around afterwards. I like it.

> > Finally, it is OK to use mq in a repository others can pull from, if you use
> > the relatively new "phases" feature to hide your mq changesets from anybody
> > doing a "pull". You do this by enabling the "secret" option in mq, to make
> > any changesets created by the extension hidden from others on pull or push:
> >
> > http://mercurial.selenic.com/wiki/MqExtension#Ensure_patch_queue_changesets_use_secret_Phase
>
> I saw this on that webpage. But I didn't want to confuse by introducing
> yet another option of which I don't if it is available at all clients.
> So how about this:
>
> Don't use MQ in a repository anyone might pull from (unless you hide
> your mq changesets away using the <a
> href='http://mercurial.selenic.com/wiki/MqExtension#Ensure_patch_queue_changesets_use_secret_Phase'>secret
> phase</a>)
>

I like this phrasing. Noting it's possible but not a good idea by
default until you take other action.

[Ben Fritz]

On Monday, March 31, 2014 9:04:50 PM UTC-5, Danek Duvall wrote:
> Christian Brabandt wrote:
>
> > > Secondly, I disagree that "It is dangerous to pull changes from the central
> > > vim repository, while there are still patches applied." Pulling with patches
> > > applied always works just fine, the mq patches act just like real Mecurial
> > > changesets. What you don't want to do, is update after a pull with the
> > > patches still applied, because then you need to back up to a different
> > > changeset to unapply the patches. But even update isn't "dangerous". What
> > > would be bad is trying to merge the upstream changes into your mq patches
> > > using Mercurial merge commands.
> >
> > Yes, that's what I meant. So, to how about this:
> >
> > It is dangerous to pull changes from the central vim repository and
> > update your working copy at the same time (-u flag), while there are
> > still patches applied. Instead, make sure to pop all patches, update the
> > repository and push your patches again:
>
> You may be better off enabling the rebase extension and running
> "hg pull --rebase", since that will enable mercurial's merging
> mechanics, rather than the qpop/qpush pair, which will (if there are
> conflicts) leave patch reject files lying around, which you then need
> to merge manually.
>

Does "rebase" integrate with MQ to set up the queue base changeset and
such properly? I really, really don't like the idea of messing around
with MQ changesets in this way, unless I know it works.

I saw recently on the Mercurial mailing list that MQ is planned for
deprecation (still supported, but not recommended for new users). One of
the suggested replacements was "changeset evolution" combined with
rebase, histedit, and strip if needed. The idea was to use normal
changesets with the "secret" phase on so you could use all the internal
Mercurial machinery better. But I don't know how mature this approach
is, and I'm having a little trouble wrapping my mind around it without
trying it out first. I already know how to work with MQ, on the other
hand.

[Nikolay Pavlov]

I am wondering why you need any rebasing in the first place? I use regular mercurial branches or bookmarks and regular "hg merge". If you are not going to release a series of related patches (which depend on each other) this approach is far better:

1. You are not losing context in which changes were made.
2. If you do a mistake while dealing with conflict this mistake will be attached to merge changeset, not to the original change.
3. It is easy to distribute changes. No secrecy, just push to your public repo. Bitbucked supports or used to support MQ, but I failed to setup it. Though I did not try hard, just was investigating whether dealing MQ is worth learning.
4. You do not need to learn MQ in addition to learning mercurial.

Additionally if the phase of changesets is secret you can as well rebase regular mercurial changesets.

>
> --
> --
> You received this message from the "vim_dev" maillist.
> Do not top-post! Type your reply below the text you are replying to.
> For more information, visit http://www.vim.org/maillist.php
>
> ---
> You received this message because you are subscribed to the Google Groups "vim_dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+u...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Ben Fritz]

On Tuesday, April 1, 2014 10:53:38 AM UTC-5, ZyX wrote:
>
> I am wondering why you need any rebasing in the first place? I use regular mercurial branches or bookmarks and regular "hg merge". If you are not going to release a series of related patches (which depend on each other) this approach is far better:
>
>
> 1. You are not losing context in which changes were made.
>
> 2. If you do a mistake while dealing with conflict this mistake will be attached to merge changeset, not to the original change.
>
> 3. It is easy to distribute changes. No secrecy, just push to your public repo. Bitbucked supports or used to support MQ, but I failed to setup it. Though I did not try hard, just was investigating whether dealing MQ is worth learning.
>
>

I agree with all this. But Bram refuses to pull directly from other repositories and then merge. So if I use normal Hg changesets with merging, I will always have my own personal branches hanging around. Even if I merge and close the branch with a commit, that merge commit will never be an ancestor of the commits I pull from Bram. So I will eventually just need to delete my changesets to remove the dead branch. Additionally, since Bram won't pull and merge, I need to send changes based off the latest changes HE has made to make him more likely to accept them.

Either rebasing changesets, or doing a qpop+update+qpush, are easy ways to do this.

> 4. You do not need to learn MQ in addition to learning mercurial.
>
> Additionally if the phase of changesets is secret you can as well rebase regular mercurial changesets.
>

This is true. I learned MQ before phases were introduced and in general am not very comfortable in editing history even in secret.

I view MQ changesets not so much as normal changesets in Hg, but rather a group of in-progress patches applied on top of an actual repository. In Vim's case, that repository is for my purposes pretty much read-only. I don't commit any Vim code changes (on the rare occasions I make them). Rather I qpush them. When the change appears in the upstream repository as an official patch, I qpop and forget about my changeset. I know under the hood this is acutally editing history...but it doesn't FEEL like that's what I'm doing.

I understand changeset obsolescence and history editing are intended to replace MQ to accomplish the same things, however.

[Nikolay Pavlov]

On Apr 2, 2014 1:54 AM, "Ben Fritz" <fritzo...@gmail.com> wrote:
>
> On Tuesday, April 1, 2014 10:53:38 AM UTC-5, ZyX wrote:
> >
> > I am wondering why you need any rebasing in the first place? I use regular mercurial branches or bookmarks and regular "hg merge". If you are not going to release a series of related patches (which depend on each other) this approach is far better:
> >
> >
> > 1. You are not losing context in which changes were made.
> >
> > 2. If you do a mistake while dealing with conflict this mistake will be attached to merge changeset, not to the original change.
> >
> > 3. It is easy to distribute changes. No secrecy, just push to your public repo. Bitbucked supports or used to support MQ, but I failed to setup it. Though I did not try hard, just was investigating whether dealing MQ is worth learning.
> >
> >
>
> I agree with all this. But Bram refuses to pull directly from other repositories and then merge. So if I use normal Hg changesets with merging, I will always have my own personal branches hanging around. Even if I merge and close the branch with a commit, that merge commit will never be an ancestor of the commits I pull from Bram. So I will eventually just need to delete my changesets to remove the dead branch. Additionally, since Bram won't pull and merge, I need to send changes based off the latest changes HE has made to make him more likely to accept them.

After you did a merge sending update is trivial (except for a sequence of patches I mentioned: in this case you either need to actually rebase or have a sequence of merge commits). Mercurial is also fine with dead branches hanging around and you can always view only ancestors of a certain changeset. Absence of upstream merges is not fine, but all attempts to convince Bram failed so far.

>
> Either rebasing changesets, or doing a qpop+update+qpush, are easy ways to do this.
>
> > 4. You do not need to learn MQ in addition to learning mercurial.
> >
> > Additionally if the phase of changesets is secret you can as well rebase regular mercurial changesets.
> >
>
> This is true. I learned MQ before phases were introduced and in general am not very comfortable in editing history even in secret.
>
> I view MQ changesets not so much as normal changesets in Hg, but rather a group of in-progress patches applied on top of an actual repository. In Vim's case, that repository is for my purposes pretty much read-only. I don't commit any Vim code changes (on the rare occasions I make them). Rather I qpush them. When the change appears in the upstream repository as an official patch, I qpop and forget about my changeset. I know under the hood this is acutally editing history...but it doesn't FEEL like that's what I'm doing.
>
> I understand changeset obsolescence and history editing are intended to replace MQ to accomplish the same things, however.

If you keep old history you can review what *Bram* has changed in your patch. You will need to rebase to/merge the changeset just before upstream inclusion though.

[Danek Duvall]

Ben Fritz wrote:

> Does "rebase" integrate with MQ to set up the queue base changeset and
> such properly? I really, really don't like the idea of messing around
> with MQ changesets in this way, unless I know it works.

Yes, it resets all the MQ-specific tags to what they would have been had
you qpushed onto the new changeset.

> I saw recently on the Mercurial mailing list that MQ is planned for
> deprecation (still supported, but not recommended for new users). One of
> the suggested replacements was "changeset evolution" combined with
> rebase, histedit, and strip if needed. The idea was to use normal
> changesets with the "secret" phase on so you could use all the internal
> Mercurial machinery better. But I don't know how mature this approach
> is, and I'm having a little trouble wrapping my mind around it without
> trying it out first. I already know how to work with MQ, on the other
> hand.

I'm in the same boat. I've been using MQ for years and am comfortable with
it, and haven't had the time to try out obsolescence in any serious way,
though I'm actually quite excited about it, since I'd love to have the
history of my patch changes for the duration of the patch. I tried out
pbranch several years ago, but it was just too wild, and the merges were
horrendous.

Danek

[Christian Brabandt]

On Mi, 02 Apr 2014, Danek Duvall wrote:

> I'm in the same boat. I've been using MQ for years and am comfortable with
> it, and haven't had the time to try out obsolescence in any serious way,
> though I'm actually quite excited about it, since I'd love to have the
> history of my patch changes for the duration of the patch. I tried out
> pbranch several years ago, but it was just too wild, and the merges were
> horrendous.

Can someone explain in little more detail how to use the new recommended
way to deal with patches if mq becomes un-supported?

Thanks.

Best,
Christian
--
Ein Schotte mit brennender Kerze vor dem Spiegel: Zweiter Advent.

[Danek Duvall]

Christian Brabandt wrote:

> On Mi, 02 Apr 2014, Danek Duvall wrote:
>
> > I'm in the same boat. I've been using MQ for years and am comfortable with
> > it, and haven't had the time to try out obsolescence in any serious way,
> > though I'm actually quite excited about it, since I'd love to have the
> > history of my patch changes for the duration of the patch. I tried out
> > pbranch several years ago, but it was just too wild, and the merges were
> > horrendous.
>
> Can someone explain in little more detail how to use the new recommended
> way to deal with patches if mq becomes un-supported?

You can start here:

http://mercurial.selenic.com/wiki/ChangesetEvolution

and the materials linked from there. There's also

http://hg-lab.logilab.org/doc/mutable-history/html/

which includes a MQ-refugee page.

Though the interfaces MQ provides won't be removed for a long time, if
ever; the mercurial guys are very strict about backwards compatibility.

Danek

[Christian Brabandt]

On Sa, 05 Apr 2014, Danek Duvall wrote:

> Christian Brabandt wrote:
>
> > On Mi, 02 Apr 2014, Danek Duvall wrote:
> >
> > > I'm in the same boat. I've been using MQ for years and am comfortable with
> > > it, and haven't had the time to try out obsolescence in any serious way,
> > > though I'm actually quite excited about it, since I'd love to have the
> > > history of my patch changes for the duration of the patch. I tried out
> > > pbranch several years ago, but it was just too wild, and the merges were
> > > horrendous.
> >
> > Can someone explain in little more detail how to use the new recommended
> > way to deal with patches if mq becomes un-supported?
>
> You can start here:
>
> http://mercurial.selenic.com/wiki/ChangesetEvolution
>
> and the materials linked from there. There's also
>
> http://hg-lab.logilab.org/doc/mutable-history/html/
>
> which includes a MQ-refugee page.

Thanks. Now I need some time to go through all the documentation and
check, if it fits my current workflow better...

> Though the interfaces MQ provides won't be removed for a long time, if
> ever; the mercurial guys are very strict about backwards compatibility.

Ah, is mercurial the Vim of the DVCS?

Best,
Christian
--
Wer ernsthaft die Wahrheit der Dinge ergründen will, darf sich keiner
einzelnen Wissenschaft verschreiben; denn alle Teile der Wissenschaft
stehen im Verbund wechselseitiger Abhängigkeit.
-- René Descartes

[Christian Brabandt]

On Sa, 12 Apr 2014, Christian Brabandt wrote:

> On Sa, 05 Apr 2014, Danek Duvall wrote:
>
> > Christian Brabandt wrote:
> >
> > > On Mi, 02 Apr 2014, Danek Duvall wrote:
> > >
> > > > I'm in the same boat. I've been using MQ for years and am comfortable with
> > > > it, and haven't had the time to try out obsolescence in any serious way,
> > > > though I'm actually quite excited about it, since I'd love to have the
> > > > history of my patch changes for the duration of the patch. I tried out
> > > > pbranch several years ago, but it was just too wild, and the merges were
> > > > horrendous.
> > >
> > > Can someone explain in little more detail how to use the new recommended
> > > way to deal with patches if mq becomes un-supported?
> >
> > You can start here:
> >
> > http://mercurial.selenic.com/wiki/ChangesetEvolution
> >
> > and the materials linked from there. There's also
> >
> > http://hg-lab.logilab.org/doc/mutable-history/html/

This link gives only a 404 now.

> >
> > which includes a MQ-refugee page.
>
> Thanks. Now I need some time to go through all the documentation and
> check, if it fits my current workflow better...
>
> > Though the interfaces MQ provides won't be removed for a long time, if
> > ever; the mercurial guys are very strict about backwards compatibility.
>
> Ah, is mercurial the Vim of the DVCS?
>
> Best,
> Christian

Mit freundlichen Grüßen
Christian
--
Es gibt nichts Schöneres in dieser Welt als einen gesunden weisen
alten Mann.
-- Lin Yutang