exclude submodules by default when refreshing

[Jacob Keller]

I work with a team that uses submodules fairly heavily, and a few of our developers have grown accustomed to stacked git. Unfortunately, there are a few problems with stacked git, when it comes to submodules.

The first major pain point is that "stg refresh" will include submodule pointer updates by default. This can cause issues if users who are new to submodules, when an environment expects rapidly changing submodules (such as our use case).

Since this leads to accidental commits of the submodule pointer, I am proposing a patch to stacked git, which causes "stg refresh" to simply exclude submodules by default, unless either (a) stgit.refreshsubmodules is "yes", (b) the --submodules command line argument is given, or (c) the submodule is included in the list of paths to update.

This ensures that in general submodules will not be "refreshed" (since they are often accidentally out of date due to the way that git detects differences of submodules) but if necessary, users are able to easily include the submodules to patches via 3 easy mechanisms, depending on the users preference and work flow.

I believe that this change will have minimal impact to other users of stgit, and will resolve a pain point that many on my team suffer from. I have already tried alternatives, such as attempting to teach users about how submodules work, and advising the use of "refresh --index" mode instead of the regular refresh mode. Unfortunately the first of these efforts helps some, but does not eliminate the accidents which occur, and the second requires users of stgit to actively change their workflow in a way that they may not find efficient. Additionally, in many cases, users who accidentally commit submodule changes are not even aware they have done so until much later in development when it causes problems.

I would accept that this may not wish to be the default, in which case swapping the default value of stgit.refreshsubmodules would achieve this result.

Thanks,

Regards

[Peter Maydell]

On 14 November 2017 at 13:23, Jacob Keller <jacob....@gmail.com> wrote:
> I work with a team that uses submodules fairly heavily, and a few of our
> developers have grown accustomed to stacked git. Unfortunately, there are a
> few problems with stacked git, when it comes to submodules.
>
> The first major pain point is that "stg refresh" will include submodule
> pointer updates by default. This can cause issues if users who are new to
> submodules, when an environment expects rapidly changing submodules (such as
> our use case).
>
> Since this leads to accidental commits of the submodule pointer, I am
> proposing a patch to stacked git, which causes "stg refresh" to simply
> exclude submodules by default, unless either (a) stgit.refreshsubmodules is
> "yes", (b) the --submodules command line argument is given, or (c) the
> submodule is included in the list of paths to update.
>
> This ensures that in general submodules will not be "refreshed" (since they
> are often accidentally out of date due to the way that git detects
> differences of submodules) but if necessary, users are able to easily
> include the submodules to patches via 3 easy mechanisms, depending on the
> users preference and work flow.

Yeah, I run into this problem a lot. I can't help feeling that stgit
is the wrong level to fix the problem though -- "git checkout" &c
should also update the submodule, rather than leaving it out of date.
Then there wouldn't generally be accidental-out-of-date situations to
get refreshed into stgit patches...

thanks
-- PMM

[Jacob Keller]

There is some work involved in the git community to fix this with
recursive checkout, but it has a few problems.

1) it does not work well in cases where you're actively working on the
submodule code and have it checked out

2) users on my team are used to the "old" world of simply using 3
separate repositories, and having the "sub" modules checked out at
master. I attempted to move people to submodules, but many team
members never quite grokked the submodule concept, so they are often
confused about how it is tracked by the parent at all, and would
rather keep the submodule checked out at master. This would be fine,
except in cases where the parent project is not yet ready to checkout
the submodule at master.

So, for cases where submodules are rarely changed, recursive checkout
makes sense, but it doesn't work quite as well in a submodule which is
fast paced and updates constantly.

I'm totally willing to switch the default on this patch and make it
something users have to configure to "enable" this behavior, since I
can usually convince people to set something in their .gitconfig once.

Thanks,
Jake

[Jacob Keller]

I made a pull request of these changes at https://github.com/ctmarinas/stgit/pull/8

It includes two patches now, the first fixes some bugs caused by how submodules don't get checked out automatically as expected which causes stgit to complain about a dirty worktree all the time. The second is the main patch proposed here with a few updates.

I still did not change the default yet, since I strongly believe this change is generally beneficial and should be the default behavior. I'd like to see what others think of that, and whether they agree.

Thanks,

Jake