Submodules workflow

[Antoine Labour]

Hi all,

Let's talk about the submodules workflow.

The wiki page (https://code.google.com/p/chromium/wiki/UsingGitSubmodules) indicates it (still) is experimental and possibly broken. It's been experimental for more than 1.5 years. OTOH, developers (that don't use it) regularly experience problems because of it: (https://groups.google.com/a/chromium.org/forum/#!topic/chromium-dev/KA0UP69Rd9w).

So, can we commit to either:

1- making it a reality, something stable and that everyone should use, or

2- removing support for it, or at least the sources of pain for developers

Either one of those within a reasonable timeframe?

Thanks,

Antoine

[James Hall]

Hi there,

Apologies if I'm throwing a spanner in the works here, but many problems with submodules have been fixed with 'git subtree'.

http://blogs.atlassian.com/2013/05/alternatives-to-git-submodule-git-subtree/

Thanks,

James

ᐧ

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev...@chromium.org.

--

James Hall

Director

Parallax

+44 113 322 6477

http://parall.ax/

Registered office: The Old Brewery, High Court, Leeds, LS2 7ES

Registered in England no. 07430032

VAT No. 101 3405 84

[Stefan Zager]

On Mon, Mar 10, 2014 at 12:53 PM, Antoine Labour <pi...@chromium.org> wrote:
> Hi all,
>
> Let's talk about the submodules workflow.
>
> The wiki page (https://code.google.com/p/chromium/wiki/UsingGitSubmodules)
> indicates it (still) is experimental and possibly broken. It's been
> experimental for more than 1.5 years. OTOH, developers (that don't use it)
> regularly experience problems because of it:
> (https://groups.google.com/a/chromium.org/forum/#!topic/chromium-dev/KA0UP69Rd9w).
>
> So, can we commit to either:
> 1- making it a reality, something stable and that everyone should use, or
> 2- removing support for it, or at least the sources of pain for developers

We will be doing #2 in conjunction with the Chrome final migration to git.

Stefan

[Adam Treat]

Even if you use gclient you are using git submodules.  Gclient just wraps them in some scripts AFAIK.

I don't see how removing git crup for those who choose not to use gclient will help any developer.

[Adam Treat]

On 03/10/2014 04:08 PM, Stefan Zager wrote:
> We will be doing #2 in conjunction with the Chrome final migration to git.
>
> Stefan

What does this mean? That you will get rid of git submodules usage
altogether or just that you will require people to use the gclient
interface to manage them?

Thanks,
Adam

[Michael Spang]

On Mon, Mar 10, 2014 at 4:08 PM, Stefan Zager <sza...@google.com> wrote:

On Mon, Mar 10, 2014 at 12:53 PM, Antoine Labour <pi...@chromium.org> wrote:
> Hi all,
>
> Let's talk about the submodules workflow.
>
> The wiki page (https://code.google.com/p/chromium/wiki/UsingGitSubmodules)
> indicates it (still) is experimental and possibly broken. It's been
> experimental for more than 1.5 years. OTOH, developers (that don't use it)
> regularly experience problems because of it:
> (https://groups.google.com/a/chromium.org/forum/#!topic/chromium-dev/KA0UP69Rd9w).

I've uploaded a fix for this particular issue - https://codereview.chromium.org/192443006/

 

>
> So, can we commit to either:
> 1- making it a reality, something stable and that everyone should use, or
> 2- removing support for it, or at least the sources of pain for developers

We will be doing #2 in conjunction with the Chrome final migration to git.

I think we should land the fix unless we do this relatively soon.

Michael

[Adam Treat]

On 03/10/2014 04:08 PM, Stefan Zager wrote:

> 2- removing support for it, or at least the sources of pain for developers
> We will be doing #2 in conjunction with the Chrome final migration to git.
>
> Stefan
>

Can you let me know whether or not this plan is about moving away from
git submodules altogether or just requiring gclient as the front-end for
all version control work? Also, where this decision was made and how
would be great... Thanks!

[Dirk Pranke]

The thing you should take away from this thread is simply that you should not be using the "UsingGitSubmodules" workflow. If you are, stop and switch back to the gclient-based flow, preferably in the "unmanaged" mode.

The only workflow that we are currently supporting -- and really have been supporting for a while -- is the gclient-based flow (which still uses git submodules underneath, but does not use or require git crup). 

Stefan is just going a bit further to say that that flow will probably break completely with the transition.

It's possible how we sync dependencies will change as well at some point, but (at least as far as I know), we will continue using the gclient solution across and after the migration.

Does that help address your concerns?

-- Dirk

[Adam Treat]

On 03/11/2014 02:36 PM, Dirk Pranke wrote:

On Tue, Mar 11, 2014 at 11:11 AM, Adam Treat <adam....@samsung.com> wrote:

On 03/10/2014 04:08 PM, Stefan Zager wrote:

2- removing support for it, or at least the sources of pain for developers
We will be doing #2 in conjunction with the Chrome final migration to git.

Stefan

Can you let me know whether or not this plan is about moving away from git submodules altogether or just requiring gclient as the front-end for all version control work?  Also, where this decision was made and how would be great... Thanks!

The thing you should take away from this thread is simply that you should not be using the "UsingGitSubmodules" workflow. If you are, stop and switch back to the gclient-based flow, preferably in the "unmanaged" mode.

OK, I see.  Can you tell me what the motivation is for requiring gclient though?  What really puzzles me is that from what I can tell gclient is an abstraction around the vcs allowing people to use svn or git.  But if we are moving to git for the server what is the necessity of using the gclient porcelain instead of just using pure git?

Does that help address your concerns?

It certainly answers the question, but I'm still left wondering about the motivation.

Thanks,
Adam

[Stefan Zager]

On Tue, Mar 11, 2014 at 11:11 AM, Adam Treat <adam....@samsung.com> wrote:

The current submodule flow relies on a complicated branching scheme
that we don't want to continue supporting:

https://sites.google.com/a/chromium.org/dev/developers/how-tos/git-repo

When we make the final switch to git, the current refs/heads/git-svn
branch is going to be our starting point. We won't do any more
branching to add submodule information, and all of the commits that
contain submodule information won't be reachable from
refs/heads/master.

It is possible that we'll bring back submodule support after the git
migration, but there's a bit of complexity(*) to that, and it's low on
our priority list right now. The reason we don't dismantle the
submodule flow before the migration is that doing so would probably
break existing tools and flows, and there's no good justification for
it. All of our energy is focused on achieving the git migration.

(*) Two problems:

1. The DEPS file -- used by gclient -- and the .gitmodules and git
submodule files -- used by the submodule flow -- contain redundant
information. We have no intention of deprecating DEPS or gclient in
the short term. To have any kind of sane submodule support
co-existing with DEPS/gclient, we must be absolutely certain that DEPS
and .gitmodules/git-submodule-files are always in sync. We can add
client git hooks that do the necessary munging when a developer is
working up a patch locally; but we don't have an airtight enforcement
mechanism to ensure that a change which introduces out-of-sync DEPS
and .gitmodules never lands in the upstream repository.

2. The code review tool currently has no support for changes to git
submodule files.


Stefan

[Stefan Zager]

On Tue, Mar 11, 2014 at 11:36 AM, Dirk Pranke <dpr...@chromium.org> wrote:
>
> The thing you should take away from this thread is simply that you should
> not be using the "UsingGitSubmodules" workflow. If you are, stop and switch
> back to the gclient-based flow, preferably in the "unmanaged" mode.
>
> The only workflow that we are currently supporting -- and really have been
> supporting for a while -- is the gclient-based flow (which still uses git
> submodules underneath, but does not use or require git crup).

Point of clarification: the gclient-based flow does NOT use submodules
underneath.

Stefan

[Dirk Pranke]

On Tue, Mar 11, 2014 at 11:43 AM, Adam Treat <adam....@samsung.com> wrote:

On 03/11/2014 02:36 PM, Dirk Pranke wrote:

On Tue, Mar 11, 2014 at 11:11 AM, Adam Treat <adam....@samsung.com> wrote:

On 03/10/2014 04:08 PM, Stefan Zager wrote:

2- removing support for it, or at least the sources of pain for developers
We will be doing #2 in conjunction with the Chrome final migration to git.

Stefan

Can you let me know whether or not this plan is about moving away from git submodules altogether or just requiring gclient as the front-end for all version control work?  Also, where this decision was made and how would be great... Thanks!

The thing you should take away from this thread is simply that you should not be using the "UsingGitSubmodules" workflow. If you are, stop and switch back to the gclient-based flow, preferably in the "unmanaged" mode.

OK, I see.  Can you tell me what the motivation is for requiring gclient though?  What really puzzles me is that from what I can tell gclient is an abstraction around the vcs allowing people to use svn or git.  But if we are moving to git for the server what is the necessity of using the gclient porcelain instead of just using pure git?

Stefan just replied w/ some of the issues that arise. 

I think perhaps the more important thing is that we don't want to change everything at once, so we'll keep gclient across the transition and then (hopefully when things settle down at some point in the future) perhaps look at switching to a pure submodule flow. 

Switching to pure submodules isn't high on at least my priority list, though (I'd much rather have people working on things like merging Blink and Chromium, getting faster checkouts and syncs, etc.).

-- Dirk

[Dirk Pranke]

Whoops, you're right, of course. Sorry for confusing things further. I don't know where my brain was at ...

-- Dirk