Correct way to create new chromium checkouts?

[Randy Smith]

TL;DR: I believe the only working way to create a chromium checkout is through "fetch chromium" producing a git submodules checkout, and the "gclient config->gclient sync" flow documented on dev.chromium.org doesn't work.  That right?  Anyone mind if I change the doc?

As part of a recent (Mac) laptop switch, I re-engaged with the process of creating a new chromium (commiter) checkout from scratch.  I'm currently using gclient (not the new git submodules flow) and decided to change a single variable at a time, i.e. to keep using gclient.  I tried the following things:

* Following the directions at http://dev.chromium.org/developers/how-tos/get-the-code#TOC-Mac-OS-X, "download via git".  That failed with:

________ running 'git checkout --quiet master' in '/Users/rdsmith/Sandboxen/2chrome/src'

Error: 14> Can't update/checkout /Users/rdsmith/Sandboxen/2chrome/src/native_client if an unversioned directory is present. Delete the directory and try again.

when I investigated, it turned out that "native_client" existed in the checkout already as a git commit link (i.e. a link pointing into a submodule), which blocked the gclient checkout as above.  I played around with this approach a bit, but didn't find a way to get a gclient based checkout.

* Using "fetch chromium".  This appeared to create a git submodules checkout.  Using --nosvn=True created a git submodules checkout without a link to git-svn.  Not using that flag prompted me for my svn committer credentials and created a git submodules checkout with a link to git-svn.  (I'm presuming that second checkout would allow me to do "git cl dcommit", but I don't know how to test that without actually doing it.)

From the above I'm concluding that we're deprecating the gclient sync based checkout, and anyone creating a new checkout should use git submodules.  That accurate?  Any reason for me not to change the documentation to reflect that?

Secondary question:  The above link says "NOTE: These instructions will pull a read-only tree.  If you are a committer, and plan to make changes to source code, use the instructions given to you when you received commit access."  At least one committer :-} doesn't have that good a memory; is there any reason we can't put instructions as to how to create a R/W checkout if you are a committer on dev.chromium.org, either with a reference to "use the SVN password that was given to you when you because a committer" or instructions as to how to retrieve that password if you've forgotten it (which I also don't remember :-} :-J)?

-- Randy

[Peter Kasting]

On Thu, Sep 26, 2013 at 10:54 AM, Randy Smith <rds...@chromium.org> wrote:

TL;DR: I believe the only working way to create a chromium checkout is through "fetch chromium" producing a git submodules checkout, and the "gclient config->gclient sync" flow documented on dev.chromium.org doesn't work.  That right?  Anyone mind if I change the doc?

At least on Windows, "fetch chromium" is not necessary (I don't use it).  I would rather not lose the gclient documentation.

 

instructions as to how to retrieve that password if you've forgotten it (which I also don't remember :-} :-J)?

https://chromium-access.appspot.com/

PK 

[Randy Smith]

On Thu, Sep 26, 2013 at 2:47 PM, Peter Kasting <pkas...@chromium.org> wrote:

On Thu, Sep 26, 2013 at 10:54 AM, Randy Smith <rds...@chromium.org> wrote:

TL;DR: I believe the only working way to create a chromium checkout is through "fetch chromium" producing a git submodules checkout, and the "gclient config->gclient sync" flow documented on dev.chromium.org doesn't work.  That right?  Anyone mind if I change the doc?

At least on Windows, "fetch chromium" is not necessary (I don't use it).  I would rather not lose the gclient documentation.

Sorry, just to be clear: You mean that you've recently created a chromium checkout from scratch via gclient config -> gclient sync, and it's worked for you?  If so, that's good news; it suggests that it might be made to work on other platforms as well.

-- Randy

[Peter Kasting]

On Thu, Sep 26, 2013 at 11:56 AM, Randy Smith <rds...@google.com> wrote:

On Thu, Sep 26, 2013 at 2:47 PM, Peter Kasting <pkas...@chromium.org> wrote:

On Thu, Sep 26, 2013 at 10:54 AM, Randy Smith <rds...@chromium.org> wrote:

TL;DR: I believe the only working way to create a chromium checkout is through "fetch chromium" producing a git submodules checkout, and the "gclient config->gclient sync" flow documented on dev.chromium.org doesn't work.  That right?  Anyone mind if I change the doc?

At least on Windows, "fetch chromium" is not necessary (I don't use it).  I would rather not lose the gclient documentation.

Sorry, just to be clear: You mean that you've recently created a chromium checkout from scratch via gclient config -> gclient sync, and it's worked for you?  If so, that's good news; it suggests that it might be made to work on other platforms as well.

Not super-recently, but definitely after the Blink changeover.  Just requires a couple changes to the config files.

PK 

[Paweł Hajdan, Jr.]

On Thu, Sep 26, 2013 at 10:54 AM, Randy Smith <rds...@chromium.org> wrote:

TL;DR: I believe the only working way to create a chromium checkout is through "fetch chromium" producing a git submodules checkout,

AFAIK "fetch chromium" will make an unmanaged git checkout and not submodules. I was working on it recently so I think this is accurate. Please let me know if you're seeing something different. The documentation for it can be found at https://code.google.com/p/chromium/wiki/UsingGit - please report any inaccuracies that may be there.

 

and the "gclient config->gclient sync" flow documented on dev.chromium.org doesn't work.  That right?  Anyone mind if I change the doc?

+1 to getting the doc up to date, and thanks for catching this. Please either let us know here what you'd like to change, or post after making the change so it can be reviewed just in case.

 

* Using "fetch chromium".  This appeared to create a git submodules checkout.  Using --nosvn=True created a git submodules checkout without a link to git-svn.  Not using that flag prompted me for my svn committer credentials and created a git submodules checkout with a link to git-svn.  (I'm presuming that second checkout would allow me to do "git cl dcommit", but I don't know how to test that without actually doing it.)

What makes you think it's using submodules? Maybe it is, but I don't think it should.

 

From the above I'm concluding that we're deprecating the gclient sync based checkout, and anyone creating a new checkout should use git submodules.  That accurate?  Any reason for me not to change the documentation to reflect that?

AFAIK, git submodules workflow is considered highly experimental, and the recommended one is https://code.google.com/p/chromium/wiki/UsingGit .

 

Secondary question:  The above link says "NOTE: These instructions will pull a read-only tree.  If you are a committer, and plan to make changes to source code, use the instructions given to you when you received commit access."  At least one committer :-} doesn't have that good a memory; is there any reason we can't put instructions as to how to create a R/W checkout if you are a committer on dev.chromium.org, either with a reference to "use the SVN password that was given to you when you because a committer" or instructions as to how to retrieve that password if you've forgotten it (which I also don't remember :-} :-J)?

I think "fetch chromium" nicely addresses that. I totally agree with what you wrote above.

Paweł

[Randy Smith]

What are the changes?  I'd be a little surprised if there was a major difference at this level between Windows and Mac, so I'd expect what worked for you to work for me.

-- Randy

PK 

[Peter Kasting]

On Thu, Sep 26, 2013 at 1:57 PM, Randy Smith <rds...@google.com> wrote:

What are the changes?  I'd be a little surprised if there was a major difference at this level between Windows and Mac, so I'd expect what worked for you to work for me.

Haven't looked at what happens by default in a while, but the first part of my .gclient is

  { "name"        : "src",

    "url"         : "svn://chrome-svn/chrome/trunk/src",

    "custom_vars" : {

      "webkit_trunk": "svn://svn.chromium.org/blink/trunk",

      "webkit_revision": "HEAD",

    },

  },

PK 

[Randy Smith]

On Thu, Sep 26, 2013 at 3:30 PM, Paweł Hajdan, Jr. <phajd...@chromium.org> wrote:

On Thu, Sep 26, 2013 at 10:54 AM, Randy Smith <rds...@chromium.org> wrote:

TL;DR: I believe the only working way to create a chromium checkout is through "fetch chromium" producing a git submodules checkout,

AFAIK "fetch chromium" will make an unmanaged git checkout and not submodules. I was working on it recently so I think this is accurate. Please let me know if you're seeing something different. The documentation for it can be found at https://code.google.com/p/chromium/wiki/UsingGit - please report any inaccuracies that may be there.

So the way I distinguish between a git submodule checkout and a gclient checkout is by doing a "git ls-tree HEAD" and seeing if I see any entries of type "commit".  On the checkout on Mac I just created with "fetch chromium" I get:

rdsmith-macbookpro:../chrome/src [master] $ git ls-tree HEAD | grep native_client

160000 commit 9dd875a9429e98c7fcb216354b5893791663a512 native_client

040000 tree 95fbb97671bf090b5d004a01c7dd7d4352e55f55 native_client_sdk

rdsmith-macbookpro:../chrome/src [master] $    

and on the legacy checkout (i.e. created a long time ago) on Linux I get:

astibar:../chrome/src [master] $ git ls-tree HEAD | grep native_client

040000 tree 05fd536d2f9e727a5cb76cc45a9ab0661b563fbc native_client_sdk

astibar:../chrome/src [master] $ 

 

and the "gclient config->gclient sync" flow documented on dev.chromium.org doesn't work.  That right?  Anyone mind if I change the doc?

+1 to getting the doc up to date, and thanks for catching this. Please either let us know here what you'd like to change, or post after making the change so it can be reviewed just in case.

No worries; I'm not a fan of flying unchecked.  I sorta wish web page changes went through Rietveld like code changes did; I'd feel more comfortable making them. 

 

 

* Using "fetch chromium".  This appeared to create a git submodules checkout.  Using --nosvn=True created a git submodules checkout without a link to git-svn.  Not using that flag prompted me for my svn committer credentials and created a git submodules checkout with a link to git-svn.  (I'm presuming that second checkout would allow me to do "git cl dcommit", but I don't know how to test that without actually doing it.)

What makes you think it's using submodules? Maybe it is, but I don't think it should.

See above.

-- Randy

[John Abd-El-Malek]

On Thu, Sep 26, 2013 at 11:47 AM, Peter Kasting <pkas...@chromium.org> wrote:

On Thu, Sep 26, 2013 at 10:54 AM, Randy Smith <rds...@chromium.org> wrote:

TL;DR: I believe the only working way to create a chromium checkout is through "fetch chromium" producing a git submodules checkout, and the "gclient config->gclient sync" flow documented on dev.chromium.org doesn't work.  That right?  Anyone mind if I change the doc?

At least on Windows, "fetch chromium" is not necessary (I don't use it).  I would rather not lose the gclient documentation.

I have heard Brett say that he couldn't get fetch to work on Windows because it kept timing out. He restored the gclient information on the wiki which had been removed. So if someone is thinking of removing it, please check with him. (he's OOO till Monday).

 

 

instructions as to how to retrieve that password if you've forgotten it (which I also don't remember :-} :-J)?

https://chromium-access.appspot.com/

PK 

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev

[Lei Zhang]

Are you talking about the Blink checkout timing out? Juan recently hit
that issue and he said: "I modified fetch.py to run gclient sync with
--no-nag-max, and that finally got past the WebKit timeouts."

[John Abd-El-Malek]

On Thu, Sep 26, 2013 at 2:11 PM, Lei Zhang <the...@chromium.org> wrote:

Are you talking about the Blink checkout timing out?

A Blink or Chromium checkout now is very similar from what I understand.

[Lei Zhang]

What I mean was, as part of a Chromium checkout, we have to checkout
Blink into third_party/WebKit. Since that git repo is large, it has a
tendency to time out.

[Scott Graham]

FWIW, I still use this one weird old trick to avoid the timeout: "fetch --dry-run chromium", and then execute the steps it prints manually.

It doesn't really seem like a feature, or at least isn't implemented properly. I will delete the nag-killing unless someone speaks up about about why they need to keep it.

[Scott Graham]

Timeout code removed at r225730.

I recommend "fetch chromium" as the best way to get the code now.

It's just a simple wrapper around gclient that creates a .gclient file and makes sure git-svn is set up.

[Randy Smith]

Quick update on this thread: When I started it, neither of the recipes at http://dev.chromium.org/developers/how-tos/get-the-code#TOC-Mac-OS-X worked for me.  Now, they both do.  I'm not ecstatic about that, but it does reduce my need to fix the web documentation :-J.

Is "fetch chromium" (with and without --nosvn=True) described anywhere on the wiki?  If not, any objection if I add it, specifically replacing the "If you are a committer, and plan to make changes to source code, use the instructions given to you when you received commit access." warning?

-- Randy

[Dirk Pranke]

Last I checked, it is not documented, and I have no objection to you fixing the documentation :).

[Rouslan Solomakhin]

On Wed, Oct 9, 2013 at 11:42 AM, Randy Smith <rds...@google.com> wrote:

Is "fetch chromium" (with and without --nosvn=True) described anywhere on the wiki?

It's documented on https://code.google.com/p/chromium/wiki/UsingGit#Initial_checkout.