PSA: New and improved Windows git available in depot_tools

199 views
Skip to first unread message

Michael Moss

unread,
Feb 7, 2014, 12:22:08 AM2/7/14
to chromium-dev, blin...@chromium.org
Background: The current version of Windows git in depot_tools is much slower on some common operations, like 'stat' and 'fetch', than git on other platforms. We've built a version with some custom patches to try to address the worst of these performance issues. Our testing shows that it is now 2-4 times faster on those operations, but we'd like to get more feedback before making it the default version in depot_tools.

Please try this new build and let us know if there are any problems. If you were disappointed with previous git performance, let us know if this version addresses your concerns.

You can install by running the following at a Windows command prompt:
set DEPOT_TOOLS_GIT_1825=1
gclient

Once installed, 'git --version' should report:
git version 1.8.5.2.msysgit.0.1.g01cc1f2

If you would like to go back to the default version of git:
set DEPOT_TOOLS_GIT_1825=0
rd /s /q git-1.8.5.2.chromium.1_bin git-1.8.0_bin
gclient

Thanks,
Michael, on behalf of chrome-infrastructure-team

Takayoshi Kochi

unread,
Feb 7, 2014, 12:47:38 AM2/7/14
to Michael Moss, chromium-dev, blink-dev
Hi,

On Fri, Feb 7, 2014 at 2:22 PM, Michael Moss <mm...@chromium.org> wrote:
Background: The current version of Windows git in depot_tools is much slower on some common operations, like 'stat' and 'fetch', than git on other platforms. We've built a version with some custom patches to try to address the worst of these performance issues. Our testing shows that it is now 2-4 times faster on those operations, but we'd like to get more feedback before making it the default version in depot_tools.

Please try this new build and let us know if there are any problems. If you were disappointed with previous git performance, let us know if this version addresses your concerns.

You can install by running the following at a Windows command prompt:
set DEPOT_TOOLS_GIT_1825=1

If I'm not mistaken, DEPOT_TOOLS_GIT_1852, not 1825.

gclient

Once installed, 'git --version' should report:
git version 1.8.5.2.msysgit.0.1.g01cc1f2

If you would like to go back to the default version of git:
set DEPOT_TOOLS_GIT_1825=0
rd /s /q git-1.8.5.2.chromium.1_bin git-1.8.0_bin
gclient

Thanks,
Michael, on behalf of chrome-infrastructure-team



--
Takayoshi Kochi

Michael Moss

unread,
Feb 7, 2014, 12:49:10 AM2/7/14
to Takayoshi Kochi, chromium-dev, blink-dev
On Thu, Feb 6, 2014 at 9:47 PM, Takayoshi Kochi <ko...@chromium.org> wrote:
Hi,

On Fri, Feb 7, 2014 at 2:22 PM, Michael Moss <mm...@chromium.org> wrote:
Background: The current version of Windows git in depot_tools is much slower on some common operations, like 'stat' and 'fetch', than git on other platforms. We've built a version with some custom patches to try to address the worst of these performance issues. Our testing shows that it is now 2-4 times faster on those operations, but we'd like to get more feedback before making it the default version in depot_tools.

Please try this new build and let us know if there are any problems. If you were disappointed with previous git performance, let us know if this version addresses your concerns.

You can install by running the following at a Windows command prompt:
set DEPOT_TOOLS_GIT_1852=1

If I'm not mistaken, DEPOT_TOOLS_GIT_1852, not 1825.
Yes, fixed, sorry. 

gclient

Once installed, 'git --version' should report:
git version 1.8.5.2.msysgit.0.1.g01cc1f2

If you would like to go back to the default version of git:
set DEPOT_TOOLS_GIT_1852=0
rd /s /q git-1.8.5.2.chromium.1_bin git-1.8.0_bin
gclient

Thanks,
Michael, on behalf of chrome-infrastructure-team



--
Takayoshi Kochi

Michael Moss

unread,
Feb 7, 2014, 2:55:34 PM2/7/14
to chromium-dev, blink-dev
FYI, If there are no reported problems, this will become the default version of Windows git on Monday.

Ken Rockot

unread,
Feb 7, 2014, 3:57:39 PM2/7/14
to Michael Moss, chromium-dev, blink-dev
Thanks for this effort. The difference is quite noticeable and indeed wonderful.


--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev

Henrik Andreasson

unread,
Feb 8, 2014, 11:04:43 AM2/8/14
to roc...@chromium.org, Michael Moss, chromium-dev, blink-dev
Works well for me. Have not done any detailed comparisons. Thanks for doing this work.

Michael Moss

unread,
Feb 10, 2014, 12:37:05 PM2/10/14
to chromium-dev, blink-dev
This is now the default. Let us know if you see any problems.


On Fri, Feb 7, 2014 at 11:55 AM, Michael Moss <mm...@chromium.org> wrote:

Reid Kleckner

unread,
Feb 10, 2014, 5:11:37 PM2/10/14
to mm...@chromium.org, chromium-dev, blink-dev
Is there any way to use depot_tools git using bash and a console other than cmd?

I spent some time trying to get it to work with mintty.  git basically thinks it's connected to a pipe (isatty() probably returns false), so it disables color and paging, making it hard to use.

I don't really care if I have to use Cygwin, MSys, the copy of bash in depot_tools's git, or something else crazy, but it'd be awesome if there was a supported way to develop Chromium on Windows in a terminal that has reverse search, long scrollback, more than a hundred columns, etc.


--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev...@chromium.org.

Stefan Zager

unread,
Feb 10, 2014, 5:28:41 PM2/10/14
to Reid Kleckner, Michael Moss, chromium-dev, blink-dev
On Mon, Feb 10, 2014 at 2:11 PM, Reid Kleckner <r...@google.com> wrote:
> Is there any way to use depot_tools git using bash and a console other than
> cmd?

You can fire up an msys bash terminal, and run your git commands from there:

> set PATH=%PATH%;C:\path\to\your\depot_tools\git-1.8.5.2.chromium.1_bin\bin
> bash
$ git ...

Stefan

Michael Moss

unread,
Feb 10, 2014, 6:39:16 PM2/10/14
to chromium-dev, blink-dev
Due to some bot failures that I still need time to investigate, I'm going to push a change to no longer make the new git the default.

If the new git is working for you, and you would like to keep using it, you must once again:
set DEPOT_TOOLS_GIT_1852=1
or depot_tools will revert to the old version. Sorry for the inconvenience.

Michael

Reid Kleckner

unread,
Feb 10, 2014, 6:42:54 PM2/10/14
to Stefan Zager, Michael Moss, chromium-dev, blink-dev
This will execute in a cmd.exe window, which I don't want for all the usual reasons: copy paste is hard, resizing the window is hard, scrollback is short.

Zachary Turner

unread,
Feb 10, 2014, 7:02:00 PM2/10/14
to Reid Kleckner, mm...@chromium.org, chromium-dev, blink-dev
A lot of people have been switching over to cmd substitutes that are syntax-compatible but much more feature rich.   Pretty sure they support all the things you mention.  I forget the name of them off the top of my head though.

Since you're ok with "something else crazy", might I suggest Powershell?  You can even add git tab completion to it: http://www.jeremyskinner.co.uk/2010/03/07/using-git-with-windows-powershell/


On Mon, Feb 10, 2014 at 2:11 PM, Reid Kleckner <r...@google.com> wrote:

Kim Gräsman

unread,
Feb 11, 2014, 3:06:20 PM2/11/14
to Reid Kleckner, mm...@chromium.org, chromium-dev, blink-dev
On Mon, Feb 10, 2014 at 11:11 PM, Reid Kleckner <r...@google.com> wrote:
> Is there any way to use depot_tools git using bash and a console other than
> cmd?
>
> I don't really care if I have to use Cygwin, MSys, the copy of bash in
> depot_tools's git, or something else crazy, but it'd be awesome if there was
> a supported way to develop Chromium on Windows in a terminal that has
> reverse search, long scrollback, more than a hundred columns, etc.

I don't do Chromium development, so I don't know how well it applies
for your scenario, but check out clink:
https://code.google.com/p/clink/

It basically turns cmd.exe into bash with a couple of extra hooks,
e.g. Ctrl+V for clipboard paste.

- Kim

Michael Moss

unread,
Feb 11, 2014, 10:53:37 PM2/11/14
to chromium-dev, blink-dev
Our custom Windows git is once again the default version in depot_tools. If you would like to keep the old version for some reason, you can do so by adding:
set DEPOT_TOOLS_GIT_1852=0
to your environment.

Michael

Roger Tawa

unread,
Feb 12, 2014, 2:02:22 PM2/12/14
to mm...@chromium.org, chromium-dev, blink-dev
Hi all,

As of today, git on windows uses a different colour scheme for the git branch command.  Its dark blue on black, see attached screenshot, which is not easily readable.

I looked through the git config docs about colours (https://www.kernel.org/pub/software/scm/git/docs/git-config.html), but I did not find a slot for the "upstream branch".  I see only slots for "local", "remote", "current", and "plain", none of which fixed this.  Does someone know what slot to use?

Thanks,
Roger

-


git-colour.gif

Stefan Zager

unread,
Feb 12, 2014, 4:37:31 PM2/12/14
to Roger Tawa, Michael Moss, chromium-dev, blink-dev
On Wed, Feb 12, 2014 at 11:02 AM, Roger Tawa <rog...@chromium.org> wrote:
> Hi all,
>
> As of today, git on windows uses a different colour scheme for the git
> branch command. Its dark blue on black, see attached screenshot, which is
> not easily readable.
>
> I looked through the git config docs about colours
> (https://www.kernel.org/pub/software/scm/git/docs/git-config.html), but I
> did not find a slot for the "upstream branch". I see only slots for
> "local", "remote", "current", and "plain", none of which fixed this. Does
> someone know what slot to use?

Is git-br an alias? I don't seem to have it, and I don't know how to
produce that style of output with git-branch.

Stefan

Zachary Turner

unread,
Feb 12, 2014, 6:06:27 PM2/12/14
to sza...@chromium.org, Roger Tawa, Michael Moss, chromium-dev, blink-dev
git branch -vv

(that's 2 v's, not a single w, which might be confusing depending on your font).


Daniel Bratell

unread,
Feb 24, 2014, 1:24:16 PM2/24/14
to chromium-dev, blin...@chromium.org, Michael Moss
I just noticed a (hopefully) unintentional difference between the old and the new git.

The new git has a hardcoded 

 autocrlf = false

in the depot_tools/git-1.8.5.2.chromium.1_bin/etc/gitconfig file. That flag makes a big difference when working on Windows. I don't think most Googlers notice this because it seems to be internal Google policy to use Unix line endings in Windows, but for us people trying to use the optimized/fast git for other projects it's a bit harmful with non-default values to core.autocrlf.

Would it be possible to switch back to the previous value so that the behaviour is more like standard git? Please?

/Daniel

Michael Moss

unread,
Feb 24, 2014, 1:31:44 PM2/24/14
to Daniel Bratell, chromium-dev, blink-dev
That was intentionally changed to enforce the Chromium policy, and to fix specific problems caused by not having it set properly (e.g. messed up checkouts on buildbots). If you have it set the way you want it in your personal preferences (git config --global or --local), does that not take precedence?

FWIW, the previous git also had a hardcoded autocrlf config, it was just set to a different value (probably the one you wanted, so you didn't notice).

Daniel Bratell

unread,
Feb 24, 2014, 1:43:20 PM2/24/14
to Michael Moss, chromium-dev, blink-dev
On Mon, 24 Feb 2014 19:31:44 +0100, Michael Moss <mm...@chromium.org> wrote:

That was intentionally changed to enforce the Chromium policy, and to fix specific problems caused by not having it set properly (e.g. messed up checkouts on buildbots). If you have it set the way you want it in your personal preferences (git config --global or --local), does that not take precedence?

FWIW, the previous git also had a hardcoded autocrlf config, it was just set to a different value (probably the one you wanted, so you didn't notice).

The previous value was the default value as in "the same value you get when you download a git binary from, well, anywhere". I understand that you want the tools to work as well as possible for chromium but there is a value to having tools behave as people are used to as well.

For me personally it's a problem because Chromium is not my only project but it's the only one with the core.autcrlf=false quirk. I was used to special case my chromium checkouts (needed anyway because gclient modifiies the global git configuration (very evil)) but now I have to be even more careful.

Is it possible to somehow put autocrlf in the repository settings instead? That would be the sanest solution if it works but maybe there is a bootstrap problem with that.

/Daniel

Michael Moss

unread,
Feb 24, 2014, 2:47:35 PM2/24/14
to Daniel Bratell, chromium-dev, blink-dev
I'm not sure of a reliable way to do that, certainly not more reliable than setting the system-level default in the git package, and certainly not something I'm going to have an opportunity to revisit anytime soon. I understand your position, but as this git is built specifically for the Chromium toolchain, enforcing Chromium policies is more important than behaving as a universal git replacement.

That said, there's also nothing stopping someone from using it as a universal git replacement. Copy the depot_tools/git-* dir somewhere else, change the default configs, and add that to your path. It's basically just a repackaged version of PortableGit (http://storage.googleapis.com/chrome-infra/PortableGit-1.8.5.2.msysgit.0.1.g01cc1f2.7z is the original), so it can happily be installed in multiple locations without conflicting with each other. All the speedups, none of the unwanted defaults.

 
/Daniel


Zachary Turner

unread,
Feb 24, 2014, 2:58:28 PM2/24/14
to Michael Moss, Daniel Bratell, chromium-dev, blink-dev
FWIW, it's possible to do what Daniel suggests by putting autocrlf in a .gitattributes file and checking that in at the root, then let the git binary keep its normal default value.  


Michael Moss

unread,
Feb 24, 2014, 3:23:33 PM2/24/14
to Zachary Turner, Daniel Bratell, chromium-dev, blink-dev
But wouldn't that need to be done for every repository? Including third-party ones?

Zachary Turner

unread,
Feb 24, 2014, 3:34:00 PM2/24/14
to Michael Moss, Daniel Bratell, chromium-dev, blink-dev
Good question, I'm not sure if the setting persists into submodules.  Either way though, I think that long-term gitattributes should be where we head.  It makes the setup flow simpler for new developers, allows us to clean up confusing documentation about how to configure git, and makes incorporating new changes downstream simpler.

It also allows us to configure things differently on a per-file basis and have changes rolled out to users automatically with a simple sync, which is far more flexible than anything else git provides.  I know this is useful in a couple of places, for example a few types of files actually require CRLF (windows batch files come to mind).

Michael Moss

unread,
Feb 24, 2014, 4:06:09 PM2/24/14
to Zachary Turner, Daniel Bratell, chromium-dev, blink-dev
On Mon, Feb 24, 2014 at 12:34 PM, Zachary Turner <ztu...@chromium.org> wrote:
Good question, I'm not sure if the setting persists into submodules.

I did some quick testing, and it doesn't seem to.
 
Either way though, I think that long-term gitattributes should be where we head.  It makes the setup flow simpler for new developers, allows us to clean up confusing documentation about how to configure git, and makes incorporating new changes downstream simpler.
 
Yeah, I'm not opposed to using .gitattributes, and agree that gives us the most flexibility. But that's going to be a good bit of work to retrofit, and to ensure it's properly maintained going forward (e.g. for new repos), so probably not something that's going to happen anytime soon. And there's still the problem of third-party repos. Like your example of batch files wanting CRLF, some files, like shell scripts, can break with CRLF. If such files were in a third-party repo where we couldn't add a .gitattributes, we'd still need to enforce an autocrlf policy.

Takayoshi Kochi

unread,
Feb 24, 2014, 8:31:24 PM2/24/14
to Michael Moss, Zachary Turner, Daniel Bratell, chromium-dev, blink-dev
FYI, while I was working on Windows with the new git, I was bitten by CRLF files accidentally committed,
ended up in having 2 CLs: 251460 and 250068.
I wasted some time finding out how I should set up core.autocrlf on Windows in vain - and finally I made
CRLF->LF CL on Linux (I gave up making line-endings only change on Windows).

Please consider refraining from making such changes that breaks daily workflow.
--
Takayoshi Kochi

Daniel Bratell

unread,
Feb 26, 2014, 11:09:29 AM2/26/14
to Zachary Turner, Michael Moss, chromium-dev, blink-dev
On Mon, 24 Feb 2014 21:23:33 +0100, Michael Moss <mm...@chromium.org>
wrote:

(regarding setting .gitattributes in the root of a repository)

> But wouldn't that need to be done for every repository? Including
> third-party ones?

Yes, and that would be a problem. Easy to forget one of them, and more
than a few of the non Google people would probably object since
core.autocrlf=false is not a recommended setting in general.

On the other hand it's not a big issue having CRLF in the source in
Windows for most people. The problems are most apparent when you mix (for
instance by using depot_tools git first and then a default git. :-) ).

Talking about git performance (going to diverge the thread a bit), what
openssl version does the bundled git use on Windows? I've heard (though my
search mojo has just left me) that it can help performance to include the
right (TM) version of the ssl/crypto libs.

/Daniel

Daniel Bratell

unread,
Feb 27, 2014, 10:39:18 AM2/27/14
to Zachary Turner, Michael Moss, chromium-dev, blink-dev
On Wed, 26 Feb 2014 17:09:29 +0100, Daniel Bratell <bra...@opera.com>
wrote:

> Talking about git performance (going to diverge the thread a bit), what
> openssl version does the bundled git use on Windows? I've heard (though
> my search mojo has just left me) that it can help performance to include
> the right (TM) version of the ssl/crypto libs.

OpenSSH not OpenSSL. We had some really good performance results with
replacing OpenSSH 4.x in msysgit with OpenSSH 5.2 (copied from an rsync
package) half a year ago. People reported up to 10x higher download speeds.

(Quote from internal mail (hi Stan): "With ssh 4.x the a clone was
happening at no more than 1024 kbps")

/Daniel

Stefan Zager

unread,
Feb 27, 2014, 12:50:00 PM2/27/14
to Daniel Bratell, blink-dev, Michael Moss, chromium-dev, Zachary Turner


On Feb 27, 2014 7:39 AM, "Daniel Bratell" <bra...@opera.com> wrote:
>
> On Wed, 26 Feb 2014 17:09:29 +0100, Daniel Bratell <bra...@opera.com> wrote:
>
>> Talking about git performance (going to diverge the thread a bit), what openssl version does the bundled git use on Windows? I've heard (though my search mojo has just left me) that it can help performance to include the right (TM) version of the ssl/crypto libs.
>
>
> OpenSSH not OpenSSL. We had some really good performance results with replacing OpenSSH 4.x in msysgit with OpenSSH 5.2 (copied from an rsync package) half a year ago. People reported up to 10x higher download speeds.

Yes, we made the same observation in the field.  However, we don't use ssh transport for any of our repos.

Stefan

Greg Thompson

unread,
Mar 3, 2014, 11:25:13 AM3/3/14
to rog...@chromium.org, mm...@chromium.org, chromium-dev, blink-dev
On Wed, Feb 12, 2014 at 2:02 PM, Roger Tawa <rog...@chromium.org> wrote:
Hi all,

As of today, git on windows uses a different colour scheme for the git branch command.  Its dark blue on black, see attached screenshot, which is not easily readable.

I looked through the git config docs about colours (https://www.kernel.org/pub/software/scm/git/docs/git-config.html), but I did not find a slot for the "upstream branch".  I see only slots for "local", "remote", "current", and "plain", none of which fixed this.  Does someone know what slot to use?

Hi Roger,

Did you figure out a way to change the colors for git branch -vv? Dark blue on black isn't working for my monitor/eyeball combination. Thanks.

Zachary Turner

unread,
Mar 3, 2014, 12:20:37 PM3/3/14
to Greg Thompson, Roger Tawa, Michael Moss, chromium-dev, blink-dev
A quick look at the git source suggests that it's hardcoded.  I guess you could submit a patch to make it an option of git config, I agree that dark blue on black is a bit difficult to read.

Greg Thompson

unread,
Mar 12, 2014, 10:51:11 AM3/12/14
to Zachary Turner, Roger Tawa, Michael Moss, chromium-dev, blink-dev
It turns out that it is configurable, but not really documented. Here's what I'm using now:

git config --global --add color.branch.upstream "normal bold"

Reply all
Reply to author
Forward
0 new messages