Actually I just realized I have a custom .gitconfig in my %HOME%, looking at its history (it's Google Drive backed), neither preloadindex, nor fscache were set whereas the default one in \depot_tools\git-1.9.5.chromium.6_bin\etc\.gitconfig has those params.
That could explain why setting them helped me.
I guess there is no way to both have a custom .gitconfig and enjoy the latest tweaks to the default .gitconfig? Or is there such a thing as a hierarchy of .gitconfigs?!
@Bruce: good point, I've now removed the automatic git gc (git config --global gc.auto 0) and added a manual "git gc" step to my automated sync all/ctags/etc :-).