Git for Windows SDK installation shows errors

[Johannes Sixt]

I observe timeout errors on installing the GfW SDK.
Am I doing something wrong?

I did the following:

1. Download git-sdk-installer-1.0.0-64.7z.exe
2. Execute it (install location: D:\Src\git-sdk-64)
3. Upon seeing errors, hit Ctrl-C and terminate it
4. Extract git-sdk-installer-1.0.0-64.7z.exe into D:\Src\git-sdk-64
with 7zip
5. In new CMD, execute setup-git-sdk.bat.
6. After seeing errors, Ctrl-C again
7. Repeat after 4 hours.

I did not let the script run to completion and I did not clear the
install location between attempts.

I also tried with git-sdk-installer-1.0.0-32.7z.exe, but there are
the same timout errors.

What should I do to get a complete setup?

Here is the log of the last attempt (after waiting 4 hours):

D:\Src\git-sdk-64>setup-git-sdk.bat
:: Synchronizing package databases...
git-for-windows is up to date0 B 0.00B/s 00:00 [---------------------] 0%
error: failed retrieving file 'mingw32.db' from repo.msys2.org : Connection time
d out after 10001 milliseconds
mingw32 is up to date
error: failed retrieving file 'mingw64.db' from repo.msys2.org : Connection time
d out after 10001 milliseconds
mingw64 197.2 KiB 64.2M/s 00:00 [#####################] 100%
mingw64.sig 96.0 B 93.8K/s 00:00 [#####################] 100%
error: failed retrieving file 'msys.db' from repo.msys2.org : Connection timed o
ut after 10000 milliseconds
msys 124.5 KiB 60.8M/s 00:00 [#####################] 100%
msys.sig 96.0 B 93.8K/s 00:00 [#####################] 100%
resolving dependencies...
looking for conflicting packages...

Packages (1) msys2-runtime-2.1.0.33407.cd22500-1

Total Download Size: 2.26 MiB
Total Installed Size: 8.39 MiB

:: Proceed with installation? [Y/n]
warning: no /var/cache/pacman/pkg/ cache exists, creating...
:: Retrieving packages ...
msys2-runtime-2.1.0... 2.3 MiB 1431K/s 00:02 [#####################] 100%
(1/1) checking keys in keyring [#####################] 100%
(1/1) checking package integrity [#####################] 100%
(1/1) loading package files [#####################] 100%
(1/1) checking for file conflicts [#####################] 100%
(1/1) checking available disk space [#####################] 100%
(1/1) installing msys2-runtime [#####################] 100%
resolving dependencies...
looking for conflicting packages...

Packages (39) bzip2-1.0.6-2 ca-certificates-20150426-1 coreutils-8.23-4
curl-7.43.0-1 findutils-4.5.14-4 gcc-libs-4.9.2-5
gettext-0.19.4-1 gmp-6.0.0-3 gnupg-1.4.19-1 gzip-1.6-1
less-479-1 libasprintf-0.19.4-1 libbz2-1.0.6-2 libcrypt-1.1-6
libcurl-7.43.0-1 libexpat-2.1.0-2 libffi-3.2.1-1
libgettextpo-0.19.4-1 libiconv-1.14-2 libidn-1.32-1
libintl-0.19.4-1 libmetalink-0.1.2-2 libopenssl-1.0.2.d-1
libp11-kit-0.23.1-1 libpcre-8.37-1 libssh2-1.6.0-1
libtasn1-4.5-1 msys2-keyring-r8.3864337-1
ncurses-6.0.20150627-2 openssl-1.0.2.d-1 p11-kit-0.23.1-1
pacman-mirrors-20150722-2 sed-4.2.2-2 which-2.21-2 xz-5.2.1-1
zlib-1.2.8-3 bash-4.3.039-2 info-6.0-1
pacman-4.2.1.6198.439d376-1

Total Download Size: 18.42 MiB
Total Installed Size: 97.95 MiB

:: Proceed with installation? [Y/n]
:: Retrieving packages ...
error: failed retrieving file 'bash-4.3.039-2-x86_64.pkg.tar.xz' from repo.msys2
.org : Connection timed out after 10001 milliseconds
bash-4.3.039-2-x86_64 1864.6 KiB 303M/s 00:00 [#####################] 100%
error: failed retrieving file 'gcc-libs-4.9.2-5-x86_64.pkg.tar.xz' from repo.msy
s2.org : Connection timed out after 10000 milliseconds
gcc-libs-4.9.2-5-x86_64 646.8 KiB 105M/s 00:00 [#####################] 100%
error: failed retrieving file 'ncurses-6.0.20150627-2-x86_64.pkg.tar.xz' from re
po.msys2.org : Connection timed out after 10000 milliseconds
ncurses-6.0.2015062... 1111.0 KiB 181M/s 00:00 [#####################] 100%

Interrupt signal received

Batchvorgang abbrechen (J/N)? j

[Johannes Schindelin]

Hi Johannes,

On 2015-08-20 22:57, Johannes Sixt wrote:
> I observe timeout errors on installing the GfW SDK.

Please note that I would prefer to move the discussion to the
git-for...@googlegroups.com list.

> 1. Download git-sdk-installer-1.0.0-64.7z.exe
> 2. Execute it (install location: D:\Src\git-sdk-64)
> 3. Upon seeing errors, hit Ctrl-C and terminate it

Which errors?

> 4. Extract git-sdk-installer-1.0.0-64.7z.exe into D:\Src\git-sdk-64
> with 7zip
> 5. In new CMD, execute setup-git-sdk.bat.
> 6. After seeing errors, Ctrl-C again

Which errors?

> 7. Repeat after 4 hours.
>
> I did not let the script run to completion and I did not clear the
> install location between attempts.
>
> I also tried with git-sdk-installer-1.0.0-32.7z.exe, but there are
> the same timout errors.
>
> What should I do to get a complete setup?
>
> Here is the log of the last attempt (after waiting 4 hours):
>
> D:\Src\git-sdk-64>setup-git-sdk.bat
> :: Synchronizing package databases...
> git-for-windows is up to date0 B 0.00B/s 00:00
> [---------------------] 0%
> error: failed retrieving file 'mingw32.db' from repo.msys2.org :
> Connection time
> d out after 10001 milliseconds
> mingw32 is up to date
> error: failed retrieving file 'mingw64.db' from repo.msys2.org :
> Connection time
> d out after 10001 milliseconds

It seems that repo.msys2.org was slow at the time you tried. This is
unfortunate, as the move away from SourceForge was intended to stabilize
the service.

> mingw64 197.2 KiB 64.2M/s 00:00
> [#####################] 100%
> mingw64.sig 96.0 B 93.8K/s 00:00
> [#####################] 100%

However, it seems that it worked, after all. So it timed out and then
just retried. And then succeeded.

> [...]

> error: failed retrieving file 'bash-4.3.039-2-x86_64.pkg.tar.xz' from
> repo.msys2
> .org : Connection timed out after 10001 milliseconds
> bash-4.3.039-2-x86_64 1864.6 KiB 303M/s 00:00
> [#####################] 100%
> error: failed retrieving file 'gcc-libs-4.9.2-5-x86_64.pkg.tar.xz' from
> repo.msy
> s2.org : Connection timed out after 10000 milliseconds
> gcc-libs-4.9.2-5-x86_64 646.8 KiB 105M/s 00:00
> [#####################] 100%
> error: failed retrieving file
> 'ncurses-6.0.20150627-2-x86_64.pkg.tar.xz' from re
> po.msys2.org : Connection timed out after 10000 milliseconds
> ncurses-6.0.2015062... 1111.0 KiB 181M/s 00:00
> [#####################] 100%
>
> Interrupt signal received
>
> Batchvorgang abbrechen (J/N)? j

The safest would be *not* to interrupt the process, in particular when
it seems to manage to download the packages after trying a couple of
times.

Since the problematic part is MSys2's server, I guess it would not make
much sense to try to set up MSys2 instead and convert it into a Git SDK
manually.

Please let me know if the problems persist; as a temporary work-around I
could .zip up my Git SDK and share it with you via instant.io or
file.pizza.

Ciao,
Johannes

[Johannes Sixt]

Am 21.08.2015 um 12:20 schrieb Johannes Schindelin:
> Hi Johannes,
>
> On 2015-08-20 22:57, Johannes Sixt wrote:
>> I observe timeout errors on installing the GfW SDK.
>
> Please note that I would prefer to move the discussion to the
> git-for...@googlegroups.com list.

No problem. Cc'd.

>> D:\Src\git-sdk-64>setup-git-sdk.bat
>> :: Synchronizing package databases...
>> git-for-windows is up to date0 B 0.00B/s 00:00
>> [---------------------] 0%
>> error: failed retrieving file 'mingw32.db' from repo.msys2.org :
>> Connection time
>> d out after 10001 milliseconds
>> mingw32 is up to date
>> error: failed retrieving file 'mingw64.db' from repo.msys2.org :
>> Connection time
>> d out after 10001 milliseconds
>
> It seems that repo.msys2.org was slow at the time you tried. This is
> unfortunate, as the move away from SourceForge was intended to stabilize
> the service.
>
>> mingw64 197.2 KiB 64.2M/s 00:00
>> [#####################] 100%
>> mingw64.sig 96.0 B 93.8K/s 00:00
>> [#####################] 100%
>
> However, it seems that it worked, after all. So it timed out and then
> just retried. And then succeeded.

Ok. I was unsure whether this would be a correct interpretation of this
output.

At any rate, I repeated yesterday, and the setup went through without
these time-outs. It even compiled and installed Git.

I observed something else, though. I have a rather peculiar setup. In
particular, I have these environment variables set by default:

HOME=H:\
HOMEDRIVE=C:
HOMEPATH=\Users\jsixt

H:\ is on a remote Samba share and points to my actual $HOME at that
Linux box (!!)

Then I see errors of this kind:

( 8/39) installing gzip [#####################] 100%
/cygdrive/h/.bashrc: line 6: /etc/profile: No such file or directory

This .bashrc sources /etc/profile. However, I wonder why .bashrc is
sourced in the first place.

( 27/169) installing grep [#####################] 100%
mkdir: cannot change permissions of '/dev/shm': Permission denied
mkdir: cannot change permissions of '/dev/mqueue': Permission denied
ln: failed to create symbolic link '/dev/stdin': No such file or directory

Can these be ignored? These errors happen again and again.

'C:\\Windows\\System32\\drivers\\etc\\hosts' -> '/etc/hosts'
'C:\\Windows\\System32\\drivers\\etc\\protocol' -> '/etc/protocols'
'C:\\Windows\\System32\\drivers\\etc\\services' -> '/etc/services'
'C:\\Windows\\System32\\drivers\\etc\\networks' -> '/etc/networks'
( 28/169) installing dash [#####################] 100%
ln: failed to create symbolic link '/dev/stdin': No such file or directory
/tmp/alpm_ndIDSO/.INSTALL: Zeile 2: usr/bin/grep: No such file or directory
/tmp/alpm_ndIDSO/.INSTALL: Zeile 2: etc/shells: No such file or directory
error: command (/usr/bin/bash /usr/bin/bash -c . /tmp/alpm_ndIDSO/.INSTALL; post
_install 0.5.8-1 ) failed to execute correctly

This one is more alarming. post_install failed! Does it mean that my
setup is now incomplete? (I think I saw this error several times.)

If I were to repeat the installation after unsetting HOME, can I
somehow avoid that everything is downloaded again?

-- Hannes

[Johannes Schindelin]

Hi Johannes,

On 2015-08-22 10:00, Johannes Sixt wrote:

> I have a rather peculiar setup. In particular, I have these environment
> variables set by default:
>
> HOME=H:\
> HOMEDRIVE=C:
> HOMEPATH=\Users\jsixt
>
> H:\ is on a remote Samba share and points to my actual $HOME at that
> Linux box (!!)
>
> Then I see errors of this kind:
>
> ( 8/39) installing gzip
> [#####################] 100%
> /cygdrive/h/.bashrc: line 6: /etc/profile: No such file or directory
>
> This .bashrc sources /etc/profile. However, I wonder why .bashrc is
> sourced in the first place.

As part of installing the gzip package, some post-install script is
executed:


https://github.com/Alexpux/MSYS2-packages/blob/master/gzip/gzip.install

It is a shell script, executed with Bash, and therefore $HOME/.bashrc
gets sourced.

> ( 27/169) installing grep
> [#####################] 100%
> mkdir: cannot change permissions of '/dev/shm': Permission denied
> mkdir: cannot change permissions of '/dev/mqueue': Permission denied

This is normal if you execute `/etc/profile` as part of `.bashrc`. We
have a couple of relatively expensive initializations that we run as
part of the first full Bash run, i.e. the first time Bash is called with
--login. This is the best time to run those initializations because in
the portable Git distribution, we lack the luxury of being able to
initialize anything should the user decide to simply unpack the file
with 7-Zip directly. Also, for setups other than yours, it should be
fine because the idea of `/etc/profile` is to run it just once per Bash
session, at the very beginning, not for every shell script that is run.

> ln: failed to create symbolic link '/dev/stdin': No such file or
> directory

I have not seen this before, but I guess this comes from `/etc/profile`
being run prematurely.

> Can these be ignored? These errors happen again and again.

They can be ignored. Or you can change your setup so that .bashrc runs
only those commands that are really, absolutely necessary to be run with
every single shell invocation, and avoids doing the entire
`/etc/profile` dance.

> 'C:\\Windows\\System32\\drivers\\etc\\hosts' -> '/etc/hosts'
> 'C:\\Windows\\System32\\drivers\\etc\\protocol' -> '/etc/protocols'
> 'C:\\Windows\\System32\\drivers\\etc\\services' -> '/etc/services'
> 'C:\\Windows\\System32\\drivers\\etc\\networks' -> '/etc/networks'

That's known, too, being done as part of `/etc/profile`.

> ( 28/169) installing dash
> [#####################] 100%
> ln: failed to create symbolic link '/dev/stdin': No such file or
> directory
> /tmp/alpm_ndIDSO/.INSTALL: Zeile 2: usr/bin/grep: No such file or
> directory
> /tmp/alpm_ndIDSO/.INSTALL: Zeile 2: etc/shells: No such file or
> directory
> error: command (/usr/bin/bash /usr/bin/bash -c .
> /tmp/alpm_ndIDSO/.INSTALL; post
> _install 0.5.8-1 ) failed to execute correctly

My guess is that this failed `/dev/stdin` linking causes some trouble
there; I have seen some (non-fatal) problems when running the Git SDK
installer before, but never this huge trouble.

> This one is more alarming. post_install failed! Does it mean that my
> setup is now incomplete? (I think I saw this error several times.)

Yes, I think that your `/etc/shells` is incomplete now, as appending to
that file through this script failed:


https://github.com/Alexpux/MSYS2-packages/blob/master/dash/dash.install

> If I were to repeat the installation after unsetting HOME, can I
> somehow avoid that everything is downloaded again?

Absolutely. You should be able to re-run this batch script in the
top-level directory of your Git SDK, forcing a re-install:


https://github.com/git-for-windows/build-extra/blob/master/sdk-installer/setup-git-sdk.bat

You will have to adjust it before that, though, e.g. replacing all the
`@@BITNESS@@` placeholders with either `64` or `32`, depending on your
setup.

Please note that all of the calls to `pacman` (the package manager) will
force the respective packages to be reinstalled (that is on purpose,
otherwise I would have specified the `--needed` option).

The only troubles you will see if you run that batch script unmodified
otherwise: it will try to re-create the `/usr/src/**` directories and
re-clone the `/usr/src/git/` repository.

Ciao,
Johannes

[Johannes Sixt]

Thank you for your help. I will follow your advice.

-- Hannes

[Johannes Schindelin]

Hi Johannes,

Thank you. Please let me know if you have any troubles. I think there
might be some value in making setup-git-sdk.bat re-entrant, i.e. to make
it possible to run it multiple times in the same directory in case
things go wrong. Of course it would never be as easy as downloading the
raw file from GitHub (because of the @@BITNESS@@ issue) but it could be
unpacked via 7-Zip from the Git SDK installer.

What do you think?

Ciao,
Johannes

[Johannes Sixt]

Things did not work out as well as I hoped.

I used the setup-git-sdk.bat that came with the installers. It has the
@@BITNESS@@ alread substituted. And, BTW, there is also @@ARCH@@ in the
raw file.

Re-running the script in a partially completed directory locked up my
machine into an unusable state. No new programs could be started, not
even the task manager, although existing programs continued normally.
The only escape was the reset button.

The lock-ups happen, according to Process Monitor, reproducibly when
pacman attempts to update itself, in particular, there is a system call
'SetRenameInformationFile' that messes with files in the Recycle bin.
Unfortunately, I accidentally deleted my notes about the details. But I
may be brave enough to generate a Process Monitor dump if that is of
interest.

I finally deleted the half-complete directory and downloaded everything
again after unsetting HOME. That worked flawlessly.

-- Hannes

[Johannes Schindelin]

Hi Johannes,

On 2015-08-24 22:46, Johannes Sixt wrote:
> Am 24.08.2015 um 12:30 schrieb Johannes Schindelin:
>
>> Please let me know if you have any troubles. I think there
>> might be some value in making setup-git-sdk.bat re-entrant, i.e. to
>> make
>> it possible to run it multiple times in the same directory in case
>> things go wrong. Of course it would never be as easy as downloading
>> the
>> raw file from GitHub (because of the @@BITNESS@@ issue) but it could
>> be
>> unpacked via 7-Zip from the Git SDK installer.
>
> Things did not work out as well as I hoped.

Sorry to hear that!

> I used the setup-git-sdk.bat that came with the installers. It has the
> @@BITNESS@@ alread substituted. And, BTW, there is also @@ARCH@@ in
> the raw file.
>
> Re-running the script in a partially completed directory locked up my
> machine into an unusable state. No new programs could be started, not
> even the task manager, although existing programs continued normally.
> The only escape was the reset button.
>
> The lock-ups happen, according to Process Monitor, reproducibly when
> pacman attempts to update itself, in particular, there is a system
> call 'SetRenameInformationFile' that messes with files in the Recycle
> bin. Unfortunately, I accidentally deleted my notes about the details.
> But I may be brave enough to generate a Process Monitor dump if that
> is of interest.

Ouch. I am a bit surprised because I did *a lot* with pacman and it
never hung up on me.

> I finally deleted the half-complete directory and downloaded
> everything again after unsetting HOME. That worked flawlessly.

Good.

I am truly sorry that it caused you so much troubles just to install the
Git for Windows 2.x SDK. I really hoped that all my work on it would
make it very, very simple to install.

My apologies,
Johannes

[Johannes Sixt]

No reason to be sorry. It is all my fault owing to my extra-ordinary
requirements. You did a perfect job. In the end, the installation was
dead simple. I appreciate your efforts. Thank you very much!

-- Hannes