[PATCH,RESEND] copy-files.sh: Preserve bit-exactness of git-*.exe for DONT_REMOVE_BUILTINS

[Kirill Smelkov]

Currently, if client (i.e. share/WinGit/portable-release.sh) asks
copy-files.sh not to remove git builtins, we just keep them in bin/ and
libexec/git-core/ . The problem is, later, when those executables are
stripped, though initially they were identical, they all become
different, because:

For PE, strip (and everything from binutils) puts current time into
produced object in PE header into TimeDateStamp [1]:

---- 8< ---- bfd/peXXigen.c
H_PUT_32 (abfd, time (0), filehdr_out->f_timdat);

and that hurts, if later a user wants to hardlink'ify portable git
install.

Though strip has --preserve-dates, as said above, it does not affect the
_contents_ of the generated object file - only mtime/ctime are
preserved.

Let's preserve builtins ourselves - we already keep list of them in
etc/fileList-builtins.txt for msysgit installer (it relinks them at
install time), and for portable version, lets first remove all builtins
except git.exe, then strip, and then restore builtins bit-to-bit equal
to git.exe . This way, later hardlinkifying will work.

[1] http://www.csn.ul.ie/~caolan/publink/winresdump/winresdump/doc/pefile2.html

Signed-off-by: Kirill Smelkov <ki...@mns.spb.ru>
---
share/WinGit/copy-files.sh | 19 +++++++++++++------
1 file changed, 13 insertions(+), 6 deletions(-)

diff --git a/share/WinGit/copy-files.sh b/share/WinGit/copy-files.sh
index 6bc9427..b94f460 100755
--- a/share/WinGit/copy-files.sh
+++ b/share/WinGit/copy-files.sh
@@ -69,12 +69,9 @@ rm -rf bin/cvs.exe &&
test -f lib/perl5/site_perl/Git.pm &&
gitmd5=$(md5sum bin/git.exe | cut -c 1-32) &&
mkdir etc &&
-if test -z "$DONT_REMOVE_BUILTINS"
-then
- md5sum {bin,libexec/git-core}/git-*.exe libexec/git-core/git.exe |
- sed -n -r "s/^$gitmd5\s+\*?(.*)/\1/p" > etc/fileList-builtins.txt &&
- rm $(cat etc/fileList-builtins.txt)
-fi &&
+md5sum {bin,libexec/git-core}/git-*.exe libexec/git-core/git.exe |
+sed -n -r "s/^$gitmd5\s+\*?(.*)/\1/p" > etc/fileList-builtins.txt &&
+rm $(cat etc/fileList-builtins.txt) && # rm builtins - if needed we'll restore them after strip
(cd $MSYSGITROOT/mingw && tar cf - \
bin/*{tcl,tk,wish,gpg,msmtp,curl.exe,*.crt}* bin/connect.exe \
bin/*{libcurl,libcrypto,libssl,libgsasl,libiconv}* \
@@ -85,6 +82,16 @@ fi &&
tar xf - &&
cp $MSYSGITROOT/mingw/bin/hd2u.exe bin/dos2unix.exe &&
strip bin/{[a-fh-z],g[a-oq-z]}*.exe libexec/git-core/*.exe &&
+if test -n "$DONT_REMOVE_BUILTINS"
+then
+ # restore builtins after git.exe was stripped
+ # (for PE, strip embeds current time into file header, and if we just
+ # pass all git builtins to strip the result will be lots of
+ # not-bit-exact exe's)
+ for b in $(cat etc/fileList-builtins.txt); do
+ ln bin/git.exe $b
+ done
+fi &&
cp $MSYSGITROOT/git/contrib/completion/git-completion.bash etc/ &&
cp $MSYSGITROOT/git/contrib/completion/git-prompt.sh etc/ &&
cp $MSYSGITROOT/etc/termcap etc/ &&
--
1.8.2.1.744.gffa8c7b

[Eric Sunshine]

On Mon, Apr 29, 2013 at 4:19 AM, Kirill Smelkov <ki...@mns.spb.ru> wrote:
> Currently, if client (i.e. share/WinGit/portable-release.sh) asks
> copy-files.sh not to remove git builtins, we just keep them in bin/ and
> libexec/git-core/ . The problem is, later, when those executables are
> stripped, though initially they were identical, they all become
> different, because:
>
> For PE, strip (and everything from binutils) puts current time into
> produced object in PE header into TimeDateStamp [1]:
>
> ---- 8< ---- bfd/peXXigen.c
> H_PUT_32 (abfd, time (0), filehdr_out->f_timdat);
>
> and that hurts, if later a user wants to hardlink'ify portable git
> install.
>
> Though strip has --preserve-dates, as said above, it does not affect the
> _contents_ of the generated object file - only mtime/ctime are
> preserved.
>
> Let's preserve builtins ourselves - we already keep list of them in
> etc/fileList-builtins.txt for msysgit installer (it relinks them at
> install time), and for portable version, lets first remove all builtins

s/lets/let's/

> except git.exe, then strip, and then restore builtins bit-to-bit equal
> to git.exe . This way, later hardlinkifying will work.

s/exe ./exe./

[Johannes Schindelin]

Hi Kirill,

On Mon, 29 Apr 2013, Kirill Smelkov wrote:

> Currently, if client (i.e. share/WinGit/portable-release.sh) asks
> copy-files.sh not to remove git builtins, we just keep them in bin/ and
> libexec/git-core/ . The problem is, later, when those executables are
> stripped, though initially they were identical, they all become
> different, because:
>
> For PE, strip (and everything from binutils) puts current time into
> produced object in PE header into TimeDateStamp [1]:
>
> ---- 8< ---- bfd/peXXigen.c
> H_PUT_32 (abfd, time (0), filehdr_out->f_timdat);
>
> and that hurts, if later a user wants to hardlink'ify portable git
> install.
>
> Though strip has --preserve-dates, as said above, it does not affect the
> _contents_ of the generated object file - only mtime/ctime are
> preserved.
>
> Let's preserve builtins ourselves - we already keep list of them in
> etc/fileList-builtins.txt for msysgit installer (it relinks them at
> install time), and for portable version, lets first remove all builtins
> except git.exe, then strip, and then restore builtins bit-to-bit equal
> to git.exe . This way, later hardlinkifying will work.

Thanks for the patch and all the research that went into it. I'm awfully
short on time -- again! -- but your nice presentation made it very easy
for me to review the patch. I committed and pushed it.

Ciao,
Dscho

[Kirill Smelkov]

Hi Johannes, Eric, everyone,

On Mon, Apr 29, 2013 at 03:01:07PM -0500, Johannes Schindelin wrote:
> Hi Kirill,
>

> On Mon, 29 Apr 2013, Kirill Smelkov wrote:
>
> > Currently, if client (i.e. share/WinGit/portable-release.sh) asks
> > copy-files.sh not to remove git builtins, we just keep them in bin/ and
> > libexec/git-core/ . The problem is, later, when those executables are
> > stripped, though initially they were identical, they all become
> > different, because:
> >
> > For PE, strip (and everything from binutils) puts current time into
> > produced object in PE header into TimeDateStamp [1]:
> >
> > ---- 8< ---- bfd/peXXigen.c
> > H_PUT_32 (abfd, time (0), filehdr_out->f_timdat);
> >
> > and that hurts, if later a user wants to hardlink'ify portable git
> > install.
> >
> > Though strip has --preserve-dates, as said above, it does not affect the
> > _contents_ of the generated object file - only mtime/ctime are
> > preserved.
> >
> > Let's preserve builtins ourselves - we already keep list of them in
> > etc/fileList-builtins.txt for msysgit installer (it relinks them at
> > install time), and for portable version, lets first remove all builtins

> > except git.exe, then strip, and then restore builtins bit-to-bit equal
> > to git.exe . This way, later hardlinkifying will work.
>

> Thanks for the patch and all the research that went into it. I'm awfully
> short on time -- again! -- but your nice presentation made it very easy
> for me to review the patch. I committed and pushed it.

Thanks a lot! Like you I'm short not only on space, but also on time --
again! and still! -- so I wish we all have more free time to work on
interesting things, etc...

Thanks again for maintaining msysGit and for accepting the patch,
Kirill


P.S. Eric, I agree about "let's". The period after git.exe was separated
on intent though, not to merge it with file name.