Fixing git-svn on msys
Marten Svanfeldt
As some others, git-svn is important to me so I spent a night
investigating the recent breakage. I've found the reason and also a
number of solutions, but would like some hints on which one you prefer
before spending (alot) of time implementing them.
The problem is in the "recent" optimizations done to git-svn upstream.
With commit b70f8ef (and later fixes to temp-file usage) git-svn
changed from piping the file data through git-hash-object to piping
the filename of the temporary file to git-hash-object --stdin-paths.
The paths to those temporary files are generated by Perl (perl-msys in
this case) and use msys-style paths (so "/tmp/...") while git-hash-
object (and other git tools) use "open" system call which requires
windows paths. Hence its unable to open those temporary files.
Proposed solutions I came up with is:
1. Make Git.pm use a dir below current one for temporary files (or
even current dir). Results in temporary files with relative paths,
which works fine.
2. Make Git.pm/git-svn unmangle the paths to native windows paths
before sending them to git-hash-object
3. Fix mingw_open to handle msys paths as well as native windows
paths.
Any suggestions which solution would be prefered?
-Marten Svanfeldt
Johannes Schindelin
I considered including a MinGW Perl instead of an MSys Perl. This would
have a few advantages, not the least of it would be higher speed (as it
would no longer have to use that POSIX-layer).
The downside is that you'd have to compile subversion in the same setting,
which is probably a bit involved -- even if using apr should abstract out
problems with lack of POSIX in the Win32 API -- because AFAIR the build
process for pure Windows is too much optimized for Microsoft's
Visualstudio.
Ciao,
Dscho
P.S.: please subscribe so that your messages do not have to be moderated.
Johannes Sixt
> The problem is in the "recent" optimizations done to git-svn upstream.
> With commit b70f8ef (and later fixes to temp-file usage) git-svn
> changed from piping the file data through git-hash-object to piping
> the filename of the temporary file to git-hash-object --stdin-paths.
> The paths to those temporary files are generated by Perl (perl-msys in
> this case) and use msys-style paths (so "/tmp/...") while git-hash-
> object (and other git tools) use "open" system call which requires
> windows paths. Hence its unable to open those temporary files.
>
> Proposed solutions I came up with is:
>
> 1. Make Git.pm use a dir below current one for temporary files (or
> even current dir). Results in temporary files with relative paths,
> which works fine.
This is the preferred solution...
> 2. Make Git.pm/git-svn unmangle the paths to native windows paths
> before sending them to git-hash-object
... because this one is just oh so yucky that it's unlikely that Eric
accepts it.
> 3. Fix mingw_open to handle msys paths as well as native windows
> paths.
This one's a no-go. mingw_open would have to know about how msys locates
the msys-style paths.
-- Hannes
Marten Svanfeldt
> Marten Svanfeldt schrieb:
> > Proposed solutions I came up with is:
>
> > 1. Make Git.pm use a dir below current one for temporary files (or
> > even current dir). Results in temporary files with relative paths,
> > which works fine.
>
> This is the preferred solution...
>
> > 2. Make Git.pm/git-svn unmangle the paths to native windows paths
> > before sending them to git-hash-object
>
> ... because this one is just oh so yucky that it's unlikely that Eric
> accepts it.
rely on temp-files being in certain directories.
>
> > 3. Fix mingw_open to handle msys paths as well as native windows
> > paths.
>
> This one's a no-go. mingw_open would have to know about how msys locates
> the msys-style paths.
I did also consider the last part, upgrading to a native Win32 Perl. I
think you are right in that building subversion under MinGW might take
some work, but it wouldn't be the first time I had to spend some time
getting a project to build in Windows. One could also build a Perl/
Subversion combination with MSVC...
-Marten
Johannes Schindelin
On Thu, 6 Nov 2008, Marten Svanfeldt wrote:
> I did also consider the last part, upgrading to a native Win32 Perl. I
> think you are right in that building subversion under MinGW might take
> some work, but it wouldn't be the first time I had to spend some time
> getting a project to build in Windows. One could also build a Perl/
> Subversion combination with MSVC...
You probably meant me (Dscho, instead of Hannes).
I am pretty much unwilling to spend any amount of money on Microsoft,
especially to buy myself the pain that is MSVC, so you would lose one
potential contributor. As a consequence, you would also use a potential
user, since I refuse to use software that I cannot fix, if there is an
alternative.
However, if you are happy to maintain that combination, I am sure msysGit
users would be thankful.
Ciao,
Dscho
Marten Svanfeldt
> You probably meant me (Dscho, instead of Hannes).
>
> I am pretty much unwilling to spend any amount of money on Microsoft,
> especially to buy myself the pain that is MSVC, so you would lose one
> potential contributor. As a consequence, you would also use a potential
> user, since I refuse to use software that I cannot fix, if there is an
> alternative.
>
> However, if you are happy to maintain that combination, I am sure msysGit
> users would be thankful.
to spend any money. However, I understand this objection, even though
I am a MSVC user I try to get everything I write to work with mingw/
msys combination. I will look into how much pain it would be to get
Perl+subversion to compile to native on mingw; if it is next to
impossible I might provide a MSVC-compiled version for now (and/or the
other fix with relative dirs).
>
> Ciao,
> Dscho
-Marten
Peter Harris
>
> The problem is in the "recent" optimizations done to git-svn upstream.
> With commit b70f8ef (and later fixes to temp-file usage) git-svn
> changed from piping the file data through git-hash-object to piping
> the filename of the temporary file to git-hash-object --stdin-paths.
> The paths to those temporary files are generated by Perl (perl-msys in
> this case) and use msys-style paths (so "/tmp/...") while git-hash-
> object (and other git tools) use "open" system call which requires
> windows paths. Hence its unable to open those temporary files.
>
> Proposed solutions I came up with is:
>
> 1. Make Git.pm use a dir below current one for temporary files (or
> even current dir). Results in temporary files with relative paths,
> which works fine.
Or maybe $ENV{GIT_DIR}, possibly even $ENV{GIT_DIR}/tmp. I've been
doing this locally, but it smells funny to me, so I don't want to send
it to Junio.
> 2. Make Git.pm/git-svn unmangle the paths to native windows paths
> before sending them to git-hash-object
>
> 3. Fix mingw_open to handle msys paths as well as native windows
> paths.
>
> Any suggestions which solution would be prefered?
4. Replace Msys perl with Mingw (vanilla/strawberry) perl. As others
have mentioned, the hardest part of this is rebuilding subversion and
everything that subversion depends on.
5. Modify msysgit\lib\perl5\5.8.8\File\Spec.pm to use the Win32
module, even when it detects Msys/Cygwin? This should produce a
Win32-style temporary dir, but I'm not sure what else that would
impact. It's a bit of a hack, but it has two advantages: 1) doesn't
cause git-svn to deviate from upstream, and 2) would be easier than
replacing all of Perl with the mingw version.
Peter Harris
Sean Blanton
my core. I was trying to recruit someone from the Chicago Perl users group.
For Git to be maximally successful on Windows, which I would like to
see, it has to work in a typical Windows environment. Currently, msysGit
does just that, even though there are issues such as git-svn.
For example, I've been using msysGit with automation tools
(Mojo/Meister) that on Windows call the simple git programs from cmd.exe
and at th same time running Perl scripts with ActiveState Perl and its
been working great even with an msys Perl in there. Add a Windows
explorer interface like Cheetah and you have that recipe for broad success.
A typical Windows developer in a typical development team will rarely,
if ever, go to a bash or a cmd prompt, they will stick with ActivePerl
for their other Perl needs and Git should be like a black box tool set.
It looks like the project has all the pieces, but I should ask is that a
goal of this project?
In this scenario, I believe the requirement would be for a minimal Perl
(still able to build modules from CPAN) to satisfy the minimal msysGit
needs. Is that what you are shooting for?
I did enjoy reading the history of Perl in the MsysGit Herald and in the
process eliminated a bit of my naivety about the problems. Any chance to
read more of your witty stylings, Dscho, in future Heralds?
Johannes Schindelin
On Thu, 6 Nov 2008, Sean Blanton wrote:
> I did enjoy reading the history of Perl in the MsysGit Herald and in the
> process eliminated a bit of my naivety about the problems. Any chance to
> read more of your witty stylings, Dscho, in future Heralds?
Heh, might be possible in the near future...
Thanks for the encouragement,
Dscho
Marten Svanfeldt
wrote:
>
> Then I'll make a patch for that and send later.
>
>
submit it to you. Which branch should I create patches against and how
do you want them? Via the tracker or somewhere else?
I also have a couple of fixes to Perl scripts for doc building, but
this does not work 100% yet so no patches yet to submit.
-Marten
Johannes Schindelin
On Wed, 12 Nov 2008, Marten Svanfeldt wrote:
> On Nov 6, 7:56 pm, Marten Svanfeldt <marten.svanfe...@gmail.com> wrote:
> > On Nov 6, 5:02 pm, Johannes Sixt <j.s...@viscovery.net> wrote:
> >
> > Then I'll make a patch for that and send later.
>
> Okay, I fixed the Perl scripts for this, however I am unsure how to
> submit it to you. Which branch should I create patches against and how
> do you want them? Via the tracker or somewhere else?
Preferably to this mailing list and to the Git mailing list, probably Cc:
Eric Wong, too.
If you committed the changes in /git/, you should be fine with the output
of git format-patch, the branch you developed against should not matter
all that much.
For more information how the Git project prefers submissions, please refer
to Documentation/SubmittingPatches.txt in /git/.
> I also have a couple of fixes to Perl scripts for doc building, but this
> does not work 100% yet so no patches yet to submit.
Looking forward to that!
Ciao,
Dscho