Re: Probably a bug with "~" symbol in filenames on Windows 7 x64 in git 1.9.5

[Johannes Schindelin]

Hi Junio,

On Wed, 7 Jan 2015, Junio C Hamano wrote:

> Dscho, this sounds to me like the additional "8.3 ambiguity
> protection" (which is only in Git for Windows) in action. Any
> thoughts?

First thought: the Git for Windows mailing list should be Cc:ed (I was
traveling yesterday and somebody else might have been able to address
Dmitry's problem).

Second thought below:

> On Wed, Jan 7, 2015 at 3:26 PM, Dmitry Bykov <pvr...@gmail.com> wrote:
> >
> > Recently I installed 1.9.5 git version and faced the problem that one
> > of the files in my cloned repository with a name ICON~714.PNG is
> > marked as deleted even repository was freshly cloned. Trying to do
> > anything with that file resulted in constant "Invalid Path" errors.
> > Reverting back to 1.9.4. fixed that problem.

ICON~714.PNG is a valid short name for a long name (such as
'icon.background.png') because it fits the shortening scheme (8.3 format,
the base name ends in ~<n>). As this can clash with a validly shortened
long name, Git for Windows refuses to check out such paths by default.

If you want the old -- unsafe -- behavior back, just set your
core.protectNTFS to false (this means that you agree that the incurred
problems are your own responsibility and cannot be blamed on anybody else
;-))

Ciao,
Johannes

[Jeff King]

On Thu, Jan 08, 2015 at 11:06:18AM +0100, Johannes Schindelin wrote:

> ICON~714.PNG is a valid short name for a long name (such as
> 'icon.background.png') because it fits the shortening scheme (8.3 format,
> the base name ends in ~<n>). As this can clash with a validly shortened
> long name, Git for Windows refuses to check out such paths by default.
>
> If you want the old -- unsafe -- behavior back, just set your
> core.protectNTFS to false (this means that you agree that the incurred
> problems are your own responsibility and cannot be blamed on anybody else
> ;-))

I wonder if it is worth having a "git-only" mode for core.protectNTFS.
Turning it off entirely would make him susceptible to GIT~1 attacks.

-Peff

[Johannes Schindelin]

Hi Peff,

That is a good idea!

Ciao,
Dscho

[Torsten Bögershausen]

There is something more then just the "tilde" protection going on, it seems:


torstenbogershausen@TORBOGWM ~/projects/test_tilde (master)
$ ~/projects/git/msysgit/git ls-files
ICON~714.PNG

torstenbogershausen@TORBOGWM ~/projects/test_tilde (master)
$ ~/projects/git/msysgit/git status
On branch master
nothing to commit, working directory clean

torstenbogershausen@TORBOGWM ~/projects/test_tilde (master)
$ ~/projects/git/tb/git status
On branch master
nothing to commit, working directory clean

torstenbogershausen@TORBOGWM ~/projects/test_tilde (master)
$ ~/projects/git/msysgit/git --version
git version 1.9.2.msysgit.0.2273.gc47d6ec

torstenbogershausen@TORBOGWM ~/projects/test_tilde (master)
$ ~/projects/git/tb/git --version
git version 2.2.1.268.g1e6f5b2

Neither latest msygit nor latest git.git reports a problem with a single
file
within a short path.
It may be, that the '~' is not accepted in a long path or there are
"problems"
when a directory is filled with many files, but that is speculation.

Dimitry,
is there a way to make the problem reproducible for other people ?
Either a public demo-repo, or a step-by-step receipe ?

[Johannes Schindelin]

Hi Torsten,

On Thu, 8 Jan 2015, Torsten Bögershausen wrote:

> There is something more then just the "tilde" protection going on, [...]

Indeed. What is going on is that you build Git yourself, from git.git,
while Dmitry obviously used Git for Windows -- which carries a couple of
patches on top of upstream git.git.

In this particular case, the tilde protection was introduced in
https://github.com/msysgit/git/commit/2e2a2d12.

Ciao,
Johannes

[Torsten Bögershausen]

My bad, I was building the msysgit master branch, which didn't have the 2e2a2d12.

However, I am wondering if 2e2a2d12 is really needed that much ?
In the same spirit that we do not prevent the checkout of "FILE" when "file" is present,
why do we prevent people from using "~" in file names ?
When a file is visible under different file names, I assume that the user chooses only
one representation to run "git add" on.
But that is another discussion.

[Junio C Hamano]

Yes, I think having distinctions would make sense, but I am not sure
if "git-only" vs "every questionable included" should be the
classification.

To classify various funniness of paths by the extent of damage they
may cause for Windows:

- ".Git" and "GIT~1" would be the worst level;

- "CON.c" and the like would be the next bad level; and

- "ICON~714.PNG" would be in "to be avoided in an ideal world but
as long as the project is comfortable with having it in, Git is
in no position to complain" level.

that is just my knee-jerk reaction.

And if we were to go with just two level, I'd throw "CON.c" into the
same level as ".Git"; they both are to break the host that checks
out such paths. The former may break Git on the host, and the
latter may break something that is not Git on the host, but they are
not that different in that the end user on that host is harmed.

"8.3 ambiguity" does not directly harm individual hosts, but the
reason to flag is primarily because such a path may make the
project's contents unreliable (e.g. "icon1234234.png" may alias
"ICON~714.PNG" on somebody's machine, causing confusion). The same
reason should make us flag a tree object as suspect if it contains
two paths that are the same case-insensitively (e.g. a tree that
represents the "net/netfilter" part of the Linux kernel source would
be flagged because it contains both "xt_tcpmss.c" and "xt_TCPMSS.c").

Let's call the former Harmful and the latter Questionable. "CON.c"
is Harmful on Windows, while "net/netfilter" is Questionable on
Windows. Orthogonal to this, we earlier discussed what to do in
fsck and receive.fsckObjects. I think we want to be express
different shades of grays between the two extremes:

- A repository of a mono-culture project would want to flag
Harmfuls and Questionables that apply to the project's intended
platform as errors. By definition, a mono-culture project would
not care about paths that may only be Harmful on others'
platforms.

- A repository for a cross-platform project would want to flag
paths that may be Harmful or Questionable on some platform, and
does not care if these paths are Harmful or Questionable on the
platform that happens to host the repository. By definition, a
cross-platform project wants to make sure everybody involved will
not get hurt.

[Jeff King]

On Thu, Jan 08, 2015 at 11:08:47AM -0800, Junio C Hamano wrote:

> > I wonder if it is worth having a "git-only" mode for core.protectNTFS.
> > Turning it off entirely would make him susceptible to GIT~1 attacks.
>
> Yes, I think having distinctions would make sense, but I am not sure
> if "git-only" vs "every questionable included" should be the
> classification.

Yeah, I agree with everything you said below (I should have said
"security-relevant" instead of "git-only" originally). I also agree that
there may be more than two classifications, but I do not know enough
about the severity of the bad effects of each type to judge. I'd leave
that to the folks on each platform.

-Peff