The different EOL behavior between libgit2 and Git for Windows(msysGit)

[b873...@student.nsysu.edu.tw]

Hi:

I got a testing repository that it stores two blob of text file with mixed EOL.

One is MIX-more_LF.txt -> all EOLs are LF, except EOL of line 2 is CRLF.

The other on is MIX-more_CRLF.txt -> all EOLs are CRLF, except EOL of line 2 is LF.

(Those files are commit under autocrlf = false)

The autocrlf is true. (set it to true)

### Git for Windows ###

I use Git for Windows 1.9.2.

I delete those files, and run these command:

$ git checkout HEAD -f -- "MIX-more_LF.txt"

$ git checkout HEAD -f -- "MIX-more_CRLF.txt"

I got those files back, and the EOLs of them are mixed EOLs as they are stored in repository.

### libgit2 ###

I use TortoiseGit and do the following steps:

1. delete those two files

2. Revert those two files (TortoiseGit use libgit2 to perform the Revert.)

I got those files back, and the EOLs of them are all CRLF.

### Other Information ###

I tried these:

    User@PC /d/Repo/TestAutoCrlf (master)

    $ git check-attr -a "MIX-more_LF.txt"

    User@PC /d/Repo/TestAutoCrlf (master)

    $ git check-attr -a "MIX-more_CRLF.txt"

    User@PC /d/Repo/TestAutoCrlf (master)

    $ git config -l | grep core

    core.symlinks=false

    core.autocrlf=true

    core.autocrlf=false

    core.safecrlf=true

    core.bare=false

    core.repositoryformatversion=0

    core.filemode=false

    core.symlinks=false

    core.ignorecase=true

    core.logallrefupdates=true

    core.autocrlf=true

### Question ###

I got confused.

Is there anyone know about this? 

Should it keep mixed EOL or change EOL anyway?

Thank you. ^_^

Yue Lin Ho

[b873...@student.nsysu.edu.tw]

Hi:

yueli...@gmail.com於 2014年6月12日星期四UTC+8上午10時18分10秒寫道：

Attached testing repository :)

Yue Lin Ho

[Thomas Braun]

Am 12.06.2014 04:18, schrieb b873...@student.nsysu.edu.tw:
> Hi:
>
> I got a testing repository that it stores two blob of text file with
> mixed EOL.
> One is MIX-more_LF.txt -> all EOLs are LF, except EOL of line 2 is CRLF.
> The other on is MIX-more_CRLF.txt -> all EOLs are CRLF, except EOL of
> line 2 is LF.
> (Those files are commit under autocrlf = false)
>

> *The autocrlf is true. (set it to true)*
>
> *### Git for Windows ###*
>
> I use Git for Windows *1.9.2*.

> I delete those files, and run these command:
>
> $ git checkout HEAD -f -- "MIX-more_LF.txt"
> $ git checkout HEAD -f -- "MIX-more_CRLF.txt"
>
> I got those files back, and the EOLs of them are mixed EOLs as they are
> stored in repository.

> *
> *
> *### libgit2 ###*

>
> I use TortoiseGit and do the following steps:
>
> 1. delete those two files
> 2. Revert those two files (TortoiseGit use libgit2 to perform the Revert.)
>
> I got those files back, and the EOLs of them are all CRLF.
>
>

> *### Other Information ###*

>
> I tried these:
>
> User@PC /d/Repo/TestAutoCrlf (master)
> $ git check-attr -a "MIX-more_LF.txt"
>
> User@PC /d/Repo/TestAutoCrlf (master)
> $ git check-attr -a "MIX-more_CRLF.txt"
>
> User@PC /d/Repo/TestAutoCrlf (master)
> $ git config -l | grep core
> core.symlinks=false
> core.autocrlf=true
> core.autocrlf=false
> core.safecrlf=true
> core.bare=false
> core.repositoryformatversion=0
> core.filemode=false
> core.symlinks=false
> core.ignorecase=true
> core.logallrefupdates=true
> core.autocrlf=true
>
>

> *### Question ###*

>
> I got confused.
> Is there anyone know about this?

> *Should it keep mixed EOL or change EOL anyway?*

Hi,

have you checked what git on linux does?
This should be the reference and all other gits may it be git for
windows or libgit should behave like it.

[Yue Lin Ho]

Hi Thomas Braun:

​Thank you. :-)

That's good idea.​

​​No, I have not.

I never use Linux system.

But I will try:

(1) install Linux (using Virtual Box)

(2) install git on Linux

(3) push my testing repository to GitHub on Windows

(4) pull my testing repository from GitHub on Linux

(5) try the above git command on Linux

(6) try to use some tool to check EOL on Linux

right? ​^_^

Yue Lin Ho

 

[Johannes Schindelin]

Hi,

There's something even easier than that: Vagrant. It is essentially the
same you mentioned above, but much more convenient:
https://github.com/msysgit/msysgit/wiki/Vagrant

Ciao,
Johannes

[Yue Lin Ho]

Hi Johannes Schindelin:

2014-06-14 0:14 GMT+08:00 Johannes Schindelin <Johannes....@gmx.de>:

On Fri, 13 Jun 2014, Yue Lin Ho wrote:

> 2014-06-12 23:39 GMT+08:00 Thomas Braun <thomas...@virtuell-zuhause.de>:
> > Am 12.06.2014 04:18, schrieb b873...@student.nsysu.edu.tw:
> > > I got a testing repository that it stores two blob of text file with

> > > mixed EOL.
> > > One is MIX-more_LF.txt -> all EOLs are LF, except EOL of line 2 is CRLF.
> > > The other on is MIX-more_CRLF.txt -> all EOLs are CRLF, except EOL of
> > > line 2 is LF.
> > > (Those files are commit under autocrlf = false)
> > > *The autocrlf is true. (set it to true)*
> > > *### Git for Windows ###*
> > > I use Git for Windows *1.9.2*.
> > > I delete those files, and run these command:
> > > $ git checkout HEAD -f -- "MIX-more_LF.txt"
> > > $ git checkout HEAD -f -- "MIX-more_CRLF.txt"
> > > I got those files back, and the EOLs of them are mixed EOLs as they are
> > > stored in repository.

​Thank you. ^_^​

​I will try it later, then report it here.

Yue Lin Ho

[b873...@student.nsysu.edu.tw]

Hi:

Johannes Schindelin於 2014年6月14日星期六UTC+8上午12時14分33秒寫道：

> ​​No, I have not.
> I never use Linux system.
> But I will try:
> (1) install Linux (using Virtual Box)
> (2) install git on Linux
> (3) push my testing repository to GitHub on Windows
> (4) pull my testing repository from GitHub on Linux
> (5) try the above git command on Linux
> (6) try to use some tool to check EOL on Linux
>
> right? ​^_^

There's something even easier than that: Vagrant. It is essentially the
same you mentioned above, but much more convenient:
https://github.com/msysgit/msysgit/wiki/Vagrant

Ciao,
Johannes

I installed Virtual Box and Vagrant.

Run C:\msysGit\msys.bat

Run "vagrant up"

Run "vagrant ssh"

Run "make -j NO_PERL_MAKEMAKER=t install"

But with some error.

GIT_VERSION = 2.0.0.msysgit.0

    .

    .

    .

    CC reflog-walk.o

    CC refs.o

    CC remote.o

cc: internal compiler error: Killed (program cc1)

Please submit a full bug report,

with preprocessed source if appropriate.

See <file:///usr/share/doc/gcc-4.6/README.Bugs> for instructions.

make: *** [alias.o] Error 4

    CC replace_object.o

make: *** Waiting for unfinished jobs....

    CC rerere.o

cc: internal compiler error: Killed (program cc1)

Please submit a full bug report,

with preprocessed source if appropriate.

See <file:///usr/share/doc/gcc-4.6/README.Bugs> for instructions.

cc: internal compiler error: Killed (program cc1)

Please submit a full bug report,

with preprocessed source if appropriate.

See <file:///usr/share/doc/gcc-4.6/README.Bugs> for instructions.

    .

    .

    .

So, I Run the following command:

  $ sudo apt-get install python-software-properties

  $ sudo add-apt-repository ppa:git-core/ppa

  $ sudo apt-get update

  $ sudo apt-get install git

Run "git --version", it shows "git version 2.0.0"

Run "cd /vagrant"

Run "git clone https://github.com/YueLinHo/TestAutoCrlf.git"

Run "cd TestAutoCrlf"

Delete all files in the working tree via Windows Explorer.

Run "git checkout HEAD -f -- "MIX-more_LF.txt""

Got the file with mixed EOLs

Run "git checkout HEAD -f -- "MIX-more_CRLF.txt""

Got the file with mixed EOLs

(Check the EOL via Notepad++ on Windows)

Anything wrong?

Based on these test,

Can I say that core git 2.0 is identical with Git for Windows 1.9.2 on this case?

Can I say libgit2 have different behavior with core git on this case?

Thank you all. ^_^

Yue Lin Ho

[Johannes Schindelin]

Hi,

On Wed, 18 Jun 2014, b873...@student.nsysu.edu.tw wrote:

> Johannes Schindelin於 2014年6月14日星期六UTC+8上午12時14分33秒寫道：

>
> > There's something even easier than that: Vagrant. It is essentially
> > the same you mentioned above, but much more convenient:
> > https://github.com/msysgit/msysgit/wiki/Vagrant
>

Wow. I never got anything like that.

> So, I Run the following command:
> $ sudo apt-get install python-software-properties
> $ sudo add-apt-repository ppa:git-core/ppa
> $ sudo apt-get update
> $ sudo apt-get install git
>

> Run "git --version", it shows "*git version 2.0.0*"

Good.

> Run "cd /vagrant"
> Run "git clone https://github.com/YueLinHo/TestAutoCrlf.git"
> Run "cd TestAutoCrlf"
> Delete all files in the working tree via Windows Explorer.
>

> *Run "git checkout HEAD -f -- "MIX-more_LF.txt""*
> *Got the file with mixed EOLs*
>
> *Run "git checkout HEAD -f -- "MIX-more_CRLF.txt""*
> *Got the file with mixed EOLs*

>
> (Check the EOL via Notepad++ on Windows)

By now, I thoroughly forgot the details from earlier in this thread,
sorry. I am assuming that the results are the same as with Git for Windows
below.

> *Based on these test,*
> *Can I say that core git 2.0 is identical with Git for Windows 1.9.2 on
> this case?*

I think you are correct.

> *Can I say libgit2 have different behavior with core git on this case?*

With the above in mind, it is safe to state that libgit2 has different
behavior here.

That is not good!

As opposed to your analysis, which is *very* good, thank you so much for
doing that.

Could you please raise this issue on the Git mailing list,
g...@vger.kernel.org? It would be good if you could accompany the report
with a *minimal* example showing the problem, and by stating clearly what
is the behavior with libgit2-based software vs the behavior with official
Git.

Thank you!
Johannes

[Yue Lin Ho]

Hi Johannes Schindelin:

​Thank you~ ^_^

Sure. I mail to g...@vger.kernel.org, but I didn't see it on http://git.661346.n2.nabble.com/

So, I created a topic.

(see http://git.661346.n2.nabble.com/The-different-EOL-behavior-between-libgit2-based-software-and-official-Git-td7613670.html)

Thanks again. :-)

​Yue Lin Ho​

[Yue Lin Ho]

Hi Torsten Bögershausen:

2014-06-19 12:58 GMT+08:00 Torsten Bögershausen :

    Could you please raise this issue on the Git mailing list, 

    g...@vger.kernel.org <mailto:g...@vger.kernel.org>? It would be good

    if you could accompany the report
    with a *minimal* example showing the problem, and by stating
    clearly what
    is the behavior with libgit2-based software vs the behavior with
    official
    Git.

    Thank you!
    Johannes

​ Thank you~ ^_^

Sure. I mail to g...@vger.kernel.org <mailto:g...@vger.kernel.org>, but I didn't see it on http://git.661346.n2.nabble.com/

So, I created a topic.
(see http://git.661346.n2.nabble.com/The-different-EOL-behavior-between-libgit2-based-software-and-official-Git-td7613670.html)

Sorry being late, I don't think there is something wrong with Git.
And the core.autocrlf is the "old" crlf handling, which has been in Git for a long time.
It may still be useful sometimes, if you exactly know what you are doing, know exactly
which tools are doing what, convince everybody who pulls or pushes to that repo to use
the same local config.

In short: I would strongly recommend to use gitattributes, please see below.


tb@msygit ~/temp
$ git clone https://github.com/YueLinHo/TestAutoCrlf.git
Cloning into 'TestAutoCrlf'...
[snip]
$ cd TestAutoCrlf/

tb@msygit ~/temp/TestAutoCrlf (master)
$ ls
CRLF.txt  LF.txt  MIX-more_CRLF.txt  MIX-more_LF.txt  Readme.md
###### Check how the file looks like:
$ od -c MIX-more_LF.txt
0000000   L   i   n   e       1  \n   l   i   n   e       (   2   ) \r
0000020  \n   l   i   n   e       3   .  \n   t   h   i   s i   s
0000040       l   i   n   e       4  \n   l       i       n       e
0000060   N   o   .       5  \n   L   i   n   e       N   u   m b   e
0000100   r       6  \n
0000104
####### The file has one CRLF, the rest is LF, exactly how it had been
####### commited. So this is what we expect.



####### Tell Git that the file is a text file:
$ echo MIX-more_LF.txt text >.gitattributes

####### Verify that MIX-more_LF.txt is text, all other files
####### are "binary" (or "-text" in Git language)
tb@msygit ~/temp/TestAutoCrlf (master)
$ git check-attr text *
CRLF.txt: text: unspecified
LF.txt: text: unspecified
MIX-more_CRLF.txt: text: unspecified
MIX-more_LF.txt: text: set
Readme.md: text: unspecified

####### Now ask Git to normalize the line endings in the working tree
$ rm  MIX-more_LF.txt

tb@msygit ~/temp/TestAutoCrlf (master)
$ git checkout   MIX-more_LF.txt

########## Check what we got:
tb@msygit ~/temp/TestAutoCrlf (master)
$ od -c MIX-more_LF.txt
0000000   L   i   n   e       1  \r  \n   l   i   n   e       ( 2   )
0000020  \r  \n   l   i   n   e       3   .  \r  \n   t   h   i   s
0000040   i   s       l   i   n   e       4  \r  \n   l i       n
0000060       e       N   o   .       5  \r  \n   L   i   n e       N
0000100   u   m   b   e   r       6  \r  \n
0000111
######### (This is under Windows, under Linux I would expect only LF)
######### See core.eol for for information

##########
########## Now we need to normalize the file in the repo,
########## All line endings should be LF, otherwise Git
########## things the file is modified:
$ git status
On branch master
Your branch is up-to-date with 'origin/master'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

        modified:   MIX-more_LF.txt

Untracked files:
  (use "git add <file>..." to include in what will be committed)

        .gitattributes

no changes added to commit (use "git add" and/or "git commit -a")


###############
############### Do the normalization:
tb@msygit ~/temp/TestAutoCrlf (master)
$ git add MIX-more_LF.txt .gitattributes

tb@msygit ~/temp/TestAutoCrlf (master)
$ git commit -m  "MIX-more_LF.txt is text"
[master 200d874] MIX-more_LF.txt is text
 Committer: unknown <xxx@xxx>
Your name and email address were configured automatically based
on your username and hostname. Please check that they are accurate.
You can suppress this message by setting them explicitly:

    git config --global user.name "Your Name"
    git config --global user.email y...@example.com

After doing this, you may fix the identity used for this commit with:

    git commit --amend --reset-author

 2 files changed, 2 insertions(+), 1 deletion(-)
 create mode 100644 .gitattributes

tb@msygit ~/temp/TestAutoCrlf (master)
$

######## From now on, MIX-more_LF.txt is treated as text by Git,
######## and get CRLF under Windows.
######## I think there is nothing wrong with Git here.
######## If libgit2 does something different, we need to ask
######## the libgit2 project, which is independent from Git

First, thanks for you example. It's nice. ^_^

Second, I do these (Note that I use "-text", instead of "text")

    User@PC /d/Repo/TestAutoCrlf (master)

    $ echo MIX-more_LF.txt -text >.gitattributes

    User@PC /d/Repo/TestAutoCrlf (master)

    $ git check-attr text *

    CRLF.txt: text: unspecified

    LF.txt: text: unspecified

    MIX-more_CRLF-ed2b4bb.txt: text: unspecified

    MIX-more_CRLF.txt: text: unspecified

    MIX-more_LF.txt: text: unset

    Readme.md: text: unspecified

By now, libgit2 checkout "MIX-more_LF.txt" with mixed EOL.

Perhaps it is a workaround of libgit2 for me. :)

Actually, I created a issue of libgit2.

(See https://github.com/libgit2/libgit2/issues/2393)

----

The following message is my balabala... no further important information. :P

Official Git, Git for Windows, libgit2 and TortoiseGit are open source projects.

I think all of them make the world better.(At least, make my life better.)

I notice they have the different behavior.

For me, I just want to make things clear.

By now, it is seems a little bug of libgit2, so I will try to fix it. (if i could :P)

All of these things/people here and there make/help me to learn more.

So, thank all of you. ^_^

----

Yue Lin Ho

[Torsten Bögershausen]

> Could you please raise this issue on the Git mailing list,
> ​​

> g...@vger.kernel.org <mailto:g...@vger.kernel.org>? It would be good

> if you could accompany the report
> with a *minimal* example showing the problem, and by stating
> clearly what
> is the behavior with libgit2-based software vs the behavior with
> official
> Git.
>
> Thank you!
> Johannes
>
>
> ​ Thank you~ ^_^
>

> Sure. I mail to g...@vger.kernel.org <mailto:g...@vger.kernel.org>, but

> I didn't see it on http://git.661346.n2.nabble.com/
> So, I created a topic.
> (see
> http://git.661346.n2.nabble.com/The-different-EOL-behavior-between-libgit2-based-software-and-official-Git-td7613670.html)
>
>

Committer: unknown <torstenbo...@torbogws01.esss.lu.se>