Bug#750687: "git status" becomes fork-bomb under some circumstances

[Axel Beckert]

Package: git
Version: 1:2.0.0~rc2-1
Severity: important
Affects: hobbit-plugins
Control: found -1 1:2.0.0-1

Hi,

I couldn't willingly reproduce it yet outside the setup where I ran
into that issue, so here's primarily a description of symptoms plus
some observations.

My current suspection is that it's permissions related:

abe@acromantula:/etc/apache2.abe$ ls -ld . .git
drwxr-xr-x 16 root root 4096 Jun 10 2011 .
drwxrwx--- 9 root 112 4096 Jun 5 18:33 .git
abe@acromantula:/etc/apache2.abe$ id
uid=1000(abe) gid=1000(abe) groups=1000(abe)

But a relation to etckeeper and git submodules cannot be ruled out.
Will investigate further after having sent this report.

abe@acromantula:/etc/apache2.abe$ strace -f -e fork git status
Process 11354 attached
Process 11355 attached
Process 11358 attached
Process 11359 attached
Process 11362 attached
Process 11365 attached
Process 11366 attached
Process 11367 attached
Process 11370 attached
Process 11371 attached

In some other shell I ran the following command at that point:

10/0/0 root@acromantula:pts/4 18:53:47 [~] # ps auxwwwf | fgrep git
abe 11352 77.1 0.0 4552 596 pts/2 R+ 18:53 0:10 | | \_ strace -f -e fork git status
abe 11353 6.3 0.1 24632 6848 pts/2 S+ 18:53 0:00 | | \_ git status
abe 11354 7.3 0.1 24632 6856 pts/2 S+ 18:53 0:00 | | \_ git status --porcelain
abe 11355 8.0 0.1 24632 6856 pts/2 S+ 18:53 0:00 | | \_ git status --porcelain
abe 11358 9.8 0.1 24632 6852 pts/2 S+ 18:53 0:00 | | \_ git status --porcelain
abe 11359 12.5 0.1 24632 6856 pts/2 S+ 18:53 0:00 | | \_ git status --porcelain
abe 11362 17.6 0.1 24632 6856 pts/2 S+ 18:53 0:00 | | \_ git status --porcelain
abe 11365 22.2 0.1 24632 6852 pts/2 S+ 18:53 0:00 | | \_ git status --porcelain
abe 11366 44.5 0.1 24632 6856 pts/2 S+ 18:53 0:00 | | \_ git status --porcelain
abe 11367 0.0 0.1 24632 6756 pts/2 t+ 18:53 0:00 | | \_ git status --porcelain
abe 11081 0.7 1.0 260160 38456 pts/3 S 18:47 0:02 | \_ /usr/bin/python /usr/bin/reportbug git
abe 11139 0.0 0.0 4316 384 pts/3 S 18:49 0:00 | \_ sh -c emacsclient -a emacs +6 '/tmp/reportbug-git-20140605-11081-PM8nfv'
abe 11140 0.2 0.4 211336 16588 pts/3 S+ 18:49 0:00 | \_ emacs +6 /tmp/reportbug-git-20140605-11081-PM8nfv
root 11369 0.0 0.0 10472 764 pts/4 S+ 18:53 0:00 \_ fgrep git
11/0/0 root@acromantula:pts/4 18:53:52 [~] #

If I don't use strace, the forking happens much quicker and when I
initially noticed it I had hundereds (if not thousands or ten
thousands) of "git status --porcelain" running and close to all RAM
and swap was used up (together 10 GB).

So this issue has quite some potential to bring down a system within
minutes and trigger an OOM condition. I didn't file it as RC because I
couldn't reproduce it on some other system yet, i.e. I don't know yet
how common the conditions are under which it can happen. (Feel free to
raise it to RC anyways. :-)

I initially run into it on a system freshly dist-upgraded from Wheezy
to Jessie where I enabled hobbit-plugins' dirtvcs monitoring
plugin. (It wasn't enabled before, so I can't tell if it would have
happened on Wheezy, too.)

The issue is present with both, the version in Jessie (1:2.0.0~rc2-1)
and the version in Sid (1:2.0.0-1).

-- System Information:
Debian Release: jessie/sid
APT prefers testing
APT policy: (990, 'testing'), (200, 'unstable')
Architecture: amd64 (x86_64)

Kernel: Linux 3.14-1-amd64 (SMP w/2 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash

Versions of packages git depends on:
ii git-man 1:2.0.0-1
ii libc6 2.18-7
ii libcurl3-gnutls 7.37.0-1
ii liberror-perl 0.17-1.1
ii libexpat1 2.1.0-5
ii libpcre3 1:8.31-5
ii perl-modules 5.18.2-4
ii zlib1g 1:1.2.8.dfsg-1

Versions of packages git recommends:
ii less 458-2
ii openssh-client [ssh-client] 1:6.6p1-5
ii patch 2.7.1-5
ii rsync 3.1.0-3

Versions of packages git suggests:
ii gettext-base 0.18.3.2-1
pn git-arch <none>
ii git-cvs 1:2.0.0-1
pn git-daemon-run | git-daemon-sysvinit <none>
pn git-doc <none>
pn git-el <none>
pn git-email <none>
ii git-gui 1:2.0.0-1
pn git-mediawiki <none>
ii git-svn 1:2.0.0-1
ii gitk 1:2.0.0-1
pn gitweb <none>

-- no debconf information


--
To UNSUBSCRIBE, email to debian-bugs-...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org

[Axel Beckert]

Control: retitle -1 "git status" becomes fork-bomb if a submodule's .git directory is not accessible
Control: severity -1 grave
Control: found -1 1:1.9.1-1~bpo70+2
Control: found -1 1:1.7.10.4-1+wheezy1

Hi,

Axel Beckert wrote:
> I couldn't willingly reproduce it yet outside the setup where I ran
> into that issue,

Here's how to reproduce:

root@kiva6 [~] # mkdir -pv /tmp/bar/foo
mkdir: created directory ‘/tmp/bar’
mkdir: created directory ‘/tmp/bar/foo’
root@kiva6 [~] # cd /tmp/bar/
root@kiva6 [/tmp/bar] # git init
Initialized empty Git repository in /tmp/bar/.git/
root@kiva6 [/tmp/bar] # cd foo/
root@kiva6 [/tmp/bar/foo] # git init
Initialized empty Git repository in /tmp/bar/foo/.git/
root@kiva6 [/tmp/bar/foo] # echo foo > foo
root@kiva6 [/tmp/bar/foo] # git add foo
root@kiva6 [/tmp/bar/foo] # git commit -m foo
[master (root-commit) 41cb8de] foo
1 file changed, 1 insertion(+)
create mode 100644 foo
root@kiva6 [/tmp/bar/foo] # cd ..
root@kiva6 [/tmp/bar] # git submodule add ./foo
Adding existing repo at 'foo' to the index
root@kiva6 [/tmp/bar] # git commit -m foo
[master (root-commit) f86b7be] foo
2 files changed, 4 insertions(+)
create mode 100644 .gitmodules
create mode 160000 foo
root@kiva6 [/tmp/bar] # chmod 700 -c foo/.git
mode of ‘foo/.git’ changed from 0755 (rwxr-xr-x) to 0700 (rwx------)
root@kiva6 [/tmp/bar] # su - abe -s /bin/bash
abe@kiva6 [~] $ cd /tmp/bar/foo
abe@kiva6 [/tmp/bar/foo] $ git status
[Fork bomb starts]

Since I consider such a setup not too seldom (especially with
etckeeper and some shell prompt using "git status" to show some
information), I'm raising the severity to grave.

I could reproduce this issue also with the versions of the git package
currently in wheezy and wheezy-backports.

For convenience, here's a shell script setting up a minimal scenario.
It needs to be run as user "root" to make the "su" command at the end
work. The remainder may also work as normal user.

---8<---
#!/bin/sh

rm -rv /tmp/bar
mkdir -pv /tmp/bar/foo
cd /tmp/bar/
git init
cd foo/
git init
echo foo > foo
git add foo
git commit -m foo
cd ..
git submodule add ./foo
git commit -m foo
chmod 700 -c foo/.git

echo ''
echo 'Now type "git status" or e.g. "strace -f -e open git status". Press Ctrl-C to abort the fork-bomb.'
su nobody -s /bin/bash
--->8---

Regards, Axel
--
,''`. | Axel Beckert <a...@debian.org>, http://people.debian.org/~abe/
: :' : | Debian Developer, ftp.ch.debian.org Admin
`. `' | 1024D: F067 EA27 26B9 C3FC 1486 202E C09E 1D89 9593 0EDE
`- | 4096R: 2517 B724 C5F6 CA99 5329 6E61 2FF9 CD59 6126 16B5

[Jonathan Nieder]

Hi,

Axel Beckert wrote:

> 10/0/0 root@acromantula:pts/4 18:53:47 [~] # ps auxwwwf | fgrep git
> abe 11352 77.1 0.0 4552 596 pts/2 R+ 18:53 0:10 | | \_ strace -f -e fork git status
> abe 11353 6.3 0.1 24632 6848 pts/2 S+ 18:53 0:00 | | \_ git status
> abe 11354 7.3 0.1 24632 6856 pts/2 S+ 18:53 0:00 | | \_ git status --porcelain
> abe 11355 8.0 0.1 24632 6856 pts/2 S+ 18:53 0:00 | | \_ git status --porcelain
> abe 11358 9.8 0.1 24632 6852 pts/2 S+ 18:53 0:00 | | \_ git status --porcelain

Yeah, this is a submodule issue. What is the output of

git ls-files -s | grep ^160000 | cut -d$'\t' -f2 | xargs ls -ld

?

[...]

> So this issue has quite some potential to bring down a system within
> minutes and trigger an OOM condition.

That's a general feature of fork bombs. I think we should try to
figure out why git is recursing into the same repository again and
again --- the fork bomb mitigation aspect is less interesting to me
(though certainly a worthwhile thing to work on for kernel hackers
or people coming up with default rlimits).

Thanks,
Jonathan

[Axel Beckert]

Hi Jonathan,

I've sent my previous mail before I received your's. :-/

Jonathan Nieder wrote:
> > 10/0/0 root@acromantula:pts/4 18:53:47 [~] # ps auxwwwf | fgrep git
> > abe 11352 77.1 0.0 4552 596 pts/2 R+ 18:53 0:10 | | \_ strace -f -e fork git status
> > abe 11353 6.3 0.1 24632 6848 pts/2 S+ 18:53 0:00 | | \_ git status
> > abe 11354 7.3 0.1 24632 6856 pts/2 S+ 18:53 0:00 | | \_ git status --porcelain
> > abe 11355 8.0 0.1 24632 6856 pts/2 S+ 18:53 0:00 | | \_ git status --porcelain
> > abe 11358 9.8 0.1 24632 6852 pts/2 S+ 18:53 0:00 | | \_ git status --porcelain
>
> Yeah, this is a submodule issue.

Indeed. See my other mail.

> What is the output of
>
> git ls-files -s | grep ^160000 | cut -d$'\t' -f2 | xargs ls -ld

For the proof of concept example in my other mail:

# git ls-files -s | grep ^160000 | cut -d$'\t' -f2 | xargs ls -ld
drwxr-xr-x 3 root root 80 Jun 5 21:12 foo

On the original system:

drwxr-xr-x 16 root root 4096 Jun 10 2011 etc/apache2
drwxr-xr-x 3 root root 4096 Jun 10 2011 etc/dokuwiki
drwxr-xr-x 3 root root 4096 Apr 5 2011 export/data1/wiki/global-include
drwxr-xr-x 3 root root 4096 Jun 10 2011 usr/share/dokuwiki/inc/auth
drwxr-xr-x 7 root root 4096 Jun 10 2011 var/lib/dokuwiki/lib/plugins/discussion
drwxr-xr-x 4 root root 4096 Jun 10 2011 var/lib/dokuwiki/lib/plugins/svgpureInsert
drwxr-xr-x 6 root root 4096 Jun 10 2011 var/lib/dokuwiki/lib/tpl

Doesn't look too suspicious to me. But it makes me suspecting that
there are more options where a "chmod 700" would cause such breakage.
:-)

> > So this issue has quite some potential to bring down a system within
> > minutes and trigger an OOM condition.
>
> That's a general feature of fork bombs.

Sure. :-)

> I think we should try to figure out why git is recursing into the
> same repository again and again

Definitely. That's why I wrote this bug report. :-)

Regards, Axel
--
,''`. | Axel Beckert <a...@debian.org>, http://people.debian.org/~abe/
: :' : | Debian Developer, ftp.ch.debian.org Admin
`. `' | 1024D: F067 EA27 26B9 C3FC 1486 202E C09E 1D89 9593 0EDE
`- | 4096R: 2517 B724 C5F6 CA99 5329 6E61 2FF9 CD59 6126 16B5

[Axel Beckert]

Hi Jonathan,

Jonathan Nieder wrote:
> > Since I consider such a setup not too seldom (especially with
> > etckeeper and some shell prompt using "git status" to show some
> > information), I'm raising the severity to grave.
>

> That's way overinflated (it would not be worth blocking a release of
> Debian).

Hrm, I'm not really convinced of that. But I accept the maintainer's
decision.

> But thanks much for tracking it down.

You're welcome.

[Greg Price]

Control: fixed -1 1:2.20.1-2+deb10u1
Control: fixed -1 1:2.26.2-1~bpo10+1

I tried reproducing this on the Git 2.20 in buster (with the
reproduction recipe from message #14), and it seems it
has been fixed! Same with the Git 2.26 backported from bullseye.

Instead of a fork bomb, there's an error message:

nobody@678fb692c9b5:/tmp/bar$ git status
fatal: 'foo/.git' not recognized as a git repository

which seems pretty OK.

Cheers,
Greg

[Axel Beckert]

Hi Greg,

Greg Price wrote:

> On Wed, Apr 29, 2020 at 12:49 AM Greg Price <gnp...@gmail.com> wrote:
> > I tried reproducing this on the Git 2.20 in buster (with the
> > reproduction recipe from message #14), and it seems it
> > has been fixed! Same with the Git 2.26 backported from bullseye.
> >
> > Instead of a fork bomb, there's an error message:
>

> I did a bit more testing, this time with containers running stretch
> and jessie. The bug reproduces in jessie -- no surprise, given the
> original report. But as of stretch, which at this point is oldstable,
> it seems to already be fixed. So I believe the bug can be closed.

Thanks for checking and closing.

Haven't run into it again either. (Didn't try to provoke it though
neither.)

Regards, Axel
--
,''`. | Axel Beckert <a...@debian.org>, https://people.debian.org/~abe/
: :' : | Debian Developer, ftp.ch.debian.org Admin
`. `' | 4096R: 2517 B724 C5F6 CA99 5329 6E61 2FF9 CD59 6126 16B5
`- | 1024D: F067 EA27 26B9 C3FC 1486 202E C09E 1D89 9593 0EDE