runc: CVE-2024-21626: high severity container breakout attack

[Aleksa Sarai]

NOTE: This advisory was sent to <security...@opencontainers.org>
two weeks ago. If you ship any Open Container Initiative software, we
highly recommend that you subscribe to our security-announce list in
order to receive more timely disclosures of future security issues. The
procedure for subscribing to security-announce is outlined here[1].

Hello,

This is a notification to vendors that use runc about a high-severity
vulnerability (CVE-2024-21626) with several exploit methods which allow
for full container breakouts due to an internal file descriptor leak.

Attached are patches which resolve this issue and provide further
hardening to prevent similar issues from happening in the future. The
provided patches apply cleanly on top of runc 1.1.11. We have also
released runc 1.1.12[3] with these patches applied.

The most severe version of this issue is assigned a CVSS of
CVSS:3.1/AV:L/AC:L/PR:N/UI:R/S:C/C:H/I:H/A:H (8.6 -- High severity). For
a full breakdown of the CVSS scoring for each sub-issue, please read
through the advisory[2] which describes each attack in more detail.

The core issue is a file descriptor leak, and while we do O_CLOEXEC all
file descriptors before executing the container code, the file
descriptor is open when doing setcwd(2) which means that the reference
can be kept alive into the container by configuring the working
directory to be a path resolved through the file descriptor (and the
non-dumpable bit is unset after execve(2) meaning that there are
multiple ways to attack this other than bad configurations).

There is also an execve(2)-based attack that makes simple verification
unworkable and was particularly hairy to fix (the patch involves doing
//go:linkname to access Go runtime internals, because the only way to
defend against it entirely is to close all unneeded file descriptors --
for the same reason that #!-based tricks meant that CVE-2019-5736
required drastic measures).

Aside from only running trusted images and never using "runc exec" on
containers, there are no generic workarounds for the issue and so users
are strongly advised to patch their installations as soon as possible.
Usage of user namespaces and LSMs like SELinux will reduce the impact of
a container breakout (and we recommend using them) but do not stop it
from happening entirely.

Credit for discovering and reporting the original vulnerability goes to
Rory McNamara from Snyk. In addition, credit goes to @lifubang from
acmcoder and Aleksa Sarai from SUSE for discovering how to adapt the
attacks in various ways to make them more severe and practical for real
SaaS workloads.

Please send any questions you have to <d...@opencontainers.org> or open
an issue on our issue tracker[4]. If you feel the issue is
security-sensitive please send a mail to <secu...@opencontainers.org>.

[1]: https://github.com/opencontainers/.github/blob/main/SECURITY.md#disclosure-distribution-list
[2]: https://github.com/opencontainers/runc/security/advisories/GHSA-xr7r-f8xq-vfvv
[3]: https://github.com/opencontainers/runc/releases/tag/v1.1.12
[4]: https://github.com/opencontainers/runc/issues/new

--
Aleksa Sarai
Senior Software Engineer (Containers)
SUSE Linux GmbH
<https://www.cyphar.com/>

[Solar Designer]

Hello Aleksa,

Thank you and others you credit for doing much more than fixing the
immediate issue, and for disclosing this in so much detail.

On Thu, Feb 01, 2024 at 07:33:01AM +1100, Aleksa Sarai wrote:
> This is a notification to vendors that use runc about a high-severity
> vulnerability (CVE-2024-21626) with several exploit methods which allow
> for full container breakouts due to an internal file descriptor leak.

> The core issue is a file descriptor leak, and while we do O_CLOEXEC all
> file descriptors before executing the container code, the file
> descriptor is open when doing setcwd(2) which means that the reference
> can be kept alive into the container by configuring the working
> directory to be a path resolved through the file descriptor (and the
> non-dumpable bit is unset after execve(2) meaning that there are
> multiple ways to attack this other than bad configurations).

What's setcwd(2)? Perhaps you meant something else?

> There is also an execve(2)-based attack that makes simple verification
> unworkable and was particularly hairy to fix (the patch involves doing
> //go:linkname to access Go runtime internals, because the only way to
> defend against it entirely is to close all unneeded file descriptors --
> for the same reason that #!-based tricks meant that CVE-2019-5736
> required drastic measures).

For reference, here are the threads you started on CVE-2019-5736 and its
exploit back in 2019:

https://www.openwall.com/lists/oss-security/2019/02/11/2
https://www.openwall.com/lists/oss-security/2019/02/13/3

In one of the messages:

https://www.openwall.com/lists/oss-security/2019/02/13/1

you mentioned having sent your "AT_THIS_ROOT patchset to LKML -- which
allows userspace processes to block resolution of magic links." What's
the current status of this effort, and does/would it help against this
new issue?

Alexander

[Aleksa Sarai]

On 2024-02-01, Solar Designer <so...@openwall.com> wrote:
> Hello Aleksa,
>
> Thank you and others you credit for doing much more than fixing the
> immediate issue, and for disclosing this in so much detail.
>
> On Thu, Feb 01, 2024 at 07:33:01AM +1100, Aleksa Sarai wrote:
> > This is a notification to vendors that use runc about a high-severity
> > vulnerability (CVE-2024-21626) with several exploit methods which allow
> > for full container breakouts due to an internal file descriptor leak.
>
> > The core issue is a file descriptor leak, and while we do O_CLOEXEC all
> > file descriptors before executing the container code, the file
> > descriptor is open when doing setcwd(2) which means that the reference
> > can be kept alive into the container by configuring the working
> > directory to be a path resolved through the file descriptor (and the
> > non-dumpable bit is unset after execve(2) meaning that there are
> > multiple ways to attack this other than bad configurations).
>
> What's setcwd(2)? Perhaps you meant something else?

I meant chdir(2), sorry. I had getcwd(3) on my mind.

> > There is also an execve(2)-based attack that makes simple verification
> > unworkable and was particularly hairy to fix (the patch involves doing
> > //go:linkname to access Go runtime internals, because the only way to
> > defend against it entirely is to close all unneeded file descriptors --
> > for the same reason that #!-based tricks meant that CVE-2019-5736
> > required drastic measures).
>
> For reference, here are the threads you started on CVE-2019-5736 and its
> exploit back in 2019:
>
> https://www.openwall.com/lists/oss-security/2019/02/11/2
> https://www.openwall.com/lists/oss-security/2019/02/13/3
>
> In one of the messages:
>
> https://www.openwall.com/lists/oss-security/2019/02/13/1
>
> you mentioned having sent your "AT_THIS_ROOT patchset to LKML -- which
> allows userspace processes to block resolution of magic links." What's
> the current status of this effort, and does/would it help against this
> new issue?

That eventually became openat2(2) which was merged in Linux 5.6.
Somewhat ironically, the switch to openat2(2) in runc was the cause of
one of the fd leaks that made this issue exploitable in runc! 😅

However, I later discovered that execve-related attacks currently cannot
be defended against -- even with a safe file descriptor inside the
container from openat2(2) passed to exeveat(AT_EMPTY_PATH) you can still
escape because the binfmt loader can open files in a way you cannot
control (the ELF loader will load shared libs and binfmt_script will
happily open everything). It might be necessary to add resolution flagst
o execveat(2) as well as a result, but I haven't gotten around to
working on that yet.

There's also some more work to do with blocking certain tricks related
to magic-links used by exploits when breaking out of containers (such as
keeping a handle to /proc/self/exe and then re-opening it for writing
afterwards). I had a talk on this topic at the last LSF/MM/BPF summit[1]
-- I have since reworked the design to block operations in a far more
systematic manner but I'm still playing around with prototypes at the
moment.

[1]: https://www.youtube.com/watch?v=NjPjEcQzCMY