Re: Kernel hardening project suggestion: Normalizing ->ctor slabs and TYPESAFE_BY_RCU slabs
1 view
Skip to first unread message
Dmitry Vyukov
Jun 23, 2020, 2:45:23 AM6/23/20
to Jann Horn, Kernel Hardening, Christoph Lameter, Pekka Enberg, David Rientjes, Joonsoo Kim, Andrew Morton, Linux-MM, Andrey Konovalov, Will Deacon, kasan-dev, Kees Cook, Alexander Potapenko, Marco Elver
On Tue, Jun 23, 2020 at 8:26 AM Jann Horn <ja...@google.com> wrote:
>
> Hi!
>
> Here's a project idea for the kernel-hardening folks:
>
> The slab allocator interface has two features that are problematic for
> security testing and/or hardening:
>
> - constructor slabs: These things come with an object constructor
> that doesn't run when an object is allocated, but instead when the
> slab allocator grabs a new page from the page allocator. This is
> problematic for use-after-free detection mechanisms such as HWASAN and
> Memory Tagging, which can only do their job properly if the address of
> an object is allowed to change every time the object is
> freed/reallocated. (You can't change the address of an object without
> reinitializing the entire object because e.g. an empty list_head
> points to itself.)
>
> - RCU slabs: These things basically permit use-after-frees by design,
> and stuff like ASAN/HWASAN/Memory Tagging essentially doesn't work on
> them.
>
>
> It would be nice to have a config flag or so that changes the SLUB
> allocator's behavior such that these slabs can be instrumented
> properly. Something like:
>
> - Let calculate_sizes() reserve space for an rcu_head on each object
> in TYPESAFE_BY_RCU slabs, make kmem_cache_free() redirect to
> call_rcu() for these slabs, and remove most of the other
> special-casing, so that KASAN can instrument these slabs.
> - For all constructor slabs, let slab_post_alloc_hook() call the
> ->ctor() function on each allocated object, so that Memory Tagging and
> HWASAN will work on them.
Hi Jann,
Both things sound good to me. I think we considered doing the ctor's
change with KASAN, but we did not get anywhere. The only argument
against it I remember now was "performance", but it's not that
important if this mode is enabled only with KASAN and other debugging
tools. Performance is definitely not as important as missing bugs. The
additional code complexity for ctors change should be minimal.
The rcu change would also be useful, but I would assume it will be larger.
Please add them to [1], that's KASAN laundry list.
+Alex, Marco, will it be useful for KFENCE [2] as well? Do ctors/rcu
affect KFENCE? Will we need any special handling for KFENCE?
I assume it will also be useful for KMSAN b/c we can re-mark objects
as uninitialized only after they have been reallocated.
[1] https://bugzilla.kernel.org/buglist.cgi?bug_status=__open__&component=Sanitizers&list_id=1063981&product=Memory%20Management
[2] https://github.com/google/kasan/commits/kfence
>
> Hi!
>
> Here's a project idea for the kernel-hardening folks:
>
> The slab allocator interface has two features that are problematic for
> security testing and/or hardening:
>
> - constructor slabs: These things come with an object constructor
> that doesn't run when an object is allocated, but instead when the
> slab allocator grabs a new page from the page allocator. This is
> problematic for use-after-free detection mechanisms such as HWASAN and
> Memory Tagging, which can only do their job properly if the address of
> an object is allowed to change every time the object is
> freed/reallocated. (You can't change the address of an object without
> reinitializing the entire object because e.g. an empty list_head
> points to itself.)
>
> - RCU slabs: These things basically permit use-after-frees by design,
> and stuff like ASAN/HWASAN/Memory Tagging essentially doesn't work on
> them.
>
>
> It would be nice to have a config flag or so that changes the SLUB
> allocator's behavior such that these slabs can be instrumented
> properly. Something like:
>
> - Let calculate_sizes() reserve space for an rcu_head on each object
> in TYPESAFE_BY_RCU slabs, make kmem_cache_free() redirect to
> call_rcu() for these slabs, and remove most of the other
> special-casing, so that KASAN can instrument these slabs.
> - For all constructor slabs, let slab_post_alloc_hook() call the
> ->ctor() function on each allocated object, so that Memory Tagging and
> HWASAN will work on them.
Hi Jann,
Both things sound good to me. I think we considered doing the ctor's
change with KASAN, but we did not get anywhere. The only argument
against it I remember now was "performance", but it's not that
important if this mode is enabled only with KASAN and other debugging
tools. Performance is definitely not as important as missing bugs. The
additional code complexity for ctors change should be minimal.
The rcu change would also be useful, but I would assume it will be larger.
Please add them to [1], that's KASAN laundry list.
+Alex, Marco, will it be useful for KFENCE [2] as well? Do ctors/rcu
affect KFENCE? Will we need any special handling for KFENCE?
I assume it will also be useful for KMSAN b/c we can re-mark objects
as uninitialized only after they have been reallocated.
[1] https://bugzilla.kernel.org/buglist.cgi?bug_status=__open__&component=Sanitizers&list_id=1063981&product=Memory%20Management
[2] https://github.com/google/kasan/commits/kfence
Marco Elver
Jun 23, 2020, 3:18:07 AM6/23/20
to Dmitry Vyukov, Jann Horn, Kernel Hardening, Christoph Lameter, Pekka Enberg, David Rientjes, Joonsoo Kim, Andrew Morton, Linux-MM, Andrey Konovalov, Will Deacon, kasan-dev, Kees Cook, Alexander Potapenko
Alexander Potapenko
Jun 23, 2020, 3:24:48 AM6/23/20
to Marco Elver, Dmitry Vyukov, Jann Horn, Kernel Hardening, Christoph Lameter, Pekka Enberg, David Rientjes, Joonsoo Kim, Andrew Morton, Linux-MM, Andrey Konovalov, Will Deacon, kasan-dev, Kees Cook
KFENCE also has to ignore both TYPESAFE_BY_RCU and ctors.
For ctors it should be pretty straightforward to fix (and won't
require any changes to SL[AU]B). Not sure if your proposal for RCU
will also work for KFENCE.
Another beneficiary of RCU/ctor normalization would be
init_on_alloc/init_on_free, which also ignore such slabs.
--
Alexander Potapenko
Software Engineer
Google Germany GmbH
Erika-Mann-Straße, 33
80636 München
Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
For ctors it should be pretty straightforward to fix (and won't
require any changes to SL[AU]B). Not sure if your proposal for RCU
will also work for KFENCE.
Another beneficiary of RCU/ctor normalization would be
init_on_alloc/init_on_free, which also ignore such slabs.
Alexander Potapenko
Software Engineer
Google Germany GmbH
Erika-Mann-Straße, 33
80636 München
Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
Dmitry Vyukov
Jun 23, 2020, 4:31:45 AM6/23/20
to Alexander Potapenko, Marco Elver, Jann Horn, Kernel Hardening, Christoph Lameter, Pekka Enberg, David Rientjes, Joonsoo Kim, Andrew Morton, Linux-MM, Andrey Konovalov, Will Deacon, kasan-dev, Kees Cook
On Tue, Jun 23, 2020 at 9:24 AM Alexander Potapenko <gli...@google.com> wrote:
>
> KFENCE also has to ignore both TYPESAFE_BY_RCU and ctors.
> For ctors it should be pretty straightforward to fix (and won't
> require any changes to SL[AU]B). Not sure if your proposal for RCU
> will also work for KFENCE.
Does it work for objects freed by call_rcu in normal slabs?
>
> KFENCE also has to ignore both TYPESAFE_BY_RCU and ctors.
> For ctors it should be pretty straightforward to fix (and won't
> require any changes to SL[AU]B). Not sure if your proposal for RCU
> will also work for KFENCE.
If yes, then I would assume it will work for TYPESAFE_BY_RCU after
this change, or is there a difference?
Alexander Potapenko
Jun 23, 2020, 4:38:14 AM6/23/20
to Dmitry Vyukov, Marco Elver, Jann Horn, Kernel Hardening, Christoph Lameter, Pekka Enberg, David Rientjes, Joonsoo Kim, Andrew Morton, Linux-MM, Andrey Konovalov, Will Deacon, kasan-dev, Kees Cook
On Tue, Jun 23, 2020 at 10:31 AM Dmitry Vyukov <dvy...@google.com> wrote:
>
> On Tue, Jun 23, 2020 at 9:24 AM Alexander Potapenko <gli...@google.com> wrote:
> >
> > KFENCE also has to ignore both TYPESAFE_BY_RCU and ctors.
> > For ctors it should be pretty straightforward to fix (and won't
> > require any changes to SL[AU]B). Not sure if your proposal for RCU
> > will also work for KFENCE.
>
> Does it work for objects freed by call_rcu in normal slabs?
> If yes, then I would assume it will work for TYPESAFE_BY_RCU after
> this change, or is there a difference?
If my understanding is correct, TYPESAFE_BY_RCU means that the object
>
> On Tue, Jun 23, 2020 at 9:24 AM Alexander Potapenko <gli...@google.com> wrote:
> >
> > KFENCE also has to ignore both TYPESAFE_BY_RCU and ctors.
> > For ctors it should be pretty straightforward to fix (and won't
> > require any changes to SL[AU]B). Not sure if your proposal for RCU
> > will also work for KFENCE.
>
> Does it work for objects freed by call_rcu in normal slabs?
> If yes, then I would assume it will work for TYPESAFE_BY_RCU after
> this change, or is there a difference?
may be used after it has been freed, that's why we cannot further
reuse or wipe it before ensuring they aren't used anymore.
Objects allocated from normal slabs cannot be used after they've been
freed, so I don't see how this change applies to them.
Dmitry Vyukov
Jun 23, 2020, 5:14:32 AM6/23/20
to Alexander Potapenko, Marco Elver, Jann Horn, Kernel Hardening, Christoph Lameter, Pekka Enberg, David Rientjes, Joonsoo Kim, Andrew Morton, Linux-MM, Andrey Konovalov, Will Deacon, kasan-dev, Kees Cook
On Tue, Jun 23, 2020 at 10:38 AM Alexander Potapenko <gli...@google.com> wrote:
> > > KFENCE also has to ignore both TYPESAFE_BY_RCU and ctors.
> > > For ctors it should be pretty straightforward to fix (and won't
> > > require any changes to SL[AU]B). Not sure if your proposal for RCU
> > > will also work for KFENCE.
> >
> > Does it work for objects freed by call_rcu in normal slabs?
> > If yes, then I would assume it will work for TYPESAFE_BY_RCU after
> > this change, or is there a difference?
>
> If my understanding is correct, TYPESAFE_BY_RCU means that the object
> may be used after it has been freed, that's why we cannot further
> reuse or wipe it before ensuring they aren't used anymore.
Yes, but only within an rcu grace period.
> > > KFENCE also has to ignore both TYPESAFE_BY_RCU and ctors.
> > > For ctors it should be pretty straightforward to fix (and won't
> > > require any changes to SL[AU]B). Not sure if your proposal for RCU
> > > will also work for KFENCE.
> >
> > Does it work for objects freed by call_rcu in normal slabs?
> > If yes, then I would assume it will work for TYPESAFE_BY_RCU after
> > this change, or is there a difference?
>
> If my understanding is correct, TYPESAFE_BY_RCU means that the object
> may be used after it has been freed, that's why we cannot further
> reuse or wipe it before ensuring they aren't used anymore.
And this proposal will take care of this: from the point of view of
slab, the object is freed after an additional rcu grace period. So
when it reaches slab free, it must not be used anymore.
Alexander Potapenko
Jun 23, 2020, 5:24:06 AM6/23/20
to Dmitry Vyukov, Marco Elver, Jann Horn, Kernel Hardening, Christoph Lameter, Pekka Enberg, David Rientjes, Joonsoo Kim, Andrew Morton, Linux-MM, Andrey Konovalov, Will Deacon, kasan-dev, Kees Cook
On Tue, Jun 23, 2020 at 11:14 AM Dmitry Vyukov <dvy...@google.com> wrote:
>
> On Tue, Jun 23, 2020 at 10:38 AM Alexander Potapenko <gli...@google.com> wrote:
> > > > KFENCE also has to ignore both TYPESAFE_BY_RCU and ctors.
> > > > For ctors it should be pretty straightforward to fix (and won't
> > > > require any changes to SL[AU]B). Not sure if your proposal for RCU
> > > > will also work for KFENCE.
> > >
> > > Does it work for objects freed by call_rcu in normal slabs?
> > > If yes, then I would assume it will work for TYPESAFE_BY_RCU after
> > > this change, or is there a difference?
> >
> > If my understanding is correct, TYPESAFE_BY_RCU means that the object
> > may be used after it has been freed, that's why we cannot further
> > reuse or wipe it before ensuring they aren't used anymore.
>
> Yes, but only within an rcu grace period.
> And this proposal will take care of this: from the point of view of
> slab, the object is freed after an additional rcu grace period. So
> when it reaches slab free, it must not be used anymore.
Thanks for clarifying!
>
> On Tue, Jun 23, 2020 at 10:38 AM Alexander Potapenko <gli...@google.com> wrote:
> > > > KFENCE also has to ignore both TYPESAFE_BY_RCU and ctors.
> > > > For ctors it should be pretty straightforward to fix (and won't
> > > > require any changes to SL[AU]B). Not sure if your proposal for RCU
> > > > will also work for KFENCE.
> > >
> > > Does it work for objects freed by call_rcu in normal slabs?
> > > If yes, then I would assume it will work for TYPESAFE_BY_RCU after
> > > this change, or is there a difference?
> >
> > If my understanding is correct, TYPESAFE_BY_RCU means that the object
> > may be used after it has been freed, that's why we cannot further
> > reuse or wipe it before ensuring they aren't used anymore.
>
> Yes, but only within an rcu grace period.
> And this proposal will take care of this: from the point of view of
> slab, the object is freed after an additional rcu grace period. So
> when it reaches slab free, it must not be used anymore.
Then both KFENCE and init_on_free should work fine with that change.
Jann Horn
Aug 25, 2023, 5:22:42 PM8/25/23
to Kernel Hardening, Christoph Lameter, Pekka Enberg, David Rientjes, Joonsoo Kim, Andrew Morton, Linux-MM, Andrey Konovalov, Dmitry Vyukov, Will Deacon, kasan-dev, Kees Cook, Alexander Potapenko, Marco Elver
On Tue, Jun 23, 2020 at 8:26 AM Jann Horn <ja...@google.com> wrote:
> Here's a project idea for the kernel-hardening folks:
>
> The slab allocator interface has two features that are problematic for
> security testing and/or hardening:
>
> - constructor slabs: These things come with an object constructor
> that doesn't run when an object is allocated, but instead when the
> slab allocator grabs a new page from the page allocator. This is
> problematic for use-after-free detection mechanisms such as HWASAN and
> Memory Tagging, which can only do their job properly if the address of
> an object is allowed to change every time the object is
> freed/reallocated. (You can't change the address of an object without
> reinitializing the entire object because e.g. an empty list_head
> points to itself.)
>
> - RCU slabs: These things basically permit use-after-frees by design,
> and stuff like ASAN/HWASAN/Memory Tagging essentially doesn't work on
> them.
>
>
> It would be nice to have a config flag or so that changes the SLUB
> allocator's behavior such that these slabs can be instrumented
> properly. Something like:
>
> - Let calculate_sizes() reserve space for an rcu_head on each object
> in TYPESAFE_BY_RCU slabs, make kmem_cache_free() redirect to
> call_rcu() for these slabs, and remove most of the other
> special-casing, so that KASAN can instrument these slabs.
I've implemented this first part now and sent it out for review:
> Here's a project idea for the kernel-hardening folks:
>
> The slab allocator interface has two features that are problematic for
> security testing and/or hardening:
>
> - constructor slabs: These things come with an object constructor
> that doesn't run when an object is allocated, but instead when the
> slab allocator grabs a new page from the page allocator. This is
> problematic for use-after-free detection mechanisms such as HWASAN and
> Memory Tagging, which can only do their job properly if the address of
> an object is allowed to change every time the object is
> freed/reallocated. (You can't change the address of an object without
> reinitializing the entire object because e.g. an empty list_head
> points to itself.)
>
> - RCU slabs: These things basically permit use-after-frees by design,
> and stuff like ASAN/HWASAN/Memory Tagging essentially doesn't work on
> them.
>
>
> It would be nice to have a config flag or so that changes the SLUB
> allocator's behavior such that these slabs can be instrumented
> properly. Something like:
>
> - Let calculate_sizes() reserve space for an rcu_head on each object
> in TYPESAFE_BY_RCU slabs, make kmem_cache_free() redirect to
> call_rcu() for these slabs, and remove most of the other
> special-casing, so that KASAN can instrument these slabs.
https://lore.kernel.org/lkml/20230825211426....@google.com/T/
Reply all
Reply to author
Forward
0 new messages