[PATCH] ARM: kasan: Only map modules if CONFIG_KASAN_VMALLOC=n

[Alexander A Sverdlin]

From: Alexander Sverdlin <alexander...@nokia.com>

In case CONFIG_KASAN_VMALLOC=y kasan_populate_vmalloc() allocates the
shadow pages dynamically. But even worse is that kasan_release_vmalloc()
releases them, which is not compatible with create_mapping() of
MODULES_VADDR..MODULES_END range:

BUG: Bad page state in process kworker/9:1 pfn:2068b
page:e5e06160 refcount:0 mapcount:0 mapping:00000000 index:0x0
flags: 0x1000(reserved)
raw: 00001000 e5e06164 e5e06164 00000000 00000000 00000000 ffffffff 00000000
page dumped because: PAGE_FLAGS_CHECK_AT_FREE flag(s) set
bad because of flags: 0x1000(reserved)
Modules linked in: ip_tables
CPU: 9 PID: 154 Comm: kworker/9:1 Not tainted 5.4.188-... #1
Hardware name: LSI Axxia AXM55XX
Workqueue: events do_free_init
unwind_backtrace
show_stack
dump_stack
bad_page
free_pcp_prepare
free_unref_page
kasan_depopulate_vmalloc_pte
__apply_to_page_range
apply_to_existing_page_range
kasan_release_vmalloc
__purge_vmap_area_lazy
_vm_unmap_aliases.part.0
__vunmap
do_free_init
process_one_work
worker_thread
kthread

Signed-off-by: Alexander Sverdlin <alexander...@nokia.com>
---
arch/arm/mm/kasan_init.c | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/arch/arm/mm/kasan_init.c b/arch/arm/mm/kasan_init.c
index 29caee9c79ce3..64790661bdc40 100644
--- a/arch/arm/mm/kasan_init.c
+++ b/arch/arm/mm/kasan_init.c
@@ -268,12 +268,14 @@ void __init kasan_init(void)

/*
* 1. The module global variables are in MODULES_VADDR ~ MODULES_END,
- * so we need to map this area.
+ * so we need to map this area if CONFIG_KASAN_VMALLOC=n.
* 2. PKMAP_BASE ~ PKMAP_BASE+PMD_SIZE's shadow and MODULES_VADDR
* ~ MODULES_END's shadow is in the same PMD_SIZE, so we can't
* use kasan_populate_zero_shadow.
*/
- create_mapping((void *)MODULES_VADDR, (void *)(PKMAP_BASE + PMD_SIZE));
+ if (!IS_ENABLED(CONFIG_KASAN_VMALLOC) && IS_ENABLED(CONFIG_MODULES))
+ create_mapping((void *)MODULES_VADDR, (void *)(MODULES_END));
+ create_mapping((void *)PKMAP_BASE, (void *)(PKMAP_BASE + PMD_SIZE));

/*
* KAsan may reuse the contents of kasan_early_shadow_pte directly, so
--
2.35.1

[Linus Walleij]

On Sat, Aug 27, 2022 at 11:32 PM Alexander A Sverdlin
<alexander...@nokia.com> wrote:

> - create_mapping((void *)MODULES_VADDR, (void *)(PKMAP_BASE + PMD_SIZE));
> + if (!IS_ENABLED(CONFIG_KASAN_VMALLOC) && IS_ENABLED(CONFIG_MODULES))
> + create_mapping((void *)MODULES_VADDR, (void *)(MODULES_END));

So the way I understand it is that modules are first and foremost loaded into
the area MODULES_VADDR .. MODULES_END, and then after that is out,
they get loaded into VMALLOC. See arch/arm/kernel/module.c, module_alloc().

If you do this, how are the addresses between MODULES_VADDR..MODULES_END
shadowed when using CONFIG_KASAN_VMALLOC?

> + create_mapping((void *)PKMAP_BASE, (void *)(PKMAP_BASE + PMD_SIZE));

(Splitting this in two steps if probably good in any case.)

Pls keep me on CC for Kasan ARM patches, thanks! (Maybe I should add some
MAINTAINERS blurb.)

Yours,
Linus Walleij

[Alexander Sverdlin]

Hello Linus,

On 31/08/2022 11:30, Linus Walleij wrote:
>> - create_mapping((void *)MODULES_VADDR, (void *)(PKMAP_BASE + PMD_SIZE));
>> + if (!IS_ENABLED(CONFIG_KASAN_VMALLOC) && IS_ENABLED(CONFIG_MODULES))
>> + create_mapping((void *)MODULES_VADDR, (void *)(MODULES_END));
> So the way I understand it is that modules are first and foremost loaded into
> the area MODULES_VADDR .. MODULES_END, and then after that is out,
> they get loaded into VMALLOC. See arch/arm/kernel/module.c, module_alloc().

yes, but both areas are managed by __vmalloc_node_range().

> If you do this, how are the addresses between MODULES_VADDR..MODULES_END
> shadowed when using CONFIG_KASAN_VMALLOC?

That's the thing, __vmalloc_node_range() doesn't differentiate between address
ranges and tries first to recreate [already existing] shadow mapping, and then
vfree() unconditionally frees the mapping and the page.

vmalloc() KASAN handling is generic, module_alloc() implemented via vmalloc()
is however ARM-specific. Even though we could teach vmalloc() about MODULES_VADDR
and MODULES_END (and don't call kasan_ instrumentation on these), but, this is
ARM-specifics that it's used for this range.

>> + create_mapping((void *)PKMAP_BASE, (void *)(PKMAP_BASE + PMD_SIZE));
> (Splitting this in two steps if probably good in any case.)
>
> Pls keep me on CC for Kasan ARM patches, thanks! (Maybe I should add some
> MAINTAINERS blurb.)

--
Best regards,
Alexander Sverdlin.

[Linus Walleij]

On Thu, Sep 1, 2022 at 10:42 AM Alexander Sverdlin
<alexander...@nokia.com> wrote:

> >> - create_mapping((void *)MODULES_VADDR, (void *)(PKMAP_BASE + PMD_SIZE));
> >> + if (!IS_ENABLED(CONFIG_KASAN_VMALLOC) && IS_ENABLED(CONFIG_MODULES))
> >> + create_mapping((void *)MODULES_VADDR, (void *)(MODULES_END));
> > So the way I understand it is that modules are first and foremost loaded into
> > the area MODULES_VADDR .. MODULES_END, and then after that is out,
> > they get loaded into VMALLOC. See arch/arm/kernel/module.c, module_alloc().
>
> yes, but both areas are managed by __vmalloc_node_range().

Owww!

> > If you do this, how are the addresses between MODULES_VADDR..MODULES_END
> > shadowed when using CONFIG_KASAN_VMALLOC?
>
> That's the thing, __vmalloc_node_range() doesn't differentiate between address
> ranges and tries first to recreate [already existing] shadow mapping, and then
> vfree() unconditionally frees the mapping and the page.
>
> vmalloc() KASAN handling is generic, module_alloc() implemented via vmalloc()
> is however ARM-specific. Even though we could teach vmalloc() about MODULES_VADDR
> and MODULES_END (and don't call kasan_ instrumentation on these), but, this is
> ARM-specifics that it's used for this range.

OK I get it. Maybe this warrants a comment in the code explaining the
above behaviour (also in commitlog) so nobody gets confused.

With that:
Reviewed-by: Linus Walleij <linus....@linaro.org>

Yours.
Linus Walleij

[Alexander A Sverdlin]

Reviewed-by: Linus Walleij <linus....@linaro.org>

Signed-off-by: Alexander Sverdlin <alexander...@nokia.com>
---

Changelog:
v2:
* more verbose comment

arch/arm/mm/kasan_init.c | 9 +++++++--
1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/arch/arm/mm/kasan_init.c b/arch/arm/mm/kasan_init.c
index 29caee9..46d9f4a 100644
--- a/arch/arm/mm/kasan_init.c
+++ b/arch/arm/mm/kasan_init.c
@@ -268,12 +268,17 @@ void __init kasan_init(void)

/*
* 1. The module global variables are in MODULES_VADDR ~ MODULES_END,
- * so we need to map this area.

+ * so we need to map this area if CONFIG_KASAN_VMALLOC=n. With
+ * VMALLOC support KASAN will manage this region dynamically,
+ * refer to kasan_populate_vmalloc() and ARM's implementation of
+ * module_alloc().

* 2. PKMAP_BASE ~ PKMAP_BASE+PMD_SIZE's shadow and MODULES_VADDR
* ~ MODULES_END's shadow is in the same PMD_SIZE, so we can't
* use kasan_populate_zero_shadow.
*/

- create_mapping((void *)MODULES_VADDR, (void *)(PKMAP_BASE + PMD_SIZE));
+ if (!IS_ENABLED(CONFIG_KASAN_VMALLOC) && IS_ENABLED(CONFIG_MODULES))
+ create_mapping((void *)MODULES_VADDR, (void *)(MODULES_END));

+ create_mapping((void *)PKMAP_BASE, (void *)(PKMAP_BASE + PMD_SIZE));

/*
* KAsan may reuse the contents of kasan_early_shadow_pte directly, so
--

2.10.2

[Linus Walleij]

Thanks Alexander, will you submit this to Russell's patch tracker please?

Yours,
Linus Walleij

[Alexander Sverdlin]

Hello Linus,

On 05/09/2022 15:38, Linus Walleij wrote:
>> In case CONFIG_KASAN_VMALLOC=y kasan_populate_vmalloc() allocates the
>> shadow pages dynamically. But even worse is that kasan_release_vmalloc()
>> releases them, which is not compatible with create_mapping() of
>> MODULES_VADDR..MODULES_END range:
>>
>> BUG: Bad page state in process kworker/9:1 pfn:2068b

[...]

>>
>> Reviewed-by: Linus Walleij <linus....@linaro.org>
>> Signed-off-by: Alexander Sverdlin <alexander...@nokia.com>
> Thanks Alexander, will you submit this to Russell's patch tracker please?

done!

https://www.arm.linux.org.uk/developer/patches/viewpatch.php?id=9242/1

Thank you for the quick review!

[Alexander Sverdlin]

Hello Linus,

On 31/08/2022 11:30, Linus Walleij wrote:

> Pls keep me on CC for Kasan ARM patches, thanks! (Maybe I should add some
> MAINTAINERS blurb.)

there is one patch which barely triggered any interest, but the problem has been spotted during
KASAN usage on ARM ("ARM: module: Teach unwinder about PLTs"):
https://lore.kernel.org/linux-arm-kernel/2bb016da-363d-5aac...@nokia.com/t/

Would you like to review it so that I can add it to Russel's patch system?