[PATCH v2 0/5] arm64: ARMv8.7-A: MTE: Add asymm in-kernel support

1 view
Skip to first unread message

Vincenzo Frascino

unread,
Oct 4, 2021, 4:23:12 PM10/4/21
to linux-ar...@lists.infradead.org, linux-...@vger.kernel.org, kasa...@googlegroups.com, vincenzo...@arm.com, Andrew Morton, Catalin Marinas, Will Deacon, Dmitry Vyukov, Andrey Ryabinin, Alexander Potapenko, Marco Elver, Evgenii Stepanov, Branislav Rankov, Andrey Konovalov, Lorenzo Pieralisi
This series implements the in-kernel asymmetric mode support for
ARMv8.7-A Memory Tagging Extension (MTE), which is a debugging feature
that allows to detect with the help of the architecture the C and C++
programmatic memory errors like buffer overflow, use-after-free,
use-after-return, etc.

MTE is built on top of the AArch64 v8.0 virtual address tagging TBI
(Top Byte Ignore) feature and allows a task to set a 4 bit tag on any
subset of its address space that is multiple of a 16 bytes granule. MTE
is based on a lock-key mechanism where the lock is the tag associated to
the physical memory and the key is the tag associated to the virtual
address.

When MTE is enabled and tags are set for ranges of address space of a task,
the PE will compare the tag related to the physical memory with the tag
related to the virtual address (tag check operation). Access to the memory
is granted only if the two tags match. In case of mismatch the PE will raise
an exception.

When asymmetric mode is present, the CPU triggers a fault on a tag mismatch
during a load operation and asynchronously updates a register when a tag
mismatch is detected during a store operation.

Note: The userspace support will be sent with a future patch series.

The series is based on linux-v5.15-rc4.

To simplify the testing a tree with the new patches on top has been made
available at [1].

[1] https://git.gitlab.arm.com/linux-arm/linux-vf.git mte/v2.asymm

Cc: Andrew Morton <ak...@linux-foundation.org>
Cc: Catalin Marinas <catalin...@arm.com>
Cc: Will Deacon <wi...@kernel.org>
Cc: Dmitry Vyukov <dvy...@google.com>
Cc: Andrey Ryabinin <arya...@virtuozzo.com>
Cc: Alexander Potapenko <gli...@google.com>
Cc: Marco Elver <el...@google.com>
Cc: Evgenii Stepanov <eug...@google.com>
Cc: Branislav Rankov <Branisla...@arm.com>
Cc: Andrey Konovalov <andre...@gmail.com>
Cc: Lorenzo Pieralisi <lorenzo....@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo...@arm.com>

Vincenzo Frascino (5):
kasan: Remove duplicate of kasan_flag_async
arm64: mte: Bitfield definitions for Asymm MTE
arm64: mte: CPU feature detection for Asymm MTE
arm64: mte: Add asymmetric mode support
kasan: Extend KASAN mode kernel parameter

Documentation/dev-tools/kasan.rst | 7 +++++--
arch/arm64/include/asm/memory.h | 1 +
arch/arm64/include/asm/mte-kasan.h | 5 +++++
arch/arm64/include/asm/sysreg.h | 3 +++
arch/arm64/kernel/cpufeature.c | 10 +++++++++
arch/arm64/kernel/mte.c | 33 +++++++++++++++++++++++++++++-
arch/arm64/tools/cpucaps | 1 +
lib/test_kasan.c | 2 +-
mm/kasan/hw_tags.c | 27 +++++++++++++++++++-----
mm/kasan/kasan.h | 24 +++++++++++++++++-----
mm/kasan/report.c | 2 +-
11 files changed, 100 insertions(+), 15 deletions(-)

--
2.33.0

Vincenzo Frascino

unread,
Oct 4, 2021, 4:23:13 PM10/4/21
to linux-ar...@lists.infradead.org, linux-...@vger.kernel.org, kasa...@googlegroups.com, vincenzo...@arm.com, Andrew Morton, Catalin Marinas, Will Deacon, Dmitry Vyukov, Andrey Ryabinin, Alexander Potapenko, Marco Elver, Evgenii Stepanov, Branislav Rankov, Andrey Konovalov, Lorenzo Pieralisi
After merging async mode for KASAN_HW_TAGS a duplicate of the
kasan_flag_async flag was left erroneously inside the code.

Remove the duplicate.

Note: This change does not bring functional changes to the code
base.

Cc: Dmitry Vyukov <dvy...@google.com>
Cc: Andrey Ryabinin <arya...@virtuozzo.com>
Cc: Alexander Potapenko <gli...@google.com>
Cc: Marco Elver <el...@google.com>
Cc: Evgenii Stepanov <eug...@google.com>
Cc: Andrey Konovalov <andre...@gmail.com>
Signed-off-by: Vincenzo Frascino <vincenzo...@arm.com>
Acked-by: Catalin Marinas <catalin...@arm.com>
Reviewed-by: Andrey Konovalov <andre...@gmail.com>
---
mm/kasan/kasan.h | 2 --
1 file changed, 2 deletions(-)

diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h
index 8bf568a80eb8..3639e7c8bb98 100644
--- a/mm/kasan/kasan.h
+++ b/mm/kasan/kasan.h
@@ -38,8 +38,6 @@ static inline bool kasan_async_mode_enabled(void)

#endif

-extern bool kasan_flag_async __ro_after_init;
-
#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
#define KASAN_GRANULE_SIZE (1UL << KASAN_SHADOW_SCALE_SHIFT)
#else
--
2.33.0

Vincenzo Frascino

unread,
Oct 4, 2021, 4:23:15 PM10/4/21
to linux-ar...@lists.infradead.org, linux-...@vger.kernel.org, kasa...@googlegroups.com, vincenzo...@arm.com, Andrew Morton, Catalin Marinas, Will Deacon, Dmitry Vyukov, Andrey Ryabinin, Alexander Potapenko, Marco Elver, Evgenii Stepanov, Branislav Rankov, Andrey Konovalov, Lorenzo Pieralisi
Add Asymmetric Memory Tagging Extension bitfield definitions.

Cc: Will Deacon <wi...@kernel.org>
Cc: Catalin Marinas <catalin...@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo...@arm.com>
Acked-by: Catalin Marinas <catalin...@arm.com>
---
arch/arm64/include/asm/sysreg.h | 3 +++
1 file changed, 3 insertions(+)

diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
index b268082d67ed..f51d5912b41c 100644
--- a/arch/arm64/include/asm/sysreg.h
+++ b/arch/arm64/include/asm/sysreg.h
@@ -621,6 +621,7 @@
#define SCTLR_ELx_TCF_NONE (UL(0x0) << SCTLR_ELx_TCF_SHIFT)
#define SCTLR_ELx_TCF_SYNC (UL(0x1) << SCTLR_ELx_TCF_SHIFT)
#define SCTLR_ELx_TCF_ASYNC (UL(0x2) << SCTLR_ELx_TCF_SHIFT)
+#define SCTLR_ELx_TCF_ASYMM (UL(0x3) << SCTLR_ELx_TCF_SHIFT)
#define SCTLR_ELx_TCF_MASK (UL(0x3) << SCTLR_ELx_TCF_SHIFT)

#define SCTLR_ELx_ENIA_SHIFT 31
@@ -666,6 +667,7 @@
#define SCTLR_EL1_TCF0_NONE (UL(0x0) << SCTLR_EL1_TCF0_SHIFT)
#define SCTLR_EL1_TCF0_SYNC (UL(0x1) << SCTLR_EL1_TCF0_SHIFT)
#define SCTLR_EL1_TCF0_ASYNC (UL(0x2) << SCTLR_EL1_TCF0_SHIFT)
+#define SCTLR_EL1_TCF0_ASYMM (UL(0x3) << SCTLR_EL1_TCF0_SHIFT)
#define SCTLR_EL1_TCF0_MASK (UL(0x3) << SCTLR_EL1_TCF0_SHIFT)

#define SCTLR_EL1_BT1 (BIT(36))
@@ -807,6 +809,7 @@
#define ID_AA64PFR1_MTE_NI 0x0
#define ID_AA64PFR1_MTE_EL0 0x1
#define ID_AA64PFR1_MTE 0x2
+#define ID_AA64PFR1_MTE_ASYMM 0x3

/* id_aa64zfr0 */
#define ID_AA64ZFR0_F64MM_SHIFT 56
--
2.33.0

Vincenzo Frascino

unread,
Oct 4, 2021, 4:23:17 PM10/4/21
to linux-ar...@lists.infradead.org, linux-...@vger.kernel.org, kasa...@googlegroups.com, vincenzo...@arm.com, Andrew Morton, Catalin Marinas, Will Deacon, Dmitry Vyukov, Andrey Ryabinin, Alexander Potapenko, Marco Elver, Evgenii Stepanov, Branislav Rankov, Andrey Konovalov, Lorenzo Pieralisi, Suzuki K Poulose, Suzuki K Poulose
Add the cpufeature entries to detect the presence of Asymmetric MTE.

Note: The tag checking mode is initialized via cpu_enable_mte() ->
kasan_init_hw_tags() hence to enable it we require asymmetric mode
to be at least on the boot CPU. If the boot CPU does not have it, it is
fine for late CPUs to have it as long as the feature is not enabled
(ARM64_CPUCAP_BOOT_CPU_FEATURE).

Cc: Will Deacon <wi...@kernel.org>
Cc: Catalin Marinas <catalin...@arm.com>
Cc: Suzuki K Poulose <Suzuki....@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo...@arm.com>
Reviewed-by: Catalin Marinas <catalin...@arm.com>
Reviewed-by: Suzuki K Poulose <suzuki....@arm.com>
---
arch/arm64/kernel/cpufeature.c | 10 ++++++++++
arch/arm64/tools/cpucaps | 1 +
2 files changed, 11 insertions(+)

diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 6ec7036ef7e1..9e3e8ad75f20 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -2321,6 +2321,16 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
.sign = FTR_UNSIGNED,
.cpu_enable = cpu_enable_mte,
},
+ {
+ .desc = "Asymmetric MTE Tag Check Fault",
+ .capability = ARM64_MTE_ASYMM,
+ .type = ARM64_CPUCAP_BOOT_CPU_FEATURE,
+ .matches = has_cpuid_feature,
+ .sys_reg = SYS_ID_AA64PFR1_EL1,
+ .field_pos = ID_AA64PFR1_MTE_SHIFT,
+ .min_field_value = ID_AA64PFR1_MTE_ASYMM,
+ .sign = FTR_UNSIGNED,
+ },
#endif /* CONFIG_ARM64_MTE */
{
.desc = "RCpc load-acquire (LDAPR)",
diff --git a/arch/arm64/tools/cpucaps b/arch/arm64/tools/cpucaps
index 49305c2e6dfd..74a569bf52d6 100644
--- a/arch/arm64/tools/cpucaps
+++ b/arch/arm64/tools/cpucaps
@@ -39,6 +39,7 @@ HW_DBM
KVM_PROTECTED_MODE
MISMATCHED_CACHE_TYPE
MTE
+MTE_ASYMM
SPECTRE_V2
SPECTRE_V3A
SPECTRE_V4
--
2.33.0

Vincenzo Frascino

unread,
Oct 4, 2021, 4:23:19 PM10/4/21
to linux-ar...@lists.infradead.org, linux-...@vger.kernel.org, kasa...@googlegroups.com, vincenzo...@arm.com, Andrew Morton, Catalin Marinas, Will Deacon, Dmitry Vyukov, Andrey Ryabinin, Alexander Potapenko, Marco Elver, Evgenii Stepanov, Branislav Rankov, Andrey Konovalov, Lorenzo Pieralisi
MTE provides an asymmetric mode for detecting tag exceptions. In
particular, when such a mode is present, the CPU triggers a fault
on a tag mismatch during a load operation and asynchronously updates
a register when a tag mismatch is detected during a store operation.

Add support for MTE asymmetric mode.

Note: If the CPU does not support MTE asymmetric mode the kernel falls
back on synchronous mode which is the default for kasan=on.

Cc: Will Deacon <wi...@kernel.org>
Cc: Catalin Marinas <catalin...@arm.com>
Cc: Andrey Konovalov <andre...@gmail.com>
Signed-off-by: Vincenzo Frascino <vincenzo...@arm.com>
Reviewed-by: Catalin Marinas <catalin...@arm.com>
---
arch/arm64/include/asm/memory.h | 1 +
arch/arm64/include/asm/mte-kasan.h | 5 +++++
arch/arm64/kernel/mte.c | 33 +++++++++++++++++++++++++++++-
3 files changed, 38 insertions(+), 1 deletion(-)

diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h
index f1745a843414..1b9a1e242612 100644
--- a/arch/arm64/include/asm/memory.h
+++ b/arch/arm64/include/asm/memory.h
@@ -243,6 +243,7 @@ static inline const void *__tag_set(const void *addr, u8 tag)
#ifdef CONFIG_KASAN_HW_TAGS
#define arch_enable_tagging_sync() mte_enable_kernel_sync()
#define arch_enable_tagging_async() mte_enable_kernel_async()
+#define arch_enable_tagging_asymm() mte_enable_kernel_asymm()
#define arch_force_async_tag_fault() mte_check_tfsr_exit()
#define arch_get_random_tag() mte_get_random_tag()
#define arch_get_mem_tag(addr) mte_get_mem_tag(addr)
diff --git a/arch/arm64/include/asm/mte-kasan.h b/arch/arm64/include/asm/mte-kasan.h
index 22420e1f8c03..478b9bcf69ad 100644
--- a/arch/arm64/include/asm/mte-kasan.h
+++ b/arch/arm64/include/asm/mte-kasan.h
@@ -130,6 +130,7 @@ static inline void mte_set_mem_tag_range(void *addr, size_t size, u8 tag,

void mte_enable_kernel_sync(void);
void mte_enable_kernel_async(void);
+void mte_enable_kernel_asymm(void);

#else /* CONFIG_ARM64_MTE */

@@ -161,6 +162,10 @@ static inline void mte_enable_kernel_async(void)
{
}

+static inline void mte_enable_kernel_asymm(void)
+{
+}
+
#endif /* CONFIG_ARM64_MTE */

#endif /* __ASSEMBLY__ */
diff --git a/arch/arm64/kernel/mte.c b/arch/arm64/kernel/mte.c
index e5e801bc5312..b6ad6b861c25 100644
--- a/arch/arm64/kernel/mte.c
+++ b/arch/arm64/kernel/mte.c
@@ -26,7 +26,12 @@
static DEFINE_PER_CPU_READ_MOSTLY(u64, mte_tcf_preferred);

#ifdef CONFIG_KASAN_HW_TAGS
-/* Whether the MTE asynchronous mode is enabled. */
+/*
+ * The MTE asynchronous and asymmetric mode have the same
+ * behavior for the store operations.
+ *
+ * Whether the MTE asynchronous or asymmetric mode is enabled.
+ */
DEFINE_STATIC_KEY_FALSE(mte_async_mode);
EXPORT_SYMBOL_GPL(mte_async_mode);
#endif
@@ -137,6 +142,32 @@ void mte_enable_kernel_async(void)
if (!system_uses_mte_async_mode())
static_branch_enable(&mte_async_mode);
}
+
+void mte_enable_kernel_asymm(void)
+{
+ if (cpus_have_cap(ARM64_MTE_ASYMM)) {
+ __mte_enable_kernel("asymmetric", SCTLR_ELx_TCF_ASYMM);
+
+ /*
+ * MTE asymm mode behaves as async mode for store
+ * operations. The mode is set system wide by the
+ * first PE that executes this function.
+ *
+ * Note: If in future KASAN acquires a runtime switching
+ * mode in between sync and async, this strategy needs
+ * to be reviewed.
+ */
+ if (!system_uses_mte_async_mode())
+ static_branch_enable(&mte_async_mode);
+ } else {
+ /*
+ * If the CPU does not support MTE asymmetric mode the
+ * kernel falls back on synchronous mode which is the
+ * default for kasan=on.
+ */
+ mte_enable_kernel_sync();
+ }
+}
#endif

#ifdef CONFIG_KASAN_HW_TAGS
--
2.33.0

Vincenzo Frascino

unread,
Oct 4, 2021, 4:23:22 PM10/4/21
to linux-ar...@lists.infradead.org, linux-...@vger.kernel.org, kasa...@googlegroups.com, vincenzo...@arm.com, Andrew Morton, Catalin Marinas, Will Deacon, Dmitry Vyukov, Andrey Ryabinin, Alexander Potapenko, Marco Elver, Evgenii Stepanov, Branislav Rankov, Andrey Konovalov, Lorenzo Pieralisi
Architectures supported by KASAN_HW_TAGS can provide an asymmetric mode
of execution. On an MTE enabled arm64 hw for example this can be
identified with the asymmetric tagging mode of execution. In particular,
when such a mode is present, the CPU triggers a fault on a tag mismatch
during a load operation and asynchronously updates a register when a tag
mismatch is detected during a store operation.

Extend the KASAN HW execution mode kernel command line parameter to
support asymmetric mode.

Cc: Dmitry Vyukov <dvy...@google.com>
Cc: Andrey Ryabinin <arya...@virtuozzo.com>
Cc: Alexander Potapenko <gli...@google.com>
Cc: Andrey Konovalov <andre...@gmail.com>
Signed-off-by: Vincenzo Frascino <vincenzo...@arm.com>
Reviewed-by: Catalin Marinas <catalin...@arm.com>
---
Documentation/dev-tools/kasan.rst | 7 +++++--
lib/test_kasan.c | 2 +-
mm/kasan/hw_tags.c | 27 ++++++++++++++++++++++-----
mm/kasan/kasan.h | 22 +++++++++++++++++++---
mm/kasan/report.c | 2 +-
5 files changed, 48 insertions(+), 12 deletions(-)

diff --git a/Documentation/dev-tools/kasan.rst b/Documentation/dev-tools/kasan.rst
index 21dc03bc10a4..8089c559d339 100644
--- a/Documentation/dev-tools/kasan.rst
+++ b/Documentation/dev-tools/kasan.rst
@@ -194,14 +194,17 @@ additional boot parameters that allow disabling KASAN or controlling features:

- ``kasan=off`` or ``=on`` controls whether KASAN is enabled (default: ``on``).

-- ``kasan.mode=sync`` or ``=async`` controls whether KASAN is configured in
- synchronous or asynchronous mode of execution (default: ``sync``).
+- ``kasan.mode=sync``, ``=async`` or ``=asymm`` controls whether KASAN
+ is configured in synchronous, asynchronous or asymmetric mode of
+ execution (default: ``sync``).
Synchronous mode: a bad access is detected immediately when a tag
check fault occurs.
Asynchronous mode: a bad access detection is delayed. When a tag check
fault occurs, the information is stored in hardware (in the TFSR_EL1
register for arm64). The kernel periodically checks the hardware and
only reports tag faults during these checks.
+ Asymmetric mode: a bad access is detected synchronously on reads and
+ asynchronously on writes.

- ``kasan.stacktrace=off`` or ``=on`` disables or enables alloc and free stack
traces collection (default: ``on``).
diff --git a/lib/test_kasan.c b/lib/test_kasan.c
index 8835e0784578..ebed755ebf34 100644
--- a/lib/test_kasan.c
+++ b/lib/test_kasan.c
@@ -88,7 +88,7 @@ static void kasan_test_exit(struct kunit *test)
*/
#define KUNIT_EXPECT_KASAN_FAIL(test, expression) do { \
if (IS_ENABLED(CONFIG_KASAN_HW_TAGS) && \
- !kasan_async_mode_enabled()) \
+ kasan_sync_fault_possible()) \
migrate_disable(); \
KUNIT_EXPECT_FALSE(test, READ_ONCE(fail_data.report_found)); \
barrier(); \
diff --git a/mm/kasan/hw_tags.c b/mm/kasan/hw_tags.c
index 05d1e9460e2e..87eb7aa13918 100644
--- a/mm/kasan/hw_tags.c
+++ b/mm/kasan/hw_tags.c
@@ -29,6 +29,7 @@ enum kasan_arg_mode {
KASAN_ARG_MODE_DEFAULT,
KASAN_ARG_MODE_SYNC,
KASAN_ARG_MODE_ASYNC,
+ KASAN_ARG_MODE_ASYMM,
};

enum kasan_arg_stacktrace {
@@ -49,6 +50,10 @@ EXPORT_SYMBOL(kasan_flag_enabled);
bool kasan_flag_async __ro_after_init;
EXPORT_SYMBOL_GPL(kasan_flag_async);

+/* Whether the asymmetric mode is enabled. */
+bool kasan_flag_asymm __ro_after_init;
+EXPORT_SYMBOL_GPL(kasan_flag_asymm);
+
/* Whether to collect alloc/free stack traces. */
DEFINE_STATIC_KEY_FALSE(kasan_flag_stacktrace);

@@ -69,7 +74,7 @@ static int __init early_kasan_flag(char *arg)
}
early_param("kasan", early_kasan_flag);

-/* kasan.mode=sync/async */
+/* kasan.mode=sync/async/asymm */
static int __init early_kasan_mode(char *arg)
{
if (!arg)
@@ -79,6 +84,8 @@ static int __init early_kasan_mode(char *arg)
kasan_arg_mode = KASAN_ARG_MODE_SYNC;
else if (!strcmp(arg, "async"))
kasan_arg_mode = KASAN_ARG_MODE_ASYNC;
+ else if (!strcmp(arg, "asymm"))
+ kasan_arg_mode = KASAN_ARG_MODE_ASYMM;
else
return -EINVAL;

@@ -116,11 +123,13 @@ void kasan_init_hw_tags_cpu(void)
return;

/*
- * Enable async mode only when explicitly requested through
- * the command line.
+ * Enable async or asymm modes only when explicitly requested
+ * through the command line.
*/
if (kasan_arg_mode == KASAN_ARG_MODE_ASYNC)
hw_enable_tagging_async();
+ else if (kasan_arg_mode == KASAN_ARG_MODE_ASYMM)
+ hw_enable_tagging_asymm();
else
hw_enable_tagging_sync();
}
@@ -143,16 +152,24 @@ void __init kasan_init_hw_tags(void)
case KASAN_ARG_MODE_DEFAULT:
/*
* Default to sync mode.
- * Do nothing, kasan_flag_async keeps its default value.
+ * Do nothing, kasan_flag_async and kasan_flag_asymm keep
+ * their default values.
*/
break;
case KASAN_ARG_MODE_SYNC:
- /* Do nothing, kasan_flag_async keeps its default value. */
+ /*
+ * Do nothing, kasan_flag_async and kasan_flag_asymm keep
+ * their default values.
+ */
break;
case KASAN_ARG_MODE_ASYNC:
/* Async mode enabled. */
kasan_flag_async = true;
break;
+ case KASAN_ARG_MODE_ASYMM:
+ /* Asymm mode enabled. */
+ kasan_flag_asymm = true;
+ break;
}

switch (kasan_arg_stacktrace) {
diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h
index 3639e7c8bb98..1d331ce67dec 100644
--- a/mm/kasan/kasan.h
+++ b/mm/kasan/kasan.h
@@ -14,15 +14,21 @@

DECLARE_STATIC_KEY_FALSE(kasan_flag_stacktrace);
extern bool kasan_flag_async __ro_after_init;
+extern bool kasan_flag_asymm __ro_after_init;

static inline bool kasan_stack_collection_enabled(void)
{
return static_branch_unlikely(&kasan_flag_stacktrace);
}

-static inline bool kasan_async_mode_enabled(void)
+static inline bool kasan_async_fault_possible(void)
{
- return kasan_flag_async;
+ return kasan_flag_async | kasan_flag_asymm;
+}
+
+static inline bool kasan_sync_fault_possible(void)
+{
+ return !kasan_flag_async | kasan_flag_asymm;
}
#else

@@ -31,11 +37,16 @@ static inline bool kasan_stack_collection_enabled(void)
return true;
}

-static inline bool kasan_async_mode_enabled(void)
+static inline bool kasan_async_fault_possible(void)
{
return false;
}

+static inline bool kasan_sync_fault_possible(void)
+{
+ return true;
+}
+
#endif

#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
@@ -287,6 +298,9 @@ static inline const void *arch_kasan_set_tag(const void *addr, u8 tag)
#ifndef arch_enable_tagging_async
#define arch_enable_tagging_async()
#endif
+#ifndef arch_enable_tagging_asymm
+#define arch_enable_tagging_asymm()
+#endif
#ifndef arch_force_async_tag_fault
#define arch_force_async_tag_fault()
#endif
@@ -302,6 +316,7 @@ static inline const void *arch_kasan_set_tag(const void *addr, u8 tag)

#define hw_enable_tagging_sync() arch_enable_tagging_sync()
#define hw_enable_tagging_async() arch_enable_tagging_async()
+#define hw_enable_tagging_asymm() arch_enable_tagging_asymm()
#define hw_force_async_tag_fault() arch_force_async_tag_fault()
#define hw_get_random_tag() arch_get_random_tag()
#define hw_get_mem_tag(addr) arch_get_mem_tag(addr)
@@ -312,6 +327,7 @@ static inline const void *arch_kasan_set_tag(const void *addr, u8 tag)

#define hw_enable_tagging_sync()
#define hw_enable_tagging_async()
+#define hw_enable_tagging_asymm()

#endif /* CONFIG_KASAN_HW_TAGS */

diff --git a/mm/kasan/report.c b/mm/kasan/report.c
index 884a950c7026..9da071ad930c 100644
--- a/mm/kasan/report.c
+++ b/mm/kasan/report.c
@@ -112,7 +112,7 @@ static void start_report(unsigned long *flags)

static void end_report(unsigned long *flags, unsigned long addr)
{
- if (!kasan_async_mode_enabled())
+ if (!kasan_async_fault_possible())
trace_error_report_end(ERROR_DETECTOR_KASAN, addr);
pr_err("==================================================================\n");
add_taint(TAINT_BAD_PAGE, LOCKDEP_NOW_UNRELIABLE);
--
2.33.0

Andrew Morton

unread,
Oct 5, 2021, 6:25:33 PM10/5/21
to Vincenzo Frascino, linux-ar...@lists.infradead.org, linux-...@vger.kernel.org, kasa...@googlegroups.com, Catalin Marinas, Will Deacon, Dmitry Vyukov, Andrey Ryabinin, Alexander Potapenko, Marco Elver, Evgenii Stepanov, Branislav Rankov, Andrey Konovalov, Lorenzo Pieralisi
On Mon, 4 Oct 2021 21:22:48 +0100 Vincenzo Frascino <vincenzo...@arm.com> wrote:

> This series implements the in-kernel asymmetric mode support for
> ARMv8.7-A Memory Tagging Extension (MTE), which is a debugging feature
> that allows to detect with the help of the architecture the C and C++
> programmatic memory errors like buffer overflow, use-after-free,
> use-after-return, etc.

I'm not sure which subsystem tree you were targeting here, so I grabbed
them. I'll drop the -mm copy if this material pops up in a linux-next
via a different tree.

Will Deacon

unread,
Oct 6, 2021, 4:05:37 AM10/6/21
to Andrew Morton, Vincenzo Frascino, linux-ar...@lists.infradead.org, linux-...@vger.kernel.org, kasa...@googlegroups.com, Catalin Marinas, Dmitry Vyukov, Andrey Ryabinin, Alexander Potapenko, Marco Elver, Evgenii Stepanov, Branislav Rankov, Andrey Konovalov, Lorenzo Pieralisi
Hi Andrew,
I'll queue 'em via the arm64 tree, as we already have some MTE work over
there and most of this series seems to be arch changes.

Cheers,

Will

Will Deacon

unread,
Oct 6, 2021, 6:58:38 AM10/6/21
to Vincenzo Frascino, linux-ar...@lists.infradead.org, linux-...@vger.kernel.org, kasa...@googlegroups.com, Andrew Morton, Catalin Marinas, Dmitry Vyukov, Andrey Ryabinin, Alexander Potapenko, Marco Elver, Evgenii Stepanov, Branislav Rankov, Andrey Konovalov, Lorenzo Pieralisi
On Mon, Oct 04, 2021 at 09:22:53PM +0100, Vincenzo Frascino wrote:
> Architectures supported by KASAN_HW_TAGS can provide an asymmetric mode
> of execution. On an MTE enabled arm64 hw for example this can be
> identified with the asymmetric tagging mode of execution. In particular,
> when such a mode is present, the CPU triggers a fault on a tag mismatch
> during a load operation and asynchronously updates a register when a tag
> mismatch is detected during a store operation.
>
> Extend the KASAN HW execution mode kernel command line parameter to
> support asymmetric mode.
>
> Cc: Dmitry Vyukov <dvy...@google.com>
> Cc: Andrey Ryabinin <arya...@virtuozzo.com>
> Cc: Alexander Potapenko <gli...@google.com>
> Cc: Andrey Konovalov <andre...@gmail.com>
> Signed-off-by: Vincenzo Frascino <vincenzo...@arm.com>
> Reviewed-by: Catalin Marinas <catalin...@arm.com>
> ---
> Documentation/dev-tools/kasan.rst | 7 +++++--
> lib/test_kasan.c | 2 +-
> mm/kasan/hw_tags.c | 27 ++++++++++++++++++++++-----
> mm/kasan/kasan.h | 22 +++++++++++++++++++---
> mm/kasan/report.c | 2 +-
> 5 files changed, 48 insertions(+), 12 deletions(-)

I'll wait for an Ack from Andrey or Marco before queueing this.

Will

Marco Elver

unread,
Oct 6, 2021, 7:35:23 AM10/6/21
to Vincenzo Frascino, linux-ar...@lists.infradead.org, linux-...@vger.kernel.org, kasa...@googlegroups.com, Andrew Morton, Catalin Marinas, Will Deacon, Dmitry Vyukov, Andrey Ryabinin, Alexander Potapenko, Evgenii Stepanov, Branislav Rankov, Andrey Konovalov, Lorenzo Pieralisi
On Mon, Oct 04, 2021 at 09:22PM +0100, Vincenzo Frascino wrote:
[...]
> DECLARE_STATIC_KEY_FALSE(kasan_flag_stacktrace);
> extern bool kasan_flag_async __ro_after_init;
> +extern bool kasan_flag_asymm __ro_after_init;
>
> static inline bool kasan_stack_collection_enabled(void)
> {
> return static_branch_unlikely(&kasan_flag_stacktrace);
> }
>
> -static inline bool kasan_async_mode_enabled(void)
> +static inline bool kasan_async_fault_possible(void)
> {
> - return kasan_flag_async;
> + return kasan_flag_async | kasan_flag_asymm;
> +}
> +
> +static inline bool kasan_sync_fault_possible(void)
> +{
> + return !kasan_flag_async | kasan_flag_asymm;
> }

Is the choice of bit-wise OR a typo? Because this should probably have
been logical OR. In this case, functionally it shouldn't matter, but is
unusual style.

Andrey Konovalov

unread,
Oct 6, 2021, 8:19:20 AM10/6/21
to Vincenzo Frascino, Linux ARM, LKML, kasan-dev, Andrew Morton, Catalin Marinas, Will Deacon, Dmitry Vyukov, Andrey Ryabinin, Alexander Potapenko, Marco Elver, Evgenii Stepanov, Branislav Rankov, Lorenzo Pieralisi
Using this variable and function here still looks confusing. Maybe
naming the variable mte_async_or_asymm_mode? Or
mte_async_fault_possible similarly to KASAN?

Andrey Konovalov

unread,
Oct 6, 2021, 8:19:39 AM10/6/21
to Vincenzo Frascino, Linux ARM, LKML, kasan-dev, Andrew Morton, Catalin Marinas, Will Deacon, Dmitry Vyukov, Andrey Ryabinin, Alexander Potapenko, Marco Elver, Evgenii Stepanov, Branislav Rankov, Lorenzo Pieralisi
On Mon, Oct 4, 2021 at 10:23 PM Vincenzo Frascino
<vincenzo...@arm.com> wrote:
>
This should be just !kasan_flag_async.

It seems that choosing one exclusive option out of 3 via two bools is
confusing. How about an enum?

enum kasan_mode {
KASAN_MODE_SYNC,
KASAN_MODE_ASYNC,
KASAN_MODE_ASYMM,
};

enum kasan_mode kasan_mode __ro_after_init;
EXPORT_SYMBOL_GPL(kasan_mode);

I also agree with Marco re using || instead of |.

Vincenzo Frascino

unread,
Oct 6, 2021, 10:21:05 AM10/6/21
to Andrey Konovalov, Linux ARM, LKML, kasan-dev, Andrew Morton, Catalin Marinas, Will Deacon, Dmitry Vyukov, Andrey Ryabinin, Alexander Potapenko, Marco Elver, Evgenii Stepanov, Branislav Rankov, Lorenzo Pieralisi
On 10/6/21 2:19 PM, Andrey Konovalov wrote:
>> + if (!system_uses_mte_async_mode())
>> + static_branch_enable(&mte_async_mode);
> Using this variable and function here still looks confusing. Maybe
> naming the variable mte_async_or_asymm_mode? Or
> mte_async_fault_possible similarly to KASAN?
>
>

mte_async_or_asymm_mode works for me. I will modify the code in v3.

Thanks.

--
Regards,
Vincenzo

Vincenzo Frascino

unread,
Oct 6, 2021, 10:22:25 AM10/6/21
to Marco Elver, linux-ar...@lists.infradead.org, linux-...@vger.kernel.org, kasa...@googlegroups.com, Andrew Morton, Catalin Marinas, Will Deacon, Dmitry Vyukov, Andrey Ryabinin, Alexander Potapenko, Evgenii Stepanov, Branislav Rankov, Andrey Konovalov, Lorenzo Pieralisi


On 10/6/21 1:35 PM, Marco Elver wrote:
>> +static inline bool kasan_sync_fault_possible(void)
>> +{
>> + return !kasan_flag_async | kasan_flag_asymm;
>> }
> Is the choice of bit-wise OR a typo? Because this should probably have
> been logical OR. In this case, functionally it shouldn't matter, but is
> unusual style.

Good catch!

This is clearly a typo and a copy-paste of a typo ;)

--
Regards,
Vincenzo

Vincenzo Frascino

unread,
Oct 6, 2021, 10:24:56 AM10/6/21
to Andrey Konovalov, Linux ARM, LKML, kasan-dev, Andrew Morton, Catalin Marinas, Will Deacon, Dmitry Vyukov, Andrey Ryabinin, Alexander Potapenko, Marco Elver, Evgenii Stepanov, Branislav Rankov, Lorenzo Pieralisi
On 10/6/21 2:19 PM, Andrey Konovalov wrote:
>> +static inline bool kasan_sync_fault_possible(void)
>> +{
>> + return !kasan_flag_async | kasan_flag_asymm;
> This should be just !kasan_flag_async.
>
> It seems that choosing one exclusive option out of 3 via two bools is
> confusing. How about an enum?
>
> enum kasan_mode {
> KASAN_MODE_SYNC,
> KASAN_MODE_ASYNC,
> KASAN_MODE_ASYMM,
> };
>
> enum kasan_mode kasan_mode __ro_after_init;
> EXPORT_SYMBOL_GPL(kasan_mode);

Fine by me. I will change the code in v3.

Thanks!

--
Regards,
Vincenzo

Vincenzo Frascino

unread,
Oct 6, 2021, 11:48:03 AM10/6/21
to linux-ar...@lists.infradead.org, linux-...@vger.kernel.org, kasa...@googlegroups.com, vincenzo...@arm.com, Andrew Morton, Catalin Marinas, Will Deacon, Dmitry Vyukov, Andrey Ryabinin, Alexander Potapenko, Marco Elver, Evgenii Stepanov, Branislav Rankov, Andrey Konovalov, Lorenzo Pieralisi
This series implements the in-kernel asymmetric mode support for
ARMv8.7-A Memory Tagging Extension (MTE), which is a debugging feature
that allows to detect with the help of the architecture the C and C++
programmatic memory errors like buffer overflow, use-after-free,
use-after-return, etc.

MTE is built on top of the AArch64 v8.0 virtual address tagging TBI
(Top Byte Ignore) feature and allows a task to set a 4 bit tag on any
subset of its address space that is multiple of a 16 bytes granule. MTE
is based on a lock-key mechanism where the lock is the tag associated to
the physical memory and the key is the tag associated to the virtual
address.

When MTE is enabled and tags are set for ranges of address space of a task,
the PE will compare the tag related to the physical memory with the tag
related to the virtual address (tag check operation). Access to the memory
is granted only if the two tags match. In case of mismatch the PE will raise
an exception.

When asymmetric mode is present, the CPU triggers a fault on a tag mismatch
during a load operation and asynchronously updates a register when a tag
mismatch is detected during a store operation.

Note: The userspace support will be sent with a future patch series.

The series is based on linux-v5.15-rc4.

To simplify the testing a tree with the new patches on top has been made
available at [1].

[1] https://git.gitlab.arm.com/linux-arm/linux-vf.git mte/v3.asymm

Cc: Andrew Morton <ak...@linux-foundation.org>
Cc: Catalin Marinas <catalin...@arm.com>
Cc: Will Deacon <wi...@kernel.org>
Cc: Dmitry Vyukov <dvy...@google.com>
Cc: Andrey Ryabinin <arya...@virtuozzo.com>
Cc: Alexander Potapenko <gli...@google.com>
Cc: Marco Elver <el...@google.com>
Cc: Evgenii Stepanov <eug...@google.com>
Cc: Branislav Rankov <Branisla...@arm.com>
Cc: Andrey Konovalov <andre...@gmail.com>
Cc: Lorenzo Pieralisi <lorenzo....@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo...@arm.com>

Vincenzo Frascino (5):
kasan: Remove duplicate of kasan_flag_async
arm64: mte: Bitfield definitions for Asymm MTE
arm64: mte: CPU feature detection for Asymm MTE
arm64: mte: Add asymmetric mode support
kasan: Extend KASAN mode kernel parameter

Documentation/dev-tools/kasan.rst | 7 +++--
arch/arm64/include/asm/memory.h | 1 +
arch/arm64/include/asm/mte-kasan.h | 5 ++++
arch/arm64/include/asm/mte.h | 8 +++---
arch/arm64/include/asm/sysreg.h | 3 +++
arch/arm64/include/asm/uaccess.h | 4 +--
arch/arm64/kernel/cpufeature.c | 10 +++++++
arch/arm64/kernel/mte.c | 43 +++++++++++++++++++++++++-----
arch/arm64/tools/cpucaps | 1 +
lib/test_kasan.c | 2 +-
mm/kasan/hw_tags.c | 28 ++++++++++++-------
mm/kasan/kasan.h | 33 ++++++++++++++++++-----
mm/kasan/report.c | 2 +-
13 files changed, 115 insertions(+), 32 deletions(-)

--
2.33.0

Vincenzo Frascino

unread,
Oct 6, 2021, 11:48:06 AM10/6/21
to linux-ar...@lists.infradead.org, linux-...@vger.kernel.org, kasa...@googlegroups.com, vincenzo...@arm.com, Andrew Morton, Catalin Marinas, Will Deacon, Dmitry Vyukov, Andrey Ryabinin, Alexander Potapenko, Marco Elver, Evgenii Stepanov, Branislav Rankov, Andrey Konovalov, Lorenzo Pieralisi
After merging async mode for KASAN_HW_TAGS a duplicate of the
kasan_flag_async flag was left erroneously inside the code.

Remove the duplicate.

Note: This change does not bring functional changes to the code
base.

Cc: Dmitry Vyukov <dvy...@google.com>
Cc: Andrey Ryabinin <arya...@virtuozzo.com>
Cc: Alexander Potapenko <gli...@google.com>
Cc: Marco Elver <el...@google.com>
Cc: Evgenii Stepanov <eug...@google.com>
Cc: Andrey Konovalov <andre...@gmail.com>
Signed-off-by: Vincenzo Frascino <vincenzo...@arm.com>
Acked-by: Catalin Marinas <catalin...@arm.com>
Reviewed-by: Andrey Konovalov <andre...@gmail.com>
---
mm/kasan/kasan.h | 2 --
1 file changed, 2 deletions(-)

diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h
index 8bf568a80eb8..3639e7c8bb98 100644
--- a/mm/kasan/kasan.h
+++ b/mm/kasan/kasan.h

Vincenzo Frascino

unread,
Oct 6, 2021, 11:48:07 AM10/6/21
to linux-ar...@lists.infradead.org, linux-...@vger.kernel.org, kasa...@googlegroups.com, vincenzo...@arm.com, Andrew Morton, Catalin Marinas, Will Deacon, Dmitry Vyukov, Andrey Ryabinin, Alexander Potapenko, Marco Elver, Evgenii Stepanov, Branislav Rankov, Andrey Konovalov, Lorenzo Pieralisi
Add Asymmetric Memory Tagging Extension bitfield definitions.

Cc: Will Deacon <wi...@kernel.org>
Cc: Catalin Marinas <catalin...@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo...@arm.com>
Acked-by: Catalin Marinas <catalin...@arm.com>

Vincenzo Frascino

unread,
Oct 6, 2021, 11:48:10 AM10/6/21
to linux-ar...@lists.infradead.org, linux-...@vger.kernel.org, kasa...@googlegroups.com, vincenzo...@arm.com, Andrew Morton, Catalin Marinas, Will Deacon, Dmitry Vyukov, Andrey Ryabinin, Alexander Potapenko, Marco Elver, Evgenii Stepanov, Branislav Rankov, Andrey Konovalov, Lorenzo Pieralisi, Suzuki K Poulose, Suzuki K Poulose
Add the cpufeature entries to detect the presence of Asymmetric MTE.

Note: The tag checking mode is initialized via cpu_enable_mte() ->
kasan_init_hw_tags() hence to enable it we require asymmetric mode
to be at least on the boot CPU. If the boot CPU does not have it, it is
fine for late CPUs to have it as long as the feature is not enabled
(ARM64_CPUCAP_BOOT_CPU_FEATURE).

Cc: Will Deacon <wi...@kernel.org>
Cc: Catalin Marinas <catalin...@arm.com>
Cc: Suzuki K Poulose <Suzuki....@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo...@arm.com>
Reviewed-by: Catalin Marinas <catalin...@arm.com>

Vincenzo Frascino

unread,
Oct 6, 2021, 11:48:12 AM10/6/21
to linux-ar...@lists.infradead.org, linux-...@vger.kernel.org, kasa...@googlegroups.com, vincenzo...@arm.com, Andrew Morton, Catalin Marinas, Will Deacon, Dmitry Vyukov, Andrey Ryabinin, Alexander Potapenko, Marco Elver, Evgenii Stepanov, Branislav Rankov, Andrey Konovalov, Lorenzo Pieralisi
MTE provides an asymmetric mode for detecting tag exceptions. In
particular, when such a mode is present, the CPU triggers a fault
on a tag mismatch during a load operation and asynchronously updates
a register when a tag mismatch is detected during a store operation.

Add support for MTE asymmetric mode.

Note: If the CPU does not support MTE asymmetric mode the kernel falls
back on synchronous mode which is the default for kasan=on.

Cc: Will Deacon <wi...@kernel.org>
Cc: Catalin Marinas <catalin...@arm.com>
Cc: Andrey Konovalov <andre...@gmail.com>
Signed-off-by: Vincenzo Frascino <vincenzo...@arm.com>
Reviewed-by: Catalin Marinas <catalin...@arm.com>
---
arch/arm64/include/asm/memory.h | 1 +
arch/arm64/include/asm/mte-kasan.h | 5 ++++
arch/arm64/include/asm/mte.h | 8 +++---
arch/arm64/include/asm/uaccess.h | 4 +--
arch/arm64/kernel/mte.c | 43 +++++++++++++++++++++++++-----
5 files changed, 49 insertions(+), 12 deletions(-)
diff --git a/arch/arm64/include/asm/mte.h b/arch/arm64/include/asm/mte.h
index 02511650cffe..075539f5f1c8 100644
--- a/arch/arm64/include/asm/mte.h
+++ b/arch/arm64/include/asm/mte.h
@@ -88,11 +88,11 @@ static inline int mte_ptrace_copy_tags(struct task_struct *child,

#ifdef CONFIG_KASAN_HW_TAGS
/* Whether the MTE asynchronous mode is enabled. */
-DECLARE_STATIC_KEY_FALSE(mte_async_mode);
+DECLARE_STATIC_KEY_FALSE(mte_async_or_asymm_mode);

-static inline bool system_uses_mte_async_mode(void)
+static inline bool system_uses_mte_async_or_asymm_mode(void)
{
- return static_branch_unlikely(&mte_async_mode);
+ return static_branch_unlikely(&mte_async_or_asymm_mode);
}

void mte_check_tfsr_el1(void);
@@ -121,7 +121,7 @@ static inline void mte_check_tfsr_exit(void)
mte_check_tfsr_el1();
}
#else
-static inline bool system_uses_mte_async_mode(void)
+static inline bool system_uses_mte_async_or_asymm_mode(void)
{
return false;
}
diff --git a/arch/arm64/include/asm/uaccess.h b/arch/arm64/include/asm/uaccess.h
index 190b494e22ab..315354047d69 100644
--- a/arch/arm64/include/asm/uaccess.h
+++ b/arch/arm64/include/asm/uaccess.h
@@ -196,13 +196,13 @@ static inline void __uaccess_enable_tco(void)
*/
static inline void __uaccess_disable_tco_async(void)
{
- if (system_uses_mte_async_mode())
+ if (system_uses_mte_async_or_asymm_mode())
__uaccess_disable_tco();
}

static inline void __uaccess_enable_tco_async(void)
{
- if (system_uses_mte_async_mode())
+ if (system_uses_mte_async_or_asymm_mode())
__uaccess_enable_tco();
}

diff --git a/arch/arm64/kernel/mte.c b/arch/arm64/kernel/mte.c
index e5e801bc5312..d7da4e3924c4 100644
--- a/arch/arm64/kernel/mte.c
+++ b/arch/arm64/kernel/mte.c
@@ -26,9 +26,14 @@
static DEFINE_PER_CPU_READ_MOSTLY(u64, mte_tcf_preferred);

#ifdef CONFIG_KASAN_HW_TAGS
-/* Whether the MTE asynchronous mode is enabled. */
-DEFINE_STATIC_KEY_FALSE(mte_async_mode);
-EXPORT_SYMBOL_GPL(mte_async_mode);
+/*
+ * The MTE asynchronous and asymmetric mode have the same
+ * behavior for the store operations.
+ *
+ * Whether the MTE asynchronous or asymmetric mode is enabled.
+ */
+DEFINE_STATIC_KEY_FALSE(mte_async_or_asymm_mode);
+EXPORT_SYMBOL_GPL(mte_async_or_asymm_mode);
#endif

static void mte_sync_page_tags(struct page *page, pte_t old_pte,
@@ -116,7 +121,7 @@ void mte_enable_kernel_sync(void)
* Make sure we enter this function when no PE has set
* async mode previously.
*/
- WARN_ONCE(system_uses_mte_async_mode(),
+ WARN_ONCE(system_uses_mte_async_or_asymm_mode(),
"MTE async mode enabled system wide!");

__mte_enable_kernel("synchronous", SCTLR_ELx_TCF_SYNC);
@@ -134,8 +139,34 @@ void mte_enable_kernel_async(void)
* mode in between sync and async, this strategy needs
* to be reviewed.
*/
- if (!system_uses_mte_async_mode())
- static_branch_enable(&mte_async_mode);
+ if (!system_uses_mte_async_or_asymm_mode())
+ static_branch_enable(&mte_async_or_asymm_mode);
+}
+
+void mte_enable_kernel_asymm(void)
+{
+ if (cpus_have_cap(ARM64_MTE_ASYMM)) {
+ __mte_enable_kernel("asymmetric", SCTLR_ELx_TCF_ASYMM);
+
+ /*
+ * MTE asymm mode behaves as async mode for store
+ * operations. The mode is set system wide by the
+ * first PE that executes this function.
+ *
+ * Note: If in future KASAN acquires a runtime switching
+ * mode in between sync and async, this strategy needs
+ * to be reviewed.
+ */
+ if (!system_uses_mte_async_or_asymm_mode())
+ static_branch_enable(&mte_async_or_asymm_mode);
+ } else {
+ /*
+ * If the CPU does not support MTE asymmetric mode the
+ * kernel falls back on synchronous mode which is the
+ * default for kasan=on.
+ */
+ mte_enable_kernel_sync();
+ }
}
#endif

--
2.33.0

Vincenzo Frascino

unread,
Oct 6, 2021, 11:48:14 AM10/6/21
to linux-ar...@lists.infradead.org, linux-...@vger.kernel.org, kasa...@googlegroups.com, vincenzo...@arm.com, Andrew Morton, Catalin Marinas, Will Deacon, Dmitry Vyukov, Andrey Ryabinin, Alexander Potapenko, Marco Elver, Evgenii Stepanov, Branislav Rankov, Andrey Konovalov, Lorenzo Pieralisi
Architectures supported by KASAN_HW_TAGS can provide an asymmetric mode
of execution. On an MTE enabled arm64 hw for example this can be
identified with the asymmetric tagging mode of execution. In particular,
when such a mode is present, the CPU triggers a fault on a tag mismatch
during a load operation and asynchronously updates a register when a tag
mismatch is detected during a store operation.

Extend the KASAN HW execution mode kernel command line parameter to
support asymmetric mode.

Cc: Dmitry Vyukov <dvy...@google.com>
Cc: Andrey Ryabinin <arya...@virtuozzo.com>
Cc: Alexander Potapenko <gli...@google.com>
Cc: Andrey Konovalov <andre...@gmail.com>
Signed-off-by: Vincenzo Frascino <vincenzo...@arm.com>
Reviewed-by: Catalin Marinas <catalin...@arm.com>
---
Documentation/dev-tools/kasan.rst | 7 +++++--
lib/test_kasan.c | 2 +-
mm/kasan/hw_tags.c | 28 ++++++++++++++++++----------
mm/kasan/kasan.h | 31 +++++++++++++++++++++++++++----
mm/kasan/report.c | 2 +-
5 files changed, 52 insertions(+), 18 deletions(-)
index 05d1e9460e2e..39e34595f2b4 100644
--- a/mm/kasan/hw_tags.c
+++ b/mm/kasan/hw_tags.c
@@ -29,6 +29,7 @@ enum kasan_arg_mode {
KASAN_ARG_MODE_DEFAULT,
KASAN_ARG_MODE_SYNC,
KASAN_ARG_MODE_ASYNC,
+ KASAN_ARG_MODE_ASYMM,
};

enum kasan_arg_stacktrace {
@@ -45,9 +46,9 @@ static enum kasan_arg_stacktrace kasan_arg_stacktrace __ro_after_init;
DEFINE_STATIC_KEY_FALSE(kasan_flag_enabled);
EXPORT_SYMBOL(kasan_flag_enabled);

-/* Whether the asynchronous mode is enabled. */
-bool kasan_flag_async __ro_after_init;
-EXPORT_SYMBOL_GPL(kasan_flag_async);
+/* Whether the selected mode is synchronous/asynchronous/asymmetric.*/
+enum kasan_mode kasan_mode __ro_after_init;
+EXPORT_SYMBOL_GPL(kasan_mode);

/* Whether to collect alloc/free stack traces. */
DEFINE_STATIC_KEY_FALSE(kasan_flag_stacktrace);
@@ -69,7 +70,7 @@ static int __init early_kasan_flag(char *arg)
}
early_param("kasan", early_kasan_flag);

-/* kasan.mode=sync/async */
+/* kasan.mode=sync/async/asymm */
static int __init early_kasan_mode(char *arg)
{
if (!arg)
@@ -79,6 +80,8 @@ static int __init early_kasan_mode(char *arg)
kasan_arg_mode = KASAN_ARG_MODE_SYNC;
else if (!strcmp(arg, "async"))
kasan_arg_mode = KASAN_ARG_MODE_ASYNC;
+ else if (!strcmp(arg, "asymm"))
+ kasan_arg_mode = KASAN_ARG_MODE_ASYMM;
else
return -EINVAL;

@@ -116,11 +119,13 @@ void kasan_init_hw_tags_cpu(void)
return;

/*
- * Enable async mode only when explicitly requested through
- * the command line.
+ * Enable async or asymm modes only when explicitly requested
+ * through the command line.
*/
if (kasan_arg_mode == KASAN_ARG_MODE_ASYNC)
hw_enable_tagging_async();
+ else if (kasan_arg_mode == KASAN_ARG_MODE_ASYMM)
+ hw_enable_tagging_asymm();
else
hw_enable_tagging_sync();
}
@@ -143,15 +148,18 @@ void __init kasan_init_hw_tags(void)
case KASAN_ARG_MODE_DEFAULT:
/*
* Default to sync mode.
- * Do nothing, kasan_flag_async keeps its default value.
*/
- break;
case KASAN_ARG_MODE_SYNC:
- /* Do nothing, kasan_flag_async keeps its default value. */
+ /* Sync mode enabled. */
+ kasan_mode = KASAN_MODE_SYNC;
break;
case KASAN_ARG_MODE_ASYNC:
/* Async mode enabled. */
- kasan_flag_async = true;
+ kasan_mode = KASAN_MODE_ASYNC;
+ break;
+ case KASAN_ARG_MODE_ASYMM:
+ /* Asymm mode enabled. */
+ kasan_mode = KASAN_MODE_ASYMM;
break;
}

diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h
index 3639e7c8bb98..71b1b5d3d97e 100644
--- a/mm/kasan/kasan.h
+++ b/mm/kasan/kasan.h
@@ -13,16 +13,29 @@
#include "../slab.h"

DECLARE_STATIC_KEY_FALSE(kasan_flag_stacktrace);
-extern bool kasan_flag_async __ro_after_init;
+
+enum kasan_mode {
+ KASAN_MODE_SYNC,
+ KASAN_MODE_ASYNC,
+ KASAN_MODE_ASYMM,
+};
+
+extern enum kasan_mode kasan_mode __ro_after_init;

static inline bool kasan_stack_collection_enabled(void)
{
return static_branch_unlikely(&kasan_flag_stacktrace);
}

-static inline bool kasan_async_mode_enabled(void)
+static inline bool kasan_async_fault_possible(void)
+{
+ return kasan_mode == KASAN_MODE_ASYNC ||
+ kasan_mode == KASAN_MODE_ASYMM;
+}
+
+static inline bool kasan_sync_fault_possible(void)
{
- return kasan_flag_async;
+ return kasan_mode != KASAN_MODE_ASYNC;
}
#else

@@ -31,11 +44,16 @@ static inline bool kasan_stack_collection_enabled(void)
return true;
}

-static inline bool kasan_async_mode_enabled(void)
+static inline bool kasan_async_fault_possible(void)
{
return false;
}

+static inline bool kasan_sync_fault_possible(void)
+{
+ return true;
+}
+
#endif

#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
@@ -287,6 +305,9 @@ static inline const void *arch_kasan_set_tag(const void *addr, u8 tag)
#ifndef arch_enable_tagging_async
#define arch_enable_tagging_async()
#endif
+#ifndef arch_enable_tagging_asymm
+#define arch_enable_tagging_asymm()
+#endif
#ifndef arch_force_async_tag_fault
#define arch_force_async_tag_fault()
#endif
@@ -302,6 +323,7 @@ static inline const void *arch_kasan_set_tag(const void *addr, u8 tag)

#define hw_enable_tagging_sync() arch_enable_tagging_sync()
#define hw_enable_tagging_async() arch_enable_tagging_async()
+#define hw_enable_tagging_asymm() arch_enable_tagging_asymm()
#define hw_force_async_tag_fault() arch_force_async_tag_fault()
#define hw_get_random_tag() arch_get_random_tag()
#define hw_get_mem_tag(addr) arch_get_mem_tag(addr)
@@ -312,6 +334,7 @@ static inline const void *arch_kasan_set_tag(const void *addr, u8 tag)

Andrey Konovalov

unread,
Oct 6, 2021, 7:06:43 PM10/6/21
to Vincenzo Frascino, Linux ARM, LKML, kasan-dev, Andrew Morton, Catalin Marinas, Will Deacon, Dmitry Vyukov, Andrey Ryabinin, Alexander Potapenko, Marco Elver, Evgenii Stepanov, Branislav Rankov, Lorenzo Pieralisi
Nit:
The asynchronous and asymmetric MTE modes have the same behavior for
store operations. This flag is set when either of these modes is
Acked-by: Andrey Konovalov <andre...@gmail.com>

Andrey Konovalov

unread,
Oct 6, 2021, 7:07:04 PM10/6/21
to Vincenzo Frascino, Linux ARM, LKML, kasan-dev, Andrew Morton, Catalin Marinas, Will Deacon, Dmitry Vyukov, Andrey Ryabinin, Alexander Potapenko, Marco Elver, Evgenii Stepanov, Branislav Rankov, Lorenzo Pieralisi
.On Wed, Oct 6, 2021 at 5:48 PM Vincenzo Frascino
kasan_mode = KASAN_MODE_SYNC;

then, since the "do nothing" comment is dropped.
kasan_mode == KASAN_MODE_SYNC || kasan_mode == KASAN_MODE_ASYMM

is more in line with the condition in kasan_async_fault_possible().
With the mentioned changes:

Reviewed-by: Andrey Konovalov <andre...@gmail.com>

Will Deacon

unread,
Oct 7, 2021, 5:48:26 AM10/7/21
to Vincenzo Frascino, linux-...@vger.kernel.org, linux-ar...@lists.infradead.org, kasa...@googlegroups.com, catalin...@arm.com, kerne...@android.com, Will Deacon, Dmitry Vyukov, Andrey Ryabinin, Branislav Rankov, Marco Elver, Andrey Konovalov, Lorenzo Pieralisi, Alexander Potapenko, Andrew Morton, Evgenii Stepanov
On Wed, 6 Oct 2021 16:47:46 +0100, Vincenzo Frascino wrote:
> This series implements the in-kernel asymmetric mode support for
> ARMv8.7-A Memory Tagging Extension (MTE), which is a debugging feature
> that allows to detect with the help of the architecture the C and C++
> programmatic memory errors like buffer overflow, use-after-free,
> use-after-return, etc.
>
> MTE is built on top of the AArch64 v8.0 virtual address tagging TBI
> (Top Byte Ignore) feature and allows a task to set a 4 bit tag on any
> subset of its address space that is multiple of a 16 bytes granule. MTE
> is based on a lock-key mechanism where the lock is the tag associated to
> the physical memory and the key is the tag associated to the virtual
> address.
>
> [...]

Applied to arm64 (for-next/mte), thanks!

[1/5] kasan: Remove duplicate of kasan_flag_async
https://git.kernel.org/arm64/c/f5627ec1ff2c
[2/5] arm64: mte: Bitfield definitions for Asymm MTE
https://git.kernel.org/arm64/c/ba1a98e8b172
[3/5] arm64: mte: CPU feature detection for Asymm MTE
https://git.kernel.org/arm64/c/d73c162e0733
[4/5] arm64: mte: Add asymmetric mode support
https://git.kernel.org/arm64/c/ec0288369f0c
[5/5] kasan: Extend KASAN mode kernel parameter
https://git.kernel.org/arm64/c/2d27e5851473

Cheers,
--
Will

https://fixes.arm64.dev
https://next.arm64.dev
https://will.arm64.dev
Reply all
Reply to author
Forward
0 new messages