[BUG] mm: thp: hugepage_vma_check has a blind spot

[Alex Thorlton]

hugepage_vma_check is called during khugepaged_scan_mm_slot to ensure
that khugepaged doesn't try to allocate THPs in vmas where they are
disallowed, either due to THPs being disabled system-wide, or through
MADV_NOHUGEPAGE.

The logic that hugepage_vma_check uses doesn't seem to cover all cases,
in my opinion. Looking at the original code:

if ((!(vma->vm_flags & VM_HUGEPAGE) && !khugepaged_always()) ||
(vma->vm_flags & VM_NOHUGEPAGE))

We can see that it's possible to have THP disabled system-wide, but still
receive THPs in this vma. It seems that it's assumed that just because
khugepaged_always == false, TRANSPARENT_HUGEPAGE_REQ_MADV_FLAG must be
set, which is not the case. We could have VM_HUGEPAGE set, but have THP
set to "never" system-wide, in which case, the condition presented in the
if will evaluate to false, and (provided the other checks pass) we can
end up giving out a THP even though the behavior is set to "never."

While we do properly check these flags in khugepaged_has_work, it looks
like it's possible to sleep after we check khugepaged_hask_work, but
before hugepage_vma_check, during which time, hugepages could have been
disabled system-wide, in which case, we could hand out THPs when we
shouldn't be.

This small fix makes hugepage_vma_check work more like
transparent_hugepage_enabled, checking if THPs are set to "always"
system-wide, then checking if THPs are set to "madvise," as well as
making sure that VM_HUGEPAGE is set for this vma.

Signed-off-by: Alex Thorlton <atho...@sgi.com>
Reported-by: Alex Thorlton <atho...@sgi.com>
Cc: Andrew Morton <ak...@linux-foundation.org>
Cc: Mel Gorman <mgo...@suse.de>
Cc: Rik van Riel <ri...@redhat.com>
Cc: Ingo Molnar <mi...@kernel.org>
Cc: Peter Zijlstra <pet...@infradead.org>
Cc: linu...@kvack.org
Cc: linux-...@vger.kernel.org

---
mm/huge_memory.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index 95d1acb..f62fba9 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -2394,7 +2394,8 @@ static struct page

static bool hugepage_vma_check(struct vm_area_struct *vma)
{
- if ((!(vma->vm_flags & VM_HUGEPAGE) && !khugepaged_always()) ||
+ if ((!khugepaged_always() ||
+ (!(vma->vm_flags & VM_HUGEPAGE) && !khugepaged_req_madv())) ||
(vma->vm_flags & VM_NOHUGEPAGE))
return false;

--
1.7.12.4

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majo...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

[David Rientjes]

On Tue, 21 Jan 2014, Alex Thorlton wrote:

> hugepage_vma_check is called during khugepaged_scan_mm_slot to ensure
> that khugepaged doesn't try to allocate THPs in vmas where they are
> disallowed, either due to THPs being disabled system-wide, or through
> MADV_NOHUGEPAGE.
>
> The logic that hugepage_vma_check uses doesn't seem to cover all cases,
> in my opinion. Looking at the original code:
>
> if ((!(vma->vm_flags & VM_HUGEPAGE) && !khugepaged_always()) ||
> (vma->vm_flags & VM_NOHUGEPAGE))
>
> We can see that it's possible to have THP disabled system-wide, but still
> receive THPs in this vma. It seems that it's assumed that just because
> khugepaged_always == false, TRANSPARENT_HUGEPAGE_REQ_MADV_FLAG must be
> set, which is not the case. We could have VM_HUGEPAGE set, but have THP
> set to "never" system-wide, in which case, the condition presented in the
> if will evaluate to false, and (provided the other checks pass) we can
> end up giving out a THP even though the behavior is set to "never."
>

You should be able to add a

BUG_ON(current != khugepaged_thread);

here since khugepaged is supposed to be the only caller to the function.

> While we do properly check these flags in khugepaged_has_work, it looks
> like it's possible to sleep after we check khugepaged_hask_work, but
> before hugepage_vma_check, during which time, hugepages could have been
> disabled system-wide, in which case, we could hand out THPs when we
> shouldn't be.
>

You're talking about when thp is set to "never" and before khugepaged has
stopped, correct?

That doesn't seem like a bug to me or anything that needs to be fixed, the
sysfs knob could be switched even after hugepage_vma_check() is called and
before a hugepage is actually collapsed so you have the same race.

The only thing that's guaranteed is that, upon writing "never" to
/sys/kernel/mm/transparent_hugepage/enabled, no more thp memory will be
collapsed after khugepaged has stopped.

[Alex Thorlton]

Yes, that's correct.

> That doesn't seem like a bug to me or anything that needs to be fixed, the
> sysfs knob could be switched even after hugepage_vma_check() is called and
> before a hugepage is actually collapsed so you have the same race.
>
> The only thing that's guaranteed is that, upon writing "never" to
> /sys/kernel/mm/transparent_hugepage/enabled, no more thp memory will be
> collapsed after khugepaged has stopped.

That makes sense, I wasn't aware that that's the expected behavior here.
I suppose this isn't something that needs to be changed, in that case.
I needed the logic broken out a bit more explicitly (madvise/never case
need to be handled separately) for a patch that I'm working on - that's
when this caught my attention. Good to know that a change to the
system-wide switch shouldn't affect khugepaged if it's already running.
I would've screwed up that behavior with my patch :)

Thanks, David!

- Alex