ANDROID: mm: skip pte_alloc during speculative page fault
Speculative page fault checks pmd to be valid before starting to handle
the page fault and pte_alloc() should do nothing if pmd stays valid.
If pmd gets changed during speculative page fault, we will detect the
change later and retry with mmap_lock. Therefore pte_alloc() can be
safely skipped and this prevents the racy pmd_lock() call which can
access pmd->ptl after pmd was cleared.
Bug: 257443051
Change-Id: Iec57df5530dba6e0e0bdf9f7500f910851c3d3fd
Signed-off-by: Suren Baghdasaryan <surenb@google.com>
diff --git a/mm/memory.c b/mm/memory.c
index 6bc34d7..6519783 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -3907,6 +3907,10 @@ static vm_fault_t do_anonymous_page(struct vm_fault *vmf)
if (vma->vm_flags & VM_SHARED)
return VM_FAULT_SIGBUS;
+ /* Do not check unstable pmd, if it's changed will retry later */
+ if (vmf->flags & FAULT_FLAG_SPECULATIVE)
+ goto skip_pmd_checks;
+
/*
* Use pte_alloc() instead of pte_alloc_map(). We can't run
* pte_offset_map() on pmds where a huge pmd might be created
@@ -3924,6 +3928,7 @@ static vm_fault_t do_anonymous_page(struct vm_fault *vmf)
if (unlikely(pmd_trans_unstable(vmf->pmd)))
return 0;
+skip_pmd_checks:
/* Use the zero-page for reads */
if (!(vmf->flags & FAULT_FLAG_WRITE) &&
!mm_forbids_zeropage(vma->vm_mm)) {