• Naoya Horiguchi's avatar
    mm,hwpoison: fix race with hugetlb page allocation · 25182f05
    Naoya Horiguchi authored
    When hugetlb page fault (under overcommitting situation) and
    memory_failure() race, VM_BUG_ON_PAGE() is triggered by the following
    race:
    
        CPU0:                           CPU1:
    
                                        gather_surplus_pages()
                                          page = alloc_surplus_huge_page()
        memory_failure_hugetlb()
          get_hwpoison_page(page)
            __get_hwpoison_page(page)
              get_page_unless_zero(page)
                                          zero = put_page_testzero(page)
                                          VM_BUG_ON_PAGE(!zero, page)
                                          enqueue_huge_page(h, page)
          put_page(page)
    
    __get_hwpoison_page() only checks the page refcount before taking an
    additional one for memory error handling, which is not enough because
    there's a time window where compound pages have non-zero refcount during
    hugetlb page initialization.
    
    So make __get_hwpoison_page() check page status a bit more for hugetlb
    pages with get_hwpoison_huge_page().  Checking hugetlb-specific flags
    under hugetlb_lock makes sure that the hugetlb page is not transitive.
    It's notable that another new function, HWPoisonHandlable(), is helpful
    to prevent a race against other transitive page states (like a generic
    compound page just before PageHuge becomes true).
    
    Link: https://lkml.kernel.org/r/20210603233632.2964832-2-nao.horiguchi@gmail.com
    Fixes: ead07f6a ("mm/memory-failure: introduce get_hwpoison_page() for consistent refcount handling")
    Signed-off-by: default avatarNaoya Horiguchi <naoya.horiguchi@nec.com>
    Reported-by: default avatarMuchun Song <songmuchun@bytedance.com>
    Acked-by: default avatarMike Kravetz <mike.kravetz@oracle.com>
    Cc: Oscar Salvador <osalvador@suse.de>
    Cc: Michal Hocko <mhocko@suse.com>
    Cc: Tony Luck <tony.luck@intel.com>
    Cc: <stable@vger.kernel.org>	[5.12+]
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    25182f05
memory-failure.c 53.5 KB