• Kairui Song's avatar
    mm: swap: add a adaptive full cluster cache reclaim · 2cacbdfd
    Kairui Song authored
    Link all full cluster with one full list, and reclaim from it when the
    allocation have ran out of all usable clusters.
    
    There are many reason a folio can end up being in the swap cache while
    having no swap count reference.  So the best way to search for such slots
    is still by iterating the swap clusters.
    
    With the list as an LRU, iterating from the oldest cluster and keep them
    rotating is a very doable and clean way to free up potentially not inuse
    clusters.
    
    When any allocation failure, try reclaim and rotate only one cluster. 
    This is adaptive for high order allocations they can tolerate fallback. 
    So this avoids latency, and give the full cluster list an fair chance to
    get reclaimed.  It release the usage stress for the fallback order 0
    allocation or following up high order allocation.
    
    If the swap device is getting very full, reclaim more aggresively to
    ensure no OOM will happen.  This ensures order 0 heavy workload won't go
    OOM as order 0 won't fail if any cluster still have any space.
    
    [ryncsn@gmail.com: fix discard of full cluster]
      Link: https://lkml.kernel.org/r/CAMgjq7CWwK75_2Zi5P40K08pk9iqOcuWKL6khu=x4Yg_nXaQag@mail.gmail.com
    Link: https://lkml.kernel.org/r/20240730-swap-allocator-v5-9-cb9c148b9297@kernel.orgSigned-off-by: default avatarKairui Song <kasong@tencent.com>
    Reported-by: default avatarBarry Song <21cnbao@gmail.com>
    Cc: Chris Li <chrisl@kernel.org>
    Cc: "Huang, Ying" <ying.huang@intel.com>
    Cc: Hugh Dickins <hughd@google.com>
    Cc: Kalesh Singh <kaleshsingh@google.com>
    Cc: Ryan Roberts <ryan.roberts@arm.com>
    Cc: David Hildenbrand <david@redhat.com>
    Cc: Kairui Song <ryncsn@gmail.com>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    2cacbdfd
swapfile.c 99.7 KB