• Mel Gorman's avatar
    mm/page_alloc: replace local_lock with normal spinlock · 01b44456
    Mel Gorman authored
    struct per_cpu_pages is no longer strictly local as PCP lists can be
    drained remotely using a lock for protection.  While the use of local_lock
    works, it goes against the intent of local_lock which is for "pure CPU
    local concurrency control mechanisms and not suited for inter-CPU
    concurrency control" (Documentation/locking/locktypes.rst)
    
    local_lock protects against migration between when the percpu pointer is
    accessed and the pcp->lock acquired.  The lock acquisition is a preemption
    point so in the worst case, a task could migrate to another NUMA node and
    accidentally allocate remote memory.  The main requirement is to pin the
    task to a CPU that is suitable for PREEMPT_RT and !PREEMPT_RT.
    
    Replace local_lock with helpers that pin a task to a CPU, lookup the
    per-cpu structure and acquire the embedded lock.  It's similar to
    local_lock without breaking the intent behind the API.  It is not a
    complete API as only the parts needed for PCP-alloc are implemented but in
    theory, the generic helpers could be promoted to a general API if there
    was demand for an embedded lock within a per-cpu struct with a guarantee
    that the per-cpu structure locked matches the running CPU and cannot use
    get_cpu_var due to RT concerns.  PCP requires these semantics to avoid
    accidentally allocating remote memory.
    
    [mgorman@techsingularity.net: use pcp_spin_trylock_irqsave instead of pcpu_spin_trylock_irqsave]
      Link: https://lkml.kernel.org/r/20220627084645.GA27531@techsingularity.net
    Link: https://lkml.kernel.org/r/20220624125423.6126-8-mgorman@techsingularity.netSigned-off-by: default avatarMel Gorman <mgorman@techsingularity.net>
    Tested-by: default avatarYu Zhao <yuzhao@google.com>
    Reviewed-by: default avatarNicolas Saenz Julienne <nsaenzju@redhat.com>
    Tested-by: default avatarNicolas Saenz Julienne <nsaenzju@redhat.com>
    Acked-by: default avatarVlastimil Babka <vbabka@suse.cz>
    Tested-by: default avatarYu Zhao <yuzhao@google.com>
    Cc: Hugh Dickins <hughd@google.com>
    Cc: Marcelo Tosatti <mtosatti@redhat.com>
    Cc: Marek Szyprowski <m.szyprowski@samsung.com>
    Cc: Michal Hocko <mhocko@kernel.org>
    Cc: Minchan Kim <minchan@kernel.org>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    01b44456
page_alloc.c 268 KB