• Yosry Ahmed's avatar
    mm: workingset: move the stats flush into workingset_test_recent() · b0068472
    Yosry Ahmed authored
    The workingset code flushes the stats in workingset_refault() to get
    accurate stats of the eviction memcg.  In preparation for more scoped
    flushed and passing the eviction memcg to the flush call, move the call to
    workingset_test_recent() where we have a pointer to the eviction memcg.
    
    The flush call is sleepable, and cannot be made in an rcu read section. 
    Hence, minimize the rcu read section by also moving it into
    workingset_test_recent().  Furthermore, instead of holding the rcu read
    lock throughout workingset_test_recent(), only hold it briefly to get a
    ref on the eviction memcg.  This allows us to make the flush call after we
    get the eviction memcg.
    
    As for workingset_refault(), nothing else there appears to be protected by
    rcu.  The memcg of the faulted folio (which is not necessarily the same as
    the eviction memcg) is protected by the folio lock, which is held from all
    callsites.  Add a VM_BUG_ON() to make sure this doesn't change from under
    us.
    
    No functional change intended.
    
    Link: https://lkml.kernel.org/r/20231129032154.3710765-5-yosryahmed@google.comSigned-off-by: default avatarYosry Ahmed <yosryahmed@google.com>
    Tested-by: default avatarDomenico Cerasuolo <cerasuolodomenico@gmail.com>
    Acked-by: default avatarShakeel Butt <shakeelb@google.com>
    Cc: Chris Li <chrisl@kernel.org>
    Cc: Greg Thelen <gthelen@google.com>
    Cc: Ivan Babrou <ivan@cloudflare.com>
    Cc: Johannes Weiner <hannes@cmpxchg.org>
    Cc: Michal Hocko <mhocko@kernel.org>
    Cc: Michal Koutny <mkoutny@suse.com>
    Cc: Muchun Song <muchun.song@linux.dev>
    Cc: Roman Gushchin <roman.gushchin@linux.dev>
    Cc: Tejun Heo <tj@kernel.org>
    Cc: Waiman Long <longman@redhat.com>
    Cc: Wei Xu <weixugc@google.com>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    b0068472
workingset.c 27 KB