• Roman Gushchin's avatar
    mm: memcg/slab: charge individual slab objects instead of pages · f2fe7b09
    Roman Gushchin authored
    Switch to per-object accounting of non-root slab objects.
    
    Charging is performed using obj_cgroup API in the pre_alloc hook.
    Obj_cgroup is charged with the size of the object and the size of
    metadata: as now it's the size of an obj_cgroup pointer.  If the amount of
    memory has been charged successfully, the actual allocation code is
    executed.  Otherwise, -ENOMEM is returned.
    
    In the post_alloc hook if the actual allocation succeeded, corresponding
    vmstats are bumped and the obj_cgroup pointer is saved.  Otherwise, the
    charge is canceled.
    
    On the free path obj_cgroup pointer is obtained and used to uncharge the
    size of the releasing object.
    
    Memcg and lruvec counters are now representing only memory used by active
    slab objects and do not include the free space.  The free space is shared
    and doesn't belong to any specific cgroup.
    
    Global per-node slab vmstats are still modified from
    (un)charge_slab_page() functions.  The idea is to keep all slab pages
    accounted as slab pages on system level.
    Signed-off-by: default avatarRoman Gushchin <guro@fb.com>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Reviewed-by: default avatarVlastimil Babka <vbabka@suse.cz>
    Reviewed-by: default avatarShakeel Butt <shakeelb@google.com>
    Cc: Christoph Lameter <cl@linux.com>
    Cc: Johannes Weiner <hannes@cmpxchg.org>
    Cc: Michal Hocko <mhocko@kernel.org>
    Cc: Tejun Heo <tj@kernel.org>
    Link: http://lkml.kernel.org/r/20200623174037.3951353-10-guro@fb.comSigned-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    f2fe7b09
slab.h 21.7 KB