• Jesper Dangaard Brouer's avatar
    slub: support for bulk free with SLUB freelists · 81084651
    Jesper Dangaard Brouer authored
    Make it possible to free a freelist with several objects by adjusting API
    of slab_free() and __slab_free() to have head, tail and an objects counter
    (cnt).
    
    Tail being NULL indicate single object free of head object.  This allow
    compiler inline constant propagation in slab_free() and
    slab_free_freelist_hook() to avoid adding any overhead in case of single
    object free.
    
    This allows a freelist with several objects (all within the same
    slab-page) to be free'ed using a single locked cmpxchg_double in
    __slab_free() and with an unlocked cmpxchg_double in slab_free().
    
    Object debugging on the free path is also extended to handle these
    freelists.  When CONFIG_SLUB_DEBUG is enabled it will also detect if
    objects don't belong to the same slab-page.
    
    These changes are needed for the next patch to bulk free the detached
    freelists it introduces and constructs.
    
    Micro benchmarking showed no performance reduction due to this change,
    when debugging is turned off (compiled with CONFIG_SLUB_DEBUG).
    Signed-off-by: default avatarJesper Dangaard Brouer <brouer@redhat.com>
    Signed-off-by: default avatarAlexander Duyck <alexander.h.duyck@redhat.com>
    Acked-by: default avatarChristoph Lameter <cl@linux.com>
    Cc: Pekka Enberg <penberg@kernel.org>
    Cc: David Rientjes <rientjes@google.com>
    Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    81084651
slub.c 131 KB