• Vlastimil Babka's avatar
    mm/slub: free KFENCE objects in slab_free_hook() · 782f8906
    Vlastimil Babka authored
    When freeing an object that was allocated from KFENCE, we do that in the
    slowpath __slab_free(), relying on the fact that KFENCE "slab" cannot be
    the cpu slab, so the fastpath has to fallback to the slowpath.
    
    This optimization doesn't help much though, because is_kfence_address()
    is checked earlier anyway during the free hook processing or detached
    freelist building. Thus we can simplify the code by making the
    slab_free_hook() free the KFENCE object immediately, similarly to KASAN
    quarantine.
    
    In slab_free_hook() we can place kfence_free() above init processing, as
    callers have been making sure to set init to false for KFENCE objects.
    This simplifies slab_free(). This places it also above kasan_slab_free()
    which is ok as that skips KFENCE objects anyway.
    
    While at it also determine the init value in slab_free_freelist_hook()
    outside of the loop.
    
    This change will also make introducing per cpu array caches easier.
    Tested-by: default avatarMarco Elver <elver@google.com>
    Reviewed-by: default avatarChengming Zhou <zhouchengming@bytedance.com>
    Signed-off-by: default avatarVlastimil Babka <vbabka@suse.cz>
    782f8906
slub.c 177 KB