• Alexander Potapenko's avatar
    mm, kfence: insert KFENCE hooks for SLAB · d3fb45f3
    Alexander Potapenko authored
    Inserts KFENCE hooks into the SLAB allocator.
    
    To pass the originally requested size to KFENCE, add an argument
    'orig_size' to slab_alloc*(). The additional argument is required to
    preserve the requested original size for kmalloc() allocations, which
    uses size classes (e.g. an allocation of 272 bytes will return an object
    of size 512). Therefore, kmem_cache::size does not represent the
    kmalloc-caller's requested size, and we must introduce the argument
    'orig_size' to propagate the originally requested size to KFENCE.
    
    Without the originally requested size, we would not be able to detect
    out-of-bounds accesses for objects placed at the end of a KFENCE object
    page if that object is not equal to the kmalloc-size class it was
    bucketed into.
    
    When KFENCE is disabled, there is no additional overhead, since
    slab_alloc*() functions are __always_inline.
    
    Link: https://lkml.kernel.org/r/20201103175841.3495947-5-elver@google.comSigned-off-by: default avatarMarco Elver <elver@google.com>
    Signed-off-by: default avatarAlexander Potapenko <glider@google.com>
    Reviewed-by: default avatarDmitry Vyukov <dvyukov@google.com>
    Co-developed-by: default avatarMarco Elver <elver@google.com>
    
    Cc: Christoph Lameter <cl@linux.com>
    Cc: Pekka Enberg <penberg@kernel.org>
    Cc: David Rientjes <rientjes@google.com>
    Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com>
    Cc: Andrey Konovalov <andreyknvl@google.com>
    Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
    Cc: Andy Lutomirski <luto@kernel.org>
    Cc: Borislav Petkov <bp@alien8.de>
    Cc: Catalin Marinas <catalin.marinas@arm.com>
    Cc: Dave Hansen <dave.hansen@linux.intel.com>
    Cc: Eric Dumazet <edumazet@google.com>
    Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
    Cc: Hillf Danton <hdanton@sina.com>
    Cc: "H. Peter Anvin" <hpa@zytor.com>
    Cc: Ingo Molnar <mingo@redhat.com>
    Cc: Jann Horn <jannh@google.com>
    Cc: Joern Engel <joern@purestorage.com>
    Cc: Jonathan Corbet <corbet@lwn.net>
    Cc: Kees Cook <keescook@chromium.org>
    Cc: Mark Rutland <mark.rutland@arm.com>
    Cc: Paul E. McKenney <paulmck@kernel.org>
    Cc: Peter Zijlstra <peterz@infradead.org>
    Cc: SeongJae Park <sjpark@amazon.de>
    Cc: Thomas Gleixner <tglx@linutronix.de>
    Cc: Vlastimil Babka <vbabka@suse.cz>
    Cc: Will Deacon <will@kernel.org>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    d3fb45f3
slab_common.c 31.6 KB