• Daniel Sanders's avatar
    slab: correct size_index table before replacing the bootstrap kmem_cache_node · 34cc6990
    Daniel Sanders authored
    This patch moves the initialization of the size_index table slightly
    earlier so that the first few kmem_cache_node's can be safely allocated
    when KMALLOC_MIN_SIZE is large.
    
    There are currently two ways to generate indices into kmalloc_caches (via
    kmalloc_index() and via the size_index table in slab_common.c) and on some
    arches (possibly only MIPS) they potentially disagree with each other
    until create_kmalloc_caches() has been called.  It seems that the
    intention is that the size_index table is a fast equivalent to
    kmalloc_index() and that create_kmalloc_caches() patches the table to
    return the correct value for the cases where kmalloc_index()'s
    if-statements apply.
    
    The failing sequence was:
    * kmalloc_caches contains NULL elements
    * kmem_cache_init initialises the element that 'struct
      kmem_cache_node' will be allocated to. For 32-bit Mips, this is a
      56-byte struct and kmalloc_index returns KMALLOC_SHIFT_LOW (7).
    * init_list is called which calls kmalloc_node to allocate a 'struct
      kmem_cache_node'.
    * kmalloc_slab selects the kmem_caches element using
      size_index[size_index_elem(size)]. For MIPS, size is 56, and the
      expression returns 6.
    * This element of kmalloc_caches is NULL and allocation fails.
    * If it had not already failed, it would have called
      create_kmalloc_caches() at this point which would have changed
      size_index[size_index_elem(size)] to 7.
    
    I don't believe the bug to be LLVM specific but GCC doesn't normally
    encounter the problem.  I haven't been able to identify exactly what GCC
    is doing better (probably inlining) but it seems that GCC is managing to
    optimize to the point that it eliminates the problematic allocations.
    This theory is supported by the fact that GCC can be made to fail in the
    same way by changing inline, __inline, __inline__, and __always_inline in
    include/linux/compiler-gcc.h such that they don't actually inline things.
    Signed-off-by: default avatarDaniel Sanders <daniel.sanders@imgtec.com>
    Acked-by: default avatarPekka Enberg <penberg@kernel.org>
    Acked-by: default avatarChristoph Lameter <cl@linux.com>
    Cc: Pekka Enberg <penberg@kernel.org>
    Cc: David Rientjes <rientjes@google.com>
    Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    34cc6990
slab.h 10.5 KB