• Linus Walleij's avatar
    ARM: 9016/2: Initialize the mapping of KASan shadow memory · 5615f69b
    Linus Walleij authored
    This patch initializes KASan shadow region's page table and memory.
    There are two stage for KASan initializing:
    
    1. At early boot stage the whole shadow region is mapped to just
       one physical page (kasan_zero_page). It is finished by the function
       kasan_early_init which is called by __mmap_switched(arch/arm/kernel/
       head-common.S)
    
    2. After the calling of paging_init, we use kasan_zero_page as zero
       shadow for some memory that KASan does not need to track, and we
       allocate a new shadow space for the other memory that KASan need to
       track. These issues are finished by the function kasan_init which is
       call by setup_arch.
    
    When using KASan we also need to increase the THREAD_SIZE_ORDER
    from 1 to 2 as the extra calls for shadow memory uses quite a bit
    of stack.
    
    As we need to make a temporary copy of the PGD when setting up
    shadow memory we create a helpful PGD_SIZE definition for both
    LPAE and non-LPAE setups.
    
    The KASan core code unconditionally calls pud_populate() so this
    needs to be changed from BUG() to do {} while (0) when building
    with KASan enabled.
    
    After the initial development by Andre Ryabinin several modifications
    have been made to this code:
    
    Abbott Liu <liuwenliang@huawei.com>
    - Add support ARM LPAE: If LPAE is enabled, KASan shadow region's
      mapping table need be copied in the pgd_alloc() function.
    - Change kasan_pte_populate,kasan_pmd_populate,kasan_pud_populate,
      kasan_pgd_populate from .meminit.text section to .init.text section.
      Reported by Florian Fainelli <f.fainelli@gmail.com>
    
    Linus Walleij <linus.walleij@linaro.org>:
    - Drop the custom mainpulation of TTBR0 and just use
      cpu_switch_mm() to switch the pgd table.
    - Adopt to handle 4th level page tabel folding.
    - Rewrite the entire page directory and page entry initialization
      sequence to be recursive based on ARM64:s kasan_init.c.
    
    Ard Biesheuvel <ardb@kernel.org>:
    - Necessary underlying fixes.
    - Crucial bug fixes to the memory set-up code.
    Co-developed-by: default avatarAndrey Ryabinin <aryabinin@virtuozzo.com>
    Co-developed-by: default avatarAbbott Liu <liuwenliang@huawei.com>
    Co-developed-by: default avatarArd Biesheuvel <ardb@kernel.org>
    
    Cc: Alexander Potapenko <glider@google.com>
    Cc: Dmitry Vyukov <dvyukov@google.com>
    Cc: kasan-dev@googlegroups.com
    Cc: Mike Rapoport <rppt@linux.ibm.com>
    Acked-by: default avatarMike Rapoport <rppt@linux.ibm.com>
    Reviewed-by: default avatarArd Biesheuvel <ardb@kernel.org>
    Tested-by: Ard Biesheuvel <ardb@kernel.org> # QEMU/KVM/mach-virt/LPAE/8G
    Tested-by: Florian Fainelli <f.fainelli@gmail.com> # Brahma SoCs
    Tested-by: Ahmad Fatoum <a.fatoum@pengutronix.de> # i.MX6Q
    Reported-by: default avatarRussell King - ARM Linux <rmk+kernel@armlinux.org.uk>
    Reported-by: default avatarFlorian Fainelli <f.fainelli@gmail.com>
    Signed-off-by: default avatarAndrey Ryabinin <aryabinin@virtuozzo.com>
    Signed-off-by: default avatarAbbott Liu <liuwenliang@huawei.com>
    Signed-off-by: default avatarFlorian Fainelli <f.fainelli@gmail.com>
    Signed-off-by: default avatarArd Biesheuvel <ardb@kernel.org>
    Signed-off-by: default avatarLinus Walleij <linus.walleij@linaro.org>
    Signed-off-by: default avatarRussell King <rmk+kernel@armlinux.org.uk>
    5615f69b
pgd.c 4.51 KB