• Ard Biesheuvel's avatar
    crypto: arm64 - revert NEON yield for fast AEAD implementations · f10dc56c
    Ard Biesheuvel authored
    As it turns out, checking the TIF_NEED_RESCHED flag after each
    iteration results in a significant performance regression (~10%)
    when running fast algorithms (i.e., ones that use special instructions
    and operate in the < 4 cycles per byte range) on in-order cores with
    comparatively slow memory accesses such as the Cortex-A53.
    
    Given the speed of these ciphers, and the fact that the page based
    nature of the AEAD scatterwalk API guarantees that the core NEON
    transform is never invoked with more than a single page's worth of
    input, we can estimate the worst case duration of any resulting
    scheduling blackout: on a 1 GHz Cortex-A53 running with 64k pages,
    processing a page's worth of input at 4 cycles per byte results in
    a delay of ~250 us, which is a reasonable upper bound.
    
    So let's remove the yield checks from the fused AES-CCM and AES-GCM
    routines entirely.
    
    This reverts commit 7b67ae4d and
    partially reverts commit 7c50136a.
    
    Fixes: 7c50136a ("crypto: arm64/aes-ghash - yield NEON after every ...")
    Fixes: 7b67ae4d ("crypto: arm64/aes-ccm - yield NEON after every ...")
    Signed-off-by: default avatarArd Biesheuvel <ard.biesheuvel@linaro.org>
    Acked-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
    Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
    f10dc56c
aes-ce-ccm-core.S 5.94 KB