- 22 Jan, 2021 15 commits
-
-
Ard Biesheuvel authored
Indirect calls are very expensive on x86, so use a static call to set the system-wide AES-NI/CTR asm helper. Signed-off-by:
Ard Biesheuvel <ardb@kernel.org> Signed-off-by:
Herbert Xu <herbert@gondor.apana.org.au>
-
Ovidiu Panait authored
src_size and aad_size are defined as u32, so the following expressions are currently being evaluated using 32-bit arithmetic: bit_len = src_size * 8; ... bit_len = aad_size * 8; However, bit_len is used afterwards in a context that expects a valid 64-bit value (the lower and upper 32-bit words of bit_len are extracted and written to hw). In order to make sure the correct bit length is generated and the 32-bit multiplication does not wrap around, cast src_size and aad_size to u64. Signed-off-by:
Ovidiu Panait <ovidiu.panait@windriver.com> Acked-by:
Daniele Alessandrelli <daniele.alessandrelli@intel.com> Signed-off-by:
-
Jason A. Donenfeld authored
With no mod_exit function, users are unable to unload the module after use. I'm not aware of any reason why module unloading should be prohibited for this one, so this commit simply adds an empty exit function. Reported-and-tested-by:
John Donnelly <john.p.donnelly@oracle.com> Acked-by:
Jason A. Donenfeld <Jason@zx2c4.com> Signed-off-by:
-
Srujana Challa authored
CPT offload module utilises the linux crypto framework to offload crypto processing. This patch registers supported algorithms by calling registration functions provided by the kernel crypto API. The module currently supports: - AES block cipher in CBC,ECB and XTS mode. - 3DES block cipher in CBC and ECB mode. - AEAD algorithms. authenc(hmac(sha1),cbc(aes)), authenc(hmac(sha256),cbc(aes)), authenc(hmac(sha384),cbc(aes)), authenc(hmac(sha512),cbc(aes)), authenc(hmac(sha1),ecb(cipher_null)), authenc(hmac(sha256),ecb(cipher_null)), authenc(hmac(sha384),ecb(cipher_null)), authenc(hmac(sha512),ecb(cipher_null)), rfc4106(gcm(aes)). Signed-off-by:
Suheil Chandran <schandran@marvell.com> Signed-off-by:
Lukasz Bartosik <lbartosik@marvell.com> Signed-off-by:
Srujana Challa <schalla@marvell.com> Signed-off-by:
-
Srujana Challa authored
Attach LFs to CPT VF to process the crypto requests and register LF interrupts. Signed-off-by:
-
Srujana Challa authored
Add support for the Marvell OcteonTX2 CPT virtual function driver. This patch includes probe, PCI specific initialization and interrupt handling. Signed-off-by:
-
Srujana Challa authored
Adds support to get engine capabilities and adds a new mailbox to share capabilities with VF driver. Signed-off-by:
-
Srujana Challa authored
CPT RVU Local Functions(LFs) needs to be attached to the PF/VF to submit the instructions to CPT. This patch adds the interface to initialize and attach the LFs. It also adds interface to register the LF's interrupts. Signed-off-by:
-
Srujana Challa authored
CPT includes microcoded GigaCypher symmetric engines(SEs), IPsec symmetric engines(IEs), and asymmetric engines (AEs). Each engine receives CPT instructions from the engine groups it has subscribed to. This patch loads microcode, configures three engine groups(one for SEs, one for IEs and one for AEs), and configures all engines. Signed-off-by:
-
Srujana Challa authored
Adds 'sriov_configure' to enable/disable virtual functions (VFs). Also Initializes VF<=>PF mailbox IRQs, register handlers for processing these mailbox messages. Admin function (AF) handles resource allocation and configuration for PFs and their VFs. PFs request the AF directly, via mailboxes. Unlike PFs, VFs cannot send a mailbox request directly. A VF sends mailbox messages to its parent PF, with which it shares a mailbox region. The PF then forwards these messages to the AF. After handling the request, the AF sends a response back to the VF, through the PF. This patch adds support for this 'VF <=> PF <=> AF' mailbox communication. Signed-off-by:
-
Srujana Challa authored
In the resource virtualization unit (RVU) each of the PF and AF (admin function) share a 64KB of reserved memory region for communication. This patch initializes PF <=> AF mailbox IRQs, registers handlers for processing these communication messages. Signed-off-by:
-
Srujana Challa authored
Adds skeleton for the Marvell OcteonTX2 CPT physical function driver which includes probe, PCI specific initialization and hardware register defines. RVU defines are present in AF driver (drivers/net/ethernet/marvell/octeontx2/af), header files from AF driver are included here to avoid duplication. Signed-off-by:
-
Ard Biesheuvel authored
The accelerated, instruction based implementations of SHA1, SHA2 and SHA3 are autoloaded based on CPU capabilities, given that the code is modest in size, and widely used, which means that resolving the algo name, loading all compatible modules and picking the one with the highest priority is taken to be suboptimal. However, if these algorithms are requested before this CPU feature based matching and autoloading occurs, these modules are not even considered, and we end up with suboptimal performance. So add the missing module aliases for the various SHA implementations. Cc: <stable@vger.kernel.org> Signed-off-by:
-
Herbert Xu authored
This patch fixes a number of sparse warnings in the bcm driver. Signed-off-by: -
Ard Biesheuvel authored
Unlike many other structure types defined in the crypto API, the 'shash_desc' structure is permitted to live on the stack, which implies its contents may not be accessed by DMA masters. (This is due to the fact that the stack may be located in the vmalloc area, which requires a different virtual-to-physical translation than the one implemented by the DMA subsystem) Our definition of CRYPTO_MINALIGN_ATTR is based on ARCH_KMALLOC_MINALIGN, which may take DMA constraints into account on architectures that support non-cache coherent DMA such as ARM and arm64. In this case, the value is chosen to reflect the largest cacheline size in the system, in order to ensure that explicit cache maintenance as required by non-coherent DMA masters does not affect adjacent, unrelated slab allocations. On arm64, this value is currently set at 128 bytes. This means that applying CRYPTO_MINALIGN_ATTR to struct shash_desc is both unnecessary (as it is never used for DMA), and undesirable, given that it wastes stack space (on arm64, performing the alignment costs 112 bytes in the worst case, and the hole between the 'tfm' and '__ctx' members takes up another 120 bytes, resulting in an increased stack footprint of up to 232 bytes.) So instead, let's switch to the minimum SLAB alignment, which does not take DMA constraints into account. Note that this is a no-op for x86. Signed-off-by:
-
- 14 Jan, 2021 25 commits
-
-
Daniele Alessandrelli authored
Add the following additional dependencies for CRYPTO_DEV_KEEMBAY_OCS_HCU: - HAS_IOMEM to prevent build failures - ARCH_KEEMBAY to prevent asking the user about this driver when configuring a kernel without Intel Keem Bay platform support. Signed-off-by:
-
Dan Carpenter authored
The first argument to WARN() is a condition and the messages is the second argument is the string, so this WARN() will only display the __func__ part of the message. Fixes: ae832e32 ("crypto: keembay-ocs-hcu - Add HMAC support") Signed-off-by:
Dan Carpenter <dan.carpenter@oracle.com> Acked-by:
-
Ard Biesheuvel authored
The Camellia, Serpent and Twofish related header files only contain declarations that are shared between different implementations of the respective algorithms residing under arch/x86/crypto, and none of their contents should be used elsewhere. So move the header files into the same location, and use local #includes instead. Acked-by:
Eric Biggers <ebiggers@google.com> Signed-off-by:
-
Ard Biesheuvel authored
All dependencies on the x86 glue helper module have been replaced by local instantiations of the new ECB/CBC preprocessor helper macros, so the glue helper module can be retired. Acked-by:
-
Ard Biesheuvel authored
Replace the glue helper dependency with implementations of ECB and CBC based on the new CPP macros, which avoid the need for indirect calls. Acked-by:
-
Ard Biesheuvel authored
Replace the glue helper dependency with implementations of ECB and CBC based on the new CPP macros, which avoid the need for indirect calls. Acked-by:
-
Ard Biesheuvel authored
Replace the glue helper dependency with implementations of ECB and CBC based on the new CPP macros, which avoid the need for indirect calls. Acked-by:
-
Ard Biesheuvel authored
Replace the glue helper dependency with implementations of ECB and CBC based on the new CPP macros, which avoid the need for indirect calls. Acked-by:
-
Ard Biesheuvel authored
Replace the glue helper dependency with implementations of ECB and CBC based on the new CPP macros, which avoid the need for indirect calls. Acked-by:
-
Ard Biesheuvel authored
The x86 glue helper module is starting to show its age: - It relies heavily on function pointers to invoke asm helper functions that operate on fixed input sizes that are relatively small. This means the performance is severely impacted by retpolines. - It goes to great lengths to amortize the cost of kernel_fpu_begin()/end() over as much work as possible, which is no longer necessary now that FPU save/restore is done lazily, and doing so may cause unbounded scheduling blackouts due to the fact that enabling the FPU in kernel mode disables preemption. - The CBC mode decryption helper makes backward strides through the input, in order to avoid a single block size memcpy() between chunks. Consuming the input in this manner is highly likely to defeat any hardware prefetchers, so it is better to go through the data linearly, and perform the extra memcpy() where needed (which is turned into direct loads and stores by the compiler anyway). Note that benchmarks won't show this effect, given that the memory they use is always cache hot. - It implements blockwise XOR in terms of le128 pointers, which imply an alignment that is not guaranteed by the API, violating the C standard. GCC does not seem to be smart enough to elide the indirect calls when the function pointers are passed as arguments to static inline helper routines modeled after the existing ones. So instead, let's create some CPP macros that encapsulate the core of the ECB and CBC processing, so we can wire them up for existing users of the glue helper module, i.e., Camellia, Serpent, Twofish and CAST6. Acked-by:
-
Ard Biesheuvel authored
Blowfish in counter mode is never used in the kernel, so there is no point in keeping an accelerated implementation around. Acked-by:
-
Ard Biesheuvel authored
DES or Triple DES in counter mode is never used in the kernel, so there is no point in keeping an accelerated implementation around. Acked-by:
-
Ard Biesheuvel authored
The glue helper's CTR routines are no longer used, so drop them. Acked-by:
-
Ard Biesheuvel authored
Twofish in CTR mode is never used by the kernel directly, and is highly unlikely to be relied upon by dm-crypt or algif_skcipher. So let's drop the accelerated CTR mode implementation, and instead, rely on the CTR template and the bare cipher. Acked-by:
-
Ard Biesheuvel authored
CAST6 in CTR mode is never used by the kernel directly, and is highly unlikely to be relied upon by dm-crypt or algif_skcipher. So let's drop the accelerated CTR mode implementation, and instead, rely on the CTR template and the bare cipher. Acked-by:
-
Ard Biesheuvel authored
CAST5 in CTR mode is never used by the kernel directly, and is highly unlikely to be relied upon by dm-crypt or algif_skcipher. So let's drop the accelerated CTR mode implementation, and instead, rely on the CTR template and the bare cipher. Acked-by:
-
Ard Biesheuvel authored
Serpent in CTR mode is never used by the kernel directly, and is highly unlikely to be relied upon by dm-crypt or algif_skcipher. So let's drop the accelerated CTR mode implementation, and instead, rely on the CTR template and the bare cipher. Acked-by:
-
Ard Biesheuvel authored
Camellia in CTR mode is never used by the kernel directly, and is highly unlikely to be relied upon by dm-crypt or algif_skcipher. So let's drop the accelerated CTR mode implementation, and instead, rely on the CTR template and the bare cipher. Acked-by:
-
Ard Biesheuvel authored
The glue helper's XTS routines are no longer used, so drop them. Acked-by:
-
Ard Biesheuvel authored
Now that the XTS template can wrap accelerated ECB modes, it can be used to implement Twofish in XTS mode as well, which turns out to be at least as fast, and sometimes even faster Acked-by:
-
Ard Biesheuvel authored
Now that the XTS template can wrap accelerated ECB modes, it can be used to implement Serpent in XTS mode as well, which turns out to be at least as fast, and sometimes even faster Acked-by:
-
Ard Biesheuvel authored
Now that the XTS template can wrap accelerated ECB modes, it can be used to implement CAST6 in XTS mode as well, which turns out to be at least as fast, and sometimes even faster Acked-by:
-
Ard Biesheuvel authored
Now that the XTS template can wrap accelerated ECB modes, it can be used to implement Camellia in XTS mode as well, which turns out to be at least as fast, and sometimes even faster. Acked-by:
-
Bhaskar Chowdhury authored
s/fautly/faulty/p Signed-off-by:
Bhaskar Chowdhury <unixbhaskar@gmail.com> Signed-off-by:
-
Kai Ye authored
Register SEC device to uacce framework for user space. Signed-off-by:
Kai Ye <yekai13@huawei.com> Reviewed-by:
Zhou Wang <wangzhou1@hisilicon.com> Reviewed-by:
Zaibo Xu <xuzaibo@huawei.com> Signed-off-by:
-
