- 07 Aug, 2018 1 commit
-
-
Ard Biesheuvel authored
As it turns out, checking the TIF_NEED_RESCHED flag after each iteration results in a significant performance regression (~10%) when running fast algorithms (i.e., ones that use special instructions and operate in the < 4 cycles per byte range) on in-order cores with comparatively slow memory accesses such as the Cortex-A53. Given the speed of these ciphers, and the fact that the page based nature of the AEAD scatterwalk API guarantees that the core NEON transform is never invoked with more than a single page's worth of input, we can estimate the worst case duration of any resulting scheduling blackout: on a 1 GHz Cortex-A53 running with 64k pages, processing a page's worth of input at 4 cycles per byte results in a delay of ~250 us, which is a reasonable upper bound. So let's remove the yield checks from the fused AES-CCM and AES-GCM routines entirely. This reverts commit 7b67ae4d and partially reverts commit 7c50136a. Fixes: 7c50136a ("crypto: arm64/aes-ghash - yield NEON after every ...") Fixes: 7b67ae4d ("crypto: arm64/aes-ccm - yield NEON after every ...") Signed-off-by:
Ard Biesheuvel <ard.biesheuvel@linaro.org> Acked-by:
Herbert Xu <herbert@gondor.apana.org.au> Signed-off-by:
-
- 20 Jul, 2018 1 commit
-
-
Herbert Xu authored
This was detected by the self-test thanks to Ard's chunking patch. I finally got around to testing this out on my ancient Via box. It turns out that the workaround got the assembly wrong and we end up doing count + initial cycles of the loop instead of just count. This obviously causes corruption, either by overwriting the source that is yet to be processed, or writing over the end of the buffer. On CPUs that don't require the workaround only ECB is affected. On Nano CPUs both ECB and CBC are affected. This patch fixes it by doing the subtraction prior to the assembly. Fixes: a76c1c23 ("crypto: padlock-aes - work around Nano CPU...") Cc: <stable@vger.kernel.org> Reported-by:
Jamie Heilman <jamie@audible.transient.net> Signed-off-by:
-
- 13 Jul, 2018 1 commit
-
-
Stephan Mueller authored
The RX SGL in processing is already registered with the RX SGL tracking list to support proper cleanup. The cleanup code path uses the sg_num_bytes variable which must therefore be always initialized, even in the error code path. Signed-off-by:
Stephan Mueller <smueller@chronox.de> Reported-by: syzbot+9c251bdd09f83b92ba95@syzkaller.appspotmail.com #syz test: https://github.com/google/kmsan.git master CC: <stable@vger.kernel.org> #4.14 Fixes: e870456d ("crypto: algif_skcipher - overhaul memory management") Fixes: d887c52d ("crypto: algif_aead - overhaul memory management") Signed-off-by:
-
- 01 Jul, 2018 2 commits
-
-
Borislav Petkov authored
Add explicit RETs to the tail calls of AEGIS and MORUS crypto algorithms otherwise they run into INT3 padding due to ("x86/asm: Pad assembly functions with INT3 instructions") leading to spurious debug exceptions. Mike Galbraith <efault@gmx.de> took care of all the remaining callsites. Signed-off-by:Borislav Petkov <bp@suse.de> Acked-by:
Ondrej Mosnacek <omosnacek@gmail.com> Signed-off-by:
-
Eric Biggers authored
Building the kernel with CONFIG_THUMB2_KERNEL=y and CONFIG_CRYPTO_SPECK_NEON set fails with the following errors: arch/arm/crypto/speck-neon-core.S: Assembler messages: arch/arm/crypto/speck-neon-core.S:419: Error: r13 not allowed here -- `bic sp,#0xf' arch/arm/crypto/speck-neon-core.S:423: Error: r13 not allowed here -- `bic sp,#0xf' arch/arm/crypto/speck-neon-core.S:427: Error: r13 not allowed here -- `bic sp,#0xf' arch/arm/crypto/speck-neon-core.S:431: Error: r13 not allowed here -- `bic sp,#0xf' The problem is that the 'bic' instruction can't operate on the 'sp' register in Thumb2 mode. Fix it by using a temporary register. This isn't in the main loop, so the performance difference is negligible. This also matches what aes-neonbs-core.S does. Reported-by:Stefan Agner <stefan@agner.ch> Fixes: ede96221 ("crypto: arm/speck - add NEON-accelerated implementation of Speck-XTS") Signed-off-by:
Eric Biggers <ebiggers@google.com> Acked-by:
-
- 15 Jun, 2018 5 commits
-
-
Michael Büsch authored
enable_best_rng() is used in hwrng_unregister() to switch away from the currently active RNG, if that is the one currently being removed. However enable_best_rng() might fail, if the next RNG's init routine fails. In that case enable_best_rng() will return an error code and the currently active RNG will remain active. After unregistering this might lead to crashes due to use-after-free. Fix this by dropping the currently active RNG, if enable_best_rng() failed. This will result in no RNG to be active, if the next-best one failed to initialize. This problem was introduced by 142a27f0 Fixes: 142a27f0 ("hwrng: core - Reset user selected rng by...") Reported-by:
Wirz <spam@lukas-wirz.de> Tested-by:
Michael Büsch <m@bues.ch> Cc: stable@vger.kernel.org Signed-off-by:
-
Ondrej Mosnáček authored
We must load the block from the temporary variable here, not directly from the input. Also add forgotten zeroing-out of the uninitialized part of the temporary block (as is done correctly in morus1280.c). Fixes: 396be41f ("crypto: morus - Add generic MORUS AEAD implementations") Reported-by: syzbot+1fafa9c4cf42df33f716@syzkaller.appspotmail.com Reported-by: syzbot+d82643ba80bf6937cd44@syzkaller.appspotmail.com Signed-off-by:
Ondrej Mosnacek <omosnace@redhat.com> Signed-off-by:
-
Dmitry Vyukov authored
keccakf() is the only function in kernel that uses __optimize() macro. __optimize() breaks frame pointer unwinder as optimized code uses RBP, and amusingly this always lead to degraded performance as gcc does not inline across different optimizations levels, so keccakf() wasn't inlined into its callers and keccakf_round() wasn't inlined into keccakf(). Drop __optimize() to resolve both problems. Signed-off-by:
Dmitry Vyukov <dvyukov@google.com> Fixes: 83dee2ce ("crypto: sha3-generic - rewrite KECCAK transform to help the compiler optimize") Reported-by: syzbot+37035ccfa9a0a017ffcf@syzkaller.appspotmail.com Reported-by: syzbot+e073e4740cfbb3ae200b@syzkaller.appspotmail.com Cc: linux-crypto@vger.kernel.org Cc: "David S. Miller" <davem@davemloft.net> Cc: Herbert Xu <herbert@gondor.apana.org.au> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org> Acked-by:
-
Jia He authored
In a arm64 server(QDF2400),I met a similar might-sleep warning as [1]: [ 7.019116] BUG: sleeping function called from invalid context at ./include/crypto/algapi.h:416 [ 7.027863] in_atomic(): 1, irqs_disabled(): 0, pid: 410, name: cryptomgr_test [ 7.035106] 1 lock held by cryptomgr_test/410: [ 7.039549] #0: (ptrval) (&drbg->drbg_mutex){+.+.}, at: drbg_instantiate+0x34/0x398 [ 7.048038] CPU: 9 PID: 410 Comm: cryptomgr_test Not tainted 4.17.0-rc6+ #27 [ 7.068228] dump_backtrace+0x0/0x1c0 [ 7.071890] show_stack+0x24/0x30 [ 7.075208] dump_stack+0xb0/0xec [ 7.078523] ___might_sleep+0x160/0x238 [ 7.082360] skcipher_walk_done+0x118/0x2c8 [ 7.086545] ctr_encrypt+0x98/0x130 [ 7.090035] simd_skcipher_encrypt+0x68/0xc0 [ 7.094304] drbg_kcapi_sym_ctr+0xd4/0x1f8 [ 7.098400] drbg_ctr_update+0x98/0x330 [ 7.102236] drbg_seed+0x1b8/0x2f0 [ 7.105637] drbg_instantiate+0x2ac/0x398 [ 7.109646] drbg_kcapi_seed+0xbc/0x188 [ 7.113482] crypto_rng_reset+0x4c/0xb0 [ 7.117319] alg_test_drbg+0xec/0x330 [ 7.120981] alg_test.part.6+0x1c8/0x3c8 [ 7.124903] alg_test+0x58/0xa0 [ 7.128044] cryptomgr_test+0x50/0x58 [ 7.131708] kthread+0x134/0x138 [ 7.134936] ret_from_fork+0x10/0x1c Seems there is a bug in Ard Biesheuvel's commit. Fixes: 68338174 ("crypto: arm64/aes-blk - move kernel mode neon en/disable into loop") [1] https://www.spinics.net/lists/linux-crypto/msg33103.html Signed-off-by: jia.he@hxt-semitech.com Acked-by:Will Deacon <will.deacon@arm.com> Signed-off-by:
-
Dan Carpenter authored
We call chtls_free_skb() but then we dereference it on the next lines. Also "skb" can't be NULL, we just dereferenced it on the line before. I have moved the free down a couple lines to fix this issue. Fixes: 17a7d24a ("crypto: chtls - generic handling of data and hdr") Signed-off-by:
Dan Carpenter <dan.carpenter@oracle.com> Signed-off-by:
-
- 30 May, 2018 26 commits
-
-
Atul Gupta authored
Reported-by:
Atul Gupta <atul.gupta@chelsio.com> Signed-off-by:
-
Atul Gupta authored
- unindented continue - check for null page - signed return Reported-by:
-
Atul Gupta authored
skb dereferenced before check in sendpage Reported-by:
-
Atul Gupta authored
address suspicious code <gustavo@embeddedor.com> 1210 set_bit(SOCK_NOSPACE, &sk->sk_socket->flags); 1211 } The issue is that in the code above, set_bit is never reached due to the 'continue' statement at line 1208. Also reported by bug report:<dan.carpenter@oracle.com> 1210 set_bit(SOCK_NOSPACE, &sk->sk_socket->flags); ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Not reachable. Its required to wait for buffer in the send path and takes care of unaddress and un-handled SOCK_NOSPACE. v2: use csk_mem_free where appropriate proper indent of goto do_nonblock replace out with do_rm_wq Reported-by:Gustavo A. R. Silva <gustavo@embeddedor.com> Reported-by:
-
Atul Gupta authored
corrected the key length to copy 128b key. Removed 192b and 256b key as user input supports key of size 128b in gcm_ctx Reported-by:
-
Eric Biggers authored
This reverts commit eb772f37, as now the x86 Salsa20 implementation has been removed and the generic helpers are no longer needed outside of salsa20_generic.c. We could keep this just in case someone else wants to add a new optimized Salsa20 implementation. But given that we have ChaCha20 now too, I think it's unlikely. And this can always be reverted back. Signed-off-by:
-
Eric Biggers authored
The x86 assembly implementations of Salsa20 use the frame base pointer register (%ebp or %rbp), which breaks frame pointer convention and breaks stack traces when unwinding from an interrupt in the crypto code. Recent (v4.10+) kernels will warn about this, e.g. WARNING: kernel stack regs at 00000000a8291e69 in syzkaller047086:4677 has bad 'bp' value 000000001077994c [...] But after looking into it, I believe there's very little reason to still retain the x86 Salsa20 code. First, these are *not* vectorized (SSE2/SSSE3/AVX2) implementations, which would be needed to get anywhere close to the best Salsa20 performance on any remotely modern x86 processor; they're just regular x86 assembly. Second, it's still unclear that anyone is actually using the kernel's Salsa20 at all, especially given that now ChaCha20 is supported too, and with much more efficient SSSE3 and AVX2 implementations. Finally, in benchmarks I did on both Intel and AMD processors with both gcc 8.1.0 and gcc 4.9.4, the x86_64 salsa20-asm is actually slightly *slower* than salsa20-generic (~3% slower on Skylake, ~10% slower on Zen), while the i686 salsa20-asm is only slightly faster than salsa20-generic (~15% faster on Skylake, ~20% faster on Zen). The gcc version made little difference. So, the x86_64 salsa20-asm is pretty clearly useless. That leaves just the i686 salsa20-asm, which based on my tests provides a 15-20% speed boost. But that's without updating the code to not use %ebp. And given the maintenance cost, the small speed difference vs. salsa20-generic, the fact that few people still use i686 kernels, the doubt that anyone is even using the kernel's Salsa20 at all, and the fact that a SSE2 implementation would almost certainly be much faster on any remotely modern x86 processor yet no one has cared enough to add one yet, I don't think it's worthwhile to keep. Thus, just remove both the x86_64 and i686 salsa20-asm implementations. Reported-by: syzbot+ffa3a158337bbc01ff09@syzkaller.appspotmail.com Signed-off-by:
-
Janakarajan Natarajan authored
The GET_ID command, added as of SEV API v0.16, allows the SEV firmware to be queried about a unique CPU ID. This unique ID can then be used to obtain the public certificate containing the Chip Endorsement Key (CEK) public key signed by the AMD SEV Signing Key (ASK). For more information please refer to "Section 5.12 GET_ID" of https://support.amd.com/TechDocs/55766_SEV-KM%20API_Specification.pdfSigned-off-by:
Janakarajan Natarajan <Janakarajan.Natarajan@amd.com> Signed-off-by:
-
Janakarajan Natarajan authored
The DOWNLOAD_FIRMWARE command, added as of SEV API v0.15, allows the OS to install SEV firmware newer than the currently active SEV firmware. For the new SEV firmware to be applied it must: * Pass the validation test performed by the existing firmware. * Be of the same build or a newer build compared to the existing firmware. For more information please refer to "Section 5.11 DOWNLOAD_FIRMWARE" of https://support.amd.com/TechDocs/55766_SEV-KM%20API_Specification.pdfSigned-off-by:
-
Conor McLoughlin authored
Signed-off-by:
Conor McLoughlin <conor.mcloughlin@intel.com> Signed-off-by:
-
Gilad Ben-Yossef authored
The cache parameter register configuration was being too verbose. Use dev_dbg() to only provide the information if needed. Signed-off-by:
Gilad Ben-Yossef <gilad@benyossef.com> Reviewed-by:
Simon Horman <horms+renesas@verge.net.au> Signed-off-by:
-
Gilad Ben-Yossef authored
Use managed clock handling, differentiate between no clock (possibly OK) and clock init failure (never OK) and correctly handle clock detection being deferred. Suggested-by:
Geert Uytterhoeven <geert@linux-m68k.org> Signed-off-by:
-
Gilad Ben-Yossef authored
The product signature and HW revision register have different offset on the older HW revisions. This fixes the problem of the driver failing sanity check on silicon despite working on the FPGA emulation systems. Fixes: 27b3b22d ("crypto: ccree - add support for older HW revs") Cc: stable@vger.kernel.org Signed-off-by:
-
Harsh Jain authored
Extends memory required for IV to include B0 Block and DMA map in single operation. Signed-off-by:
Harsh Jain <harsh@chelsio.com> Signed-off-by:
-
Harsh Jain authored
Send IV in WR as immediate instead of dma mapped entry for cipher. Signed-off-by:
-
Harsh Jain authored
Change the return type based on following patch https://www.mail-archive.com/linux-crypto@vger.kernel.org/msg28552.htmlSigned-off-by:
-
Horia Geantă authored
Coverity warns about an "Unintentional integer overflow (OVERFLOW_BEFORE_WIDEN)" when computing the congestion threshold value. Even though it is highly unlikely for an overflow to happen, use this as an opportunity to simplify the code. Signed-off-by:
Horia Geantă <horia.geanta@nxp.com> Signed-off-by:
-
Horia Geantă authored
In some cases the CCB DMA-based internal transfer started by the MOVE command (src=M3 register, dst=descriptor buffer) does not finish in time and DECO executes the unpatched descriptor. This leads eventually to a DECO Watchdog Timer timeout error. To make sure the transfer ends, change the MOVE command to be blocking. Signed-off-by:
-
Horia Geantă authored
Management Complex (MC) f/w detection is based on CTPR_MS[DPAA2] bit. This is incorrect since: -the bit is set for all CAAM blocks integrated in SoCs with a certain Layerscape Chassis -some SoCs with LS Chassis don't have an MC block (thus no MC f/w) To fix this, MC f/w detection will be based on the presence of "fsl,qoriq-mc" compatible string in the device tree. Fixes: 297b9ceb ("crypto: caam/jr - add support for DPAA2 parts") Signed-off-by:
-
Adam Langley authored
Several source files have been taken from OpenSSL. In some of them a comment that "permission to use under GPL terms is granted" was included below a contradictory license statement. In several cases, there was no indication that the license of the code was compatible with the GPLv2. This change clarifies the licensing for all of these files. I've confirmed with the author (Andy Polyakov) that a) he has licensed the files with the GPLv2 comment under that license and b) that he's also happy to license the other files under GPLv2 too. In one case, the file is already contained in his CRYPTOGAMS bundle, which has a GPLv2 option, and so no special measures are needed. In all cases, the license status of code has been clarified by making the GPLv2 license prominent. The .S files have been regenerated from the updated .pl files. This is a comment-only change. No code is changed. Signed-off-by:
Adam Langley <agl@chromium.org> Signed-off-by:
-
Ondrej Mosnacek authored
Commit 56e8e57f ("crypto: morus - Add common SIMD glue code for MORUS") accidetally consiedered the glue code to be usable by different architectures, but it seems to be only usable on x86. This patch moves it under arch/x86/crypto and adds 'depends on X86' to the Kconfig options and also removes the prompt to hide these internal options from the user. Reported-by:
kbuild test robot <lkp@intel.com> Signed-off-by:
-
Eric Biggers authored
Currently testmgr has separate encryption and decryption test vectors for symmetric ciphers. That's massively redundant, since with few exceptions (mostly mistakes, apparently), all decryption tests are identical to the encryption tests, just with the input/result flipped. Therefore, eliminate the redundancy by removing the decryption test vectors and updating testmgr to test both encryption and decryption using what used to be the encryption test vectors. Naming is adjusted accordingly: each cipher_testvec now has a 'ptext' (plaintext), 'ctext' (ciphertext), and 'len' instead of an 'input', 'result', 'ilen', and 'rlen'. Note that it was always the case that 'ilen == rlen'. AES keywrap ("kw(aes)") is special because its IV is generated by the encryption. Previously this was handled by specifying 'iv_out' for encryption and 'iv' for decryption. To make it work cleanly with only one set of test vectors, put the IV in 'iv', remove 'iv_out', and add a boolean that indicates that the IV is generated by the encryption. In total, this removes over 10000 lines from testmgr.h, with no reduction in test coverage since prior patches already copied the few unique decryption test vectors into the encryption test vectors. This covers all algorithms that used 'struct cipher_testvec', e.g. any block cipher in the ECB, CBC, CTR, XTS, LRW, CTS-CBC, PCBC, OFB, or keywrap modes, and Salsa20 and ChaCha20. No change is made to AEAD tests, though we probably can eliminate a similar redundancy there too. The testmgr.h portion of this patch was automatically generated using the following awk script, with some slight manual fixups on top (updated 'struct cipher_testvec' definition, updated a few comments, and fixed up the AES keywrap test vectors): BEGIN { OTHER = 0; ENCVEC = 1; DECVEC = 2; DECVEC_TAIL = 3; mode = OTHER } /^static const struct cipher_testvec.*_enc_/ { sub("_enc", ""); mode = ENCVEC } /^static const struct cipher_testvec.*_dec_/ { mode = DECVEC } mode == ENCVEC && !/\.ilen[[:space:]]*=/ { sub(/\.input[[:space:]]*=$/, ".ptext =") sub(/\.input[[:space:]]*=/, ".ptext\t=") sub(/\.result[[:space:]]*=$/, ".ctext =") sub(/\.result[[:space:]]*=/, ".ctext\t=") sub(/\.rlen[[:space:]]*=/, ".len\t=") print } mode == DECVEC_TAIL && /[^[:space:]]/ { mode = OTHER } mode == OTHER { print } mode == ENCVEC && /^};/ { mode = OTHER } mode == DECVEC && /^};/ { mode = DECVEC_TAIL } Note that git's default diff algorithm gets confused by the testmgr.h portion of this patch, and reports too many lines added and removed. It's better viewed with 'git diff --minimal' (or 'git show --minimal'), which reports "2 files changed, 919 insertions(+), 11723 deletions(-)". Signed-off-by: -
Eric Biggers authored
One "kw(aes)" decryption test vector doesn't exactly match an encryption test vector with input and result swapped. In preparation for removing the decryption test vectors, add this test vector to the encryption test vectors, so we don't lose any test coverage. Signed-off-by:
-
Eric Biggers authored
None of the four "ecb(tnepres)" decryption test vectors exactly match an encryption test vector with input and result swapped. In preparation for removing the decryption test vectors, add these to the encryption test vectors, so we don't lose any test coverage. Signed-off-by:
-
Eric Biggers authored
One "cbc(des)" decryption test vector doesn't exactly match an encryption test vector with input and result swapped. It's *almost* the same as one, but the decryption version is "chunked" while the encryption version is "unchunked". In preparation for removing the decryption test vectors, make the encryption one both chunked and unchunked, so we don't lose any test coverage. Signed-off-by:
-
Eric Biggers authored
Two "ecb(des)" decryption test vectors don't exactly match any of the encryption test vectors with input and result swapped. In preparation for removing the decryption test vectors, add these to the encryption test vectors, so we don't lose any test coverage. Signed-off-by:
-
- 26 May, 2018 4 commits
-
-
Ondrej Mosnacek authored
AEGIS-256 key is two blocks, not one. Fixes: 1d373d4e ("crypto: x86 - Add optimized AEGIS implementations") Reported-by:
Eric Biggers <ebiggers3@gmail.com> Signed-off-by:
-
Eric Biggers authored
crc32c has an unkeyed test vector but crc32 did not. Add the crc32c one (which uses an empty input) to crc32 too, and also add a new one to both that uses a nonempty input. These test vectors verify that crc32 and crc32c implementations use the correct default initial state. Signed-off-by:
-
Eric Biggers authored
Since testmgr uses a single tfm for all tests of each hash algorithm, once a key is set the tfm won't be unkeyed anymore. But with crc32 and crc32c, the key is really the "default initial state" and is optional; those algorithms should have both keyed and unkeyed test vectors, to verify that implementations use the correct default key. Simply listing the unkeyed test vectors first isn't guaranteed to work yet because testmgr makes multiple passes through the test vectors. crc32c does have an unkeyed test vector listed first currently, but it only works by chance because the last crc32c test vector happens to use a key that is the same as the default key. Therefore, teach testmgr to split hash test vectors into unkeyed and keyed sections, and do all the unkeyed ones before the keyed ones. Signed-off-by:
-
Eric Biggers authored
The Blackfin CRC driver was removed by commit 9678a8dc ("crypto: bfin_crc - remove blackfin CRC driver"), but it was forgotten to remove the corresponding "hmac(crc32)" test vectors. I see no point in keeping them since nothing else appears to implement or use "hmac(crc32)", which isn't an algorithm that makes sense anyway because HMAC is meant to be used with a cryptographically secure hash function, which CRC's are not. Thus, remove the unneeded test vectors. Signed-off-by:
-
