When many threads submit multiple requests they get blocked until all
responses are processed, which prevents them from submitting more requests
even though there is space on the rings.
To fix this we need to decrement the inflight counter early to in the callback.
Signed-off-by: Harvijay Saini <harvijayx.saini@intel.com>
Signed-off-by: Tadeusz Struk <tadeusz.struk@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Fix numerous spelling error in include/crypto/akcipher.h
Signed-off-by: LABBE Corentin <clabbe.montjoie@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

The sahara_dump_descriptors and sahara_dump_links functions attempt
to print a dma_addr_t value with a 0x%08x format string, which
produces a warning when dma_addr_t is 64-bit wide:

drivers/crypto/sahara.c:419:120: warning: format '%x' expects argument of type 'unsigned int', but argument 5 has type 'dma_addr_t {aka long long unsigned int}' [-Wformat=]

This changes the code to use the %pad format string that is meant
for dma_addr_t, which avoids the warning and gives us the correct
output in all configurations.
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

The sahara hardware uses DMA descriptors with 32-bit addresses, but
dma_addr_t is variable size depending on whether we want to support
any devices that use 64-bit DMA addresses in hardware.
This means that the definition of the DMA descriptor structure is wrong,
and we helpfully get a compiler warning about them too:

drivers/crypto/sahara.c:423:372: warning: format '%x' expects argument of type 'unsigned int', but argument 4 has type 'dma_addr_t {aka long long unsigned int}' [-Wformat=]

This changes the definition of the sahara_hw_desc and sahara_hw_link
structures to only contain fixed-length members, which is required
to make the driver work on ARM LPAE mode, and avoids most of the
gcc warnings we get.
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

The drbg_state_ops structures are never modified, so declare them as const.

Done with the help of Coccinelle.
Signed-off-by: Julia Lawall <Julia.Lawall@lip6.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

If the length of the plaintext is zero, there's no need to waste cycles
on encryption and decryption. Using the chacha20poly1305 construction
for zero-length plaintexts is a common way of using a shared encryption
key for AAD authentication.
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

This patch adds PKCS#1 v1.5 standard RSA padding as a separate template.
This way an RSA cipher with padding can be obtained by instantiating
"pkcs1pad(rsa)".  The reason for adding this is that RSA is almost
never used without this padding (or OAEP) so it will be needed for
either certificate work in the kernel or the userspace, and I also hear
that it is likely implemented by hardware RSA in which case hardware
implementations of the whole of pkcs1pad(rsa) can be provided.
Signed-off-by: Andrew Zaborowski <andrew.zaborowski@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Add a struct akcipher_instance and struct akcipher_spawn similar to
how AEAD declares them and the macros for converting to/from
crypto_instance/crypto_spawn.  Also add register functions to
avoid exposing crypto_akcipher_type.
Signed-off-by: Andrew Zaborowski <andrew.zaborowski@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Add support for c62x accelerator Virtual Function
Signed-off-by: Tadeusz Struk <tadeusz.struk@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Add support for c3xxx accelerator Virtual Function
Signed-off-by: Tadeusz Struk <tadeusz.struk@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Add support for qat c62x accel type
Signed-off-by: Tadeusz Struk <tadeusz.struk@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Add support for c3xxx accel type.
Signed-off-by: Tadeusz Struk <tadeusz.struk@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Move qat_isr.c and qat_isrvf.c files to qat_common dir
so that they can be reused by all devices.
Remove adf_drv.h files because thay are not longer needed.
Move adf_dev_configure() function to qat_common so it can be reused.
Also some minor updates to common code for multidevice.
Signed-off-by: Tadeusz Struk <tadeusz.struk@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

FW loader updates for new qat devices
Signed-off-by: Tadeusz Struk <tadeusz.struk@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Add dev ids and names for the new device types.
Signed-off-by: Tadeusz Struk <tadeusz.struk@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Fix 21 occurrences of this kernel-doc warning in <crypto/aead.h>:

..//include/crypto/aead.h:149: warning: No description found for parameter 'base'
Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

These new helpers simplify implementing multi-driver modules and
properly handle failure to register one driver by unregistering all
previously registered drivers.
Signed-off-by: Thierry Reding <treding@nvidia.com>
Acked-by: David S. Miller <davem@davemloft.net>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

This patch adds the following algorithms to the talitos driver:
* ecb(aes)
* ctr(aes)
* ecb(des)
* cbc(des)
* ecb(des3_ede)
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

IV size was zero on CBC and CTR modes,
causing a bug triggered by skcipher.

Fixing this  adding a correct size.
Signed-off-by: Leonidas Da Silva Barbosa <leosilva@linux.vnet.ibm.com>
Signed-off-by: Paulo Smorigo <pfsmorigo@linux.vnet.ibm.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

When buffer 0 is used we should use buflen_0 instead of buflen_1.

Fix it.
Signed-off-by: Fabio Estevam <fabio.estevam@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

cryptd_create_hash() fails by returning -EINVAL.  It is because after
8996eafd ("crypto: ahash - ensure statesize is non-zero") all ahash
drivers must have a non-zero statesize.

This patch fixes the problem by properly assigning the statesize.
Signed-off-by: Rui Wang <rui.y.wang@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

ghash_clmulni_intel fails to load on Linux 4.3+ with the following message:
"modprobe: ERROR: could not insert 'ghash_clmulni_intel': Invalid argument"

After 8996eafd ("crypto: ahash - ensure statesize is non-zero") all ahash
drivers are required to implement import()/export(), and must have a non-
zero statesize.

This patch has been tested with the algif_hash interface. The calculated
digest values, after several rounds of import()s and export()s, match those
calculated by tcrypt.
Signed-off-by: Rui Wang <rui.y.wang@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Lockdep warns about a possible deadlock resulting from the use of regular
spin_locks:

=================================
[ INFO: inconsistent lock state ]
4.4.0-rc2+ #2724 Not tainted
---------------------------------
inconsistent {SOFTIRQ-ON-W} -> {IN-SOFTIRQ-W} usage.
ksoftirqd/0/3 [HC0[0]:SC1[1]:HE1:SE0] takes:
(&(&crypto_info->lock)->rlock){+.?...}, at: [<bf14a65c>] rk_crypto_tasklet_cb+0x24/0xb4 [rk_crypto]
{SOFTIRQ-ON-W} state was registered at:
  [<c007f4ac>] lock_acquire+0x178/0x218
  [<c0759bac>] _raw_spin_lock+0x54/0x64
  [<bf14af88>] rk_handle_req+0x7c/0xbc [rk_crypto]
  [<bf14b040>] rk_des_ecb_encrypt+0x2c/0x30 [rk_crypto]
  [<bf14b05c>] rk_aes_ecb_encrypt+0x18/0x1c [rk_crypto]
  [<c028c820>] skcipher_encrypt_ablkcipher+0x64/0x68
  [<c0290770>] __test_skcipher+0x2a8/0x8dc
  [<c0292e94>] test_skcipher+0x38/0xc4
  [<c0292fb0>] alg_test_skcipher+0x90/0xb0
  [<c0292158>] alg_test+0x1e8/0x280
  [<c028f6f4>] cryptomgr_test+0x34/0x54
  [<c004bbe8>] kthread+0xf4/0x10c
  [<c0010010>] ret_from_fork+0x14/0x24
irq event stamp: 10672
hardirqs last  enabled at (10672): [<c002fac8>] tasklet_action+0x48/0x104
hardirqs last disabled at (10671): [<c002faa0>] tasklet_action+0x20/0x104
softirqs last  enabled at (10658): [<c002ef84>] __do_softirq+0x358/0x49c
softirqs last disabled at (10669): [<c002f108>] run_ksoftirqd+0x40/0x80

other info that might help us debug this:
Possible unsafe locking scenario:

    CPU0
    ----
  lock(&(&crypto_info->lock)->rlock);
  <Interrupt>
    lock(&(&crypto_info->lock)->rlock);

 *** DEADLOCK ***

Fix this by moving to irq-disabling spinlocks.
Signed-off-by: Heiko Stuebner <heiko@sntech.de>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

hwrng kthread can be waiting via hwrng_fillfn for some data from a rng
like virtio-rng:
hwrng           D ffff880093e17798     0   382      2 0x00000000
...
Call Trace:
 [<ffffffff817339c6>] wait_for_completion_killable+0x96/0x210
 [<ffffffffa00aa1b7>] virtio_read+0x57/0xf0 [virtio_rng]
 [<ffffffff814f4a35>] hwrng_fillfn+0x75/0x130
 [<ffffffff810aa243>] kthread+0xf3/0x110

And when some user program tries to read the /dev node in this state,
we get:
rngd            D ffff880093e17798     0   762      1 0x00000004
...
Call Trace:
 [<ffffffff817351ac>] mutex_lock_nested+0x15c/0x3e0
 [<ffffffff814f478e>] rng_dev_read+0x6e/0x240
 [<ffffffff81231958>] __vfs_read+0x28/0xe0
 [<ffffffff81232393>] vfs_read+0x83/0x130

And this is indeed unkillable. So use mutex_lock_interruptible
instead of mutex_lock in rng_dev_read and exit immediatelly when
interrupted. And possibly return already read data, if any (as POSIX
allows).

v2: use ERESTARTSYS instead of EINTR
Signed-off-by: Jiri Slaby <jslaby@suse.cz>
Cc: Matt Mackall <mpm@selenic.com>
Cc: Herbert Xu <herbert@gondor.apana.org.au>
Cc: <linux-crypto@vger.kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

The <linux/crypto.h> provides inline function - crypto_skcipher_type().
Let's use it in the cryptd_alloc_ablkcipher() instead of direct calculation.
Signed-off-by: Alexander Kuleshov <kuleshovmail@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Crypto driver support:
     ecb(aes) cbc(aes) ecb(des) cbc(des) ecb(des3_ede) cbc(des3_ede)
You can alloc tags above in your case.

And other algorithms and platforms will be added later on.
Signed-off-by: Zain Wang <zain.wang@rock-chips.com>
Tested-by: Heiko Stuebner <heiko@sntech.de>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Add DT bindings documentation for the rk3288 crypto drivers.
Signed-off-by: Zain Wang <zain.wang@rock-chips.com>
Acked-by: Rob Herring <robh@kernel.org>
Tested-by: Heiko Stuebner <heiko@sntech.de>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

use of_propert_read_u32() for reading int value,
it can help reducing number of variables used
Signed-off-by: Saurabh Sengar <saurabh.truth@gmail.com>
Acked-by: Dan Streetman <ddstreet@ieee.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Use offset_in_page macro instead of (addr & ~PAGE_MASK).
Signed-off-by: Geliang Tang <geliangtang@163.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

A typo in the Makefile leaves qat_rsaprivkey-asn1.h hanging around.
Signed-off-by: Jim Davis <jim.epost@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

The unsigned int variables [src|dst]_nents and nents can be assigned
signed value (-EINVAL) from sg_nents_for_len().
Furthermore they are used only by dma_map_sg and dma_unmap_sg which wait
for an signed int, so they must be set as int.

Fixes: f051f95e ("crypto: picoxcell - check return value of sg_nents_for_len")
Reported-by: Dan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: LABBE Corentin <clabbe.montjoie@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

The two unsigned int variables nb_in_sg and nb_out_sg can be assigned
signed value (-EINVAL) from sg_nents_for_len().
Furthermore they are used only by dma_map_sg and dma_unmap_sg which wait
for an signed int, so they must be set as int.

Fixes: 6c2b74d4 ("crypto: sahara - check return value of sg_nents_for_len")
Reported-by: Dan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: LABBE Corentin <clabbe.montjoie@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

We cannot put the HW RNG to idle using a timer because we cannot disable
clocks from atomic context. Use a delayed work instead.

Fixes a warning with CONFIG_DEBUG_MUTEXES on Nokia N900 during boot.
Reported-by: Sebastian Reichel <sre@kernel.org>
Signed-off-by: Aaro Koskinen <aaro.koskinen@iki.fi>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

The Atmel at91 crypto driver unconditionally selects AT_HDMAC,
which results in a Kconfig warning if that driver is not enabled:

warning: (CRYPTO_DEV_ATMEL_AES) selects AT_HDMAC which has unmet direct dependencies (DMADEVICES && ARCH_AT91)

The crypto driver itself does not actually have a dependency
on a particular dma engine, other than this being the one that
is used in at91.

Removing the 'select' gets rid of the warning, but can cause
the driver to be unusable if the HDMAC is not enabled at the
same time. To work around that, this patch clarifies the runtime
dependency to be 'AT_HDMAC || AT_XDMAC', but adds an alternative
for COMPILE_TEST, which lets the driver get build on all systems.

The ARCH_AT91 dependency is implied by AT_XDMAC || AT_HDMAC now
and no longer needs to be listed separately.
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

The key sizes used by AES in GCM mode should be 128, 192 or 256 bits (16,
24 or 32 bytes).
There is no additional 4byte nonce as for RFC 4106.
Signed-off-by: Cyrille Pitchen <cyrille.pitchen@atmel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

The atmel AES driver assumes that 'int' and 'size_t' are the same
type in multiple locations, which the compiler warns about when
building it for 64-bit systems:

In file included from ../drivers/crypto/atmel-aes.c:17:0:
drivers/crypto/atmel-aes.c: In function 'atmel_aes_sg_copy':
include/linux/kernel.h:724:17: warning: comparison of distinct pointer types lacks a cast
drivers/crypto/atmel-aes.c:448:11: note: in expansion of macro 'min'

drivers/crypto/atmel-aes.c: In function 'atmel_aes_crypt_dma_stop':
include/linux/kern_levels.h:4:18: warning: format '%u' expects argument of type 'unsigned int', but argument 2 has type 'size_t {aka long unsigned int}' [-Wformat=]

This changes the format strings to use the %z modifier when printing
a size_t, and makes sure that we use the correct size_t type where
needed. In case of sg_dma_len(), the type of the result depends
on CONFIG_NEED_SG_DMA_LENGTH, so we have to use min_t to get it to
work in all configurations.
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Acked-by: Cyrille Pitchen <cyrille.pitchen@atmel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Simplify mcryptd_opportunistic_flush() with list_first_entry_or_null().
Signed-off-by: Geliang Tang <geliangtang@163.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Simplify crypto_more_spawns() with list_first_entry_or_null()
and list_next_entry().
Signed-off-by: Geliang Tang <geliangtang@163.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

sun4i-ss implementaton of md5/sha1 is via ahash algorithms.
Commit 8996eafd ("crypto: ahash - ensure statesize is non-zero")
made impossible to load them without giving statesize. This patch
specifiy statesize for sha1 and md5.

Fixes: 6298e948 ("crypto: sunxi-ss - Add Allwinner Security System crypto accelerator")
Cc: <stable@vger.kernel.org> # v4.3+
Tested-by: Chen-Yu Tsai <wens@csie.org>
Signed-off-by: LABBE Corentin <clabbe.montjoie@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

The dma_pool_destroy() function tests whether its argument is NULL
and then returns immediately. Thus the test around the calls is not needed.

This issue was detected by using the Coccinelle software.
Signed-off-by: Markus Elfring <elfring@users.sourceforge.net>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>