We rename aesni-intel_avx.S to aesni-intel_avx-x86_64.S to indicate
that it is only used by x86_64 architecture.
Signed-off-by: Tim Chen <tim.c.chen@linux.intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Fixes the following sparse warning:

drivers/crypto/mxs-dcp.c:103:1: warning:
 symbol 'global_mutex' was not declared. Should it be static?
Signed-off-by: Wei Yongjun <yongjun_wei@trendmicro.com.cn>
Acked-by: Marek Vasut <marex@denx.de>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

The CCP cannot be hot-plugged so it will either be there
or it won't.  Do not allow the driver to stay loaded if the
CCP does not successfully initialize.

Provide stub routines in the ccp.h file that return -ENODEV
if the CCP has not been configured in the build.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Cleanup the ahash digest invocations to check the init
return code and make use of the finup routine.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

When performing a hash operation if the amount of data buffered and a
request at or near the maximum data length is received then the length
calcuation could wrap causing an error in executing the hash operation.
Fix this by using a u64 type for the input and output data lengths in
all CCP operations.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

For a hash operation, the caller doesn't have to supply a result
area on every call so don't use it / update it if it hasn't
been supplied.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Cleanup up the usage of scatterlists to make the code cleaner
and avoid extra memory allocations when not needed.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Fix some memory allocations to use the appropriate gfp_t type based
on the CRYPTO_TFM_REQ_MAY_SLEEP flag.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

The order in the Makefile was a mess, sort it.
Signed-off-by: Marek Vasut <marex@denx.de>
Cc: David S. Miller <davem@davemloft.net>
Cc: Fabio Estevam <fabio.estevam@freescale.com>
Cc: Shawn Guo <shawn.guo@linaro.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Enable the DCP by default on both i.MX23 and i.MX28.
Signed-off-by: Marek Vasut <marex@denx.de>
Cc: David S. Miller <davem@davemloft.net>
Cc: Fabio Estevam <fabio.estevam@freescale.com>
Cc: Shawn Guo <shawn.guo@linaro.org>
Cc: devicetree@vger.kernel.org
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Add support for the MXS DCP block. The driver currently supports
SHA-1/SHA-256 hashing and AES-128 CBC/ECB modes. The non-standard
CRC32 is not yet supported.
Signed-off-by: Marek Vasut <marex@denx.de>
Cc: David S. Miller <davem@davemloft.net>
Cc: Fabio Estevam <fabio.estevam@freescale.com>
Cc: Shawn Guo <shawn.guo@linaro.org>
Cc: devicetree@vger.kernel.org
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Remove the old DCP driver as it had multiple severe issues. The driver
will be replaced by a more robust implementation. Here is a short list
of problems with this driver:

1) It only supports AES_CBC
2) The driver was apparently never ran behind anyone working with MXS. ie.:
   -> Restarting the DCP block is not done via mxs_reset_block()
   -> The DT name is not "fsl,dcp" or "fsl,mxs-dcp" as other MXS drivers
3) Introduces new ad-hoc IOCTLs
4) The IRQ handler can't use usual completion() in the driver because that'd
   trigger "scheduling while atomic" oops, yes?
Signed-off-by: Marek Vasut <marex@denx.de>
Cc: David S. Miller <davem@davemloft.net>
Cc: Fabio Estevam <fabio.estevam@freescale.com>
Cc: Shawn Guo <shawn.guo@linaro.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

When finishing the ahash request, the ahash_op_unaligned_done() will
call complete() on the request. Yet, this will not call the correct
complete callback. The correct complete callback was previously stored
in the requests' private data, as seen in ahash_op_unaligned(). This
patch restores the correct complete callback and .data field of the
request before calling complete() on it.
Signed-off-by: Marek Vasut <marex@denx.de>
Cc: David S. Miller <davem@davemloft.net>
Cc: Fabio Estevam <fabio.estevam@freescale.com>
Cc: Shawn Guo <shawn.guo@linaro.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

It seems commit d764593a "crypto: aesni - AVX and AVX2 version of AESNI-GCM
encode and decode" breaks a build on x86_32 since it's designed only for
x86_64. This patch makes a compilation unit conditional to CONFIG_64BIT and
functions usage to CONFIG_X86_64.
Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Driver core sets it to NULL upon probe failure or release.
Signed-off-by: Sachin Kamat <sachin.kamat@linaro.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Driver core sets it to NULL upon probe failure or release.
Signed-off-by: Sachin Kamat <sachin.kamat@linaro.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Driver core sets it to NULL upon probe failure or release.
Signed-off-by: Sachin Kamat <sachin.kamat@linaro.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Use the common helper function crypto_authenc_extractkeys() for key
parsing.
Also fix the key buffer overflow condition: use split key pad length
instead of authentication key length.
Signed-off-by: Horia Geanta <horia.geanta@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Command "tcrypt sec=1 mode=403" give the follwoing error for Polling
mode:
root@am335x-evm:/# insmod tcrypt.ko sec=1 mode=403
[...]

[  346.982754] test 15 ( 4096 byte blocks, 1024 bytes per update,   4 updates):   4352 opers/sec,  17825792 bytes/sec
[  347.992661] test 16 ( 4096 byte blocks, 4096 bytes per update,   1 updates):   7095 opers/sec,  29061120 bytes/sec
[  349.002667] test 17 ( 8192 byte blocks,   16 bytes per update, 512 updates):
[  349.010882] Unable to handle kernel NULL pointer dereference at virtual address 00000000
[  349.020037] pgd = ddeac000
[  349.022884] [00000000] *pgd=9dcb4831, *pte=00000000, *ppte=00000000
[  349.029816] Internal error: Oops: 17 [#1] PREEMPT SMP ARM
[  349.035482] Modules linked in: tcrypt(+)
[  349.039617] CPU: 0 PID: 1473 Comm: insmod Not tainted 3.12.4-01566-g6279006-dirty #38
[  349.047832] task: dda91540 ti: ddcd2000 task.ti: ddcd2000
[  349.053517] PC is at omap_sham_xmit_dma+0x6c/0x238
[  349.058544] LR is at omap_sham_xmit_dma+0x38/0x238
[  349.063570] pc : [<c04eb7cc>]    lr : [<c04eb798>]    psr: 20000013
[  349.063570] sp : ddcd3c78  ip : 00000000  fp : 9d8980b8
[  349.075610] r10: 00000000  r9 : 00000000  r8 : 00000000
[  349.081090] r7 : 00001000  r6 : dd898000  r5 : 00000040  r4 : ddb10550
[  349.087935] r3 : 00000004  r2 : 00000010  r1 : 53100080  r0 : 00000000
[  349.094783] Flags: nzCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment user
[  349.102268] Control: 10c5387d  Table: 9deac019  DAC: 00000015
[  349.108294] Process insmod (pid: 1473, stack limit = 0xddcd2248)

[...]

This is because polling_mode is not enabled for ctx without FLAGS_FINUP.

For polling mode the bufcnt is made 0 unconditionally. But it should be made 0
only if it is a final update or a total is not zero(This condition is similar
to what is done in DMA case). Because of this wrong hashes are produced.

Fixing the same.
Signed-off-by: Lokesh Vutla <lokeshvutla@ti.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Adding simple speed tests for a range of block sizes for AEAD crypto
algorithms.
Signed-off-by: Tim Chen <tim.c.chen@linux.intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

We have added AVX and AVX2 routines that optimize AESNI-GCM encode/decode.
These routines are optimized for encrypt and decrypt of large buffers.
In tests we have seen up to 6% speedup for 1K, 11% speedup for 2K and
18% speedup for 8K buffer over the existing SSE version.  These routines
should provide even better speedup for future Intel x86_64 cpus.
Signed-off-by: Tim Chen <tim.c.chen@linux.intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Replace remaining occurences (just as we did in crypto/) under arch/*/crypto/
that make use of memcmp() for comparing keys or authentication tags for
usage with crypto_memneq(). It can simply be used as a drop-in replacement
for the normal memcmp().
Signed-off-by: Daniel Borkmann <dborkman@redhat.com>
Cc: James Yonan <james@openvpn.net>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Remove the pr_err calls that are issued during parameter
checking in some AES operations. This will eliminate the
possibility of filling up syslog through these paths.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Update the Kconfig to include PCI on the 'depends on'
and add 'select HW_RANDOM' to insure the necessary PCI
and HW_RANDOM functions are available/included in the
build.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

The sha initialization data generated the following sparse warnings:

   sparse: incorrect type in initializer (different base types)
      expected unsigned int
      got restricted __be32 [usertype] <noident>

Change the initialization data type from u32 to __be32.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Commit fe8c8a12 introduced a possible build error for archs
that do not have CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS set. :/
Fix this up by bringing else braces outside of the ifdef.
Reported-by: Fengguang Wu <fengguang.wu@intel.com>
Fixes: fe8c8a12 ("crypto: more robust crypto_memneq")
Signed-off-by: Daniel Borkmann <dborkman@redhat.com>
Acked-By: Cesar Eduardo Barros <cesarb@cesarb.eti.br>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

drivers/crypto/ccp/ccp-crypto-aes.c:344:1-7: Replace memcpy with struct assignment
drivers/crypto/ccp/ccp-crypto-sha.c:398:1-7: Replace memcpy with struct assignment
drivers/crypto/ccp/ccp-dev.c:578:2-3: Unneeded semicolon
/c/kernel-tests/src/cocci/drivers/crypto/ccp/ccp-dev.c:565:2-3: Unneeded semicolon

Generated by: coccinelle/misc/memcpy-assign.cocci

CC: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Fengguang Wu <fengguang.wu@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

The AES driver currently assumes that pm_runtime_get_sync will always
succeed, which may not always be true, so add error handling for the
same.

This scenario was reported in the following bug:
place.  https://bugzilla.kernel.org/show_bug.cgi?id=66441Reported-by: Tobias Jakobi <tjakobi@math.uni-bielefeld.de>
Signed-off-by: Nishanth Menon <nm@ti.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

A kernel with enabled lockdep complains about the wrong usage of
rcu_dereference() under a rcu_read_lock_bh() protected region.

  ===============================
  [ INFO: suspicious RCU usage. ]
  3.13.0-rc1+ #126 Not tainted
  -------------------------------
  linux/kernel/padata.c:115 suspicious rcu_dereference_check() usage!

  other info that might help us debug this:

  rcu_scheduler_active = 1, debug_locks = 1
  1 lock held by cryptomgr_test/153:
   #0:  (rcu_read_lock_bh){.+....}, at: [<ffffffff8115c235>] padata_do_parallel+0x5/0x270

Fix that by using rcu_dereference_bh() instead.
Signed-off-by: Mathias Krause <minipli@googlemail.com>
Acked-by: Steffen Klassert <steffen.klassert@secunet.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

A kernel with enabled lockdep complains about the wrong usage of
rcu_dereference() under a rcu_read_lock_bh() protected region.

  ===============================
  [ INFO: suspicious RCU usage. ]
  3.13.0-rc1+ #126 Not tainted
  -------------------------------
  linux/crypto/pcrypt.c:81 suspicious rcu_dereference_check() usage!

  other info that might help us debug this:

  rcu_scheduler_active = 1, debug_locks = 1
  1 lock held by cryptomgr_test/153:
   #0:  (rcu_read_lock_bh){.+....}, at: [<ffffffff812c8075>] pcrypt_do_parallel.isra.2+0x5/0x200

Fix that by using rcu_dereference_bh() instead.
Signed-off-by: Mathias Krause <minipli@googlemail.com>
Cc: "David S. Miller" <davem@davemloft.net>
Acked-by: Steffen Klassert <steffen.klassert@secunet.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Disabling compiler optimizations can be fragile, since a new
optimization could be added to -O0 or -Os that breaks the assumptions
the code is making.

Instead of disabling compiler optimizations, use a dummy inline assembly
(based on RELOC_HIDE) to block the problematic kinds of optimization,
while still allowing other optimizations to be applied to the code.

The dummy inline assembly is added after every OR, and has the
accumulator variable as its input and output. The compiler is forced to
assume that the dummy inline assembly could both depend on the
accumulator variable and change the accumulator variable, so it is
forced to compute the value correctly before the inline assembly, and
cannot assume anything about its value after the inline assembly.

This change should be enough to make crypto_memneq work correctly (with
data-independent timing) even if it is inlined at its call sites. That
can be done later in a followup patch.

Compile-tested on x86_64.
Signed-off-by: Cesar Eduardo Barros <cesarb@cesarb.eti.br>
Acked-by: Daniel Borkmann <dborkman@redhat.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Commit 3e721aeb
("crypto: talitos - handle descriptor not found in error path")
tried to address the fact that CDPR (Current Descriptor Pointer Register)
is unreliable.

As it turns out, there are still issues in the function detecting the
offending descriptor:
-only 32 bits of the descriptor address are read, however the address is
36-bit - since reset_channel() initializes channels with EAE (extended
address) bit set
-reading CDPR can return zero in cur_desc; when searching the channel
fifo for this address, cur_desc == dma_desc (= 0) case might happen,
leading to an oops when trying to return desc->hdr (desc is zero)
-read channel's .tail only once; the tail is a moving target; use a
local variable for the end of search condition
Signed-off-by: Lei Xu <Lei.Xu@freescale.com>
Signed-off-by: Horia Geanta <horia.geanta@freescale.com>
Tested-by: Kalyani Chowdhury <Kalyani.Chowdhury@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

In omap_sham_probe() and omap_sham_remove(), 'dd->dma_lch'
is released without checking to see if it was successfully
requested or not.  This is a bug and was identified and
reported by Dan Carpenter here:

	http://www.spinics.net/lists/devicetree/msg11023.html

Add code to only release 'dd->dma_lch' when its not NULL
(that is, when it was successfully requested).
Reported-by: Dan Carpenter <dan.carpenter@oracle.com>
CC: Joel Fernandes <joelf@ti.com>
Signed-off-by: Mark A. Greer <mgreer@animalcreek.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Update the MAINTAINERS file for the AMD CCP device driver.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

These files provide the ability to configure and build the
AMD CCP device driver and crypto API support.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

These routines provide crypto API support for SHA1, SHA224 and SHA256
on the AMD CCP.  HMAC support for these SHA modes is also provided.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

These routines provide crypto API support for the XTS-AES mode of AES
on the AMD CCP.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

These routines provide crypto API support for the CMAC mode of AES
on the AMD CCP.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

These routines provide crypto API support for AES on the AMD CCP.

Support for AES modes: ECB, CBC, OFB, CFB and CTR
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

These routines provide the support for the interface between the crypto API
and the AMD CCP. This includes insuring that requests associated with a
given tfm on the same cpu are processed in the order received.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>