Fix the following clang warning:

drivers/crypto/chelsio/chcr_algo.c:129:19: warning: unused function
'is_ofld_imm' [-Wunused-function].
Reported-by: Abaci Robot <abaci@linux.alibaba.com>
Signed-off-by: Jiapeng Chong <jiapeng.chong@linux.alibaba.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

The sa2ul module in am64 have limited support for algorithms, and the
priv and priv_id used on the platform is different compared to AM654 or
j721e.

Use match data to get the SoC specific information and use it throughout
the driver.
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: Vaibhav Gupta <v_gupta@ti.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

On AM64 the DMA channel for sa2ul can be configured to be coherent or
non coherent via DT binding.

Use the dmaengine_get_device_for_dma_api() to get the device pointer which
should be used for with the dma_api to use matching dma_ops for the
channel coherency/non coherency.
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: Vaibhav Gupta <v_gupta@ti.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Add the AM64 version of sa2ul to the compatible list.

[v_gupta@ti.com: Conditional dma-coherent requirement, clocks]
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: Vaibhav Gupta <v_gupta@ti.com>
Reviewed-by: Rob Herring <robh@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

QM adds 'qm_mailbox_timeout' and 'qm_flr_timeout' hardware error types on
Kunpeng930. This patch enables the new error types and configures the error
types as NFE.
Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Kunpeng930 SEC adds several new hardware error types. This patch enables
the new error types and configures the error types as NFE.
Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Kunpeng930 ZIP adds 'zip_axi_poison_err' 'zip_sva_err' and
'QM_ACC_DO_TASK_TIMEOUT' hardware error types. This patch enables the error
types and configures the error types as NFE.
Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

'err_info' does not support dynamic configuration since it is const type.
Therefore, in order to support new error type later, 'err_info' is changed
to dynamic configuration.
Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Fix function name in chacha.c kernel-doc comment to remove a warning.

Convert af_alg.c to kernel-doc notation to eliminate many kernel-doc
warnings.

../lib/crypto/chacha.c:77: warning: expecting prototype for chacha_block(). Prototype was for chacha_block_generic() instead
chacha.c:104: warning: Excess function parameter 'out' description in 'hchacha_block_generic'

af_alg.c:498: warning: Function parameter or member 'sk' not described in 'af_alg_alloc_tsgl'
../crypto/af_alg.c:539: warning: expecting prototype for aead_count_tsgl(). Prototype was for af_alg_count_tsgl() instead
../crypto/af_alg.c:596: warning: expecting prototype for aead_pull_tsgl(). Prototype was for af_alg_pull_tsgl() instead
af_alg.c:663: warning: Function parameter or member 'areq' not described in 'af_alg_free_areq_sgls'
af_alg.c:700: warning: Function parameter or member 'sk' not described in 'af_alg_wait_for_wmem'
af_alg.c:700: warning: Function parameter or member 'flags' not described in 'af_alg_wait_for_wmem'
af_alg.c:731: warning: Function parameter or member 'sk' not described in 'af_alg_wmem_wakeup'
af_alg.c:757: warning: Function parameter or member 'sk' not described in 'af_alg_wait_for_data'
af_alg.c:757: warning: Function parameter or member 'flags' not described in 'af_alg_wait_for_data'
af_alg.c:757: warning: Function parameter or member 'min' not described in 'af_alg_wait_for_data'
af_alg.c:796: warning: Function parameter or member 'sk' not described in 'af_alg_data_wakeup'
af_alg.c:832: warning: Function parameter or member 'sock' not described in 'af_alg_sendmsg'
af_alg.c:832: warning: Function parameter or member 'msg' not described in 'af_alg_sendmsg'
af_alg.c:832: warning: Function parameter or member 'size' not described in 'af_alg_sendmsg'
af_alg.c:832: warning: Function parameter or member 'ivsize' not described in 'af_alg_sendmsg'
af_alg.c:985: warning: Function parameter or member 'sock' not described in 'af_alg_sendpage'
af_alg.c:985: warning: Function parameter or member 'page' not described in 'af_alg_sendpage'
af_alg.c:985: warning: Function parameter or member 'offset' not described in 'af_alg_sendpage'
af_alg.c:985: warning: Function parameter or member 'size' not described in 'af_alg_sendpage'
af_alg.c:985: warning: Function parameter or member 'flags' not described in 'af_alg_sendpage'
af_alg.c:1040: warning: Function parameter or member 'areq' not described in 'af_alg_free_resources'
af_alg.c:1059: warning: Function parameter or member '_req' not described in 'af_alg_async_cb'
af_alg.c:1059: warning: Function parameter or member 'err' not described in 'af_alg_async_cb'
af_alg.c:1083: warning: Function parameter or member 'file' not described in 'af_alg_poll'
af_alg.c:1083: warning: Function parameter or member 'sock' not described in 'af_alg_poll'
af_alg.c:1083: warning: Function parameter or member 'wait' not described in 'af_alg_poll'
af_alg.c:1114: warning: Function parameter or member 'sk' not described in 'af_alg_alloc_areq'
af_alg.c:1114: warning: Function parameter or member 'areqlen' not described in 'af_alg_alloc_areq'
af_alg.c:1146: warning: Function parameter or member 'sk' not described in 'af_alg_get_rsgl'
af_alg.c:1146: warning: Function parameter or member 'msg' not described in 'af_alg_get_rsgl'
af_alg.c:1146: warning: Function parameter or member 'flags' not described in 'af_alg_get_rsgl'
af_alg.c:1146: warning: Function parameter or member 'areq' not described in 'af_alg_get_rsgl'
af_alg.c:1146: warning: Function parameter or member 'maxsize' not described in 'af_alg_get_rsgl'
af_alg.c:1146: warning: Function parameter or member 'outlen' not described in 'af_alg_get_rsgl'
Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
Cc: Herbert Xu <herbert@gondor.apana.org.au>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: linux-crypto@vger.kernel.org
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Use SPDX-License-Identifier: GPL-2.0-only, instead of hand writing it.

This also removes a reference to http://www.xyratex.com which seems to be
down.
Signed-off-by: Christophe JAILLET <christophe.jaillet@wanadoo.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

'hpre_cfg_by_dsm' has checked and printed error path internally. It is not
necessary to do it here, so remove it.

It should return error immediately when return value of 'hpre_cfg_by_dsm'
is non-zero, and no need to execute the remaining sentences.
Signed-off-by: Hui Tang <tanghui20@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

When the received sqe is abnormal, the error message in the sqe written
back by the hardware is printed to help to analyze the abnormal causes.
Signed-off-by: Hui Tang <tanghui20@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

The return value of 'le32_to_cpu' is unsigned, so change the
variable type from 'int' to 'unsigned int'.
Signed-off-by: Hui Tang <tanghui20@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

There are two spaces after return, just keep one.
Signed-off-by: Hui Tang <tanghui20@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Debian's clang carries a patch that makes the default FPU mode
'vfp3-d16' instead of 'neon' for 'armv7-a' to avoid generating NEON
instructions on hardware that does not support them:

https://salsa.debian.org/pkg-llvm-team/llvm-toolchain/-/raw/5a61ca6f21b4ad8c6ac4970e5ea5a7b5b4486d22/debian/patches/clang-arm-default-vfp3-on-armv7a.patch
https://bugs.debian.org/841474
https://bugs.debian.org/842142
https://bugs.debian.org/914268

This results in the following build error when clang's integrated
assembler is used because the '.arch' directive overrides the '.fpu'
directive:

arch/arm/crypto/curve25519-core.S:25:2: error: instruction requires: NEON
 vmov.i32 q0, #1
 ^
arch/arm/crypto/curve25519-core.S:26:2: error: instruction requires: NEON
 vshr.u64 q1, q0, #7
 ^
arch/arm/crypto/curve25519-core.S:27:2: error: instruction requires: NEON
 vshr.u64 q0, q0, #8
 ^
arch/arm/crypto/curve25519-core.S:28:2: error: instruction requires: NEON
 vmov.i32 d4, #19
 ^

Shuffle the order of the '.arch' and '.fpu' directives so that the code
builds regardless of the default FPU mode. This has been tested against
both clang with and without Debian's patch and GCC.

Cc: stable@vger.kernel.org
Fixes: d8f1308a ("crypto: arm/curve25519 - wire up NEON implementation")
Link: https://github.com/ClangBuiltLinux/continuous-integration2/issues/118Reported-by: Arnd Bergmann <arnd@arndb.de>
Suggested-by: Arnd Bergmann <arnd@arndb.de>
Suggested-by: Jessica Clarke <jrtc27@jrtc27.com>
Signed-off-by: Nathan Chancellor <nathan@kernel.org>
Acked-by: Jason A. Donenfeld <Jason@zx2c4.com>
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
Tested-by: Nick Desaulniers <ndesaulniers@google.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Enable the detection of hangs by setting watchdog timers (WDTs) on
generations that supports that feature.

The default timeout value comes from HW specs. WTDs are reset each time
an accelerator wins arbitration and is able to send/read a command to/from
an accelerator.

The value has added significant margin to make sure there are no spurious
timeouts. The scope of watchdog is per QAT device.

If a timeout is detected, the firmware resets the accelerator and
returns a response descriptor with an appropriate error code.
Signed-off-by: Wojciech Ziemba <wojciech.ziemba@intel.com>
Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Add data cleared operation for sge data.
Signed-off-by: Kai Ye <yekai13@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

The buffer of the hardware sge needs to be initialized by
soft sgl.
Signed-off-by: Kai Ye <yekai13@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Add some dfx logs in some abnormal exit situations.
Signed-off-by: Kai Ye <yekai13@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Delete unneeded variable initialization
Signed-off-by: Kai Ye <yekai13@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

This seems "32" and "31" is obfuscating, It might be better to add a comment,
which explain it.
Signed-off-by: Kai Ye <yekai13@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

According to Documentation/core-api/printk-formats.rst, Use
the correct print format. Printing an unsigned int value should use %u
instead of %d.
Signed-off-by: Kai Ye <yekai13@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

According to Documentation/core-api/printk-formats.rst, Use
the correct print format. Printing an unsigned int value should use %u
instead of %d.
Signed-off-by: Kai Ye <yekai13@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

pm_runtime_get_sync will increment pm usage counter even it failed.
Forgetting to putting operation will result in reference leak here.
Fix it by replacing it with pm_runtime_resume_and_get to keep usage
counter balanced.
Signed-off-by: Shixin Liu <liushixin2@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

pm_runtime_get_sync will increment pm usage counter even it failed.
Forgetting to putting operation will result in reference leak here.
Fix it by replacing it with pm_runtime_resume_and_get to keep usage
counter balanced.
Signed-off-by: Shixin Liu <liushixin2@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

pm_runtime_get_sync will increment pm usage counter even it failed.
Forgetting to putting operation will result in reference leak here.
Fix it by replacing it with pm_runtime_resume_and_get to keep usage
counter balanced.
Signed-off-by: Shixin Liu <liushixin2@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

pm_runtime_get_sync will increment pm usage counter even it failed.
Forgetting to putting operation will result in reference leak here.
Fix it by replacing it with pm_runtime_resume_and_get to keep usage
counter balanced.
Signed-off-by: Shixin Liu <liushixin2@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

pm_runtime_get_sync will increment pm usage counter even it failed.
Forgetting to putting operation will result in reference leak here.
Fix it by replacing it with pm_runtime_resume_and_get to keep usage
counter balanced.
Signed-off-by: Shixin Liu <liushixin2@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

pm_runtime_get_sync will increment pm usage counter even it failed.
Forgetting to putting operation will result in reference leak here.
Fix it by replacing it with pm_runtime_resume_and_get to keep usage
counter balanced.
Signed-off-by: Shixin Liu <liushixin2@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

pm_runtime_get_sync will increment pm usage counter even it failed.
Forgetting to putting operation will result in reference leak here.
Fix it by replacing it with pm_runtime_resume_and_get to keep usage
counter balanced.
Signed-off-by: Shixin Liu <liushixin2@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Remove including <linux/version.h> that don't need it.
Signed-off-by: Tian Tao <tiantao6@hisilicon.com>
Signed-off-by: Zhiqi Song <songzhiqi1@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

There is a error message within devm_ioremap_resource
already, so remove the dev_err call to avoid redundant
error message.
Signed-off-by: YueHaibing <yuehaibing@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

There is a error message within devm_ioremap_resource
already, so remove the dev_err call to avoid redundant
error message.
Signed-off-by: YueHaibing <yuehaibing@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

There is a error message within devm_ioremap_resource
already, so remove the dev_err call to avoid redundant
error message.
Signed-off-by: YueHaibing <yuehaibing@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

devm_platform_ioremap_resource() and devm_ioremap_resource()
will print err msg while failing, so the redundant dev_err call
can be removed.
Signed-off-by: YueHaibing <yuehaibing@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

There is a error message within devm_ioremap_resource
already, so remove the dev_err call to avoid redundant
error message.
Signed-off-by: YueHaibing <yuehaibing@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Kunpeng930 supports trng and prng, but Kunpeng920 only supports trng.

Therefore, version information is added to ensure that prng is not
registered to Crypto subsystem on Kunpeng920.
Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

The new carry handling code in the CTR driver can deal with a carry
occurring in the 4x/5x parallel code path, by using a computed goto to
jump into the carry sequence at the right place as to only apply the
carry to a subset of the blocks being processed.

If the lower half of the counter wraps and ends up at exactly 0x0, a
carry needs to be applied to the counter, but not to the counter values
taken for the 4x/5x parallel sequence. In this case, the computed goto
skips all register assignments, and branches straight to the jump
instruction that gets us back to the fast path. This produces the
correct result, but due to the fact that this branch target does not
carry the correct BTI annotation, this fails when BTI is enabled.

Let's omit the computed goto entirely in this case, and jump straight
back to the fast path after applying the carry to the main counter.

Fixes: 5318d3db ("crypto: arm64/aes-ctr - improve tail handling")
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Kunpeng930 could be able to stop queue by writing hardware registers,
which will trigger tasks in device to be flushed out.

In order to be compatible with the kunpeng920 driver, add 'qm_hw_ops_v3' to
adapt Kunpeng930. And 'stop_qp' callback is added in 'qm_hw_ops_v3' to
write hardware registers. Call 'stop_qp' to drain the tasks in device
before releasing queue.
Signed-off-by: Weili Qian <qianweili@huawei.com>
Reviewed-by: Longfang Liu <liulongfang@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

spinlock can be initialized automatically with DEFINE_SPINLOCK()
rather than explicitly calling spin_lock_init().
Reported-by: Hulk Robot <hulkci@huawei.com>
Signed-off-by: Guobin Huang <huangguobin4@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>