The user space Jitter RNG library uses the oversampling rate of 3 which
implies that each time stamp is credited with 1/3 bit of entropy. To
obtain 256 bits of entropy, 768 time stamps need to be sampled. The
increase in OSR is applied based on a report where the Jitter RNG is
used on a system exhibiting a challenging environment to collect
entropy.

This OSR default value is now applied to the Linux kernel version of
the Jitter RNG as well.

The increase in the OSR from 1 to 3 also implies that the Jitter RNG is
now slower by default.
Reported-by: Jeff Barnes <jeffbarnes@microsoft.com>
Signed-off-by: Stephan Mueller <smueller@chronox.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

The custom swap functions used in octeontx2 driver do not perform any
special operations and can be replaced with the built-in swap function
of sort. This change not only reduces code size but also improves
efficiency, especially in scenarios where CONFIG_RETPOLINE is enabled,
as it makes indirect function calls more expensive.

By using the built-in swap, we avoid these costly indirect function
calls, leading to better performance.
Signed-off-by: Kuan-Wei Chiu <visitorckw@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

The custom swap function used in octeontx driver do not perform any
special operations and can be replaced with the built-in swap function
of sort. This change not only reduces code size but also improves
efficiency, especially in scenarios where CONFIG_RETPOLINE is enabled,
as it makes indirect function calls more expensive.

By using the built-in swap, we avoid these costly indirect function
calls, leading to better performance.
Signed-off-by: Kuan-Wei Chiu <visitorckw@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Fixes: 6637e11e ("crypto: rsa - allow only odd e and restrict value in FIPS mode")
Fixes: f145d411 ("crypto: rsa - implement Chinese Remainder Theorem for faster private key operation")
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Now that mpi_rshift can return errors, check them.

Fixes: 35d2bf20 ("crypto: dh - calculate Q from P for the full public key verification")
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

The remaining functions added by commit
a8ea8bdd did not check for memory
allocation errors.  Add the checks and change the API to allow errors
to be returned.

Fixes: a8ea8bdd ("lib/mpi: Extend the MPI library")
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

This partially reverts commit a8ea8bdd.

Most of it is no longer needed since sm2 has been removed.  However,
the following functions have been kept as they have developed other
uses:

mpi_copy

mpi_mod

mpi_test_bit
mpi_set_bit
mpi_rshift

mpi_add
mpi_sub
mpi_addm
mpi_subm

mpi_mul
mpi_mulm

mpi_tdiv_r
mpi_fdiv_r
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

In aes-neonbs, instead of going through the crypto API for the parts
that the bit-sliced AES code doesn't handle, namely AES-CBC encryption
and single-block AES, just call the ARM scalar AES cipher directly.

This basically goes back to the original approach that was used before
commit b56f5cbc ("crypto: arm/aes-neonbs - resolve fallback cipher
at runtime").  Calling the ARM scalar AES cipher directly is faster,
simpler, and avoids any chance of bugs specific to the use of fallback
ciphers such as module loading deadlocks which have happened twice.  The
deadlocks turned out to be fixable in other ways, but there's no need to
rely on anything so fragile in the first place.

The rationale for the above-mentioned commit was to allow people to
choose to use a time-invariant AES implementation for the fallback
cipher.  There are a couple problems with that rationale, though:

- In practice the ARM scalar AES cipher (aes-arm) was used anyway, since
  it has a higher priority than aes-fixed-time.  Users *could* go out of
  their way to disable or blacklist aes-arm, or to lower its priority
  using NETLINK_CRYPTO, but very few users customize the crypto API to
  this extent.  Systems with the ARMv8 Crypto Extensions used aes-ce,
  but the bit-sliced algorithms are irrelevant on such systems anyway.

- Since commit 913a3aa0 ("crypto: arm/aes - add some hardening
  against cache-timing attacks"), the ARM scalar AES cipher is partially
  hardened against cache-timing attacks.  It actually works like
  aes-fixed-time, in that it disables interrupts and prefetches its
  lookup table.  It does use a larger table than aes-fixed-time, but
  even so, it is not clear that aes-fixed-time is meaningfully more
  time-invariant than aes-arm.  And of course, the real solution for
  time-invariant AES is to use a CPU that supports AES instructions.
Signed-off-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

When submitting more than 2^32 padata objects to padata_do_serial, the
current sorting implementation incorrectly sorts padata objects with
overflowed seq_nr, causing them to be placed before existing objects in
the reorder list. This leads to a deadlock in the serialization process
as padata_find_next cannot match padata->seq_nr and pd->processed
because the padata instance with overflowed seq_nr will be selected
next.

To fix this, we use an unsigned integer wrap around to correctly sort
padata objects in scenarios with integer overflow.

Fixes: bfde23ce ("padata: unbind parallel jobs from specific CPUs")
Cc: <stable@vger.kernel.org>
Co-developed-by: Christian Gafert <christian.gafert@rohde-schwarz.com>
Signed-off-by: Christian Gafert <christian.gafert@rohde-schwarz.com>
Co-developed-by: Max Ferger <max.ferger@rohde-schwarz.com>
Signed-off-by: Max Ferger <max.ferger@rohde-schwarz.com>
Signed-off-by: Van Giang Nguyen <vangiang.nguyen@rohde-schwarz.com>
Acked-by: Daniel Jordan <daniel.m.jordan@oracle.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Commit 1e6b251c ("crypto: nx - Avoid -Wflex-array-member-not-at-end
warning") introduced tagged `struct nx842_crypto_header_hdr`. We want
to ensure that when new members need to be added to the flexible
structure, they are always included within this tagged struct.

So, we use `static_assert()` to ensure that the memory layout for
both the flexible structure and the tagged struct is the same after
any changes.
Signed-off-by: Gustavo A. R. Silva <gustavoars@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Commit 140e4c85 ("crypto: qat - Avoid -Wflex-array-member-not-at-end
warnings") introduced tagged `struct qat_alg_buf_list_hdr`. We want to
ensure that when new members need to be added to the flexible structure,
they are always included within this tagged struct.

So, we use `static_assert()` to ensure that the memory layout for
both the flexible structure and the tagged struct is the same after
any changes.
Signed-off-by: Gustavo A. R. Silva <gustavoars@kernel.org>
Acked-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

When objtool gains support for ARM in the future, it may encounter issues
disassembling the following data in the .text section:
> .Lzeros:
> .long   0,0,0,0,0,0,0,0
> .asciz  "Poly1305 for ARMv8, CRYPTOGAMS by \@dot-asm"
> .align  2

Move it to .rodata which is a more appropriate section for read-only data.

There is a limit on how far the label can be from the instruction, hence
use "adrp" and low 12bits offset of the label to avoid the compilation
error.
Signed-off-by: Jia He <justin.he@arm.com>
Tested-by: Daniel Gomez <da.gomez@samsung.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Add the __counted_by compiler attribute to the flexible array member
salt to improve access bounds-checking via CONFIG_UBSAN_BOUNDS and
CONFIG_FORTIFY_SOURCE.
Reviewed-by: Kees Cook <kees@kernel.org>
Signed-off-by: Thorsten Blum <thorsten.blum@toblux.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Fixup for "crypto: spacc - Add SPAcc Skcipher support"
interacting with commit

  1a251f52 ("minmax: make generic MIN() and MAX() macros available everywhere")

from Linus' tree.
Signed-off-by: Stephen Rothwell <sfr@canb.auug.org.au>

Reintroduced MIN macro with ifndef around it.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

On PREEMPT_RT, kfree() takes sleeping locks and must not be called with
preemption disabled.  Therefore, on PREEMPT_RT skcipher_walk_done() must
not be called from within a kernel_fpu_{begin,end}() pair, even when
it's the last call which is guaranteed to not allocate memory.

Therefore, move the last skcipher_walk_done() in gcm_crypt() to the end
of the function so that it goes after the kernel_fpu_end().  To make
this work cleanly, rework the data processing loop to handle only
non-last data segments.

Fixes: b06affb1 ("crypto: x86/aes-gcm - add VAES and AVX512 / AVX10 optimized AES-GCM")
Reported-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Closes: https://lore.kernel.org/linux-crypto/20240802102333.itejxOsJ@linutronix.deSigned-off-by: Eric Biggers <ebiggers@google.com>
Tested-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Add the missing clk_disable_unprepare() before return in
cctrng_resume().

Fixes: a583ed31 ("hwrng: cctrng - introduce Arm CryptoCell driver")
Cc: <stable@vger.kernel.org>
Signed-off-by: Gaosheng Cui <cuigaosheng1@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Add the missing clk_disable_unprepare() before return in
bcm2835_rng_init().

Fixes: e5f9f41d ("hwrng: bcm2835 - add reset support")
Cc: <stable@vger.kernel.org>
Signed-off-by: Gaosheng Cui <cuigaosheng1@huawei.com>
Reviewed-by: Florian Fainelli <florian.fainelli@broadcom.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

This reverts commit d58bb7e5.

It's no longer needed since sm2 has been removed.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Rockchip SoCs used to have a random number generator as part of their
crypto device, and support for it has to be added to the corresponding
driver. However newer Rockchip SoCs like the RK3568 have an independent
True Random Number Generator device. This patch adds a driver for it,
greatly inspired from the downstream driver.

The TRNG device does not seem to have a signal conditionner and the FIPS
140-2 test returns a lot of failures. They can be reduced by increasing
RK_RNG_SAMPLE_CNT, in a tradeoff between quality and speed. This value
has been adjusted to get ~90% of successes and the quality value has
been set accordingly.
Signed-off-by: Aurelien Jarno <aurelien@aurel32.net>
[daniel@makrotpia.org: code style fixes]
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
Acked-by: Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Add the True Random Number Generator on the Rockchip RK3568 SoC.
Signed-off-by: Aurelien Jarno <aurelien@aurel32.net>
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
Reviewed-by: Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Switch cpumask_t to cpumask_var_t as the former may be too big
for the stack:

  CC [M]  drivers/crypto/caam/qi.o
  CC [M]  drivers/crypto/caam/caamalg_qi2.o
../drivers/crypto/caam/qi.c: In function ‘caam_qi_init’:
../drivers/crypto/caam/qi.c:808:1: warning: the frame size of 1056 bytes is larger than 1024 bytes [-Wframe-larger-than=]
  808 | }
      | ^
  CHECK   ../drivers/crypto/caam/qi.c
../drivers/crypto/caam/caamalg_qi2.c: In function ‘dpaa2_dpseci_setup’:
../drivers/crypto/caam/caamalg_qi2.c:5135:1: warning: the frame size of 1032 bytes is larger than 1024 bytes [-Wframe-larger-than=]
 5135 | }
      | ^

Also fix the error path handling in qi.c.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Before commit addea585 ("hwrng: Kconfig - Do not enable by default
CN10K driver") the Marvell CN10K Random Number Generator was always
enabled when HW_RANDOM was enabled.

This was changed with that commit to prevent having this driver being
always enabled on arm64. To prevent introducing regression with some old
defconfig enable the driver when ARCH_THUNDER is enabled.

Fixes: addea585 ("hwrng: Kconfig - Do not enable by default CN10K driver")
Closes: https://lore.kernel.org/all/SN7PR18MB53144B37B82ADEEC5D35AE0CE3AC2@SN7PR18MB5314.namprd18.prod.outlook.com/Signed-off-by: Francesco Dolcini <francesco.dolcini@toradex.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Signed-off-by: Bhoomika K <bhoomikak@vayavyalabs.com>
Signed-off-by: Pavitrakumar M <pavitrakumarm@vayavyalabs.com>
Acked-by: Ruud Derwig <Ruud.Derwig@synopsys.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Signed-off-by: shwetar <shwetar@vayavyalabs.com>
Signed-off-by: Pavitrakumar M <pavitrakumarm@vayavyalabs.com>
Acked-by: Ruud Derwig <Ruud.Derwig@synopsys.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Signed-off-by: shwetar <shwetar@vayavyalabs.com>
Signed-off-by: Pavitrakumar M <pavitrakumarm@vayavyalabs.com>
Acked-by: Ruud Derwig <Ruud.Derwig@synopsys.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Signed-off-by: shwetar <shwetar@vayavyalabs.com>
Signed-off-by: Pavitrakumar M <pavitrakumarm@vayavyalabs.com>
Acked-by: Ruud Derwig <Ruud.Derwig@synopsys.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Signed-off-by: Bhoomika K <bhoomikak@vayavyalabs.com>
Signed-off-by: Pavitrakumar M <pavitrakumarm@vayavyalabs.com>
Acked-by: Ruud Derwig <Ruud.Derwig@synopsys.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Signed-off-by: Bhoomika K <bhoomikak@vayavyalabs.com>
Signed-off-by: Pavitrakumar M <pavitrakumarm@vayavyalabs.com>
Acked-by: Ruud Derwig <Ruud.Derwig@synopsys.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

This reverts commit 47d96252.

It causes build issues as detected by the kernel test robot.
Reported-by: kernel test robot <lkp@intel.com>
Closes: https://lore.kernel.org/oe-kbuild-all/202408040817.OWKXtCv6-lkp@intel.com/Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

The free_device_compression_mode(iaa_device, device_mode) function frees
"device_mode" but it iss passed to iaa_compression_modes[i]->free() a few
lines later resulting in a use after free.

The good news is that, so far as I can tell, nothing implements the
->free() function and the use after free happens in dead code.  But, with
this fix, when something does implement it, we'll be ready.  :)

Fixes: b190447e ("crypto: iaa - Add compression mode management along with fixed mode")
Signed-off-by: Dan Carpenter <dan.carpenter@linaro.org>
Reviewed-by: Tom Zanussi <tom.zanussi@linux.intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Since commit 1fffe7a3 ("script: modpost: emit a warning when the
description is missing"), a module without a MODULE_DESCRIPTION() will
result in a warning with make W=1. The following warning is being
observed when building ppc64le with CRYPTO_CURVE25519_PPC64=m:

WARNING: modpost: missing MODULE_DESCRIPTION() in arch/powerpc/crypto/curve25519-ppc64le.o

Add the missing invocation of the MODULE_DESCRIPTION() macro.
Signed-off-by: Jeff Johnson <quic_jjohnson@quicinc.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

The QAT driver allows enabling SR-IOV VFs but does not allow them to be
disabled through a write to sysfs.
Disabling SR-IOV VFs can be only achieved by bringing down and up a
device using the attribute /sys/bus/pci/devices/<BDF>/qat/state.

The documentation for the sysfs attribute `sriov_numvfs` specifies
that "a userspace application wanting to disable the VFs would write a
zero to this file".

Add support for disabling SR-IOV VFs by writing '0' to the
'sriov_numvfs' attribute in sysfs.

Enabling or disabling SR-IOV always requires adf_dev_down() to be
called. This action subsequently leads to the deletion of the
ADF_KERNEL_SEC configuration section. The keys ADF_NUM_CY and ADF_NUM_DC
within that section must be set to '0', otherwise, the driver will
register into the Linux Crypto Framework. Because of this, the
configuration in the ADF_KERNEL_SEC section must be added before every
sriov_enable.
Signed-off-by: Michal Witwicki <michal.witwicki@intel.com>
Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: Przemek Kitszel <przemyslaw.kitszel@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

In the process of sending the ADF_PF2VF_MSGTYPE_RESTARTING message to
Virtual Functions (VFs), the Physical Function (PF) should set the
`vf->restarting` flag to true before dispatching the message.
This change is necessary to prevent a race condition where the handling
of the ADF_VF2PF_MSGTYPE_RESTARTING_COMPLETE message (which sets the
`vf->restarting` flag to false) runs immediately after the message is sent,
but before the flag is set to true.

Set the `vf->restarting` to true before sending the message
ADF_PF2VF_MSGTYPE_RESTARTING, if supported by the version of the
protocol and if the VF is started.

Fixes: ec26f8e6 ("crypto: qat - update PFVF protocol for recovery")
Signed-off-by: Michal Witwicki <michal.witwicki@intel.com>
Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: Przemek Kitszel <przemyslaw.kitszel@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

When the PFVF protocol was updated to support version 5, i.e.
ADF_PFVF_COMPAT_FALLBACK, the compatibility version for the VF was
updated without supporting the message RESTARTING_COMPLETE required for
such version.

Add support for the ADF_VF2PF_MSGTYPE_RESTARTING_COMPLETE message in the
VF drivers. This message is sent by the VF driver to the PF to notify
the completion of the shutdown flow.

Fixes: ec26f8e6 ("crypto: qat - update PFVF protocol for recovery")
Signed-off-by: Michal Witwicki <michal.witwicki@intel.com>
Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: Przemek Kitszel <przemyslaw.kitszel@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Disabling IOV has the side effect of re-enabling the AEs that might
attempt to do DMAs into the heartbeat buffers.
Move the disable_iov() function in adf_dev_stop() before the AEs are
stopped.

Fixes: ed8ccaef ("crypto: qat - Add support for SRIOV")
Signed-off-by: Michal Witwicki <michal.witwicki@intel.com>
Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: Przemek Kitszel <przemyslaw.kitszel@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

The ADF_GENERAL_SEC configuration section contains values that must be
preserved during state transitions (down -> up, up -> down).
This patch modifies the logic in adf_dev_shutdown() to maintain all
key values within this section, rather than selectively saving and
restoring only the ADF_SERVICES_ENABLED attribute.

To achieve this, a new function has been introduced that deletes all
configuration sections except for the one specified by name.
This function is invoked during adf_dev_down(), with ADF_GENERAL_SEC
as the argument. Consequently, the adf_dev_shutdown_cache_cfg() function
has been removed as it is now redundant.

Additionally, this patch eliminates the cache_config parameter from
the adf_dev_down() function since ADF_GENERAL_SEC should always be
retained. This change does not cause any side effects because all
entries in the key-value store are cleared when a module is unloaded.
Signed-off-by: Adam Guerin <adam.guerin@intel.com>
Co-developed-by: Michal Witwicki <michal.witwicki@intel.com>
Signed-off-by: Michal Witwicki <michal.witwicki@intel.com>
Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: Przemek Kitszel <przemyslaw.kitszel@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

When objtool gains support for ARM in the future, it may encounter issues
disassembling the following data in the .text section:
> .Lzeros:
> .long   0,0,0,0,0,0,0,0
> .asciz  "Poly1305 for ARMv8, CRYPTOGAMS by \@dot-asm"
> .align  2

Move it to .rodata which is a more appropriate section for read-only data.
Signed-off-by: Jia He <justin.he@arm.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Commit c055e3ea ("crypto: xor - use ktime for template benchmarking")
switched from using jiffies to ktime-based performance benchmarking.

This works nicely on machines which have a fine-grained ktime()
clocksource as e.g. x86 machines with TSC.
But other machines, e.g. my 4-way HP PARISC server, don't have such
fine-grained clocksources, which is why it seems that 800 xor loops
take zero seconds, which then shows up in the logs as:

 xor: measuring software checksum speed
    8regs           : -1018167296 MB/sec
    8regs_prefetch  : -1018167296 MB/sec
    32regs          : -1018167296 MB/sec
    32regs_prefetch : -1018167296 MB/sec

Fix this with some small modifications to the existing code to improve
the algorithm to always produce correct results without introducing
major delays for architectures with a fine-grained ktime()
clocksource:
a) Delay start of the timing until ktime() just advanced. On machines
with a fast ktime() this should be just one additional ktime() call.
b) Count the number of loops. Run at minimum 800 loops and finish
earliest when the ktime() counter has progressed.

With that the throughput can now be calculated more accurately under all
conditions.

Fixes: c055e3ea ("crypto: xor - use ktime for template benchmarking")
Signed-off-by: Helge Deller <deller@gmx.de>
Tested-by: John David Anglin <dave.anglin@bell.net>

v2:
- clean up coding style (noticed & suggested by Herbert Xu)
- rephrased & fixed typo in commit message
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Merge tag 'kbuild-fixes-v6.11' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild

Pull Kbuild fixes from Masahiro Yamada:

 - Fix RPM package build error caused by an incorrect locale setup

 - Mark modules.weakdep as ghost in RPM package

 - Fix the odd combination of -S and -c in stack protector scripts,
   which is an error with the latest Clang

* tag 'kbuild-fixes-v6.11' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild:
  kbuild: Fix '-S -c' in x86 stack protector scripts
  kbuild: rpm-pkg: ghost modules.weakdep file
  kbuild: rpm-pkg: Fix C locale setup