Skip to content

L
linux
Commit 7a46b17d authored by Linus Torvalds's avatar Linus Torvalds
Browse files
Options

Merge tag 'dmaengine-6.11-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/vkoul/dmaengine 

Pull dmaengine updates from Vinod Koul:
 "New support:

   - New dmaengine_prep_peripheral_dma_vec() to support transfers using
     dma vectors and documentation and user in AXI dma

   - STMicro STM32 DMA3 support and new capabilities of cyclic dma

  Updates:

   - Yaml conversion for Freescale imx dma and qdma bindings,
     sprd sc9860 dma binding

   - Altera msgdma updates for descriptor management"

* tag 'dmaengine-6.11-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/vkoul/dmaengine: (35 commits)
  dt-bindings: fsl-qdma: fix interrupts 'if' check logic
  dt-bindings: dma: sprd,sc9860-dma: convert to YAML
  dmaengine: fsl-dpaa2-qdma: add missing MODULE_DESCRIPTION() macro
  dmaengine: ti: add missing MODULE_DESCRIPTION() macros
  dmaengine: ti: cppi41: add missing MODULE_DESCRIPTION() macro
  dmaengine: virt-dma: add missing MODULE_DESCRIPTION() macro
  dmaengine: ti: k3-udma: Fix BCHAN count with UHC and HC channels
  dmaengine: sh: rz-dmac: Fix lockdep assert warning
  dmaengine: qcom: gpi: clean up the IRQ disable/enable in gpi_reset_chan()
  dmaengine: fsl-edma: change the memory access from local into remote mode in i.MX 8QM
  dmaengine: qcom: gpi: remove unused struct 'reg_info'
  dmaengine: moxart-dma: remove unused struct 'moxart_filter_data'
  dt-bindings: fsl-qdma: Convert to yaml format
  dmaengine: fsl-edma: remove redundant "idle" field from fsl_chan
  dmaengine: fsl-edma: request per-channel IRQ only when channel is allocated
  dmaengine: stm32-dma3: defer channel registration to specify channel name
  dmaengine: add channel device name to channel registration
  dmaengine: stm32-dma3: improve residue granularity
  dmaengine: stm32-dma3: add device_pause and device_resume ops
  dmaengine: stm32-dma3: add DMA_MEMCPY capability
  ...
parents 7a3fad30 b8ec9dba
Hide whitespace changes
Inline Side-by-side
Showing with 2492 additions and 282 deletions
Documentation/devicetree/bindings/dma/fsl,imx-dma.yaml 0 → 100644
View file @ 7a46b17d
# SPDX-License-Identifier: GPL-2.0-only OR BSD-2-Clause
%YAML 1.2
---
$id: http://devicetree.org/schemas/dma/fsl,imx-dma.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
title: Freescale Direct Memory Access (DMA) Controller for i.MX
maintainers:
- Animesh Agarwal <animeshagarwal28@gmail.com>
allOf:
- $ref: dma-controller.yaml#
properties:
compatible:
enum:
- fsl,imx1-dma
- fsl,imx21-dma
- fsl,imx27-dma
reg:
maxItems: 1
interrupts:
items:
- description: DMA complete interrupt
- description: DMA Error interrupt
minItems: 1
"#dma-cells":
const: 1
dma-channels:
const: 16
dma-requests:
description: Number of DMA requests supported.
required:
- compatible
- reg
- interrupts
- "#dma-cells"
additionalProperties: false
examples:
- |
dma-controller@10001000 {
compatible = "fsl,imx27-dma";
reg = <0x10001000 0x1000>;
interrupts = <32 33>;
#dma-cells = <1>;
dma-channels = <16>;
};
Documentation/devicetree/bindings/dma/fsl-imx-dma.txt deleted 100644 → 0
View file @ 7a3fad30
* Freescale Direct Memory Access (DMA) Controller for i.MX
This document will only describe differences to the generic DMA Controller and
DMA request bindings as described in dma/dma.txt .
* DMA controller
Required properties:
- compatible : Should be "fsl,<chip>-dma". chip can be imx1, imx21 or imx27
- reg : Should contain DMA registers location and length
- interrupts : First item should be DMA interrupt, second one is optional and
should contain DMA Error interrupt
- #dma-cells : Has to be 1. imx-dma does not support anything else.
Optional properties:
- dma-channels : Number of DMA channels supported. Should be 16.
- #dma-channels : deprecated
- dma-requests : Number of DMA requests supported.
- #dma-requests : deprecated
Example:
dma: dma@10001000 {
compatible = "fsl,imx27-dma";
reg = <0x10001000 0x1000>;
interrupts = <32 33>;
#dma-cells = <1>;
dma-channels = <16>;
};
* DMA client
Clients have to specify the DMA requests with phandles in a list.
Required properties:
- dmas: List of one or more DMA request specifiers. One DMA request specifier
consists of a phandle to the DMA controller followed by the integer
specifying the request line.
- dma-names: List of string identifiers for the DMA requests. For the correct
names, have a look at the specific client driver.
Example:
sdhci1: sdhci@10013000 {
...
dmas = <&dma 7>;
dma-names = "rx-tx";
...
};
Documentation/devicetree/bindings/dma/fsl-qdma.txt deleted 100644 → 0
View file @ 7a3fad30
NXP Layerscape SoC qDMA Controller
==================================
This device follows the generic DMA bindings defined in dma/dma.txt.
Required properties:
- compatible: Must be one of
"fsl,ls1021a-qdma": for LS1021A Board
"fsl,ls1028a-qdma": for LS1028A Board
"fsl,ls1043a-qdma": for ls1043A Board
"fsl,ls1046a-qdma": for ls1046A Board
- reg: Should contain the register's base address and length.
- interrupts: Should contain a reference to the interrupt used by this
device.
- interrupt-names: Should contain interrupt names:
"qdma-queue0": the block0 interrupt
"qdma-queue1": the block1 interrupt
"qdma-queue2": the block2 interrupt
"qdma-queue3": the block3 interrupt
"qdma-error": the error interrupt
- fsl,dma-queues: Should contain number of queues supported.
- dma-channels: Number of DMA channels supported
- block-number: the virtual block number
- block-offset: the offset of different virtual block
- status-sizes: status queue size of per virtual block
- queue-sizes: command queue size of per virtual block, the size number
based on queues
Optional properties:
- dma-channels: Number of DMA channels supported by the controller.
- big-endian: If present registers and hardware scatter/gather descriptors
of the qDMA are implemented in big endian mode, otherwise in little
mode.
Examples:
qdma: dma-controller@8390000 {
compatible = "fsl,ls1021a-qdma";
reg = <0x0 0x8388000 0x0 0x1000>, /* Controller regs */
<0x0 0x8389000 0x0 0x1000>, /* Status regs */
<0x0 0x838a000 0x0 0x2000>; /* Block regs */
interrupts = <GIC_SPI 185 IRQ_TYPE_LEVEL_HIGH>,
<GIC_SPI 76 IRQ_TYPE_LEVEL_HIGH>,
<GIC_SPI 77 IRQ_TYPE_LEVEL_HIGH>;
interrupt-names = "qdma-error",
"qdma-queue0", "qdma-queue1";
dma-channels = <8>;
block-number = <2>;
block-offset = <0x1000>;
fsl,dma-queues = <2>;
status-sizes = <64>;
queue-sizes = <64 64>;
big-endian;
};
DMA clients must use the format described in dma/dma.txt file.
Documentation/devicetree/bindings/dma/fsl-qdma.yaml 0 → 100644
View file @ 7a46b17d
# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
%YAML 1.2
---
$id: http://devicetree.org/schemas/dma/fsl-qdma.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
title: NXP Layerscape SoC qDMA Controller
maintainers:
- Frank Li <Frank.Li@nxp.com>
properties:
compatible:
enum:
- fsl,ls1021a-qdma
- fsl,ls1028a-qdma
- fsl,ls1043a-qdma
- fsl,ls1046a-qdma
reg:
items:
- description: Controller regs
- description: Status regs
- description: Block regs
interrupts:
minItems: 2
maxItems: 5
interrupt-names:
minItems: 2
items:
- const: qdma-error
- const: qdma-queue0
- const: qdma-queue1
- const: qdma-queue2
- const: qdma-queue3
dma-channels:
minimum: 1
maximum: 64
fsl,dma-queues:
$ref: /schemas/types.yaml#/definitions/uint32
description: Should contain number of queues supported.
minimum: 1
maximum: 4
block-number:
$ref: /schemas/types.yaml#/definitions/uint32
description: the virtual block number
block-offset:
$ref: /schemas/types.yaml#/definitions/uint32
description: the offset of different virtual block
status-sizes:
$ref: /schemas/types.yaml#/definitions/uint32
description: status queue size of per virtual block
queue-sizes:
$ref: /schemas/types.yaml#/definitions/uint32-array
description:
command queue size of per virtual block, the size number
based on queues
big-endian:
$ref: /schemas/types.yaml#/definitions/flag
description:
If present registers and hardware scatter/gather descriptors
of the qDMA are implemented in big endian mode, otherwise in little
mode.
required:
- compatible
- reg
- interrupts
- interrupt-names
- fsl,dma-queues
- block-number
- block-offset
- status-sizes
- queue-sizes
allOf:
- $ref: dma-controller.yaml#
- if:
properties:
compatible:
contains:
enum:
- fsl,ls1028a-qdma
- fsl,ls1043a-qdma
- fsl,ls1046a-qdma
then:
properties:
interrupts:
minItems: 5
interrupt-names:
minItems: 5
else:
properties:
interrupts:
maxItems: 3
interrupt-names:
maxItems: 3
unevaluatedProperties: false
examples:
- |
#include <dt-bindings/interrupt-controller/arm-gic.h>
dma-controller@8390000 {
compatible = "fsl,ls1021a-qdma";
reg = <0x8388000 0x1000>, /* Controller regs */
<0x8389000 0x1000>, /* Status regs */
<0x838a000 0x2000>; /* Block regs */
interrupts = <GIC_SPI 185 IRQ_TYPE_LEVEL_HIGH>,
<GIC_SPI 76 IRQ_TYPE_LEVEL_HIGH>,
<GIC_SPI 77 IRQ_TYPE_LEVEL_HIGH>;
interrupt-names = "qdma-error", "qdma-queue0", "qdma-queue1";
#dma-cells = <1>;
dma-channels = <8>;
block-number = <2>;
block-offset = <0x1000>;
status-sizes = <64>;
queue-sizes = <64 64>;
big-endian;
fsl,dma-queues = <2>;
};
Documentation/devicetree/bindings/dma/sprd,sc9860-dma.yaml 0 → 100644
View file @ 7a46b17d
# SPDX-License-Identifier: GPL-2.0-only OR BSD-2-Clause
%YAML 1.2
---
$id: http://devicetree.org/schemas/dma/sprd,sc9860-dma.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
title: Spreadtrum SC9860 DMA controller
description: |
There are three DMA controllers: AP DMA, AON DMA and AGCP DMA. For AGCP
DMA controller, it can or do not request the IRQ, which will save
system power without resuming system by DMA interrupts if AGCP DMA
does not request the IRQ.
maintainers:
- Orson Zhai <orsonzhai@gmail.com>
- Baolin Wang <baolin.wang7@gmail.com>
- Chunyan Zhang <zhang.lyra@gmail.com>
properties:
compatible:
const: sprd,sc9860-dma
reg:
maxItems: 1
interrupts:
maxItems: 1
clocks:
minItems: 1
items:
- description: DMA enable clock
- description: optional ashb_eb clock, only for the AGCP DMA controller
clock-names:
minItems: 1
items:
- const: enable
- const: ashb_eb
'#dma-cells':
const: 1
dma-channels:
const: 32
'#dma-channels':
const: 32
deprecated: true
required:
- compatible
- reg
- clocks
- clock-names
- '#dma-cells'
- dma-channels
allOf:
- $ref: dma-controller.yaml#
unevaluatedProperties: false
examples:
- |
#include <dt-bindings/clock/sprd,sc9860-clk.h>
#include <dt-bindings/interrupt-controller/arm-gic.h>
#include <dt-bindings/interrupt-controller/irq.h>
/* AP DMA controller */
dma-controller@20100000 {
compatible = "sprd,sc9860-dma";
reg = <0x20100000 0x4000>;
interrupts = <GIC_SPI 42 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&apahb_gate CLK_DMA_EB>;
clock-names = "enable";
#dma-cells = <1>;
dma-channels = <32>;
};
/* AGCP DMA controller */
dma-controller@41580000 {
compatible = "sprd,sc9860-dma";
reg = <0x41580000 0x4000>;
clocks = <&agcp_gate CLK_AGCP_DMAAP_EB>,
<&agcp_gate CLK_AGCP_AP_ASHB_EB>;
clock-names = "enable", "ashb_eb";
#dma-cells = <1>;
dma-channels = <32>;
};
...
Documentation/devicetree/bindings/dma/sprd-dma.txt deleted 100644 → 0
View file @ 7a3fad30
* Spreadtrum DMA controller
This binding follows the generic DMA bindings defined in dma.txt.
Required properties:
- compatible: Should be "sprd,sc9860-dma".
- reg: Should contain DMA registers location and length.
- interrupts: Should contain one interrupt shared by all channel.
- #dma-cells: must be <1>. Used to represent the number of integer
cells in the dmas property of client device.
- dma-channels : Number of DMA channels supported. Should be 32.
- clock-names: Should contain the clock of the DMA controller.
- clocks: Should contain a clock specifier for each entry in clock-names.
Deprecated properties:
- #dma-channels : Number of DMA channels supported. Should be 32.
Example:
Controller:
apdma: dma-controller@20100000 {
compatible = "sprd,sc9860-dma";
reg = <0x20100000 0x4000>;
interrupts = <GIC_SPI 50 IRQ_TYPE_LEVEL_HIGH>;
#dma-cells = <1>;
dma-channels = <32>;
clock-names = "enable";
clocks = <&clk_ap_ahb_gates 5>;
};
Client:
DMA clients connected to the Spreadtrum DMA controller must use the format
described in the dma.txt file, using a two-cell specifier for each channel.
The two cells in order are:
1. A phandle pointing to the DMA controller.
2. The slave id.
spi0: spi@70a00000{
...
dma-names = "rx_chn", "tx_chn";
dmas = <&apdma 11>, <&apdma 12>;
...
};
Documentation/devicetree/bindings/dma/st,stm32-dma.yaml → Documentation/devicetree/bindings/dma/stm32/st,stm32-dma.yaml
View file @ 7a46b17d
# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
%YAML 1.2
---
$id: http://devicetree.org/schemas/dma/st,stm32-dma.yaml#
$id: http://devicetree.org/schemas/dma/stm32/st,stm32-dma.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
title: STMicroelectronics STM32 DMA Controller
......@@ -53,7 +53,7 @@ maintainers:
- Amelie Delaunay <amelie.delaunay@foss.st.com>
allOf:
- $ref: dma-controller.yaml#
- $ref: /schemas/dma/dma-controller.yaml#
properties:
"#dma-cells":
......
Documentation/devicetree/bindings/dma/stm32/st,stm32-dma3.yaml 0 → 100644
View file @ 7a46b17d
# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
%YAML 1.2
---
$id: http://devicetree.org/schemas/dma/stm32/st,stm32-dma3.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
title: STMicroelectronics STM32 DMA3 Controller
description: |
The STM32 DMA3 is a direct memory access controller with different features
depending on its hardware configuration.
It is either called LPDMA (Low Power), GPDMA (General Purpose) or HPDMA (High
Performance).
Its hardware configuration registers allow to dynamically expose its features.
GPDMA and HPDMA support 16 independent DMA channels, while only 4 for LPDMA.
GPDMA and HPDMA support 256 DMA requests from peripherals, 8 for LPDMA.
Bindings are generic for these 3 STM32 DMA3 configurations.
DMA clients connected to the STM32 DMA3 controller must use the format
described in "#dma-cells" property description below, using a three-cell
specifier for each channel.
maintainers:
- Amelie Delaunay <amelie.delaunay@foss.st.com>
allOf:
- $ref: /schemas/dma/dma-controller.yaml#
properties:
compatible:
const: st,stm32mp25-dma3
reg:
maxItems: 1
interrupts:
minItems: 4
maxItems: 16
description:
Should contain all of the per-channel DMA interrupts in ascending order
with respect to the DMA channel index.
clocks:
maxItems: 1
resets:
maxItems: 1
power-domains:
maxItems: 1
"#dma-cells":
const: 3
description: |
Specifies the number of cells needed to provide DMA controller specific
information.
The first cell is the request line number.
The second cell is a 32-bit mask specifying the DMA channel requirements:
-bit 0-1: The priority level
0x0: low priority, low weight
0x1: low priority, mid weight
0x2: low priority, high weight
0x3: high priority
-bit 4-7: The FIFO requirement for queuing source/destination transfers
0x0: no FIFO requirement/any channel can fit
0x2: FIFO of 8 bytes (2^2+1)
0x4: FIFO of 32 bytes (2^4+1)
0x6: FIFO of 128 bytes (2^6+1)
0x7: FIFO of 256 bytes (2^7+1)
The third cell is a 32-bit mask specifying the DMA transfer requirements:
-bit 0: The source incrementing burst
0x0: fixed burst
0x1: contiguously incremented burst
-bit 1: The source allocated port
0x0: port 0 is allocated to the source transfer
0x1: port 1 is allocated to the source transfer
-bit 4: The destination incrementing burst
0x0: fixed burst
0x1: contiguously incremented burst
-bit 5: The destination allocated port
0x0: port 0 is allocated to the destination transfer
0x1: port 1 is allocated to the destination transfer
-bit 8: The type of hardware request
0x0: burst
0x1: block
-bit 9: The control mode
0x0: DMA controller control mode
0x1: peripheral control mode
-bit 12-13: The transfer complete event mode
0x0: at block level, transfer complete event is generated at the end
of a block
0x2: at LLI level, the transfer complete event is generated at the end
of the LLI transfer
including the update of the LLI if any
0x3: at channel level, the transfer complete event is generated at the
end of the last LLI
required:
- compatible
- reg
- interrupts
- clocks
- "#dma-cells"
unevaluatedProperties: false
examples:
- |
#include <dt-bindings/interrupt-controller/arm-gic.h>
#include <dt-bindings/clock/st,stm32mp25-rcc.h>
dma-controller@40400000 {
compatible = "st,stm32mp25-dma3";
reg = <0x40400000 0x1000>;
interrupts = <GIC_SPI 33 IRQ_TYPE_LEVEL_HIGH>,
<GIC_SPI 34 IRQ_TYPE_LEVEL_HIGH>,
<GIC_SPI 35 IRQ_TYPE_LEVEL_HIGH>,
<GIC_SPI 36 IRQ_TYPE_LEVEL_HIGH>,
<GIC_SPI 37 IRQ_TYPE_LEVEL_HIGH>,
<GIC_SPI 38 IRQ_TYPE_LEVEL_HIGH>,
<GIC_SPI 39 IRQ_TYPE_LEVEL_HIGH>,
<GIC_SPI 40 IRQ_TYPE_LEVEL_HIGH>,
<GIC_SPI 41 IRQ_TYPE_LEVEL_HIGH>,
<GIC_SPI 42 IRQ_TYPE_LEVEL_HIGH>,
<GIC_SPI 43 IRQ_TYPE_LEVEL_HIGH>,
<GIC_SPI 44 IRQ_TYPE_LEVEL_HIGH>,
<GIC_SPI 45 IRQ_TYPE_LEVEL_HIGH>,
<GIC_SPI 46 IRQ_TYPE_LEVEL_HIGH>,
<GIC_SPI 47 IRQ_TYPE_LEVEL_HIGH>,
<GIC_SPI 48 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&rcc CK_BUS_HPDMA1>;
#dma-cells = <3>;
};
...
Documentation/devicetree/bindings/dma/st,stm32-dmamux.yaml → Documentation/devicetree/bindings/dma/stm32/st,stm32-dmamux.yaml
View file @ 7a46b17d
# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
%YAML 1.2
---
$id: http://devicetree.org/schemas/dma/st,stm32-dmamux.yaml#
$id: http://devicetree.org/schemas/dma/stm32/st,stm32-dmamux.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
title: STMicroelectronics STM32 DMA MUX (DMA request router)
......@@ -10,7 +10,7 @@ maintainers:
- Amelie Delaunay <amelie.delaunay@foss.st.com>
allOf:
- $ref: dma-router.yaml#
- $ref: /schemas/dma/dma-router.yaml#
properties:
"#dma-cells":
......
Documentation/devicetree/bindings/dma/st,stm32-mdma.yaml → Documentation/devicetree/bindings/dma/stm32/st,stm32-mdma.yaml
View file @ 7a46b17d
# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
%YAML 1.2
---
$id: http://devicetree.org/schemas/dma/st,stm32-mdma.yaml#
$id: http://devicetree.org/schemas/dma/stm32/st,stm32-mdma.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
title: STMicroelectronics STM32 MDMA Controller
......@@ -53,7 +53,7 @@ maintainers:
- Amelie Delaunay <amelie.delaunay@foss.st.com>
allOf:
- $ref: dma-controller.yaml#
- $ref: /schemas/dma/dma-controller.yaml#
properties:
"#dma-cells":
......
Documentation/devicetree/bindings/spi/st,stm32-spi.yaml
View file @ 7a46b17d
......@@ -42,7 +42,7 @@ properties:
dmas:
description: |
DMA specifiers for tx and rx dma. DMA fifo mode must be used. See
the STM32 DMA bindings Documentation/devicetree/bindings/dma/st,stm32-dma.yaml.
the STM32 DMA controllers bindings Documentation/devicetree/bindings/dma/stm32/*.yaml.
items:
- description: rx DMA channel
- description: tx DMA channel
......
MAINTAINERS
View file @ 7a46b17d
......@@ -21839,6 +21839,15 @@ F: Documentation/devicetree/bindings/iio/adc/st,stm32-dfsdm-adc.yaml
F: Documentation/devicetree/bindings/sound/st,stm32-*.yaml
F: sound/soc/stm/
STM32 DMA DRIVERS
M: Amélie Delaunay <amelie.delaunay@foss.st.com>
L: dmaengine@vger.kernel.org
L: linux-stm32@st-md-mailman.stormreply.com (moderated for non-subscribers)
S: Maintained
F: Documentation/arch/arm/stm32/stm32-dma-mdma-chaining.rst
F: Documentation/devicetree/bindings/dma/stm32/
F: drivers/dma/stm32/
STM32 TIMER/LPTIMER DRIVERS
M: Fabrice Gasnier <fabrice.gasnier@foss.st.com>
S: Maintained
......
drivers/base/firmware_loader/main.c
View file @ 7a46b17d
......@@ -1172,34 +1172,11 @@ static void request_firmware_work_func(struct work_struct *work)
kfree(fw_work);
}
/**
* request_firmware_nowait() - asynchronous version of request_firmware
* @module: module requesting the firmware
* @uevent: sends uevent to copy the firmware image if this flag
* is non-zero else the firmware copy must be done manually.
* @name: name of firmware file
* @device: device for which firmware is being loaded
* @gfp: allocation flags
* @context: will be passed over to @cont, and
* @fw may be %NULL if firmware request fails.
* @cont: function will be called asynchronously when the firmware
* request is over.
*
* Caller must hold the reference count of @device.
*
* Asynchronous variant of request_firmware() for user contexts:
* - sleep for as small periods as possible since it may
* increase kernel boot time of built-in device drivers
* requesting firmware in their ->probe() methods, if
* @gfp is GFP_KERNEL.
*
* - can't sleep at all if @gfp is GFP_ATOMIC.
**/
int
request_firmware_nowait(
static int _request_firmware_nowait(
struct module *module, bool uevent,
const char *name, struct device *device, gfp_t gfp, void *context,
void (*cont)(const struct firmware *fw, void *context))
void (*cont)(const struct firmware *fw, void *context), bool nowarn)
{
struct firmware_work *fw_work;
......@@ -1217,7 +1194,8 @@ request_firmware_nowait(
fw_work->context = context;
fw_work->cont = cont;
fw_work->opt_flags = FW_OPT_NOWAIT |
(uevent ? FW_OPT_UEVENT : FW_OPT_USERHELPER);
(uevent ? FW_OPT_UEVENT : FW_OPT_USERHELPER) |
(nowarn ? FW_OPT_NO_WARN : 0);
if (!uevent && fw_cache_is_setup(device, name)) {
kfree_const(fw_work->name);
......@@ -1236,8 +1214,66 @@ request_firmware_nowait(
schedule_work(&fw_work->work);
return 0;
}
/**
* request_firmware_nowait() - asynchronous version of request_firmware
* @module: module requesting the firmware
* @uevent: sends uevent to copy the firmware image if this flag
* is non-zero else the firmware copy must be done manually.
* @name: name of firmware file
* @device: device for which firmware is being loaded
* @gfp: allocation flags
* @context: will be passed over to @cont, and
* @fw may be %NULL if firmware request fails.
* @cont: function will be called asynchronously when the firmware
* request is over.
*
* Caller must hold the reference count of @device.
*
* Asynchronous variant of request_firmware() for user contexts:
* - sleep for as small periods as possible since it may
* increase kernel boot time of built-in device drivers
* requesting firmware in their ->probe() methods, if
* @gfp is GFP_KERNEL.
*
* - can't sleep at all if @gfp is GFP_ATOMIC.
**/
int request_firmware_nowait(
struct module *module, bool uevent,
const char *name, struct device *device, gfp_t gfp, void *context,
void (*cont)(const struct firmware *fw, void *context))
{
return _request_firmware_nowait(module, uevent, name, device, gfp,
context, cont, false);
}
EXPORT_SYMBOL(request_firmware_nowait);
/**
* firmware_request_nowait_nowarn() - async version of request_firmware_nowarn
* @module: module requesting the firmware
* @name: name of firmware file
* @device: device for which firmware is being loaded
* @gfp: allocation flags
* @context: will be passed over to @cont, and
* @fw may be %NULL if firmware request fails.
* @cont: function will be called asynchronously when the firmware
* request is over.
*
* Similar in function to request_firmware_nowait(), but doesn't print a warning
* when the firmware file could not be found and always sends a uevent to copy
* the firmware image.
*/
int firmware_request_nowait_nowarn(
struct module *module, const char *name,
struct device *device, gfp_t gfp, void *context,
void (*cont)(const struct firmware *fw, void *context))
{
return _request_firmware_nowait(module, FW_ACTION_UEVENT, name, device,
gfp, context, cont, true);
}
EXPORT_SYMBOL_GPL(firmware_request_nowait_nowarn);
#ifdef CONFIG_FW_CACHE
static ASYNC_DOMAIN_EXCLUSIVE(fw_cache_domain);
......
drivers/dma/Kconfig
View file @ 7a46b17d
......@@ -568,38 +568,6 @@ config ST_FDMA
Say Y here if you have such a chipset.
If unsure, say N.
config STM32_DMA
bool "STMicroelectronics STM32 DMA support"
depends on ARCH_STM32 || COMPILE_TEST
select DMA_ENGINE
select DMA_VIRTUAL_CHANNELS
help
Enable support for the on-chip DMA controller on STMicroelectronics
STM32 MCUs.
If you have a board based on such a MCU and wish to use DMA say Y
here.
config STM32_DMAMUX
bool "STMicroelectronics STM32 dma multiplexer support"
depends on STM32_DMA || COMPILE_TEST
help
Enable support for the on-chip DMA multiplexer on STMicroelectronics
STM32 MCUs.
If you have a board based on such a MCU and wish to use DMAMUX say Y
here.
config STM32_MDMA
bool "STMicroelectronics STM32 master dma support"
depends on ARCH_STM32 || COMPILE_TEST
depends on OF
select DMA_ENGINE
select DMA_VIRTUAL_CHANNELS
help
Enable support for the on-chip MDMA controller on STMicroelectronics
STM32 platforms.
If you have a board based on STM32 SoC and wish to use the master DMA
say Y here.
config SPRD_DMA
tristate "Spreadtrum DMA support"
depends on ARCH_SPRD || COMPILE_TEST
......@@ -772,6 +740,8 @@ source "drivers/dma/fsl-dpaa2-qdma/Kconfig"
source "drivers/dma/lgm/Kconfig"
source "drivers/dma/stm32/Kconfig"
# clients
comment "DMA Clients"
depends on DMA_ENGINE
......
drivers/dma/Makefile
View file @ 7a46b17d
......@@ -70,9 +70,6 @@ obj-$(CONFIG_PXA_DMA) += pxa_dma.o
obj-$(CONFIG_RENESAS_DMA) += sh/
obj-$(CONFIG_SF_PDMA) += sf-pdma/
obj-$(CONFIG_STE_DMA40) += ste_dma40.o ste_dma40_ll.o
obj-$(CONFIG_STM32_DMA) += stm32-dma.o
obj-$(CONFIG_STM32_DMAMUX) += stm32-dmamux.o
obj-$(CONFIG_STM32_MDMA) += stm32-mdma.o
obj-$(CONFIG_SPRD_DMA) += sprd-dma.o
obj-$(CONFIG_TXX9_DMAC) += txx9dmac.o
obj-$(CONFIG_TEGRA186_GPC_DMA) += tegra186-gpc-dma.o
......@@ -88,5 +85,6 @@ obj-$(CONFIG_INTEL_LDMA) += lgm/
obj-y += mediatek/
obj-y += qcom/
obj-y += stm32/
obj-y += ti/
obj-y += xilinx/
drivers/dma/altera-msgdma.c
View file @ 7a46b17d
......@@ -233,7 +233,7 @@ static void msgdma_free_descriptor(struct msgdma_device *mdev,
struct msgdma_sw_desc *child, *next;
mdev->desc_free_cnt++;
list_add_tail(&desc->node, &mdev->free_list);
list_move_tail(&desc->node, &mdev->free_list);
list_for_each_entry_safe(child, next, &desc->tx_list, node) {
mdev->desc_free_cnt++;
list_move_tail(&child->node, &mdev->free_list);
......@@ -583,22 +583,25 @@ static void msgdma_issue_pending(struct dma_chan *chan)
static void msgdma_chan_desc_cleanup(struct msgdma_device *mdev)
{
struct msgdma_sw_desc *desc, *next;
unsigned long irqflags;
spin_lock_irqsave(&mdev->lock, irqflags);
list_for_each_entry_safe(desc, next, &mdev->done_list, node) {
struct dmaengine_desc_callback cb;
list_del(&desc->node);
dmaengine_desc_get_callback(&desc->async_tx, &cb);
if (dmaengine_desc_callback_valid(&cb)) {
spin_unlock(&mdev->lock);
spin_unlock_irqrestore(&mdev->lock, irqflags);
dmaengine_desc_callback_invoke(&cb, NULL);
spin_lock(&mdev->lock);
spin_lock_irqsave(&mdev->lock, irqflags);
}
/* Run any dependencies, then free the descriptor */
msgdma_free_descriptor(mdev, desc);
}
spin_unlock_irqrestore(&mdev->lock, irqflags);
}
/**
......@@ -713,10 +716,11 @@ static void msgdma_tasklet(struct tasklet_struct *t)
}
msgdma_complete_descriptor(mdev);
msgdma_chan_desc_cleanup(mdev);
}
spin_unlock_irqrestore(&mdev->lock, flags);
msgdma_chan_desc_cleanup(mdev);
}
/**
......
drivers/dma/dmaengine.c
View file @ 7a46b17d
......@@ -1037,7 +1037,8 @@ static int get_dma_id(struct dma_device *device)
}
static int __dma_async_device_channel_register(struct dma_device *device,
struct dma_chan *chan)
struct dma_chan *chan,
const char *name)
{
int rc;
......@@ -1066,8 +1067,10 @@ static int __dma_async_device_channel_register(struct dma_device *device,
chan->dev->device.parent = device->dev;
chan->dev->chan = chan;
chan->dev->dev_id = device->dev_id;
dev_set_name(&chan->dev->device, "dma%dchan%d",
device->dev_id, chan->chan_id);
if (!name)
dev_set_name(&chan->dev->device, "dma%dchan%d", device->dev_id, chan->chan_id);
else
dev_set_name(&chan->dev->device, name);
rc = device_register(&chan->dev->device);
if (rc)
goto err_out_ida;
......@@ -1087,11 +1090,12 @@ static int __dma_async_device_channel_register(struct dma_device *device,
}
int dma_async_device_channel_register(struct dma_device *device,
struct dma_chan *chan)
struct dma_chan *chan,
const char *name)
{
int rc;
rc = __dma_async_device_channel_register(device, chan);
rc = __dma_async_device_channel_register(device, chan, name);
if (rc < 0)
return rc;
......@@ -1203,7 +1207,7 @@ int dma_async_device_register(struct dma_device *device)
/* represent channels in sysfs. Probably want devs too */
list_for_each_entry(chan, &device->channels, device_node) {
rc = __dma_async_device_channel_register(device, chan);
rc = __dma_async_device_channel_register(device, chan, NULL);
if (rc < 0)
goto err_out;
}
......
drivers/dma/dmatest.c
View file @ 7a46b17d
......@@ -1372,4 +1372,5 @@ static void __exit dmatest_exit(void)
module_exit(dmatest_exit);
MODULE_AUTHOR("Haavard Skinnemoen (Atmel)");
MODULE_DESCRIPTION("DMA Engine test module");
MODULE_LICENSE("GPL v2");
drivers/dma/fsl-dpaa2-qdma/dpdmai.c
View file @ 7a46b17d
......@@ -367,4 +367,5 @@ int dpdmai_get_tx_queue(struct fsl_mc_io *mc_io, u32 cmd_flags,
}
EXPORT_SYMBOL_GPL(dpdmai_get_tx_queue);
MODULE_DESCRIPTION("NXP DPAA2 QDMA driver");
MODULE_LICENSE("GPL v2");
drivers/dma/fsl-edma-common.c
View file @ 7a46b17d
......@@ -59,7 +59,6 @@ void fsl_edma_tx_chan_handler(struct fsl_edma_chan *fsl_chan)
vchan_cookie_complete(&fsl_chan->edesc->vdesc);
fsl_chan->edesc = NULL;
fsl_chan->status = DMA_COMPLETE;
fsl_chan->idle = true;
} else {
vchan_cyclic_callback(&fsl_chan->edesc->vdesc);
}
......@@ -239,7 +238,7 @@ int fsl_edma_terminate_all(struct dma_chan *chan)
spin_lock_irqsave(&fsl_chan->vchan.lock, flags);
fsl_edma_disable_request(fsl_chan);
fsl_chan->edesc = NULL;
fsl_chan->idle = true;
fsl_chan->status = DMA_COMPLETE;
vchan_get_all_descriptors(&fsl_chan->vchan, &head);
spin_unlock_irqrestore(&fsl_chan->vchan.lock, flags);
vchan_dma_desc_free_list(&fsl_chan->vchan, &head);
......@@ -259,7 +258,6 @@ int fsl_edma_pause(struct dma_chan *chan)
if (fsl_chan->edesc) {
fsl_edma_disable_request(fsl_chan);
fsl_chan->status = DMA_PAUSED;
fsl_chan->idle = true;
}
spin_unlock_irqrestore(&fsl_chan->vchan.lock, flags);
return 0;
......@@ -274,7 +272,6 @@ int fsl_edma_resume(struct dma_chan *chan)
if (fsl_chan->edesc) {
fsl_edma_enable_request(fsl_chan);
fsl_chan->status = DMA_IN_PROGRESS;
fsl_chan->idle = false;
}
spin_unlock_irqrestore(&fsl_chan->vchan.lock, flags);
return 0;
......@@ -758,6 +755,8 @@ struct dma_async_tx_descriptor *fsl_edma_prep_memcpy(struct dma_chan *chan,
fsl_desc->iscyclic = false;
fsl_chan->is_sw = true;
if (fsl_edma_drvflags(fsl_chan) & FSL_EDMA_DRV_MEM_REMOTE)
fsl_chan->is_remote = true;
/* To match with copy_align and max_seg_size so 1 tcd is enough */
fsl_edma_fill_tcd(fsl_chan, fsl_desc->tcd[0].vtcd, dma_src, dma_dst,
......@@ -780,7 +779,6 @@ void fsl_edma_xfer_desc(struct fsl_edma_chan *fsl_chan)
fsl_edma_set_tcd_regs(fsl_chan, fsl_chan->edesc->tcd[0].vtcd);
fsl_edma_enable_request(fsl_chan);
fsl_chan->status = DMA_IN_PROGRESS;
fsl_chan->idle = false;
}
void fsl_edma_issue_pending(struct dma_chan *chan)
......@@ -805,6 +803,7 @@ void fsl_edma_issue_pending(struct dma_chan *chan)
int fsl_edma_alloc_chan_resources(struct dma_chan *chan)
{
struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan);
int ret;
if (fsl_edma_drvflags(fsl_chan) & FSL_EDMA_DRV_HAS_CHCLK)
clk_prepare_enable(fsl_chan->clk);
......@@ -813,6 +812,17 @@ int fsl_edma_alloc_chan_resources(struct dma_chan *chan)
fsl_edma_drvflags(fsl_chan) & FSL_EDMA_DRV_TCD64 ?
sizeof(struct fsl_edma_hw_tcd64) : sizeof(struct fsl_edma_hw_tcd),
32, 0);
if (fsl_chan->txirq) {
ret = request_irq(fsl_chan->txirq, fsl_chan->irq_handler, IRQF_SHARED,
fsl_chan->chan_name, fsl_chan);
if (ret) {
dma_pool_destroy(fsl_chan->tcd_pool);
return ret;
}
}
return 0;
}
......@@ -832,11 +842,15 @@ void fsl_edma_free_chan_resources(struct dma_chan *chan)
fsl_edma_unprep_slave_dma(fsl_chan);
spin_unlock_irqrestore(&fsl_chan->vchan.lock, flags);
if (fsl_chan->txirq)
free_irq(fsl_chan->txirq, fsl_chan);
vchan_dma_desc_free_list(&fsl_chan->vchan, &head);
dma_pool_destroy(fsl_chan->tcd_pool);
fsl_chan->tcd_pool = NULL;
fsl_chan->is_sw = false;
fsl_chan->srcid = 0;
fsl_chan->is_remote = false;
if (fsl_edma_drvflags(fsl_chan) & FSL_EDMA_DRV_HAS_CHCLK)
clk_disable_unprepare(fsl_chan->clk);
}
......
drivers/dma/fsl-edma-common.h
View file @ 7a46b17d
......@@ -150,7 +150,6 @@ struct fsl_edma_chan {
struct virt_dma_chan vchan;
enum dma_status status;
enum fsl_edma_pm_state pm_state;
bool idle;
struct fsl_edma_engine *edma;
struct fsl_edma_desc *edesc;
struct dma_slave_config cfg;
......@@ -172,6 +171,7 @@ struct fsl_edma_chan {
int priority;
int hw_chanid;
int txirq;
irqreturn_t (*irq_handler)(int irq, void *dev_id);
bool is_rxchan;
bool is_remote;
bool is_multi_fifo;
......@@ -194,6 +194,7 @@ struct fsl_edma_desc {
#define FSL_EDMA_DRV_HAS_PD BIT(5)
#define FSL_EDMA_DRV_HAS_CHCLK BIT(6)
#define FSL_EDMA_DRV_HAS_CHMUX BIT(7)
#define FSL_EDMA_DRV_MEM_REMOTE BIT(8)
/* control and status register is in tcd address space, edma3 reg layout */
#define FSL_EDMA_DRV_SPLIT_REG BIT(9)
#define FSL_EDMA_DRV_BUS_8BYTE BIT(10)
......@@ -455,7 +456,6 @@ static inline struct fsl_edma_desc *to_fsl_edma_desc(struct virt_dma_desc *vd)
static inline void fsl_edma_err_chan_handler(struct fsl_edma_chan *fsl_chan)
{
fsl_chan->status = DMA_ERROR;
fsl_chan->idle = true;
}
void fsl_edma_tx_chan_handler(struct fsl_edma_chan *fsl_chan);
......
drivers/dma/fsl-edma-main.c
View file @ 7a46b17d
......@@ -65,6 +65,13 @@ static irqreturn_t fsl_edma3_tx_handler(int irq, void *dev_id)
return IRQ_HANDLED;
}
static irqreturn_t fsl_edma2_tx_handler(int irq, void *devi_id)
{
struct fsl_edma_chan *fsl_chan = devi_id;
return fsl_edma_tx_handler(irq, fsl_chan->edma);
}
static irqreturn_t fsl_edma_err_handler(int irq, void *dev_id)
{
struct fsl_edma_engine *fsl_edma = dev_id;
......@@ -228,7 +235,6 @@ fsl_edma_irq_init(struct platform_device *pdev, struct fsl_edma_engine *fsl_edma
static int fsl_edma3_irq_init(struct platform_device *pdev, struct fsl_edma_engine *fsl_edma)
{
int ret;
int i;
for (i = 0; i < fsl_edma->n_chans; i++) {
......@@ -243,13 +249,7 @@ static int fsl_edma3_irq_init(struct platform_device *pdev, struct fsl_edma_engi
if (fsl_chan->txirq < 0)
return -EINVAL;
ret = devm_request_irq(&pdev->dev, fsl_chan->txirq,
fsl_edma3_tx_handler, IRQF_SHARED,
fsl_chan->chan_name, fsl_chan);
if (ret) {
dev_err(&pdev->dev, "Can't register chan%d's IRQ.\n", i);
return -EINVAL;
}
fsl_chan->irq_handler = fsl_edma3_tx_handler;
}
return 0;
......@@ -278,19 +278,20 @@ fsl_edma2_irq_init(struct platform_device *pdev,
*/
for (i = 0; i < count; i++) {
irq = platform_get_irq(pdev, i);
ret = 0;
if (irq < 0)
return -ENXIO;
/* The last IRQ is for eDMA err */
if (i == count - 1)
if (i == count - 1) {
ret = devm_request_irq(&pdev->dev, irq,
fsl_edma_err_handler,
0, "eDMA2-ERR", fsl_edma);
else
ret = devm_request_irq(&pdev->dev, irq,
fsl_edma_tx_handler, 0,
fsl_edma->chans[i].chan_name,
fsl_edma);
} else {
fsl_edma->chans[i].txirq = irq;
fsl_edma->chans[i].irq_handler = fsl_edma2_tx_handler;
}
if (ret)
return ret;
}
......@@ -342,7 +343,7 @@ static struct fsl_edma_drvdata imx7ulp_data = {
};
static struct fsl_edma_drvdata imx8qm_data = {
.flags = FSL_EDMA_DRV_HAS_PD | FSL_EDMA_DRV_EDMA3,
.flags = FSL_EDMA_DRV_HAS_PD | FSL_EDMA_DRV_EDMA3 | FSL_EDMA_DRV_MEM_REMOTE,
.chreg_space_sz = 0x10000,
.chreg_off = 0x10000,
.setup_irq = fsl_edma3_irq_init,
......@@ -543,7 +544,6 @@ static int fsl_edma_probe(struct platform_device *pdev)
fsl_chan->edma = fsl_edma;
fsl_chan->pm_state = RUNNING;
fsl_chan->srcid = 0;
fsl_chan->idle = true;
fsl_chan->dma_dir = DMA_NONE;
fsl_chan->vchan.desc_free = fsl_edma_free_desc;
......@@ -668,7 +668,7 @@ static int fsl_edma_suspend_late(struct device *dev)
continue;
spin_lock_irqsave(&fsl_chan->vchan.lock, flags);
/* Make sure chan is idle or will force disable. */
if (unlikely(!fsl_chan->idle)) {
if (unlikely(fsl_chan->status == DMA_IN_PROGRESS)) {
dev_warn(dev, "WARN: There is non-idle channel.");
fsl_edma_disable_request(fsl_chan);
fsl_edma_chan_mux(fsl_chan, 0, false);
......
drivers/dma/idxd/dma.c
View file @ 7a46b17d
......@@ -269,7 +269,7 @@ static int idxd_register_dma_channel(struct idxd_wq *wq)
desc->txd.tx_submit = idxd_dma_tx_submit;
}
rc = dma_async_device_channel_register(dma, chan);
rc = dma_async_device_channel_register(dma, chan, NULL);
if (rc < 0) {
kfree(idxd_chan);
return rc;
......
drivers/dma/idxd/init.c
View file @ 7a46b17d
......@@ -22,6 +22,7 @@
#include "perfmon.h"
MODULE_VERSION(IDXD_DRIVER_VERSION);
MODULE_DESCRIPTION("Intel Data Streaming Accelerator and In-Memory Analytics Accelerator common driver");
MODULE_LICENSE("GPL v2");
MODULE_AUTHOR("Intel Corporation");
MODULE_IMPORT_NS(IDXD);
......
drivers/dma/imx-sdma.c
View file @ 7a46b17d
......@@ -2107,9 +2107,8 @@ static int sdma_get_firmware(struct sdma_engine *sdma,
{
int ret;
ret = request_firmware_nowait(THIS_MODULE,
FW_ACTION_UEVENT, fw_name, sdma->dev,
GFP_KERNEL, sdma, sdma_load_firmware);
ret = firmware_request_nowait_nowarn(THIS_MODULE, fw_name, sdma->dev,
GFP_KERNEL, sdma, sdma_load_firmware);
return ret;
}
......
drivers/dma/ioat/init.c
View file @ 7a46b17d
......@@ -23,6 +23,7 @@
#include "../dmaengine.h"
MODULE_VERSION(IOAT_DMA_VERSION);
MODULE_DESCRIPTION("Intel I/OAT DMA Linux driver");
MODULE_LICENSE("Dual BSD/GPL");
MODULE_AUTHOR("Intel Corporation");
......
drivers/dma/mcf-edma-main.c
View file @ 7a46b17d
......@@ -64,7 +64,6 @@ static irqreturn_t mcf_edma_err_handler(int irq, void *dev_id)
fsl_edma_disable_request(&mcf_edma->chans[ch]);
iowrite8(EDMA_CERR_CERR(ch), regs->cerr);
mcf_edma->chans[ch].status = DMA_ERROR;
mcf_edma->chans[ch].idle = true;
}
}
......@@ -196,7 +195,6 @@ static int mcf_edma_probe(struct platform_device *pdev)
mcf_chan->edma = mcf_edma;
mcf_chan->srcid = i;
mcf_chan->idle = true;
mcf_chan->dma_dir = DMA_NONE;
mcf_chan->vchan.desc_free = fsl_edma_free_desc;
vchan_init(&mcf_chan->vchan, &mcf_edma->dma_dev);
......
drivers/dma/moxart-dma.c
View file @ 7a46b17d
......@@ -148,11 +148,6 @@ struct moxart_dmadev {
unsigned int irq;
};
struct moxart_filter_data {
struct moxart_dmadev *mdc;
struct of_phandle_args *dma_spec;
};
static const unsigned int es_bytes[] = {
[MOXART_DMA_DATA_TYPE_S8] = 1,
[MOXART_DMA_DATA_TYPE_S16] = 2,
......
drivers/dma/qcom/gpi.c
View file @ 7a46b17d
......@@ -476,12 +476,6 @@ struct gpi_dev {
struct gpii *gpiis;
};
struct reg_info {
char *name;
u32 offset;
u32 val;
};
struct gchan {
struct virt_dma_chan vc;
u32 chid;
......@@ -1197,7 +1191,6 @@ static int gpi_reset_chan(struct gchan *gchan, enum gpi_cmd gpi_cmd)
{
struct gpii *gpii = gchan->gpii;
struct gpi_ring *ch_ring = &gchan->ch_ring;
unsigned long flags;
LIST_HEAD(list);
int ret;
......@@ -1220,9 +1213,9 @@ static int gpi_reset_chan(struct gchan *gchan, enum gpi_cmd gpi_cmd)
gpi_mark_stale_events(gchan);
/* remove all async descriptors */
spin_lock_irqsave(&gchan->vc.lock, flags);
spin_lock(&gchan->vc.lock);
vchan_get_all_descriptors(&gchan->vc, &list);
spin_unlock_irqrestore(&gchan->vc.lock, flags);
spin_unlock(&gchan->vc.lock);
write_unlock_irq(&gpii->pm_lock);
vchan_dma_desc_free_list(&gchan->vc, &list);
......
drivers/dma/qcom/hidma.c
View file @ 7a46b17d
......@@ -957,4 +957,5 @@ static struct platform_driver hidma_driver = {
};
module_platform_driver(hidma_driver);
MODULE_DESCRIPTION("Qualcomm Technologies HIDMA Channel support");
MODULE_LICENSE("GPL v2");
drivers/dma/qcom/hidma_mgmt.c
View file @ 7a46b17d
......@@ -331,4 +331,5 @@ static struct platform_driver hidma_mgmt_driver = {
};
module_platform_driver(hidma_mgmt_driver);
MODULE_DESCRIPTION("Qualcomm Technologies HIDMA DMA engine interface");
MODULE_LICENSE("GPL v2");
drivers/dma/sh/rz-dmac.c
View file @ 7a46b17d
......@@ -540,8 +540,8 @@ static int rz_dmac_terminate_all(struct dma_chan *chan)
spin_lock_irqsave(&channel->vc.lock, flags);
list_splice_tail_init(&channel->ld_active, &channel->ld_free);
list_splice_tail_init(&channel->ld_queue, &channel->ld_free);
spin_unlock_irqrestore(&channel->vc.lock, flags);
vchan_get_all_descriptors(&channel->vc, &head);
spin_unlock_irqrestore(&channel->vc.lock, flags);
vchan_dma_desc_free_list(&channel->vc, &head);
return 0;
......
drivers/dma/stm32/Kconfig 0 → 100644
View file @ 7a46b17d
# SPDX-License-Identifier: GPL-2.0-only
#
# STM32 DMA controllers drivers
#
if ARCH_STM32 || COMPILE_TEST
config STM32_DMA
bool "STMicroelectronics STM32 DMA support"
select DMA_ENGINE
select DMA_VIRTUAL_CHANNELS
help
Enable support for the on-chip DMA controller on STMicroelectronics
STM32 platforms.
If you have a board based on STM32 SoC with such DMA controller
and want to use DMA say Y here.
config STM32_DMAMUX
bool "STMicroelectronics STM32 DMA multiplexer support"
depends on STM32_DMA
help
Enable support for the on-chip DMA multiplexer on STMicroelectronics
STM32 platforms.
If you have a board based on STM32 SoC with such DMA multiplexer
and want to use DMAMUX say Y here.
config STM32_MDMA
bool "STMicroelectronics STM32 master DMA support"
depends on OF
select DMA_ENGINE
select DMA_VIRTUAL_CHANNELS
help
Enable support for the on-chip MDMA controller on STMicroelectronics
STM32 platforms.
If you have a board based on STM32 SoC with such DMA controller
and want to use MDMA say Y here.
config STM32_DMA3
tristate "STMicroelectronics STM32 DMA3 support"
select DMA_ENGINE
select DMA_VIRTUAL_CHANNELS
help
Enable support for the on-chip DMA3 controller on STMicroelectronics
STM32 platforms.
If you have a board based on STM32 SoC with such DMA3 controller
and want to use DMA3, say Y here.
endif
drivers/dma/stm32/Makefile 0 → 100644
View file @ 7a46b17d
# SPDX-License-Identifier: GPL-2.0-only
obj-$(CONFIG_STM32_DMA) += stm32-dma.o
obj-$(CONFIG_STM32_DMAMUX) += stm32-dmamux.o
obj-$(CONFIG_STM32_MDMA) += stm32-mdma.o
obj-$(CONFIG_STM32_DMA3) += stm32-dma3.o
drivers/dma/stm32-dma.c → drivers/dma/stm32/stm32-dma.c
View file @ 7a46b17d
......@@ -28,7 +28,7 @@
#include <linux/sched.h>
#include <linux/slab.h>
#include "virt-dma.h"
#include "../virt-dma.h"
#define STM32_DMA_LISR 0x0000 /* DMA Low Int Status Reg */
#define STM32_DMA_HISR 0x0004 /* DMA High Int Status Reg */
......
drivers/dma/stm32/stm32-dma3.c 0 → 100644
View file @ 7a46b17d
// SPDX-License-Identifier: GPL-2.0-only
/*
* STM32 DMA3 controller driver
*
* Copyright (C) STMicroelectronics 2024
* Author(s): Amelie Delaunay <amelie.delaunay@foss.st.com>
*/
#include <linux/bitfield.h>
#include <linux/clk.h>
#include <linux/dma-mapping.h>
#include <linux/dmaengine.h>
#include <linux/dmapool.h>
#include <linux/init.h>
#include <linux/iopoll.h>
#include <linux/list.h>
#include <linux/module.h>
#include <linux/of_dma.h>
#include <linux/platform_device.h>
#include <linux/pm_runtime.h>
#include <linux/reset.h>
#include <linux/slab.h>
#include "../virt-dma.h"
#define STM32_DMA3_SECCFGR 0x00
#define STM32_DMA3_PRIVCFGR 0x04
#define STM32_DMA3_RCFGLOCKR 0x08
#define STM32_DMA3_MISR 0x0c
#define STM32_DMA3_SMISR 0x10
#define STM32_DMA3_CLBAR(x) (0x50 + 0x80 * (x))
#define STM32_DMA3_CCIDCFGR(x) (0x54 + 0x80 * (x))
#define STM32_DMA3_CSEMCR(x) (0x58 + 0x80 * (x))
#define STM32_DMA3_CFCR(x) (0x5c + 0x80 * (x))
#define STM32_DMA3_CSR(x) (0x60 + 0x80 * (x))
#define STM32_DMA3_CCR(x) (0x64 + 0x80 * (x))
#define STM32_DMA3_CTR1(x) (0x90 + 0x80 * (x))
#define STM32_DMA3_CTR2(x) (0x94 + 0x80 * (x))
#define STM32_DMA3_CBR1(x) (0x98 + 0x80 * (x))
#define STM32_DMA3_CSAR(x) (0x9c + 0x80 * (x))
#define STM32_DMA3_CDAR(x) (0xa0 + 0x80 * (x))
#define STM32_DMA3_CLLR(x) (0xcc + 0x80 * (x))
#define STM32_DMA3_HWCFGR13 0xfc0 /* G_PER_CTRL(X) x=8..15 */
#define STM32_DMA3_HWCFGR12 0xfc4 /* G_PER_CTRL(X) x=0..7 */
#define STM32_DMA3_HWCFGR4 0xfe4 /* G_FIFO_SIZE(X) x=8..15 */
#define STM32_DMA3_HWCFGR3 0xfe8 /* G_FIFO_SIZE(X) x=0..7 */
#define STM32_DMA3_HWCFGR2 0xfec /* G_MAX_REQ_ID */
#define STM32_DMA3_HWCFGR1 0xff0 /* G_MASTER_PORTS, G_NUM_CHANNELS, G_Mx_DATA_WIDTH */
#define STM32_DMA3_VERR 0xff4
/* SECCFGR DMA secure configuration register */
#define SECCFGR_SEC(x) BIT(x)
/* MISR DMA non-secure/secure masked interrupt status register */
#define MISR_MIS(x) BIT(x)
/* CxLBAR DMA channel x linked_list base address register */
#define CLBAR_LBA GENMASK(31, 16)
/* CxCIDCFGR DMA channel x CID register */
#define CCIDCFGR_CFEN BIT(0)
#define CCIDCFGR_SEM_EN BIT(1)
#define CCIDCFGR_SCID GENMASK(5, 4)
#define CCIDCFGR_SEM_WLIST_CID0 BIT(16)
#define CCIDCFGR_SEM_WLIST_CID1 BIT(17)
#define CCIDCFGR_SEM_WLIST_CID2 BIT(18)
enum ccidcfgr_cid {
CCIDCFGR_CID0,
CCIDCFGR_CID1,
CCIDCFGR_CID2,
};
/* CxSEMCR DMA channel x semaphore control register */
#define CSEMCR_SEM_MUTEX BIT(0)
#define CSEMCR_SEM_CCID GENMASK(5, 4)
/* CxFCR DMA channel x flag clear register */
#define CFCR_TCF BIT(8)
#define CFCR_HTF BIT(9)
#define CFCR_DTEF BIT(10)
#define CFCR_ULEF BIT(11)
#define CFCR_USEF BIT(12)
#define CFCR_SUSPF BIT(13)
/* CxSR DMA channel x status register */
#define CSR_IDLEF BIT(0)
#define CSR_TCF BIT(8)
#define CSR_HTF BIT(9)
#define CSR_DTEF BIT(10)
#define CSR_ULEF BIT(11)
#define CSR_USEF BIT(12)
#define CSR_SUSPF BIT(13)
#define CSR_ALL_F GENMASK(13, 8)
#define CSR_FIFOL GENMASK(24, 16)
/* CxCR DMA channel x control register */
#define CCR_EN BIT(0)
#define CCR_RESET BIT(1)
#define CCR_SUSP BIT(2)
#define CCR_TCIE BIT(8)
#define CCR_HTIE BIT(9)
#define CCR_DTEIE BIT(10)
#define CCR_ULEIE BIT(11)
#define CCR_USEIE BIT(12)
#define CCR_SUSPIE BIT(13)
#define CCR_ALLIE GENMASK(13, 8)
#define CCR_LSM BIT(16)
#define CCR_LAP BIT(17)
#define CCR_PRIO GENMASK(23, 22)
enum ccr_prio {
CCR_PRIO_LOW,
CCR_PRIO_MID,
CCR_PRIO_HIGH,
CCR_PRIO_VERY_HIGH,
};
/* CxTR1 DMA channel x transfer register 1 */
#define CTR1_SINC BIT(3)
#define CTR1_SBL_1 GENMASK(9, 4)
#define CTR1_DINC BIT(19)
#define CTR1_DBL_1 GENMASK(25, 20)
#define CTR1_SDW_LOG2 GENMASK(1, 0)
#define CTR1_PAM GENMASK(12, 11)
#define CTR1_SAP BIT(14)
#define CTR1_DDW_LOG2 GENMASK(17, 16)
#define CTR1_DAP BIT(30)
enum ctr1_dw {
CTR1_DW_BYTE,
CTR1_DW_HWORD,
CTR1_DW_WORD,
CTR1_DW_DWORD, /* Depends on HWCFGR1.G_M0_DATA_WIDTH_ENC and .G_M1_DATA_WIDTH_ENC */
};
enum ctr1_pam {
CTR1_PAM_0S_LT, /* if DDW > SDW, padded with 0s else left-truncated */
CTR1_PAM_SE_RT, /* if DDW > SDW, sign extended else right-truncated */
CTR1_PAM_PACK_UNPACK, /* FIFO queued */
};
/* CxTR2 DMA channel x transfer register 2 */
#define CTR2_REQSEL GENMASK(7, 0)
#define CTR2_SWREQ BIT(9)
#define CTR2_DREQ BIT(10)
#define CTR2_BREQ BIT(11)
#define CTR2_PFREQ BIT(12)
#define CTR2_TCEM GENMASK(31, 30)
enum ctr2_tcem {
CTR2_TCEM_BLOCK,
CTR2_TCEM_REPEAT_BLOCK,
CTR2_TCEM_LLI,
CTR2_TCEM_CHANNEL,
};
/* CxBR1 DMA channel x block register 1 */
#define CBR1_BNDT GENMASK(15, 0)
/* CxLLR DMA channel x linked-list address register */
#define CLLR_LA GENMASK(15, 2)
#define CLLR_ULL BIT(16)
#define CLLR_UDA BIT(27)
#define CLLR_USA BIT(28)
#define CLLR_UB1 BIT(29)
#define CLLR_UT2 BIT(30)
#define CLLR_UT1 BIT(31)
/* HWCFGR13 DMA hardware configuration register 13 x=8..15 */
/* HWCFGR12 DMA hardware configuration register 12 x=0..7 */
#define G_PER_CTRL(x) (ULL(0x1) << (4 * (x)))
/* HWCFGR4 DMA hardware configuration register 4 x=8..15 */
/* HWCFGR3 DMA hardware configuration register 3 x=0..7 */
#define G_FIFO_SIZE(x) (ULL(0x7) << (4 * (x)))
#define get_chan_hwcfg(x, mask, reg) (((reg) & (mask)) >> (4 * (x)))
/* HWCFGR2 DMA hardware configuration register 2 */
#define G_MAX_REQ_ID GENMASK(7, 0)
/* HWCFGR1 DMA hardware configuration register 1 */
#define G_MASTER_PORTS GENMASK(2, 0)
#define G_NUM_CHANNELS GENMASK(12, 8)
#define G_M0_DATA_WIDTH_ENC GENMASK(25, 24)
#define G_M1_DATA_WIDTH_ENC GENMASK(29, 28)
enum stm32_dma3_master_ports {
AXI64, /* 1x AXI: 64-bit port 0 */
AHB32, /* 1x AHB: 32-bit port 0 */
AHB32_AHB32, /* 2x AHB: 32-bit port 0 and 32-bit port 1 */
AXI64_AHB32, /* 1x AXI 64-bit port 0 and 1x AHB 32-bit port 1 */
AXI64_AXI64, /* 2x AXI: 64-bit port 0 and 64-bit port 1 */
AXI128_AHB32, /* 1x AXI 128-bit port 0 and 1x AHB 32-bit port 1 */
};
enum stm32_dma3_port_data_width {
DW_32, /* 32-bit, for AHB */
DW_64, /* 64-bit, for AXI */
DW_128, /* 128-bit, for AXI */
DW_INVALID,
};
/* VERR DMA version register */
#define VERR_MINREV GENMASK(3, 0)
#define VERR_MAJREV GENMASK(7, 4)
/* Device tree */
/* struct stm32_dma3_dt_conf */
/* .ch_conf */
#define STM32_DMA3_DT_PRIO GENMASK(1, 0) /* CCR_PRIO */
#define STM32_DMA3_DT_FIFO GENMASK(7, 4)
/* .tr_conf */
#define STM32_DMA3_DT_SINC BIT(0) /* CTR1_SINC */
#define STM32_DMA3_DT_SAP BIT(1) /* CTR1_SAP */
#define STM32_DMA3_DT_DINC BIT(4) /* CTR1_DINC */
#define STM32_DMA3_DT_DAP BIT(5) /* CTR1_DAP */
#define STM32_DMA3_DT_BREQ BIT(8) /* CTR2_BREQ */
#define STM32_DMA3_DT_PFREQ BIT(9) /* CTR2_PFREQ */
#define STM32_DMA3_DT_TCEM GENMASK(13, 12) /* CTR2_TCEM */
/* struct stm32_dma3_chan .config_set bitfield */
#define STM32_DMA3_CFG_SET_DT BIT(0)
#define STM32_DMA3_CFG_SET_DMA BIT(1)
#define STM32_DMA3_CFG_SET_BOTH (STM32_DMA3_CFG_SET_DT | STM32_DMA3_CFG_SET_DMA)
#define STM32_DMA3_MAX_BLOCK_SIZE ALIGN_DOWN(CBR1_BNDT, 64)
#define port_is_ahb(maxdw) ({ typeof(maxdw) (_maxdw) = (maxdw); \
((_maxdw) != DW_INVALID) && ((_maxdw) == DW_32); })
#define port_is_axi(maxdw) ({ typeof(maxdw) (_maxdw) = (maxdw); \
((_maxdw) != DW_INVALID) && ((_maxdw) != DW_32); })
#define get_chan_max_dw(maxdw, maxburst)((port_is_ahb(maxdw) || \
(maxburst) < DMA_SLAVE_BUSWIDTH_8_BYTES) ? \
DMA_SLAVE_BUSWIDTH_4_BYTES : DMA_SLAVE_BUSWIDTH_8_BYTES)
/* Static linked-list data structure (depends on update bits UT1/UT2/UB1/USA/UDA/ULL) */
struct stm32_dma3_hwdesc {
u32 ctr1;
u32 ctr2;
u32 cbr1;
u32 csar;
u32 cdar;
u32 cllr;
} __packed __aligned(32);
/*
* CLLR_LA / sizeof(struct stm32_dma3_hwdesc) represents the number of hdwdesc that can be addressed
* by the pointer to the next linked-list data structure. The __aligned forces the 32-byte
* alignment. So use hardcoded 32. Multiplied by the max block size of each item, it represents
* the sg size limitation.
*/
#define STM32_DMA3_MAX_SEG_SIZE ((CLLR_LA / 32) * STM32_DMA3_MAX_BLOCK_SIZE)
/*
* Linked-list items
*/
struct stm32_dma3_lli {
struct stm32_dma3_hwdesc *hwdesc;
dma_addr_t hwdesc_addr;
};
struct stm32_dma3_swdesc {
struct virt_dma_desc vdesc;
u32 ccr;
bool cyclic;
u32 lli_size;
struct stm32_dma3_lli lli[] __counted_by(lli_size);
};
struct stm32_dma3_dt_conf {
u32 ch_id;
u32 req_line;
u32 ch_conf;
u32 tr_conf;
};
struct stm32_dma3_chan {
struct virt_dma_chan vchan;
u32 id;
int irq;
u32 fifo_size;
u32 max_burst;
bool semaphore_mode;
struct stm32_dma3_dt_conf dt_config;
struct dma_slave_config dma_config;
u8 config_set;
struct dma_pool *lli_pool;
struct stm32_dma3_swdesc *swdesc;
enum ctr2_tcem tcem;
u32 dma_status;
};
struct stm32_dma3_ddata {
struct dma_device dma_dev;
void __iomem *base;
struct clk *clk;
struct stm32_dma3_chan *chans;
u32 dma_channels;
u32 dma_requests;
enum stm32_dma3_port_data_width ports_max_dw[2];
};
static inline struct stm32_dma3_ddata *to_stm32_dma3_ddata(struct stm32_dma3_chan *chan)
{
return container_of(chan->vchan.chan.device, struct stm32_dma3_ddata, dma_dev);
}
static inline struct stm32_dma3_chan *to_stm32_dma3_chan(struct dma_chan *c)
{
return container_of(c, struct stm32_dma3_chan, vchan.chan);
}
static inline struct stm32_dma3_swdesc *to_stm32_dma3_swdesc(struct virt_dma_desc *vdesc)
{
return container_of(vdesc, struct stm32_dma3_swdesc, vdesc);
}
static struct device *chan2dev(struct stm32_dma3_chan *chan)
{
return &chan->vchan.chan.dev->device;
}
static void stm32_dma3_chan_dump_reg(struct stm32_dma3_chan *chan)
{
struct stm32_dma3_ddata *ddata = to_stm32_dma3_ddata(chan);
struct device *dev = chan2dev(chan);
u32 id = chan->id, offset;
offset = STM32_DMA3_SECCFGR;
dev_dbg(dev, "SECCFGR(0x%03x): %08x\n", offset, readl_relaxed(ddata->base + offset));
offset = STM32_DMA3_PRIVCFGR;
dev_dbg(dev, "PRIVCFGR(0x%03x): %08x\n", offset, readl_relaxed(ddata->base + offset));
offset = STM32_DMA3_CCIDCFGR(id);
dev_dbg(dev, "C%dCIDCFGR(0x%03x): %08x\n", id, offset, readl_relaxed(ddata->base + offset));
offset = STM32_DMA3_CSEMCR(id);
dev_dbg(dev, "C%dSEMCR(0x%03x): %08x\n", id, offset, readl_relaxed(ddata->base + offset));
offset = STM32_DMA3_CSR(id);
dev_dbg(dev, "C%dSR(0x%03x): %08x\n", id, offset, readl_relaxed(ddata->base + offset));
offset = STM32_DMA3_CCR(id);
dev_dbg(dev, "C%dCR(0x%03x): %08x\n", id, offset, readl_relaxed(ddata->base + offset));
offset = STM32_DMA3_CTR1(id);
dev_dbg(dev, "C%dTR1(0x%03x): %08x\n", id, offset, readl_relaxed(ddata->base + offset));
offset = STM32_DMA3_CTR2(id);
dev_dbg(dev, "C%dTR2(0x%03x): %08x\n", id, offset, readl_relaxed(ddata->base + offset));
offset = STM32_DMA3_CBR1(id);
dev_dbg(dev, "C%dBR1(0x%03x): %08x\n", id, offset, readl_relaxed(ddata->base + offset));
offset = STM32_DMA3_CSAR(id);
dev_dbg(dev, "C%dSAR(0x%03x): %08x\n", id, offset, readl_relaxed(ddata->base + offset));
offset = STM32_DMA3_CDAR(id);
dev_dbg(dev, "C%dDAR(0x%03x): %08x\n", id, offset, readl_relaxed(ddata->base + offset));
offset = STM32_DMA3_CLLR(id);
dev_dbg(dev, "C%dLLR(0x%03x): %08x\n", id, offset, readl_relaxed(ddata->base + offset));
offset = STM32_DMA3_CLBAR(id);
dev_dbg(dev, "C%dLBAR(0x%03x): %08x\n", id, offset, readl_relaxed(ddata->base + offset));
}
static void stm32_dma3_chan_dump_hwdesc(struct stm32_dma3_chan *chan,
struct stm32_dma3_swdesc *swdesc)
{
struct stm32_dma3_hwdesc *hwdesc;
int i;
for (i = 0; i < swdesc->lli_size; i++) {
hwdesc = swdesc->lli[i].hwdesc;
if (i)
dev_dbg(chan2dev(chan), "V\n");
dev_dbg(chan2dev(chan), "[%d]@%pad\n", i, &swdesc->lli[i].hwdesc_addr);
dev_dbg(chan2dev(chan), "| C%dTR1: %08x\n", chan->id, hwdesc->ctr1);
dev_dbg(chan2dev(chan), "| C%dTR2: %08x\n", chan->id, hwdesc->ctr2);
dev_dbg(chan2dev(chan), "| C%dBR1: %08x\n", chan->id, hwdesc->cbr1);
dev_dbg(chan2dev(chan), "| C%dSAR: %08x\n", chan->id, hwdesc->csar);
dev_dbg(chan2dev(chan), "| C%dDAR: %08x\n", chan->id, hwdesc->cdar);
dev_dbg(chan2dev(chan), "| C%dLLR: %08x\n", chan->id, hwdesc->cllr);
}
if (swdesc->cyclic) {
dev_dbg(chan2dev(chan), "|\n");
dev_dbg(chan2dev(chan), "-->[0]@%pad\n", &swdesc->lli[0].hwdesc_addr);
} else {
dev_dbg(chan2dev(chan), "X\n");
}
}
static struct stm32_dma3_swdesc *stm32_dma3_chan_desc_alloc(struct stm32_dma3_chan *chan, u32 count)
{
struct stm32_dma3_ddata *ddata = to_stm32_dma3_ddata(chan);
struct stm32_dma3_swdesc *swdesc;
int i;
/*
* If the memory to be allocated for the number of hwdesc (6 u32 members but 32-bytes
* aligned) is greater than the maximum address of CLLR_LA, then the last items can't be
* addressed, so abort the allocation.
*/
if ((count * 32) > CLLR_LA) {
dev_err(chan2dev(chan), "Transfer is too big (> %luB)\n", STM32_DMA3_MAX_SEG_SIZE);
return NULL;
}
swdesc = kzalloc(struct_size(swdesc, lli, count), GFP_NOWAIT);
if (!swdesc)
return NULL;
for (i = 0; i < count; i++) {
swdesc->lli[i].hwdesc = dma_pool_zalloc(chan->lli_pool, GFP_NOWAIT,
&swdesc->lli[i].hwdesc_addr);
if (!swdesc->lli[i].hwdesc)
goto err_pool_free;
}
swdesc->lli_size = count;
swdesc->ccr = 0;
/* Set LL base address */
writel_relaxed(swdesc->lli[0].hwdesc_addr & CLBAR_LBA,
ddata->base + STM32_DMA3_CLBAR(chan->id));
/* Set LL allocated port */
swdesc->ccr &= ~CCR_LAP;
return swdesc;
err_pool_free:
dev_err(chan2dev(chan), "Failed to alloc descriptors\n");
while (--i >= 0)
dma_pool_free(chan->lli_pool, swdesc->lli[i].hwdesc, swdesc->lli[i].hwdesc_addr);
kfree(swdesc);
return NULL;
}
static void stm32_dma3_chan_desc_free(struct stm32_dma3_chan *chan,
struct stm32_dma3_swdesc *swdesc)
{
int i;
for (i = 0; i < swdesc->lli_size; i++)
dma_pool_free(chan->lli_pool, swdesc->lli[i].hwdesc, swdesc->lli[i].hwdesc_addr);
kfree(swdesc);
}
static void stm32_dma3_chan_vdesc_free(struct virt_dma_desc *vdesc)
{
struct stm32_dma3_swdesc *swdesc = to_stm32_dma3_swdesc(vdesc);
struct stm32_dma3_chan *chan = to_stm32_dma3_chan(vdesc->tx.chan);
stm32_dma3_chan_desc_free(chan, swdesc);
}
static void stm32_dma3_check_user_setting(struct stm32_dma3_chan *chan)
{
struct stm32_dma3_ddata *ddata = to_stm32_dma3_ddata(chan);
struct device *dev = chan2dev(chan);
u32 ctr1 = readl_relaxed(ddata->base + STM32_DMA3_CTR1(chan->id));
u32 cbr1 = readl_relaxed(ddata->base + STM32_DMA3_CBR1(chan->id));
u32 csar = readl_relaxed(ddata->base + STM32_DMA3_CSAR(chan->id));
u32 cdar = readl_relaxed(ddata->base + STM32_DMA3_CDAR(chan->id));
u32 cllr = readl_relaxed(ddata->base + STM32_DMA3_CLLR(chan->id));
u32 bndt = FIELD_GET(CBR1_BNDT, cbr1);
u32 sdw = 1 << FIELD_GET(CTR1_SDW_LOG2, ctr1);
u32 ddw = 1 << FIELD_GET(CTR1_DDW_LOG2, ctr1);
u32 sap = FIELD_GET(CTR1_SAP, ctr1);
u32 dap = FIELD_GET(CTR1_DAP, ctr1);
if (!bndt && !FIELD_GET(CLLR_UB1, cllr))
dev_err(dev, "null source block size and no update of this value\n");
if (bndt % sdw)
dev_err(dev, "source block size not multiple of src data width\n");
if (FIELD_GET(CTR1_PAM, ctr1) == CTR1_PAM_PACK_UNPACK && bndt % ddw)
dev_err(dev, "(un)packing mode w/ src block size not multiple of dst data width\n");
if (csar % sdw)
dev_err(dev, "unaligned source address not multiple of src data width\n");
if (cdar % ddw)
dev_err(dev, "unaligned destination address not multiple of dst data width\n");
if (sdw == DMA_SLAVE_BUSWIDTH_8_BYTES && port_is_ahb(ddata->ports_max_dw[sap]))
dev_err(dev, "double-word source data width not supported on port %u\n", sap);
if (ddw == DMA_SLAVE_BUSWIDTH_8_BYTES && port_is_ahb(ddata->ports_max_dw[dap]))
dev_err(dev, "double-word destination data width not supported on port %u\n", dap);
}
static void stm32_dma3_chan_prep_hwdesc(struct stm32_dma3_chan *chan,
struct stm32_dma3_swdesc *swdesc,
u32 curr, dma_addr_t src, dma_addr_t dst, u32 len,
u32 ctr1, u32 ctr2, bool is_last, bool is_cyclic)
{
struct stm32_dma3_hwdesc *hwdesc;
dma_addr_t next_lli;
u32 next = curr + 1;
hwdesc = swdesc->lli[curr].hwdesc;
hwdesc->ctr1 = ctr1;
hwdesc->ctr2 = ctr2;
hwdesc->cbr1 = FIELD_PREP(CBR1_BNDT, len);
hwdesc->csar = src;
hwdesc->cdar = dst;
if (is_last) {
if (is_cyclic)
next_lli = swdesc->lli[0].hwdesc_addr;
else
next_lli = 0;
} else {
next_lli = swdesc->lli[next].hwdesc_addr;
}
hwdesc->cllr = 0;
if (next_lli) {
hwdesc->cllr |= CLLR_UT1 | CLLR_UT2 | CLLR_UB1;
hwdesc->cllr |= CLLR_USA | CLLR_UDA | CLLR_ULL;
hwdesc->cllr |= (next_lli & CLLR_LA);
}
/*
* Make sure to flush the CPU's write buffers so that the descriptors are ready to be read
* by DMA3. By explicitly using a write memory barrier here, instead of doing it with writel
* to enable the channel, we avoid an unnecessary barrier in the case where the descriptors
* are reused (DMA_CTRL_REUSE).
*/
if (is_last)
dma_wmb();
}
static enum dma_slave_buswidth stm32_dma3_get_max_dw(u32 chan_max_burst,
enum stm32_dma3_port_data_width port_max_dw,
u32 len, dma_addr_t addr)
{
enum dma_slave_buswidth max_dw = get_chan_max_dw(port_max_dw, chan_max_burst);
/* len and addr must be a multiple of dw */
return 1 << __ffs(len | addr | max_dw);
}
static u32 stm32_dma3_get_max_burst(u32 len, enum dma_slave_buswidth dw, u32 chan_max_burst)
{
u32 max_burst = chan_max_burst ? chan_max_burst / dw : 1;
/* len is a multiple of dw, so if len is < chan_max_burst, shorten burst */
if (len < chan_max_burst)
max_burst = len / dw;
/*
* HW doesn't modify the burst if burst size <= half of the fifo size.
* If len is not a multiple of burst size, last burst is shortened by HW.
*/
return max_burst;
}
static int stm32_dma3_chan_prep_hw(struct stm32_dma3_chan *chan, enum dma_transfer_direction dir,
u32 *ccr, u32 *ctr1, u32 *ctr2,
dma_addr_t src_addr, dma_addr_t dst_addr, u32 len)
{
struct stm32_dma3_ddata *ddata = to_stm32_dma3_ddata(chan);
struct dma_device dma_device = ddata->dma_dev;
u32 sdw, ddw, sbl_max, dbl_max, tcem, init_dw, init_bl_max;
u32 _ctr1 = 0, _ctr2 = 0;
u32 ch_conf = chan->dt_config.ch_conf;
u32 tr_conf = chan->dt_config.tr_conf;
u32 sap = FIELD_GET(STM32_DMA3_DT_SAP, tr_conf), sap_max_dw;
u32 dap = FIELD_GET(STM32_DMA3_DT_DAP, tr_conf), dap_max_dw;
dev_dbg(chan2dev(chan), "%s from %pad to %pad\n",
dmaengine_get_direction_text(dir), &src_addr, &dst_addr);
sdw = chan->dma_config.src_addr_width ? : get_chan_max_dw(sap, chan->max_burst);
ddw = chan->dma_config.dst_addr_width ? : get_chan_max_dw(dap, chan->max_burst);
sbl_max = chan->dma_config.src_maxburst ? : 1;
dbl_max = chan->dma_config.dst_maxburst ? : 1;
/* Following conditions would raise User Setting Error interrupt */
if (!(dma_device.src_addr_widths & BIT(sdw)) || !(dma_device.dst_addr_widths & BIT(ddw))) {
dev_err(chan2dev(chan), "Bus width (src=%u, dst=%u) not supported\n", sdw, ddw);
return -EINVAL;
}
if (ddata->ports_max_dw[1] == DW_INVALID && (sap || dap)) {
dev_err(chan2dev(chan), "Only one master port, port 1 is not supported\n");
return -EINVAL;
}
sap_max_dw = ddata->ports_max_dw[sap];
dap_max_dw = ddata->ports_max_dw[dap];
if ((port_is_ahb(sap_max_dw) && sdw == DMA_SLAVE_BUSWIDTH_8_BYTES) ||
(port_is_ahb(dap_max_dw) && ddw == DMA_SLAVE_BUSWIDTH_8_BYTES)) {
dev_err(chan2dev(chan),
"8 bytes buswidth (src=%u, dst=%u) not supported on port (sap=%u, dap=%u\n",
sdw, ddw, sap, dap);
return -EINVAL;
}
if (FIELD_GET(STM32_DMA3_DT_SINC, tr_conf))
_ctr1 |= CTR1_SINC;
if (sap)
_ctr1 |= CTR1_SAP;
if (FIELD_GET(STM32_DMA3_DT_DINC, tr_conf))
_ctr1 |= CTR1_DINC;
if (dap)
_ctr1 |= CTR1_DAP;
_ctr2 |= FIELD_PREP(CTR2_REQSEL, chan->dt_config.req_line) & ~CTR2_SWREQ;
if (FIELD_GET(STM32_DMA3_DT_BREQ, tr_conf))
_ctr2 |= CTR2_BREQ;
if (dir == DMA_DEV_TO_MEM && FIELD_GET(STM32_DMA3_DT_PFREQ, tr_conf))
_ctr2 |= CTR2_PFREQ;
tcem = FIELD_GET(STM32_DMA3_DT_TCEM, tr_conf);
_ctr2 |= FIELD_PREP(CTR2_TCEM, tcem);
/* Store TCEM to know on which event TC flag occurred */
chan->tcem = tcem;
/* Store direction for residue computation */
chan->dma_config.direction = dir;
switch (dir) {
case DMA_MEM_TO_DEV:
/* Set destination (device) data width and burst */
ddw = min_t(u32, ddw, stm32_dma3_get_max_dw(chan->max_burst, dap_max_dw,
len, dst_addr));
dbl_max = min_t(u32, dbl_max, stm32_dma3_get_max_burst(len, ddw, chan->max_burst));
/* Set source (memory) data width and burst */
sdw = stm32_dma3_get_max_dw(chan->max_burst, sap_max_dw, len, src_addr);
sbl_max = stm32_dma3_get_max_burst(len, sdw, chan->max_burst);
_ctr1 |= FIELD_PREP(CTR1_SDW_LOG2, ilog2(sdw));
_ctr1 |= FIELD_PREP(CTR1_SBL_1, sbl_max - 1);
_ctr1 |= FIELD_PREP(CTR1_DDW_LOG2, ilog2(ddw));
_ctr1 |= FIELD_PREP(CTR1_DBL_1, dbl_max - 1);
if (ddw != sdw) {
_ctr1 |= FIELD_PREP(CTR1_PAM, CTR1_PAM_PACK_UNPACK);
/* Should never reach this case as ddw is clamped down */
if (len & (ddw - 1)) {
dev_err(chan2dev(chan),
"Packing mode is enabled and len is not multiple of ddw");
return -EINVAL;
}
}
/* dst = dev */
_ctr2 |= CTR2_DREQ;
break;
case DMA_DEV_TO_MEM:
/* Set source (device) data width and burst */
sdw = min_t(u32, sdw, stm32_dma3_get_max_dw(chan->max_burst, sap_max_dw,
len, src_addr));
sbl_max = min_t(u32, sbl_max, stm32_dma3_get_max_burst(len, sdw, chan->max_burst));
/* Set destination (memory) data width and burst */
ddw = stm32_dma3_get_max_dw(chan->max_burst, dap_max_dw, len, dst_addr);
dbl_max = stm32_dma3_get_max_burst(len, ddw, chan->max_burst);
_ctr1 |= FIELD_PREP(CTR1_SDW_LOG2, ilog2(sdw));
_ctr1 |= FIELD_PREP(CTR1_SBL_1, sbl_max - 1);
_ctr1 |= FIELD_PREP(CTR1_DDW_LOG2, ilog2(ddw));
_ctr1 |= FIELD_PREP(CTR1_DBL_1, dbl_max - 1);
if (ddw != sdw) {
_ctr1 |= FIELD_PREP(CTR1_PAM, CTR1_PAM_PACK_UNPACK);
/* Should never reach this case as ddw is clamped down */
if (len & (ddw - 1)) {
dev_err(chan2dev(chan),
"Packing mode is enabled and len is not multiple of ddw\n");
return -EINVAL;
}
}
/* dst = mem */
_ctr2 &= ~CTR2_DREQ;
break;
case DMA_MEM_TO_MEM:
/* Set source (memory) data width and burst */
init_dw = sdw;
init_bl_max = sbl_max;
sdw = stm32_dma3_get_max_dw(chan->max_burst, sap_max_dw, len, src_addr);
sbl_max = stm32_dma3_get_max_burst(len, sdw, chan->max_burst);
if (chan->config_set & STM32_DMA3_CFG_SET_DMA) {
sdw = min_t(u32, init_dw, sdw);
sbl_max = min_t(u32, init_bl_max,
stm32_dma3_get_max_burst(len, sdw, chan->max_burst));
}
/* Set destination (memory) data width and burst */
init_dw = ddw;
init_bl_max = dbl_max;
ddw = stm32_dma3_get_max_dw(chan->max_burst, dap_max_dw, len, dst_addr);
dbl_max = stm32_dma3_get_max_burst(len, ddw, chan->max_burst);
if (chan->config_set & STM32_DMA3_CFG_SET_DMA) {
ddw = min_t(u32, init_dw, ddw);
dbl_max = min_t(u32, init_bl_max,
stm32_dma3_get_max_burst(len, ddw, chan->max_burst));
}
_ctr1 |= FIELD_PREP(CTR1_SDW_LOG2, ilog2(sdw));
_ctr1 |= FIELD_PREP(CTR1_SBL_1, sbl_max - 1);
_ctr1 |= FIELD_PREP(CTR1_DDW_LOG2, ilog2(ddw));
_ctr1 |= FIELD_PREP(CTR1_DBL_1, dbl_max - 1);
if (ddw != sdw) {
_ctr1 |= FIELD_PREP(CTR1_PAM, CTR1_PAM_PACK_UNPACK);
/* Should never reach this case as ddw is clamped down */
if (len & (ddw - 1)) {
dev_err(chan2dev(chan),
"Packing mode is enabled and len is not multiple of ddw");
return -EINVAL;
}
}
/* CTR2_REQSEL/DREQ/BREQ/PFREQ are ignored with CTR2_SWREQ=1 */
_ctr2 |= CTR2_SWREQ;
break;
default:
dev_err(chan2dev(chan), "Direction %s not supported\n",
dmaengine_get_direction_text(dir));
return -EINVAL;
}
*ccr |= FIELD_PREP(CCR_PRIO, FIELD_GET(STM32_DMA3_DT_PRIO, ch_conf));
*ctr1 = _ctr1;
*ctr2 = _ctr2;
dev_dbg(chan2dev(chan), "%s: sdw=%u bytes sbl=%u beats ddw=%u bytes dbl=%u beats\n",
__func__, sdw, sbl_max, ddw, dbl_max);
return 0;
}
static void stm32_dma3_chan_start(struct stm32_dma3_chan *chan)
{
struct stm32_dma3_ddata *ddata = to_stm32_dma3_ddata(chan);
struct virt_dma_desc *vdesc;
struct stm32_dma3_hwdesc *hwdesc;
u32 id = chan->id;
u32 csr, ccr;
vdesc = vchan_next_desc(&chan->vchan);
if (!vdesc) {
chan->swdesc = NULL;
return;
}
list_del(&vdesc->node);
chan->swdesc = to_stm32_dma3_swdesc(vdesc);
hwdesc = chan->swdesc->lli[0].hwdesc;
stm32_dma3_chan_dump_hwdesc(chan, chan->swdesc);
writel_relaxed(chan->swdesc->ccr, ddata->base + STM32_DMA3_CCR(id));
writel_relaxed(hwdesc->ctr1, ddata->base + STM32_DMA3_CTR1(id));
writel_relaxed(hwdesc->ctr2, ddata->base + STM32_DMA3_CTR2(id));
writel_relaxed(hwdesc->cbr1, ddata->base + STM32_DMA3_CBR1(id));
writel_relaxed(hwdesc->csar, ddata->base + STM32_DMA3_CSAR(id));
writel_relaxed(hwdesc->cdar, ddata->base + STM32_DMA3_CDAR(id));
writel_relaxed(hwdesc->cllr, ddata->base + STM32_DMA3_CLLR(id));
/* Clear any pending interrupts */
csr = readl_relaxed(ddata->base + STM32_DMA3_CSR(id));
if (csr & CSR_ALL_F)
writel_relaxed(csr, ddata->base + STM32_DMA3_CFCR(id));
stm32_dma3_chan_dump_reg(chan);
ccr = readl_relaxed(ddata->base + STM32_DMA3_CCR(id));
writel_relaxed(ccr | CCR_EN, ddata->base + STM32_DMA3_CCR(id));
chan->dma_status = DMA_IN_PROGRESS;
dev_dbg(chan2dev(chan), "vchan %pK: started\n", &chan->vchan);
}
static int stm32_dma3_chan_suspend(struct stm32_dma3_chan *chan, bool susp)
{
struct stm32_dma3_ddata *ddata = to_stm32_dma3_ddata(chan);
u32 csr, ccr = readl_relaxed(ddata->base + STM32_DMA3_CCR(chan->id)) & ~CCR_EN;
int ret = 0;
if (susp)
ccr |= CCR_SUSP;
else
ccr &= ~CCR_SUSP;
writel_relaxed(ccr, ddata->base + STM32_DMA3_CCR(chan->id));
if (susp) {
ret = readl_relaxed_poll_timeout_atomic(ddata->base + STM32_DMA3_CSR(chan->id), csr,
csr & CSR_SUSPF, 1, 10);
if (!ret)
writel_relaxed(CFCR_SUSPF, ddata->base + STM32_DMA3_CFCR(chan->id));
stm32_dma3_chan_dump_reg(chan);
}
return ret;
}
static void stm32_dma3_chan_reset(struct stm32_dma3_chan *chan)
{
struct stm32_dma3_ddata *ddata = to_stm32_dma3_ddata(chan);
u32 ccr = readl_relaxed(ddata->base + STM32_DMA3_CCR(chan->id)) & ~CCR_EN;
writel_relaxed(ccr |= CCR_RESET, ddata->base + STM32_DMA3_CCR(chan->id));
}
static int stm32_dma3_chan_get_curr_hwdesc(struct stm32_dma3_swdesc *swdesc, u32 cllr, u32 *residue)
{
u32 i, lli_offset, next_lli_offset = cllr & CLLR_LA;
/* If cllr is null, it means it is either the last or single item */
if (!cllr)
return swdesc->lli_size - 1;
/* In cyclic mode, go fast and first check we are not on the last item */
if (swdesc->cyclic && next_lli_offset == (swdesc->lli[0].hwdesc_addr & CLLR_LA))
return swdesc->lli_size - 1;
/* As transfer is in progress, look backward from the last item */
for (i = swdesc->lli_size - 1; i > 0; i--) {
*residue += FIELD_GET(CBR1_BNDT, swdesc->lli[i].hwdesc->cbr1);
lli_offset = swdesc->lli[i].hwdesc_addr & CLLR_LA;
if (lli_offset == next_lli_offset)
return i - 1;
}
return -EINVAL;
}
static void stm32_dma3_chan_set_residue(struct stm32_dma3_chan *chan,
struct stm32_dma3_swdesc *swdesc,
struct dma_tx_state *txstate)
{
struct stm32_dma3_ddata *ddata = to_stm32_dma3_ddata(chan);
struct device *dev = chan2dev(chan);
struct stm32_dma3_hwdesc *hwdesc;
u32 residue, curr_lli, csr, cdar, cbr1, cllr, bndt, fifol;
bool pack_unpack;
int ret;
csr = readl_relaxed(ddata->base + STM32_DMA3_CSR(chan->id));
if (!(csr & CSR_IDLEF) && chan->dma_status != DMA_PAUSED) {
/* Suspend current transfer to read registers for a snapshot */
writel_relaxed(swdesc->ccr | CCR_SUSP, ddata->base + STM32_DMA3_CCR(chan->id));
ret = readl_relaxed_poll_timeout_atomic(ddata->base + STM32_DMA3_CSR(chan->id), csr,
csr & (CSR_SUSPF | CSR_IDLEF), 1, 10);
if (ret || ((csr & CSR_TCF) && (csr & CSR_IDLEF))) {
writel_relaxed(CFCR_SUSPF, ddata->base + STM32_DMA3_CFCR(chan->id));
writel_relaxed(swdesc->ccr, ddata->base + STM32_DMA3_CCR(chan->id));
if (ret)
dev_err(dev, "Channel suspension timeout, csr=%08x\n", csr);
}
}
/* If channel is still active (CSR_IDLEF is not set), can't get a reliable residue */
if (!(csr & CSR_IDLEF))
dev_warn(dev, "Can't get residue: channel still active, csr=%08x\n", csr);
/*
* If channel is not suspended, but Idle and Transfer Complete are set,
* linked-list is over, no residue
*/
if (!(csr & CSR_SUSPF) && (csr & CSR_TCF) && (csr & CSR_IDLEF))
return;
/* Read registers to have a snapshot */
cllr = readl_relaxed(ddata->base + STM32_DMA3_CLLR(chan->id));
cbr1 = readl_relaxed(ddata->base + STM32_DMA3_CBR1(chan->id));
cdar = readl_relaxed(ddata->base + STM32_DMA3_CDAR(chan->id));
/* Resume current transfer */
if (csr & CSR_SUSPF) {
writel_relaxed(CFCR_SUSPF, ddata->base + STM32_DMA3_CFCR(chan->id));
writel_relaxed(swdesc->ccr, ddata->base + STM32_DMA3_CCR(chan->id));
}
/* Add current BNDT */
bndt = FIELD_GET(CBR1_BNDT, cbr1);
residue = bndt;
/* Get current hwdesc and cumulate residue of pending hwdesc BNDT */
ret = stm32_dma3_chan_get_curr_hwdesc(swdesc, cllr, &residue);
if (ret < 0) {
dev_err(chan2dev(chan), "Can't get residue: current hwdesc not found\n");
return;
}
curr_lli = ret;
/* Read current FIFO level - in units of programmed destination data width */
hwdesc = swdesc->lli[curr_lli].hwdesc;
fifol = FIELD_GET(CSR_FIFOL, csr) * (1 << FIELD_GET(CTR1_DDW_LOG2, hwdesc->ctr1));
/* If the FIFO contains as many bytes as its size, it can't contain more */
if (fifol == (1 << (chan->fifo_size + 1)))
goto skip_fifol_update;
/*
* In case of PACKING (Destination burst length > Source burst length) or UNPACKING
* (Source burst length > Destination burst length), bytes could be pending in the FIFO
* (to be packed up to Destination burst length or unpacked into Destination burst length
* chunks).
* BNDT is not reliable, as it reflects the number of bytes read from the source but not the
* number of bytes written to the destination.
* FIFOL is also not sufficient, because it reflects the number of available write beats in
* units of Destination data width but not the bytes not yet packed or unpacked.
* In case of Destination increment DINC, it is possible to compute the number of bytes in
* the FIFO:
* fifol_in_bytes = bytes_read - bytes_written.
*/
pack_unpack = !!(FIELD_GET(CTR1_PAM, hwdesc->ctr1) == CTR1_PAM_PACK_UNPACK);
if (pack_unpack && (hwdesc->ctr1 & CTR1_DINC)) {
int bytes_read = FIELD_GET(CBR1_BNDT, hwdesc->cbr1) - bndt;
int bytes_written = cdar - hwdesc->cdar;
if (bytes_read > 0)
fifol = bytes_read - bytes_written;
}
skip_fifol_update:
if (fifol) {
dev_dbg(chan2dev(chan), "%u byte(s) in the FIFO\n", fifol);
dma_set_in_flight_bytes(txstate, fifol);
/*
* Residue is already accurate for DMA_MEM_TO_DEV as BNDT reflects data read from
* the source memory buffer, so just need to add fifol to residue in case of
* DMA_DEV_TO_MEM transfer because these bytes are not yet written in destination
* memory buffer.
*/
if (chan->dma_config.direction == DMA_DEV_TO_MEM)
residue += fifol;
}
dma_set_residue(txstate, residue);
}
static int stm32_dma3_chan_stop(struct stm32_dma3_chan *chan)
{
struct stm32_dma3_ddata *ddata = to_stm32_dma3_ddata(chan);
u32 ccr;
int ret = 0;
chan->dma_status = DMA_COMPLETE;
/* Disable interrupts */
ccr = readl_relaxed(ddata->base + STM32_DMA3_CCR(chan->id));
writel_relaxed(ccr & ~(CCR_ALLIE | CCR_EN), ddata->base + STM32_DMA3_CCR(chan->id));
if (!(ccr & CCR_SUSP) && (ccr & CCR_EN)) {
/* Suspend the channel */
ret = stm32_dma3_chan_suspend(chan, true);
if (ret)
dev_warn(chan2dev(chan), "%s: timeout, data might be lost\n", __func__);
}
/*
* Reset the channel: this causes the reset of the FIFO and the reset of the channel
* internal state, the reset of CCR_EN and CCR_SUSP bits.
*/
stm32_dma3_chan_reset(chan);
return ret;
}
static void stm32_dma3_chan_complete(struct stm32_dma3_chan *chan)
{
if (!chan->swdesc)
return;
vchan_cookie_complete(&chan->swdesc->vdesc);
chan->swdesc = NULL;
stm32_dma3_chan_start(chan);
}
static irqreturn_t stm32_dma3_chan_irq(int irq, void *devid)
{
struct stm32_dma3_chan *chan = devid;
struct stm32_dma3_ddata *ddata = to_stm32_dma3_ddata(chan);
u32 misr, csr, ccr;
spin_lock(&chan->vchan.lock);
misr = readl_relaxed(ddata->base + STM32_DMA3_MISR);
if (!(misr & MISR_MIS(chan->id))) {
spin_unlock(&chan->vchan.lock);
return IRQ_NONE;
}
csr = readl_relaxed(ddata->base + STM32_DMA3_CSR(chan->id));
ccr = readl_relaxed(ddata->base + STM32_DMA3_CCR(chan->id)) & CCR_ALLIE;
if (csr & CSR_TCF && ccr & CCR_TCIE) {
if (chan->swdesc->cyclic)
vchan_cyclic_callback(&chan->swdesc->vdesc);
else
stm32_dma3_chan_complete(chan);
}
if (csr & CSR_USEF && ccr & CCR_USEIE) {
dev_err(chan2dev(chan), "User setting error\n");
chan->dma_status = DMA_ERROR;
/* CCR.EN automatically cleared by HW */
stm32_dma3_check_user_setting(chan);
stm32_dma3_chan_reset(chan);
}
if (csr & CSR_ULEF && ccr & CCR_ULEIE) {
dev_err(chan2dev(chan), "Update link transfer error\n");
chan->dma_status = DMA_ERROR;
/* CCR.EN automatically cleared by HW */
stm32_dma3_chan_reset(chan);
}
if (csr & CSR_DTEF && ccr & CCR_DTEIE) {
dev_err(chan2dev(chan), "Data transfer error\n");
chan->dma_status = DMA_ERROR;
/* CCR.EN automatically cleared by HW */
stm32_dma3_chan_reset(chan);
}
/*
* Half Transfer Interrupt may be disabled but Half Transfer Flag can be set,
* ensure HTF flag to be cleared, with other flags.
*/
csr &= (ccr | CCR_HTIE);
if (csr)
writel_relaxed(csr, ddata->base + STM32_DMA3_CFCR(chan->id));
spin_unlock(&chan->vchan.lock);
return IRQ_HANDLED;
}
static int stm32_dma3_alloc_chan_resources(struct dma_chan *c)
{
struct stm32_dma3_chan *chan = to_stm32_dma3_chan(c);
struct stm32_dma3_ddata *ddata = to_stm32_dma3_ddata(chan);
u32 id = chan->id, csemcr, ccid;
int ret;
ret = pm_runtime_resume_and_get(ddata->dma_dev.dev);
if (ret < 0)
return ret;
/* Ensure the channel is free */
if (chan->semaphore_mode &&
readl_relaxed(ddata->base + STM32_DMA3_CSEMCR(chan->id)) & CSEMCR_SEM_MUTEX) {
ret = -EBUSY;
goto err_put_sync;
}
chan->lli_pool = dmam_pool_create(dev_name(&c->dev->device), c->device->dev,
sizeof(struct stm32_dma3_hwdesc),
__alignof__(struct stm32_dma3_hwdesc), SZ_64K);
if (!chan->lli_pool) {
dev_err(chan2dev(chan), "Failed to create LLI pool\n");
ret = -ENOMEM;
goto err_put_sync;
}
/* Take the channel semaphore */
if (chan->semaphore_mode) {
writel_relaxed(CSEMCR_SEM_MUTEX, ddata->base + STM32_DMA3_CSEMCR(id));
csemcr = readl_relaxed(ddata->base + STM32_DMA3_CSEMCR(id));
ccid = FIELD_GET(CSEMCR_SEM_CCID, csemcr);
/* Check that the channel is well taken */
if (ccid != CCIDCFGR_CID1) {
dev_err(chan2dev(chan), "Not under CID1 control (in-use by CID%d)\n", ccid);
ret = -EPERM;
goto err_pool_destroy;
}
dev_dbg(chan2dev(chan), "Under CID1 control (semcr=0x%08x)\n", csemcr);
}
return 0;
err_pool_destroy:
dmam_pool_destroy(chan->lli_pool);
chan->lli_pool = NULL;
err_put_sync:
pm_runtime_put_sync(ddata->dma_dev.dev);
return ret;
}
static void stm32_dma3_free_chan_resources(struct dma_chan *c)
{
struct stm32_dma3_chan *chan = to_stm32_dma3_chan(c);
struct stm32_dma3_ddata *ddata = to_stm32_dma3_ddata(chan);
unsigned long flags;
/* Ensure channel is in idle state */
spin_lock_irqsave(&chan->vchan.lock, flags);
stm32_dma3_chan_stop(chan);
chan->swdesc = NULL;
spin_unlock_irqrestore(&chan->vchan.lock, flags);
vchan_free_chan_resources(to_virt_chan(c));
dmam_pool_destroy(chan->lli_pool);
chan->lli_pool = NULL;
/* Release the channel semaphore */
if (chan->semaphore_mode)
writel_relaxed(0, ddata->base + STM32_DMA3_CSEMCR(chan->id));
pm_runtime_put_sync(ddata->dma_dev.dev);
/* Reset configuration */
memset(&chan->dt_config, 0, sizeof(chan->dt_config));
memset(&chan->dma_config, 0, sizeof(chan->dma_config));
chan->config_set = 0;
}
static void stm32_dma3_init_chan_config_for_memcpy(struct stm32_dma3_chan *chan,
dma_addr_t dst, dma_addr_t src)
{
struct stm32_dma3_ddata *ddata = to_stm32_dma3_ddata(chan);
u32 dw = get_chan_max_dw(ddata->ports_max_dw[0], chan->max_burst); /* port 0 by default */
u32 burst = chan->max_burst / dw;
/* Initialize dt_config if channel not pre-configured through DT */
if (!(chan->config_set & STM32_DMA3_CFG_SET_DT)) {
chan->dt_config.ch_conf = FIELD_PREP(STM32_DMA3_DT_PRIO, CCR_PRIO_VERY_HIGH);
chan->dt_config.ch_conf |= FIELD_PREP(STM32_DMA3_DT_FIFO, chan->fifo_size);
chan->dt_config.tr_conf = STM32_DMA3_DT_SINC | STM32_DMA3_DT_DINC;
chan->dt_config.tr_conf |= FIELD_PREP(STM32_DMA3_DT_TCEM, CTR2_TCEM_CHANNEL);
}
/* Initialize dma_config if dmaengine_slave_config() not used */
if (!(chan->config_set & STM32_DMA3_CFG_SET_DMA)) {
chan->dma_config.src_addr_width = dw;
chan->dma_config.dst_addr_width = dw;
chan->dma_config.src_maxburst = burst;
chan->dma_config.dst_maxburst = burst;
chan->dma_config.src_addr = src;
chan->dma_config.dst_addr = dst;
}
}
static struct dma_async_tx_descriptor *stm32_dma3_prep_dma_memcpy(struct dma_chan *c,
dma_addr_t dst, dma_addr_t src,
size_t len, unsigned long flags)
{
struct stm32_dma3_chan *chan = to_stm32_dma3_chan(c);
struct stm32_dma3_swdesc *swdesc;
size_t next_size, offset;
u32 count, i, ctr1, ctr2;
count = DIV_ROUND_UP(len, STM32_DMA3_MAX_BLOCK_SIZE);
swdesc = stm32_dma3_chan_desc_alloc(chan, count);
if (!swdesc)
return NULL;
if (chan->config_set != STM32_DMA3_CFG_SET_BOTH)
stm32_dma3_init_chan_config_for_memcpy(chan, dst, src);
for (i = 0, offset = 0; offset < len; i++, offset += next_size) {
size_t remaining;
int ret;
remaining = len - offset;
next_size = min_t(size_t, remaining, STM32_DMA3_MAX_BLOCK_SIZE);
ret = stm32_dma3_chan_prep_hw(chan, DMA_MEM_TO_MEM, &swdesc->ccr, &ctr1, &ctr2,
src + offset, dst + offset, next_size);
if (ret)
goto err_desc_free;
stm32_dma3_chan_prep_hwdesc(chan, swdesc, i, src + offset, dst + offset, next_size,
ctr1, ctr2, next_size == remaining, false);
}
/* Enable Errors interrupts */
swdesc->ccr |= CCR_USEIE | CCR_ULEIE | CCR_DTEIE;
/* Enable Transfer state interrupts */
swdesc->ccr |= CCR_TCIE;
swdesc->cyclic = false;
return vchan_tx_prep(&chan->vchan, &swdesc->vdesc, flags);
err_desc_free:
stm32_dma3_chan_desc_free(chan, swdesc);
return NULL;
}
static struct dma_async_tx_descriptor *stm32_dma3_prep_slave_sg(struct dma_chan *c,
struct scatterlist *sgl,
unsigned int sg_len,
enum dma_transfer_direction dir,
unsigned long flags, void *context)
{
struct stm32_dma3_chan *chan = to_stm32_dma3_chan(c);
struct stm32_dma3_swdesc *swdesc;
struct scatterlist *sg;
size_t len;
dma_addr_t sg_addr, dev_addr, src, dst;
u32 i, j, count, ctr1, ctr2;
int ret;
count = sg_len;
for_each_sg(sgl, sg, sg_len, i) {
len = sg_dma_len(sg);
if (len > STM32_DMA3_MAX_BLOCK_SIZE)
count += DIV_ROUND_UP(len, STM32_DMA3_MAX_BLOCK_SIZE) - 1;
}
swdesc = stm32_dma3_chan_desc_alloc(chan, count);
if (!swdesc)
return NULL;
/* sg_len and i correspond to the initial sgl; count and j correspond to the hwdesc LL */
j = 0;
for_each_sg(sgl, sg, sg_len, i) {
sg_addr = sg_dma_address(sg);
dev_addr = (dir == DMA_MEM_TO_DEV) ? chan->dma_config.dst_addr :
chan->dma_config.src_addr;
len = sg_dma_len(sg);
do {
size_t chunk = min_t(size_t, len, STM32_DMA3_MAX_BLOCK_SIZE);
if (dir == DMA_MEM_TO_DEV) {
src = sg_addr;
dst = dev_addr;
ret = stm32_dma3_chan_prep_hw(chan, dir, &swdesc->ccr, &ctr1, &ctr2,
src, dst, chunk);
if (FIELD_GET(CTR1_DINC, ctr1))
dev_addr += chunk;
} else { /* (dir == DMA_DEV_TO_MEM || dir == DMA_MEM_TO_MEM) */
src = dev_addr;
dst = sg_addr;
ret = stm32_dma3_chan_prep_hw(chan, dir, &swdesc->ccr, &ctr1, &ctr2,
src, dst, chunk);
if (FIELD_GET(CTR1_SINC, ctr1))
dev_addr += chunk;
}
if (ret)
goto err_desc_free;
stm32_dma3_chan_prep_hwdesc(chan, swdesc, j, src, dst, chunk,
ctr1, ctr2, j == (count - 1), false);
sg_addr += chunk;
len -= chunk;
j++;
} while (len);
}
/* Enable Error interrupts */
swdesc->ccr |= CCR_USEIE | CCR_ULEIE | CCR_DTEIE;
/* Enable Transfer state interrupts */
swdesc->ccr |= CCR_TCIE;
swdesc->cyclic = false;
return vchan_tx_prep(&chan->vchan, &swdesc->vdesc, flags);
err_desc_free:
stm32_dma3_chan_desc_free(chan, swdesc);
return NULL;
}
static struct dma_async_tx_descriptor *stm32_dma3_prep_dma_cyclic(struct dma_chan *c,
dma_addr_t buf_addr,
size_t buf_len, size_t period_len,
enum dma_transfer_direction dir,
unsigned long flags)
{
struct stm32_dma3_chan *chan = to_stm32_dma3_chan(c);
struct stm32_dma3_swdesc *swdesc;
dma_addr_t src, dst;
u32 count, i, ctr1, ctr2;
int ret;
if (!buf_len || !period_len || period_len > STM32_DMA3_MAX_BLOCK_SIZE) {
dev_err(chan2dev(chan), "Invalid buffer/period length\n");
return NULL;
}
if (buf_len % period_len) {
dev_err(chan2dev(chan), "Buffer length not multiple of period length\n");
return NULL;
}
count = buf_len / period_len;
swdesc = stm32_dma3_chan_desc_alloc(chan, count);
if (!swdesc)
return NULL;
if (dir == DMA_MEM_TO_DEV) {
src = buf_addr;
dst = chan->dma_config.dst_addr;
ret = stm32_dma3_chan_prep_hw(chan, DMA_MEM_TO_DEV, &swdesc->ccr, &ctr1, &ctr2,
src, dst, period_len);
} else if (dir == DMA_DEV_TO_MEM) {
src = chan->dma_config.src_addr;
dst = buf_addr;
ret = stm32_dma3_chan_prep_hw(chan, DMA_DEV_TO_MEM, &swdesc->ccr, &ctr1, &ctr2,
src, dst, period_len);
} else {
dev_err(chan2dev(chan), "Invalid direction\n");
ret = -EINVAL;
}
if (ret)
goto err_desc_free;
for (i = 0; i < count; i++) {
if (dir == DMA_MEM_TO_DEV) {
src = buf_addr + i * period_len;
dst = chan->dma_config.dst_addr;
} else { /* (dir == DMA_DEV_TO_MEM) */
src = chan->dma_config.src_addr;
dst = buf_addr + i * period_len;
}
stm32_dma3_chan_prep_hwdesc(chan, swdesc, i, src, dst, period_len,
ctr1, ctr2, i == (count - 1), true);
}
/* Enable Error interrupts */
swdesc->ccr |= CCR_USEIE | CCR_ULEIE | CCR_DTEIE;
/* Enable Transfer state interrupts */
swdesc->ccr |= CCR_TCIE;
swdesc->cyclic = true;
return vchan_tx_prep(&chan->vchan, &swdesc->vdesc, flags);
err_desc_free:
stm32_dma3_chan_desc_free(chan, swdesc);
return NULL;
}
static void stm32_dma3_caps(struct dma_chan *c, struct dma_slave_caps *caps)
{
struct stm32_dma3_chan *chan = to_stm32_dma3_chan(c);
if (!chan->fifo_size) {
caps->max_burst = 0;
caps->src_addr_widths &= ~BIT(DMA_SLAVE_BUSWIDTH_8_BYTES);
caps->dst_addr_widths &= ~BIT(DMA_SLAVE_BUSWIDTH_8_BYTES);
} else {
/* Burst transfer should not exceed half of the fifo size */
caps->max_burst = chan->max_burst;
if (caps->max_burst < DMA_SLAVE_BUSWIDTH_8_BYTES) {
caps->src_addr_widths &= ~BIT(DMA_SLAVE_BUSWIDTH_8_BYTES);
caps->dst_addr_widths &= ~BIT(DMA_SLAVE_BUSWIDTH_8_BYTES);
}
}
}
static int stm32_dma3_config(struct dma_chan *c, struct dma_slave_config *config)
{
struct stm32_dma3_chan *chan = to_stm32_dma3_chan(c);
memcpy(&chan->dma_config, config, sizeof(*config));
chan->config_set |= STM32_DMA3_CFG_SET_DMA;
return 0;
}
static int stm32_dma3_pause(struct dma_chan *c)
{
struct stm32_dma3_chan *chan = to_stm32_dma3_chan(c);
int ret;
ret = stm32_dma3_chan_suspend(chan, true);
if (ret)
return ret;
chan->dma_status = DMA_PAUSED;
dev_dbg(chan2dev(chan), "vchan %pK: paused\n", &chan->vchan);
return 0;
}
static int stm32_dma3_resume(struct dma_chan *c)
{
struct stm32_dma3_chan *chan = to_stm32_dma3_chan(c);
stm32_dma3_chan_suspend(chan, false);
chan->dma_status = DMA_IN_PROGRESS;
dev_dbg(chan2dev(chan), "vchan %pK: resumed\n", &chan->vchan);
return 0;
}
static int stm32_dma3_terminate_all(struct dma_chan *c)
{
struct stm32_dma3_chan *chan = to_stm32_dma3_chan(c);
unsigned long flags;
LIST_HEAD(head);
spin_lock_irqsave(&chan->vchan.lock, flags);
if (chan->swdesc) {
vchan_terminate_vdesc(&chan->swdesc->vdesc);
chan->swdesc = NULL;
}
stm32_dma3_chan_stop(chan);
vchan_get_all_descriptors(&chan->vchan, &head);
spin_unlock_irqrestore(&chan->vchan.lock, flags);
vchan_dma_desc_free_list(&chan->vchan, &head);
dev_dbg(chan2dev(chan), "vchan %pK: terminated\n", &chan->vchan);
return 0;
}
static void stm32_dma3_synchronize(struct dma_chan *c)
{
struct stm32_dma3_chan *chan = to_stm32_dma3_chan(c);
vchan_synchronize(&chan->vchan);
}
static enum dma_status stm32_dma3_tx_status(struct dma_chan *c, dma_cookie_t cookie,
struct dma_tx_state *txstate)
{
struct stm32_dma3_chan *chan = to_stm32_dma3_chan(c);
struct stm32_dma3_swdesc *swdesc = NULL;
enum dma_status status;
unsigned long flags;
struct virt_dma_desc *vd;
status = dma_cookie_status(c, cookie, txstate);
if (status == DMA_COMPLETE)
return status;
if (!txstate)
return chan->dma_status;
spin_lock_irqsave(&chan->vchan.lock, flags);
vd = vchan_find_desc(&chan->vchan, cookie);
if (vd)
swdesc = to_stm32_dma3_swdesc(vd);
else if (chan->swdesc && chan->swdesc->vdesc.tx.cookie == cookie)
swdesc = chan->swdesc;
/* Get residue/in_flight_bytes only if a transfer is currently running (swdesc != NULL) */
if (swdesc)
stm32_dma3_chan_set_residue(chan, swdesc, txstate);
spin_unlock_irqrestore(&chan->vchan.lock, flags);
return chan->dma_status;
}
static void stm32_dma3_issue_pending(struct dma_chan *c)
{
struct stm32_dma3_chan *chan = to_stm32_dma3_chan(c);
unsigned long flags;
spin_lock_irqsave(&chan->vchan.lock, flags);
if (vchan_issue_pending(&chan->vchan) && !chan->swdesc) {
dev_dbg(chan2dev(chan), "vchan %pK: issued\n", &chan->vchan);
stm32_dma3_chan_start(chan);
}
spin_unlock_irqrestore(&chan->vchan.lock, flags);
}
static bool stm32_dma3_filter_fn(struct dma_chan *c, void *fn_param)
{
struct stm32_dma3_chan *chan = to_stm32_dma3_chan(c);
struct stm32_dma3_ddata *ddata = to_stm32_dma3_ddata(chan);
struct stm32_dma3_dt_conf *conf = fn_param;
u32 mask, semcr;
int ret;
dev_dbg(c->device->dev, "%s(%s): req_line=%d ch_conf=%08x tr_conf=%08x\n",
__func__, dma_chan_name(c), conf->req_line, conf->ch_conf, conf->tr_conf);
if (!of_property_read_u32(c->device->dev->of_node, "dma-channel-mask", &mask))
if (!(mask & BIT(chan->id)))
return false;
ret = pm_runtime_resume_and_get(ddata->dma_dev.dev);
if (ret < 0)
return false;
semcr = readl_relaxed(ddata->base + STM32_DMA3_CSEMCR(chan->id));
pm_runtime_put_sync(ddata->dma_dev.dev);
/* Check if chan is free */
if (semcr & CSEMCR_SEM_MUTEX)
return false;
/* Check if chan fifo fits well */
if (FIELD_GET(STM32_DMA3_DT_FIFO, conf->ch_conf) != chan->fifo_size)
return false;
return true;
}
static struct dma_chan *stm32_dma3_of_xlate(struct of_phandle_args *dma_spec, struct of_dma *ofdma)
{
struct stm32_dma3_ddata *ddata = ofdma->of_dma_data;
dma_cap_mask_t mask = ddata->dma_dev.cap_mask;
struct stm32_dma3_dt_conf conf;
struct stm32_dma3_chan *chan;
struct dma_chan *c;
if (dma_spec->args_count < 3) {
dev_err(ddata->dma_dev.dev, "Invalid args count\n");
return NULL;
}
conf.req_line = dma_spec->args[0];
conf.ch_conf = dma_spec->args[1];
conf.tr_conf = dma_spec->args[2];
if (conf.req_line >= ddata->dma_requests) {
dev_err(ddata->dma_dev.dev, "Invalid request line\n");
return NULL;
}
/* Request dma channel among the generic dma controller list */
c = dma_request_channel(mask, stm32_dma3_filter_fn, &conf);
if (!c) {
dev_err(ddata->dma_dev.dev, "No suitable channel found\n");
return NULL;
}
chan = to_stm32_dma3_chan(c);
chan->dt_config = conf;
chan->config_set |= STM32_DMA3_CFG_SET_DT;
return c;
}
static u32 stm32_dma3_check_rif(struct stm32_dma3_ddata *ddata)
{
u32 chan_reserved, mask = 0, i, ccidcfgr, invalid_cid = 0;
/* Reserve Secure channels */
chan_reserved = readl_relaxed(ddata->base + STM32_DMA3_SECCFGR);
/*
* CID filtering must be configured to ensure that the DMA3 channel will inherit the CID of
* the processor which is configuring and using the given channel.
* In case CID filtering is not configured, dma-channel-mask property can be used to
* specify available DMA channels to the kernel.
*/
of_property_read_u32(ddata->dma_dev.dev->of_node, "dma-channel-mask", &mask);
/* Reserve !CID-filtered not in dma-channel-mask, static CID != CID1, CID1 not allowed */
for (i = 0; i < ddata->dma_channels; i++) {
ccidcfgr = readl_relaxed(ddata->base + STM32_DMA3_CCIDCFGR(i));
if (!(ccidcfgr & CCIDCFGR_CFEN)) { /* !CID-filtered */
invalid_cid |= BIT(i);
if (!(mask & BIT(i))) /* Not in dma-channel-mask */
chan_reserved |= BIT(i);
} else { /* CID-filtered */
if (!(ccidcfgr & CCIDCFGR_SEM_EN)) { /* Static CID mode */
if (FIELD_GET(CCIDCFGR_SCID, ccidcfgr) != CCIDCFGR_CID1)
chan_reserved |= BIT(i);
} else { /* Semaphore mode */
if (!FIELD_GET(CCIDCFGR_SEM_WLIST_CID1, ccidcfgr))
chan_reserved |= BIT(i);
ddata->chans[i].semaphore_mode = true;
}
}
dev_dbg(ddata->dma_dev.dev, "chan%d: %s mode, %s\n", i,
!(ccidcfgr & CCIDCFGR_CFEN) ? "!CID-filtered" :
ddata->chans[i].semaphore_mode ? "Semaphore" : "Static CID",
(chan_reserved & BIT(i)) ? "denied" :
mask & BIT(i) ? "force allowed" : "allowed");
}
if (invalid_cid)
dev_warn(ddata->dma_dev.dev, "chan%*pbl have invalid CID configuration\n",
ddata->dma_channels, &invalid_cid);
return chan_reserved;
}
static const struct of_device_id stm32_dma3_of_match[] = {
{ .compatible = "st,stm32mp25-dma3", },
{ /* sentinel */ },
};
MODULE_DEVICE_TABLE(of, stm32_dma3_of_match);
static int stm32_dma3_probe(struct platform_device *pdev)
{
struct device_node *np = pdev->dev.of_node;
struct stm32_dma3_ddata *ddata;
struct reset_control *reset;
struct stm32_dma3_chan *chan;
struct dma_device *dma_dev;
u32 master_ports, chan_reserved, i, verr;
u64 hwcfgr;
int ret;
ddata = devm_kzalloc(&pdev->dev, sizeof(*ddata), GFP_KERNEL);
if (!ddata)
return -ENOMEM;
platform_set_drvdata(pdev, ddata);
dma_dev = &ddata->dma_dev;
ddata->base = devm_platform_ioremap_resource(pdev, 0);
if (IS_ERR(ddata->base))
return PTR_ERR(ddata->base);
ddata->clk = devm_clk_get(&pdev->dev, NULL);
if (IS_ERR(ddata->clk))
return dev_err_probe(&pdev->dev, PTR_ERR(ddata->clk), "Failed to get clk\n");
reset = devm_reset_control_get_optional(&pdev->dev, NULL);
if (IS_ERR(reset))
return dev_err_probe(&pdev->dev, PTR_ERR(reset), "Failed to get reset\n");
ret = clk_prepare_enable(ddata->clk);
if (ret)
return dev_err_probe(&pdev->dev, ret, "Failed to enable clk\n");
reset_control_reset(reset);
INIT_LIST_HEAD(&dma_dev->channels);
dma_cap_set(DMA_SLAVE, dma_dev->cap_mask);
dma_cap_set(DMA_PRIVATE, dma_dev->cap_mask);
dma_cap_set(DMA_CYCLIC, dma_dev->cap_mask);
dma_cap_set(DMA_MEMCPY, dma_dev->cap_mask);
dma_dev->dev = &pdev->dev;
/*
* This controller supports up to 8-byte buswidth depending on the port used and the
* channel, and can only access address at even boundaries, multiple of the buswidth.
*/
dma_dev->copy_align = DMAENGINE_ALIGN_8_BYTES;
dma_dev->src_addr_widths = BIT(DMA_SLAVE_BUSWIDTH_1_BYTE) |
BIT(DMA_SLAVE_BUSWIDTH_2_BYTES) |
BIT(DMA_SLAVE_BUSWIDTH_4_BYTES) |
BIT(DMA_SLAVE_BUSWIDTH_8_BYTES);
dma_dev->dst_addr_widths = BIT(DMA_SLAVE_BUSWIDTH_1_BYTE) |
BIT(DMA_SLAVE_BUSWIDTH_2_BYTES) |
BIT(DMA_SLAVE_BUSWIDTH_4_BYTES) |
BIT(DMA_SLAVE_BUSWIDTH_8_BYTES);
dma_dev->directions = BIT(DMA_DEV_TO_MEM) | BIT(DMA_MEM_TO_DEV) | BIT(DMA_MEM_TO_MEM);
dma_dev->descriptor_reuse = true;
dma_dev->max_sg_burst = STM32_DMA3_MAX_SEG_SIZE;
dma_dev->residue_granularity = DMA_RESIDUE_GRANULARITY_BURST;
dma_dev->device_alloc_chan_resources = stm32_dma3_alloc_chan_resources;
dma_dev->device_free_chan_resources = stm32_dma3_free_chan_resources;
dma_dev->device_prep_dma_memcpy = stm32_dma3_prep_dma_memcpy;
dma_dev->device_prep_slave_sg = stm32_dma3_prep_slave_sg;
dma_dev->device_prep_dma_cyclic = stm32_dma3_prep_dma_cyclic;
dma_dev->device_caps = stm32_dma3_caps;
dma_dev->device_config = stm32_dma3_config;
dma_dev->device_pause = stm32_dma3_pause;
dma_dev->device_resume = stm32_dma3_resume;
dma_dev->device_terminate_all = stm32_dma3_terminate_all;
dma_dev->device_synchronize = stm32_dma3_synchronize;
dma_dev->device_tx_status = stm32_dma3_tx_status;
dma_dev->device_issue_pending = stm32_dma3_issue_pending;
/* if dma_channels is not modified, get it from hwcfgr1 */
if (of_property_read_u32(np, "dma-channels", &ddata->dma_channels)) {
hwcfgr = readl_relaxed(ddata->base + STM32_DMA3_HWCFGR1);
ddata->dma_channels = FIELD_GET(G_NUM_CHANNELS, hwcfgr);
}
/* if dma_requests is not modified, get it from hwcfgr2 */
if (of_property_read_u32(np, "dma-requests", &ddata->dma_requests)) {
hwcfgr = readl_relaxed(ddata->base + STM32_DMA3_HWCFGR2);
ddata->dma_requests = FIELD_GET(G_MAX_REQ_ID, hwcfgr) + 1;
}
/* G_MASTER_PORTS, G_M0_DATA_WIDTH_ENC, G_M1_DATA_WIDTH_ENC in HWCFGR1 */
hwcfgr = readl_relaxed(ddata->base + STM32_DMA3_HWCFGR1);
master_ports = FIELD_GET(G_MASTER_PORTS, hwcfgr);
ddata->ports_max_dw[0] = FIELD_GET(G_M0_DATA_WIDTH_ENC, hwcfgr);
if (master_ports == AXI64 || master_ports == AHB32) /* Single master port */
ddata->ports_max_dw[1] = DW_INVALID;
else /* Dual master ports */
ddata->ports_max_dw[1] = FIELD_GET(G_M1_DATA_WIDTH_ENC, hwcfgr);
ddata->chans = devm_kcalloc(&pdev->dev, ddata->dma_channels, sizeof(*ddata->chans),
GFP_KERNEL);
if (!ddata->chans) {
ret = -ENOMEM;
goto err_clk_disable;
}
chan_reserved = stm32_dma3_check_rif(ddata);
if (chan_reserved == GENMASK(ddata->dma_channels - 1, 0)) {
ret = -ENODEV;
dev_err_probe(&pdev->dev, ret, "No channel available, abort registration\n");
goto err_clk_disable;
}
/* G_FIFO_SIZE x=0..7 in HWCFGR3 and G_FIFO_SIZE x=8..15 in HWCFGR4 */
hwcfgr = readl_relaxed(ddata->base + STM32_DMA3_HWCFGR3);
hwcfgr |= ((u64)readl_relaxed(ddata->base + STM32_DMA3_HWCFGR4)) << 32;
for (i = 0; i < ddata->dma_channels; i++) {
if (chan_reserved & BIT(i))
continue;
chan = &ddata->chans[i];
chan->id = i;
chan->fifo_size = get_chan_hwcfg(i, G_FIFO_SIZE(i), hwcfgr);
/* If chan->fifo_size > 0 then half of the fifo size, else no burst when no FIFO */
chan->max_burst = (chan->fifo_size) ? (1 << (chan->fifo_size + 1)) / 2 : 0;
}
ret = dmaenginem_async_device_register(dma_dev);
if (ret)
goto err_clk_disable;
for (i = 0; i < ddata->dma_channels; i++) {
char name[12];
if (chan_reserved & BIT(i))
continue;
chan = &ddata->chans[i];
snprintf(name, sizeof(name), "dma%dchan%d", ddata->dma_dev.dev_id, chan->id);
chan->vchan.desc_free = stm32_dma3_chan_vdesc_free;
vchan_init(&chan->vchan, dma_dev);
ret = dma_async_device_channel_register(&ddata->dma_dev, &chan->vchan.chan, name);
if (ret) {
dev_err_probe(&pdev->dev, ret, "Failed to register channel %s\n", name);
goto err_clk_disable;
}
ret = platform_get_irq(pdev, i);
if (ret < 0)
goto err_clk_disable;
chan->irq = ret;
ret = devm_request_irq(&pdev->dev, chan->irq, stm32_dma3_chan_irq, 0,
dev_name(chan2dev(chan)), chan);
if (ret) {
dev_err_probe(&pdev->dev, ret, "Failed to request channel %s IRQ\n",
dev_name(chan2dev(chan)));
goto err_clk_disable;
}
}
ret = of_dma_controller_register(np, stm32_dma3_of_xlate, ddata);
if (ret) {
dev_err_probe(&pdev->dev, ret, "Failed to register controller\n");
goto err_clk_disable;
}
verr = readl_relaxed(ddata->base + STM32_DMA3_VERR);
pm_runtime_set_active(&pdev->dev);
pm_runtime_enable(&pdev->dev);
pm_runtime_get_noresume(&pdev->dev);
pm_runtime_put(&pdev->dev);
dev_info(&pdev->dev, "STM32 DMA3 registered rev:%lu.%lu\n",
FIELD_GET(VERR_MAJREV, verr), FIELD_GET(VERR_MINREV, verr));
return 0;
err_clk_disable:
clk_disable_unprepare(ddata->clk);
return ret;
}
static void stm32_dma3_remove(struct platform_device *pdev)
{
pm_runtime_disable(&pdev->dev);
}
static int stm32_dma3_runtime_suspend(struct device *dev)
{
struct stm32_dma3_ddata *ddata = dev_get_drvdata(dev);
clk_disable_unprepare(ddata->clk);
return 0;
}
static int stm32_dma3_runtime_resume(struct device *dev)
{
struct stm32_dma3_ddata *ddata = dev_get_drvdata(dev);
int ret;
ret = clk_prepare_enable(ddata->clk);
if (ret)
dev_err(dev, "Failed to enable clk: %d\n", ret);
return ret;
}
static const struct dev_pm_ops stm32_dma3_pm_ops = {
SYSTEM_SLEEP_PM_OPS(pm_runtime_force_suspend, pm_runtime_force_resume)
RUNTIME_PM_OPS(stm32_dma3_runtime_suspend, stm32_dma3_runtime_resume, NULL)
};
static struct platform_driver stm32_dma3_driver = {
.probe = stm32_dma3_probe,
.remove_new = stm32_dma3_remove,
.driver = {
.name = "stm32-dma3",
.of_match_table = stm32_dma3_of_match,
.pm = pm_ptr(&stm32_dma3_pm_ops),
},
};
static int __init stm32_dma3_init(void)
{
return platform_driver_register(&stm32_dma3_driver);
}
subsys_initcall(stm32_dma3_init);
MODULE_DESCRIPTION("STM32 DMA3 controller driver");
MODULE_AUTHOR("Amelie Delaunay <amelie.delaunay@foss.st.com>");
MODULE_LICENSE("GPL");
drivers/dma/stm32-mdma.c → drivers/dma/stm32/stm32-mdma.c
View file @ 7a46b17d
......@@ -30,7 +30,7 @@
#include <linux/reset.h>
#include <linux/slab.h>
#include "virt-dma.h"
#include "../virt-dma.h"
#define STM32_MDMA_GISR0 0x0000 /* MDMA Int Status Reg 1 */
......
drivers/dma/ti/cppi41.c
View file @ 7a46b17d
......@@ -1252,5 +1252,6 @@ static struct platform_driver cpp41_dma_driver = {
};
module_platform_driver(cpp41_dma_driver);
MODULE_DESCRIPTION("Texas Instruments CPPI 4.1 DMA support");
MODULE_LICENSE("GPL");
MODULE_AUTHOR("Sebastian Andrzej Siewior <bigeasy@linutronix.de>");
drivers/dma/ti/k3-psil.c
View file @ 7a46b17d
......@@ -106,4 +106,5 @@ int psil_set_new_ep_config(struct device *dev, const char *name,
return 0;
}
EXPORT_SYMBOL_GPL(psil_set_new_ep_config);
MODULE_DESCRIPTION("K3 PSI-L endpoint configuration");
MODULE_LICENSE("GPL v2");
drivers/dma/ti/k3-udma-glue.c
View file @ 7a46b17d
......@@ -1574,4 +1574,5 @@ static int __init k3_udma_glue_class_init(void)
}
module_init(k3_udma_glue_class_init);
MODULE_DESCRIPTION("TI K3 NAVSS DMA glue interface");
MODULE_LICENSE("GPL v2");
drivers/dma/ti/k3-udma.c
View file @ 7a46b17d
......@@ -4405,6 +4405,7 @@ static const struct of_device_id udma_of_match[] = {
},
{ /* Sentinel */ },
};
MODULE_DEVICE_TABLE(of, udma_of_match);
static struct udma_soc_data am654_soc_data = {
.oes = {
......@@ -4472,7 +4473,9 @@ static int udma_get_mmrs(struct platform_device *pdev, struct udma_dev *ud)
ud->rchan_cnt = UDMA_CAP2_RCHAN_CNT(cap2);
break;
case DMA_TYPE_BCDMA:
ud->bchan_cnt = BCDMA_CAP2_BCHAN_CNT(cap2);
ud->bchan_cnt = BCDMA_CAP2_BCHAN_CNT(cap2) +
BCDMA_CAP3_HBCHAN_CNT(cap3) +
BCDMA_CAP3_UBCHAN_CNT(cap3);
ud->tchan_cnt = BCDMA_CAP2_TCHAN_CNT(cap2);
ud->rchan_cnt = BCDMA_CAP2_RCHAN_CNT(cap2);
ud->rflow_cnt = ud->rchan_cnt;
......@@ -5621,6 +5624,7 @@ static struct platform_driver udma_driver = {
};
module_platform_driver(udma_driver);
MODULE_DESCRIPTION("Texas Instruments UDMA support");
MODULE_LICENSE("GPL v2");
/* Private interfaces to UDMA */
......
drivers/dma/ti/omap-dma.c
View file @ 7a46b17d
......@@ -1950,4 +1950,5 @@ static void __exit omap_dma_exit(void)
module_exit(omap_dma_exit);
MODULE_AUTHOR("Russell King");
MODULE_DESCRIPTION("Texas Instruments sDMA DMAengine support");
MODULE_LICENSE("GPL");
drivers/dma/virt-dma.c
View file @ 7a46b17d
......@@ -139,4 +139,5 @@ void vchan_init(struct virt_dma_chan *vc, struct dma_device *dmadev)
EXPORT_SYMBOL_GPL(vchan_init);
MODULE_AUTHOR("Russell King");
MODULE_DESCRIPTION("Virtual DMA channel support for DMAengine");
MODULE_LICENSE("GPL");
include/linux/dmaengine.h
View file @ 7a46b17d
......@@ -1608,7 +1608,8 @@ int dma_async_device_register(struct dma_device *device);
int dmaenginem_async_device_register(struct dma_device *device);
void dma_async_device_unregister(struct dma_device *device);
int dma_async_device_channel_register(struct dma_device *device,
struct dma_chan *chan);
struct dma_chan *chan,
const char *name);
void dma_async_device_channel_unregister(struct dma_device *device,
struct dma_chan *chan);
void dma_run_dependencies(struct dma_async_tx_descriptor *tx);
......
include/linux/firmware.h
View file @ 7a46b17d
......@@ -98,6 +98,10 @@ static inline bool firmware_request_builtin(struct firmware *fw,
#if IS_REACHABLE(CONFIG_FW_LOADER)
int request_firmware(const struct firmware **fw, const char *name,
struct device *device);
int firmware_request_nowait_nowarn(
struct module *module, const char *name,
struct device *device, gfp_t gfp, void *context,
void (*cont)(const struct firmware *fw, void *context));
int firmware_request_nowarn(const struct firmware **fw, const char *name,
struct device *device);
int firmware_request_platform(const struct firmware **fw, const char *name,
......@@ -123,6 +127,14 @@ static inline int request_firmware(const struct firmware **fw,
return -EINVAL;
}
static inline int firmware_request_nowait_nowarn(
struct module *module, const char *name,
struct device *device, gfp_t gfp, void *context,
void (*cont)(const struct firmware *fw, void *context))
{
return -EINVAL;
}
static inline int firmware_request_nowarn(const struct firmware **fw,
const char *name,
struct device *device)
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7