dmaengine: imx-sdma: restart cyclic channel if needed

Under heavy load resulting in high interrupt latencies, it is possible
for imx UART requests to completely fill DMA buffer. When DMA channel
is triggered and no SDMA owned buffer is available, SDMA stops. Thanks
to the autoRTS feature, there is no data loss due to the SDMA stop if
the UART is using hardware flow control.

According to DMA Engine API Guide, DMA cyclic operation is performed
until explicitly stopped. Restart the buffer after handling channel loop
if the channel was stopped by SDMA.
Signed-off-by: Tomasz Moń <tomasz.mon@camlingroup.com>
Link: https://lore.kernel.org/r/20220117091955.1038937-1-tomasz.mon@camlingroup.comSigned-off-by: Vinod Koul <vkoul@kernel.org>

drivers/dma/imx-sdma.c

@@ -701,6 +701,11 @@ static int sdma_config_ownership(struct sdma_channel *sdmac,
 	return 0;
 }
 
+static int is_sdma_channel_enabled(struct sdma_engine *sdma, int channel)
+{
+	return !!(readl(sdma->regs + SDMA_H_STATSTOP) & BIT(channel));
+}
+
 static void sdma_enable_channel(struct sdma_engine *sdma, int channel)
 {
 	writel(BIT(channel), sdma->regs + SDMA_H_START);
@@ -860,6 +865,15 @@ static void sdma_update_channel_loop(struct sdma_channel *sdmac)
 		if (error)
 			sdmac->status = old_status;
 	}
+
+	/*
+	 * SDMA stops cyclic channel when DMA request triggers a channel and no SDMA
+	 * owned buffer is available (i.e. BD_DONE was set too late).
+	 */
+	if (!is_sdma_channel_enabled(sdmac->sdma, sdmac->channel)) {
+		dev_warn(sdmac->sdma->dev, "restart cyclic channel %d\n", sdmac->channel);
+		sdma_enable_channel(sdmac->sdma, sdmac->channel);
+	}
 }
 
 static void mxc_sdma_handle_channel_normal(struct sdma_channel *data)