Make use of the swap macro and remove unnecessary variables swap.
This makes the code easier to read and maintain.

This code was detected with the help of Coccinelle.
Signed-off-by: Gustavo A. R. Silva <garsilva@embeddedor.com>
Acked-by: Han Xu <han.xu@nxp.com>
Signed-off-by: Boris Brezillon <boris.brezillon@free-electrons.com>