This reverts commit 156e96ff.

This patch was found to cause some division issues on the i.MX8MP
which causes the video clocks to not properly divide when division
was greate than 8.  This causes video failures on disp1_pix and
disp2_pix clocks.

Until a better solution is found, we'll have to revert this.
Signed-off-by: Adam Ford <aford173@gmail.com>
Link: https://lore.kernel.org/r/20230423123828.1346511-1-aford173@gmail.comAcked-by: Peng Fan <peng.fan@nxp.com>
Signed-off-by: Stephen Boyd <sboyd@kernel.org>