Make the code cleaner and avoid call clk_disable_unprepare()
Signed-off-by: Wu Bo <bo.wu@vivo.com>
Link: https://patch.msgid.link/20240822095249.1642713-1-bo.wu@vivo.comSigned-off-by: Mark Brown <broonie@kernel.org>