It seems that the kernel has no business managing this clock: once the SP
clock is disabled, it's not sufficient to just enable in order to bring the
SP core back up. Just let the firmware keep it enabled and don't expose it
to drivers.

This reverts commit fc27c239.

Link: https://lore.kernel.org/lkml/154783267051.169631.3197836544646625747@swboyd.mtv.corp.google.com/Signed-off-by: Lubomir Rintel <lkundrak@v3.sk>
Signed-off-by: Stephen Boyd <sboyd@kernel.org>