When the clock is specified, there could be other errors besides
the EPROBE_DEFER so don't ignore them.
Signed-off-by: Nicolin Chen <nicoleotsuka@gmail.com>
Acked-by: Paul Handrigan <Paul.Handrigan@cirrus.com>
Signed-off-by: Mark Brown <broonie@kernel.org>