Only -ENOENT from devm_clk_get() means that clock is not present in
device tree.  Other errors have their own meaning and should not be
ignored.

Simplify getting the clock which is in fact optional and also use
dev_err_probe() for handling deferred.
Signed-off-by: Krzysztof Kozlowski <krzk@kernel.org>
Acked-by: Florian Fainelli <f.fainelli@gmail.com>
Link: https://lore.kernel.org/r/20200902193658.20539-7-krzk@kernel.orgSigned-off-by: Ulf Hansson <ulf.hansson@linaro.org>