25165f79
("ASoC: rsnd: enable clock-frequency for both 44.1kHz/48kHz")
supports both 44.1kHz/48kHz clock-frequency settings for ADG
which will be used for AUDIO_OLKOUTn.
But some board doesn't need it, thus, it is not mandatory.

But, above patch didn't care about the case of "clock-frequency" DT
property was not present.
This patch ignores ADG settings if AUDIO_OLKOUTn was not used.
Signed-off-by: Marek Vasut <marek.vasut+renesas@gmail.com>
[Kuninori: tidyup not to break non AUDIO_OLKOUTn case]
Signed-off-by: Kuninori Morimoto <kuninori.morimoto.gx@renesas.com>
Signed-off-by: Mark Brown <broonie@kernel.org>