This reverts
commit fe51bfb9.
Author: Ville Syrjälä <ville.syrjala@linux.intel.com>
Date:   Thu Mar 12 17:10:38 2015 +0200

CHV does not support intermediate frequencies so reverting the
patch that added it in the first place

Cc: stable@vger.kernel.org # v4.1+
Reviewed-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: Sivakumar Thulasimani <sivakumar.thulasimani@intel.com>
Signed-off-by: Jani Nikula <jani.nikula@intel.com>