From testing on hardware the poll_enable_time isn't required and
sometimes causes the driver probe to fail so let's remove it.
Signed-off-by: Alistair Francis <alistair@alistair23.me>
Link: https://lore.kernel.org/r/20210803084456.198-5-alistair@alistair23.meSigned-off-by: Mark Brown <broonie@kernel.org>