Fix the regression caused by commit ad78c918 ("usb: musb: dsps: just
start polling already") which causes polling the ID pin status even in
device-only mode.

Fixes: ad78c918 ("usb: musb: dsps: just start polling already")
Signed-off-by: Bin Liu <b-liu@ti.com>
Signed-off-by: Felipe Balbi <balbi@ti.com>