commit 16b972a5 upstream.

We are going to disable runtime_pm and we're
removing the driver, we must disable the device
now.
Signed-off-by: Felipe Balbi <balbi@ti.com>
Signed-off-by: Ben Hutchings <ben@decadent.org.uk>