driver core protects us with semaphores during probe/disconnect. We can
stop messing with the module usage counts.