Only cancel any ongoing work after making sure, that no new work
can be scheduled. This fixes a race condition in the remove handler.
Signed-off-by: Ian Molton <ian@mnementh.co.uk>
Reviewed-by: Sebastian Reichel <sebastian.reichel@collabora.co.uk>
Signed-off-by: Marcel Holtmann <marcel@holtmann.org>