Having 2 ways of tracking disconnection is too much, remove both and
instead simply set pdev->udev to NULL on disconnect. Also check for
pdev->udev being NULL in all possible entry paths.
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@redhat.com>