And drop it when we detach.  If the shared buffer is in vram,
we need to make sure we don't put the device into runtime
suspend.
Acked-by: Shashank Sharma <shashank.sharma@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>