In that case, the firmware work will never be scheduled, will
never complete and thus the firmware will never be released.
Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>