During interface removal, don't adjust sleep_auth if we are during
recovery. Since the FW is potentially dead we shouldn't talk to it.
Reported-by: Yossi Wortzel <yossiw@ti.com>
Signed-off-by: Arik Nemtsov <arik@wizery.com>
Signed-off-by: Luciano Coelho <coelho@ti.com>