In testing I've never seen it go past 1 retry anyways but better
safe than sorry.

Reported by Droste on irc.
Signed-off-by: Dave Airlie <airlied@redhat.com>