If we stop the kthread, we may end up up'ing the sem twice, which seems
unintended.
Reported-by: Dan Carpenter <error27@gmail.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>