While investigating the previous patch, a bug caused poll to return
POLLHUP on the listening socket, which caused us to spin.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>