The tty below tty_port might get destroyed by the tty layer
while we hold a reference to it.
So we have to carry tty_port around...
Signed-off-by: Richard Weinberger <richard@nod.at>