From: Colin Leroy <colin@colino.net>

rmmod'ing pmac_zilog currently oopses because uart_unregister_driver(),
which nullifies drv->tty_driver, is called before uart_remove_one_port(),
which uses said drv->tty_driver.

The comment at top of uart_unregister_driver() specifically says we have
to have removed all our ports via uart_remove_one_port() before.