usb: musb: only remove host/udc if it has been added

musb_shutdown() removes always USB host and device.
musb_init_controller() adds host and device depending on port_mode. If
port mode is set to HOST then the removal of UDC leads only to:
|(NULL device *): gadget not registered.
and nothing else happens. If port mode is set to DEVICE and we remove
the host then we oops in usb_remove_hcd().
This patch ensures that we only remove host in OTG/host mode and device
only in OTG/device mode to avoid any trouble.
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Signed-off-by: Felipe Balbi <balbi@ti.com>

drivers/usb/musb/musb_gadget.c

@@ -1812,6 +1812,8 @@ int musb_gadget_setup(struct musb *musb)
 
 void musb_gadget_cleanup(struct musb *musb)
 {
+	if (musb->port_mode == MUSB_PORT_MODE_HOST)
+		return;
 	usb_del_gadget_udc(&musb->g);
 }
 

drivers/usb/musb/musb_host.c

@@ -2628,6 +2628,8 @@ int musb_host_alloc(struct musb *musb)
 
 void musb_host_cleanup(struct musb *musb)
 {
+	if (musb->port_mode == MUSB_PORT_MODE_GADGET)
+		return;
 	usb_remove_hcd(musb->hcd);
 	musb->hcd = NULL;
 }