usb: gadget: core: do not try to disconnect gadget if it is not connected

Current UDC core connects gadget during the loading gadget flow
(udc_bind_to_driver->usb_udc_connect_control), but for
platforms which do not connect gadget if the VBUS is not there,
they call usb_gadget_disconnect, but the gadget is not connected
at this time, notify disconnecton for the gadget driver is meaningless
at this situation.
Signed-off-by: Peter Chen <peter.chen@nxp.com>
Signed-off-by: Felipe Balbi <balbi@kernel.org>

drivers/usb/gadget/udc/core.c

@@ -715,6 +715,9 @@ int usb_gadget_disconnect(struct usb_gadget *gadget)
 		goto out;
 	}
 
+	if (!gadget->connected)
+		goto out;
+
 	if (gadget->deactivated) {
 		/*
 		 * If gadget is deactivated we only save new state.