usb: gadget: udc: lpc32xx: do not rely on 'driver' argument

future patches will remove the extra 'driver'
argument to ->udc_stop(), in order to do that,
we must make sure that our UDC does not rely
on it first.
Signed-off-by: Felipe Balbi <balbi@ti.com>

drivers/usb/gadget/udc/lpc32xx_udc.c

@@ -2967,9 +2967,6 @@ static int lpc32xx_stop(struct usb_gadget *gadget,
 	int i;
 	struct lpc32xx_udc *udc = to_udc(gadget);
 
-	if (!driver || driver != udc->driver)
-		return -EINVAL;
-
 	for (i = IRQ_USB_LP; i <= IRQ_USB_ATX; i++)
 		disable_irq(udc->udp_irq[i]);