usb: gadget: core: do not try to disconnect gadget if it is not connected

Current UDC core connects gadget during the loading gadget flow
(udc_bind_to_driver->usb_udc_connect_control), but for
platforms which do not connect gadget if the VBUS is not there,
they call usb_gadget_disconnect, but the gadget is not connected
at this time, notify disconnecton for the gadget driver is meaningless
at this situation.

Signed-off-by: Peter Chen <peter.chen@nxp.com>
Signed-off-by: Felipe Balbi <balbi@kernel.org>