Add dwc_otg driver

Signed-off-by: popcornmix <popcornmix@gmail.com>

usb: dwc: fix lockdep false positive

Signed-off-by: Kari Suvanto <karis79@gmail.com>

usb: dwc: fix inconsistent lock state

Signed-off-by: Kari Suvanto <karis79@gmail.com>

Add FIQ patch to dwc_otg driver. Enable with dwc_otg.fiq_fix_enable=1. Should give about 10% more ARM performance.
Thanks to Gordon and Costas

Avoid dynamic memory allocation for channel lock in USB driver. Thanks ddv2005.

Add NAK holdoff scheme. Enabled by default, disable with dwc_otg.nak_holdoff_enable=0. Thanks gsh

Make sure we wait for the reset to finish

dwc_otg: fix bug in dwc_otg_hcd.c resulting in silent kernel
	 memory corruption, escalating to OOPS under high USB load.

dwc_otg: Fix unsafe access of QTD during URB enqueue

In dwc_otg_hcd_urb_enqueue during qtd creation, it was possible that the
transaction could complete almost immediately after the qtd was assigned
to a host channel during URB enqueue, which meant the qtd pointer was no
longer valid having been completed and removed. Usually, this resulted in
an OOPS during URB submission. By predetermining whether transactions
need to be queued or not, this unsafe pointer access is avoided.

This bug was only evident on the Pi model A where a device was attached
that had no periodic endpoints (e.g. USB pendrive or some wlan devices).

dwc_otg: Fix incorrect URB allocation error handling

If the memory allocation for a dwc_otg_urb failed, the kernel would OOPS
because for some reason a member of the *unallocated* struct was set to
zero. Error handling changed to fail correctly.

dwc_otg: fix potential use-after-free case in interrupt handler

If a transaction had previously aborted, certain interrupts are
enabled to track error counts and reset where necessary. On IN
endpoints the host generates an ACK interrupt near-simultaneously
with completion of transfer. In the case where this transfer had
previously had an error, this results in a use-after-free on
the QTD memory space with a 1-byte length being overwritten to
0x00.

dwc_otg: add handling of SPLIT transaction data toggle errors

Previously a data toggle error on packets from a USB1.1 device behind
a TT would result in the Pi locking up as the driver never handled
the associated interrupt. Patch adds basic retry mechanism and
interrupt acknowledgement to cater for either a chance toggle error or
for devices that have a broken initial toggle state (FT8U232/FT232BM).

dwc_otg: implement tasklet for returning URBs to usbcore hcd layer

The dwc_otg driver interrupt handler for transfer completion will spend
a very long time with interrupts disabled when a URB is completed -
this is because usb_hcd_giveback_urb is called from within the handler
which for a USB device driver with complicated processing (e.g. webcam)
will take an exorbitant amount of time to complete. This results in
missed completion interrupts for other USB packets which lead to them
being dropped due to microframe overruns.

This patch splits returning the URB to the usb hcd layer into a
high-priority tasklet. This will have most benefit for isochronous IN
transfers but will also have incidental benefit where multiple periodic
devices are active at once.

dwc_otg: fix NAK holdoff and allow on split transactions only

This corrects a bug where if a single active non-periodic endpoint
had at least one transaction in its qh, on frnum == MAX_FRNUM the qh
would get skipped and never get queued again. This would result in
a silent device until error detection (automatic or otherwise) would
either reset the device or flush and requeue the URBs.

Additionally the NAK holdoff was enabled for all transactions - this
would potentially stall a HS endpoint for 1ms if a previous error state
enabled this interrupt and the next response was a NAK. Fix so that
only split transactions get held off.

dwc_otg: Call usb_hcd_unlink_urb_from_ep with lock held in completion handler

usb_hcd_unlink_urb_from_ep must be called with the HCD lock held.  Calling it
asynchronously in the tasklet was not safe (regression in
c4564d4a).

This change unlinks it from the endpoint prior to queueing it for handling in
the tasklet, and also adds a check to ensure the urb is OK to be unlinked
before doing so.

NULL pointer dereference kernel oopses had been observed in usb_hcd_giveback_urb
when a USB device was unplugged/replugged during data transfer.  This effect
was reproduced using automated USB port power control, hundreds of replug
events were performed during active transfers to confirm that the problem was
eliminated.

USB fix using a FIQ to implement split transactions

This commit adds a FIQ implementaion that schedules
the split transactions using a FIQ so we don't get
held off by the interrupt latency of Linux

dwc_otg: fix device attributes and avoid kernel warnings on boot

dcw_otg: avoid logging function that can cause panics

See: https://github.com/raspberrypi/firmware/issues/21


Thanks to cleverca22 for fix

dwc_otg: mask correct interrupts after transaction error recovery

The dwc_otg driver will unmask certain interrupts on a transaction
that previously halted in the error state in order to reset the
QTD error count. The various fine-grained interrupt handlers do not
consider that other interrupts besides themselves were unmasked.

By disabling the two other interrupts only ever enabled in DMA mode
for this purpose, we can avoid unnecessary function calls in the
IRQ handler. This will also prevent an unneccesary FIQ interrupt
from being generated if the FIQ is enabled.

dwc_otg: fiq: prevent FIQ thrash and incorrect state passing to IRQ

In the case of a transaction to a device that had previously aborted
due to an error, several interrupts are enabled to reset the error
count when a device responds. This has the side-effect of making the
FIQ thrash because the hardware will generate multiple instances of
a NAK on an IN bulk/interrupt endpoint and multiple instances of ACK
on an OUT bulk/interrupt endpoint. Make the FIQ mask and clear the
associated interrupts.

Additionally, on non-split transactions make sure that only unmasked
interrupts are cleared. This caused a hard-to-trigger but serious
race condition when you had the combination of an endpoint awaiting
error recovery and a transaction completed on an endpoint - due to
the sequencing and timing of interrupts generated by the dwc_otg core,
it was possible to confuse the IRQ handler.

Fix function tracing

dwc_otg: whitespace cleanup in dwc_otg_urb_enqueue

dwc_otg: prevent OOPSes during device disconnects

The dwc_otg_urb_enqueue function is thread-unsafe. In particular the
access of urb->hcpriv, usb_hcd_link_urb_to_ep, dwc_otg_urb->qtd and
friends does not occur within a critical section and so if a device
was unplugged during activity there was a high chance that the
usbcore hub_thread would try to disable the endpoint with partially-
formed entries in the URB queue. This would result in BUG() or null
pointer dereferences.

Fix so that access of urb->hcpriv, enqueuing to the hardware and
adding to usbcore endpoint URB lists is contained within a single
critical section.

dwc_otg: prevent BUG() in TT allocation if hub address is > 16

A fixed-size array is used to track TT allocation. This was
previously set to 16 which caused a crash because
dwc_otg_hcd_allocate_port would read past the end of the array.

This was hit if a hub was plugged in which enumerated as addr > 16,
due to previous device resets or unplugs.

Also add #ifdef FIQ_DEBUG around hcd->hub_port_alloc[], which grows
to a large size if 128 hub addresses are supported. This field is
for debug only for tracking which frame an allocate happened in.

dwc_otg: make channel halts with unknown state less damaging

If the IRQ received a channel halt interrupt through the FIQ
with no other bits set, the IRQ would not release the host
channel and never complete the URB.

Add catchall handling to treat as a transaction error and retry.

dwc_otg: fiq_split: use TTs with more granularity

This fixes certain issues with split transaction scheduling.

- Isochronous multi-packet OUT transactions now hog the TT until
  they are completed - this prevents hubs aborting transactions
  if they get a periodic start-split out-of-order
- Don't perform TT allocation on non-periodic endpoints - this
  allows simultaneous use of the TT's bulk/control and periodic
  transaction buffers

This commit will mainly affect USB audio playback.

dwc_otg: fix potential sleep while atomic during urb enqueue

Fixes a regression introduced with eb1b482a. Kmalloc called from
dwc_otg_hcd_qtd_add / dwc_otg_hcd_qtd_create did not always have
the GPF_ATOMIC flag set. Force this flag when inside the larger
critical section.

dwc_otg: make fiq_split_enable imply fiq_fix_enable

Failing to set up the FIQ correctly would result in
"IRQ 32: nobody cared" errors in dmesg.

dwc_otg: prevent crashes on host port disconnects

Fix several issues resulting in crashes or inconsistent state
if a Model A root port was disconnected.

- Clean up queue heads properly in kill_urbs_in_qh_list by
  removing the empty QHs from the schedule lists
- Set the halt status properly to prevent IRQ handlers from
  using freed memory
- Add fiq_split related cleanup for saved registers
- Make microframe scheduling reclaim host channels if
  active during a disconnect
- Abort URBs with -ESHUTDOWN status response, informing
  device drivers so they respond in a more correct fashion
  and don't try to resubmit URBs
- Prevent IRQ handlers from attempting to handle channel
  interrupts if the associated URB was dequeued (and the
  driver state was cleared)

dwc_otg: prevent leaking URBs during enqueue

A dwc_otg_urb would get leaked if the HCD enqueue function
failed for any reason. Free the URB at the appropriate points.

dwc_otg: Enable NAK holdoff for control split transactions

Certain low-speed devices take a very long time to complete a
data or status stage of a control transaction, producing NAK
responses until they complete internal processing - the USB2.0
spec limit is up to 500mS. This causes the same type of interrupt
storm as seen with USB-serial dongles prior to c8edb238.

In certain circumstances, usually while booting, this interrupt
storm could cause SD card timeouts.

dwc_otg: Fix for occasional lockup on boot when doing a USB reset

dwc_otg: Don't issue traffic to LS devices in FS mode

Issuing low-speed packets when the root port is in full-speed mode
causes the root port to stop responding. Explicitly fail when
enqueuing URBs to a LS endpoint on a FS bus.

Fix ARM architecture issue with local_irq_restore()

If local_fiq_enable() is called before a local_irq_restore(flags) where
the flags variable has the F bit set, the FIQ will be erroneously disabled.

Fixup arch_local_irq_restore to avoid trampling the F bit in CPSR.

Also fix some of the hacks previously implemented for previous dwc_otg
incarnations.

dwc_otg: fiq_fsm: Base commit for driver rewrite

This commit removes the previous FIQ fixes entirely and adds fiq_fsm.

This rewrite features much more complete support for split transactions
and takes into account several OTG hardware bugs. High-speed
isochronous transactions are also capable of being performed by fiq_fsm.

All driver options have been removed and replaced with:
  - dwc_otg.fiq_enable (bool)
  - dwc_otg.fiq_fsm_enable (bool)
  - dwc_otg.fiq_fsm_mask (bitmask)
  - dwc_otg.nak_holdoff (unsigned int)

Defaults are specified such that fiq_fsm behaves similarly to the
previously implemented FIQ fixes.

fiq_fsm: Push error recovery into the FIQ when fiq_fsm is used

If the transfer associated with a QTD failed due to a bus error, the HCD
would retry the transfer up to 3 times (implementing the USB2.0
three-strikes retry in software).

Due to the masking mechanism used by fiq_fsm, it is only possible to pass
a single interrupt through to the HCD per-transfer.

In this instance host channels would fall off the radar because the error
reset would function, but the subsequent channel halt would be lost.

Push the error count reset into the FIQ handler.

fiq_fsm: Implement timeout mechanism

For full-speed endpoints with a large packet size, interrupt latency
runs the risk of the FIQ starting a transaction too late in a full-speed
frame. If the device is still transmitting data when EOF2 for the
downstream frame occurs, the hub will disable the port. This change is
not reflected in the hub status endpoint and the device becomes
unresponsive.

Prevent high-bandwidth transactions from being started too late in a
frame. The mechanism is not guaranteed: a combination of bit stuffing
and hub latency may still result in a device overrunning.

fiq_fsm: fix bounce buffer utilisation for Isochronous OUT

Multi-packet isochronous OUT transactions were subject to a few bounday
bugs. Fix them.

Audio playback is now much more robust: however, an issue stands with
devices that have adaptive sinks - ALSA plays samples too fast.

dwc_otg: Return full-speed frame numbers in HS mode

The frame counter increments on every *microframe* in high-speed mode.
Most device drivers expect this number to be in full-speed frames - this
caused considerable confusion to e.g. snd_usb_audio which uses the
frame counter to estimate the number of samples played.

fiq_fsm: save PID on completion of interrupt OUT transfers

Also add edge case handling for interrupt transports.

Note that for periodic split IN, data toggles are unimplemented in the
OTG host hardware - it unconditionally accepts any PID.

fiq_fsm: add missing case for fiq_fsm_tt_in_use()

Certain combinations of bitrate and endpoint activity could
result in a periodic transaction erroneously getting started
while the previous Isochronous OUT was still active.

fiq_fsm: clear hcintmsk for aborted transactions

Prevents the FIQ from erroneously handling interrupts
on a timed out channel.

fiq_fsm: enable by default

fiq_fsm: fix dequeues for non-periodic split transactions

If a dequeue happened between the SSPLIT and CSPLIT phases of the
transaction, the HCD would never receive an interrupt.

fiq_fsm: Disable by default

fiq_fsm: Handle HC babble errors

The HCTSIZ transfer size field raises a babble interrupt if
the counter wraps. Handle the resulting interrupt in this case.

dwc_otg: fix interrupt registration for fiq_enable=0

Additionally make the module parameter conditional for wherever
hcd->fiq_state is touched.

fiq_fsm: Enable by default

dwc_otg: Fix various issues with root port and transaction errors

Process the host port interrupts correctly (and don't trample them).
Root port hotplug now functional again.

Fix a few thinkos with the transaction error passthrough for fiq_fsm.

fiq_fsm: Implement hack for Split Interrupt transactions

Hubs aren't too picky about which endpoint we send Control type split
transactions to. By treating Interrupt transfers as Control, it is
possible to use the non-periodic queue in the OTG core as well as the
non-periodic FIFOs in the hub itself. This massively reduces the
microframe exclusivity/contention that periodic split transactions
otherwise have to enforce.

It goes without saying that this is a fairly egregious USB specification
violation, but it works.

Original idea by Hans Petter Selasky @ FreeBSD.org.

dwc_otg: FIQ support on SMP. Set up FIQ stack and handler on Core 0 only.

dwc_otg: introduce fiq_fsm_spin(un|)lock()

SMP safety for the FIQ relies on register read-modify write cycles being
completed in the correct order. Several places in the DWC code modify
registers also touched by the FIQ. Protect these by a bare-bones lock
mechanism.

This also makes it possible to run the FIQ and IRQ handlers on different
cores.

fiq_fsm: fix build on bcm2708 and bcm2709 platforms

dwc_otg: put some barriers back where they should be for UP

bcm2709/dwc_otg: Setup FIQ on core 1 if >1 core active

dwc_otg: fixup read-modify-write in critical paths

Be more careful about read-modify-write on registers that the FIQ
also touches.

Guard fiq_fsm_spin_lock with fiq_enable check

fiq_fsm: Falling out of the state machine isn't fatal

This edge case can be hit if the port is disabled while the FIQ is
in the middle of a transaction. Make the effects less severe.

Also get rid of the useless return value.

squash: dwc_otg: Allow to build without SMP

usb: core: make overcurrent messages more prominent

Hub overcurrent messages are more serious than "debug". Increase loglevel.

usb: dwc_otg: Don't use dma_to_virt()

Commit 6ce0d200 changes dma_to_virt() which breaks this driver.
Open code the old dma_to_virt() implementation to work around this.

Limit the use of __bus_to_virt() to cases where transfer_buffer_length
is set and transfer_buffer is not set. This is done to increase the
chance that this driver will also work on ARCH_BCM2835.

transfer_buffer should not be NULL if the length is set, but the
comment in the code indicates that there are situations where this
might happen. drivers/usb/isp1760/isp1760-hcd.c also has a similar
comment pointing to a possible: 'usb storage / SCSI bug'.

Signed-off-by: Noralf Trønnes <noralf@tronnes.org>

dwc_otg: Fix crash when fiq_enable=0

dwc_otg: fiq_fsm: Make high-speed isochronous strided transfers work properly

Certain low-bandwidth high-speed USB devices (specialist audio devices,
compressed-frame webcams) have packet intervals > 1 microframe.

Stride these transfers in the FIQ by using the start-of-frame interrupt
to restart the channel at the right time.

dwc_otg: Force host mode to fix incorrect compute module boards

dwc_otg: Add ARCH_BCM2835 support

Signed-off-by: Noralf Trønnes <noralf@tronnes.org>