The PCIe programming sequence in TRM suggests CLKSTCTRL of PCIe should be
set to SW_WKUP. There are no issues when CLKSTCTRL is set to HW_AUTO in RC
mode. However in EP mode, the host system is not able to access the
MEMSPACE and setting the CLKSTCTRL to SW_WKUP fixes it.

Acked-by: Tony Lindgren <tony@atomide.com>
Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

Add maintainer for the newly introduced PCI Endpoint framework.

Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

Add documentation to help users use pci-epf-test function driver and
pci_endpoint_test host driver for testing PCI.

Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

Add a simple test script that invokes the pcitest userspace tool to perform
all the PCI endpoint tests (BAR tests, interrupt tests, read tests, write
tests and copy tests).

Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

Add a userspace tool to invoke the ioctls exposed by the PCI endpoint test
driver to perform various PCI tests.

Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

Add Documentation for pci-endpoint-test driver.

Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

Add PCI endpoint test driver that can verify base address register, legacy
interrupt/MSI interrupt and read/write/copy buffers between host and
device. The corresponding pci-epf-test function driver should be used on
the EP side.

Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

Add device IDs for DRA74x and DRA72x devices. These devices have
configurable PCI endpoint.

Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

Update device tree binding documentation of TI's dra7xx PCI controller to
include property for enabling unaligned mem access.

Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Acked-by: Rob Herring <robh@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

According to errata i870, access to the PCIe slave port that are not 32-bit
aligned will result in incorrect mapping to TLP Address and Byte enable
fields.

Accessing non 32-bit aligned data causes incorrect data in the target
buffer if memcpy is used. Implement the workaround for this errata here.

Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

Add device tree binding documentation for PCI dra7xx EP mode.

Acked-by: Rob Herring <robh@kernel.org>
Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

The PCIe controller integrated in dra7xx SoCs is capable of operating in
endpoint mode. Add endpoint mode support to dra7xx driver.

Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

No functional change. Split dra7xx_pcie_enable_interrupts() into
dra7xx_pcie_enable_wrapper_interrupts() and
dra7xx_pcie_enable_msi_interrupts() so that wrapper interrupts and MSI
interrupts can be enabled independently.  This is in preparation for adding
EP mode support to dra7xx driver since EP mode doesn't have to enable
msi_interrupts.

Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

Add device tree binding documentation for PCI designware EP mode.

Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Acked-by: Rob Herring <robh@kernel.org>

Add endpoint mode support to designware driver. This uses the EP Core layer
introduced recently to add endpoint mode support.  *Any* function driver
can now use this designware device in order to achieve the EP
functionality.

Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

Add binding documentation for pci-test endpoint function that helps in
adding and configuring pci-test endpoint function.

Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

Adds a new endpoint function driver (to program the virtual test device)
making use of the EP-core library.

[bhelgaas: fold in pci_epf_test_probe() -ENOMEM test from Wei Yongjun
<weiyongjun1@huawei.com>]
Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

Add specification for the *PCI test* virtual function device. The endpoint
function driver and the host PCI driver should be created based on this
specification.

Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

Invoke APIs provided by pci-ep-cfs to create configfs entry for every EPC
device and EPF driver to help users in creating EPF device and binding the
EPF device to the EPC device.

Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

Add Documentation to help users use PCI endpoint to configure PCI endpoint
function and to bind the endpoint function with endpoint controller.

Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Acked-By: Joao Pinto <jpinto@synopsys.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

Introduce a new configfs entry to configure the EP function (like
configuring the standard configuration header entries) and to bind the EP
function with EP controller.

Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

Add Documentation to help users use endpoint library to enable endpoint
mode in the PCI controller and add new PCI endpoint functions.

Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Acked-By: Joao Pinto <jpinto@synopsys.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

Introduce a new EP core layer in order to support endpoint functions in
linux kernel. This comprises the EPC library (Endpoint Controller Library)
and EPF library (Endpoint Function Library). EPC library implements
functions specific to an endpoint controller and EPF library implements
functions specific to an endpoint function.

Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Acked-by: Joao Pinto <jpinto@synopsys.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

Currently devm_request_irq() is being called before base, PCI fields of
dra7xx_pcie structure are populated. It is called even before
pm_runtime_enable() and pm_runtime_get_sync() are called. This will lead
to exceptions if in case an interrupt is triggered before the all of the
above are done. Hence push the devm_request_irq() call to the end of the
probe.

Signed-off-by: Keerthy <j-keerthy@ti.com>
Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

No functional change. Rename dw_pcie_writel_unroll/dw_pcie_readl_unroll to
dw_pcie_writel_ob_unroll/dw_pcie_readl_ob_unroll respectively as these
functions are used to perform only outbound configurations. Also move
these _unroll configurations to a separate function.

Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

Previously dbi accessors can be used to access data of size 4 bytes. But
there might be situations (like accessing MSI_MESSAGE_CONTROL in order to
set/get the number of required MSI interrupts in EP mode) where dbi
accessors must be used to access data of size 2. This is in preparation
for adding endpoint mode support to designware driver.

Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Acked-by: Niklas Cassel <niklas.cassel@axis.com>
Cc: Jingoo Han <jingoohan1@gmail.com>
Cc: Joao Pinto <Joao.Pinto@synopsys.com>

dwc has 2 dbi address space labeled dbics and dbics2. The existing helper
to access dbi address space can access only dbics. However dbics2 has to
be accessed for programming the BAR registers in the case of EP mode. This
is in preparation for adding EP mode support to dwc driver.

Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Acked-by: Niklas Cassel <niklas.cassel@axis.com>
Cc: Jingoo Han <jingoohan1@gmail.com>
Cc: Joao Pinto <Joao.Pinto@synopsys.com>

Populate cpu_addr_fixup ops to extract the least 28 bits of the
corresponding CPU address.

Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Acked-by: Niklas Cassel <niklas.cassel@axis.com>
Acked-by: Joao Pinto <jpinto@synopsys.com>

Populate cpu_addr_fixup ops to extract the least 28 bits of the
corresponding CPU address.

Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Acked-by: Joao Pinto <jpinto@synopsys.com>

Some platforms (like dra7xx) require only the least 28 bits of the
corresponding 32 bit CPU address to be programmed in the address
translation unit. This modified address is stored in io_base/mem_base/
cfg0_base/cfg1_base in dra7xx_pcie_host_init(). While this is okay for
host mode where the address range is fixed, device mode requires different
addresses to be programmed based on the host buffer address.  Add a new
ops to get the least 28 bits of the corresponding 32 bit CPU address and
invoke it before programming the address translation unit.

Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Acked-by: Joao Pinto <jpinto@synopsys.com>

The bug is that "val" is unsigned long but we only initialize 32 bits of
it.  Then we test "if (val)" and that might be true not because we set the
bits but because some were never initialized.

Fixes: f342d940

 ("PCI: exynos: Add support for MSI")
Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

Use "continue" to skip rest of the loop when possible to save an indent
level.  No functional change intended.

Suggested-by: walter harms <wharms@bfs.de>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

Fix a crash from dereferencing a NULL dw_pcie_ops pointer.  For example,
on ARTPEC-6:

  Unable to handle kernel NULL pointer dereference at virtual address 00000004
  pgd = c0204000
  [00000004] *pgd=00000000
  Internal error: Oops: 5 [#1] SMP ARM
  Modules linked in:
  CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.11.0-rc3-next-20170321 #1
  Hardware name: Axis ARTPEC-6 Platform
  task: db098000 task.stack: db096000
  PC is at dw_pcie_writel_dbi+0x2c/0xd0

Prior to 442ec4c0 ("PCI: dwc: all: Split struct pcie_port into
host-only and core structures"), every driver had a struct pcie_host_ops
with function pointers, typically used as:

  if (pp->ops->readl_rc)
    return pp->ops->readl_rc(...);

442ec4c0 split struct pcie_host_ops into two pieces: struct
dw_pcie_host_ops and struct dw_pcie_ops, so the above became:

  if (pci->ops->readl_dbi)
    return pci->ops->readl_dbi(...);

But pcie-artpec6.c and pcie-designware-plat.c don't need the dw_pcie_ops
pointers and didn't supply a pci->ops struct, which leads to NULL pointer
dereferences.

Supply an empty struct dw_pcie_ops to avoid the NULL pointer dereferences.

[bhelgaas: changelog]
Fixes: 442ec4c0

 ("PCI: dwc: all: Split struct pcie_port into host-only and core structures")
Signed-off-by: Niklas Cassel <niklas.cassel@axis.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Acked-by: Kishon Vijay Abraham I <kishon@ti.com>
Acked-by: Joao Pinto <jpinto@synopsys.com>

Without PCI_HOST_COMMON support enabled, we get a link error:

  drivers/pci/dwc/built-in.o: In function `hisi_pcie_map_bus':
  pcie-hisi.c:(.text+0x8860): undefined reference to `pci_ecam_map_bus'
  drivers/pci/dwc/built-in.o: In function `hisi_pcie_almost_ecam_probe':
  pcie-hisi.c:(.text+0x88b4): undefined reference to `pci_host_common_probe'

Add an explicit 'select', as the other users have.

Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Acked-by: Jingoo Han <jingoohan1@gmail.com>

SZ_16M PEM resource size includes PEM-specific register and its children
resources. Reservation of the whole SZ_16M range leads to child device
driver failure when pcieport driver is requesting resources:

  pcieport 0004:1f:00.0: can't enable device: BAR 0 [mem 0x87e0c0f00000-0x87e0c0ffffff 64bit] not claimed

So we cannot reserve full 16M here and instead we want to reserve
PEM-specific register only which is SZ_64K.

At the end increase PEM resource to SZ_16M since this is what
thunder_pem_init() call expects for proper initialization.

Fixes: 9abb27c7

 ("PCI: thunder-pem: Add legacy firmware support for Cavium ThunderX host controller")
Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
CC: stable@vger.kernel.org	# v4.10+

During early days of PCI quirks support, ThunderX firmware did not provide
PNP0c02 node with PCI configuration space and PEM-specific register ranges.
This means that for legacy FW we are not reserving these resources and
cannot gather PEM-specific resources for further PEM initialization.

To support already deployed legacy FW, calculate PEM-specific ranges and
provide resources reservation as fallback scenario into PEM driver when we
could not gather PEM reg base from ACPI tables.

Tested-by: Robert Richter <rrichter@cavium.com>
Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
Signed-off-by: Vadim Lomovtsev <Vadim.Lomovtsev@caviumnetworks.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Acked-by: Robert Richter <rrichter@cavium.com>
CC: stable@vger.kernel.org	# v4.10+

"CAV" is the only PNP/ACPI hardware ID vendor prefix assigned to Cavium so
fix this as it should be from day one.

Fixes: 44f22bd9

 ("PCI: Add MCFG quirks for Cavium ThunderX pass2.x host controller")
Tested-by: Robert Richter <rrichter@cavium.com>
Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Acked-by: Robert Richter <rrichter@cavium.com>
CC: stable@vger.kernel.org	# v4.10+

The host bridge memory window resource is inserted into the iomem_resource
tree and cannot be deallocated until the host bridge itself is removed.

Previously, the window was on the stack, which meant the iomem_resource
entry pointed into the stack and was corrupted as soon as the probe
function returned, which caused memory corruption and errors like this:

  pcie_iproc_bcma bcma0:8: resource collision: [mem 0x40000000-0x47ffffff] conflicts with PCIe MEM space [mem 0x40000000-0x47ffffff]

Move the memory window resource from the stack into struct iproc_pcie so
its lifetime matches that of the host bridge.

Fixes: c3245a56

 ("PCI: iproc: Request host bridge window resources")
Reported-and-tested-by: Rafał Miłecki <zajec5@gmail.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
CC: stable@vger.kernel.org	# v4.8+

We call pcie_aspm_exit_link_state() when we remove a device.  If the device
is the last PCIe function to be removed below a bridge and the bridge has
an ASPM link_state struct, we disable ASPM on the link.  Disabling ASPM
requires link->downstream (used in pcie_config_aspm_link()).

We previously set link->downstream in pcie_aspm_cap_init(), but only if the
device was not blacklisted.  Removing the blacklisted device caused a NULL
pointer dereference in the pcie_aspm_exit_link_state() ->
pcie_config_aspm_link() path:

  # echo 1 > /sys/bus/pci/devices/0000\:0b\:00.0/remove
  ...
   BUG: unable to handle kernel NULL pointer dereference at 0000000000000080
   IP: pcie_config_aspm_link+0x5d/0x2b0
   Call Trace:
    pcie_aspm_exit_link_state+0x75/0x130
    pci_stop_bus_device+0xa4/0xb0
    pci_stop_and_remove_bus_device_locked+0x1a/0x30
    remove_store+0x50/0x70
    dev_attr_store+0x18/0x30
    sysfs_kf_write+0x44/0x60
    kernfs_fop_write+0x10e/0x190
    __vfs_write+0x28/0x110
    ? rcu_read_lock_sched_held+0x5d/0x80
    ? rcu_sync_lockdep_assert+0x2c/0x60
    ? __sb_start_write+0x173/0x1a0
    ? vfs_write+0xb3/0x180
    vfs_write+0xc4/0x180
    SyS_write+0x49/0xa0
    do_syscall_64+0xa6/0x1c0
    entry_SYSCALL64_slow_path+0x25/0x25
   ---[ end trace bd187ee0267df5d9 ]---

To avoid this, set link->downstream in alloc_pcie_link_state(), so every
pcie_link_state structure has a valid link->downstream pointer.

[bhelgaas: changelog]
Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Acked-by: Rajat Jain <rajatja@google.com>
CC: stable@vger.kernel.org

QLogic ISP2722-based 16/32Gb Fibre Channel to PCIe Adapter has the VPD
access issue too, while read the common pci-sysfs access interface shown as

 /sys/devices/pci0000:00/0000:00:03.2/0000:0b:00.0/vpd

with simple 'cat' could cause system hang and panic:

  Kernel panic - not syncing: An NMI occurred. Depending on your system the reason for the NMI is logged in any one of the following resources:
  1. Integrated Management Log (IML)
  2. OA Syslog
  3. OA Forward Progress Log
  4. iLO Event Log
  CPU: 0 PID: 15070 Comm: udevadm Not tainted 4.1.12
  Hardware name: HP ProLiant DL380 Gen9/ProLiant DL380 Gen9, BIOS P89 12/27/2015
   0000000000000086 000000007f0cdf51 ffff880c4fa05d58 ffffffff817193de
   ffffffffa00b42d8 0000000000000075 ffff880c4fa05dd8 ffffffff81714072
   0000000000000008 ffff880c4fa05de8 ffff880c4fa05d88 000000007f0cdf51
  Call Trace:
   <NMI>  [<ffffffff817193de>] dump_stack+0x63/0x81
   [<ffffffff81714072>] panic+0xd0/0x20e
   [<ffffffffa00b390d>] hpwdt_pretimeout+0xdd/0xe0 [hpwdt]
   [<ffffffff81021fc9>] ? sched_clock+0x9/0x10
   [<ffffffff8101c101>] nmi_handle+0x91/0x170
   [<ffffffff8101c10c>] ? nmi_handle+0x9c/0x170
   [<ffffffff8101c5fe>] io_check_error+0x1e/0xa0
   [<ffffffff8101c719>] default_do_nmi+0x99/0x140
   [<ffffffff8101c8b4>] do_nmi+0xf4/0x170
   [<ffffffff817232c5>] end_repeat_nmi+0x1a/0x1e
   [<ffffffff815d724b>] ? pci_conf1_read+0xeb/0x120
   [<ffffffff815d724b>] ? pci_conf1_read+0xeb/0x120
   [<ffffffff815d724b>] ? pci_conf1_read+0xeb/0x120
   <<EOE>>  [<ffffffff815db4b3>] raw_pci_read+0x23/0x40
   [<ffffffff815db4fc>] pci_read+0x2c/0x30
   [<ffffffff8136f612>] pci_user_read_config_word+0x72/0x110
   [<ffffffff8136f746>] pci_vpd_pci22_wait+0x96/0x130
   [<ffffffff8136ff9b>] pci_vpd_pci22_read+0xdb/0x1a0
   [<ffffffff8136ea30>] pci_read_vpd+0x20/0x30
   [<ffffffff8137d590>] read_vpd_attr+0x30/0x40
   [<ffffffff8128e037>] sysfs_kf_bin_read+0x47/0x70
   [<ffffffff8128d24e>] kernfs_fop_read+0xae/0x180
   [<ffffffff8120dd97>] __vfs_read+0x37/0x100
   [<ffffffff812ba7e4>] ? security_file_permission+0x84/0xa0
   [<ffffffff8120e366>] ? rw_verify_area+0x56/0xe0
   [<ffffffff8120e476>] vfs_read+0x86/0x140
   [<ffffffff8120f3f5>] SyS_read+0x55/0xd0
   [<ffffffff81720f2e>] system_call_fastpath+0x12/0x71
  Shutting down cpus with NMI
  Kernel Offset: disabled
  drm_kms_helper: panic occurred, switching back to text console

So blacklist the access to its VPD.

Signed-off-by: Ethan Zhao <ethan.zhao@oracle.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
CC: stable@vger.kernel.org	# v4.6+