Commit 5b47b10e authored by Linus Torvalds's avatar Linus Torvalds
Browse files

Merge tag 'pci-v5.12-changes' of git://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci 

Pull PCI updates from Bjorn Helgaas:
 "Enumeration:
   - Remove unnecessary locking around _OSC (Bjorn Helgaas)
   - Clarify message about _OSC failure (Bjorn Helgaas)
   - Remove notification of PCIe bandwidth changes (Bjorn Helgaas)
   - Tidy checking of syscall user config accessors (Heiner Kallweit)

  Resource management:
   - Decline to resize resources if boot config must be preserved (Ard
     Biesheuvel)
   - Fix pci_register_io_range() memory leak (Geert Uytterhoeven)

  Error handling (Keith Busch):
   - Clear error status from the correct device
   - Retain error recovery status so drivers can use it after reset
   - Log the type of Port (Root or Switch Downstream) that we reset
   - Always request a reset for Downstream Ports in frozen state

  Endpoint framework and NTB (Kishon Vijay Abraham I):
   - Make *_get_first_free_bar() take into account 64 bit BAR
   - Add helper API to get the 'next' unreserved BAR
   - Make *_free_bar() return error codes on failure
   - Remove unused pci_epf_match_device()
   - Add support to associate secondary EPC with EPF
   - Add support in configfs to associate two EPCs with EPF
   - Add pci_epc_ops to map MSI IRQ
   - Add pci_epf_ops to expose function-specific attrs
   - Allow user to create sub-directory of 'EPF Device' directory
   - Implement ->msi_map_irq() ops for cadence
   - Configure LM_EP_FUNC_CFG based on epc->function_num_map for cadence
   - Add EP function driver to provide NTB functionality
   - Add support for EPF PCI Non-Transparent Bridge
   - Add specification for PCI NTB function device
   - Add PCI endpoint NTB function user guide
   - Add configfs binding documentation for pci-ntb endpoint function

  Broadcom STB PCIe controller driver:
   - Add support for BCM4908 and external PERST# signal controller
     (Rafał Miłecki)

  Cadence PCIe controller driver:
   - Retrain Link to work around Gen2 training defect (Nadeem Athani)
   - Fix merge botch in cdns_pcie_host_map_dma_ranges() (Krzysztof
     Wilczyński)

  Freescale Layerscape PCIe controller driver:
   - Add LX2160A rev2 EP mode support (Hou Zhiqiang)
   - Convert to builtin_platform_driver() (Michael Walle)

  MediaTek PCIe controller driver:
   - Fix OF node reference leak (Krzysztof Wilczyński)

  Microchip PolarFlare PCIe controller driver:
   - Add Microchip PolarFire PCIe controller driver (Daire McNamara)

  Qualcomm PCIe controller driver:
   - Use PHY_REFCLK_USE_PAD only for ipq8064 (Ansuel Smith)
   - Add support for ddrss_sf_tbu clock for sm8250 (Dmitry Baryshkov)

  Renesas R-Car PCIe controller driver:
   - Drop PCIE_RCAR config option (Lad Prabhakar)
   - Always allocate MSI addresses in 32bit space (Marek Vasut)

  Rockchip PCIe controller driver:
   - Add FriendlyARM NanoPi M4B DT binding (Chen-Yu Tsai)
   - Make 'ep-gpios' DT property optional (Chen-Yu Tsai)

  Synopsys DesignWare PCIe controller driver:
   - Work around ECRC configuration hardware defect (Vidya Sagar)
   - Drop support for config space in DT 'ranges' (Rob Herring)
   - Change size to u64 for EP outbound iATU (Shradha Todi)
   - Add upper limit address for outbound iATU (Shradha Todi)
   - Make dw_pcie ops optional (Jisheng Zhang)
   - Remove unnecessary dw_pcie_ops from al driver (Jisheng Zhang)

  Xilinx Versal CPM PCIe controller driver:
   - Fix OF node reference leak (Pan Bian)

  Miscellaneous:
   - Remove tango host controller driver (Arnd Bergmann)
   - Remove IRQ handler & data together (altera-msi, brcmstb, dwc)
     (Martin Kaiser)
   - Fix xgene-msi race in installing chained IRQ handler (Martin
     Kaiser)
   - Apply CONFIG_PCI_DEBUG to entire drivers/pci hierarchy (Junhao He)
   - Fix pci-bridge-emul array overruns (Russell King)
   - Remove obsolete uses of WARN_ON(in_interrupt()) (Sebastian Andrzej
     Siewior)"

* tag 'pci-v5.12-changes' of git://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci: (69 commits)
  PCI: qcom: Use PHY_REFCLK_USE_PAD only for ipq8064
  PCI: qcom: Add support for ddrss_sf_tbu clock
  dt-bindings: PCI: qcom: Document ddrss_sf_tbu clock for sm8250
  PCI: al: Remove useless dw_pcie_ops
  PCI: dwc: Don't assume the ops in dw_pcie always exist
  PCI: dwc: Add upper limit address for outbound iATU
  PCI: dwc: Change size to u64 for EP outbound iATU
  PCI: dwc: Drop support for config space in 'ranges'
  PCI: layerscape: Convert to builtin_platform_driver()
  PCI: layerscape: Add LX2160A rev2 EP mode support
  dt-bindings: PCI: layerscape: Add LX2160A rev2 compatible strings
  PCI: dwc: Work around ECRC configuration issue
  PCI/portdrv: Report reset for frozen channel
  PCI/AER: Specify the type of Port that was reset
  PCI/ERR: Retain status from error notification
  PCI/AER: Clear AER status from Root Port when resetting Downstream Port
  PCI/ERR: Clear status of the reporting device
  dt-bindings: arm: rockchip: Add FriendlyARM NanoPi M4B
  PCI: rockchip: Make 'ep-gpios' DT property optional
  Documentation: PCI: Add PCI endpoint NTB function user guide
  ...
parents 6c15f9e8 e18fb64b

Documentation/PCI/endpoint/function/binding/pci-ntb.rst

0 → 100644
+38 −0
Original line number Diff line number Diff line 
.. SPDX-License-Identifier: GPL-2.0



==========================

PCI NTB Endpoint Function

==========================



1) Create a subdirectory to pci_epf_ntb directory in configfs.



Standard EPF Configurable Fields:



================   ===========================================================

vendorid	   should be 0x104c

deviceid	   should be 0xb00d for TI's J721E SoC

revid		   don't care

progif_code	   don't care

subclass_code	   should be 0x00

baseclass_code	   should be 0x5

cache_line_size	   don't care

subsys_vendor_id   don't care

subsys_id	   don't care

interrupt_pin	   don't care

msi_interrupts	   don't care

msix_interrupts	   don't care

================   ===========================================================



2) Create a subdirectory to directory created in 1



NTB EPF specific configurable fields:



================   ===========================================================

db_count	   Number of doorbells; default = 4

mw1     	   size of memory window1

mw2     	   size of memory window2

mw3     	   size of memory window3

mw4     	   size of memory window4

num_mws     	   Number of memory windows; max = 4

spad_count     	   Number of scratchpad registers; default = 64

================   ===========================================================

Documentation/PCI/endpoint/index.rst

+3 −0
Original line number Diff line number Diff line
@@ -11,5 +11,8 @@ PCI Endpoint Framework 
   pci-endpoint-cfs

   pci-test-function

   pci-test-howto

   pci-ntb-function

   pci-ntb-howto



   function/binding/pci-test

   function/binding/pci-ntb

Documentation/PCI/endpoint/pci-endpoint-cfs.rst

+10 −0
Original line number Diff line number Diff line
@@ -68,6 +68,16 @@ created) 
				... subsys_vendor_id

				... subsys_id

				... interrupt_pin

                                ... primary/

			                ... <Symlink EPC Device1>/

                                ... secondary/

			                ... <Symlink EPC Device2>/



If an EPF device has to be associated with 2 EPCs (like in the case of

Non-transparent bridge), symlink of endpoint controller connected to primary

interface should be added in 'primary' directory and symlink of endpoint

controller connected to secondary interface should be added in 'secondary'

directory.



EPC Device

==========

Documentation/PCI/endpoint/pci-ntb-function.rst

0 → 100644
+348 −0
Original line number Diff line number Diff line 
.. SPDX-License-Identifier: GPL-2.0



=================

PCI NTB Function

=================



:Author: Kishon Vijay Abraham I <kishon@ti.com>



PCI Non-Transparent Bridges (NTB) allow two host systems to communicate

with each other by exposing each host as a device to the other host.

NTBs typically support the ability to generate interrupts on the remote

machine, expose memory ranges as BARs, and perform DMA.  They also support

scratchpads, which are areas of memory within the NTB that are accessible

from both machines.



PCI NTB Function allows two different systems (or hosts) to communicate

with each other by configuring the endpoint instances in such a way that

transactions from one system are routed to the other system.



In the below diagram, PCI NTB function configures the SoC with multiple

PCI Endpoint (EP) instances in such a way that transactions from one EP

controller are routed to the other EP controller. Once PCI NTB function

configures the SoC with multiple EP instances, HOST1 and HOST2 can

communicate with each other using SoC as a bridge.



.. code-block:: text



    +-------------+                                   +-------------+

    |             |                                   |             |

    |    HOST1    |                                   |    HOST2    |

    |             |                                   |             |

    +------^------+                                   +------^------+

           |                                                 |

           |                                                 |

 +---------|-------------------------------------------------|---------+

 |  +------v------+                                   +------v------+  |

 |  |             |                                   |             |  |

 |  |     EP      |                                   |     EP      |  |

 |  | CONTROLLER1 |                                   | CONTROLLER2 |  |

 |  |             <----------------------------------->             |  |

 |  |             |                                   |             |  |

 |  |             |                                   |             |  |

 |  |             |  SoC With Multiple EP Instances   |             |  |

 |  |             |  (Configured using NTB Function)  |             |  |

 |  +-------------+                                   +-------------+  |

 +---------------------------------------------------------------------+



Constructs used for Implementing NTB

====================================



	1) Config Region

	2) Self Scratchpad Registers

	3) Peer Scratchpad Registers

	4) Doorbell (DB) Registers

	5) Memory Window (MW)





Config Region:

--------------



Config Region is a construct that is specific to NTB implemented using NTB

Endpoint Function Driver. The host and endpoint side NTB function driver will

exchange information with each other using this region. Config Region has

Control/Status Registers for configuring the Endpoint Controller. Host can

write into this region for configuring the outbound Address Translation Unit

(ATU) and to indicate the link status. Endpoint can indicate the status of

commands issued by host in this region. Endpoint can also indicate the

scratchpad offset and number of memory windows to the host using this region.



The format of Config Region is given below. All the fields here are 32 bits.



.. code-block:: text



	+------------------------+

	|         COMMAND        |

	+------------------------+

	|         ARGUMENT       |

	+------------------------+

	|         STATUS         |

	+------------------------+

	|         TOPOLOGY       |

	+------------------------+

	|    ADDRESS (LOWER 32)  |

	+------------------------+

	|    ADDRESS (UPPER 32)  |

	+------------------------+

	|           SIZE         |

	+------------------------+

	|   NO OF MEMORY WINDOW  |

	+------------------------+

	|  MEMORY WINDOW1 OFFSET |

	+------------------------+

	|       SPAD OFFSET      |

	+------------------------+

	|        SPAD COUNT      |

	+------------------------+

	|      DB ENTRY SIZE     |

	+------------------------+

	|         DB DATA        |

	+------------------------+

	|            :           |

	+------------------------+

	|            :           |

	+------------------------+

	|         DB DATA        |

	+------------------------+





  COMMAND:



	NTB function supports three commands:



	  CMD_CONFIGURE_DOORBELL (0x1): Command to configure doorbell. Before

	invoking this command, the host should allocate and initialize

	MSI/MSI-X vectors (i.e., initialize the MSI/MSI-X Capability in the

	Endpoint). The endpoint on receiving this command will configure

	the outbound ATU such that transactions to Doorbell BAR will be routed

	to the MSI/MSI-X address programmed by the host. The ARGUMENT

	register should be populated with number of DBs to configure (in the

	lower 16 bits) and if MSI or MSI-X should be configured (BIT 16).



	  CMD_CONFIGURE_MW (0x2): Command to configure memory window (MW). The

	host invokes this command after allocating a buffer that can be

	accessed by remote host. The allocated address should be programmed

	in the ADDRESS register (64 bit), the size should be programmed in

	the SIZE register and the memory window index should be programmed

	in the ARGUMENT register. The endpoint on receiving this command

	will configure the outbound ATU such that transactions to MW BAR

	are routed to the address provided by the host.



	  CMD_LINK_UP (0x3): Command to indicate an NTB application is

	bound to the EP device on the host side. Once the endpoint

	receives this command from both the hosts, the endpoint will

	raise a LINK_UP event to both the hosts to indicate the host

	NTB applications can start communicating with each other.



  ARGUMENT:



	The value of this register is based on the commands issued in

	command register. See COMMAND section for more information.



  TOPOLOGY:



	Set to NTB_TOPO_B2B_USD for Primary interface

	Set to NTB_TOPO_B2B_DSD for Secondary interface



  ADDRESS/SIZE:



	Address and Size to be used while configuring the memory window.

	See "CMD_CONFIGURE_MW" for more info.



  MEMORY WINDOW1 OFFSET:



	Memory Window 1 and Doorbell registers are packed together in the

	same BAR. The initial portion of the region will have doorbell

	registers and the latter portion of the region is for memory window 1.

	This register will specify the offset of the memory window 1.



  NO OF MEMORY WINDOW:



	Specifies the number of memory windows supported by the NTB device.



  SPAD OFFSET:



	Self scratchpad region and config region are packed together in the

	same BAR. The initial portion of the region will have config region

	and the latter portion of the region is for self scratchpad. This

	register will specify the offset of the self scratchpad registers.



  SPAD COUNT:



	Specifies the number of scratchpad registers supported by the NTB

	device.



  DB ENTRY SIZE:



	Used to determine the offset within the DB BAR that should be written

	in order to raise doorbell. EPF NTB can use either MSI or MSI-X to

	ring doorbell (MSI-X support will be added later). MSI uses same

	address for all the interrupts and MSI-X can provide different

	addresses for different interrupts. The MSI/MSI-X address is provided

	by the host and the address it gives is based on the MSI/MSI-X

	implementation supported by the host. For instance, ARM platform

	using GIC ITS will have the same MSI-X address for all the interrupts.

	In order to support all the combinations and use the same mechanism

	for both MSI and MSI-X, EPF NTB allocates a separate region in the

	Outbound Address Space for each of the interrupts. This region will

	be mapped to the MSI/MSI-X address provided by the host. If a host

	provides the same address for all the interrupts, all the regions

	will be translated to the same address. If a host provides different

	addresses, the regions will be translated to different addresses. This

	will ensure there is no difference while raising the doorbell.



  DB DATA:



	EPF NTB supports 32 interrupts, so there are 32 DB DATA registers.

	This holds the MSI/MSI-X data that has to be written to MSI address

	for raising doorbell interrupt. This will be populated by EPF NTB

	while invoking CMD_CONFIGURE_DOORBELL.



Scratchpad Registers:

---------------------



  Each host has its own register space allocated in the memory of NTB endpoint

  controller. They are both readable and writable from both sides of the bridge.

  They are used by applications built over NTB and can be used to pass control

  and status information between both sides of a device.



  Scratchpad registers has 2 parts

	1) Self Scratchpad: Host's own register space

	2) Peer Scratchpad: Remote host's register space.



Doorbell Registers:

-------------------



  Doorbell Registers are used by the hosts to interrupt each other.



Memory Window:

--------------



  Actual transfer of data between the two hosts will happen using the

  memory window.



Modeling Constructs:

====================



There are 5 or more distinct regions (config, self scratchpad, peer

scratchpad, doorbell, one or more memory windows) to be modeled to achieve

NTB functionality. At least one memory window is required while more than

one is permitted. All these regions should be mapped to BARs for hosts to

access these regions.



If one 32-bit BAR is allocated for each of these regions, the scheme would

look like this:



======  ===============

BAR NO  CONSTRUCTS USED

======  ===============

BAR0    Config Region

BAR1    Self Scratchpad

BAR2    Peer Scratchpad

BAR3    Doorbell

BAR4    Memory Window 1

BAR5    Memory Window 2

======  ===============



However if we allocate a separate BAR for each of the regions, there would not

be enough BARs for all the regions in a platform that supports only 64-bit

BARs.



In order to be supported by most of the platforms, the regions should be

packed and mapped to BARs in a way that provides NTB functionality and

also makes sure the host doesn't access any region that it is not supposed

to.



The following scheme is used in EPF NTB Function:



======  ===============================

BAR NO  CONSTRUCTS USED

======  ===============================

BAR0    Config Region + Self Scratchpad

BAR1    Peer Scratchpad

BAR2    Doorbell + Memory Window 1

BAR3    Memory Window 2

BAR4    Memory Window 3

BAR5    Memory Window 4

======  ===============================



With this scheme, for the basic NTB functionality 3 BARs should be sufficient.



Modeling Config/Scratchpad Region:

----------------------------------



.. code-block:: text



 +-----------------+------->+------------------+        +-----------------+

 |       BAR0      |        |  CONFIG REGION   |        |       BAR0      |

 +-----------------+----+   +------------------+<-------+-----------------+

 |       BAR1      |    |   |SCRATCHPAD REGION |        |       BAR1      |

 +-----------------+    +-->+------------------+<-------+-----------------+

 |       BAR2      |            Local Memory            |       BAR2      |

 +-----------------+                                    +-----------------+

 |       BAR3      |                                    |       BAR3      |

 +-----------------+                                    +-----------------+

 |       BAR4      |                                    |       BAR4      |

 +-----------------+                                    +-----------------+

 |       BAR5      |                                    |       BAR5      |

 +-----------------+                                    +-----------------+

   EP CONTROLLER 1                                        EP CONTROLLER 2



Above diagram shows Config region + Scratchpad region for HOST1 (connected to

EP controller 1) allocated in local memory. The HOST1 can access the config

region and scratchpad region (self scratchpad) using BAR0 of EP controller 1.

The peer host (HOST2 connected to EP controller 2) can also access this

scratchpad region (peer scratchpad) using BAR1 of EP controller 2. This

diagram shows the case where Config region and Scratchpad regions are allocated

for HOST1, however the same is applicable for HOST2.



Modeling Doorbell/Memory Window 1:

----------------------------------



.. code-block:: text



 +-----------------+    +----->+----------------+-----------+-----------------+

 |       BAR0      |    |      |   Doorbell 1   +-----------> MSI-X ADDRESS 1 |

 +-----------------+    |      +----------------+           +-----------------+

 |       BAR1      |    |      |   Doorbell 2   +---------+ |                 |

 +-----------------+----+      +----------------+         | |                 |

 |       BAR2      |           |   Doorbell 3   +-------+ | +-----------------+

 +-----------------+----+      +----------------+       | +-> MSI-X ADDRESS 2 |

 |       BAR3      |    |      |   Doorbell 4   +-----+ |   +-----------------+

 +-----------------+    |      |----------------+     | |   |                 |

 |       BAR4      |    |      |                |     | |   +-----------------+

 +-----------------+    |      |      MW1       +---+ | +-->+ MSI-X ADDRESS 3||

 |       BAR5      |    |      |                |   | |     +-----------------+

 +-----------------+    +----->-----------------+   | |     |                 |

   EP CONTROLLER 1             |                |   | |     +-----------------+

                               |                |   | +---->+ MSI-X ADDRESS 4 |

                               +----------------+   |       +-----------------+

                                EP CONTROLLER 2     |       |                 |

                                  (OB SPACE)        |       |                 |

                                                    +------->      MW1        |

                                                            |                 |

                                                            |                 |

                                                            +-----------------+

                                                            |                 |

                                                            |                 |

                                                            |                 |

                                                            |                 |

                                                            |                 |

                                                            +-----------------+

                                                             PCI Address Space

                                                             (Managed by HOST2)



Above diagram shows how the doorbell and memory window 1 is mapped so that

HOST1 can raise doorbell interrupt on HOST2 and also how HOST1 can access

buffers exposed by HOST2 using memory window1 (MW1). Here doorbell and

memory window 1 regions are allocated in EP controller 2 outbound (OB) address

space. Allocating and configuring BARs for doorbell and memory window1

is done during the initialization phase of NTB endpoint function driver.

Mapping from EP controller 2 OB space to PCI address space is done when HOST2

sends CMD_CONFIGURE_MW/CMD_CONFIGURE_DOORBELL.



Modeling Optional Memory Windows:

---------------------------------



This is modeled the same was as MW1 but each of the additional memory windows

is mapped to separate BARs.

Documentation/PCI/endpoint/pci-ntb-howto.rst

0 → 100644
+161 −0
Original line number Diff line number Diff line 
.. SPDX-License-Identifier: GPL-2.0



===================================================================

PCI Non-Transparent Bridge (NTB) Endpoint Function (EPF) User Guide

===================================================================



:Author: Kishon Vijay Abraham I <kishon@ti.com>



This document is a guide to help users use pci-epf-ntb function driver

and ntb_hw_epf host driver for NTB functionality. The list of steps to

be followed in the host side and EP side is given below. For the hardware

configuration and internals of NTB using configurable endpoints see

Documentation/PCI/endpoint/pci-ntb-function.rst



Endpoint Device

===============



Endpoint Controller Devices

---------------------------



For implementing NTB functionality at least two endpoint controller devices

are required.



To find the list of endpoint controller devices in the system::



	# ls /sys/class/pci_epc/

	2900000.pcie-ep  2910000.pcie-ep



If PCI_ENDPOINT_CONFIGFS is enabled::



	# ls /sys/kernel/config/pci_ep/controllers

	2900000.pcie-ep  2910000.pcie-ep





Endpoint Function Drivers

-------------------------



To find the list of endpoint function drivers in the system::



	# ls /sys/bus/pci-epf/drivers

	pci_epf_ntb   pci_epf_ntb



If PCI_ENDPOINT_CONFIGFS is enabled::



	# ls /sys/kernel/config/pci_ep/functions

	pci_epf_ntb   pci_epf_ntb





Creating pci-epf-ntb Device

----------------------------



PCI endpoint function device can be created using the configfs. To create

pci-epf-ntb device, the following commands can be used::



	# mount -t configfs none /sys/kernel/config

	# cd /sys/kernel/config/pci_ep/

	# mkdir functions/pci_epf_ntb/func1



The "mkdir func1" above creates the pci-epf-ntb function device that will

be probed by pci_epf_ntb driver.



The PCI endpoint framework populates the directory with the following

configurable fields::



	# ls functions/pci_epf_ntb/func1

	baseclass_code    deviceid          msi_interrupts    pci-epf-ntb.0

	progif_code       secondary         subsys_id         vendorid

	cache_line_size   interrupt_pin     msix_interrupts   primary

	revid             subclass_code     subsys_vendor_id



The PCI endpoint function driver populates these entries with default values

when the device is bound to the driver. The pci-epf-ntb driver populates

vendorid with 0xffff and interrupt_pin with 0x0001::



	# cat functions/pci_epf_ntb/func1/vendorid

	0xffff

	# cat functions/pci_epf_ntb/func1/interrupt_pin

	0x0001





Configuring pci-epf-ntb Device

-------------------------------



The user can configure the pci-epf-ntb device using its configfs entry. In order

to change the vendorid and the deviceid, the following

commands can be used::



	# echo 0x104c > functions/pci_epf_ntb/func1/vendorid

	# echo 0xb00d > functions/pci_epf_ntb/func1/deviceid



In order to configure NTB specific attributes, a new sub-directory to func1

should be created::



	# mkdir functions/pci_epf_ntb/func1/pci_epf_ntb.0/



The NTB function driver will populate this directory with various attributes

that can be configured by the user::



	# ls functions/pci_epf_ntb/func1/pci_epf_ntb.0/

	db_count    mw1         mw2         mw3         mw4         num_mws

	spad_count



A sample configuration for NTB function is given below::



	# echo 4 > functions/pci_epf_ntb/func1/pci_epf_ntb.0/db_count

	# echo 128 > functions/pci_epf_ntb/func1/pci_epf_ntb.0/spad_count

	# echo 2 > functions/pci_epf_ntb/func1/pci_epf_ntb.0/num_mws

	# echo 0x100000 > functions/pci_epf_ntb/func1/pci_epf_ntb.0/mw1

	# echo 0x100000 > functions/pci_epf_ntb/func1/pci_epf_ntb.0/mw2



Binding pci-epf-ntb Device to EP Controller

--------------------------------------------



NTB function device should be attached to two PCI endpoint controllers

connected to the two hosts. Use the 'primary' and 'secondary' entries

inside NTB function device to attach one PCI endpoint controller to

primary interface and the other PCI endpoint controller to the secondary

interface::



	# ln -s controllers/2900000.pcie-ep/ functions/pci-epf-ntb/func1/primary

	# ln -s controllers/2910000.pcie-ep/ functions/pci-epf-ntb/func1/secondary



Once the above step is completed, both the PCI endpoint controllers are ready to

establish a link with the host.





Start the Link

--------------



In order for the endpoint device to establish a link with the host, the _start_

field should be populated with '1'. For NTB, both the PCI endpoint controllers

should establish link with the host::



	# echo 1 > controllers/2900000.pcie-ep/start

	# echo 1 > controllers/2910000.pcie-ep/start





RootComplex Device

==================



lspci Output

------------



Note that the devices listed here correspond to the values populated in

"Creating pci-epf-ntb Device" section above::



	# lspci

	0000:00:00.0 PCI bridge: Texas Instruments Device b00d

	0000:01:00.0 RAM memory: Texas Instruments Device b00d





Using ntb_hw_epf Device

-----------------------



The host side software follows the standard NTB software architecture in Linux.

All the existing client side NTB utilities like NTB Transport Client and NTB

Netdev, NTB Ping Pong Test Client and NTB Tool Test Client can be used with NTB

function device.



For more information on NTB see

:doc:`Non-Transparent Bridge <../../driver-api/ntb>`