Commit 1ecee653 authored by Lucas Segarra Fernandez's avatar Lucas Segarra Fernandez Committed by Aichun Shi
Browse files

crypto: qat - add support for ring pair level telemetry 

mainline inclusion
from mainline-v6.8-rc1
commit eb52707716e3f2cdf16f4e95e3a800cca190504f
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I9A5BW
CVE: N/A
Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=eb52707716e3f2cdf16f4e95e3a800cca190504f



-------------------------------------

Expose through debugfs ring pair telemetry data for QAT GEN4 devices.

This allows to gather metrics about the PCIe channel and device TLB for
a selected ring pair. It is possible to monitor maximum 4 ring pairs at
the time per device.

For details, refer to debugfs-driver-qat_telemetry in Documentation/ABI.

This patch is based on earlier work done by Wojciech Ziemba.

Intel-SIG: commit eb52707716e3 crypto: qat - add support for ring pair level telemetry
Backport to support QAT in-tree driver

Signed-off-by: default avatarLucas Segarra Fernandez <lucas.segarra.fernandez@intel.com>
Reviewed-by: default avatarGiovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: default avatarDamian Muszynski <damian.muszynski@intel.com>
Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
[ Aichun Shi: amend commit log ]
Signed-off-by: default avatarAichun Shi <aichun.shi@intel.com>
parent 3f99c75f

Documentation/ABI/testing/debugfs-driver-qat_telemetry

+125 −0
Original line number Diff line number Diff line
@@ -101,3 +101,128 @@ Description: (RO) Reports device telemetry counters. 
		attribute is not reported.



		This attribute is only available for qat_4xxx devices.



What:		/sys/kernel/debug/qat_<device>_<BDF>/telemetry/rp_<A/B/C/D>_data

Date:		March 2024

KernelVersion:	6.8

Contact:	qat-linux@intel.com

Description:	(RW) Selects up to 4 Ring Pairs (RP) to monitor, one per file,

		and report telemetry counters related to each.



		Allowed values to write:

		========================

		* 0 to ``<num_rps - 1>``:

		  Ring pair to be monitored. The value of ``num_rps`` can be

		  retrieved through ``/sys/bus/pci/devices/<BDF>/qat/num_rps``.

		  See Documentation/ABI/testing/sysfs-driver-qat.



		Reads report metrics about performance and utilization of

		the selected RP:



		=======================	========================================

		Field			Description

		=======================	========================================

		sample_cnt		number of acquisitions of telemetry data

					from the device. Reads are performed

					every 1000 ms

		rp_num			RP number associated with slot <A/B/C/D>

		service_type		service associated to the RP

		pci_trans_cnt		number of PCIe partial transactions

		gp_lat_acc_avg		average get to put latency [ns]

		bw_in			PCIe, write bandwidth [Mbps]

		bw_out			PCIe, read bandwidth [Mbps]

		at_glob_devtlb_hit	Message descriptor DevTLB hit rate

		at_glob_devtlb_miss	Message descriptor DevTLB miss rate

		tl_at_payld_devtlb_hit	Payload DevTLB hit rate

		tl_at_payld_devtlb_miss	Payload DevTLB miss rate

		======================= ========================================



		Example.



		Writing the value '32' to the file ``rp_C_data`` starts the

		collection of telemetry metrics for ring pair 32::



		  echo 32 > /sys/kernel/debug/qat_4xxx_0000:6b:00.0/telemetry/rp_C_data



		Once a ring pair is selected, statistics can be read accessing

		the file::



		  cat /sys/kernel/debug/qat_4xxx_0000:6b:00.0/telemetry/rp_C_data



		If ``control`` is set to 1, only the current values of the

		counters are displayed::



		  <counter_name> <current>



		If ``control`` is 2, 3 or 4, counters are displayed in the

		following format::



		  <counter_name> <current> <min> <max> <avg>





		On QAT GEN4 devices there are 64 RPs on a PF, so the allowed

		values are 0..63. This number is absolute to the device.

		If Virtual Functions (VF) are used, the ring pair number can

		be derived from the Bus, Device, Function of the VF:



		============ ====== ====== ====== ======

		PCI BDF/VF   RP0    RP1    RP2    RP3

		============ ====== ====== ====== ======

		0000:6b:0.1  RP  0  RP  1  RP  2  RP  3

		0000:6b:0.2  RP  4  RP  5  RP  6  RP  7

		0000:6b:0.3  RP  8  RP  9  RP 10  RP 11

		0000:6b:0.4  RP 12  RP 13  RP 14  RP 15

		0000:6b:0.5  RP 16  RP 17  RP 18  RP 19

		0000:6b:0.6  RP 20  RP 21  RP 22  RP 23

		0000:6b:0.7  RP 24  RP 25  RP 26  RP 27

		0000:6b:1.0  RP 28  RP 29  RP 30  RP 31

		0000:6b:1.1  RP 32  RP 33  RP 34  RP 35

		0000:6b:1.2  RP 36  RP 37  RP 38  RP 39

		0000:6b:1.3  RP 40  RP 41  RP 42  RP 43

		0000:6b:1.4  RP 44  RP 45  RP 46  RP 47

		0000:6b:1.5  RP 48  RP 49  RP 50  RP 51

		0000:6b:1.6  RP 52  RP 53  RP 54  RP 55

		0000:6b:1.7  RP 56  RP 57  RP 58  RP 59

		0000:6b:2.0  RP 60  RP 61  RP 62  RP 63

		============ ====== ====== ====== ======



		The mapping is only valid for the BDFs of VFs on the host.





		The service provided on a ring-pair varies depending on the

		configuration. The configuration for a given device can be

		queried and set using ``cfg_services``.

		See Documentation/ABI/testing/sysfs-driver-qat for details.



		The following table reports how ring pairs are mapped to VFs

		on the PF 0000:6b:0.0 configured for `sym;asym` or `asym;sym`:



		=========== ============ =========== ============ ===========

		PCI BDF/VF  RP0/service  RP1/service RP2/service  RP3/service

		=========== ============ =========== ============ ===========

		0000:6b:0.1 RP 0 asym    RP 1 sym    RP 2 asym    RP 3 sym

		0000:6b:0.2 RP 4 asym    RP 5 sym    RP 6 asym    RP 7 sym

		0000:6b:0.3 RP 8 asym    RP 9 sym    RP10 asym    RP11 sym

		...         ...          ...         ...          ...

		=========== ============ =========== ============ ===========



		All VFs follow the same pattern.





		The following table reports how ring pairs are mapped to VFs on

		the PF 0000:6b:0.0 configured for `dc`:



		=========== ============ =========== ============ ===========

		PCI BDF/VF  RP0/service  RP1/service RP2/service  RP3/service

		=========== ============ =========== ============ ===========

		0000:6b:0.1 RP 0 dc      RP 1 dc     RP 2 dc      RP 3 dc

		0000:6b:0.2 RP 4 dc      RP 5 dc     RP 6 dc      RP 7 dc

		0000:6b:0.3 RP 8 dc      RP 9 dc     RP10 dc      RP11 dc

		...         ...          ...         ...          ...

		=========== ============ =========== ============ ===========



		The mapping of a RP to a service can be retrieved using

		``rp2srv`` from sysfs.

		See Documentation/ABI/testing/sysfs-driver-qat for details.



		This attribute is only available for qat_4xxx devices.

drivers/crypto/intel/qat/qat_420xx/adf_420xx_hw_data.c

+1 −0
Original line number Diff line number Diff line
@@ -520,6 +520,7 @@ void adf_init_hw_data_420xx(struct adf_hw_device_data *hw_data, u32 dev_id) 
	hw_data->init_device = adf_gen4_init_device;

	hw_data->reset_device = adf_reset_flr;

	hw_data->admin_ae_mask = ADF_420XX_ADMIN_AE_MASK;

	hw_data->num_rps = ADF_GEN4_MAX_RPS;

	hw_data->fw_name = ADF_420XX_FW;

	hw_data->fw_mmp_name = ADF_420XX_MMP;

	hw_data->uof_get_name = uof_get_name_420xx;

drivers/crypto/intel/qat/qat_4xxx/adf_4xxx_hw_data.c

+1 −0
Original line number Diff line number Diff line
@@ -421,6 +421,7 @@ void adf_init_hw_data_4xxx(struct adf_hw_device_data *hw_data, u32 dev_id) 
	hw_data->init_device = adf_gen4_init_device;

	hw_data->reset_device = adf_reset_flr;

	hw_data->admin_ae_mask = ADF_4XXX_ADMIN_AE_MASK;

	hw_data->num_rps = ADF_GEN4_MAX_RPS;

	switch (dev_id) {

	case ADF_402XX_PCI_DEVICE_ID:

		hw_data->fw_name = ADF_402XX_FW;

drivers/crypto/intel/qat/qat_common/adf_accel_devices.h

+1 −0
Original line number Diff line number Diff line
@@ -278,6 +278,7 @@ struct adf_hw_device_data { 
	u8 num_logical_accel;

	u8 num_engines;

	u32 num_hb_ctrs;

	u8 num_rps;

};



/* CSR write macro */

drivers/crypto/intel/qat/qat_common/adf_gen4_hw_data.h

+1 −0
Original line number Diff line number Diff line
@@ -36,6 +36,7 @@ 
#define ADF_GEN4_MSIX_RTTABLE_OFFSET(i)		(0x409000 + ((i) * 0x04))



/* Bank and ring configuration */

#define ADF_GEN4_MAX_RPS		64

#define ADF_GEN4_NUM_RINGS_PER_BANK	2

#define ADF_GEN4_NUM_BANKS_PER_VF	4

#define ADF_GEN4_ETR_MAX_BANKS		64