diff --git a/.mailmap b/.mailmap index ccba4cf0d893849363c2d9a44d304d3f23f5de6a..8deff4cec169c509c7219a48a1c499a8e4febb70 100644 --- a/.mailmap +++ b/.mailmap @@ -371,6 +371,7 @@ Rémi Denis-Courmont Ricardo Ribalda Ricardo Ribalda Ricardo Ribalda Delgado Ricardo Ribalda +Robert Foss Roman Gushchin Roman Gushchin Roman Gushchin @@ -422,6 +423,7 @@ Tony Luck TripleX Chung TripleX Chung Tsuneo Yoshioka +Tudor Ambarus Tycho Andersen Tzung-Bi Shih Uwe Kleine-König diff --git a/CREDITS b/CREDITS index 4e302a459ddf069ce3b59b3dd7bbebcd8476ef46..acac06b6563e266f52224b466b8cf4ef39199b09 100644 --- a/CREDITS +++ b/CREDITS @@ -2489,6 +2489,13 @@ D: XF86_Mach8 D: XF86_8514 D: cfdisk (curses based disk partitioning program) +N: Mat Martineau +E: mat@martineau.name +D: MPTCP subsystem co-maintainer 2020-2023 +D: Keyctl restricted keyring and Diffie-Hellman UAPI +D: Bluetooth L2CAP ERTM mode and AMP +S: USA + N: John S. Marvin E: jsm@fc.hp.com D: PA-RISC port diff --git a/Documentation/ABI/testing/debugfs-pktcdvd b/Documentation/ABI/testing/debugfs-pktcdvd new file mode 100644 index 0000000000000000000000000000000000000000..f6f65a4faea06c0fc3286b4c4b54c927f0d0fa5c --- /dev/null +++ b/Documentation/ABI/testing/debugfs-pktcdvd @@ -0,0 +1,18 @@ +What: /sys/kernel/debug/pktcdvd/pktcdvd[0-7] +Date: Oct. 2006 +KernelVersion: 2.6.20 +Contact: Thomas Maier +Description: + +The pktcdvd module (packet writing driver) creates +these files in debugfs: + +/sys/kernel/debug/pktcdvd/pktcdvd[0-7]/ + + ==== ====== ==================================== + info 0444 Lots of driver statistics and infos. + ==== ====== ==================================== + +Example:: + + cat /sys/kernel/debug/pktcdvd/pktcdvd0/info diff --git a/Documentation/ABI/testing/sysfs-class-pktcdvd b/Documentation/ABI/testing/sysfs-class-pktcdvd new file mode 100644 index 0000000000000000000000000000000000000000..ba1ce626591d9c4e666fea449075c44a3f447b17 --- /dev/null +++ b/Documentation/ABI/testing/sysfs-class-pktcdvd @@ -0,0 +1,97 @@ +sysfs interface +--------------- +The pktcdvd module (packet writing driver) creates the following files in the +sysfs: ( is in the format major:minor) + +What: /sys/class/pktcdvd/add +What: /sys/class/pktcdvd/remove +What: /sys/class/pktcdvd/device_map +Date: Oct. 2006 +KernelVersion: 2.6.20 +Contact: Thomas Maier +Description: + + ========== ============================================== + add (WO) Write a block device id (major:minor) to + create a new pktcdvd device and map it to the + block device. + + remove (WO) Write the pktcdvd device id (major:minor) + to remove the pktcdvd device. + + device_map (RO) Shows the device mapping in format: + pktcdvd[0-7] + ========== ============================================== + + +What: /sys/class/pktcdvd/pktcdvd[0-7]/dev +What: /sys/class/pktcdvd/pktcdvd[0-7]/uevent +Date: Oct. 2006 +KernelVersion: 2.6.20 +Contact: Thomas Maier +Description: + dev: (RO) Device id + + uevent: (WO) To send a uevent + + +What: /sys/class/pktcdvd/pktcdvd[0-7]/stat/packets_started +What: /sys/class/pktcdvd/pktcdvd[0-7]/stat/packets_finished +What: /sys/class/pktcdvd/pktcdvd[0-7]/stat/kb_written +What: /sys/class/pktcdvd/pktcdvd[0-7]/stat/kb_read +What: /sys/class/pktcdvd/pktcdvd[0-7]/stat/kb_read_gather +What: /sys/class/pktcdvd/pktcdvd[0-7]/stat/reset +Date: Oct. 2006 +KernelVersion: 2.6.20 +Contact: Thomas Maier +Description: + packets_started: (RO) Number of started packets. + + packets_finished: (RO) Number of finished packets. + + kb_written: (RO) kBytes written. + + kb_read: (RO) kBytes read. + + kb_read_gather: (RO) kBytes read to fill write packets. + + reset: (WO) Write any value to it to reset + pktcdvd device statistic values, like + bytes read/written. + + +What: /sys/class/pktcdvd/pktcdvd[0-7]/write_queue/size +What: /sys/class/pktcdvd/pktcdvd[0-7]/write_queue/congestion_off +What: /sys/class/pktcdvd/pktcdvd[0-7]/write_queue/congestion_on +Date: Oct. 2006 +KernelVersion: 2.6.20 +Contact: Thomas Maier +Description: + ============== ================================================ + size (RO) Contains the size of the bio write queue. + + congestion_off (RW) If bio write queue size is below this mark, + accept new bio requests from the block layer. + + congestion_on (RW) If bio write queue size is higher as this + mark, do no longer accept bio write requests + from the block layer and wait till the pktcdvd + device has processed enough bio's so that bio + write queue size is below congestion off mark. + A value of <= 0 disables congestion control. + ============== ================================================ + + +Example: +-------- +To use the pktcdvd sysfs interface directly, you can do:: + + # create a new pktcdvd device mapped to /dev/hdc + echo "22:0" >/sys/class/pktcdvd/add + cat /sys/class/pktcdvd/device_map + # assuming device pktcdvd0 was created, look at stat's + cat /sys/class/pktcdvd/pktcdvd0/stat/kb_written + # print the device id of the mapped block device + fgrep pktcdvd0 /sys/class/pktcdvd/device_map + # remove device, using pktcdvd0 device id 253:0 + echo "253:0" >/sys/class/pktcdvd/remove diff --git a/Documentation/ABI/testing/sysfs-driver-habanalabs b/Documentation/ABI/testing/sysfs-driver-habanalabs index 13b5b2ec3be7698678beba7be0c1b8654be10f73..1b98b6503b23f5a50f09885c5ba86b2fe58b3b5e 100644 --- a/Documentation/ABI/testing/sysfs-driver-habanalabs +++ b/Documentation/ABI/testing/sysfs-driver-habanalabs @@ -201,7 +201,19 @@ What: /sys/class/habanalabs/hl/status Date: Jan 2019 KernelVersion: 5.1 Contact: ogabbay@kernel.org -Description: Status of the card: "Operational", "Malfunction", "In reset". +Description: Status of the card: + + * "operational" - Device is available for work. + * "in reset" - Device is going through reset, will be + available shortly. + * "disabled" - Device is not usable. + * "needs reset" - Device is not usable until a hard reset + is initiated. + * "in device creation" - Device is not available yet, as it + is still initializing. + * "in reset after device release" - Device is going through + a compute-reset which is executed after a device release + (relevant for Gaudi2 only). What: /sys/class/habanalabs/hl/thermal_ver Date: Jan 2019 diff --git a/Documentation/accel/introduction.rst b/Documentation/accel/introduction.rst index 6f31af14b1fc4f3f28e6924d6b7ba907e58f575a..89984dfececf0b0b07a937179808d27b8268cf4b 100644 --- a/Documentation/accel/introduction.rst +++ b/Documentation/accel/introduction.rst @@ -67,9 +67,9 @@ tree - drivers/accel/. The accelerator devices will be exposed to the user space with the dedicated 261 major number and will have the following convention: -- device char files - /dev/accel/accel* -- sysfs - /sys/class/accel/accel*/ -- debugfs - /sys/kernel/debug/accel/accel*/ +- device char files - /dev/accel/accel\* +- sysfs - /sys/class/accel/accel\*/ +- debugfs - /sys/kernel/debug/accel/\*/ Getting Started =============== diff --git a/Documentation/admin-guide/mm/zswap.rst b/Documentation/admin-guide/mm/zswap.rst index f67de481c7f60559c169ac0ba7e8dcaff24b5da9..6dd74a18268baa6c74acfd55499ad769e04c12b2 100644 --- a/Documentation/admin-guide/mm/zswap.rst +++ b/Documentation/admin-guide/mm/zswap.rst @@ -70,9 +70,7 @@ e.g. ``zswap.zpool=zbud``. It can also be changed at runtime using the sysfs The zbud type zpool allocates exactly 1 page to store 2 compressed pages, which means the compression ratio will always be 2:1 or worse (because of half-full zbud pages). The zsmalloc type zpool has a more complex compressed page -storage method, and it can achieve greater storage densities. However, -zsmalloc does not implement compressed page eviction, so once zswap fills it -cannot evict the oldest page, it can only reject new pages. +storage method, and it can achieve greater storage densities. When a swap page is passed from frontswap to zswap, zswap maintains a mapping of the swap entry, a combination of the swap type and swap offset, to the zpool diff --git a/Documentation/arm64/silicon-errata.rst b/Documentation/arm64/silicon-errata.rst index 808ade4cc008ac7c41b0a13e5685fe5afae1dfe3..ec5f889d76819fd2fae733997d61ff1c33114ace 100644 --- a/Documentation/arm64/silicon-errata.rst +++ b/Documentation/arm64/silicon-errata.rst @@ -120,6 +120,8 @@ stable kernels. +----------------+-----------------+-----------------+-----------------------------+ | ARM | Cortex-A710 | #2224489 | ARM64_ERRATUM_2224489 | +----------------+-----------------+-----------------+-----------------------------+ +| ARM | Cortex-A715 | #2645198 | ARM64_ERRATUM_2645198 | ++----------------+-----------------+-----------------+-----------------------------+ | ARM | Cortex-X2 | #2119858 | ARM64_ERRATUM_2119858 | +----------------+-----------------+-----------------+-----------------------------+ | ARM | Cortex-X2 | #2224489 | ARM64_ERRATUM_2224489 | diff --git a/Documentation/conf.py b/Documentation/conf.py index a5c45df0bd83978028691abca71ab9d6c5155d8c..d927737e3c10fe207456f9eb360958c8b8a500a4 100644 --- a/Documentation/conf.py +++ b/Documentation/conf.py @@ -31,6 +31,12 @@ def have_command(cmd): # Get Sphinx version major, minor, patch = sphinx.version_info[:3] +# +# Warn about older versions that we don't want to support for much +# longer. +# +if (major < 2) or (major == 2 and minor < 4): + print('WARNING: support for Sphinx < 2.4 will be removed soon.') # If extensions (or modules to document with autodoc) are in another directory, # add these directories to sys.path here. If the directory is relative to the @@ -339,7 +345,11 @@ html_use_smartypants = False # Custom sidebar templates, maps document names to template names. # Note that the RTD theme ignores this -html_sidebars = { '**': ["about.html", 'searchbox.html', 'localtoc.html', 'sourcelink.html']} +html_sidebars = { '**': ['searchbox.html', 'localtoc.html', 'sourcelink.html']} + +# about.html is available for alabaster theme. Add it at the front. +if html_theme == 'alabaster': + html_sidebars['**'].insert(0, 'about.html') # Output file base name for HTML help builder. htmlhelp_basename = 'TheLinuxKerneldoc' diff --git a/Documentation/devicetree/bindings/cpufreq/cpufreq-qcom-hw.yaml b/Documentation/devicetree/bindings/cpufreq/cpufreq-qcom-hw.yaml index 903b31129f012b1d755ab612e1ec804ca3407f87..99e159bc5fb13b3f141015621b91249bce20dc1a 100644 --- a/Documentation/devicetree/bindings/cpufreq/cpufreq-qcom-hw.yaml +++ b/Documentation/devicetree/bindings/cpufreq/cpufreq-qcom-hw.yaml @@ -54,6 +54,17 @@ properties: - const: xo - const: alternate + interrupts: + minItems: 1 + maxItems: 3 + + interrupt-names: + minItems: 1 + items: + - const: dcvsh-irq-0 + - const: dcvsh-irq-1 + - const: dcvsh-irq-2 + '#freq-domain-cells': const: 1 diff --git a/Documentation/devicetree/bindings/crypto/atmel,at91sam9g46-aes.yaml b/Documentation/devicetree/bindings/crypto/atmel,at91sam9g46-aes.yaml index 0ccaab16dc614ea6c72cd67279b945ed8a117b41..0b7383b3106b51aa937bb25f569d78a5bea474d7 100644 --- a/Documentation/devicetree/bindings/crypto/atmel,at91sam9g46-aes.yaml +++ b/Documentation/devicetree/bindings/crypto/atmel,at91sam9g46-aes.yaml @@ -8,7 +8,7 @@ $schema: http://devicetree.org/meta-schemas/core.yaml# title: Atmel Advanced Encryption Standard (AES) HW cryptographic accelerator maintainers: - - Tudor Ambarus + - Tudor Ambarus properties: compatible: diff --git a/Documentation/devicetree/bindings/crypto/atmel,at91sam9g46-sha.yaml b/Documentation/devicetree/bindings/crypto/atmel,at91sam9g46-sha.yaml index 5163c51b4547b512f9aace6777d7d4be5ce7fec7..ee2ffb0343251f5a2ce7c4d94286ae898db43baa 100644 --- a/Documentation/devicetree/bindings/crypto/atmel,at91sam9g46-sha.yaml +++ b/Documentation/devicetree/bindings/crypto/atmel,at91sam9g46-sha.yaml @@ -8,7 +8,7 @@ $schema: http://devicetree.org/meta-schemas/core.yaml# title: Atmel Secure Hash Algorithm (SHA) HW cryptographic accelerator maintainers: - - Tudor Ambarus + - Tudor Ambarus properties: compatible: diff --git a/Documentation/devicetree/bindings/crypto/atmel,at91sam9g46-tdes.yaml b/Documentation/devicetree/bindings/crypto/atmel,at91sam9g46-tdes.yaml index fcc5adf03cadba45800bc1082f5e51a20adddadb..3d6ed24b1b006480d9e4d86793078b9a5422eb72 100644 --- a/Documentation/devicetree/bindings/crypto/atmel,at91sam9g46-tdes.yaml +++ b/Documentation/devicetree/bindings/crypto/atmel,at91sam9g46-tdes.yaml @@ -8,7 +8,7 @@ $schema: http://devicetree.org/meta-schemas/core.yaml# title: Atmel Triple Data Encryption Standard (TDES) HW cryptographic accelerator maintainers: - - Tudor Ambarus + - Tudor Ambarus properties: compatible: diff --git a/Documentation/devicetree/bindings/display/bridge/renesas,dsi-csi2-tx.yaml b/Documentation/devicetree/bindings/display/bridge/renesas,dsi-csi2-tx.yaml index afeeb967393d9e5df4b983e112cfd71c3a1446b7..d33026f85e19135544d2076b163681bacf2e4c99 100644 --- a/Documentation/devicetree/bindings/display/bridge/renesas,dsi-csi2-tx.yaml +++ b/Documentation/devicetree/bindings/display/bridge/renesas,dsi-csi2-tx.yaml @@ -11,13 +11,14 @@ maintainers: description: | This binding describes the MIPI DSI/CSI-2 encoder embedded in the Renesas - R-Car V3U SoC. The encoder can operate in either DSI or CSI-2 mode, with up + R-Car Gen4 SoCs. The encoder can operate in either DSI or CSI-2 mode, with up to four data lanes. properties: compatible: enum: - renesas,r8a779a0-dsi-csi2-tx # for V3U + - renesas,r8a779g0-dsi-csi2-tx # for V4H reg: maxItems: 1 diff --git a/Documentation/devicetree/bindings/display/msm/dsi-controller-main.yaml b/Documentation/devicetree/bindings/display/msm/dsi-controller-main.yaml index f2c143730a551fe8b6f68dd35fc18a3c6a633b22..6e2fd6e9fa7f0aec445c2d04dfecc892d1806dfc 100644 --- a/Documentation/devicetree/bindings/display/msm/dsi-controller-main.yaml +++ b/Documentation/devicetree/bindings/display/msm/dsi-controller-main.yaml @@ -32,7 +32,7 @@ properties: - description: Display byte clock - description: Display byte interface clock - description: Display pixel clock - - description: Display escape clock + - description: Display core clock - description: Display AHB clock - description: Display AXI clock @@ -137,8 +137,6 @@ required: - phys - assigned-clocks - assigned-clock-parents - - power-domains - - operating-points-v2 - ports additionalProperties: false diff --git a/Documentation/devicetree/bindings/display/msm/dsi-phy-10nm.yaml b/Documentation/devicetree/bindings/display/msm/dsi-phy-10nm.yaml index d9ad8b659f58ee0f5bf3b64b696beb64914c9609..3ec466c3ab38bfe31588b5154a675dd5af5126b3 100644 --- a/Documentation/devicetree/bindings/display/msm/dsi-phy-10nm.yaml +++ b/Documentation/devicetree/bindings/display/msm/dsi-phy-10nm.yaml @@ -69,7 +69,6 @@ required: - compatible - reg - reg-names - - vdds-supply unevaluatedProperties: false diff --git a/Documentation/devicetree/bindings/display/msm/dsi-phy-14nm.yaml b/Documentation/devicetree/bindings/display/msm/dsi-phy-14nm.yaml index 819de5ce0bc9170b8949f21724172da863ced56a..a43e11d3b00d2cfe74b24187114415336d876b2e 100644 --- a/Documentation/devicetree/bindings/display/msm/dsi-phy-14nm.yaml +++ b/Documentation/devicetree/bindings/display/msm/dsi-phy-14nm.yaml @@ -39,7 +39,6 @@ required: - compatible - reg - reg-names - - vcca-supply unevaluatedProperties: false diff --git a/Documentation/devicetree/bindings/display/msm/dsi-phy-28nm.yaml b/Documentation/devicetree/bindings/display/msm/dsi-phy-28nm.yaml index 3d8540a06fe22a4865f9d45b2bcc3b8e009a646f..2f1fd140c87df95ec60fe37614b724e97ba3395c 100644 --- a/Documentation/devicetree/bindings/display/msm/dsi-phy-28nm.yaml +++ b/Documentation/devicetree/bindings/display/msm/dsi-phy-28nm.yaml @@ -34,6 +34,10 @@ properties: vddio-supply: description: Phandle to vdd-io regulator device node. + qcom,dsi-phy-regulator-ldo-mode: + type: boolean + description: Indicates if the LDO mode PHY regulator is wanted. + required: - compatible - reg diff --git a/Documentation/devicetree/bindings/display/msm/qcom,qcm2290-mdss.yaml b/Documentation/devicetree/bindings/display/msm/qcom,qcm2290-mdss.yaml index d6f043a4b08d2dbf86108d15168510dacd78e4f3..4795e13c7b597baaffa5a30db4df769bc51a3cfd 100644 --- a/Documentation/devicetree/bindings/display/msm/qcom,qcm2290-mdss.yaml +++ b/Documentation/devicetree/bindings/display/msm/qcom,qcm2290-mdss.yaml @@ -72,7 +72,7 @@ examples: #include #include - mdss@5e00000 { + display-subsystem@5e00000 { #address-cells = <1>; #size-cells = <1>; compatible = "qcom,qcm2290-mdss"; diff --git a/Documentation/devicetree/bindings/display/msm/qcom,sm6115-mdss.yaml b/Documentation/devicetree/bindings/display/msm/qcom,sm6115-mdss.yaml index a86d7f53fa84dfcf2c8fe12d12c38a7c5d07757e..886858ef67000537e05e031650dfd897eb9311cc 100644 --- a/Documentation/devicetree/bindings/display/msm/qcom,sm6115-mdss.yaml +++ b/Documentation/devicetree/bindings/display/msm/qcom,sm6115-mdss.yaml @@ -62,7 +62,7 @@ examples: #include #include - mdss@5e00000 { + display-subsystem@5e00000 { #address-cells = <1>; #size-cells = <1>; compatible = "qcom,sm6115-mdss"; diff --git a/Documentation/devicetree/bindings/display/renesas,du.yaml b/Documentation/devicetree/bindings/display/renesas,du.yaml index b3e588022082d8209ccd01874ffe5f1fa3eb1d04..d4830f52c5122d677dab2f92491c28cbdeac4917 100644 --- a/Documentation/devicetree/bindings/display/renesas,du.yaml +++ b/Documentation/devicetree/bindings/display/renesas,du.yaml @@ -40,6 +40,7 @@ properties: - renesas,du-r8a77990 # for R-Car E3 compatible DU - renesas,du-r8a77995 # for R-Car D3 compatible DU - renesas,du-r8a779a0 # for R-Car V3U compatible DU + - renesas,du-r8a779g0 # for R-Car V4H compatible DU reg: maxItems: 1 @@ -762,6 +763,7 @@ allOf: contains: enum: - renesas,du-r8a779a0 + - renesas,du-r8a779g0 then: properties: clocks: diff --git a/Documentation/devicetree/bindings/i2c/renesas,rzv2m.yaml b/Documentation/devicetree/bindings/i2c/renesas,rzv2m.yaml index c46378efc12399dda5b85ce961cfe8ea53453868..92e899905ef88915179a3a92d604cbc3149e8e51 100644 --- a/Documentation/devicetree/bindings/i2c/renesas,rzv2m.yaml +++ b/Documentation/devicetree/bindings/i2c/renesas,rzv2m.yaml @@ -16,7 +16,7 @@ properties: compatible: items: - enum: - - renesas,i2c-r9a09g011 # RZ/V2M + - renesas,r9a09g011-i2c # RZ/V2M - const: renesas,rzv2m-i2c reg: @@ -66,7 +66,7 @@ examples: #include i2c0: i2c@a4030000 { - compatible = "renesas,i2c-r9a09g011", "renesas,rzv2m-i2c"; + compatible = "renesas,r9a09g011-i2c", "renesas,rzv2m-i2c"; reg = <0xa4030000 0x80>; interrupts = , ; diff --git a/Documentation/devicetree/bindings/interconnect/qcom,rpm.yaml b/Documentation/devicetree/bindings/interconnect/qcom,rpm.yaml index 4b37aa88a375b12ebffd09f43e6539a61cfb9c65..5e6be4e79201eeb09903ea8a1af78ffab305f44f 100644 --- a/Documentation/devicetree/bindings/interconnect/qcom,rpm.yaml +++ b/Documentation/devicetree/bindings/interconnect/qcom,rpm.yaml @@ -84,7 +84,6 @@ allOf: - qcom,msm8939-pcnoc - qcom,msm8939-snoc - qcom,msm8996-a1noc - - qcom,msm8996-a2noc - qcom,msm8996-bimc - qcom,msm8996-cnoc - qcom,msm8996-pnoc @@ -186,6 +185,29 @@ allOf: required: - power-domains + - if: + properties: + compatible: + contains: + enum: + - qcom,msm8996-a2noc + + then: + properties: + clock-names: + items: + - const: bus + - const: bus_a + - const: aggre2_ufs_axi + - const: ufs_axi + + clocks: + items: + - description: Bus Clock + - description: Bus A Clock + - description: Aggregate2 NoC UFS AXI Clock + - description: UFS AXI Clock + - if: properties: compatible: diff --git a/Documentation/devicetree/bindings/net/allwinner,sun8i-a83t-emac.yaml b/Documentation/devicetree/bindings/net/allwinner,sun8i-a83t-emac.yaml index 1432fda3b603f74e11d287f8a29d7bf03a042e79..47bc2057e6292a829dc25b2ef5e2f70ea6a3efa8 100644 --- a/Documentation/devicetree/bindings/net/allwinner,sun8i-a83t-emac.yaml +++ b/Documentation/devicetree/bindings/net/allwinner,sun8i-a83t-emac.yaml @@ -40,6 +40,9 @@ properties: clock-names: const: stmmaceth + phy-supply: + description: PHY regulator + syscon: $ref: /schemas/types.yaml#/definitions/phandle description: diff --git a/Documentation/devicetree/bindings/net/marvell,orion-mdio.yaml b/Documentation/devicetree/bindings/net/marvell,orion-mdio.yaml index d2906b4a0f5957bb13b20554a6e404ecd4ed489b..e35da8b01dc257f9d5a96fcad1807ae5ea3df06e 100644 --- a/Documentation/devicetree/bindings/net/marvell,orion-mdio.yaml +++ b/Documentation/devicetree/bindings/net/marvell,orion-mdio.yaml @@ -16,9 +16,6 @@ description: | 8k has a second unit which provides an interface with the xMDIO bus. This driver handles these interfaces. -allOf: - - $ref: "mdio.yaml#" - properties: compatible: enum: @@ -39,13 +36,38 @@ required: - compatible - reg +allOf: + - $ref: mdio.yaml# + + - if: + required: + - interrupts + + then: + properties: + reg: + items: + - items: + - $ref: /schemas/types.yaml#/definitions/cell + - const: 0x84 + + else: + properties: + reg: + items: + - items: + - $ref: /schemas/types.yaml#/definitions/cell + - enum: + - 0x4 + - 0x10 + unevaluatedProperties: false examples: - | mdio@d0072004 { compatible = "marvell,orion-mdio"; - reg = <0xd0072004 0x4>; + reg = <0xd0072004 0x84>; #address-cells = <1>; #size-cells = <0>; interrupts = <30>; diff --git a/Documentation/devicetree/bindings/phy/amlogic,meson-g12a-usb2-phy.yaml b/Documentation/devicetree/bindings/phy/amlogic,g12a-usb2-phy.yaml similarity index 85% rename from Documentation/devicetree/bindings/phy/amlogic,meson-g12a-usb2-phy.yaml rename to Documentation/devicetree/bindings/phy/amlogic,g12a-usb2-phy.yaml index f3a5fbabbbb5971b343ea6f892a49033dddbf5d0..bb01c6b34dabcb1f5bbbf3ae51aba0fba4027e07 100644 --- a/Documentation/devicetree/bindings/phy/amlogic,meson-g12a-usb2-phy.yaml +++ b/Documentation/devicetree/bindings/phy/amlogic,g12a-usb2-phy.yaml @@ -2,7 +2,7 @@ # Copyright 2019 BayLibre, SAS %YAML 1.2 --- -$id: "http://devicetree.org/schemas/phy/amlogic,meson-g12a-usb2-phy.yaml#" +$id: "http://devicetree.org/schemas/phy/amlogic,g12a-usb2-phy.yaml#" $schema: "http://devicetree.org/meta-schemas/core.yaml#" title: Amlogic G12A USB2 PHY @@ -13,8 +13,8 @@ maintainers: properties: compatible: enum: - - amlogic,meson-g12a-usb2-phy - - amlogic,meson-a1-usb2-phy + - amlogic,g12a-usb2-phy + - amlogic,a1-usb2-phy reg: maxItems: 1 @@ -68,7 +68,7 @@ additionalProperties: false examples: - | phy@36000 { - compatible = "amlogic,meson-g12a-usb2-phy"; + compatible = "amlogic,g12a-usb2-phy"; reg = <0x36000 0x2000>; clocks = <&xtal>; clock-names = "xtal"; diff --git a/Documentation/devicetree/bindings/phy/amlogic,meson-g12a-usb3-pcie-phy.yaml b/Documentation/devicetree/bindings/phy/amlogic,g12a-usb3-pcie-phy.yaml similarity index 82% rename from Documentation/devicetree/bindings/phy/amlogic,meson-g12a-usb3-pcie-phy.yaml rename to Documentation/devicetree/bindings/phy/amlogic,g12a-usb3-pcie-phy.yaml index 868b4e6fde71f1fb68d49b9ce8f73fc5176a12e6..129d26e99776b6e730dbfe23cf8e742b6de9a9a6 100644 --- a/Documentation/devicetree/bindings/phy/amlogic,meson-g12a-usb3-pcie-phy.yaml +++ b/Documentation/devicetree/bindings/phy/amlogic,g12a-usb3-pcie-phy.yaml @@ -2,7 +2,7 @@ # Copyright 2019 BayLibre, SAS %YAML 1.2 --- -$id: "http://devicetree.org/schemas/phy/amlogic,meson-g12a-usb3-pcie-phy.yaml#" +$id: "http://devicetree.org/schemas/phy/amlogic,g12a-usb3-pcie-phy.yaml#" $schema: "http://devicetree.org/meta-schemas/core.yaml#" title: Amlogic G12A USB3 + PCIE Combo PHY @@ -13,7 +13,7 @@ maintainers: properties: compatible: enum: - - amlogic,meson-g12a-usb3-pcie-phy + - amlogic,g12a-usb3-pcie-phy reg: maxItems: 1 @@ -49,7 +49,7 @@ additionalProperties: false examples: - | phy@46000 { - compatible = "amlogic,meson-g12a-usb3-pcie-phy"; + compatible = "amlogic,g12a-usb3-pcie-phy"; reg = <0x46000 0x2000>; clocks = <&ref_clk>; clock-names = "ref_clk"; diff --git a/Documentation/devicetree/bindings/phy/qcom,usb-hs-28nm.yaml b/Documentation/devicetree/bindings/phy/qcom,usb-hs-28nm.yaml index abcc4373f39e1898244abeca9f7308f4de357d51..ca6a0836b53c4168e29830b1a5591cfc9453ed70 100644 --- a/Documentation/devicetree/bindings/phy/qcom,usb-hs-28nm.yaml +++ b/Documentation/devicetree/bindings/phy/qcom,usb-hs-28nm.yaml @@ -16,7 +16,6 @@ properties: compatible: enum: - qcom,usb-hs-28nm-femtophy - - qcom,usb-hs-28nm-mdm9607 reg: maxItems: 1 diff --git a/Documentation/devicetree/bindings/regulator/samsung,s2mps14.yaml b/Documentation/devicetree/bindings/regulator/samsung,s2mps14.yaml index 01f9d4e236e94901e9eb6e7f2d731d00c06cb77a..a7feb497eb89bdc278b4c315df1aa8f499a8bf41 100644 --- a/Documentation/devicetree/bindings/regulator/samsung,s2mps14.yaml +++ b/Documentation/devicetree/bindings/regulator/samsung,s2mps14.yaml @@ -19,8 +19,8 @@ description: | additional information and example. patternProperties: - # 25 LDOs - "^LDO([1-9]|[1][0-9]|2[0-5])$": + # 25 LDOs, without LDO10-12 + "^LDO([1-9]|1[3-9]|2[0-5])$": type: object $ref: regulator.yaml# unevaluatedProperties: false @@ -30,6 +30,23 @@ patternProperties: required: - regulator-name + "^LDO(1[0-2])$": + type: object + $ref: regulator.yaml# + unevaluatedProperties: false + description: + Properties for single LDO regulator. + + properties: + samsung,ext-control-gpios: + maxItems: 1 + description: + LDO10, LDO11 and LDO12 can be configured to external control over + GPIO. + + required: + - regulator-name + # 5 bucks "^BUCK[1-5]$": type: object diff --git a/Documentation/devicetree/bindings/riscv/cpus.yaml b/Documentation/devicetree/bindings/riscv/cpus.yaml index c6720764e765c70af73cc0dffc6e837888c74a19..a2884e3113daded85e13b8a218b472bf5c61cf80 100644 --- a/Documentation/devicetree/bindings/riscv/cpus.yaml +++ b/Documentation/devicetree/bindings/riscv/cpus.yaml @@ -83,7 +83,7 @@ properties: insensitive, letters in the riscv,isa string must be all lowercase to simplify parsing. $ref: "/schemas/types.yaml#/definitions/string" - pattern: ^rv(?:64|32)imaf?d?q?c?b?v?k?h?(?:_[hsxz](?:[a-z])+)*$ + pattern: ^rv(?:64|32)imaf?d?q?c?b?k?j?p?v?h?(?:[hsxz](?:[a-z])+)?(?:_[hsxz](?:[a-z])+)*$ # RISC-V requires 'timebase-frequency' in /cpus, so disallow it here timebase-frequency: false diff --git a/Documentation/devicetree/bindings/soc/qcom/qcom,apr-services.yaml b/Documentation/devicetree/bindings/soc/qcom/qcom,apr-services.yaml index 290555426c39ca90ebeb087cce514f51c4cc92ff..bdf482db32aac1fc84f42aa9b72cf6d64c798933 100644 --- a/Documentation/devicetree/bindings/soc/qcom/qcom,apr-services.yaml +++ b/Documentation/devicetree/bindings/soc/qcom/qcom,apr-services.yaml @@ -39,8 +39,8 @@ properties: qcom,protection-domain: $ref: /schemas/types.yaml#/definitions/string-array description: | - Protection domain service name and path for APR service - possible values are:: + Protection domain service name and path for APR service (if supported). + Possible values are:: "avs/audio", "msm/adsp/audio_pd". "kernel/elf_loader", "msm/modem/wlan_pd". "tms/servreg", "msm/adsp/audio_pd". @@ -49,6 +49,5 @@ properties: required: - reg - - qcom,protection-domain additionalProperties: true diff --git a/Documentation/devicetree/bindings/sound/everest,es8326.yaml b/Documentation/devicetree/bindings/sound/everest,es8326.yaml old mode 100755 new mode 100644 diff --git a/Documentation/devicetree/bindings/sound/mt8186-mt6366-rt1019-rt5682s.yaml b/Documentation/devicetree/bindings/sound/mt8186-mt6366-rt1019-rt5682s.yaml index 9d3139990237408ce2ff823cfb9b869cd84d4b56..aa23b0024c461d596f81071a70833daa565c7358 100644 --- a/Documentation/devicetree/bindings/sound/mt8186-mt6366-rt1019-rt5682s.yaml +++ b/Documentation/devicetree/bindings/sound/mt8186-mt6366-rt1019-rt5682s.yaml @@ -16,6 +16,7 @@ properties: compatible: enum: - mediatek,mt8186-mt6366-rt1019-rt5682s-sound + - mediatek,mt8186-mt6366-rt5682s-max98360-sound mediatek,platform: $ref: "/schemas/types.yaml#/definitions/phandle" diff --git a/Documentation/devicetree/bindings/sound/qcom,lpass-tx-macro.yaml b/Documentation/devicetree/bindings/sound/qcom,lpass-tx-macro.yaml index 66431aade3b70a1794bd317af60a9a2cadd8cefd..da5f70910da5204ee0c37bba62f3f7473852983d 100644 --- a/Documentation/devicetree/bindings/sound/qcom,lpass-tx-macro.yaml +++ b/Documentation/devicetree/bindings/sound/qcom,lpass-tx-macro.yaml @@ -30,7 +30,9 @@ properties: const: 0 clocks: - maxItems: 5 + oneOf: + - maxItems: 3 + - maxItems: 5 clock-names: oneOf: diff --git a/Documentation/devicetree/bindings/sound/qcom,lpass-wsa-macro.yaml b/Documentation/devicetree/bindings/sound/qcom,lpass-wsa-macro.yaml index 2bf8d082f8f1044d64bb9493eea892f14b629378..66cbb1f5e31a5cdaf53126972b1f32c4122fed67 100644 --- a/Documentation/devicetree/bindings/sound/qcom,lpass-wsa-macro.yaml +++ b/Documentation/devicetree/bindings/sound/qcom,lpass-wsa-macro.yaml @@ -9,9 +9,6 @@ title: LPASS(Low Power Audio Subsystem) VA Macro audio codec maintainers: - Srinivas Kandagatla -allOf: - - $ref: dai-common.yaml# - properties: compatible: enum: @@ -30,15 +27,12 @@ properties: const: 0 clocks: - maxItems: 5 + minItems: 5 + maxItems: 6 clock-names: - items: - - const: mclk - - const: npl - - const: macro - - const: dcodec - - const: fsgen + minItems: 5 + maxItems: 6 clock-output-names: maxItems: 1 @@ -55,10 +49,51 @@ required: - reg - "#sound-dai-cells" +allOf: + - $ref: dai-common.yaml# + + - if: + properties: + compatible: + enum: + - qcom,sc7280-lpass-wsa-macro + - qcom,sm8450-lpass-wsa-macro + - qcom,sc8280xp-lpass-wsa-macro + then: + properties: + clocks: + maxItems: 5 + clock-names: + items: + - const: mclk + - const: npl + - const: macro + - const: dcodec + - const: fsgen + + - if: + properties: + compatible: + enum: + - qcom,sm8250-lpass-wsa-macro + then: + properties: + clocks: + minItems: 6 + clock-names: + items: + - const: mclk + - const: npl + - const: macro + - const: dcodec + - const: va + - const: fsgen + unevaluatedProperties: false examples: - | + #include #include codec@3240000 { compatible = "qcom,sm8250-lpass-wsa-macro"; @@ -69,7 +104,8 @@ examples: <&audiocc 0>, <&q6afecc LPASS_HW_MACRO_VOTE LPASS_CLK_ATTRIBUTE_COUPLE_NO>, <&q6afecc LPASS_HW_DCODEC_VOTE LPASS_CLK_ATTRIBUTE_COUPLE_NO>, + <&aoncc LPASS_CDC_VA_MCLK>, <&vamacro>; - clock-names = "mclk", "npl", "macro", "dcodec", "fsgen"; + clock-names = "mclk", "npl", "macro", "dcodec", "va", "fsgen"; clock-output-names = "mclk"; }; diff --git a/Documentation/devicetree/bindings/soundwire/qcom,soundwire.yaml b/Documentation/devicetree/bindings/soundwire/qcom,soundwire.yaml index bcbfa71536cda602d40e1e6a4a0d3b9bfb468a31..3efdc192ab019182571338fef555dd3a9653c6f7 100644 --- a/Documentation/devicetree/bindings/soundwire/qcom,soundwire.yaml +++ b/Documentation/devicetree/bindings/soundwire/qcom,soundwire.yaml @@ -80,7 +80,7 @@ properties: or applicable for the respective data port. More info in MIPI Alliance SoundWire 1.0 Specifications. minItems: 3 - maxItems: 5 + maxItems: 8 qcom,ports-sinterval-low: $ref: /schemas/types.yaml#/definitions/uint8-array @@ -124,7 +124,7 @@ properties: or applicable for the respective data port. More info in MIPI Alliance SoundWire 1.0 Specifications. minItems: 3 - maxItems: 5 + maxItems: 8 qcom,ports-block-pack-mode: $ref: /schemas/types.yaml#/definitions/uint8-array @@ -154,7 +154,7 @@ properties: or applicable for the respective data port. More info in MIPI Alliance SoundWire 1.0 Specifications. minItems: 3 - maxItems: 5 + maxItems: 8 items: oneOf: - minimum: 0 @@ -171,7 +171,7 @@ properties: or applicable for the respective data port. More info in MIPI Alliance SoundWire 1.0 Specifications. minItems: 3 - maxItems: 5 + maxItems: 8 items: oneOf: - minimum: 0 @@ -187,7 +187,7 @@ properties: or applicable for the respective data port. More info in MIPI Alliance SoundWire 1.0 Specifications. minItems: 3 - maxItems: 5 + maxItems: 8 items: oneOf: - minimum: 0 diff --git a/Documentation/devicetree/bindings/spi/atmel,at91rm9200-spi.yaml b/Documentation/devicetree/bindings/spi/atmel,at91rm9200-spi.yaml index 4dd973e341e6c772a2f6f1d945ce31eb6aab325e..6c57dd6c3a3614d3ed883bff2d1ed2fb20edf483 100644 --- a/Documentation/devicetree/bindings/spi/atmel,at91rm9200-spi.yaml +++ b/Documentation/devicetree/bindings/spi/atmel,at91rm9200-spi.yaml @@ -8,7 +8,7 @@ $schema: http://devicetree.org/meta-schemas/core.yaml# title: Atmel SPI device maintainers: - - Tudor Ambarus + - Tudor Ambarus allOf: - $ref: spi-controller.yaml# diff --git a/Documentation/devicetree/bindings/spi/atmel,quadspi.yaml b/Documentation/devicetree/bindings/spi/atmel,quadspi.yaml index 1d493add4053dd11add73e67e0e35ca81a12f16b..b0d99bc105352c5bcae56708896d508b2680850d 100644 --- a/Documentation/devicetree/bindings/spi/atmel,quadspi.yaml +++ b/Documentation/devicetree/bindings/spi/atmel,quadspi.yaml @@ -7,7 +7,7 @@ $schema: http://devicetree.org/meta-schemas/core.yaml# title: Atmel Quad Serial Peripheral Interface (QSPI) maintainers: - - Tudor Ambarus + - Tudor Ambarus allOf: - $ref: spi-controller.yaml# diff --git a/Documentation/devicetree/bindings/spi/spi-peripheral-props.yaml b/Documentation/devicetree/bindings/spi/spi-peripheral-props.yaml index ead2cccf658fdee57208afaa695fabe68668d1a4..9a60c0664bbe7df45fb113c3a314105c2ad5e4c0 100644 --- a/Documentation/devicetree/bindings/spi/spi-peripheral-props.yaml +++ b/Documentation/devicetree/bindings/spi/spi-peripheral-props.yaml @@ -44,9 +44,9 @@ properties: description: Maximum SPI clocking speed of the device in Hz. - spi-cs-setup-ns: + spi-cs-setup-delay-ns: description: - Delay in nanosecods to be introduced by the controller after CS is + Delay in nanoseconds to be introduced by the controller after CS is asserted. spi-rx-bus-width: diff --git a/Documentation/filesystems/erofs.rst b/Documentation/filesystems/erofs.rst index 067fd1670b1f1d6937d214f33309c4fc55e29923..a43aacf1494e778585aea093fb4c099dc4459be7 100644 --- a/Documentation/filesystems/erofs.rst +++ b/Documentation/filesystems/erofs.rst @@ -120,6 +120,8 @@ dax={always,never} Use direct access (no page cache). See dax A legacy option which is an alias for ``dax=always``. device=%s Specify a path to an extra device to be used together. fsid=%s Specify a filesystem image ID for Fscache back-end. +domain_id=%s Specify a domain ID in fscache mode so that different images + with the same blobs under a given domain ID can share storage. =================== ========================================================= Sysfs Entries diff --git a/Documentation/kbuild/makefiles.rst b/Documentation/kbuild/makefiles.rst index 6b7368d1f51639c016e4be5f4c3c4f51b0f29991..38bc74eaa547403935a2ae645cb0ec875a1a0279 100644 --- a/Documentation/kbuild/makefiles.rst +++ b/Documentation/kbuild/makefiles.rst @@ -1042,7 +1042,7 @@ $(clean-files). When executing "make clean", the file "crc32table.h" will be deleted. Kbuild will assume files to be in the same relative directory as the -Makefile, except if prefixed with $(objtree). +Makefile. To exclude certain files or directories from make clean, use the $(no-clean-files) variable. diff --git a/Documentation/networking/bridge.rst b/Documentation/networking/bridge.rst index 4aef9cddde2fd68bf75a1ec8c3453ad6cb88237c..c859f3c1636edfa30a4f2cde792608fb949a58b3 100644 --- a/Documentation/networking/bridge.rst +++ b/Documentation/networking/bridge.rst @@ -8,7 +8,7 @@ In order to use the Ethernet bridging functionality, you'll need the userspace tools. Documentation for Linux bridging is on: - http://www.linuxfoundation.org/collaborate/workgroups/networking/bridge + https://wiki.linuxfoundation.org/networking/bridge The bridge-utilities are maintained at: git://git.kernel.org/pub/scm/linux/kernel/git/shemminger/bridge-utils.git diff --git a/Documentation/networking/nf_conntrack-sysctl.rst b/Documentation/networking/nf_conntrack-sysctl.rst index 49db1d11d7c4a8788f5068e4ac97a420202044f7..8b1045c3b59e0ef0675d0ed113139b4e84b8b81d 100644 --- a/Documentation/networking/nf_conntrack-sysctl.rst +++ b/Documentation/networking/nf_conntrack-sysctl.rst @@ -173,7 +173,9 @@ nf_conntrack_sctp_timeout_cookie_echoed - INTEGER (seconds) default 3 nf_conntrack_sctp_timeout_established - INTEGER (seconds) - default 432000 (5 days) + default 210 + + Default is set to (hb_interval * path_max_retrans + rto_max) nf_conntrack_sctp_timeout_shutdown_sent - INTEGER (seconds) default 0.3 @@ -190,12 +192,6 @@ nf_conntrack_sctp_timeout_heartbeat_sent - INTEGER (seconds) This timeout is used to setup conntrack entry on secondary paths. Default is set to hb_interval. -nf_conntrack_sctp_timeout_heartbeat_acked - INTEGER (seconds) - default 210 - - This timeout is used to setup conntrack entry on secondary paths. - Default is set to (hb_interval * path_max_retrans + rto_max) - nf_conntrack_udp_timeout - INTEGER (seconds) default 30 diff --git a/Documentation/networking/rxrpc.rst b/Documentation/networking/rxrpc.rst index 39494a6ea739cf02217f088721ff23e1751521fe..e1af54424192b375f701992a95ffeeb68a76de8c 100644 --- a/Documentation/networking/rxrpc.rst +++ b/Documentation/networking/rxrpc.rst @@ -880,8 +880,8 @@ The kernel interface functions are as follows: notify_end_rx can be NULL or it can be used to specify a function to be called when the call changes state to end the Tx phase. This function is - called with the call-state spinlock held to prevent any reply or final ACK - from being delivered first. + called with a spinlock held to prevent the last DATA packet from being + transmitted until the function returns. (#) Receive data from a call:: diff --git a/Documentation/process/maintainer-netdev.rst b/Documentation/process/maintainer-netdev.rst index 1fa5ab8754d358a80f707e544ab03d4b9734e3b1..4a75686d35ab4a646babcadb8d187bde8600038a 100644 --- a/Documentation/process/maintainer-netdev.rst +++ b/Documentation/process/maintainer-netdev.rst @@ -2,9 +2,9 @@ .. _netdev-FAQ: -========== -netdev FAQ -========== +============================= +Networking subsystem (netdev) +============================= tl;dr ----- @@ -15,14 +15,15 @@ tl;dr - don't repost your patches within one 24h period - reverse xmas tree -What is netdev? ---------------- -It is a mailing list for all network-related Linux stuff. This +netdev +------ + +netdev is a mailing list for all network-related Linux stuff. This includes anything found under net/ (i.e. core code like IPv6) and drivers/net (i.e. hardware specific drivers) in the Linux source tree. Note that some subsystems (e.g. wireless drivers) which have a high -volume of traffic have their own specific mailing lists. +volume of traffic have their own specific mailing lists and trees. The netdev list is managed (like many other Linux mailing lists) through VGER (http://vger.kernel.org/) with archives available at @@ -32,32 +33,10 @@ Aside from subsystems like those mentioned above, all network-related Linux development (i.e. RFC, review, comments, etc.) takes place on netdev. -How do the changes posted to netdev make their way into Linux? --------------------------------------------------------------- -There are always two trees (git repositories) in play. Both are -driven by David Miller, the main network maintainer. There is the -``net`` tree, and the ``net-next`` tree. As you can probably guess from -the names, the ``net`` tree is for fixes to existing code already in the -mainline tree from Linus, and ``net-next`` is where the new code goes -for the future release. You can find the trees here: - -- https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git -- https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next.git - -How do I indicate which tree (net vs. net-next) my patch should be in? ----------------------------------------------------------------------- -To help maintainers and CI bots you should explicitly mark which tree -your patch is targeting. Assuming that you use git, use the prefix -flag:: - - git format-patch --subject-prefix='PATCH net-next' start..finish +Development cycle +----------------- -Use ``net`` instead of ``net-next`` (always lower case) in the above for -bug-fix ``net`` content. - -How often do changes from these trees make it to the mainline Linus tree? -------------------------------------------------------------------------- -To understand this, you need to know a bit of background information on +Here is a bit of background information on the cadence of Linux development. Each new release starts off with a two week "merge window" where the main maintainers feed their new stuff to Linus for merging into the mainline tree. After the two weeks, the @@ -69,9 +48,33 @@ rc2 is released. This repeats on a roughly weekly basis until rc7 state of churn), and a week after the last vX.Y-rcN was done, the official vX.Y is released. -Relating that to netdev: At the beginning of the 2-week merge window, -the ``net-next`` tree will be closed - no new changes/features. The -accumulated new content of the past ~10 weeks will be passed onto +To find out where we are now in the cycle - load the mainline (Linus) +page here: + + https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git + +and note the top of the "tags" section. If it is rc1, it is early in +the dev cycle. If it was tagged rc7 a week ago, then a release is +probably imminent. If the most recent tag is a final release tag +(without an ``-rcN`` suffix) - we are most likely in a merge window +and ``net-next`` is closed. + +git trees and patch flow +------------------------ + +There are two networking trees (git repositories) in play. Both are +driven by David Miller, the main network maintainer. There is the +``net`` tree, and the ``net-next`` tree. As you can probably guess from +the names, the ``net`` tree is for fixes to existing code already in the +mainline tree from Linus, and ``net-next`` is where the new code goes +for the future release. You can find the trees here: + +- https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git +- https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next.git + +Relating that to kernel development: At the beginning of the 2-week +merge window, the ``net-next`` tree will be closed - no new changes/features. +The accumulated new content of the past ~10 weeks will be passed onto mainline/Linus via a pull request for vX.Y -- at the same time, the ``net`` tree will start accumulating fixes for this pulled content relating to vX.Y @@ -103,22 +106,14 @@ focus for ``net`` is on stabilization and bug fixes. Finally, the vX.Y gets released, and the whole cycle starts over. -So where are we now in this cycle? ----------------------------------- +netdev patch review +------------------- -Load the mainline (Linus) page here: +Patch status +~~~~~~~~~~~~ - https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git - -and note the top of the "tags" section. If it is rc1, it is early in -the dev cycle. If it was tagged rc7 a week ago, then a release is -probably imminent. If the most recent tag is a final release tag -(without an ``-rcN`` suffix) - we are most likely in a merge window -and ``net-next`` is closed. - -How can I tell the status of a patch I've sent? ------------------------------------------------ -Start by looking at the main patchworks queue for netdev: +Status of a patch can be checked by looking at the main patchwork +queue for netdev: https://patchwork.kernel.org/project/netdevbpf/list/ @@ -127,73 +122,141 @@ patch. Patches are indexed by the ``Message-ID`` header of the emails which carried them so if you have trouble finding your patch append the value of ``Message-ID`` to the URL above. -How long before my patch is accepted? -------------------------------------- -Generally speaking, the patches get triaged quickly (in less than -48h). But be patient, if your patch is active in patchwork (i.e. it's -listed on the project's patch list) the chances it was missed are close to zero. -Asking the maintainer for status updates on your -patch is a good way to ensure your patch is ignored or pushed to the -bottom of the priority list. +Updating patch status +~~~~~~~~~~~~~~~~~~~~~ -Should I directly update patchwork state of my own patches? ------------------------------------------------------------ It may be tempting to help the maintainers and update the state of your -own patches when you post a new version or spot a bug. Please do not do that. +own patches when you post a new version or spot a bug. Please **do not** +do that. Interfering with the patch status on patchwork will only cause confusion. Leave it to the maintainer to figure out what is the most recent and current version that should be applied. If there is any doubt, the maintainer will reply and ask what should be done. -How do I divide my work into patches? -------------------------------------- +Review timelines +~~~~~~~~~~~~~~~~ -Put yourself in the shoes of the reviewer. Each patch is read separately -and therefore should constitute a comprehensible step towards your stated -goal. +Generally speaking, the patches get triaged quickly (in less than +48h). But be patient, if your patch is active in patchwork (i.e. it's +listed on the project's patch list) the chances it was missed are close to zero. +Asking the maintainer for status updates on your +patch is a good way to ensure your patch is ignored or pushed to the +bottom of the priority list. -Avoid sending series longer than 15 patches. Larger series takes longer -to review as reviewers will defer looking at it until they find a large -chunk of time. A small series can be reviewed in a short time, so Maintainers -just do it. As a result, a sequence of smaller series gets merged quicker and -with better review coverage. Re-posting large series also increases the mailing -list traffic. +Partial resends +~~~~~~~~~~~~~~~ -I made changes to only a few patches in a patch series should I resend only those changed? ------------------------------------------------------------------------------------------- -No, please resend the entire patch series and make sure you do number your +Please always resend the entire patch series and make sure you do number your patches such that it is clear this is the latest and greatest set of patches -that can be applied. - -I have received review feedback, when should I post a revised version of the patches? -------------------------------------------------------------------------------------- -Allow at least 24 hours to pass between postings. This will ensure reviewers -from all geographical locations have a chance to chime in. Do not wait -too long (weeks) between postings either as it will make it harder for reviewers -to recall all the context. +that can be applied. Do not try to resend just the patches which changed. -Make sure you address all the feedback in your new posting. Do not post a new -version of the code if the discussion about the previous version is still -ongoing, unless directly instructed by a reviewer. +Handling misapplied patches +~~~~~~~~~~~~~~~~~~~~~~~~~~~ -I submitted multiple versions of a patch series and it looks like a version other than the last one has been accepted, what should I do? ----------------------------------------------------------------------------------------------------------------------------------------- +Occasionally a patch series gets applied before receiving critical feedback, +or the wrong version of a series gets applied. There is no revert possible, once it is pushed out, it stays like that. Please send incremental versions on top of what has been merged in order to fix the patches the way they would look like if your latest patch series was to be merged. -Are there special rules regarding stable submissions on netdev? ---------------------------------------------------------------- +Stable tree +~~~~~~~~~~~ + While it used to be the case that netdev submissions were not supposed to carry explicit ``CC: stable@vger.kernel.org`` tags that is no longer the case today. Please follow the standard stable rules in :ref:`Documentation/process/stable-kernel-rules.rst `, and make sure you include appropriate Fixes tags! -Is the comment style convention different for the networking content? ---------------------------------------------------------------------- -Yes, in a largely trivial way. Instead of this:: +Security fixes +~~~~~~~~~~~~~~ + +Do not email netdev maintainers directly if you think you discovered +a bug that might have possible security implications. +The current netdev maintainer has consistently requested that +people use the mailing lists and not reach out directly. If you aren't +OK with that, then perhaps consider mailing security@kernel.org or +reading about http://oss-security.openwall.org/wiki/mailing-lists/distros +as possible alternative mechanisms. + + +Co-posting changes to user space components +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +User space code exercising kernel features should be posted +alongside kernel patches. This gives reviewers a chance to see +how any new interface is used and how well it works. + +When user space tools reside in the kernel repo itself all changes +should generally come as one series. If series becomes too large +or the user space project is not reviewed on netdev include a link +to a public repo where user space patches can be seen. + +In case user space tooling lives in a separate repository but is +reviewed on netdev (e.g. patches to ``iproute2`` tools) kernel and +user space patches should form separate series (threads) when posted +to the mailing list, e.g.:: + + [PATCH net-next 0/3] net: some feature cover letter + └─ [PATCH net-next 1/3] net: some feature prep + └─ [PATCH net-next 2/3] net: some feature do it + └─ [PATCH net-next 3/3] selftest: net: some feature + + [PATCH iproute2-next] ip: add support for some feature + +Posting as one thread is discouraged because it confuses patchwork +(as of patchwork 2.2.2). + +Preparing changes +----------------- + +Attention to detail is important. Re-read your own work as if you were the +reviewer. You can start with using ``checkpatch.pl``, perhaps even with +the ``--strict`` flag. But do not be mindlessly robotic in doing so. +If your change is a bug fix, make sure your commit log indicates the +end-user visible symptom, the underlying reason as to why it happens, +and then if necessary, explain why the fix proposed is the best way to +get things done. Don't mangle whitespace, and as is common, don't +mis-indent function arguments that span multiple lines. If it is your +first patch, mail it to yourself so you can test apply it to an +unpatched tree to confirm infrastructure didn't mangle it. + +Finally, go back and read +:ref:`Documentation/process/submitting-patches.rst ` +to be sure you are not repeating some common mistake documented there. + +Indicating target tree +~~~~~~~~~~~~~~~~~~~~~~ + +To help maintainers and CI bots you should explicitly mark which tree +your patch is targeting. Assuming that you use git, use the prefix +flag:: + + git format-patch --subject-prefix='PATCH net-next' start..finish + +Use ``net`` instead of ``net-next`` (always lower case) in the above for +bug-fix ``net`` content. + +Dividing work into patches +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Put yourself in the shoes of the reviewer. Each patch is read separately +and therefore should constitute a comprehensible step towards your stated +goal. + +Avoid sending series longer than 15 patches. Larger series takes longer +to review as reviewers will defer looking at it until they find a large +chunk of time. A small series can be reviewed in a short time, so Maintainers +just do it. As a result, a sequence of smaller series gets merged quicker and +with better review coverage. Re-posting large series also increases the mailing +list traffic. + +Multi-line comments +~~~~~~~~~~~~~~~~~~~ + +Comment style convention is slightly different for networking and most of +the tree. Instead of this:: /* * foobar blah blah blah @@ -206,8 +269,8 @@ it is requested that you make it look like this:: * another line of text */ -What is "reverse xmas tree"? ----------------------------- +Local variable ordering ("reverse xmas tree", "RCS") +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Netdev has a convention for ordering local variables in functions. Order the variable declaration lines longest to shortest, e.g.:: @@ -219,21 +282,31 @@ Order the variable declaration lines longest to shortest, e.g.:: If there are dependencies between the variables preventing the ordering move the initialization out of line. -I am working in existing code which uses non-standard formatting. Which formatting should I use? ------------------------------------------------------------------------------------------------- -Make your code follow the most recent guidelines, so that eventually all code +Format precedence +~~~~~~~~~~~~~~~~~ + +When working in existing code which uses nonstandard formatting make +your code follow the most recent guidelines, so that eventually all code in the domain of netdev is in the preferred format. -I found a bug that might have possible security implications or similar. Should I mail the main netdev maintainer off-list? ---------------------------------------------------------------------------------------------------------------------------- -No. The current netdev maintainer has consistently requested that -people use the mailing lists and not reach out directly. If you aren't -OK with that, then perhaps consider mailing security@kernel.org or -reading about http://oss-security.openwall.org/wiki/mailing-lists/distros -as possible alternative mechanisms. +Resending after review +~~~~~~~~~~~~~~~~~~~~~~ + +Allow at least 24 hours to pass between postings. This will ensure reviewers +from all geographical locations have a chance to chime in. Do not wait +too long (weeks) between postings either as it will make it harder for reviewers +to recall all the context. + +Make sure you address all the feedback in your new posting. Do not post a new +version of the code if the discussion about the previous version is still +ongoing, unless directly instructed by a reviewer. + +Testing +------- + +Expected level of testing +~~~~~~~~~~~~~~~~~~~~~~~~~ -What level of testing is expected before I submit my change? ------------------------------------------------------------- At the very minimum your changes must survive an ``allyesconfig`` and an ``allmodconfig`` build with ``W=1`` set without new warnings or failures. @@ -244,86 +317,42 @@ and the patch series contains a set of kernel selftest for You are expected to test your changes on top of the relevant networking tree (``net`` or ``net-next``) and not e.g. a stable tree or ``linux-next``. -How do I post corresponding changes to user space components? -------------------------------------------------------------- -User space code exercising kernel features should be posted -alongside kernel patches. This gives reviewers a chance to see -how any new interface is used and how well it works. - -When user space tools reside in the kernel repo itself all changes -should generally come as one series. If series becomes too large -or the user space project is not reviewed on netdev include a link -to a public repo where user space patches can be seen. - -In case user space tooling lives in a separate repository but is -reviewed on netdev (e.g. patches to ``iproute2`` tools) kernel and -user space patches should form separate series (threads) when posted -to the mailing list, e.g.:: - - [PATCH net-next 0/3] net: some feature cover letter - └─ [PATCH net-next 1/3] net: some feature prep - └─ [PATCH net-next 2/3] net: some feature do it - └─ [PATCH net-next 3/3] selftest: net: some feature - - [PATCH iproute2-next] ip: add support for some feature - -Posting as one thread is discouraged because it confuses patchwork -(as of patchwork 2.2.2). - -Can I reproduce the checks from patchwork on my local machine? --------------------------------------------------------------- +patchwork checks +~~~~~~~~~~~~~~~~ Checks in patchwork are mostly simple wrappers around existing kernel scripts, the sources are available at: https://github.com/kuba-moo/nipa/tree/master/tests -Running all the builds and checks locally is a pain, can I post my patches and have the patchwork bot validate them? --------------------------------------------------------------------------------------------------------------------- - -No, you must ensure that your patches are ready by testing them locally +**Do not** post your patches just to run them through the checks. +You must ensure that your patches are ready by testing them locally before posting to the mailing list. The patchwork build bot instance gets overloaded very easily and netdev@vger really doesn't need more traffic if we can help it. -netdevsim is great, can I extend it for my out-of-tree tests? -------------------------------------------------------------- +netdevsim +~~~~~~~~~ -No, ``netdevsim`` is a test vehicle solely for upstream tests. -(Please add your tests under ``tools/testing/selftests/``.) +``netdevsim`` is a test driver which can be used to exercise driver +configuration APIs without requiring capable hardware. +Mock-ups and tests based on ``netdevsim`` are strongly encouraged when +adding new APIs, but ``netdevsim`` in itself is **not** considered +a use case/user. You must also implement the new APIs in a real driver. -We also give no guarantees that ``netdevsim`` won't change in the future +We give no guarantees that ``netdevsim`` won't change in the future in a way which would break what would normally be considered uAPI. -Is netdevsim considered a "user" of an API? -------------------------------------------- - -Linux kernel has a long standing rule that no API should be added unless -it has a real, in-tree user. Mock-ups and tests based on ``netdevsim`` are -strongly encouraged when adding new APIs, but ``netdevsim`` in itself -is **not** considered a use case/user. - -Any other tips to help ensure my net/net-next patch gets OK'd? --------------------------------------------------------------- -Attention to detail. Re-read your own work as if you were the -reviewer. You can start with using ``checkpatch.pl``, perhaps even with -the ``--strict`` flag. But do not be mindlessly robotic in doing so. -If your change is a bug fix, make sure your commit log indicates the -end-user visible symptom, the underlying reason as to why it happens, -and then if necessary, explain why the fix proposed is the best way to -get things done. Don't mangle whitespace, and as is common, don't -mis-indent function arguments that span multiple lines. If it is your -first patch, mail it to yourself so you can test apply it to an -unpatched tree to confirm infrastructure didn't mangle it. - -Finally, go back and read -:ref:`Documentation/process/submitting-patches.rst ` -to be sure you are not repeating some common mistake documented there. +``netdevsim`` is reserved for use by upstream tests only, so any +new ``netdevsim`` features must be accompanied by selftests under +``tools/testing/selftests/``. -My company uses peer feedback in employee performance reviews. Can I ask netdev maintainers for feedback? ---------------------------------------------------------------------------------------------------------- +Testimonials / feedback +----------------------- -Yes, especially if you spend significant amount of time reviewing code +Some companies use peer feedback in employee performance reviews. +Please feel free to request feedback from netdev maintainers, +especially if you spend significant amount of time reviewing code and go out of your way to improve shared infrastructure. The feedback must be requested by you, the contributor, and will always diff --git a/Documentation/sphinx/load_config.py b/Documentation/sphinx/load_config.py index eeb394b39e2cc8dc8d8f866a379cf492f5d8b73b..8b416bfd75ac1711d0a6275a5bf84f71c3e40a88 100644 --- a/Documentation/sphinx/load_config.py +++ b/Documentation/sphinx/load_config.py @@ -3,7 +3,7 @@ import os import sys -from sphinx.util.pycompat import execfile_ +from sphinx.util.osutil import fs_encoding # ------------------------------------------------------------------------------ def loadConfig(namespace): @@ -48,7 +48,9 @@ def loadConfig(namespace): sys.stdout.write("load additional sphinx-config: %s\n" % config_file) config = namespace.copy() config['__file__'] = config_file - execfile_(config_file, config) + with open(config_file, 'rb') as f: + code = compile(f.read(), fs_encoding, 'exec') + exec(code, config) del config['__file__'] namespace.update(config) else: diff --git a/Documentation/userspace-api/media/v4l/pixfmt-packed-yuv.rst b/Documentation/userspace-api/media/v4l/pixfmt-packed-yuv.rst index bf283a1b5581dab5f0ad408eb80385f324ce6bc2..24a7715420594a1b8e5c17f09aef0f7b8a07a16b 100644 --- a/Documentation/userspace-api/media/v4l/pixfmt-packed-yuv.rst +++ b/Documentation/userspace-api/media/v4l/pixfmt-packed-yuv.rst @@ -262,7 +262,12 @@ the second byte and Y'\ :sub:`7-0` in the third byte. ================= These formats, commonly referred to as YUYV or YUY2, subsample the chroma -components horizontally by 2, storing 2 pixels in 4 bytes. +components horizontally by 2, storing 2 pixels in a container. The container +is 32-bits for 8-bit formats, and 64-bits for 10+-bit formats. + +The packed YUYV formats with more than 8 bits per component are stored as four +16-bit little-endian words. Each word's most significant bits contain one +component, and the least significant bits are zero padding. .. raw:: latex @@ -270,7 +275,7 @@ components horizontally by 2, storing 2 pixels in 4 bytes. .. tabularcolumns:: |p{3.4cm}|p{1.2cm}|p{0.8cm}|p{0.8cm}|p{0.8cm}|p{0.8cm}|p{0.8cm}|p{0.8cm}|p{0.8cm}|p{0.8cm}| -.. flat-table:: Packed YUV 4:2:2 Formats +.. flat-table:: Packed YUV 4:2:2 Formats in 32-bit container :header-rows: 1 :stub-columns: 0 @@ -337,6 +342,46 @@ components horizontally by 2, storing 2 pixels in 4 bytes. - Y'\ :sub:`3` - Cb\ :sub:`2` +.. tabularcolumns:: |p{3.4cm}|p{1.2cm}|p{0.8cm}|p{0.8cm}|p{0.8cm}|p{0.8cm}|p{0.8cm}|p{0.8cm}|p{0.8cm}|p{0.8cm}| + +.. flat-table:: Packed YUV 4:2:2 Formats in 64-bit container + :header-rows: 1 + :stub-columns: 0 + + * - Identifier + - Code + - Word 0 + - Word 1 + - Word 2 + - Word 3 + * .. _V4L2-PIX-FMT-Y210: + + - ``V4L2_PIX_FMT_Y210`` + - 'Y210' + + - Y'\ :sub:`0` (bits 15-6) + - Cb\ :sub:`0` (bits 15-6) + - Y'\ :sub:`1` (bits 15-6) + - Cr\ :sub:`0` (bits 15-6) + * .. _V4L2-PIX-FMT-Y212: + + - ``V4L2_PIX_FMT_Y212`` + - 'Y212' + + - Y'\ :sub:`0` (bits 15-4) + - Cb\ :sub:`0` (bits 15-4) + - Y'\ :sub:`1` (bits 15-4) + - Cr\ :sub:`0` (bits 15-4) + * .. _V4L2-PIX-FMT-Y216: + + - ``V4L2_PIX_FMT_Y216`` + - 'Y216' + + - Y'\ :sub:`0` (bits 15-0) + - Cb\ :sub:`0` (bits 15-0) + - Y'\ :sub:`1` (bits 15-0) + - Cr\ :sub:`0` (bits 15-0) + .. raw:: latex \normalsize diff --git a/Documentation/userspace-api/media/v4l/pixfmt-rgb.rst b/Documentation/userspace-api/media/v4l/pixfmt-rgb.rst index 30f51cd33f99a88861d4c6d1d3a5ce8eee6a67fb..d330aeb4d3ebf74af6e931d839c70fabf8e832d2 100644 --- a/Documentation/userspace-api/media/v4l/pixfmt-rgb.rst +++ b/Documentation/userspace-api/media/v4l/pixfmt-rgb.rst @@ -763,6 +763,200 @@ nomenclature that instead use the order of components as seen in a 24- or \normalsize +10 Bits Per Component +===================== + +These formats store a 30-bit RGB triplet with an optional 2 bit alpha in four +bytes. They are named based on the order of the RGB components as seen in a +32-bit word, which is then stored in memory in little endian byte order +(unless otherwise noted by the presence of bit 31 in the 4CC value), and on the +number of bits for each component. + +.. raw:: latex + + \begingroup + \tiny + \setlength{\tabcolsep}{2pt} + +.. tabularcolumns:: |p{2.8cm}|p{2.0cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}| + + +.. flat-table:: RGB Formats 10 Bits Per Color Component + :header-rows: 2 + :stub-columns: 0 + + * - Identifier + - Code + - :cspan:`7` Byte 0 in memory + - :cspan:`7` Byte 1 + - :cspan:`7` Byte 2 + - :cspan:`7` Byte 3 + * - + - + - 7 + - 6 + - 5 + - 4 + - 3 + - 2 + - 1 + - 0 + + - 7 + - 6 + - 5 + - 4 + - 3 + - 2 + - 1 + - 0 + + - 7 + - 6 + - 5 + - 4 + - 3 + - 2 + - 1 + - 0 + + - 7 + - 6 + - 5 + - 4 + - 3 + - 2 + - 1 + - 0 + * .. _V4L2-PIX-FMT-RGBX1010102: + + - ``V4L2_PIX_FMT_RGBX1010102`` + - 'RX30' + + - b\ :sub:`5` + - b\ :sub:`4` + - b\ :sub:`3` + - b\ :sub:`2` + - b\ :sub:`1` + - b\ :sub:`0` + - x + - x + + - g\ :sub:`3` + - g\ :sub:`2` + - g\ :sub:`1` + - g\ :sub:`0` + - b\ :sub:`9` + - b\ :sub:`8` + - b\ :sub:`7` + - b\ :sub:`6` + + - r\ :sub:`1` + - r\ :sub:`0` + - g\ :sub:`9` + - g\ :sub:`8` + - g\ :sub:`7` + - g\ :sub:`6` + - g\ :sub:`5` + - g\ :sub:`4` + + - r\ :sub:`9` + - r\ :sub:`8` + - r\ :sub:`7` + - r\ :sub:`6` + - r\ :sub:`5` + - r\ :sub:`4` + - r\ :sub:`3` + - r\ :sub:`2` + - + * .. _V4L2-PIX-FMT-RGBA1010102: + + - ``V4L2_PIX_FMT_RGBA1010102`` + - 'RA30' + + - b\ :sub:`5` + - b\ :sub:`4` + - b\ :sub:`3` + - b\ :sub:`2` + - b\ :sub:`1` + - b\ :sub:`0` + - a\ :sub:`1` + - a\ :sub:`0` + + - g\ :sub:`3` + - g\ :sub:`2` + - g\ :sub:`1` + - g\ :sub:`0` + - b\ :sub:`9` + - b\ :sub:`8` + - b\ :sub:`7` + - b\ :sub:`6` + + - r\ :sub:`1` + - r\ :sub:`0` + - g\ :sub:`9` + - g\ :sub:`8` + - g\ :sub:`7` + - g\ :sub:`6` + - g\ :sub:`5` + - g\ :sub:`4` + + - r\ :sub:`9` + - r\ :sub:`8` + - r\ :sub:`7` + - r\ :sub:`6` + - r\ :sub:`5` + - r\ :sub:`4` + - r\ :sub:`3` + - r\ :sub:`2` + - + * .. _V4L2-PIX-FMT-ARGB2101010: + + - ``V4L2_PIX_FMT_ARGB2101010`` + - 'AR30' + + - b\ :sub:`7` + - b\ :sub:`6` + - b\ :sub:`5` + - b\ :sub:`4` + - b\ :sub:`3` + - b\ :sub:`2` + - b\ :sub:`1` + - b\ :sub:`0` + + - g\ :sub:`5` + - g\ :sub:`4` + - g\ :sub:`3` + - g\ :sub:`2` + - g\ :sub:`1` + - g\ :sub:`0` + - b\ :sub:`9` + - b\ :sub:`8` + + - r\ :sub:`3` + - r\ :sub:`2` + - r\ :sub:`1` + - r\ :sub:`0` + - g\ :sub:`9` + - g\ :sub:`8` + - g\ :sub:`7` + - g\ :sub:`6` + + - a\ :sub:`1` + - a\ :sub:`0` + - r\ :sub:`9` + - r\ :sub:`8` + - r\ :sub:`7` + - r\ :sub:`6` + - r\ :sub:`5` + - r\ :sub:`4` + - + +.. raw:: latex + + \endgroup + + Deprecated RGB Formats ====================== diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst index deb494f759ed39c4b0f70bf19af63ec5d921dd6c..9807b05a1b57109f1051ac5fb24823cdf13fed12 100644 --- a/Documentation/virt/kvm/api.rst +++ b/Documentation/virt/kvm/api.rst @@ -1354,6 +1354,14 @@ the memory region are automatically reflected into the guest. For example, an mmap() that affects the region will be made visible immediately. Another example is madvise(MADV_DROP). +Note: On arm64, a write generated by the page-table walker (to update +the Access and Dirty flags, for example) never results in a +KVM_EXIT_MMIO exit when the slot has the KVM_MEM_READONLY flag. This +is because KVM cannot provide the data that would be written by the +page-table walker, making it impossible to emulate the access. +Instead, an abort (data abort if the cause of the page-table update +was a load or a store, instruction abort if it was an instruction +fetch) is injected in the guest. 4.36 KVM_SET_TSS_ADDR --------------------- @@ -8310,6 +8318,20 @@ CPU[EAX=1]:ECX[24] (TSC_DEADLINE) is not reported by ``KVM_GET_SUPPORTED_CPUID`` It can be enabled if ``KVM_CAP_TSC_DEADLINE_TIMER`` is present and the kernel has enabled in-kernel emulation of the local APIC. +CPU topology +~~~~~~~~~~~~ + +Several CPUID values include topology information for the host CPU: +0x0b and 0x1f for Intel systems, 0x8000001e for AMD systems. Different +versions of KVM return different values for this information and userspace +should not rely on it. Currently they return all zeroes. + +If userspace wishes to set up a guest topology, it should be careful that +the values of these three leaves differ for each CPU. In particular, +the APIC ID is found in EDX for all subleaves of 0x0b and 0x1f, and in EAX +for 0x8000001e; the latter also encodes the core id and node id in bits +7:0 of EBX and ECX respectively. + Obsolete ioctls and capabilities ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ diff --git a/Documentation/virt/kvm/locking.rst b/Documentation/virt/kvm/locking.rst index a3ca76f9be751091e1f0a437a1e2693d9ee30e06..a0146793d1972daabb5bb424ae07484c0c5188e7 100644 --- a/Documentation/virt/kvm/locking.rst +++ b/Documentation/virt/kvm/locking.rst @@ -24,21 +24,22 @@ The acquisition orders for mutexes are as follows: For SRCU: -- ``synchronize_srcu(&kvm->srcu)`` is called _inside_ - the kvm->slots_lock critical section, therefore kvm->slots_lock - cannot be taken inside a kvm->srcu read-side critical section. - Instead, kvm->slots_arch_lock is released before the call - to ``synchronize_srcu()`` and _can_ be taken inside a - kvm->srcu read-side critical section. - -- kvm->lock is taken inside kvm->srcu, therefore - ``synchronize_srcu(&kvm->srcu)`` cannot be called inside - a kvm->lock critical section. If you cannot delay the - call until after kvm->lock is released, use ``call_srcu``. +- ``synchronize_srcu(&kvm->srcu)`` is called inside critical sections + for kvm->lock, vcpu->mutex and kvm->slots_lock. These locks _cannot_ + be taken inside a kvm->srcu read-side critical section; that is, the + following is broken:: + + srcu_read_lock(&kvm->srcu); + mutex_lock(&kvm->slots_lock); + +- kvm->slots_arch_lock instead is released before the call to + ``synchronize_srcu()``. It _can_ therefore be taken inside a + kvm->srcu read-side critical section, for example while processing + a vmexit. On x86: -- vcpu->mutex is taken outside kvm->arch.hyperv.hv_lock +- vcpu->mutex is taken outside kvm->arch.hyperv.hv_lock and kvm->arch.xen.xen_lock - kvm->arch.mmu_lock is an rwlock. kvm->arch.tdp_mmu_pages_lock and kvm->arch.mmu_unsync_pages_lock are taken inside kvm->arch.mmu_lock, and diff --git a/Documentation/x86/amd-memory-encryption.rst b/Documentation/x86/amd-memory-encryption.rst index a1940ebe7be50cb17260b240a3ce78ec4f6a2242..934310ce7258298008e4c6ba5be611105993e1c1 100644 --- a/Documentation/x86/amd-memory-encryption.rst +++ b/Documentation/x86/amd-memory-encryption.rst @@ -95,3 +95,39 @@ by supplying mem_encrypt=on on the kernel command line. However, if BIOS does not enable SME, then Linux will not be able to activate memory encryption, even if configured to do so by default or the mem_encrypt=on command line parameter is specified. + +Secure Nested Paging (SNP) +========================== + +SEV-SNP introduces new features (SEV_FEATURES[1:63]) which can be enabled +by the hypervisor for security enhancements. Some of these features need +guest side implementation to function correctly. The below table lists the +expected guest behavior with various possible scenarios of guest/hypervisor +SNP feature support. + ++-----------------+---------------+---------------+------------------+ +| Feature Enabled | Guest needs | Guest has | Guest boot | +| by the HV | implementation| implementation| behaviour | ++=================+===============+===============+==================+ +| No | No | No | Boot | +| | | | | ++-----------------+---------------+---------------+------------------+ +| No | Yes | No | Boot | +| | | | | ++-----------------+---------------+---------------+------------------+ +| No | Yes | Yes | Boot | +| | | | | ++-----------------+---------------+---------------+------------------+ +| Yes | No | No | Boot with | +| | | | feature enabled | ++-----------------+---------------+---------------+------------------+ +| Yes | Yes | No | Graceful boot | +| | | | failure | ++-----------------+---------------+---------------+------------------+ +| Yes | Yes | Yes | Boot with | +| | | | feature enabled | ++-----------------+---------------+---------------+------------------+ + +More details in AMD64 APM[1] Vol 2: 15.34.10 SEV_STATUS MSR + +[1] https://www.amd.com/system/files/TechDocs/40332.pdf diff --git a/MAINTAINERS b/MAINTAINERS index 2758e7a64fa5c6eed8b9405abbd86806bc3d6fad..97d814a19475e294942448cc30b54ec57c78eea4 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -383,7 +383,7 @@ ACPI COMPONENT ARCHITECTURE (ACPICA) M: Robert Moore M: "Rafael J. Wysocki" L: linux-acpi@vger.kernel.org -L: devel@acpica.org +L: acpica-devel@lists.linuxfoundation.org S: Supported W: https://acpica.org/ W: https://github.com/acpica/acpica/ @@ -1104,7 +1104,6 @@ S: Supported F: arch/arm64/boot/dts/amd/ AMD XGBE DRIVER -M: Tom Lendacky M: "Shyam Sundar S K" L: netdev@vger.kernel.org S: Supported @@ -6893,6 +6892,7 @@ C: irc://irc.oftc.net/dri-devel T: git https://git.kernel.org/pub/scm/linux/kernel/git/ogabbay/accel.git F: Documentation/accel/ F: drivers/accel/ +F: include/drm/drm_accel.h DRM ACCEL DRIVERS FOR INTEL VPU M: Jacek Lawrynowicz @@ -6936,7 +6936,7 @@ F: drivers/gpu/drm/atmel-hlcdc/ DRM DRIVERS FOR BRIDGE CHIPS M: Andrzej Hajda M: Neil Armstrong -M: Robert Foss +M: Robert Foss R: Laurent Pinchart R: Jonas Karlman R: Jernej Skrabec @@ -7039,7 +7039,7 @@ M: Thierry Reding L: dri-devel@lists.freedesktop.org L: linux-tegra@vger.kernel.org S: Supported -T: git git://anongit.freedesktop.org/tegra/linux.git +T: git https://gitlab.freedesktop.org/drm/tegra.git F: Documentation/devicetree/bindings/display/tegra/nvidia,tegra20-host1x.yaml F: Documentation/devicetree/bindings/gpu/host1x/ F: drivers/gpu/drm/tegra/ @@ -7605,7 +7605,6 @@ S: Maintained F: drivers/firmware/efi/test/ EFI VARIABLE FILESYSTEM -M: Matthew Garrett M: Jeremy Kerr M: Ard Biesheuvel L: linux-efi@vger.kernel.org @@ -7884,7 +7883,11 @@ F: include/linux/extcon/ EXTRA BOOT CONFIG M: Masami Hiramatsu +L: linux-kernel@vger.kernel.org +L: linux-trace-kernel@vger.kernel.org +Q: https://patchwork.kernel.org/project/linux-trace-kernel/list/ S: Maintained +T: git git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace.git F: Documentation/admin-guide/bootconfig.rst F: fs/proc/bootconfig.c F: include/linux/bootconfig.h @@ -8457,16 +8460,16 @@ F: fs/fscache/ F: include/linux/fscache*.h FSCRYPT: FILE SYSTEM LEVEL ENCRYPTION SUPPORT +M: Eric Biggers M: Theodore Y. Ts'o M: Jaegeuk Kim -M: Eric Biggers L: linux-fscrypt@vger.kernel.org S: Supported Q: https://patchwork.kernel.org/project/linux-fscrypt/list/ -T: git git://git.kernel.org/pub/scm/fs/fscrypt/fscrypt.git +T: git https://git.kernel.org/pub/scm/fs/fscrypt/linux.git F: Documentation/filesystems/fscrypt.rst F: fs/crypto/ -F: include/linux/fscrypt*.h +F: include/linux/fscrypt.h F: include/uapi/linux/fscrypt.h FSI SUBSYSTEM @@ -8509,10 +8512,10 @@ F: include/linux/fsnotify*.h FSVERITY: READ-ONLY FILE-BASED AUTHENTICITY PROTECTION M: Eric Biggers M: Theodore Y. Ts'o -L: linux-fscrypt@vger.kernel.org +L: fsverity@lists.linux.dev S: Supported -Q: https://patchwork.kernel.org/project/linux-fscrypt/list/ -T: git git://git.kernel.org/pub/scm/fs/fscrypt/fscrypt.git fsverity +Q: https://patchwork.kernel.org/project/fsverity/list/ +T: git https://git.kernel.org/pub/scm/fs/fsverity/linux.git F: Documentation/filesystems/fsverity.rst F: fs/verity/ F: include/linux/fsverity.h @@ -8560,6 +8563,7 @@ F: kernel/trace/fgraph.c F: arch/*/*/*/*ftrace* F: arch/*/*/*ftrace* F: include/*/ftrace.h +F: samples/ftrace FUNGIBLE ETHERNET DRIVERS M: Dimitris Michailidis @@ -9032,13 +9036,15 @@ F: block/partitions/efi.* HABANALABS PCI DRIVER M: Oded Gabbay +L: dri-devel@lists.freedesktop.org S: Supported +C: irc://irc.oftc.net/dri-devel T: git https://git.kernel.org/pub/scm/linux/kernel/git/ogabbay/linux.git F: Documentation/ABI/testing/debugfs-driver-habanalabs F: Documentation/ABI/testing/sysfs-driver-habanalabs -F: drivers/misc/habanalabs/ +F: drivers/accel/habanalabs/ F: include/trace/events/habanalabs.h -F: include/uapi/misc/habanalabs.h +F: include/uapi/drm/habanalabs_accel.h HACKRF MEDIA DRIVER M: Antti Palosaari @@ -9288,7 +9294,7 @@ F: net/dsa/tag_hellcreek.c HISILICON DMA DRIVER M: Zhou Wang -M: Jie Hai +M: Jie Hai L: dmaengine@vger.kernel.org S: Maintained F: drivers/dma/hisi_dma.c @@ -11345,9 +11351,9 @@ F: virt/kvm/* KERNEL VIRTUAL MACHINE FOR ARM64 (KVM/arm64) M: Marc Zyngier R: James Morse -R: Alexandru Elisei R: Suzuki K Poulose R: Oliver Upton +R: Zenghui Yu L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers) L: kvmarm@lists.linux.dev L: kvmarm@lists.cs.columbia.edu (deprecated, moderated for non-subscribers) @@ -13609,7 +13615,7 @@ F: arch/microblaze/ MICROCHIP AT91 DMA DRIVERS M: Ludovic Desroches -M: Tudor Ambarus +M: Tudor Ambarus L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers) L: dmaengine@vger.kernel.org S: Supported @@ -13654,7 +13660,7 @@ F: Documentation/devicetree/bindings/media/microchip,csi2dc.yaml F: drivers/media/platform/microchip/microchip-csi2dc.c MICROCHIP ECC DRIVER -M: Tudor Ambarus +M: Tudor Ambarus L: linux-crypto@vger.kernel.org S: Maintained F: drivers/crypto/atmel-ecc.* @@ -13751,7 +13757,7 @@ S: Maintained F: drivers/mmc/host/atmel-mci.c MICROCHIP NAND DRIVER -M: Tudor Ambarus +M: Tudor Ambarus L: linux-mtd@lists.infradead.org S: Supported F: Documentation/devicetree/bindings/mtd/atmel-nand.txt @@ -13803,7 +13809,7 @@ S: Supported F: drivers/power/reset/at91-sama5d2_shdwc.c MICROCHIP SPI DRIVER -M: Tudor Ambarus +M: Tudor Ambarus S: Supported F: drivers/spi/spi-atmel.* @@ -14623,7 +14629,6 @@ F: net/netfilter/xt_SECMARK.c F: net/netlabel/ NETWORKING [MPTCP] -M: Mat Martineau M: Matthieu Baerts L: netdev@vger.kernel.org L: mptcp@lists.linux.dev @@ -14908,7 +14913,8 @@ T: git://git.infradead.org/nvme.git F: Documentation/nvme/ F: drivers/nvme/host/ F: drivers/nvme/common/ -F: include/linux/nvme* +F: include/linux/nvme.h +F: include/linux/nvme-*.h F: include/uapi/linux/nvme_ioctl.h NVM EXPRESS FABRICS AUTHENTICATION @@ -15738,6 +15744,12 @@ S: Maintained W: https://wireless.wiki.kernel.org/en/users/Drivers/p54 F: drivers/net/wireless/intersil/p54/ +PACKET SOCKETS +M: Willem de Bruijn +S: Maintained +F: include/uapi/linux/if_packet.h +F: net/packet/af_packet.c + PACKING M: Vladimir Oltean L: netdev@vger.kernel.org @@ -16599,6 +16611,13 @@ S: Supported F: Documentation/devicetree/bindings/input/pine64,pinephone-keyboard.yaml F: drivers/input/keyboard/pinephone-keyboard.c +PKTCDVD DRIVER +M: linux-block@vger.kernel.org +S: Orphan +F: drivers/block/pktcdvd.c +F: include/linux/pktcdvd.h +F: include/uapi/linux/pktcdvd.h + PLANTOWER PMS7003 AIR POLLUTION SENSOR DRIVER M: Tomasz Duszynski S: Maintained @@ -17219,7 +17238,7 @@ F: Documentation/devicetree/bindings/net/qcom,bam-dmux.yaml F: drivers/net/wwan/qcom_bam_dmux.c QUALCOMM CAMERA SUBSYSTEM DRIVER -M: Robert Foss +M: Robert Foss M: Todor Tomov L: linux-media@vger.kernel.org S: Maintained @@ -17299,7 +17318,7 @@ F: drivers/dma/qcom/hidma* QUALCOMM I2C CCI DRIVER M: Loic Poulain -M: Robert Foss +M: Robert Foss L: linux-i2c@vger.kernel.org L: linux-arm-msm@vger.kernel.org S: Maintained @@ -17938,6 +17957,7 @@ M: Albert Ou L: linux-riscv@lists.infradead.org S: Supported Q: https://patchwork.kernel.org/project/linux-riscv/list/ +C: irc://irc.libera.chat/riscv P: Documentation/riscv/patch-acceptance.rst T: git git://git.kernel.org/pub/scm/linux/kernel/git/riscv/linux.git F: arch/riscv/ @@ -19307,6 +19327,13 @@ L: alsa-devel@alsa-project.org (moderated for non-subscribers) S: Orphan F: sound/soc/uniphier/ +SOCKET TIMESTAMPING +M: Willem de Bruijn +S: Maintained +F: Documentation/networking/timestamping.rst +F: include/uapi/linux/net_tstamp.h +F: tools/testing/selftests/net/so_txtime.c + SOEKRIS NET48XX LED SUPPORT M: Chris Boot S: Maintained @@ -19654,7 +19681,7 @@ F: drivers/clk/spear/ F: drivers/pinctrl/spear/ SPI NOR SUBSYSTEM -M: Tudor Ambarus +M: Tudor Ambarus M: Pratyush Yadav R: Michael Walle L: linux-mtd@lists.infradead.org @@ -21727,6 +21754,13 @@ T: git git://linuxtv.org/media_tree.git F: Documentation/admin-guide/media/zr364xx* F: drivers/staging/media/deprecated/zr364xx/ +USER DATAGRAM PROTOCOL (UDP) +M: Willem de Bruijn +S: Maintained +F: include/linux/udp.h +F: net/ipv4/udp.c +F: net/ipv6/udp.c + USER-MODE LINUX (UML) M: Richard Weinberger M: Anton Ivanov @@ -22235,7 +22269,9 @@ F: drivers/scsi/vmw_pvscsi.c F: drivers/scsi/vmw_pvscsi.h VMWARE VIRTUAL PTP CLOCK DRIVER -M: Vivek Thampi +M: Srivatsa S. Bhat (VMware) +M: Deep Shah +R: Alexey Makhalov R: VMware PV-Drivers Reviewers L: netdev@vger.kernel.org S: Supported diff --git a/Makefile b/Makefile index c05b4fb7121e5fecefa3865af5eefd26a356b49b..749bb455546e43b87b18e01eac82fb4e58b504b5 100644 --- a/Makefile +++ b/Makefile @@ -2,7 +2,7 @@ VERSION = 6 PATCHLEVEL = 2 SUBLEVEL = 0 -EXTRAVERSION = -rc2 +EXTRAVERSION = -rc6 NAME = Hurr durr I'ma ninja sloth # *DOCUMENTATION* @@ -549,7 +549,7 @@ LDFLAGS_MODULE = CFLAGS_KERNEL = RUSTFLAGS_KERNEL = AFLAGS_KERNEL = -export LDFLAGS_vmlinux = +LDFLAGS_vmlinux = # Use USERINCLUDE when you must reference the UAPI directories only. USERINCLUDE := \ @@ -1248,6 +1248,18 @@ vmlinux.o modules.builtin.modinfo modules.builtin: vmlinux_o @: PHONY += vmlinux +# LDFLAGS_vmlinux in the top Makefile defines linker flags for the top vmlinux, +# not for decompressors. LDFLAGS_vmlinux in arch/*/boot/compressed/Makefile is +# unrelated; the decompressors just happen to have the same base name, +# arch/*/boot/compressed/vmlinux. +# Export LDFLAGS_vmlinux only to scripts/Makefile.vmlinux. +# +# _LDFLAGS_vmlinux is a workaround for the 'private export' bug: +# https://savannah.gnu.org/bugs/?61463 +# For Make > 4.4, the following simple code will work: +# vmlinux: private export LDFLAGS_vmlinux := $(LDFLAGS_vmlinux) +vmlinux: private _LDFLAGS_vmlinux := $(LDFLAGS_vmlinux) +vmlinux: export LDFLAGS_vmlinux = $(_LDFLAGS_vmlinux) vmlinux: vmlinux.o $(KBUILD_LDS) modpost $(Q)$(MAKE) -f $(srctree)/scripts/Makefile.vmlinux @@ -1533,6 +1545,7 @@ endif # *.ko are usually independent of vmlinux, but CONFIG_DEBUG_INFOBTF_MODULES # is an exception. ifdef CONFIG_DEBUG_INFO_BTF_MODULES +KBUILD_BUILTIN := 1 modules: vmlinux endif @@ -1986,7 +1999,7 @@ $(single-no-ko): $(build-dir) # Remove MODORDER when done because it is not the real one. PHONY += single_modules single_modules: $(single-no-ko) modules_prepare - $(Q){ $(foreach m, $(single-ko), echo $(extmod_prefix)$m;) } > $(MODORDER) + $(Q){ $(foreach m, $(single-ko), echo $(extmod_prefix)$(m:%.ko=%.o);) } > $(MODORDER) $(Q)$(MAKE) -f $(srctree)/scripts/Makefile.modpost ifneq ($(KBUILD_MODPOST_NOFINAL),1) $(Q)$(MAKE) -f $(srctree)/scripts/Makefile.modfinal diff --git a/arch/arm/Makefile b/arch/arm/Makefile index 4067f5169144246b22e96e734d9af369a32b1e45..955b0362cdfb7a4736a901a7cc63a8c7a289f95f 100644 --- a/arch/arm/Makefile +++ b/arch/arm/Makefile @@ -132,7 +132,7 @@ AFLAGS_NOWARN :=$(call as-option,-Wa$(comma)-mno-warn-deprecated,-Wa$(comma)-W) ifeq ($(CONFIG_THUMB2_KERNEL),y) CFLAGS_ISA :=-Wa,-mimplicit-it=always $(AFLAGS_NOWARN) -AFLAGS_ISA :=$(CFLAGS_ISA) -Wa$(comma)-mthumb -D__thumb2__=2 +AFLAGS_ISA :=$(CFLAGS_ISA) -Wa$(comma)-mthumb CFLAGS_ISA +=-mthumb else CFLAGS_ISA :=$(call cc-option,-marm,) $(AFLAGS_NOWARN) diff --git a/arch/arm/boot/dts/armada-38x.dtsi b/arch/arm/boot/dts/armada-38x.dtsi index 12933eff419ff877366f804e0488b666a5121230..446861b6b17b2ae628a42a4a32dca94a0b5b1f21 100644 --- a/arch/arm/boot/dts/armada-38x.dtsi +++ b/arch/arm/boot/dts/armada-38x.dtsi @@ -304,7 +304,7 @@ }; gpio0: gpio@18100 { - compatible = "marvell,armadaxp-gpio", + compatible = "marvell,armada-370-gpio", "marvell,orion-gpio"; reg = <0x18100 0x40>, <0x181c0 0x08>; reg-names = "gpio", "pwm"; @@ -323,7 +323,7 @@ }; gpio1: gpio@18140 { - compatible = "marvell,armadaxp-gpio", + compatible = "marvell,armada-370-gpio", "marvell,orion-gpio"; reg = <0x18140 0x40>, <0x181c8 0x08>; reg-names = "gpio", "pwm"; diff --git a/arch/arm/boot/dts/armada-39x.dtsi b/arch/arm/boot/dts/armada-39x.dtsi index 1e05208d9f3415f3ee5cd72084a69bf4460d9f8c..9d1cac49c022f07c315efabc57c5f08158c2833a 100644 --- a/arch/arm/boot/dts/armada-39x.dtsi +++ b/arch/arm/boot/dts/armada-39x.dtsi @@ -213,7 +213,7 @@ }; gpio0: gpio@18100 { - compatible = "marvell,armadaxp-gpio", "marvell,orion-gpio"; + compatible = "marvell,orion-gpio"; reg = <0x18100 0x40>; ngpios = <32>; gpio-controller; @@ -227,7 +227,7 @@ }; gpio1: gpio@18140 { - compatible = "marvell,armadaxp-gpio", "marvell,orion-gpio"; + compatible = "marvell,orion-gpio"; reg = <0x18140 0x40>; ngpios = <28>; gpio-controller; diff --git a/arch/arm/boot/dts/imx53-ppd.dts b/arch/arm/boot/dts/imx53-ppd.dts index 37d0cffea99c56ec7e47bd817fe99eb366fc8375..70c4a4852256c4ef2321e25b878eb3b01e2780df 100644 --- a/arch/arm/boot/dts/imx53-ppd.dts +++ b/arch/arm/boot/dts/imx53-ppd.dts @@ -488,7 +488,7 @@ scl-gpios = <&gpio3 21 GPIO_ACTIVE_HIGH>; status = "okay"; - i2c-switch@70 { + i2c-mux@70 { compatible = "nxp,pca9547"; #address-cells = <1>; #size-cells = <0>; diff --git a/arch/arm/boot/dts/imx6qdl-gw560x.dtsi b/arch/arm/boot/dts/imx6qdl-gw560x.dtsi index 4bc4371e6bae56e02ee910b98f0f03656a3b4441..4b81a975c979ded3024fdb87d984a29490ebdc96 100644 --- a/arch/arm/boot/dts/imx6qdl-gw560x.dtsi +++ b/arch/arm/boot/dts/imx6qdl-gw560x.dtsi @@ -632,7 +632,6 @@ &uart1 { pinctrl-names = "default"; pinctrl-0 = <&pinctrl_uart1>; - uart-has-rtscts; rts-gpios = <&gpio7 1 GPIO_ACTIVE_HIGH>; status = "okay"; }; diff --git a/arch/arm/boot/dts/imx6ul-pico-dwarf.dts b/arch/arm/boot/dts/imx6ul-pico-dwarf.dts index 162dc259edc8c5cb0c14b5c2b7dd9b948c861b61..5a74c7f68eb62dbf03b1af91a3b83241b5137717 100644 --- a/arch/arm/boot/dts/imx6ul-pico-dwarf.dts +++ b/arch/arm/boot/dts/imx6ul-pico-dwarf.dts @@ -32,7 +32,7 @@ }; &i2c2 { - clock_frequency = <100000>; + clock-frequency = <100000>; pinctrl-names = "default"; pinctrl-0 = <&pinctrl_i2c2>; status = "okay"; diff --git a/arch/arm/boot/dts/imx7d-pico-dwarf.dts b/arch/arm/boot/dts/imx7d-pico-dwarf.dts index 5162fe227d1ea3eedd035577970f01ce70e69f54..fdc10563f1473eed1f9cd7aaec7d7c59820db1f1 100644 --- a/arch/arm/boot/dts/imx7d-pico-dwarf.dts +++ b/arch/arm/boot/dts/imx7d-pico-dwarf.dts @@ -32,7 +32,7 @@ }; &i2c1 { - clock_frequency = <100000>; + clock-frequency = <100000>; pinctrl-names = "default"; pinctrl-0 = <&pinctrl_i2c1>; status = "okay"; @@ -52,7 +52,7 @@ }; &i2c4 { - clock_frequency = <100000>; + clock-frequency = <100000>; pinctrl-names = "default"; pinctrl-0 = <&pinctrl_i2c1>; status = "okay"; diff --git a/arch/arm/boot/dts/imx7d-pico-nymph.dts b/arch/arm/boot/dts/imx7d-pico-nymph.dts index 104a85254adbbd8688f5294205e08b6250972bca..5afb1674e012518add059d5936fd438977c5a7e8 100644 --- a/arch/arm/boot/dts/imx7d-pico-nymph.dts +++ b/arch/arm/boot/dts/imx7d-pico-nymph.dts @@ -43,7 +43,7 @@ }; &i2c1 { - clock_frequency = <100000>; + clock-frequency = <100000>; pinctrl-names = "default"; pinctrl-0 = <&pinctrl_i2c1>; status = "okay"; @@ -64,7 +64,7 @@ }; &i2c2 { - clock_frequency = <100000>; + clock-frequency = <100000>; pinctrl-names = "default"; pinctrl-0 = <&pinctrl_i2c2>; status = "okay"; diff --git a/arch/arm/boot/dts/qcom-apq8084-ifc6540.dts b/arch/arm/boot/dts/qcom-apq8084-ifc6540.dts index 44cd72f1b1be4795e391d63c5088d8f00b54fa92..116e59a3b76d01e54306ced831dfbc2019b05754 100644 --- a/arch/arm/boot/dts/qcom-apq8084-ifc6540.dts +++ b/arch/arm/boot/dts/qcom-apq8084-ifc6540.dts @@ -19,16 +19,16 @@ serial@f995e000 { status = "okay"; }; + }; +}; - sdhci@f9824900 { - bus-width = <8>; - non-removable; - status = "okay"; - }; +&sdhc_1 { + bus-width = <8>; + non-removable; + status = "okay"; +}; - sdhci@f98a4900 { - cd-gpios = <&tlmm 122 GPIO_ACTIVE_LOW>; - bus-width = <4>; - }; - }; +&sdhc_2 { + cd-gpios = <&tlmm 122 GPIO_ACTIVE_LOW>; + bus-width = <4>; }; diff --git a/arch/arm/boot/dts/qcom-apq8084.dtsi b/arch/arm/boot/dts/qcom-apq8084.dtsi index fe30abfff90a77d10f51ca48105da0bbb7ca5944..4b0d2b4f4b6a9017352eb0b847126c3b723151d2 100644 --- a/arch/arm/boot/dts/qcom-apq8084.dtsi +++ b/arch/arm/boot/dts/qcom-apq8084.dtsi @@ -421,7 +421,7 @@ status = "disabled"; }; - mmc@f9824900 { + sdhc_1: mmc@f9824900 { compatible = "qcom,apq8084-sdhci", "qcom,sdhci-msm-v4"; reg = <0xf9824900 0x11c>, <0xf9824000 0x800>; reg-names = "hc", "core"; @@ -434,7 +434,7 @@ status = "disabled"; }; - mmc@f98a4900 { + sdhc_2: mmc@f98a4900 { compatible = "qcom,apq8084-sdhci", "qcom,sdhci-msm-v4"; reg = <0xf98a4900 0x11c>, <0xf98a4000 0x800>; reg-names = "hc", "core"; diff --git a/arch/arm/boot/dts/sam9x60.dtsi b/arch/arm/boot/dts/sam9x60.dtsi index 8f5477e307dd49e444d66289663f03048b59b5e2..37a5d96aaf6421a854b96a7d04efc1b54bd11254 100644 --- a/arch/arm/boot/dts/sam9x60.dtsi +++ b/arch/arm/boot/dts/sam9x60.dtsi @@ -564,7 +564,7 @@ mpddrc: mpddrc@ffffe800 { compatible = "microchip,sam9x60-ddramc", "atmel,sama5d3-ddramc"; reg = <0xffffe800 0x200>; - clocks = <&pmc PMC_TYPE_SYSTEM 2>, <&pmc PMC_TYPE_CORE PMC_MCK>; + clocks = <&pmc PMC_TYPE_SYSTEM 2>, <&pmc PMC_TYPE_PERIPHERAL 49>; clock-names = "ddrck", "mpddr"; }; diff --git a/arch/arm/boot/dts/stm32mp151a-prtt1l.dtsi b/arch/arm/boot/dts/stm32mp151a-prtt1l.dtsi index d865ab5d866b93bf16eac00e33080cecf40041cc..dd23de85100c483386d35307fc93f1e03c987982 100644 --- a/arch/arm/boot/dts/stm32mp151a-prtt1l.dtsi +++ b/arch/arm/boot/dts/stm32mp151a-prtt1l.dtsi @@ -101,8 +101,12 @@ &qspi { pinctrl-names = "default", "sleep"; - pinctrl-0 = <&qspi_clk_pins_a &qspi_bk1_pins_a>; - pinctrl-1 = <&qspi_clk_sleep_pins_a &qspi_bk1_sleep_pins_a>; + pinctrl-0 = <&qspi_clk_pins_a + &qspi_bk1_pins_a + &qspi_cs1_pins_a>; + pinctrl-1 = <&qspi_clk_sleep_pins_a + &qspi_bk1_sleep_pins_a + &qspi_cs1_sleep_pins_a>; reg = <0x58003000 0x1000>, <0x70000000 0x4000000>; #address-cells = <1>; #size-cells = <0>; diff --git a/arch/arm/boot/dts/stm32mp157c-emstamp-argon.dtsi b/arch/arm/boot/dts/stm32mp157c-emstamp-argon.dtsi index aef02e6421a36ddf93a975b0798fb92c4326b5d1..7d11c50b9e4087afa226adc052449c3a38b3dd6e 100644 --- a/arch/arm/boot/dts/stm32mp157c-emstamp-argon.dtsi +++ b/arch/arm/boot/dts/stm32mp157c-emstamp-argon.dtsi @@ -391,8 +391,12 @@ &qspi { pinctrl-names = "default", "sleep"; - pinctrl-0 = <&qspi_clk_pins_a &qspi_bk1_pins_a>; - pinctrl-1 = <&qspi_clk_sleep_pins_a &qspi_bk1_sleep_pins_a>; + pinctrl-0 = <&qspi_clk_pins_a + &qspi_bk1_pins_a + &qspi_cs1_pins_a>; + pinctrl-1 = <&qspi_clk_sleep_pins_a + &qspi_bk1_sleep_pins_a + &qspi_cs1_sleep_pins_a>; reg = <0x58003000 0x1000>, <0x70000000 0x4000000>; #address-cells = <1>; #size-cells = <0>; diff --git a/arch/arm/boot/dts/stm32mp15xx-dhcom-som.dtsi b/arch/arm/boot/dts/stm32mp15xx-dhcom-som.dtsi index 002f221f1694c79c70aba0d9d9e120c006de0a64..c06edd2eacb0cd3f35850f0247bda1dedab4f718 100644 --- a/arch/arm/boot/dts/stm32mp15xx-dhcom-som.dtsi +++ b/arch/arm/boot/dts/stm32mp15xx-dhcom-som.dtsi @@ -428,8 +428,12 @@ &qspi { pinctrl-names = "default", "sleep"; - pinctrl-0 = <&qspi_clk_pins_a &qspi_bk1_pins_a>; - pinctrl-1 = <&qspi_clk_sleep_pins_a &qspi_bk1_sleep_pins_a>; + pinctrl-0 = <&qspi_clk_pins_a + &qspi_bk1_pins_a + &qspi_cs1_pins_a>; + pinctrl-1 = <&qspi_clk_sleep_pins_a + &qspi_bk1_sleep_pins_a + &qspi_cs1_sleep_pins_a>; reg = <0x58003000 0x1000>, <0x70000000 0x4000000>; #address-cells = <1>; #size-cells = <0>; diff --git a/arch/arm/boot/dts/stm32mp15xx-dhcor-som.dtsi b/arch/arm/boot/dts/stm32mp15xx-dhcor-som.dtsi index 134a798ad3f23f628e8673fadaf58b777cf64e41..bb40fb46da81d559084f680c11c5c072efa83073 100644 --- a/arch/arm/boot/dts/stm32mp15xx-dhcor-som.dtsi +++ b/arch/arm/boot/dts/stm32mp15xx-dhcor-som.dtsi @@ -247,8 +247,12 @@ &qspi { pinctrl-names = "default", "sleep"; - pinctrl-0 = <&qspi_clk_pins_a &qspi_bk1_pins_a>; - pinctrl-1 = <&qspi_clk_sleep_pins_a &qspi_bk1_sleep_pins_a>; + pinctrl-0 = <&qspi_clk_pins_a + &qspi_bk1_pins_a + &qspi_cs1_pins_a>; + pinctrl-1 = <&qspi_clk_sleep_pins_a + &qspi_bk1_sleep_pins_a + &qspi_cs1_sleep_pins_a>; reg = <0x58003000 0x1000>, <0x70000000 0x200000>; #address-cells = <1>; #size-cells = <0>; diff --git a/arch/arm/boot/dts/vf610-zii-dev-rev-b.dts b/arch/arm/boot/dts/vf610-zii-dev-rev-b.dts index 42ed4a04a12e250b3e4d3421d221658397560ead..6280c5e86a124e2f1547a2a7b74f7f4628db878d 100644 --- a/arch/arm/boot/dts/vf610-zii-dev-rev-b.dts +++ b/arch/arm/boot/dts/vf610-zii-dev-rev-b.dts @@ -345,7 +345,7 @@ }; &i2c2 { - tca9548@70 { + i2c-mux@70 { compatible = "nxp,pca9548"; pinctrl-0 = <&pinctrl_i2c_mux_reset>; pinctrl-names = "default"; diff --git a/arch/arm/boot/dts/vf610-zii-dev-rev-c.dts b/arch/arm/boot/dts/vf610-zii-dev-rev-c.dts index f892977da9e4e0311fd1e3dda5308a8b2811a2f6..c00d39562a10bb9487785b8f77f5fb181fc0cb90 100644 --- a/arch/arm/boot/dts/vf610-zii-dev-rev-c.dts +++ b/arch/arm/boot/dts/vf610-zii-dev-rev-c.dts @@ -340,7 +340,7 @@ }; &i2c2 { - tca9548@70 { + i2c-mux@70 { compatible = "nxp,pca9548"; pinctrl-0 = <&pinctrl_i2c_mux_reset>; pinctrl-names = "default"; diff --git a/arch/arm/crypto/Makefile b/arch/arm/crypto/Makefile index 971e74546fb1bf31d6c25f3c74a326b3a51a0e2a..13e62c7c25dca8e103527f776f9fb094b7b350e7 100644 --- a/arch/arm/crypto/Makefile +++ b/arch/arm/crypto/Makefile @@ -53,7 +53,12 @@ $(obj)/%-core.S: $(src)/%-armv4.pl clean-files += poly1305-core.S sha256-core.S sha512-core.S +aflags-thumb2-$(CONFIG_THUMB2_KERNEL) := -U__thumb2__ -D__thumb2__=1 + +AFLAGS_sha256-core.o += $(aflags-thumb2-y) +AFLAGS_sha512-core.o += $(aflags-thumb2-y) + # massage the perlasm code a bit so we only get the NEON routine if we need it poly1305-aflags-$(CONFIG_CPU_V7) := -U__LINUX_ARM_ARCH__ -D__LINUX_ARM_ARCH__=5 poly1305-aflags-$(CONFIG_KERNEL_MODE_NEON) := -U__LINUX_ARM_ARCH__ -D__LINUX_ARM_ARCH__=7 -AFLAGS_poly1305-core.o += $(poly1305-aflags-y) +AFLAGS_poly1305-core.o += $(poly1305-aflags-y) $(aflags-thumb2-y) diff --git a/arch/arm/include/asm/thread_info.h b/arch/arm/include/asm/thread_info.h index aecc403b2880493d6925eed075bf405b77ee03cb..7f092cb55a4171548da8e4df3c72708512df9264 100644 --- a/arch/arm/include/asm/thread_info.h +++ b/arch/arm/include/asm/thread_info.h @@ -128,15 +128,16 @@ extern int vfp_restore_user_hwstate(struct user_vfp *, #define TIF_NEED_RESCHED 1 /* rescheduling necessary */ #define TIF_NOTIFY_RESUME 2 /* callback before returning to user */ #define TIF_UPROBE 3 /* breakpointed or singlestepping */ -#define TIF_SYSCALL_TRACE 4 /* syscall trace active */ -#define TIF_SYSCALL_AUDIT 5 /* syscall auditing active */ -#define TIF_SYSCALL_TRACEPOINT 6 /* syscall tracepoint instrumentation */ -#define TIF_SECCOMP 7 /* seccomp syscall filtering active */ -#define TIF_NOTIFY_SIGNAL 8 /* signal notifications exist */ +#define TIF_NOTIFY_SIGNAL 4 /* signal notifications exist */ #define TIF_USING_IWMMXT 17 #define TIF_MEMDIE 18 /* is terminating due to OOM killer */ -#define TIF_RESTORE_SIGMASK 20 +#define TIF_RESTORE_SIGMASK 19 +#define TIF_SYSCALL_TRACE 20 /* syscall trace active */ +#define TIF_SYSCALL_AUDIT 21 /* syscall auditing active */ +#define TIF_SYSCALL_TRACEPOINT 22 /* syscall tracepoint instrumentation */ +#define TIF_SECCOMP 23 /* seccomp syscall filtering active */ + #define _TIF_SIGPENDING (1 << TIF_SIGPENDING) #define _TIF_NEED_RESCHED (1 << TIF_NEED_RESCHED) diff --git a/arch/arm/mach-footbridge/isa-rtc.c b/arch/arm/mach-footbridge/isa-rtc.c index b8f741a3a37ea02cee2558f951eb9bf297936168..237b828dd2f10a7d99f910edb262e14e90c3e050 100644 --- a/arch/arm/mach-footbridge/isa-rtc.c +++ b/arch/arm/mach-footbridge/isa-rtc.c @@ -20,7 +20,6 @@ #include #include -#include #include #include "common.h" diff --git a/arch/arm/mach-imx/cpu-imx25.c b/arch/arm/mach-imx/cpu-imx25.c index 3e63445cde062e08b68865707689da1f9f09b0c1..cc86977d0a3407aa13a8068a91eaebd436e997df 100644 --- a/arch/arm/mach-imx/cpu-imx25.c +++ b/arch/arm/mach-imx/cpu-imx25.c @@ -23,6 +23,7 @@ static int mx25_read_cpu_rev(void) np = of_find_compatible_node(NULL, NULL, "fsl,imx25-iim"); iim_base = of_iomap(np, 0); + of_node_put(np); BUG_ON(!iim_base); rev = readl(iim_base + MXC_IIMSREV); iounmap(iim_base); diff --git a/arch/arm/mach-imx/cpu-imx27.c b/arch/arm/mach-imx/cpu-imx27.c index bf70e13bbe9ee0ad41155a407006becf8f51d9f2..1d2893908368380499119e4ab30d955c32245053 100644 --- a/arch/arm/mach-imx/cpu-imx27.c +++ b/arch/arm/mach-imx/cpu-imx27.c @@ -28,6 +28,7 @@ static int mx27_read_cpu_rev(void) np = of_find_compatible_node(NULL, NULL, "fsl,imx27-ccm"); ccm_base = of_iomap(np, 0); + of_node_put(np); BUG_ON(!ccm_base); /* * now we have access to the IO registers. As we need diff --git a/arch/arm/mach-imx/cpu-imx31.c b/arch/arm/mach-imx/cpu-imx31.c index b9c24b851d1abd1fd18a7e30199b048335addc85..35c544924e509593759a1f59abca62173bea1c1b 100644 --- a/arch/arm/mach-imx/cpu-imx31.c +++ b/arch/arm/mach-imx/cpu-imx31.c @@ -39,6 +39,7 @@ static int mx31_read_cpu_rev(void) np = of_find_compatible_node(NULL, NULL, "fsl,imx31-iim"); iim_base = of_iomap(np, 0); + of_node_put(np); BUG_ON(!iim_base); /* read SREV register from IIM module */ diff --git a/arch/arm/mach-imx/cpu-imx35.c b/arch/arm/mach-imx/cpu-imx35.c index 80e7d8ab9f1bb81ad4a70fb8805a45bb170ff01f..1fe75b39c2d995d2aeca9ff923f561e0ba3c8bbd 100644 --- a/arch/arm/mach-imx/cpu-imx35.c +++ b/arch/arm/mach-imx/cpu-imx35.c @@ -21,6 +21,7 @@ static int mx35_read_cpu_rev(void) np = of_find_compatible_node(NULL, NULL, "fsl,imx35-iim"); iim_base = of_iomap(np, 0); + of_node_put(np); BUG_ON(!iim_base); rev = imx_readl(iim_base + MXC_IIMSREV); diff --git a/arch/arm/mach-imx/cpu-imx5.c b/arch/arm/mach-imx/cpu-imx5.c index ad56263778f936d2e1f0f9a2e6a548b201da7d1a..a67c89bf155dd3723a7406caac6ed3bf82ad0f98 100644 --- a/arch/arm/mach-imx/cpu-imx5.c +++ b/arch/arm/mach-imx/cpu-imx5.c @@ -28,6 +28,7 @@ static u32 imx5_read_srev_reg(const char *compat) np = of_find_compatible_node(NULL, NULL, compat); iim_base = of_iomap(np, 0); + of_node_put(np); WARN_ON(!iim_base); srev = readl(iim_base + IIM_SREV) & 0xff; diff --git a/arch/arm/mach-omap1/Kconfig b/arch/arm/mach-omap1/Kconfig index 538a960257cc70602f59ee7a07215d7adb838e10..7ec7ada287e050692ee7421a87a1c3b7865cd8c9 100644 --- a/arch/arm/mach-omap1/Kconfig +++ b/arch/arm/mach-omap1/Kconfig @@ -4,6 +4,7 @@ menuconfig ARCH_OMAP1 depends on ARCH_MULTI_V4T || ARCH_MULTI_V5 depends on CPU_LITTLE_ENDIAN depends on ATAGS + select ARCH_OMAP select ARCH_HAS_HOLES_MEMORYMODEL select ARCH_OMAP select CLKSRC_MMIO @@ -45,10 +46,6 @@ config ARCH_OMAP16XX select CPU_ARM926T select OMAP_DM_TIMER -config ARCH_OMAP1_ANY - select ARCH_OMAP - def_bool ARCH_OMAP730 || ARCH_OMAP850 || ARCH_OMAP15XX || ARCH_OMAP16XX - config ARCH_OMAP bool diff --git a/arch/arm/mach-omap1/Makefile b/arch/arm/mach-omap1/Makefile index 506074b86333fd67d0a705378eea2eab89a4b8b3..0615cb0ba580b03b57c78887dcf24e02e37b0e85 100644 --- a/arch/arm/mach-omap1/Makefile +++ b/arch/arm/mach-omap1/Makefile @@ -3,8 +3,6 @@ # Makefile for the linux kernel. # -ifdef CONFIG_ARCH_OMAP1_ANY - # Common support obj-y := io.o id.o sram-init.o sram.o time.o irq.o mux.o flash.o \ serial.o devices.o dma.o omap-dma.o fb.o @@ -59,5 +57,3 @@ obj-$(CONFIG_ARCH_OMAP730) += gpio7xx.o obj-$(CONFIG_ARCH_OMAP850) += gpio7xx.o obj-$(CONFIG_ARCH_OMAP15XX) += gpio15xx.o obj-$(CONFIG_ARCH_OMAP16XX) += gpio16xx.o - -endif diff --git a/arch/arm/mach-omap1/gpio15xx.c b/arch/arm/mach-omap1/gpio15xx.c index c675f11de99dbdf5e30da119bcc53098e18a5f6d..61fa26efd865340312e12b6fc914c85fbcb6df17 100644 --- a/arch/arm/mach-omap1/gpio15xx.c +++ b/arch/arm/mach-omap1/gpio15xx.c @@ -11,6 +11,7 @@ #include #include #include +#include #include "irqs.h" diff --git a/arch/arm/mach-omap1/io.c b/arch/arm/mach-omap1/io.c index d2db9b8aed3fb016b6cbecda12a0ace8c351bf13..0074b011a05a46b34c56ddc0a72e301c9b602aef 100644 --- a/arch/arm/mach-omap1/io.c +++ b/arch/arm/mach-omap1/io.c @@ -22,17 +22,14 @@ * The machine specific code may provide the extra mapping besides the * default mapping provided here. */ -static struct map_desc omap_io_desc[] __initdata = { +#if defined (CONFIG_ARCH_OMAP730) || defined (CONFIG_ARCH_OMAP850) +static struct map_desc omap7xx_io_desc[] __initdata = { { .virtual = OMAP1_IO_VIRT, .pfn = __phys_to_pfn(OMAP1_IO_PHYS), .length = OMAP1_IO_SIZE, .type = MT_DEVICE - } -}; - -#if defined (CONFIG_ARCH_OMAP730) || defined (CONFIG_ARCH_OMAP850) -static struct map_desc omap7xx_io_desc[] __initdata = { + }, { .virtual = OMAP7XX_DSP_BASE, .pfn = __phys_to_pfn(OMAP7XX_DSP_START), @@ -49,6 +46,12 @@ static struct map_desc omap7xx_io_desc[] __initdata = { #ifdef CONFIG_ARCH_OMAP15XX static struct map_desc omap1510_io_desc[] __initdata = { + { + .virtual = OMAP1_IO_VIRT, + .pfn = __phys_to_pfn(OMAP1_IO_PHYS), + .length = OMAP1_IO_SIZE, + .type = MT_DEVICE + }, { .virtual = OMAP1510_DSP_BASE, .pfn = __phys_to_pfn(OMAP1510_DSP_START), @@ -65,6 +68,12 @@ static struct map_desc omap1510_io_desc[] __initdata = { #if defined(CONFIG_ARCH_OMAP16XX) static struct map_desc omap16xx_io_desc[] __initdata = { + { + .virtual = OMAP1_IO_VIRT, + .pfn = __phys_to_pfn(OMAP1_IO_PHYS), + .length = OMAP1_IO_SIZE, + .type = MT_DEVICE + }, { .virtual = OMAP16XX_DSP_BASE, .pfn = __phys_to_pfn(OMAP16XX_DSP_START), @@ -79,18 +88,9 @@ static struct map_desc omap16xx_io_desc[] __initdata = { }; #endif -/* - * Maps common IO regions for omap1 - */ -static void __init omap1_map_common_io(void) -{ - iotable_init(omap_io_desc, ARRAY_SIZE(omap_io_desc)); -} - #if defined (CONFIG_ARCH_OMAP730) || defined (CONFIG_ARCH_OMAP850) void __init omap7xx_map_io(void) { - omap1_map_common_io(); iotable_init(omap7xx_io_desc, ARRAY_SIZE(omap7xx_io_desc)); } #endif @@ -98,7 +98,6 @@ void __init omap7xx_map_io(void) #ifdef CONFIG_ARCH_OMAP15XX void __init omap15xx_map_io(void) { - omap1_map_common_io(); iotable_init(omap1510_io_desc, ARRAY_SIZE(omap1510_io_desc)); } #endif @@ -106,7 +105,6 @@ void __init omap15xx_map_io(void) #if defined(CONFIG_ARCH_OMAP16XX) void __init omap16xx_map_io(void) { - omap1_map_common_io(); iotable_init(omap16xx_io_desc, ARRAY_SIZE(omap16xx_io_desc)); } #endif diff --git a/arch/arm/mach-omap1/mcbsp.c b/arch/arm/mach-omap1/mcbsp.c index 05c25c432449fd8f2a0f609ae644d16aecf3e379..b1632cbe37e6f93d327ad79cbba1c62ce6687d86 100644 --- a/arch/arm/mach-omap1/mcbsp.c +++ b/arch/arm/mach-omap1/mcbsp.c @@ -89,7 +89,6 @@ static struct omap_mcbsp_ops omap1_mcbsp_ops = { #define OMAP1610_MCBSP2_BASE 0xfffb1000 #define OMAP1610_MCBSP3_BASE 0xe1017000 -#if defined(CONFIG_ARCH_OMAP730) || defined(CONFIG_ARCH_OMAP850) struct resource omap7xx_mcbsp_res[][6] = { { { @@ -159,14 +158,7 @@ static struct omap_mcbsp_platform_data omap7xx_mcbsp_pdata[] = { }; #define OMAP7XX_MCBSP_RES_SZ ARRAY_SIZE(omap7xx_mcbsp_res[1]) #define OMAP7XX_MCBSP_COUNT ARRAY_SIZE(omap7xx_mcbsp_res) -#else -#define omap7xx_mcbsp_res_0 NULL -#define omap7xx_mcbsp_pdata NULL -#define OMAP7XX_MCBSP_RES_SZ 0 -#define OMAP7XX_MCBSP_COUNT 0 -#endif -#ifdef CONFIG_ARCH_OMAP15XX struct resource omap15xx_mcbsp_res[][6] = { { { @@ -266,14 +258,7 @@ static struct omap_mcbsp_platform_data omap15xx_mcbsp_pdata[] = { }; #define OMAP15XX_MCBSP_RES_SZ ARRAY_SIZE(omap15xx_mcbsp_res[1]) #define OMAP15XX_MCBSP_COUNT ARRAY_SIZE(omap15xx_mcbsp_res) -#else -#define omap15xx_mcbsp_res_0 NULL -#define omap15xx_mcbsp_pdata NULL -#define OMAP15XX_MCBSP_RES_SZ 0 -#define OMAP15XX_MCBSP_COUNT 0 -#endif -#ifdef CONFIG_ARCH_OMAP16XX struct resource omap16xx_mcbsp_res[][6] = { { { @@ -373,12 +358,6 @@ static struct omap_mcbsp_platform_data omap16xx_mcbsp_pdata[] = { }; #define OMAP16XX_MCBSP_RES_SZ ARRAY_SIZE(omap16xx_mcbsp_res[1]) #define OMAP16XX_MCBSP_COUNT ARRAY_SIZE(omap16xx_mcbsp_res) -#else -#define omap16xx_mcbsp_res_0 NULL -#define omap16xx_mcbsp_pdata NULL -#define OMAP16XX_MCBSP_RES_SZ 0 -#define OMAP16XX_MCBSP_COUNT 0 -#endif static void omap_mcbsp_register_board_cfg(struct resource *res, int res_count, struct omap_mcbsp_platform_data *config, int size) diff --git a/arch/arm/mach-omap1/pm.h b/arch/arm/mach-omap1/pm.h index d9165709c532388316d2a351a2dc9c9f02b6da95..0d1f092821ff8ff7a8c0a57c759db31879b2c9b0 100644 --- a/arch/arm/mach-omap1/pm.h +++ b/arch/arm/mach-omap1/pm.h @@ -106,13 +106,6 @@ #define OMAP7XX_IDLECT3 0xfffece24 #define OMAP7XX_IDLE_LOOP_REQUEST 0x0C00 -#if !defined(CONFIG_ARCH_OMAP730) && \ - !defined(CONFIG_ARCH_OMAP850) && \ - !defined(CONFIG_ARCH_OMAP15XX) && \ - !defined(CONFIG_ARCH_OMAP16XX) -#warning "Power management for this processor not implemented yet" -#endif - #ifndef __ASSEMBLER__ #include diff --git a/arch/arm/mach-pxa/Kconfig b/arch/arm/mach-pxa/Kconfig index b90d98bae68d74b12eac0eba44f9fc49c1df6871..03e25af6f48c97ff4db9b06063836de3b82ce206 100644 --- a/arch/arm/mach-pxa/Kconfig +++ b/arch/arm/mach-pxa/Kconfig @@ -45,6 +45,8 @@ config MACH_PXA27X_DT config MACH_PXA3XX_DT bool "Support PXA3xx platforms from device tree" select CPU_PXA300 + select CPU_PXA310 + select CPU_PXA320 select PINCTRL select POWER_SUPPLY select PXA3xx diff --git a/arch/arm/mm/nommu.c b/arch/arm/mm/nommu.c index c1494a4dee25b535f9153be56050520eb6c3521b..53f2d8774fdb9aaaeb04fe818da08ccfc0bc4803 100644 --- a/arch/arm/mm/nommu.c +++ b/arch/arm/mm/nommu.c @@ -161,7 +161,7 @@ void __init paging_init(const struct machine_desc *mdesc) mpu_setup(); /* allocate the zero page. */ - zero_page = memblock_alloc(PAGE_SIZE, PAGE_SIZE); + zero_page = (void *)memblock_alloc(PAGE_SIZE, PAGE_SIZE); if (!zero_page) panic("%s: Failed to allocate %lu bytes align=0x%lx\n", __func__, PAGE_SIZE, PAGE_SIZE); diff --git a/arch/arm/mm/proc-macros.S b/arch/arm/mm/proc-macros.S index fa6999e24b074a16ca69ac53cedcf1d583f04893..e43f6d716b4b877bff461385a940cff9957a3a6f 100644 --- a/arch/arm/mm/proc-macros.S +++ b/arch/arm/mm/proc-macros.S @@ -6,6 +6,7 @@ * VM_EXEC */ #include +#include #include #ifdef CONFIG_CPU_V7M diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 03934808b2ed0717cbc616dbcc49d962b75f0090..c5ccca26a40870f54d96ed33535beee25bfc153a 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -184,8 +184,6 @@ config ARM64 select HAVE_DEBUG_KMEMLEAK select HAVE_DMA_CONTIGUOUS select HAVE_DYNAMIC_FTRACE - select HAVE_DYNAMIC_FTRACE_WITH_ARGS \ - if $(cc-option,-fpatchable-function-entry=2) select FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY \ if DYNAMIC_FTRACE_WITH_ARGS select HAVE_EFFICIENT_UNALIGNED_ACCESS @@ -972,6 +970,22 @@ config ARM64_ERRATUM_2457168 If unsure, say Y. +config ARM64_ERRATUM_2645198 + bool "Cortex-A715: 2645198: Workaround possible [ESR|FAR]_ELx corruption" + default y + help + This option adds the workaround for ARM Cortex-A715 erratum 2645198. + + If a Cortex-A715 cpu sees a page mapping permissions change from executable + to non-executable, it may corrupt the ESR_ELx and FAR_ELx registers on the + next instruction abort caused by permission fault. + + Only user-space does executable to non-executable permission transition via + mprotect() system call. Workaround the problem by doing a break-before-make + TLB invalidation, for all changes to executable user space mappings. + + If unsure, say Y. + config CAVIUM_ERRATUM_22375 bool "Cavium erratum 22375, 24313" default y diff --git a/arch/arm64/boot/dts/amlogic/meson-sm1-odroid-hc4.dts b/arch/arm64/boot/dts/amlogic/meson-sm1-odroid-hc4.dts index e3486f60645a41726e6c9594986e363e02ded05c..a1f0c38ccaddad76baad2efd7e7865830f737c38 100644 --- a/arch/arm64/boot/dts/amlogic/meson-sm1-odroid-hc4.dts +++ b/arch/arm64/boot/dts/amlogic/meson-sm1-odroid-hc4.dts @@ -131,10 +131,6 @@ }; &usb { - phys = <&usb2_phy1>; - phy-names = "usb2-phy1"; -}; - -&usb2_phy0 { - status = "disabled"; + phys = <&usb2_phy0>, <&usb2_phy1>; + phy-names = "usb2-phy0", "usb2-phy1"; }; diff --git a/arch/arm64/boot/dts/freescale/fsl-ls1012a-qds.dts b/arch/arm64/boot/dts/freescale/fsl-ls1012a-qds.dts index 5a8d85a7d1612d44ce0fcc3ffeb31e731f50a811..bbdf989058ff7795f3ba514d667630f514bc8b53 100644 --- a/arch/arm64/boot/dts/freescale/fsl-ls1012a-qds.dts +++ b/arch/arm64/boot/dts/freescale/fsl-ls1012a-qds.dts @@ -110,7 +110,7 @@ &i2c0 { status = "okay"; - pca9547@77 { + i2c-mux@77 { compatible = "nxp,pca9547"; reg = <0x77>; #address-cells = <1>; diff --git a/arch/arm64/boot/dts/freescale/fsl-ls1043a-qds.dts b/arch/arm64/boot/dts/freescale/fsl-ls1043a-qds.dts index 9b726c2a4842671a5a8cdb0db49e1c5eccdf5c2d..dda27ed7aaf2b15cda135c5c1b3e9a4e8f4cdaff 100644 --- a/arch/arm64/boot/dts/freescale/fsl-ls1043a-qds.dts +++ b/arch/arm64/boot/dts/freescale/fsl-ls1043a-qds.dts @@ -89,7 +89,7 @@ &i2c0 { status = "okay"; - pca9547@77 { + i2c-mux@77 { compatible = "nxp,pca9547"; reg = <0x77>; #address-cells = <1>; diff --git a/arch/arm64/boot/dts/freescale/fsl-ls1046a-qds.dts b/arch/arm64/boot/dts/freescale/fsl-ls1046a-qds.dts index b2fcbba60d3accdf03ef8fe51d7de98e6f0818a8..3b0ed9305f2bd4558d9c51d64ce40b421797997d 100644 --- a/arch/arm64/boot/dts/freescale/fsl-ls1046a-qds.dts +++ b/arch/arm64/boot/dts/freescale/fsl-ls1046a-qds.dts @@ -88,7 +88,7 @@ &i2c0 { status = "okay"; - pca9547@77 { + i2c-mux@77 { compatible = "nxp,pca9547"; reg = <0x77>; #address-cells = <1>; diff --git a/arch/arm64/boot/dts/freescale/fsl-ls1088a-qds.dts b/arch/arm64/boot/dts/freescale/fsl-ls1088a-qds.dts index 41d8b15f25a540ba087e70d67558932768404d81..aa52ff73ff9e027208a2e1df6ef1c5937edd4c02 100644 --- a/arch/arm64/boot/dts/freescale/fsl-ls1088a-qds.dts +++ b/arch/arm64/boot/dts/freescale/fsl-ls1088a-qds.dts @@ -53,7 +53,7 @@ &i2c0 { status = "okay"; - i2c-switch@77 { + i2c-mux@77 { compatible = "nxp,pca9547"; reg = <0x77>; #address-cells = <1>; diff --git a/arch/arm64/boot/dts/freescale/fsl-ls1088a-rdb.dts b/arch/arm64/boot/dts/freescale/fsl-ls1088a-rdb.dts index 1bfbce69cc8b77bb5cb25680cc3d57baf9cfe5a8..ee8e932628d17f1928a5bed57bbb871566ed291d 100644 --- a/arch/arm64/boot/dts/freescale/fsl-ls1088a-rdb.dts +++ b/arch/arm64/boot/dts/freescale/fsl-ls1088a-rdb.dts @@ -136,7 +136,7 @@ &i2c0 { status = "okay"; - i2c-switch@77 { + i2c-mux@77 { compatible = "nxp,pca9547"; reg = <0x77>; #address-cells = <1>; diff --git a/arch/arm64/boot/dts/freescale/fsl-ls1088a-ten64.dts b/arch/arm64/boot/dts/freescale/fsl-ls1088a-ten64.dts index ef6c8967533efa45216dde21fe07063507d5b2fb..d4867d6cf47cdc79a8a8ebf507b96e7e0683717d 100644 --- a/arch/arm64/boot/dts/freescale/fsl-ls1088a-ten64.dts +++ b/arch/arm64/boot/dts/freescale/fsl-ls1088a-ten64.dts @@ -245,7 +245,7 @@ &i2c3 { status = "okay"; - i2c-switch@70 { + i2c-mux@70 { compatible = "nxp,pca9540"; #address-cells = <1>; #size-cells = <0>; diff --git a/arch/arm64/boot/dts/freescale/fsl-ls208xa-qds.dtsi b/arch/arm64/boot/dts/freescale/fsl-ls208xa-qds.dtsi index f598669e742fc940d444683f0ecaba5235e65331..52c5a43b30a0fb22b4c1f43fe7a7af0b30a68daf 100644 --- a/arch/arm64/boot/dts/freescale/fsl-ls208xa-qds.dtsi +++ b/arch/arm64/boot/dts/freescale/fsl-ls208xa-qds.dtsi @@ -103,7 +103,7 @@ &i2c0 { status = "okay"; - pca9547@77 { + i2c-mux@77 { compatible = "nxp,pca9547"; reg = <0x77>; #address-cells = <1>; diff --git a/arch/arm64/boot/dts/freescale/fsl-ls208xa-rdb.dtsi b/arch/arm64/boot/dts/freescale/fsl-ls208xa-rdb.dtsi index 3d9647b3da14731420f72c4678aae20b3aedf5eb..537cecb13dd08a0fc997c42563feda7326f3757c 100644 --- a/arch/arm64/boot/dts/freescale/fsl-ls208xa-rdb.dtsi +++ b/arch/arm64/boot/dts/freescale/fsl-ls208xa-rdb.dtsi @@ -44,7 +44,7 @@ &i2c0 { status = "okay"; - pca9547@75 { + i2c-mux@75 { compatible = "nxp,pca9547"; reg = <0x75>; #address-cells = <1>; diff --git a/arch/arm64/boot/dts/freescale/fsl-lx2160a-cex7.dtsi b/arch/arm64/boot/dts/freescale/fsl-lx2160a-cex7.dtsi index afb455210bd07ed68319890c6fed3549d63f38e7..d32a52ab00a42a28fdb339f1805ae722ed759a04 100644 --- a/arch/arm64/boot/dts/freescale/fsl-lx2160a-cex7.dtsi +++ b/arch/arm64/boot/dts/freescale/fsl-lx2160a-cex7.dtsi @@ -54,7 +54,7 @@ &i2c0 { status = "okay"; - i2c-switch@77 { + i2c-mux@77 { compatible = "nxp,pca9547"; #address-cells = <1>; #size-cells = <0>; diff --git a/arch/arm64/boot/dts/freescale/imx8mm-beacon-baseboard.dtsi b/arch/arm64/boot/dts/freescale/imx8mm-beacon-baseboard.dtsi index 03266bd90a06baa86204491021f6b08a9806a3c8..169f047fbca50e5d443f12a5f46412a9b18d30d7 100644 --- a/arch/arm64/boot/dts/freescale/imx8mm-beacon-baseboard.dtsi +++ b/arch/arm64/boot/dts/freescale/imx8mm-beacon-baseboard.dtsi @@ -120,7 +120,7 @@ &ecspi2 { pinctrl-names = "default"; pinctrl-0 = <&pinctrl_espi2>; - cs-gpios = <&gpio5 9 GPIO_ACTIVE_LOW>; + cs-gpios = <&gpio5 13 GPIO_ACTIVE_LOW>; status = "okay"; eeprom@0 { @@ -316,7 +316,7 @@ MX8MM_IOMUXC_ECSPI2_SCLK_ECSPI2_SCLK 0x82 MX8MM_IOMUXC_ECSPI2_MOSI_ECSPI2_MOSI 0x82 MX8MM_IOMUXC_ECSPI2_MISO_ECSPI2_MISO 0x82 - MX8MM_IOMUXC_ECSPI1_SS0_GPIO5_IO9 0x41 + MX8MM_IOMUXC_ECSPI2_SS0_GPIO5_IO13 0x41 >; }; diff --git a/arch/arm64/boot/dts/freescale/imx8mm-data-modul-edm-sbc.dts b/arch/arm64/boot/dts/freescale/imx8mm-data-modul-edm-sbc.dts index 24f61db33eba2a7b0f14adc55e87466a0dae0213..752f409a30b16eb4a96c0ca7cef66885e26b8625 100644 --- a/arch/arm64/boot/dts/freescale/imx8mm-data-modul-edm-sbc.dts +++ b/arch/arm64/boot/dts/freescale/imx8mm-data-modul-edm-sbc.dts @@ -275,7 +275,7 @@ compatible = "rohm,bd71847"; reg = <0x4b>; #clock-cells = <0>; - clocks = <&clk_xtal32k 0>; + clocks = <&clk_xtal32k>; clock-output-names = "clk-32k-out"; pinctrl-names = "default"; pinctrl-0 = <&pinctrl_pmic>; diff --git a/arch/arm64/boot/dts/freescale/imx8mm-nitrogen-r2.dts b/arch/arm64/boot/dts/freescale/imx8mm-nitrogen-r2.dts index 74c09891600f2525928321e13dbae633a9517560..6357078185eddb44d64d4ce0cbb4b89231a55111 100644 --- a/arch/arm64/boot/dts/freescale/imx8mm-nitrogen-r2.dts +++ b/arch/arm64/boot/dts/freescale/imx8mm-nitrogen-r2.dts @@ -214,7 +214,7 @@ pinctrl-0 = <&pinctrl_i2c3>; status = "okay"; - i2cmux@70 { + i2c-mux@70 { compatible = "nxp,pca9540"; reg = <0x70>; #address-cells = <1>; diff --git a/arch/arm64/boot/dts/freescale/imx8mm-venice-gw7901.dts b/arch/arm64/boot/dts/freescale/imx8mm-venice-gw7901.dts index 750a1f07ecb7a67ff17b4dc69a11e5d5424ad1b3..6433c205f8dded33d4694080e843cde859aa05c4 100644 --- a/arch/arm64/boot/dts/freescale/imx8mm-venice-gw7901.dts +++ b/arch/arm64/boot/dts/freescale/imx8mm-venice-gw7901.dts @@ -771,6 +771,7 @@ &usbotg2 { dr_mode = "host"; vbus-supply = <®_usb2_vbus>; + over-current-active-low; status = "okay"; }; diff --git a/arch/arm64/boot/dts/freescale/imx8mm-verdin-dahlia.dtsi b/arch/arm64/boot/dts/freescale/imx8mm-verdin-dahlia.dtsi index c2a5c2f7b204b9f41f193375201b996b38e5c1c1..7c3f5c54f04006a0284ce206a9f48ab3743073c7 100644 --- a/arch/arm64/boot/dts/freescale/imx8mm-verdin-dahlia.dtsi +++ b/arch/arm64/boot/dts/freescale/imx8mm-verdin-dahlia.dtsi @@ -9,6 +9,7 @@ simple-audio-card,bitclock-master = <&dailink_master>; simple-audio-card,format = "i2s"; simple-audio-card,frame-master = <&dailink_master>; + simple-audio-card,mclk-fs = <256>; simple-audio-card,name = "imx8mm-wm8904"; simple-audio-card,routing = "Headphone Jack", "HPOUTL", diff --git a/arch/arm64/boot/dts/freescale/imx8mm-verdin-dev.dtsi b/arch/arm64/boot/dts/freescale/imx8mm-verdin-dev.dtsi index 73cc3fafa01808bb49ffb51a6eb8f7debb7e162e..b2bcd22821702073ae513e394ddb1422bb9d7514 100644 --- a/arch/arm64/boot/dts/freescale/imx8mm-verdin-dev.dtsi +++ b/arch/arm64/boot/dts/freescale/imx8mm-verdin-dev.dtsi @@ -11,6 +11,7 @@ simple-audio-card,bitclock-master = <&dailink_master>; simple-audio-card,format = "i2s"; simple-audio-card,frame-master = <&dailink_master>; + simple-audio-card,mclk-fs = <256>; simple-audio-card,name = "imx8mm-nau8822"; simple-audio-card,routing = "Headphones", "LHP", diff --git a/arch/arm64/boot/dts/freescale/imx8mp-evk.dts b/arch/arm64/boot/dts/freescale/imx8mp-evk.dts index d4c7ca16abd057cc6bc6ff45de0404c9d51d5bae..f2d93437084be815a14c764f7f327ff975cf37df 100644 --- a/arch/arm64/boot/dts/freescale/imx8mp-evk.dts +++ b/arch/arm64/boot/dts/freescale/imx8mp-evk.dts @@ -36,8 +36,8 @@ pcie0_refclk: pcie0-refclk { compatible = "fixed-clock"; - #clock-cells = <0>; - clock-frequency = <100000000>; + #clock-cells = <0>; + clock-frequency = <100000000>; }; reg_can1_stby: regulator-can1-stby { diff --git a/arch/arm64/boot/dts/freescale/imx8mp-phycore-som.dtsi b/arch/arm64/boot/dts/freescale/imx8mp-phycore-som.dtsi index 79b290a002c19e8b201921247c73967a1d860bb4..ecc4bce6db97c0b5a8672c57bd4709337a347bf3 100644 --- a/arch/arm64/boot/dts/freescale/imx8mp-phycore-som.dtsi +++ b/arch/arm64/boot/dts/freescale/imx8mp-phycore-som.dtsi @@ -99,7 +99,6 @@ regulators { buck1: BUCK1 { - regulator-compatible = "BUCK1"; regulator-min-microvolt = <600000>; regulator-max-microvolt = <2187500>; regulator-boot-on; @@ -108,7 +107,6 @@ }; buck2: BUCK2 { - regulator-compatible = "BUCK2"; regulator-min-microvolt = <600000>; regulator-max-microvolt = <2187500>; regulator-boot-on; @@ -119,7 +117,6 @@ }; buck4: BUCK4 { - regulator-compatible = "BUCK4"; regulator-min-microvolt = <600000>; regulator-max-microvolt = <3400000>; regulator-boot-on; @@ -127,7 +124,6 @@ }; buck5: BUCK5 { - regulator-compatible = "BUCK5"; regulator-min-microvolt = <600000>; regulator-max-microvolt = <3400000>; regulator-boot-on; @@ -135,7 +131,6 @@ }; buck6: BUCK6 { - regulator-compatible = "BUCK6"; regulator-min-microvolt = <600000>; regulator-max-microvolt = <3400000>; regulator-boot-on; @@ -143,7 +138,6 @@ }; ldo1: LDO1 { - regulator-compatible = "LDO1"; regulator-min-microvolt = <1600000>; regulator-max-microvolt = <3300000>; regulator-boot-on; @@ -151,7 +145,6 @@ }; ldo2: LDO2 { - regulator-compatible = "LDO2"; regulator-min-microvolt = <800000>; regulator-max-microvolt = <1150000>; regulator-boot-on; @@ -159,7 +152,6 @@ }; ldo3: LDO3 { - regulator-compatible = "LDO3"; regulator-min-microvolt = <800000>; regulator-max-microvolt = <3300000>; regulator-boot-on; @@ -167,13 +159,11 @@ }; ldo4: LDO4 { - regulator-compatible = "LDO4"; regulator-min-microvolt = <800000>; regulator-max-microvolt = <3300000>; }; ldo5: LDO5 { - regulator-compatible = "LDO5"; regulator-min-microvolt = <1800000>; regulator-max-microvolt = <3300000>; regulator-boot-on; diff --git a/arch/arm64/boot/dts/freescale/imx8mp.dtsi b/arch/arm64/boot/dts/freescale/imx8mp.dtsi index 7a6e6221f4219cb87624e567f9578b1804366e64..03034b439c1f79ed7ea1a3094f0a0c42f8369297 100644 --- a/arch/arm64/boot/dts/freescale/imx8mp.dtsi +++ b/arch/arm64/boot/dts/freescale/imx8mp.dtsi @@ -524,6 +524,7 @@ compatible = "fsl,imx8mp-gpc"; reg = <0x303a0000 0x1000>; interrupt-parent = <&gic>; + interrupts = ; interrupt-controller; #interrupt-cells = <3>; @@ -590,7 +591,7 @@ reg = ; }; - pgc_hsiomix: power-domains@17 { + pgc_hsiomix: power-domain@17 { #power-domain-cells = <0>; reg = ; clocks = <&clk IMX8MP_CLK_HSIO_AXI>, @@ -1297,7 +1298,7 @@ reg = <0x32f10100 0x8>, <0x381f0000 0x20>; clocks = <&clk IMX8MP_CLK_HSIO_ROOT>, - <&clk IMX8MP_CLK_USB_ROOT>; + <&clk IMX8MP_CLK_USB_SUSP>; clock-names = "hsio", "suspend"; interrupts = ; power-domains = <&hsio_blk_ctrl IMX8MP_HSIOBLK_PD_USB>; @@ -1310,9 +1311,9 @@ usb_dwc3_0: usb@38100000 { compatible = "snps,dwc3"; reg = <0x38100000 0x10000>; - clocks = <&clk IMX8MP_CLK_HSIO_AXI>, + clocks = <&clk IMX8MP_CLK_USB_ROOT>, <&clk IMX8MP_CLK_USB_CORE_REF>, - <&clk IMX8MP_CLK_USB_ROOT>; + <&clk IMX8MP_CLK_USB_SUSP>; clock-names = "bus_early", "ref", "suspend"; interrupts = ; phys = <&usb3_phy0>, <&usb3_phy0>; @@ -1339,7 +1340,7 @@ reg = <0x32f10108 0x8>, <0x382f0000 0x20>; clocks = <&clk IMX8MP_CLK_HSIO_ROOT>, - <&clk IMX8MP_CLK_USB_ROOT>; + <&clk IMX8MP_CLK_USB_SUSP>; clock-names = "hsio", "suspend"; interrupts = ; power-domains = <&hsio_blk_ctrl IMX8MP_HSIOBLK_PD_USB>; @@ -1352,9 +1353,9 @@ usb_dwc3_1: usb@38200000 { compatible = "snps,dwc3"; reg = <0x38200000 0x10000>; - clocks = <&clk IMX8MP_CLK_HSIO_AXI>, + clocks = <&clk IMX8MP_CLK_USB_ROOT>, <&clk IMX8MP_CLK_USB_CORE_REF>, - <&clk IMX8MP_CLK_USB_ROOT>; + <&clk IMX8MP_CLK_USB_SUSP>; clock-names = "bus_early", "ref", "suspend"; interrupts = ; phys = <&usb3_phy1>, <&usb3_phy1>; diff --git a/arch/arm64/boot/dts/freescale/imx8mq-nitrogen.dts b/arch/arm64/boot/dts/freescale/imx8mq-nitrogen.dts index 9dda2a1554c3284890cf56ef62e8a3449225418f..8614c18b5998c242d7638b7cb3381f726d8c6a86 100644 --- a/arch/arm64/boot/dts/freescale/imx8mq-nitrogen.dts +++ b/arch/arm64/boot/dts/freescale/imx8mq-nitrogen.dts @@ -133,7 +133,7 @@ pinctrl-0 = <&pinctrl_i2c1>; status = "okay"; - i2cmux@70 { + i2c-mux@70 { compatible = "nxp,pca9546"; pinctrl-names = "default"; pinctrl-0 = <&pinctrl_i2c1_pca9546>; @@ -216,7 +216,7 @@ pinctrl-0 = <&pinctrl_i2c4>; status = "okay"; - pca9546: i2cmux@70 { + pca9546: i2c-mux@70 { compatible = "nxp,pca9546"; reg = <0x70>; #address-cells = <1>; diff --git a/arch/arm64/boot/dts/freescale/imx8mq-thor96.dts b/arch/arm64/boot/dts/freescale/imx8mq-thor96.dts index 5d5aa6537225fe79b0c270e6e00bbc34d5f682d5..6e6182709d2201c0155c9c0ef9085355956d90ee 100644 --- a/arch/arm64/boot/dts/freescale/imx8mq-thor96.dts +++ b/arch/arm64/boot/dts/freescale/imx8mq-thor96.dts @@ -339,7 +339,7 @@ bus-width = <4>; non-removable; no-sd; - no-emmc; + no-mmc; status = "okay"; brcmf: wifi@1 { @@ -359,7 +359,7 @@ cd-gpios = <&gpio2 12 GPIO_ACTIVE_LOW>; bus-width = <4>; no-sdio; - no-emmc; + no-mmc; disable-wp; status = "okay"; }; diff --git a/arch/arm64/boot/dts/freescale/imx8qxp-mek.dts b/arch/arm64/boot/dts/freescale/imx8qxp-mek.dts index 07d8dd8160f68169001722cb3ca72ab8955c91ad..afa883389456ce772fb489eeb7dbd144820d0618 100644 --- a/arch/arm64/boot/dts/freescale/imx8qxp-mek.dts +++ b/arch/arm64/boot/dts/freescale/imx8qxp-mek.dts @@ -61,7 +61,7 @@ pinctrl-0 = <&pinctrl_lpi2c1 &pinctrl_ioexp_rst>; status = "okay"; - i2c-switch@71 { + i2c-mux@71 { compatible = "nxp,pca9646", "nxp,pca9546"; #address-cells = <1>; #size-cells = <0>; diff --git a/arch/arm64/boot/dts/freescale/imx93-11x11-evk.dts b/arch/arm64/boot/dts/freescale/imx93-11x11-evk.dts index 69786c326db000c64590e2bd3af55f6e3afe9bf7..27f9a9f3313466a9b6b9b9d61a7fe7e5aea38862 100644 --- a/arch/arm64/boot/dts/freescale/imx93-11x11-evk.dts +++ b/arch/arm64/boot/dts/freescale/imx93-11x11-evk.dts @@ -74,7 +74,7 @@ pinctrl_usdhc1: usdhc1grp { fsl,pins = < - MX93_PAD_SD1_CLK__USDHC1_CLK 0x17fe + MX93_PAD_SD1_CLK__USDHC1_CLK 0x15fe MX93_PAD_SD1_CMD__USDHC1_CMD 0x13fe MX93_PAD_SD1_DATA0__USDHC1_DATA0 0x13fe MX93_PAD_SD1_DATA1__USDHC1_DATA1 0x13fe @@ -84,7 +84,7 @@ MX93_PAD_SD1_DATA5__USDHC1_DATA5 0x13fe MX93_PAD_SD1_DATA6__USDHC1_DATA6 0x13fe MX93_PAD_SD1_DATA7__USDHC1_DATA7 0x13fe - MX93_PAD_SD1_STROBE__USDHC1_STROBE 0x17fe + MX93_PAD_SD1_STROBE__USDHC1_STROBE 0x15fe >; }; @@ -102,7 +102,7 @@ pinctrl_usdhc2: usdhc2grp { fsl,pins = < - MX93_PAD_SD2_CLK__USDHC2_CLK 0x17fe + MX93_PAD_SD2_CLK__USDHC2_CLK 0x15fe MX93_PAD_SD2_CMD__USDHC2_CMD 0x13fe MX93_PAD_SD2_DATA0__USDHC2_DATA0 0x13fe MX93_PAD_SD2_DATA1__USDHC2_DATA1 0x13fe diff --git a/arch/arm64/boot/dts/marvell/ac5-98dx25xx.dtsi b/arch/arm64/boot/dts/marvell/ac5-98dx25xx.dtsi index 7308f7b6b22c94e72d7336ee6ccf0c41050b18d4..8bce6406913875d38b7d2759561ac8c7db786726 100644 --- a/arch/arm64/boot/dts/marvell/ac5-98dx25xx.dtsi +++ b/arch/arm64/boot/dts/marvell/ac5-98dx25xx.dtsi @@ -98,7 +98,7 @@ uart1: serial@12100 { compatible = "snps,dw-apb-uart"; - reg = <0x11000 0x100>; + reg = <0x12100 0x100>; reg-shift = <2>; interrupts = ; reg-io-width = <1>; diff --git a/arch/arm64/boot/dts/qcom/msm8992-lg-bullhead.dtsi b/arch/arm64/boot/dts/qcom/msm8992-lg-bullhead.dtsi index 87c90e93667f73c65035e649f70f448f16f7d3ca..79de9cc395c4cfe3527943c5117b3b9861ca8735 100644 --- a/arch/arm64/boot/dts/qcom/msm8992-lg-bullhead.dtsi +++ b/arch/arm64/boot/dts/qcom/msm8992-lg-bullhead.dtsi @@ -3,6 +3,7 @@ * Copyright (c) 2015, LGE Inc. All rights reserved. * Copyright (c) 2016, The Linux Foundation. All rights reserved. * Copyright (c) 2021, Petr Vorel + * Copyright (c) 2022, Dominik Kobinski */ /dts-v1/; @@ -51,6 +52,11 @@ reg = <0 0x03400000 0 0x1200000>; no-map; }; + + removed_region: reserved@5000000 { + reg = <0 0x05000000 0 0x2200000>; + no-map; + }; }; }; diff --git a/arch/arm64/boot/dts/qcom/msm8992-xiaomi-libra.dts b/arch/arm64/boot/dts/qcom/msm8992-xiaomi-libra.dts index b242c272d2af14ee5b5607c8280de48cb06fa006..fcca1ba94da699d0917a6c9fae9fdf09193259b6 100644 --- a/arch/arm64/boot/dts/qcom/msm8992-xiaomi-libra.dts +++ b/arch/arm64/boot/dts/qcom/msm8992-xiaomi-libra.dts @@ -11,6 +11,12 @@ #include #include +/delete-node/ &adsp_mem; +/delete-node/ &audio_mem; +/delete-node/ &mpss_mem; +/delete-node/ &peripheral_region; +/delete-node/ &rmtfs_mem; + / { model = "Xiaomi Mi 4C"; compatible = "xiaomi,libra", "qcom,msm8992"; @@ -70,25 +76,67 @@ #size-cells = <2>; ranges; - /* This is for getting crash logs using Android downstream kernels */ - ramoops@dfc00000 { - compatible = "ramoops"; - reg = <0x0 0xdfc00000 0x0 0x40000>; - console-size = <0x10000>; - record-size = <0x10000>; - ftrace-size = <0x10000>; - pmsg-size = <0x20000>; + memory_hole: hole@6400000 { + reg = <0 0x06400000 0 0x600000>; + no-map; + }; + + memory_hole2: hole2@6c00000 { + reg = <0 0x06c00000 0 0x2400000>; + no-map; + }; + + mpss_mem: mpss@9000000 { + reg = <0 0x09000000 0 0x5a00000>; + no-map; + }; + + tzapp: tzapp@ea00000 { + reg = <0 0x0ea00000 0 0x1900000>; + no-map; + }; + + mdm_rfsa_mem: mdm-rfsa@ca0b0000 { + reg = <0 0xca0b0000 0 0x10000>; + no-map; + }; + + rmtfs_mem: rmtfs@ca100000 { + compatible = "qcom,rmtfs-mem"; + reg = <0 0xca100000 0 0x180000>; + no-map; + + qcom,client-id = <1>; }; - modem_region: modem_region@9000000 { - reg = <0x0 0x9000000 0x0 0x5a00000>; + audio_mem: audio@cb400000 { + reg = <0 0xcb000000 0 0x400000>; + no-mem; + }; + + qseecom_mem: qseecom@cb400000 { + reg = <0 0xcb400000 0 0x1c00000>; + no-mem; + }; + + adsp_rfsa_mem: adsp-rfsa@cd000000 { + reg = <0 0xcd000000 0 0x10000>; no-map; }; - tzapp: modem_region@ea00000 { - reg = <0x0 0xea00000 0x0 0x1900000>; + sensor_rfsa_mem: sensor-rfsa@cd010000 { + reg = <0 0xcd010000 0 0x10000>; no-map; }; + + ramoops@dfc00000 { + compatible = "ramoops"; + reg = <0 0xdfc00000 0 0x40000>; + console-size = <0x10000>; + record-size = <0x10000>; + ftrace-size = <0x10000>; + pmsg-size = <0x20000>; + }; }; }; @@ -130,11 +178,6 @@ status = "okay"; }; -&peripheral_region { - reg = <0x0 0x7400000 0x0 0x1c00000>; - no-map; -}; - &pm8994_spmi_regulators { VDD_APC0: s8 { regulator-min-microvolt = <680000>; diff --git a/arch/arm64/boot/dts/qcom/msm8992.dtsi b/arch/arm64/boot/dts/qcom/msm8992.dtsi index 10adb4986ef1b97d79e12bbd0a2925dd6ae96ed2..02fc3795dbfd73a5c2905f630a88e42b520e7a2e 100644 --- a/arch/arm64/boot/dts/qcom/msm8992.dtsi +++ b/arch/arm64/boot/dts/qcom/msm8992.dtsi @@ -37,10 +37,6 @@ compatible = "qcom,rpmcc-msm8992", "qcom,rpmcc"; }; -&tcsr_mutex { - compatible = "qcom,sfpb-mutex"; -}; - &timer { interrupts = , , diff --git a/arch/arm64/boot/dts/qcom/msm8994-huawei-angler-rev-101.dts b/arch/arm64/boot/dts/qcom/msm8994-huawei-angler-rev-101.dts index 85abff0e9b3f7c6913f26cbbf42d53d3725b7b85..7b0f62144c3eeafa473c12a61fac8293fa7fe793 100644 --- a/arch/arm64/boot/dts/qcom/msm8994-huawei-angler-rev-101.dts +++ b/arch/arm64/boot/dts/qcom/msm8994-huawei-angler-rev-101.dts @@ -9,9 +9,6 @@ #include "msm8994.dtsi" -/* Angler's firmware does not report where the memory is allocated */ -/delete-node/ &cont_splash_mem; - / { model = "Huawei Nexus 6P"; compatible = "huawei,angler", "qcom,msm8994"; @@ -28,6 +25,22 @@ chosen { stdout-path = "serial0:115200n8"; }; + + reserved-memory { + #address-cells = <2>; + #size-cells = <2>; + ranges; + + tzapp_mem: tzapp@4800000 { + reg = <0 0x04800000 0 0x1900000>; + no-map; + }; + + removed_region: reserved@6300000 { + reg = <0 0x06300000 0 0xD00000>; + no-map; + }; + }; }; &blsp1_uart2 { diff --git a/arch/arm64/boot/dts/qcom/sc8280xp.dtsi b/arch/arm64/boot/dts/qcom/sc8280xp.dtsi index 109c9d2b684d115235c895093b7154386bf3e898..71cf81a8eb4da67e0724a5ea36a6f69c93265f42 100644 --- a/arch/arm64/boot/dts/qcom/sc8280xp.dtsi +++ b/arch/arm64/boot/dts/qcom/sc8280xp.dtsi @@ -10,6 +10,7 @@ #include #include #include +#include #include #include #include @@ -762,7 +763,7 @@ <0>, <0>, <0>, - <&usb_0_ssphy>, + <&usb_0_qmpphy QMP_USB43DP_USB3_PIPE_CLK>, <0>, <0>, <0>, @@ -770,7 +771,7 @@ <0>, <0>, <0>, - <&usb_1_ssphy>, + <&usb_1_qmpphy QMP_USB43DP_USB3_PIPE_CLK>, <0>, <0>, <0>, @@ -1673,42 +1674,26 @@ }; }; - usb_0_qmpphy: phy-wrapper@88ec000 { + usb_0_qmpphy: phy@88eb000 { compatible = "qcom,sc8280xp-qmp-usb43dp-phy"; - reg = <0 0x088ec000 0 0x1e4>, - <0 0x088eb000 0 0x40>, - <0 0x088ed000 0 0x1c8>; - #address-cells = <2>; - #size-cells = <2>; - ranges; + reg = <0 0x088eb000 0 0x4000>; clocks = <&gcc GCC_USB3_PRIM_PHY_AUX_CLK>, - <&rpmhcc RPMH_CXO_CLK>, <&gcc GCC_USB4_EUD_CLKREF_CLK>, - <&gcc GCC_USB3_PRIM_PHY_COM_AUX_CLK>; - clock-names = "aux", "ref_clk_src", "ref", "com_aux"; + <&gcc GCC_USB3_PRIM_PHY_COM_AUX_CLK>, + <&gcc GCC_USB3_PRIM_PHY_PIPE_CLK>; + clock-names = "aux", "ref", "com_aux", "usb3_pipe"; + + power-domains = <&gcc USB30_PRIM_GDSC>; resets = <&gcc GCC_USB3_PHY_PRIM_BCR>, - <&gcc GCC_USB3_DP_PHY_PRIM_BCR>; + <&gcc GCC_USB4_DP_PHY_PRIM_BCR>; reset-names = "phy", "common"; - power-domains = <&gcc USB30_PRIM_GDSC>; + #clock-cells = <1>; + #phy-cells = <1>; status = "disabled"; - - usb_0_ssphy: usb3-phy@88eb400 { - reg = <0 0x088eb400 0 0x100>, - <0 0x088eb600 0 0x3ec>, - <0 0x088ec400 0 0x364>, - <0 0x088eba00 0 0x100>, - <0 0x088ebc00 0 0x3ec>, - <0 0x088ec200 0 0x18>; - #phy-cells = <0>; - #clock-cells = <0>; - clocks = <&gcc GCC_USB3_PRIM_PHY_PIPE_CLK>; - clock-names = "pipe0"; - clock-output-names = "usb0_phy_pipe_clk_src"; - }; }; usb_1_hsphy: phy@8902000 { @@ -1725,42 +1710,26 @@ status = "disabled"; }; - usb_1_qmpphy: phy-wrapper@8904000 { + usb_1_qmpphy: phy@8903000 { compatible = "qcom,sc8280xp-qmp-usb43dp-phy"; - reg = <0 0x08904000 0 0x1e4>, - <0 0x08903000 0 0x40>, - <0 0x08905000 0 0x1c8>; - #address-cells = <2>; - #size-cells = <2>; - ranges; + reg = <0 0x08903000 0 0x4000>; clocks = <&gcc GCC_USB3_SEC_PHY_AUX_CLK>, - <&rpmhcc RPMH_CXO_CLK>, <&gcc GCC_USB4_CLKREF_CLK>, - <&gcc GCC_USB3_SEC_PHY_COM_AUX_CLK>; - clock-names = "aux", "ref_clk_src", "ref", "com_aux"; + <&gcc GCC_USB3_SEC_PHY_COM_AUX_CLK>, + <&gcc GCC_USB3_SEC_PHY_PIPE_CLK>; + clock-names = "aux", "ref", "com_aux", "usb3_pipe"; + + power-domains = <&gcc USB30_SEC_GDSC>; resets = <&gcc GCC_USB3_PHY_SEC_BCR>, <&gcc GCC_USB4_1_DP_PHY_PRIM_BCR>; reset-names = "phy", "common"; - power-domains = <&gcc USB30_SEC_GDSC>; + #clock-cells = <1>; + #phy-cells = <1>; status = "disabled"; - - usb_1_ssphy: usb3-phy@8903400 { - reg = <0 0x08903400 0 0x100>, - <0 0x08903600 0 0x3ec>, - <0 0x08904400 0 0x364>, - <0 0x08903a00 0 0x100>, - <0 0x08903c00 0 0x3ec>, - <0 0x08904200 0 0x18>; - #phy-cells = <0>; - #clock-cells = <0>; - clocks = <&gcc GCC_USB3_SEC_PHY_PIPE_CLK>; - clock-names = "pipe0"; - clock-output-names = "usb1_phy_pipe_clk_src"; - }; }; pmu@9091000 { @@ -1910,7 +1879,7 @@ reg = <0 0x0a600000 0 0xcd00>; interrupts = ; iommus = <&apps_smmu 0x820 0x0>; - phys = <&usb_0_hsphy>, <&usb_0_ssphy>; + phys = <&usb_0_hsphy>, <&usb_0_qmpphy QMP_USB43DP_USB3_PHY>; phy-names = "usb2-phy", "usb3-phy"; }; }; @@ -1964,7 +1933,7 @@ reg = <0 0x0a800000 0 0xcd00>; interrupts = ; iommus = <&apps_smmu 0x860 0x0>; - phys = <&usb_1_hsphy>, <&usb_1_ssphy>; + phys = <&usb_1_hsphy>, <&usb_1_qmpphy QMP_USB43DP_USB3_PHY>; phy-names = "usb2-phy", "usb3-phy"; }; }; diff --git a/arch/arm64/boot/dts/qcom/sm8250.dtsi b/arch/arm64/boot/dts/qcom/sm8250.dtsi index dab5579946f351f1a43314f121fc5682556f29ff..927032863e2f15e9d63b57a3b66956d6243ea9ed 100644 --- a/arch/arm64/boot/dts/qcom/sm8250.dtsi +++ b/arch/arm64/boot/dts/qcom/sm8250.dtsi @@ -334,7 +334,6 @@ exit-latency-us = <6562>; min-residency-us = <9987>; local-timer-stop; - status = "disabled"; }; }; }; diff --git a/arch/arm64/boot/dts/qcom/sm8350.dtsi b/arch/arm64/boot/dts/qcom/sm8350.dtsi index 245dce24ec599c0d8b062f9953a234a792508616..fb3cd20a82b5eea676ccadcfa108031a747fd957 100644 --- a/arch/arm64/boot/dts/qcom/sm8350.dtsi +++ b/arch/arm64/boot/dts/qcom/sm8350.dtsi @@ -2382,8 +2382,8 @@ <&rpmhcc RPMH_CXO_CLK>; clock-names = "iface", "core", "xo"; resets = <&gcc GCC_SDCC2_BCR>; - interconnects = <&aggre2_noc MASTER_SDCC_2 0 &mc_virt SLAVE_EBI1 0>, - <&gem_noc MASTER_APPSS_PROC 0 &config_noc SLAVE_SDCC_2 0>; + interconnects = <&aggre2_noc MASTER_SDCC_2 &mc_virt SLAVE_EBI1>, + <&gem_noc MASTER_APPSS_PROC &config_noc SLAVE_SDCC_2>; interconnect-names = "sdhc-ddr","cpu-sdhc"; iommus = <&apps_smmu 0x4a0 0x0>; power-domains = <&rpmhpd SM8350_CX>; diff --git a/arch/arm64/crypto/sm4-ce-ccm-core.S b/arch/arm64/crypto/sm4-ce-ccm-core.S index 028207c4afd0f76037336cf030375dd9b3ab4011..fa85856f33ceeed098f5bba2963d301e9280d544 100644 --- a/arch/arm64/crypto/sm4-ce-ccm-core.S +++ b/arch/arm64/crypto/sm4-ce-ccm-core.S @@ -8,6 +8,7 @@ */ #include +#include #include #include "sm4-ce-asm.h" @@ -104,7 +105,7 @@ SYM_FUNC_START(sm4_ce_ccm_final) SYM_FUNC_END(sm4_ce_ccm_final) .align 3 -SYM_FUNC_START(sm4_ce_ccm_enc) +SYM_TYPED_FUNC_START(sm4_ce_ccm_enc) /* input: * x0: round key array, CTX * x1: dst @@ -216,7 +217,7 @@ SYM_FUNC_START(sm4_ce_ccm_enc) SYM_FUNC_END(sm4_ce_ccm_enc) .align 3 -SYM_FUNC_START(sm4_ce_ccm_dec) +SYM_TYPED_FUNC_START(sm4_ce_ccm_dec) /* input: * x0: round key array, CTX * x1: dst diff --git a/arch/arm64/crypto/sm4-ce-gcm-core.S b/arch/arm64/crypto/sm4-ce-gcm-core.S index 7aa3ec18a28912bc13d74c6b187b55a5b2d2cf2e..347f25d7572793a1b9d1f853e62ecc553521167b 100644 --- a/arch/arm64/crypto/sm4-ce-gcm-core.S +++ b/arch/arm64/crypto/sm4-ce-gcm-core.S @@ -9,6 +9,7 @@ */ #include +#include #include #include "sm4-ce-asm.h" @@ -370,7 +371,7 @@ SYM_FUNC_START(pmull_ghash_update) SYM_FUNC_END(pmull_ghash_update) .align 3 -SYM_FUNC_START(sm4_ce_pmull_gcm_enc) +SYM_TYPED_FUNC_START(sm4_ce_pmull_gcm_enc) /* input: * x0: round key array, CTX * x1: dst @@ -581,7 +582,7 @@ SYM_FUNC_END(sm4_ce_pmull_gcm_enc) #define RH3 v20 .align 3 -SYM_FUNC_START(sm4_ce_pmull_gcm_dec) +SYM_TYPED_FUNC_START(sm4_ce_pmull_gcm_dec) /* input: * x0: round key array, CTX * x1: dst diff --git a/arch/arm64/include/asm/atomic_ll_sc.h b/arch/arm64/include/asm/atomic_ll_sc.h index 0890e4f568fb7f9804566fbf82b2b79bea0732df..cbb3d961123b17aba398f29c2a346ac2e7f5bf47 100644 --- a/arch/arm64/include/asm/atomic_ll_sc.h +++ b/arch/arm64/include/asm/atomic_ll_sc.h @@ -315,7 +315,7 @@ __ll_sc__cmpxchg_double##name(unsigned long old1, \ " cbnz %w0, 1b\n" \ " " #mb "\n" \ "2:" \ - : "=&r" (tmp), "=&r" (ret), "+Q" (*(unsigned long *)ptr) \ + : "=&r" (tmp), "=&r" (ret), "+Q" (*(__uint128_t *)ptr) \ : "r" (old1), "r" (old2), "r" (new1), "r" (new2) \ : cl); \ \ diff --git a/arch/arm64/include/asm/atomic_lse.h b/arch/arm64/include/asm/atomic_lse.h index 52075e93de6c01a0b7034e0a14887e43a9167137..a94d6dacc0292e3e92b644636c923338556b3222 100644 --- a/arch/arm64/include/asm/atomic_lse.h +++ b/arch/arm64/include/asm/atomic_lse.h @@ -311,7 +311,7 @@ __lse__cmpxchg_double##name(unsigned long old1, \ " eor %[old2], %[old2], %[oldval2]\n" \ " orr %[old1], %[old1], %[old2]" \ : [old1] "+&r" (x0), [old2] "+&r" (x1), \ - [v] "+Q" (*(unsigned long *)ptr) \ + [v] "+Q" (*(__uint128_t *)ptr) \ : [new1] "r" (x2), [new2] "r" (x3), [ptr] "r" (x4), \ [oldval1] "r" (oldval1), [oldval2] "r" (oldval2) \ : cl); \ diff --git a/arch/arm64/include/asm/cputype.h b/arch/arm64/include/asm/cputype.h index 4e8b66c74ea2b20b1ee1e22623ed76ac05e6750e..683ca3af408485aa6cdf4aa44b5988d91cd6a0f3 100644 --- a/arch/arm64/include/asm/cputype.h +++ b/arch/arm64/include/asm/cputype.h @@ -124,6 +124,8 @@ #define APPLE_CPU_PART_M1_FIRESTORM_PRO 0x025 #define APPLE_CPU_PART_M1_ICESTORM_MAX 0x028 #define APPLE_CPU_PART_M1_FIRESTORM_MAX 0x029 +#define APPLE_CPU_PART_M2_BLIZZARD 0x032 +#define APPLE_CPU_PART_M2_AVALANCHE 0x033 #define AMPERE_CPU_PART_AMPERE1 0xAC3 @@ -177,6 +179,8 @@ #define MIDR_APPLE_M1_FIRESTORM_PRO MIDR_CPU_MODEL(ARM_CPU_IMP_APPLE, APPLE_CPU_PART_M1_FIRESTORM_PRO) #define MIDR_APPLE_M1_ICESTORM_MAX MIDR_CPU_MODEL(ARM_CPU_IMP_APPLE, APPLE_CPU_PART_M1_ICESTORM_MAX) #define MIDR_APPLE_M1_FIRESTORM_MAX MIDR_CPU_MODEL(ARM_CPU_IMP_APPLE, APPLE_CPU_PART_M1_FIRESTORM_MAX) +#define MIDR_APPLE_M2_BLIZZARD MIDR_CPU_MODEL(ARM_CPU_IMP_APPLE, APPLE_CPU_PART_M2_BLIZZARD) +#define MIDR_APPLE_M2_AVALANCHE MIDR_CPU_MODEL(ARM_CPU_IMP_APPLE, APPLE_CPU_PART_M2_AVALANCHE) #define MIDR_AMPERE1 MIDR_CPU_MODEL(ARM_CPU_IMP_AMPERE, AMPERE_CPU_PART_AMPERE1) /* Fujitsu Erratum 010001 affects A64FX 1.0 and 1.1, (v0r0 and v1r0) */ diff --git a/arch/arm64/include/asm/efi.h b/arch/arm64/include/asm/efi.h index 31d13a6001df49c482a6bd426d0a8718288746ad..de4ff90785b2c96a15ad08484dedbda289951fae 100644 --- a/arch/arm64/include/asm/efi.h +++ b/arch/arm64/include/asm/efi.h @@ -48,8 +48,17 @@ int efi_set_mapping_permissions(struct mm_struct *mm, efi_memory_desc_t *md); }) extern spinlock_t efi_rt_lock; +extern u64 *efi_rt_stack_top; efi_status_t __efi_rt_asm_wrapper(void *, const char *, ...); +/* + * efi_rt_stack_top[-1] contains the value the stack pointer had before + * switching to the EFI runtime stack. + */ +#define current_in_efi() \ + (!preemptible() && efi_rt_stack_top != NULL && \ + on_task_stack(current, READ_ONCE(efi_rt_stack_top[-1]), 1)) + #define ARCH_EFI_IRQ_FLAGS_MASK (PSR_D_BIT | PSR_A_BIT | PSR_I_BIT | PSR_F_BIT) /* diff --git a/arch/arm64/include/asm/esr.h b/arch/arm64/include/asm/esr.h index 15b34fbfca66735fe657d746149105d0b8d5a32d..206de10524e338c9406d57641170f374527dd178 100644 --- a/arch/arm64/include/asm/esr.h +++ b/arch/arm64/include/asm/esr.h @@ -114,6 +114,15 @@ #define ESR_ELx_FSC_ACCESS (0x08) #define ESR_ELx_FSC_FAULT (0x04) #define ESR_ELx_FSC_PERM (0x0C) +#define ESR_ELx_FSC_SEA_TTW0 (0x14) +#define ESR_ELx_FSC_SEA_TTW1 (0x15) +#define ESR_ELx_FSC_SEA_TTW2 (0x16) +#define ESR_ELx_FSC_SEA_TTW3 (0x17) +#define ESR_ELx_FSC_SECC (0x18) +#define ESR_ELx_FSC_SECC_TTW0 (0x1c) +#define ESR_ELx_FSC_SECC_TTW1 (0x1d) +#define ESR_ELx_FSC_SECC_TTW2 (0x1e) +#define ESR_ELx_FSC_SECC_TTW3 (0x1f) /* ISS field definitions for Data Aborts */ #define ESR_ELx_ISV_SHIFT (24) diff --git a/arch/arm64/include/asm/hugetlb.h b/arch/arm64/include/asm/hugetlb.h index d20f5da2d76fa3ef876e99b14c28705da0d0ace1..6a4a1ab8eb238f148177a30e2c32f7f912e8bd31 100644 --- a/arch/arm64/include/asm/hugetlb.h +++ b/arch/arm64/include/asm/hugetlb.h @@ -49,6 +49,15 @@ extern pte_t huge_ptep_get(pte_t *ptep); void __init arm64_hugetlb_cma_reserve(void); +#define huge_ptep_modify_prot_start huge_ptep_modify_prot_start +extern pte_t huge_ptep_modify_prot_start(struct vm_area_struct *vma, + unsigned long addr, pte_t *ptep); + +#define huge_ptep_modify_prot_commit huge_ptep_modify_prot_commit +extern void huge_ptep_modify_prot_commit(struct vm_area_struct *vma, + unsigned long addr, pte_t *ptep, + pte_t old_pte, pte_t new_pte); + #include #endif /* __ASM_HUGETLB_H */ diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h index 0df3fc3a017371cf402efc50c32fe326a4e14711..26b0c97df98636274163ab0bb6a5ea5e5c6f38bc 100644 --- a/arch/arm64/include/asm/kvm_arm.h +++ b/arch/arm64/include/asm/kvm_arm.h @@ -319,21 +319,6 @@ BIT(18) | \ GENMASK(16, 15)) -/* For compatibility with fault code shared with 32-bit */ -#define FSC_FAULT ESR_ELx_FSC_FAULT -#define FSC_ACCESS ESR_ELx_FSC_ACCESS -#define FSC_PERM ESR_ELx_FSC_PERM -#define FSC_SEA ESR_ELx_FSC_EXTABT -#define FSC_SEA_TTW0 (0x14) -#define FSC_SEA_TTW1 (0x15) -#define FSC_SEA_TTW2 (0x16) -#define FSC_SEA_TTW3 (0x17) -#define FSC_SECC (0x18) -#define FSC_SECC_TTW0 (0x1c) -#define FSC_SECC_TTW1 (0x1d) -#define FSC_SECC_TTW2 (0x1e) -#define FSC_SECC_TTW3 (0x1f) - /* Hyp Prefetch Fault Address Register (HPFAR/HDFAR) */ #define HPFAR_MASK (~UL(0xf)) /* diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h index 9bdba47f7e149f90055657aa251155708a806ad6..193583df2d9c45761ded5a28a750e7d539d96945 100644 --- a/arch/arm64/include/asm/kvm_emulate.h +++ b/arch/arm64/include/asm/kvm_emulate.h @@ -349,16 +349,16 @@ static __always_inline u8 kvm_vcpu_trap_get_fault_level(const struct kvm_vcpu *v static __always_inline bool kvm_vcpu_abt_issea(const struct kvm_vcpu *vcpu) { switch (kvm_vcpu_trap_get_fault(vcpu)) { - case FSC_SEA: - case FSC_SEA_TTW0: - case FSC_SEA_TTW1: - case FSC_SEA_TTW2: - case FSC_SEA_TTW3: - case FSC_SECC: - case FSC_SECC_TTW0: - case FSC_SECC_TTW1: - case FSC_SECC_TTW2: - case FSC_SECC_TTW3: + case ESR_ELx_FSC_EXTABT: + case ESR_ELx_FSC_SEA_TTW0: + case ESR_ELx_FSC_SEA_TTW1: + case ESR_ELx_FSC_SEA_TTW2: + case ESR_ELx_FSC_SEA_TTW3: + case ESR_ELx_FSC_SECC: + case ESR_ELx_FSC_SECC_TTW0: + case ESR_ELx_FSC_SECC_TTW1: + case ESR_ELx_FSC_SECC_TTW2: + case ESR_ELx_FSC_SECC_TTW3: return true; default: return false; @@ -373,8 +373,26 @@ static __always_inline int kvm_vcpu_sys_get_rt(struct kvm_vcpu *vcpu) static inline bool kvm_is_write_fault(struct kvm_vcpu *vcpu) { - if (kvm_vcpu_abt_iss1tw(vcpu)) - return true; + if (kvm_vcpu_abt_iss1tw(vcpu)) { + /* + * Only a permission fault on a S1PTW should be + * considered as a write. Otherwise, page tables baked + * in a read-only memslot will result in an exception + * being delivered in the guest. + * + * The drawback is that we end-up faulting twice if the + * guest is using any of HW AF/DB: a translation fault + * to map the page containing the PT (read only at + * first), then a permission fault to allow the flags + * to be set. + */ + switch (kvm_vcpu_trap_get_fault_type(vcpu)) { + case ESR_ELx_FSC_PERM: + return true; + default: + return false; + } + } if (kvm_vcpu_trap_is_iabt(vcpu)) return false; diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h index b4bbeed80fb6ed1d37334a35b6ed49ce2bc0dcbc..65e78999c75d76ea8e1443baec542d5f80c871d9 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -681,7 +681,7 @@ static inline unsigned long pmd_page_vaddr(pmd_t pmd) #define pud_leaf(pud) (pud_present(pud) && !pud_table(pud)) #define pud_valid(pud) pte_valid(pud_pte(pud)) #define pud_user(pud) pte_user(pud_pte(pud)) - +#define pud_user_exec(pud) pte_user_exec(pud_pte(pud)) static inline void set_pud(pud_t *pudp, pud_t pud) { @@ -730,6 +730,7 @@ static inline pmd_t *pud_pgtable(pud_t pud) #else #define pud_page_paddr(pud) ({ BUILD_BUG(); 0; }) +#define pud_user_exec(pud) pud_user(pud) /* Always 0 with folding */ /* Match pmd_offset folding in */ #define pmd_set_fixmap(addr) NULL @@ -862,12 +863,12 @@ static inline bool pte_user_accessible_page(pte_t pte) static inline bool pmd_user_accessible_page(pmd_t pmd) { - return pmd_leaf(pmd) && (pmd_user(pmd) || pmd_user_exec(pmd)); + return pmd_leaf(pmd) && !pmd_present_invalid(pmd) && (pmd_user(pmd) || pmd_user_exec(pmd)); } static inline bool pud_user_accessible_page(pud_t pud) { - return pud_leaf(pud) && pud_user(pud); + return pud_leaf(pud) && (pud_user(pud) || pud_user_exec(pud)); } #endif @@ -1093,6 +1094,15 @@ static inline bool pud_sect_supported(void) } +#define __HAVE_ARCH_PTEP_MODIFY_PROT_TRANSACTION +#define ptep_modify_prot_start ptep_modify_prot_start +extern pte_t ptep_modify_prot_start(struct vm_area_struct *vma, + unsigned long addr, pte_t *ptep); + +#define ptep_modify_prot_commit ptep_modify_prot_commit +extern void ptep_modify_prot_commit(struct vm_area_struct *vma, + unsigned long addr, pte_t *ptep, + pte_t old_pte, pte_t new_pte); #endif /* !__ASSEMBLY__ */ #endif /* __ASM_PGTABLE_H */ diff --git a/arch/arm64/include/asm/stacktrace.h b/arch/arm64/include/asm/stacktrace.h index 4e5354beafb01bac900f854ea7cfecbbf068ab5c..66ec8caa6ac07fa0d70865e40b594bdd7079d585 100644 --- a/arch/arm64/include/asm/stacktrace.h +++ b/arch/arm64/include/asm/stacktrace.h @@ -106,4 +106,19 @@ static inline struct stack_info stackinfo_get_sdei_critical(void) #define stackinfo_get_sdei_critical() stackinfo_get_unknown() #endif +#ifdef CONFIG_EFI +extern u64 *efi_rt_stack_top; + +static inline struct stack_info stackinfo_get_efi(void) +{ + unsigned long high = (u64)efi_rt_stack_top; + unsigned long low = high - THREAD_SIZE; + + return (struct stack_info) { + .low = low, + .high = high, + }; +} +#endif + #endif /* __ASM_STACKTRACE_H */ diff --git a/arch/arm64/include/asm/uprobes.h b/arch/arm64/include/asm/uprobes.h index ba4bff5ca674943100a51fcfc3810d295514a9d4..2b09495499c6183a155e81b3fc33d13cb6e62012 100644 --- a/arch/arm64/include/asm/uprobes.h +++ b/arch/arm64/include/asm/uprobes.h @@ -16,7 +16,7 @@ #define UPROBE_SWBP_INSN_SIZE AARCH64_INSN_SIZE #define UPROBE_XOL_SLOT_BYTES MAX_UINSN_BYTES -typedef u32 uprobe_opcode_t; +typedef __le32 uprobe_opcode_t; struct arch_uprobe_task { }; diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c index 89ac00084f38a4584924bd2da8e2e0e9c5341a07..307faa2b4395ed9fcacdbf9f992111cd376dc601 100644 --- a/arch/arm64/kernel/cpu_errata.c +++ b/arch/arm64/kernel/cpu_errata.c @@ -661,6 +661,13 @@ const struct arm64_cpu_capabilities arm64_errata[] = { CAP_MIDR_RANGE_LIST(trbe_write_out_of_range_cpus), }, #endif +#ifdef CONFIG_ARM64_ERRATUM_2645198 + { + .desc = "ARM erratum 2645198", + .capability = ARM64_WORKAROUND_2645198, + ERRATA_MIDR_ALL_VERSIONS(MIDR_CORTEX_A715) + }, +#endif #ifdef CONFIG_ARM64_ERRATUM_2077057 { .desc = "ARM erratum 2077057", diff --git a/arch/arm64/kernel/efi-rt-wrapper.S b/arch/arm64/kernel/efi-rt-wrapper.S index a00886410537d6a615c7f415a884631b1f0e4a1d..e8ae803662cf14d1de635e2f89fe062d3a72209d 100644 --- a/arch/arm64/kernel/efi-rt-wrapper.S +++ b/arch/arm64/kernel/efi-rt-wrapper.S @@ -4,6 +4,7 @@ */ #include +#include SYM_FUNC_START(__efi_rt_asm_wrapper) stp x29, x30, [sp, #-112]! @@ -45,7 +46,10 @@ SYM_FUNC_START(__efi_rt_asm_wrapper) mov x4, x6 blr x8 + mov x16, sp mov sp, x29 + str xzr, [x16, #8] // clear recorded task SP value + ldp x1, x2, [sp, #16] cmp x2, x18 ldp x29, x30, [sp], #112 @@ -70,6 +74,9 @@ SYM_FUNC_END(__efi_rt_asm_wrapper) SYM_CODE_START(__efi_rt_asm_recover) mov sp, x30 + ldr_l x16, efi_rt_stack_top // clear recorded task SP value + str xzr, [x16, #-8] + ldp x19, x20, [sp, #32] ldp x21, x22, [sp, #48] ldp x23, x24, [sp, #64] diff --git a/arch/arm64/kernel/efi.c b/arch/arm64/kernel/efi.c index fab05de2e12dd5d8ad28b1a647fea66c6ac1651e..b273900f4566858710bee8d427721accc55fa4b4 100644 --- a/arch/arm64/kernel/efi.c +++ b/arch/arm64/kernel/efi.c @@ -11,6 +11,7 @@ #include #include +#include static bool region_is_misaligned(const efi_memory_desc_t *md) { @@ -154,7 +155,7 @@ asmlinkage efi_status_t __efi_rt_asm_recover(void); bool efi_runtime_fixup_exception(struct pt_regs *regs, const char *msg) { /* Check whether the exception occurred while running the firmware */ - if (current_work() != &efi_rts_work.work || regs->pc >= TASK_SIZE_64) + if (!current_in_efi() || regs->pc >= TASK_SIZE_64) return false; pr_err(FW_BUG "Unable to handle %s in EFI runtime service\n", msg); diff --git a/arch/arm64/kernel/elfcore.c b/arch/arm64/kernel/elfcore.c index 353009d7f307f40e982adf627a3ad9e408c80a50..2e94d20c4ac7a780095703b69b3da0fccce37537 100644 --- a/arch/arm64/kernel/elfcore.c +++ b/arch/arm64/kernel/elfcore.c @@ -8,28 +8,27 @@ #include #include -#define for_each_mte_vma(vmi, vma) \ +#define for_each_mte_vma(cprm, i, m) \ if (system_supports_mte()) \ - for_each_vma(vmi, vma) \ - if (vma->vm_flags & VM_MTE) + for (i = 0, m = cprm->vma_meta; \ + i < cprm->vma_count; \ + i++, m = cprm->vma_meta + i) \ + if (m->flags & VM_MTE) -static unsigned long mte_vma_tag_dump_size(struct vm_area_struct *vma) +static unsigned long mte_vma_tag_dump_size(struct core_vma_metadata *m) { - if (vma->vm_flags & VM_DONTDUMP) - return 0; - - return vma_pages(vma) * MTE_PAGE_TAG_STORAGE; + return (m->dump_size >> PAGE_SHIFT) * MTE_PAGE_TAG_STORAGE; } /* Derived from dump_user_range(); start/end must be page-aligned */ static int mte_dump_tag_range(struct coredump_params *cprm, - unsigned long start, unsigned long end) + unsigned long start, unsigned long len) { int ret = 1; unsigned long addr; void *tags = NULL; - for (addr = start; addr < end; addr += PAGE_SIZE) { + for (addr = start; addr < start + len; addr += PAGE_SIZE) { struct page *page = get_dump_page(addr); /* @@ -65,7 +64,6 @@ static int mte_dump_tag_range(struct coredump_params *cprm, mte_save_page_tags(page_address(page), tags); put_page(page); if (!dump_emit(cprm, tags, MTE_PAGE_TAG_STORAGE)) { - mte_free_tag_storage(tags); ret = 0; break; } @@ -77,13 +75,13 @@ static int mte_dump_tag_range(struct coredump_params *cprm, return ret; } -Elf_Half elf_core_extra_phdrs(void) +Elf_Half elf_core_extra_phdrs(struct coredump_params *cprm) { - struct vm_area_struct *vma; + int i; + struct core_vma_metadata *m; int vma_count = 0; - VMA_ITERATOR(vmi, current->mm, 0); - for_each_mte_vma(vmi, vma) + for_each_mte_vma(cprm, i, m) vma_count++; return vma_count; @@ -91,18 +89,18 @@ Elf_Half elf_core_extra_phdrs(void) int elf_core_write_extra_phdrs(struct coredump_params *cprm, loff_t offset) { - struct vm_area_struct *vma; - VMA_ITERATOR(vmi, current->mm, 0); + int i; + struct core_vma_metadata *m; - for_each_mte_vma(vmi, vma) { + for_each_mte_vma(cprm, i, m) { struct elf_phdr phdr; phdr.p_type = PT_AARCH64_MEMTAG_MTE; phdr.p_offset = offset; - phdr.p_vaddr = vma->vm_start; + phdr.p_vaddr = m->start; phdr.p_paddr = 0; - phdr.p_filesz = mte_vma_tag_dump_size(vma); - phdr.p_memsz = vma->vm_end - vma->vm_start; + phdr.p_filesz = mte_vma_tag_dump_size(m); + phdr.p_memsz = m->end - m->start; offset += phdr.p_filesz; phdr.p_flags = 0; phdr.p_align = 0; @@ -114,28 +112,25 @@ int elf_core_write_extra_phdrs(struct coredump_params *cprm, loff_t offset) return 1; } -size_t elf_core_extra_data_size(void) +size_t elf_core_extra_data_size(struct coredump_params *cprm) { - struct vm_area_struct *vma; + int i; + struct core_vma_metadata *m; size_t data_size = 0; - VMA_ITERATOR(vmi, current->mm, 0); - for_each_mte_vma(vmi, vma) - data_size += mte_vma_tag_dump_size(vma); + for_each_mte_vma(cprm, i, m) + data_size += mte_vma_tag_dump_size(m); return data_size; } int elf_core_write_extra_data(struct coredump_params *cprm) { - struct vm_area_struct *vma; - VMA_ITERATOR(vmi, current->mm, 0); - - for_each_mte_vma(vmi, vma) { - if (vma->vm_flags & VM_DONTDUMP) - continue; + int i; + struct core_vma_metadata *m; - if (!mte_dump_tag_range(cprm, vma->vm_start, vma->vm_end)) + for_each_mte_vma(cprm, i, m) { + if (!mte_dump_tag_range(cprm, m->start, m->dump_size)) return 0; } diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c index dcc81e7200d4011c99a448f90853263d321a7e65..b6ef1af0122ebde4984b504e8a14578fe6996ffb 100644 --- a/arch/arm64/kernel/fpsimd.c +++ b/arch/arm64/kernel/fpsimd.c @@ -385,7 +385,7 @@ static void task_fpsimd_load(void) WARN_ON(!system_supports_fpsimd()); WARN_ON(!have_cpu_fpsimd_context()); - if (system_supports_sve()) { + if (system_supports_sve() || system_supports_sme()) { switch (current->thread.fp_type) { case FP_STATE_FPSIMD: /* Stop tracking SVE for this task until next use. */ diff --git a/arch/arm64/kernel/ptrace.c b/arch/arm64/kernel/ptrace.c index 2686ab15760173d1dc11805f579daf4d7e563ba5..0c321ad23cd3a48d60a52f3a0a1209c4b7218ac2 100644 --- a/arch/arm64/kernel/ptrace.c +++ b/arch/arm64/kernel/ptrace.c @@ -1357,7 +1357,7 @@ enum aarch64_regset { #ifdef CONFIG_ARM64_SVE REGSET_SVE, #endif -#ifdef CONFIG_ARM64_SVE +#ifdef CONFIG_ARM64_SME REGSET_SSVE, REGSET_ZA, #endif diff --git a/arch/arm64/kernel/signal.c b/arch/arm64/kernel/signal.c index e0d09bf5b01b80a8ed637bc1f0dd51168f0a55a2..be279fd482480de2a3b2dfff80a37300907555d8 100644 --- a/arch/arm64/kernel/signal.c +++ b/arch/arm64/kernel/signal.c @@ -281,7 +281,12 @@ static int restore_sve_fpsimd_context(struct user_ctxs *user) vl = task_get_sme_vl(current); } else { - if (!system_supports_sve()) + /* + * A SME only system use SVE for streaming mode so can + * have a SVE formatted context with a zero VL and no + * payload data. + */ + if (!system_supports_sve() && !system_supports_sme()) return -EINVAL; vl = task_get_sve_vl(current); @@ -732,7 +737,7 @@ static int setup_sigframe_layout(struct rt_sigframe_user_layout *user, return err; } - if (system_supports_sve()) { + if (system_supports_sve() || system_supports_sme()) { unsigned int vq = 0; if (add_all || test_thread_flag(TIF_SVE) || diff --git a/arch/arm64/kernel/stacktrace.c b/arch/arm64/kernel/stacktrace.c index 117e2c180f3c77d8f65af2ca51012c2ba075212a..83154303e682c8b69aa78fa37f3e63d150bd1470 100644 --- a/arch/arm64/kernel/stacktrace.c +++ b/arch/arm64/kernel/stacktrace.c @@ -5,6 +5,7 @@ * Copyright (C) 2012 ARM Ltd. */ #include +#include #include #include #include @@ -12,6 +13,7 @@ #include #include +#include #include #include #include @@ -186,6 +188,13 @@ void show_stack(struct task_struct *tsk, unsigned long *sp, const char *loglvl) : stackinfo_get_unknown(); \ }) +#define STACKINFO_EFI \ + ({ \ + ((task == current) && current_in_efi()) \ + ? stackinfo_get_efi() \ + : stackinfo_get_unknown(); \ + }) + noinline noinstr void arch_stack_walk(stack_trace_consume_fn consume_entry, void *cookie, struct task_struct *task, struct pt_regs *regs) @@ -199,6 +208,9 @@ noinline noinstr void arch_stack_walk(stack_trace_consume_fn consume_entry, #if defined(CONFIG_VMAP_STACK) && defined(CONFIG_ARM_SDE_INTERFACE) STACKINFO_SDEI(normal), STACKINFO_SDEI(critical), +#endif +#ifdef CONFIG_EFI + STACKINFO_EFI, #endif }; struct unwind_state state = { diff --git a/arch/arm64/kvm/guest.c b/arch/arm64/kvm/guest.c index 5626ddb540ce3333d561f9e61c13d3a695f859fe..cf4c495a4321332b7bc5a1979fc84d3084da355e 100644 --- a/arch/arm64/kvm/guest.c +++ b/arch/arm64/kvm/guest.c @@ -1079,7 +1079,7 @@ long kvm_vm_ioctl_mte_copy_tags(struct kvm *kvm, /* uaccess failed, don't leave stale tags */ if (num_tags != MTE_GRANULES_PER_PAGE) - mte_clear_page_tags(page); + mte_clear_page_tags(maddr); set_page_mte_tagged(page); kvm_release_pfn_dirty(pfn); diff --git a/arch/arm64/kvm/hyp/include/hyp/fault.h b/arch/arm64/kvm/hyp/include/hyp/fault.h index 1b8a2dcd712f321c94b3304012770a62ef6bc61a..9ddcfe2c3e574fc8d85b01d91f244f2091350247 100644 --- a/arch/arm64/kvm/hyp/include/hyp/fault.h +++ b/arch/arm64/kvm/hyp/include/hyp/fault.h @@ -60,7 +60,7 @@ static inline bool __get_fault_info(u64 esr, struct kvm_vcpu_fault_info *fault) */ if (!(esr & ESR_ELx_S1PTW) && (cpus_have_final_cap(ARM64_WORKAROUND_834220) || - (esr & ESR_ELx_FSC_TYPE) == FSC_PERM)) { + (esr & ESR_ELx_FSC_TYPE) == ESR_ELx_FSC_PERM)) { if (!__translate_far_to_hpfar(far, &hpfar)) return false; } else { diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h index 3330d1b76bdd2015592d35e9a6ef55502050e885..07d37ff88a3f2a69352ae123a08cf37ebfcb273e 100644 --- a/arch/arm64/kvm/hyp/include/hyp/switch.h +++ b/arch/arm64/kvm/hyp/include/hyp/switch.h @@ -367,7 +367,7 @@ static bool kvm_hyp_handle_dabt_low(struct kvm_vcpu *vcpu, u64 *exit_code) if (static_branch_unlikely(&vgic_v2_cpuif_trap)) { bool valid; - valid = kvm_vcpu_trap_get_fault_type(vcpu) == FSC_FAULT && + valid = kvm_vcpu_trap_get_fault_type(vcpu) == ESR_ELx_FSC_FAULT && kvm_vcpu_dabt_isvalid(vcpu) && !kvm_vcpu_abt_issea(vcpu) && !kvm_vcpu_abt_iss1tw(vcpu); diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index 31d7fa4c7c140513f6b69034cafd5f23a8a88567..a3ee3b605c9b808ccd091ef8651401006ea96d26 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -1212,7 +1212,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, exec_fault = kvm_vcpu_trap_is_exec_fault(vcpu); VM_BUG_ON(write_fault && exec_fault); - if (fault_status == FSC_PERM && !write_fault && !exec_fault) { + if (fault_status == ESR_ELx_FSC_PERM && !write_fault && !exec_fault) { kvm_err("Unexpected L2 read permission error\n"); return -EFAULT; } @@ -1277,7 +1277,8 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, * only exception to this is when dirty logging is enabled at runtime * and a write fault needs to collapse a block entry into a table. */ - if (fault_status != FSC_PERM || (logging_active && write_fault)) { + if (fault_status != ESR_ELx_FSC_PERM || + (logging_active && write_fault)) { ret = kvm_mmu_topup_memory_cache(memcache, kvm_mmu_cache_min_pages(kvm)); if (ret) @@ -1342,7 +1343,8 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, * backed by a THP and thus use block mapping if possible. */ if (vma_pagesize == PAGE_SIZE && !(force_pte || device)) { - if (fault_status == FSC_PERM && fault_granule > PAGE_SIZE) + if (fault_status == ESR_ELx_FSC_PERM && + fault_granule > PAGE_SIZE) vma_pagesize = fault_granule; else vma_pagesize = transparent_hugepage_adjust(kvm, memslot, @@ -1350,7 +1352,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, &fault_ipa); } - if (fault_status != FSC_PERM && !device && kvm_has_mte(kvm)) { + if (fault_status != ESR_ELx_FSC_PERM && !device && kvm_has_mte(kvm)) { /* Check the VMM hasn't introduced a new disallowed VMA */ if (kvm_vma_mte_allowed(vma)) { sanitise_mte_tags(kvm, pfn, vma_pagesize); @@ -1376,7 +1378,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, * permissions only if vma_pagesize equals fault_granule. Otherwise, * kvm_pgtable_stage2_map() should be called to change block size. */ - if (fault_status == FSC_PERM && vma_pagesize == fault_granule) + if (fault_status == ESR_ELx_FSC_PERM && vma_pagesize == fault_granule) ret = kvm_pgtable_stage2_relax_perms(pgt, fault_ipa, prot); else ret = kvm_pgtable_stage2_map(pgt, fault_ipa, vma_pagesize, @@ -1441,7 +1443,7 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu) fault_ipa = kvm_vcpu_get_fault_ipa(vcpu); is_iabt = kvm_vcpu_trap_is_iabt(vcpu); - if (fault_status == FSC_FAULT) { + if (fault_status == ESR_ELx_FSC_FAULT) { /* Beyond sanitised PARange (which is the IPA limit) */ if (fault_ipa >= BIT_ULL(get_kvm_ipa_limit())) { kvm_inject_size_fault(vcpu); @@ -1476,8 +1478,9 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu) kvm_vcpu_get_hfar(vcpu), fault_ipa); /* Check the stage-2 fault is trans. fault or write fault */ - if (fault_status != FSC_FAULT && fault_status != FSC_PERM && - fault_status != FSC_ACCESS) { + if (fault_status != ESR_ELx_FSC_FAULT && + fault_status != ESR_ELx_FSC_PERM && + fault_status != ESR_ELx_FSC_ACCESS) { kvm_err("Unsupported FSC: EC=%#x xFSC=%#lx ESR_EL2=%#lx\n", kvm_vcpu_trap_get_class(vcpu), (unsigned long)kvm_vcpu_trap_get_fault(vcpu), @@ -1539,7 +1542,7 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu) /* Userspace should not be able to register out-of-bounds IPAs */ VM_BUG_ON(fault_ipa >= kvm_phys_size(vcpu->kvm)); - if (fault_status == FSC_ACCESS) { + if (fault_status == ESR_ELx_FSC_ACCESS) { handle_access_fault(vcpu, fault_ipa); ret = 1; goto out_unlock; diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c index d5ee52d6bf7326f9254298631ef2fc0824c0e176..c6cbfe6b854b310f74608c87136b81b5691ef51f 100644 --- a/arch/arm64/kvm/sys_regs.c +++ b/arch/arm64/kvm/sys_regs.c @@ -646,7 +646,7 @@ static void reset_pmcr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r) return; /* Only preserve PMCR_EL0.N, and reset the rest to 0 */ - pmcr = read_sysreg(pmcr_el0) & ARMV8_PMU_PMCR_N_MASK; + pmcr = read_sysreg(pmcr_el0) & (ARMV8_PMU_PMCR_N_MASK << ARMV8_PMU_PMCR_N_SHIFT); if (!kvm_supports_32bit_el0()) pmcr |= ARMV8_PMU_PMCR_LC; diff --git a/arch/arm64/kvm/vgic/vgic-v3.c b/arch/arm64/kvm/vgic/vgic-v3.c index 826ff6f2a4e7b772fa4c51a3eb901399b1d03b0f..2624963cb95b3a7709989ca1de2748a7d658c0ac 100644 --- a/arch/arm64/kvm/vgic/vgic-v3.c +++ b/arch/arm64/kvm/vgic/vgic-v3.c @@ -350,26 +350,23 @@ int vgic_v3_lpi_sync_pending_status(struct kvm *kvm, struct vgic_irq *irq) * The deactivation of the doorbell interrupt will trigger the * unmapping of the associated vPE. */ -static void unmap_all_vpes(struct vgic_dist *dist) +static void unmap_all_vpes(struct kvm *kvm) { - struct irq_desc *desc; + struct vgic_dist *dist = &kvm->arch.vgic; int i; - for (i = 0; i < dist->its_vm.nr_vpes; i++) { - desc = irq_to_desc(dist->its_vm.vpes[i]->irq); - irq_domain_deactivate_irq(irq_desc_get_irq_data(desc)); - } + for (i = 0; i < dist->its_vm.nr_vpes; i++) + free_irq(dist->its_vm.vpes[i]->irq, kvm_get_vcpu(kvm, i)); } -static void map_all_vpes(struct vgic_dist *dist) +static void map_all_vpes(struct kvm *kvm) { - struct irq_desc *desc; + struct vgic_dist *dist = &kvm->arch.vgic; int i; - for (i = 0; i < dist->its_vm.nr_vpes; i++) { - desc = irq_to_desc(dist->its_vm.vpes[i]->irq); - irq_domain_activate_irq(irq_desc_get_irq_data(desc), false); - } + for (i = 0; i < dist->its_vm.nr_vpes; i++) + WARN_ON(vgic_v4_request_vpe_irq(kvm_get_vcpu(kvm, i), + dist->its_vm.vpes[i]->irq)); } /** @@ -394,7 +391,7 @@ int vgic_v3_save_pending_tables(struct kvm *kvm) * and enabling of the doorbells have already been done. */ if (kvm_vgic_global_state.has_gicv4_1) { - unmap_all_vpes(dist); + unmap_all_vpes(kvm); vlpi_avail = true; } @@ -444,7 +441,7 @@ int vgic_v3_save_pending_tables(struct kvm *kvm) out: if (vlpi_avail) - map_all_vpes(dist); + map_all_vpes(kvm); return ret; } @@ -616,6 +613,8 @@ static const struct midr_range broken_seis[] = { MIDR_ALL_VERSIONS(MIDR_APPLE_M1_FIRESTORM_PRO), MIDR_ALL_VERSIONS(MIDR_APPLE_M1_ICESTORM_MAX), MIDR_ALL_VERSIONS(MIDR_APPLE_M1_FIRESTORM_MAX), + MIDR_ALL_VERSIONS(MIDR_APPLE_M2_BLIZZARD), + MIDR_ALL_VERSIONS(MIDR_APPLE_M2_AVALANCHE), {}, }; diff --git a/arch/arm64/kvm/vgic/vgic-v4.c b/arch/arm64/kvm/vgic/vgic-v4.c index ad06ba6c9b0091c911c84903d455cfa53e056b8e..a413718be92b8703c18ef8dacfe3dcfb1e3027a0 100644 --- a/arch/arm64/kvm/vgic/vgic-v4.c +++ b/arch/arm64/kvm/vgic/vgic-v4.c @@ -222,6 +222,11 @@ void vgic_v4_get_vlpi_state(struct vgic_irq *irq, bool *val) *val = !!(*ptr & mask); } +int vgic_v4_request_vpe_irq(struct kvm_vcpu *vcpu, int irq) +{ + return request_irq(irq, vgic_v4_doorbell_handler, 0, "vcpu", vcpu); +} + /** * vgic_v4_init - Initialize the GICv4 data structures * @kvm: Pointer to the VM being initialized @@ -283,8 +288,7 @@ int vgic_v4_init(struct kvm *kvm) irq_flags &= ~IRQ_NOAUTOEN; irq_set_status_flags(irq, irq_flags); - ret = request_irq(irq, vgic_v4_doorbell_handler, - 0, "vcpu", vcpu); + ret = vgic_v4_request_vpe_irq(vcpu, irq); if (ret) { kvm_err("failed to allocate vcpu IRQ%d\n", irq); /* diff --git a/arch/arm64/kvm/vgic/vgic.h b/arch/arm64/kvm/vgic/vgic.h index 0c8da72953f06bf97b67ed69e20a560fd641171e..23e280fa0a16f58c3ac880756d6fec467605fd92 100644 --- a/arch/arm64/kvm/vgic/vgic.h +++ b/arch/arm64/kvm/vgic/vgic.h @@ -331,5 +331,6 @@ int vgic_v4_init(struct kvm *kvm); void vgic_v4_teardown(struct kvm *kvm); void vgic_v4_configure_vsgis(struct kvm *kvm); void vgic_v4_get_vlpi_state(struct vgic_irq *irq, bool *val); +int vgic_v4_request_vpe_irq(struct kvm_vcpu *vcpu, int irq); #endif diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c index 35e9a468d13e6ac68093c7516350815df5b009b5..95364e8bdc194491ce298197d1a52896af511c53 100644 --- a/arch/arm64/mm/hugetlbpage.c +++ b/arch/arm64/mm/hugetlbpage.c @@ -559,3 +559,24 @@ bool __init arch_hugetlb_valid_size(unsigned long size) { return __hugetlb_valid_size(size); } + +pte_t huge_ptep_modify_prot_start(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep) +{ + if (IS_ENABLED(CONFIG_ARM64_ERRATUM_2645198) && + cpus_have_const_cap(ARM64_WORKAROUND_2645198)) { + /* + * Break-before-make (BBM) is required for all user space mappings + * when the permission changes from executable to non-executable + * in cases where cpu is affected with errata #2645198. + */ + if (pte_user_exec(READ_ONCE(*ptep))) + return huge_ptep_clear_flush(vma, addr, ptep); + } + return huge_ptep_get_and_clear(vma->vm_mm, addr, ptep); +} + +void huge_ptep_modify_prot_commit(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep, + pte_t old_pte, pte_t pte) +{ + set_huge_pte_at(vma->vm_mm, addr, ptep, pte); +} diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 14c87e8d69d8391c995616a749c665cabf4aaddf..d77c9f56b7b435d43f906445ecaef3554530c693 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -1630,3 +1630,24 @@ static int __init prevent_bootmem_remove_init(void) } early_initcall(prevent_bootmem_remove_init); #endif + +pte_t ptep_modify_prot_start(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep) +{ + if (IS_ENABLED(CONFIG_ARM64_ERRATUM_2645198) && + cpus_have_const_cap(ARM64_WORKAROUND_2645198)) { + /* + * Break-before-make (BBM) is required for all user space mappings + * when the permission changes from executable to non-executable + * in cases where cpu is affected with errata #2645198. + */ + if (pte_user_exec(READ_ONCE(*ptep))) + return ptep_clear_flush(vma, addr, ptep); + } + return ptep_get_and_clear(vma->vm_mm, addr, ptep); +} + +void ptep_modify_prot_commit(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep, + pte_t old_pte, pte_t pte) +{ + set_pte_at(vma->vm_mm, addr, ptep, pte); +} diff --git a/arch/arm64/tools/cpucaps b/arch/arm64/tools/cpucaps index a86ee376920a08ddfdb7f116336a90ed13e88629..dfeb2c51e2573dc07d58be19e58cb89d568051ab 100644 --- a/arch/arm64/tools/cpucaps +++ b/arch/arm64/tools/cpucaps @@ -71,6 +71,7 @@ WORKAROUND_2038923 WORKAROUND_2064142 WORKAROUND_2077057 WORKAROUND_2457168 +WORKAROUND_2645198 WORKAROUND_2658417 WORKAROUND_TRBE_OVERWRITE_FILL_MODE WORKAROUND_TSB_FLUSH_FAILURE diff --git a/arch/ia64/kernel/elfcore.c b/arch/ia64/kernel/elfcore.c index 94680521fbf919cf736b96ec1264dc9e50e9d92b..8895df12154047913fd7099fa4bdf487e3ade8d3 100644 --- a/arch/ia64/kernel/elfcore.c +++ b/arch/ia64/kernel/elfcore.c @@ -7,7 +7,7 @@ #include -Elf64_Half elf_core_extra_phdrs(void) +Elf64_Half elf_core_extra_phdrs(struct coredump_params *cprm) { return GATE_EHDR->e_phnum; } @@ -60,7 +60,7 @@ int elf_core_write_extra_data(struct coredump_params *cprm) return 1; } -size_t elf_core_extra_data_size(void) +size_t elf_core_extra_data_size(struct coredump_params *cprm) { const struct elf_phdr *const gate_phdrs = (const struct elf_phdr *) (GATE_ADDR + GATE_EHDR->e_phoff); diff --git a/arch/loongarch/include/asm/ftrace.h b/arch/loongarch/include/asm/ftrace.h index 90f9d3399b2aafe62a495e93f0735898d4b28715..3418d32d4fc7d2d24a96563132d42f2bcc012d38 100644 --- a/arch/loongarch/include/asm/ftrace.h +++ b/arch/loongarch/include/asm/ftrace.h @@ -10,8 +10,6 @@ #define FTRACE_REGS_PLT_IDX 1 #define NR_FTRACE_PLTS 2 -#define GRAPH_FAKE_OFFSET (sizeof(struct pt_regs) - offsetof(struct pt_regs, regs[1])) - #ifdef CONFIG_FUNCTION_TRACER #define MCOUNT_INSN_SIZE 4 /* sizeof mcount call */ diff --git a/arch/loongarch/include/asm/inst.h b/arch/loongarch/include/asm/inst.h index c00e1512d4fa33e49bf4b6c4627460938510de75..7eedd83fd0d72127fc570cd87dc759f1d8b6b12f 100644 --- a/arch/loongarch/include/asm/inst.h +++ b/arch/loongarch/include/asm/inst.h @@ -377,14 +377,6 @@ static inline bool unsigned_imm_check(unsigned long val, unsigned int bit) return val < (1UL << bit); } -static inline unsigned long sign_extend(unsigned long val, unsigned int idx) -{ - if (!is_imm_negative(val, idx + 1)) - return ((1UL << idx) - 1) & val; - else - return ~((1UL << idx) - 1) | val; -} - #define DEF_EMIT_REG0I26_FORMAT(NAME, OP) \ static inline void emit_##NAME(union loongarch_instruction *insn, \ int offset) \ @@ -401,6 +393,7 @@ static inline void emit_##NAME(union loongarch_instruction *insn, \ } DEF_EMIT_REG0I26_FORMAT(b, b_op) +DEF_EMIT_REG0I26_FORMAT(bl, bl_op) #define DEF_EMIT_REG1I20_FORMAT(NAME, OP) \ static inline void emit_##NAME(union loongarch_instruction *insn, \ diff --git a/arch/loongarch/include/asm/unwind.h b/arch/loongarch/include/asm/unwind.h index f2b52b9ea93d2862d69e9e7f1075f6e1bcfee619..b9dce87afd2e058a3dd786ccbb513a3bbbff637c 100644 --- a/arch/loongarch/include/asm/unwind.h +++ b/arch/loongarch/include/asm/unwind.h @@ -8,7 +8,9 @@ #define _ASM_UNWIND_H #include +#include +#include #include enum unwinder_type { @@ -20,11 +22,13 @@ struct unwind_state { char type; /* UNWINDER_XXX */ struct stack_info stack_info; struct task_struct *task; - bool first, error, is_ftrace; + bool first, error, reset; int graph_idx; unsigned long sp, pc, ra; }; +bool default_next_frame(struct unwind_state *state); + void unwind_start(struct unwind_state *state, struct task_struct *task, struct pt_regs *regs); bool unwind_next_frame(struct unwind_state *state); @@ -40,4 +44,39 @@ static inline bool unwind_error(struct unwind_state *state) return state->error; } +#define GRAPH_FAKE_OFFSET (sizeof(struct pt_regs) - offsetof(struct pt_regs, regs[1])) + +static inline unsigned long unwind_graph_addr(struct unwind_state *state, + unsigned long pc, unsigned long cfa) +{ + return ftrace_graph_ret_addr(state->task, &state->graph_idx, + pc, (unsigned long *)(cfa - GRAPH_FAKE_OFFSET)); +} + +static __always_inline void __unwind_start(struct unwind_state *state, + struct task_struct *task, struct pt_regs *regs) +{ + memset(state, 0, sizeof(*state)); + if (regs) { + state->sp = regs->regs[3]; + state->pc = regs->csr_era; + state->ra = regs->regs[1]; + } else if (task && task != current) { + state->sp = thread_saved_fp(task); + state->pc = thread_saved_ra(task); + state->ra = 0; + } else { + state->sp = (unsigned long)__builtin_frame_address(0); + state->pc = (unsigned long)__builtin_return_address(0); + state->ra = 0; + } + state->task = task; + get_stack_info(state->sp, state->task, &state->stack_info); + state->pc = unwind_graph_addr(state, state->pc, state->sp); +} + +static __always_inline unsigned long __unwind_get_return_address(struct unwind_state *state) +{ + return unwind_done(state) ? 0 : state->pc; +} #endif /* _ASM_UNWIND_H */ diff --git a/arch/loongarch/kernel/Makefile b/arch/loongarch/kernel/Makefile index fcaa024a685ec6df4a983daa59dbf2f967b038af..c8cfbd562921d63fe2a15aa6fb8a6940ff356fd4 100644 --- a/arch/loongarch/kernel/Makefile +++ b/arch/loongarch/kernel/Makefile @@ -8,7 +8,7 @@ extra-y := vmlinux.lds obj-y += head.o cpu-probe.o cacheinfo.o env.o setup.o entry.o genex.o \ traps.o irq.o idle.o process.o dma.o mem.o io.o reset.o switch.o \ elf.o syscall.o signal.o time.o topology.o inst.o ptrace.o vdso.o \ - alternative.o unaligned.o + alternative.o unaligned.o unwind.o obj-$(CONFIG_ACPI) += acpi.o obj-$(CONFIG_EFI) += efi.o diff --git a/arch/loongarch/kernel/alternative.c b/arch/loongarch/kernel/alternative.c index c5aebeac960b6e19908550185a49363c54f0768e..4ad13847e9626eab6dfa5022b0d51ff5a40459ad 100644 --- a/arch/loongarch/kernel/alternative.c +++ b/arch/loongarch/kernel/alternative.c @@ -74,7 +74,7 @@ static void __init_or_module recompute_jump(union loongarch_instruction *buf, switch (src->reg0i26_format.opcode) { case b_op: case bl_op: - jump_addr = cur_pc + sign_extend((si_h << 16 | si_l) << 2, 27); + jump_addr = cur_pc + sign_extend64((si_h << 16 | si_l) << 2, 27); if (in_alt_jump(jump_addr, start, end)) return; offset = jump_addr - pc; @@ -93,7 +93,7 @@ static void __init_or_module recompute_jump(union loongarch_instruction *buf, fallthrough; case beqz_op: case bnez_op: - jump_addr = cur_pc + sign_extend((si_h << 16 | si_l) << 2, 22); + jump_addr = cur_pc + sign_extend64((si_h << 16 | si_l) << 2, 22); if (in_alt_jump(jump_addr, start, end)) return; offset = jump_addr - pc; @@ -112,7 +112,7 @@ static void __init_or_module recompute_jump(union loongarch_instruction *buf, case bge_op: case bltu_op: case bgeu_op: - jump_addr = cur_pc + sign_extend(si << 2, 17); + jump_addr = cur_pc + sign_extend64(si << 2, 17); if (in_alt_jump(jump_addr, start, end)) return; offset = jump_addr - pc; diff --git a/arch/loongarch/kernel/cpu-probe.c b/arch/loongarch/kernel/cpu-probe.c index 255a09876ef28d1b10c67fd1b9f8ca4814f8aefc..3a3fce2d784611e1118e6a6d5dab1b509501dd9c 100644 --- a/arch/loongarch/kernel/cpu-probe.c +++ b/arch/loongarch/kernel/cpu-probe.c @@ -94,7 +94,7 @@ static void cpu_probe_common(struct cpuinfo_loongarch *c) c->options = LOONGARCH_CPU_CPUCFG | LOONGARCH_CPU_CSR | LOONGARCH_CPU_TLB | LOONGARCH_CPU_VINT | LOONGARCH_CPU_WATCH; - elf_hwcap |= HWCAP_LOONGARCH_CRC32; + elf_hwcap = HWCAP_LOONGARCH_CPUCFG | HWCAP_LOONGARCH_CRC32; config = read_cpucfg(LOONGARCH_CPUCFG1); if (config & CPUCFG1_UAL) { diff --git a/arch/loongarch/kernel/genex.S b/arch/loongarch/kernel/genex.S index 75e5be807a0d5004feb5f1e6eb42f8bcc9efef5f..7e5c293ed89f70c4872d54dfcad422673dc65112 100644 --- a/arch/loongarch/kernel/genex.S +++ b/arch/loongarch/kernel/genex.S @@ -67,14 +67,17 @@ SYM_FUNC_END(except_vec_cex) .macro BUILD_HANDLER exception handler prep .align 5 SYM_FUNC_START(handle_\exception) + 666: BACKUP_T0T1 SAVE_ALL build_prep_\prep move a0, sp la.abs t0, do_\handler jirl ra, t0, 0 + 668: RESTORE_ALL_AND_RET SYM_FUNC_END(handle_\exception) + SYM_DATA(unwind_hint_\exception, .word 668b - 666b) .endm BUILD_HANDLER ade ade badv diff --git a/arch/loongarch/kernel/inst.c b/arch/loongarch/kernel/inst.c index 512579d79b221ccb632cbff1c3140233e3a5dda0..badc590870423433495616c47e53022e4cf96f35 100644 --- a/arch/loongarch/kernel/inst.c +++ b/arch/loongarch/kernel/inst.c @@ -58,7 +58,6 @@ u32 larch_insn_gen_nop(void) u32 larch_insn_gen_b(unsigned long pc, unsigned long dest) { long offset = dest - pc; - unsigned int immediate_l, immediate_h; union loongarch_instruction insn; if ((offset & 3) || offset < -SZ_128M || offset >= SZ_128M) { @@ -66,15 +65,7 @@ u32 larch_insn_gen_b(unsigned long pc, unsigned long dest) return INSN_BREAK; } - offset >>= 2; - - immediate_l = offset & 0xffff; - offset >>= 16; - immediate_h = offset & 0x3ff; - - insn.reg0i26_format.opcode = b_op; - insn.reg0i26_format.immediate_l = immediate_l; - insn.reg0i26_format.immediate_h = immediate_h; + emit_b(&insn, offset >> 2); return insn.word; } @@ -82,7 +73,6 @@ u32 larch_insn_gen_b(unsigned long pc, unsigned long dest) u32 larch_insn_gen_bl(unsigned long pc, unsigned long dest) { long offset = dest - pc; - unsigned int immediate_l, immediate_h; union loongarch_instruction insn; if ((offset & 3) || offset < -SZ_128M || offset >= SZ_128M) { @@ -90,15 +80,7 @@ u32 larch_insn_gen_bl(unsigned long pc, unsigned long dest) return INSN_BREAK; } - offset >>= 2; - - immediate_l = offset & 0xffff; - offset >>= 16; - immediate_h = offset & 0x3ff; - - insn.reg0i26_format.opcode = bl_op; - insn.reg0i26_format.immediate_l = immediate_l; - insn.reg0i26_format.immediate_h = immediate_h; + emit_bl(&insn, offset >> 2); return insn.word; } @@ -107,10 +89,7 @@ u32 larch_insn_gen_or(enum loongarch_gpr rd, enum loongarch_gpr rj, enum loongar { union loongarch_instruction insn; - insn.reg3_format.opcode = or_op; - insn.reg3_format.rd = rd; - insn.reg3_format.rj = rj; - insn.reg3_format.rk = rk; + emit_or(&insn, rd, rj, rk); return insn.word; } @@ -124,9 +103,7 @@ u32 larch_insn_gen_lu12iw(enum loongarch_gpr rd, int imm) { union loongarch_instruction insn; - insn.reg1i20_format.opcode = lu12iw_op; - insn.reg1i20_format.rd = rd; - insn.reg1i20_format.immediate = imm; + emit_lu12iw(&insn, rd, imm); return insn.word; } @@ -135,9 +112,7 @@ u32 larch_insn_gen_lu32id(enum loongarch_gpr rd, int imm) { union loongarch_instruction insn; - insn.reg1i20_format.opcode = lu32id_op; - insn.reg1i20_format.rd = rd; - insn.reg1i20_format.immediate = imm; + emit_lu32id(&insn, rd, imm); return insn.word; } @@ -146,10 +121,7 @@ u32 larch_insn_gen_lu52id(enum loongarch_gpr rd, enum loongarch_gpr rj, int imm) { union loongarch_instruction insn; - insn.reg2i12_format.opcode = lu52id_op; - insn.reg2i12_format.rd = rd; - insn.reg2i12_format.rj = rj; - insn.reg2i12_format.immediate = imm; + emit_lu52id(&insn, rd, rj, imm); return insn.word; } @@ -158,10 +130,7 @@ u32 larch_insn_gen_jirl(enum loongarch_gpr rd, enum loongarch_gpr rj, unsigned l { union loongarch_instruction insn; - insn.reg2i16_format.opcode = jirl_op; - insn.reg2i16_format.rd = rd; - insn.reg2i16_format.rj = rj; - insn.reg2i16_format.immediate = (dest - pc) >> 2; + emit_jirl(&insn, rj, rd, (dest - pc) >> 2); return insn.word; } diff --git a/arch/loongarch/kernel/process.c b/arch/loongarch/kernel/process.c index c583b1ef1f44cefaada16d0d9c450b2019515ec9..edfd220a3737aadad190d43a2ecaf91fdcaf44e5 100644 --- a/arch/loongarch/kernel/process.c +++ b/arch/loongarch/kernel/process.c @@ -191,20 +191,14 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args) unsigned long __get_wchan(struct task_struct *task) { - unsigned long pc; + unsigned long pc = 0; struct unwind_state state; if (!try_get_task_stack(task)) return 0; - unwind_start(&state, task, NULL); - state.sp = thread_saved_fp(task); - get_stack_info(state.sp, state.task, &state.stack_info); - state.pc = thread_saved_ra(task); -#ifdef CONFIG_UNWINDER_PROLOGUE - state.type = UNWINDER_PROLOGUE; -#endif - for (; !unwind_done(&state); unwind_next_frame(&state)) { + for (unwind_start(&state, task, NULL); + !unwind_done(&state); unwind_next_frame(&state)) { pc = unwind_get_return_address(&state); if (!pc) break; diff --git a/arch/loongarch/kernel/traps.c b/arch/loongarch/kernel/traps.c index 7ea62faeeadb561362b0228d4937bedcb7466e51..c38a146a973b45beafc0e984bdff77631fbea0bb 100644 --- a/arch/loongarch/kernel/traps.c +++ b/arch/loongarch/kernel/traps.c @@ -72,9 +72,6 @@ static void show_backtrace(struct task_struct *task, const struct pt_regs *regs, if (!task) task = current; - if (user_mode(regs)) - state.type = UNWINDER_GUESS; - printk("%sCall Trace:", loglvl); for (unwind_start(&state, task, pregs); !unwind_done(&state); unwind_next_frame(&state)) { diff --git a/arch/loongarch/kernel/unwind.c b/arch/loongarch/kernel/unwind.c new file mode 100644 index 0000000000000000000000000000000000000000..a463d6961344c0899aacd72a8951b9cc5d18ea83 --- /dev/null +++ b/arch/loongarch/kernel/unwind.c @@ -0,0 +1,32 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (C) 2022-2023 Loongson Technology Corporation Limited + */ +#include +#include + +#include + +bool default_next_frame(struct unwind_state *state) +{ + struct stack_info *info = &state->stack_info; + unsigned long addr; + + if (unwind_done(state)) + return false; + + do { + for (state->sp += sizeof(unsigned long); + state->sp < info->end; state->sp += sizeof(unsigned long)) { + addr = *(unsigned long *)(state->sp); + state->pc = unwind_graph_addr(state, addr, state->sp + 8); + if (__kernel_text_address(state->pc)) + return true; + } + + state->sp = info->next_sp; + + } while (!get_stack_info(state->sp, state->task, info)); + + return false; +} diff --git a/arch/loongarch/kernel/unwind_guess.c b/arch/loongarch/kernel/unwind_guess.c index e2d2e4f3001f490a76762ad336fbcf696c4bbfe3..98379b7d41475b89bb85f1c4d32be29343612cc8 100644 --- a/arch/loongarch/kernel/unwind_guess.c +++ b/arch/loongarch/kernel/unwind_guess.c @@ -2,37 +2,18 @@ /* * Copyright (C) 2022 Loongson Technology Corporation Limited */ -#include -#include - #include unsigned long unwind_get_return_address(struct unwind_state *state) { - if (unwind_done(state)) - return 0; - else if (state->first) - return state->pc; - - return *(unsigned long *)(state->sp); + return __unwind_get_return_address(state); } EXPORT_SYMBOL_GPL(unwind_get_return_address); void unwind_start(struct unwind_state *state, struct task_struct *task, struct pt_regs *regs) { - memset(state, 0, sizeof(*state)); - - if (regs) { - state->sp = regs->regs[3]; - state->pc = regs->csr_era; - } - - state->task = task; - state->first = true; - - get_stack_info(state->sp, state->task, &state->stack_info); - + __unwind_start(state, task, regs); if (!unwind_done(state) && !__kernel_text_address(state->pc)) unwind_next_frame(state); } @@ -40,30 +21,6 @@ EXPORT_SYMBOL_GPL(unwind_start); bool unwind_next_frame(struct unwind_state *state) { - struct stack_info *info = &state->stack_info; - unsigned long addr; - - if (unwind_done(state)) - return false; - - if (state->first) - state->first = false; - - do { - for (state->sp += sizeof(unsigned long); - state->sp < info->end; - state->sp += sizeof(unsigned long)) { - addr = *(unsigned long *)(state->sp); - state->pc = ftrace_graph_ret_addr(state->task, &state->graph_idx, - addr, (unsigned long *)(state->sp - GRAPH_FAKE_OFFSET)); - if (__kernel_text_address(addr)) - return true; - } - - state->sp = info->next_sp; - - } while (!get_stack_info(state->sp, state->task, info)); - - return false; + return default_next_frame(state); } EXPORT_SYMBOL_GPL(unwind_next_frame); diff --git a/arch/loongarch/kernel/unwind_prologue.c b/arch/loongarch/kernel/unwind_prologue.c index 0f8d1451ebb848d8a88f8a5c9405903a8d4c93c0..9095fde8e55d5c57177c83ab0548ab888c348f16 100644 --- a/arch/loongarch/kernel/unwind_prologue.c +++ b/arch/loongarch/kernel/unwind_prologue.c @@ -2,61 +2,116 @@ /* * Copyright (C) 2022 Loongson Technology Corporation Limited */ +#include #include #include #include +#include #include +#include #include -static inline void unwind_state_fixup(struct unwind_state *state) -{ -#ifdef CONFIG_DYNAMIC_FTRACE - static unsigned long ftrace = (unsigned long)ftrace_call + 4; - - if (state->pc == ftrace) - state->is_ftrace = true; +extern const int unwind_hint_ade; +extern const int unwind_hint_ale; +extern const int unwind_hint_bp; +extern const int unwind_hint_fpe; +extern const int unwind_hint_fpu; +extern const int unwind_hint_lsx; +extern const int unwind_hint_lasx; +extern const int unwind_hint_lbt; +extern const int unwind_hint_ri; +extern const int unwind_hint_watch; +extern unsigned long eentry; +#ifdef CONFIG_NUMA +extern unsigned long pcpu_handlers[NR_CPUS]; #endif -} -unsigned long unwind_get_return_address(struct unwind_state *state) +static inline bool scan_handlers(unsigned long entry_offset) { + int idx, offset; - if (unwind_done(state)) - return 0; - else if (state->type) - return state->pc; - else if (state->first) - return state->pc; - - return *(unsigned long *)(state->sp); + if (entry_offset >= EXCCODE_INT_START * VECSIZE) + return false; + idx = entry_offset / VECSIZE; + offset = entry_offset % VECSIZE; + switch (idx) { + case EXCCODE_ADE: + return offset == unwind_hint_ade; + case EXCCODE_ALE: + return offset == unwind_hint_ale; + case EXCCODE_BP: + return offset == unwind_hint_bp; + case EXCCODE_FPE: + return offset == unwind_hint_fpe; + case EXCCODE_FPDIS: + return offset == unwind_hint_fpu; + case EXCCODE_LSXDIS: + return offset == unwind_hint_lsx; + case EXCCODE_LASXDIS: + return offset == unwind_hint_lasx; + case EXCCODE_BTDIS: + return offset == unwind_hint_lbt; + case EXCCODE_INE: + return offset == unwind_hint_ri; + case EXCCODE_WATCH: + return offset == unwind_hint_watch; + default: + return false; + } } -EXPORT_SYMBOL_GPL(unwind_get_return_address); -static bool unwind_by_guess(struct unwind_state *state) +static inline bool fix_exception(unsigned long pc) { - struct stack_info *info = &state->stack_info; - unsigned long addr; - - for (state->sp += sizeof(unsigned long); - state->sp < info->end; - state->sp += sizeof(unsigned long)) { - addr = *(unsigned long *)(state->sp); - state->pc = ftrace_graph_ret_addr(state->task, &state->graph_idx, - addr, (unsigned long *)(state->sp - GRAPH_FAKE_OFFSET)); - if (__kernel_text_address(addr)) +#ifdef CONFIG_NUMA + int cpu; + + for_each_possible_cpu(cpu) { + if (!pcpu_handlers[cpu]) + continue; + if (scan_handlers(pc - pcpu_handlers[cpu])) return true; } +#endif + return scan_handlers(pc - eentry); +} +/* + * As we meet ftrace_regs_entry, reset first flag like first doing + * tracing. Prologue analysis will stop soon because PC is at entry. + */ +static inline bool fix_ftrace(unsigned long pc) +{ +#ifdef CONFIG_DYNAMIC_FTRACE + return pc == (unsigned long)ftrace_call + LOONGARCH_INSN_SIZE; +#else return false; +#endif } +static inline bool unwind_state_fixup(struct unwind_state *state) +{ + if (!fix_exception(state->pc) && !fix_ftrace(state->pc)) + return false; + + state->reset = true; + return true; +} + +/* + * LoongArch function prologue is like follows, + * [instructions not use stack var] + * addi.d sp, sp, -imm + * st.d xx, sp, offset <- save callee saved regs and + * st.d yy, sp, offset save ra if function is nest. + * [others instructions] + */ static bool unwind_by_prologue(struct unwind_state *state) { long frame_ra = -1; unsigned long frame_size = 0; - unsigned long size, offset, pc = state->pc; + unsigned long size, offset, pc; struct pt_regs *regs; struct stack_info *info = &state->stack_info; union loongarch_instruction *ip, *ip_end; @@ -64,20 +119,21 @@ static bool unwind_by_prologue(struct unwind_state *state) if (state->sp >= info->end || state->sp < info->begin) return false; - if (state->is_ftrace) { - /* - * As we meet ftrace_regs_entry, reset first flag like first doing - * tracing. Prologue analysis will stop soon because PC is at entry. - */ + if (state->reset) { regs = (struct pt_regs *)state->sp; state->first = true; - state->is_ftrace = false; + state->reset = false; state->pc = regs->csr_era; state->ra = regs->regs[1]; state->sp = regs->regs[3]; return true; } + /* + * When first is not set, the PC is a return address in the previous frame. + * We need to adjust its value in case overflow to the next symbol. + */ + pc = state->pc - (state->first ? 0 : LOONGARCH_INSN_SIZE); if (!kallsyms_lookup_size_offset(pc, &size, &offset)) return false; @@ -93,6 +149,10 @@ static bool unwind_by_prologue(struct unwind_state *state) ip++; } + /* + * Can't find stack alloc action, PC may be in a leaf function. Only the + * first being true is reasonable, otherwise indicate analysis is broken. + */ if (!frame_size) { if (state->first) goto first; @@ -110,6 +170,7 @@ static bool unwind_by_prologue(struct unwind_state *state) ip++; } + /* Can't find save $ra action, PC may be in a leaf function, too. */ if (frame_ra < 0) { if (state->first) { state->sp = state->sp + frame_size; @@ -118,88 +179,47 @@ static bool unwind_by_prologue(struct unwind_state *state) return false; } - if (state->first) - state->first = false; - state->pc = *(unsigned long *)(state->sp + frame_ra); state->sp = state->sp + frame_size; goto out; first: - state->first = false; - if (state->pc == state->ra) - return false; - state->pc = state->ra; out: - unwind_state_fixup(state); - return !!__kernel_text_address(state->pc); -} - -void unwind_start(struct unwind_state *state, struct task_struct *task, - struct pt_regs *regs) -{ - memset(state, 0, sizeof(*state)); - - if (regs && __kernel_text_address(regs->csr_era)) { - state->pc = regs->csr_era; - state->sp = regs->regs[3]; - state->ra = regs->regs[1]; - state->type = UNWINDER_PROLOGUE; - } - - state->task = task; - state->first = true; - - get_stack_info(state->sp, state->task, &state->stack_info); - - if (!unwind_done(state) && !__kernel_text_address(state->pc)) - unwind_next_frame(state); + state->first = false; + return unwind_state_fixup(state) || __kernel_text_address(state->pc); } -EXPORT_SYMBOL_GPL(unwind_start); -bool unwind_next_frame(struct unwind_state *state) +static bool next_frame(struct unwind_state *state) { - struct stack_info *info = &state->stack_info; - struct pt_regs *regs; unsigned long pc; + struct pt_regs *regs; + struct stack_info *info = &state->stack_info; if (unwind_done(state)) return false; do { - switch (state->type) { - case UNWINDER_GUESS: - state->first = false; - if (unwind_by_guess(state)) - return true; - break; + if (unwind_by_prologue(state)) { + state->pc = unwind_graph_addr(state, state->pc, state->sp); + return true; + } + + if (info->type == STACK_TYPE_IRQ && info->end == state->sp) { + regs = (struct pt_regs *)info->next_sp; + pc = regs->csr_era; + + if (user_mode(regs) || !__kernel_text_address(pc)) + return false; + + state->first = true; + state->pc = pc; + state->ra = regs->regs[1]; + state->sp = regs->regs[3]; + get_stack_info(state->sp, state->task, info); - case UNWINDER_PROLOGUE: - if (unwind_by_prologue(state)) { - state->pc = ftrace_graph_ret_addr(state->task, &state->graph_idx, - state->pc, (unsigned long *)(state->sp - GRAPH_FAKE_OFFSET)); - return true; - } - - if (info->type == STACK_TYPE_IRQ && - info->end == state->sp) { - regs = (struct pt_regs *)info->next_sp; - pc = regs->csr_era; - - if (user_mode(regs) || !__kernel_text_address(pc)) - return false; - - state->first = true; - state->ra = regs->regs[1]; - state->sp = regs->regs[3]; - state->pc = ftrace_graph_ret_addr(state->task, &state->graph_idx, - pc, (unsigned long *)(state->sp - GRAPH_FAKE_OFFSET)); - get_stack_info(state->sp, state->task, info); - - return true; - } + return true; } state->sp = info->next_sp; @@ -208,4 +228,36 @@ bool unwind_next_frame(struct unwind_state *state) return false; } + +unsigned long unwind_get_return_address(struct unwind_state *state) +{ + return __unwind_get_return_address(state); +} +EXPORT_SYMBOL_GPL(unwind_get_return_address); + +void unwind_start(struct unwind_state *state, struct task_struct *task, + struct pt_regs *regs) +{ + __unwind_start(state, task, regs); + state->type = UNWINDER_PROLOGUE; + state->first = true; + + /* + * The current PC is not kernel text address, we cannot find its + * relative symbol. Thus, prologue analysis will be broken. Luckily, + * we can use the default_next_frame(). + */ + if (!__kernel_text_address(state->pc)) { + state->type = UNWINDER_GUESS; + if (!unwind_done(state)) + unwind_next_frame(state); + } +} +EXPORT_SYMBOL_GPL(unwind_start); + +bool unwind_next_frame(struct unwind_state *state) +{ + return state->type == UNWINDER_PROLOGUE ? + next_frame(state) : default_next_frame(state); +} EXPORT_SYMBOL_GPL(unwind_next_frame); diff --git a/arch/loongarch/mm/tlb.c b/arch/loongarch/mm/tlb.c index da3681f131c8d8a722b7f479775f7a48be47fc0a..8bad6b0cff59b96b2ebdda34214fdc1e4758e27e 100644 --- a/arch/loongarch/mm/tlb.c +++ b/arch/loongarch/mm/tlb.c @@ -251,7 +251,7 @@ static void output_pgtable_bits_defines(void) } #ifdef CONFIG_NUMA -static unsigned long pcpu_handlers[NR_CPUS]; +unsigned long pcpu_handlers[NR_CPUS]; #endif extern long exception_handlers[VECSIZE * 128 / sizeof(long)]; diff --git a/arch/mips/ralink/of.c b/arch/mips/ralink/of.c index 01c132bc33d540fd0d7aad9f205db23f50763986..4d06de77d92a69a80af6a1e996b9d8a47a2e1f0d 100644 --- a/arch/mips/ralink/of.c +++ b/arch/mips/ralink/of.c @@ -64,7 +64,7 @@ void __init plat_mem_setup(void) dtb = get_fdt(); __dt_setup_arch(dtb); - if (!early_init_dt_scan_memory()) + if (early_init_dt_scan_memory()) return; if (soc_info.mem_detect) diff --git a/arch/powerpc/boot/dts/fsl/t2081si-post.dtsi b/arch/powerpc/boot/dts/fsl/t2081si-post.dtsi index 74e17e134387d6b97b86b1beb047ff1befc723ee..27714dc2f04a541a739c4dc97bcf2b4fe78265d7 100644 --- a/arch/powerpc/boot/dts/fsl/t2081si-post.dtsi +++ b/arch/powerpc/boot/dts/fsl/t2081si-post.dtsi @@ -659,3 +659,19 @@ interrupts = <16 2 1 9>; }; }; + +&fman0_rx_0x08 { + /delete-property/ fsl,fman-10g-port; +}; + +&fman0_tx_0x28 { + /delete-property/ fsl,fman-10g-port; +}; + +&fman0_rx_0x09 { + /delete-property/ fsl,fman-10g-port; +}; + +&fman0_tx_0x29 { + /delete-property/ fsl,fman-10g-port; +}; diff --git a/arch/powerpc/boot/wrapper b/arch/powerpc/boot/wrapper index af04cea82b941f8174e5e9a5c7fb01e4b9ac55b9..352d7de24018fbd4677e663a62afae46b1a52743 100755 --- a/arch/powerpc/boot/wrapper +++ b/arch/powerpc/boot/wrapper @@ -210,6 +210,10 @@ ld_version() gsub(".*version ", ""); gsub("-.*", ""); split($1,a, "."); + if( length(a[3]) == "8" ) + # a[3] is probably a date of format yyyymmdd used for release snapshots. We + # can assume it to be zero as it does not signify a new version as such. + a[3] = 0; print a[1]*100000000 + a[2]*1000000 + a[3]*10000; exit }' diff --git a/arch/powerpc/include/asm/imc-pmu.h b/arch/powerpc/include/asm/imc-pmu.h index 4f897993b7107537c25483df2d5cfd8613a4bd81..699a88584ae16f194c7c7ccf4f8212af9977a876 100644 --- a/arch/powerpc/include/asm/imc-pmu.h +++ b/arch/powerpc/include/asm/imc-pmu.h @@ -137,7 +137,7 @@ struct imc_pmu { * are inited. */ struct imc_pmu_ref { - struct mutex lock; + spinlock_t lock; unsigned int id; int refc; }; diff --git a/arch/powerpc/kernel/vmlinux.lds.S b/arch/powerpc/kernel/vmlinux.lds.S index 8c3862b4c259d6a2b89d40cfd712ce4d2f974179..958e77a24f85bb2ce8a25f6cf204a27e70ffd883 100644 --- a/arch/powerpc/kernel/vmlinux.lds.S +++ b/arch/powerpc/kernel/vmlinux.lds.S @@ -8,6 +8,7 @@ #define BSS_FIRST_SECTIONS *(.bss.prominit) #define EMITS_PT_NOTE #define RO_EXCEPTION_TABLE_ALIGN 0 +#define RUNTIME_DISCARD_EXIT #define SOFT_MASK_TABLE(align) \ . = ALIGN(align); \ @@ -410,9 +411,12 @@ SECTIONS DISCARDS /DISCARD/ : { *(*.EMB.apuinfo) - *(.glink .iplt .plt .rela* .comment) + *(.glink .iplt .plt) *(.gnu.version*) *(.gnu.attributes) *(.eh_frame) +#ifndef CONFIG_RELOCATABLE + *(.rela*) +#endif } } diff --git a/arch/powerpc/mm/book3s64/hash_utils.c b/arch/powerpc/mm/book3s64/hash_utils.c index 80a148c57de8137aba48022373d3475a567c219f..44a35ed4f68605490bb2950d1ba0f0724e2f8e35 100644 --- a/arch/powerpc/mm/book3s64/hash_utils.c +++ b/arch/powerpc/mm/book3s64/hash_utils.c @@ -1012,7 +1012,7 @@ static void __init hash_init_partition_table(phys_addr_t hash_table, void hpt_clear_stress(void); static struct timer_list stress_hpt_timer; -void stress_hpt_timer_fn(struct timer_list *timer) +static void stress_hpt_timer_fn(struct timer_list *timer) { int next_cpu; diff --git a/arch/powerpc/perf/imc-pmu.c b/arch/powerpc/perf/imc-pmu.c index d517aba94d1bc4cb8e176700e2b29894a71a19fa..100e97daf76ba140de318eb029cc9f69cba2df3d 100644 --- a/arch/powerpc/perf/imc-pmu.c +++ b/arch/powerpc/perf/imc-pmu.c @@ -14,6 +14,7 @@ #include #include #include +#include /* Nest IMC data structures and variables */ @@ -21,7 +22,7 @@ * Used to avoid races in counting the nest-pmu units during hotplug * register and unregister */ -static DEFINE_MUTEX(nest_init_lock); +static DEFINE_SPINLOCK(nest_init_lock); static DEFINE_PER_CPU(struct imc_pmu_ref *, local_nest_imc_refc); static struct imc_pmu **per_nest_pmu_arr; static cpumask_t nest_imc_cpumask; @@ -50,7 +51,7 @@ static int trace_imc_mem_size; * core and trace-imc */ static struct imc_pmu_ref imc_global_refc = { - .lock = __MUTEX_INITIALIZER(imc_global_refc.lock), + .lock = __SPIN_LOCK_INITIALIZER(imc_global_refc.lock), .id = 0, .refc = 0, }; @@ -400,7 +401,7 @@ static int ppc_nest_imc_cpu_offline(unsigned int cpu) get_hard_smp_processor_id(cpu)); /* * If this is the last cpu in this chip then, skip the reference - * count mutex lock and make the reference count on this chip zero. + * count lock and make the reference count on this chip zero. */ ref = get_nest_pmu_ref(cpu); if (!ref) @@ -462,15 +463,15 @@ static void nest_imc_counters_release(struct perf_event *event) /* * See if we need to disable the nest PMU. * If no events are currently in use, then we have to take a - * mutex to ensure that we don't race with another task doing + * lock to ensure that we don't race with another task doing * enable or disable the nest counters. */ ref = get_nest_pmu_ref(event->cpu); if (!ref) return; - /* Take the mutex lock for this node and then decrement the reference count */ - mutex_lock(&ref->lock); + /* Take the lock for this node and then decrement the reference count */ + spin_lock(&ref->lock); if (ref->refc == 0) { /* * The scenario where this is true is, when perf session is @@ -482,7 +483,7 @@ static void nest_imc_counters_release(struct perf_event *event) * an OPAL call to disable the engine in that node. * */ - mutex_unlock(&ref->lock); + spin_unlock(&ref->lock); return; } ref->refc--; @@ -490,7 +491,7 @@ static void nest_imc_counters_release(struct perf_event *event) rc = opal_imc_counters_stop(OPAL_IMC_COUNTERS_NEST, get_hard_smp_processor_id(event->cpu)); if (rc) { - mutex_unlock(&ref->lock); + spin_unlock(&ref->lock); pr_err("nest-imc: Unable to stop the counters for core %d\n", node_id); return; } @@ -498,7 +499,7 @@ static void nest_imc_counters_release(struct perf_event *event) WARN(1, "nest-imc: Invalid event reference count\n"); ref->refc = 0; } - mutex_unlock(&ref->lock); + spin_unlock(&ref->lock); } static int nest_imc_event_init(struct perf_event *event) @@ -557,26 +558,25 @@ static int nest_imc_event_init(struct perf_event *event) /* * Get the imc_pmu_ref struct for this node. - * Take the mutex lock and then increment the count of nest pmu events - * inited. + * Take the lock and then increment the count of nest pmu events inited. */ ref = get_nest_pmu_ref(event->cpu); if (!ref) return -EINVAL; - mutex_lock(&ref->lock); + spin_lock(&ref->lock); if (ref->refc == 0) { rc = opal_imc_counters_start(OPAL_IMC_COUNTERS_NEST, get_hard_smp_processor_id(event->cpu)); if (rc) { - mutex_unlock(&ref->lock); + spin_unlock(&ref->lock); pr_err("nest-imc: Unable to start the counters for node %d\n", node_id); return rc; } } ++ref->refc; - mutex_unlock(&ref->lock); + spin_unlock(&ref->lock); event->destroy = nest_imc_counters_release; return 0; @@ -612,9 +612,8 @@ static int core_imc_mem_init(int cpu, int size) return -ENOMEM; mem_info->vbase = page_address(page); - /* Init the mutex */ core_imc_refc[core_id].id = core_id; - mutex_init(&core_imc_refc[core_id].lock); + spin_lock_init(&core_imc_refc[core_id].lock); rc = opal_imc_counters_init(OPAL_IMC_COUNTERS_CORE, __pa((void *)mem_info->vbase), @@ -703,9 +702,8 @@ static int ppc_core_imc_cpu_offline(unsigned int cpu) perf_pmu_migrate_context(&core_imc_pmu->pmu, cpu, ncpu); } else { /* - * If this is the last cpu in this core then, skip taking refernce - * count mutex lock for this core and directly zero "refc" for - * this core. + * If this is the last cpu in this core then skip taking reference + * count lock for this core and directly zero "refc" for this core. */ opal_imc_counters_stop(OPAL_IMC_COUNTERS_CORE, get_hard_smp_processor_id(cpu)); @@ -720,11 +718,11 @@ static int ppc_core_imc_cpu_offline(unsigned int cpu) * last cpu in this core and core-imc event running * in this cpu. */ - mutex_lock(&imc_global_refc.lock); + spin_lock(&imc_global_refc.lock); if (imc_global_refc.id == IMC_DOMAIN_CORE) imc_global_refc.refc--; - mutex_unlock(&imc_global_refc.lock); + spin_unlock(&imc_global_refc.lock); } return 0; } @@ -739,7 +737,7 @@ static int core_imc_pmu_cpumask_init(void) static void reset_global_refc(struct perf_event *event) { - mutex_lock(&imc_global_refc.lock); + spin_lock(&imc_global_refc.lock); imc_global_refc.refc--; /* @@ -751,7 +749,7 @@ static void reset_global_refc(struct perf_event *event) imc_global_refc.refc = 0; imc_global_refc.id = 0; } - mutex_unlock(&imc_global_refc.lock); + spin_unlock(&imc_global_refc.lock); } static void core_imc_counters_release(struct perf_event *event) @@ -764,17 +762,17 @@ static void core_imc_counters_release(struct perf_event *event) /* * See if we need to disable the IMC PMU. * If no events are currently in use, then we have to take a - * mutex to ensure that we don't race with another task doing + * lock to ensure that we don't race with another task doing * enable or disable the core counters. */ core_id = event->cpu / threads_per_core; - /* Take the mutex lock and decrement the refernce count for this core */ + /* Take the lock and decrement the refernce count for this core */ ref = &core_imc_refc[core_id]; if (!ref) return; - mutex_lock(&ref->lock); + spin_lock(&ref->lock); if (ref->refc == 0) { /* * The scenario where this is true is, when perf session is @@ -786,7 +784,7 @@ static void core_imc_counters_release(struct perf_event *event) * an OPAL call to disable the engine in that core. * */ - mutex_unlock(&ref->lock); + spin_unlock(&ref->lock); return; } ref->refc--; @@ -794,7 +792,7 @@ static void core_imc_counters_release(struct perf_event *event) rc = opal_imc_counters_stop(OPAL_IMC_COUNTERS_CORE, get_hard_smp_processor_id(event->cpu)); if (rc) { - mutex_unlock(&ref->lock); + spin_unlock(&ref->lock); pr_err("IMC: Unable to stop the counters for core %d\n", core_id); return; } @@ -802,7 +800,7 @@ static void core_imc_counters_release(struct perf_event *event) WARN(1, "core-imc: Invalid event reference count\n"); ref->refc = 0; } - mutex_unlock(&ref->lock); + spin_unlock(&ref->lock); reset_global_refc(event); } @@ -840,7 +838,6 @@ static int core_imc_event_init(struct perf_event *event) if ((!pcmi->vbase)) return -ENODEV; - /* Get the core_imc mutex for this core */ ref = &core_imc_refc[core_id]; if (!ref) return -EINVAL; @@ -848,22 +845,22 @@ static int core_imc_event_init(struct perf_event *event) /* * Core pmu units are enabled only when it is used. * See if this is triggered for the first time. - * If yes, take the mutex lock and enable the core counters. + * If yes, take the lock and enable the core counters. * If not, just increment the count in core_imc_refc struct. */ - mutex_lock(&ref->lock); + spin_lock(&ref->lock); if (ref->refc == 0) { rc = opal_imc_counters_start(OPAL_IMC_COUNTERS_CORE, get_hard_smp_processor_id(event->cpu)); if (rc) { - mutex_unlock(&ref->lock); + spin_unlock(&ref->lock); pr_err("core-imc: Unable to start the counters for core %d\n", core_id); return rc; } } ++ref->refc; - mutex_unlock(&ref->lock); + spin_unlock(&ref->lock); /* * Since the system can run either in accumulation or trace-mode @@ -874,7 +871,7 @@ static int core_imc_event_init(struct perf_event *event) * to know whether any other trace/thread imc * events are running. */ - mutex_lock(&imc_global_refc.lock); + spin_lock(&imc_global_refc.lock); if (imc_global_refc.id == 0 || imc_global_refc.id == IMC_DOMAIN_CORE) { /* * No other trace/thread imc events are running in @@ -883,10 +880,10 @@ static int core_imc_event_init(struct perf_event *event) imc_global_refc.id = IMC_DOMAIN_CORE; imc_global_refc.refc++; } else { - mutex_unlock(&imc_global_refc.lock); + spin_unlock(&imc_global_refc.lock); return -EBUSY; } - mutex_unlock(&imc_global_refc.lock); + spin_unlock(&imc_global_refc.lock); event->hw.event_base = (u64)pcmi->vbase + (config & IMC_EVENT_OFFSET_MASK); event->destroy = core_imc_counters_release; @@ -958,10 +955,10 @@ static int ppc_thread_imc_cpu_offline(unsigned int cpu) mtspr(SPRN_LDBAR, (mfspr(SPRN_LDBAR) & (~(1UL << 63)))); /* Reduce the refc if thread-imc event running on this cpu */ - mutex_lock(&imc_global_refc.lock); + spin_lock(&imc_global_refc.lock); if (imc_global_refc.id == IMC_DOMAIN_THREAD) imc_global_refc.refc--; - mutex_unlock(&imc_global_refc.lock); + spin_unlock(&imc_global_refc.lock); return 0; } @@ -1001,7 +998,7 @@ static int thread_imc_event_init(struct perf_event *event) if (!target) return -EINVAL; - mutex_lock(&imc_global_refc.lock); + spin_lock(&imc_global_refc.lock); /* * Check if any other trace/core imc events are running in the * system, if not set the global id to thread-imc. @@ -1010,10 +1007,10 @@ static int thread_imc_event_init(struct perf_event *event) imc_global_refc.id = IMC_DOMAIN_THREAD; imc_global_refc.refc++; } else { - mutex_unlock(&imc_global_refc.lock); + spin_unlock(&imc_global_refc.lock); return -EBUSY; } - mutex_unlock(&imc_global_refc.lock); + spin_unlock(&imc_global_refc.lock); event->pmu->task_ctx_nr = perf_sw_context; event->destroy = reset_global_refc; @@ -1135,25 +1132,25 @@ static int thread_imc_event_add(struct perf_event *event, int flags) /* * imc pmus are enabled only when it is used. * See if this is triggered for the first time. - * If yes, take the mutex lock and enable the counters. + * If yes, take the lock and enable the counters. * If not, just increment the count in ref count struct. */ ref = &core_imc_refc[core_id]; if (!ref) return -EINVAL; - mutex_lock(&ref->lock); + spin_lock(&ref->lock); if (ref->refc == 0) { if (opal_imc_counters_start(OPAL_IMC_COUNTERS_CORE, get_hard_smp_processor_id(smp_processor_id()))) { - mutex_unlock(&ref->lock); + spin_unlock(&ref->lock); pr_err("thread-imc: Unable to start the counter\ for core %d\n", core_id); return -EINVAL; } } ++ref->refc; - mutex_unlock(&ref->lock); + spin_unlock(&ref->lock); return 0; } @@ -1170,12 +1167,12 @@ static void thread_imc_event_del(struct perf_event *event, int flags) return; } - mutex_lock(&ref->lock); + spin_lock(&ref->lock); ref->refc--; if (ref->refc == 0) { if (opal_imc_counters_stop(OPAL_IMC_COUNTERS_CORE, get_hard_smp_processor_id(smp_processor_id()))) { - mutex_unlock(&ref->lock); + spin_unlock(&ref->lock); pr_err("thread-imc: Unable to stop the counters\ for core %d\n", core_id); return; @@ -1183,7 +1180,7 @@ static void thread_imc_event_del(struct perf_event *event, int flags) } else if (ref->refc < 0) { ref->refc = 0; } - mutex_unlock(&ref->lock); + spin_unlock(&ref->lock); /* Set bit 0 of LDBAR to zero, to stop posting updates to memory */ mtspr(SPRN_LDBAR, (mfspr(SPRN_LDBAR) & (~(1UL << 63)))); @@ -1224,9 +1221,8 @@ static int trace_imc_mem_alloc(int cpu_id, int size) } } - /* Init the mutex, if not already */ trace_imc_refc[core_id].id = core_id; - mutex_init(&trace_imc_refc[core_id].lock); + spin_lock_init(&trace_imc_refc[core_id].lock); mtspr(SPRN_LDBAR, 0); return 0; @@ -1246,10 +1242,10 @@ static int ppc_trace_imc_cpu_offline(unsigned int cpu) * Reduce the refc if any trace-imc event running * on this cpu. */ - mutex_lock(&imc_global_refc.lock); + spin_lock(&imc_global_refc.lock); if (imc_global_refc.id == IMC_DOMAIN_TRACE) imc_global_refc.refc--; - mutex_unlock(&imc_global_refc.lock); + spin_unlock(&imc_global_refc.lock); return 0; } @@ -1371,17 +1367,17 @@ static int trace_imc_event_add(struct perf_event *event, int flags) } mtspr(SPRN_LDBAR, ldbar_value); - mutex_lock(&ref->lock); + spin_lock(&ref->lock); if (ref->refc == 0) { if (opal_imc_counters_start(OPAL_IMC_COUNTERS_TRACE, get_hard_smp_processor_id(smp_processor_id()))) { - mutex_unlock(&ref->lock); + spin_unlock(&ref->lock); pr_err("trace-imc: Unable to start the counters for core %d\n", core_id); return -EINVAL; } } ++ref->refc; - mutex_unlock(&ref->lock); + spin_unlock(&ref->lock); return 0; } @@ -1414,19 +1410,19 @@ static void trace_imc_event_del(struct perf_event *event, int flags) return; } - mutex_lock(&ref->lock); + spin_lock(&ref->lock); ref->refc--; if (ref->refc == 0) { if (opal_imc_counters_stop(OPAL_IMC_COUNTERS_TRACE, get_hard_smp_processor_id(smp_processor_id()))) { - mutex_unlock(&ref->lock); + spin_unlock(&ref->lock); pr_err("trace-imc: Unable to stop the counters for core %d\n", core_id); return; } } else if (ref->refc < 0) { ref->refc = 0; } - mutex_unlock(&ref->lock); + spin_unlock(&ref->lock); trace_imc_event_stop(event, flags); } @@ -1448,7 +1444,7 @@ static int trace_imc_event_init(struct perf_event *event) * no other thread is running any core/thread imc * events */ - mutex_lock(&imc_global_refc.lock); + spin_lock(&imc_global_refc.lock); if (imc_global_refc.id == 0 || imc_global_refc.id == IMC_DOMAIN_TRACE) { /* * No core/thread imc events are running in the @@ -1457,10 +1453,10 @@ static int trace_imc_event_init(struct perf_event *event) imc_global_refc.id = IMC_DOMAIN_TRACE; imc_global_refc.refc++; } else { - mutex_unlock(&imc_global_refc.lock); + spin_unlock(&imc_global_refc.lock); return -EBUSY; } - mutex_unlock(&imc_global_refc.lock); + spin_unlock(&imc_global_refc.lock); event->hw.idx = -1; @@ -1533,10 +1529,10 @@ static int init_nest_pmu_ref(void) i = 0; for_each_node(nid) { /* - * Mutex lock to avoid races while tracking the number of + * Take the lock to avoid races while tracking the number of * sessions using the chip's nest pmu units. */ - mutex_init(&nest_imc_refc[i].lock); + spin_lock_init(&nest_imc_refc[i].lock); /* * Loop to init the "id" with the node_id. Variable "i" initialized to @@ -1633,7 +1629,7 @@ static void imc_common_mem_free(struct imc_pmu *pmu_ptr) static void imc_common_cpuhp_mem_free(struct imc_pmu *pmu_ptr) { if (pmu_ptr->domain == IMC_DOMAIN_NEST) { - mutex_lock(&nest_init_lock); + spin_lock(&nest_init_lock); if (nest_pmus == 1) { cpuhp_remove_state(CPUHP_AP_PERF_POWERPC_NEST_IMC_ONLINE); kfree(nest_imc_refc); @@ -1643,7 +1639,7 @@ static void imc_common_cpuhp_mem_free(struct imc_pmu *pmu_ptr) if (nest_pmus > 0) nest_pmus--; - mutex_unlock(&nest_init_lock); + spin_unlock(&nest_init_lock); } /* Free core_imc memory */ @@ -1800,11 +1796,11 @@ int init_imc_pmu(struct device_node *parent, struct imc_pmu *pmu_ptr, int pmu_id * rest. To handle the cpuhotplug callback unregister, we track * the number of nest pmus in "nest_pmus". */ - mutex_lock(&nest_init_lock); + spin_lock(&nest_init_lock); if (nest_pmus == 0) { ret = init_nest_pmu_ref(); if (ret) { - mutex_unlock(&nest_init_lock); + spin_unlock(&nest_init_lock); kfree(per_nest_pmu_arr); per_nest_pmu_arr = NULL; goto err_free_mem; @@ -1812,7 +1808,7 @@ int init_imc_pmu(struct device_node *parent, struct imc_pmu *pmu_ptr, int pmu_id /* Register for cpu hotplug notification. */ ret = nest_pmu_cpumask_init(); if (ret) { - mutex_unlock(&nest_init_lock); + spin_unlock(&nest_init_lock); kfree(nest_imc_refc); kfree(per_nest_pmu_arr); per_nest_pmu_arr = NULL; @@ -1820,7 +1816,7 @@ int init_imc_pmu(struct device_node *parent, struct imc_pmu *pmu_ptr, int pmu_id } } nest_pmus++; - mutex_unlock(&nest_init_lock); + spin_unlock(&nest_init_lock); break; case IMC_DOMAIN_CORE: ret = core_imc_pmu_cpumask_init(); diff --git a/arch/riscv/boot/dts/sifive/fu740-c000.dtsi b/arch/riscv/boot/dts/sifive/fu740-c000.dtsi index 43bed6c0a84fe81b10189f53842d739348b25f34..5235fd1c9cb6773d2f23562fe999ffa0ca21dc6b 100644 --- a/arch/riscv/boot/dts/sifive/fu740-c000.dtsi +++ b/arch/riscv/boot/dts/sifive/fu740-c000.dtsi @@ -328,7 +328,7 @@ bus-range = <0x0 0xff>; ranges = <0x81000000 0x0 0x60080000 0x0 0x60080000 0x0 0x10000>, /* I/O */ <0x82000000 0x0 0x60090000 0x0 0x60090000 0x0 0xff70000>, /* mem */ - <0x82000000 0x0 0x70000000 0x0 0x70000000 0x0 0x1000000>, /* mem */ + <0x82000000 0x0 0x70000000 0x0 0x70000000 0x0 0x10000000>, /* mem */ <0xc3000000 0x20 0x00000000 0x20 0x00000000 0x20 0x00000000>; /* mem prefetchable */ num-lanes = <0x8>; interrupts = <56>, <57>, <58>, <59>, <60>, <61>, <62>, <63>, <64>; diff --git a/arch/riscv/include/asm/alternative-macros.h b/arch/riscv/include/asm/alternative-macros.h index 7226e2462584b65a78f7601c47e744ed7d6a54fc..2c0f4c887289a53dc457b356a1e754114aba4185 100644 --- a/arch/riscv/include/asm/alternative-macros.h +++ b/arch/riscv/include/asm/alternative-macros.h @@ -46,7 +46,7 @@ .macro ALTERNATIVE_CFG_2 old_c, new_c_1, vendor_id_1, errata_id_1, enable_1, \ new_c_2, vendor_id_2, errata_id_2, enable_2 - ALTERNATIVE_CFG \old_c, \new_c_1, \vendor_id_1, \errata_id_1, \enable_1 + ALTERNATIVE_CFG "\old_c", "\new_c_1", \vendor_id_1, \errata_id_1, \enable_1 ALT_NEW_CONTENT \vendor_id_2, \errata_id_2, \enable_2, \new_c_2 .endm diff --git a/arch/riscv/include/asm/uaccess.h b/arch/riscv/include/asm/uaccess.h index 855450bed9f52c2b7b06360dc5976b5519e5592b..ec0cab9fbddd0da98cb415af2732a4ede083886b 100644 --- a/arch/riscv/include/asm/uaccess.h +++ b/arch/riscv/include/asm/uaccess.h @@ -165,7 +165,7 @@ do { \ might_fault(); \ access_ok(__p, sizeof(*__p)) ? \ __get_user((x), __p) : \ - ((x) = 0, -EFAULT); \ + ((x) = (__force __typeof__(x))0, -EFAULT); \ }) #define __put_user_asm(insn, x, ptr, err) \ diff --git a/arch/riscv/kernel/head.S b/arch/riscv/kernel/head.S index b865046e4dbbc489ff4ed3c21e231bfeeba45158..4bf6c449d78b6a0719bfb3c7e9bbed927b71deb2 100644 --- a/arch/riscv/kernel/head.S +++ b/arch/riscv/kernel/head.S @@ -326,7 +326,7 @@ clear_bss_done: call soc_early_init tail start_kernel -#if CONFIG_RISCV_BOOT_SPINWAIT +#ifdef CONFIG_RISCV_BOOT_SPINWAIT .Lsecondary_start: /* Set trap vector to spin forever to help debug */ la a3, .Lsecondary_park diff --git a/arch/riscv/kernel/probes/simulate-insn.c b/arch/riscv/kernel/probes/simulate-insn.c index d73e96f6ed7c59dafdaebb916598c69df9f15c1b..a20568bd1f1a885909009b4e14460ab73ada2b90 100644 --- a/arch/riscv/kernel/probes/simulate-insn.c +++ b/arch/riscv/kernel/probes/simulate-insn.c @@ -71,11 +71,11 @@ bool __kprobes simulate_jalr(u32 opcode, unsigned long addr, struct pt_regs *reg u32 rd_index = (opcode >> 7) & 0x1f; u32 rs1_index = (opcode >> 15) & 0x1f; - ret = rv_insn_reg_set_val(regs, rd_index, addr + 4); + ret = rv_insn_reg_get_val(regs, rs1_index, &base_addr); if (!ret) return ret; - ret = rv_insn_reg_get_val(regs, rs1_index, &base_addr); + ret = rv_insn_reg_set_val(regs, rd_index, addr + 4); if (!ret) return ret; diff --git a/arch/riscv/kernel/probes/simulate-insn.h b/arch/riscv/kernel/probes/simulate-insn.h index cb6ff7dccb92e18bde5f9b517e42ec851a98b986..de8474146a9b6e71e025fcb79daef39e1cd8d2e7 100644 --- a/arch/riscv/kernel/probes/simulate-insn.h +++ b/arch/riscv/kernel/probes/simulate-insn.h @@ -31,9 +31,9 @@ __RISCV_INSN_FUNCS(fence, 0x7f, 0x0f); } while (0) __RISCV_INSN_FUNCS(c_j, 0xe003, 0xa001); -__RISCV_INSN_FUNCS(c_jr, 0xf007, 0x8002); +__RISCV_INSN_FUNCS(c_jr, 0xf07f, 0x8002); __RISCV_INSN_FUNCS(c_jal, 0xe003, 0x2001); -__RISCV_INSN_FUNCS(c_jalr, 0xf007, 0x9002); +__RISCV_INSN_FUNCS(c_jalr, 0xf07f, 0x9002); __RISCV_INSN_FUNCS(c_beqz, 0xe003, 0xc001); __RISCV_INSN_FUNCS(c_bnez, 0xe003, 0xe001); __RISCV_INSN_FUNCS(c_ebreak, 0xffff, 0x9002); diff --git a/arch/riscv/kernel/smpboot.c b/arch/riscv/kernel/smpboot.c index 3373df413c8820682ea4770f622b6fbbc0003865..ddb2afba6d25558c26748f8f63dd1dc1702b5e89 100644 --- a/arch/riscv/kernel/smpboot.c +++ b/arch/riscv/kernel/smpboot.c @@ -39,7 +39,6 @@ static DECLARE_COMPLETION(cpu_running); void __init smp_prepare_boot_cpu(void) { - init_cpu_topology(); } void __init smp_prepare_cpus(unsigned int max_cpus) @@ -48,6 +47,8 @@ void __init smp_prepare_cpus(unsigned int max_cpus) int ret; unsigned int curr_cpuid; + init_cpu_topology(); + curr_cpuid = smp_processor_id(); store_cpu_topology(curr_cpuid); numa_store_cpu_info(curr_cpuid); diff --git a/arch/s390/boot/decompressor.c b/arch/s390/boot/decompressor.c index e27c2140d620689eb8270695059f6815b38b981e..8dcd7af2911a07c36d792aa0366314579b512e28 100644 --- a/arch/s390/boot/decompressor.c +++ b/arch/s390/boot/decompressor.c @@ -23,9 +23,9 @@ #define memmove memmove #define memzero(s, n) memset((s), 0, (n)) -#ifdef CONFIG_KERNEL_BZIP2 +#if defined(CONFIG_KERNEL_BZIP2) #define BOOT_HEAP_SIZE 0x400000 -#elif CONFIG_KERNEL_ZSTD +#elif defined(CONFIG_KERNEL_ZSTD) #define BOOT_HEAP_SIZE 0x30000 #else #define BOOT_HEAP_SIZE 0x10000 diff --git a/arch/s390/configs/debug_defconfig b/arch/s390/configs/debug_defconfig index a7b4e1d82758072cb23cd5e988bd8d492d5a4490..74b35ec2ad28a800f25b3b211e219c9bf32ae2df 100644 --- a/arch/s390/configs/debug_defconfig +++ b/arch/s390/configs/debug_defconfig @@ -190,7 +190,6 @@ CONFIG_NFT_CT=m CONFIG_NFT_LOG=m CONFIG_NFT_LIMIT=m CONFIG_NFT_NAT=m -CONFIG_NFT_OBJREF=m CONFIG_NFT_REJECT=m CONFIG_NFT_COMPAT=m CONFIG_NFT_HASH=m @@ -569,6 +568,7 @@ CONFIG_INPUT_EVDEV=y # CONFIG_INPUT_MOUSE is not set # CONFIG_SERIO is not set CONFIG_LEGACY_PTY_COUNT=0 +# CONFIG_LEGACY_TIOCSTI is not set CONFIG_VIRTIO_CONSOLE=m CONFIG_HW_RANDOM_VIRTIO=m CONFIG_HANGCHECK_TIMER=m @@ -660,6 +660,7 @@ CONFIG_CONFIGFS_FS=m CONFIG_ECRYPT_FS=m CONFIG_CRAMFS=m CONFIG_SQUASHFS=m +CONFIG_SQUASHFS_CHOICE_DECOMP_BY_MOUNT=y CONFIG_SQUASHFS_XATTR=y CONFIG_SQUASHFS_LZ4=y CONFIG_SQUASHFS_LZO=y @@ -705,6 +706,7 @@ CONFIG_SECURITY_LOCKDOWN_LSM_EARLY=y CONFIG_SECURITY_LANDLOCK=y CONFIG_INTEGRITY_SIGNATURE=y CONFIG_INTEGRITY_ASYMMETRIC_KEYS=y +CONFIG_INTEGRITY_PLATFORM_KEYRING=y CONFIG_IMA=y CONFIG_IMA_DEFAULT_HASH_SHA256=y CONFIG_IMA_WRITE_POLICY=y @@ -781,6 +783,7 @@ CONFIG_ZCRYPT=m CONFIG_PKEY=m CONFIG_CRYPTO_PAES_S390=m CONFIG_CRYPTO_DEV_VIRTIO=m +CONFIG_SYSTEM_BLACKLIST_KEYRING=y CONFIG_CORDIC=m CONFIG_CRYPTO_LIB_CURVE25519=m CONFIG_CRYPTO_LIB_CHACHA20POLY1305=m @@ -848,7 +851,6 @@ CONFIG_PREEMPT_TRACER=y CONFIG_SCHED_TRACER=y CONFIG_FTRACE_SYSCALLS=y CONFIG_BLK_DEV_IO_TRACE=y -CONFIG_BPF_KPROBE_OVERRIDE=y CONFIG_HIST_TRIGGERS=y CONFIG_FTRACE_STARTUP_TEST=y # CONFIG_EVENT_TRACE_STARTUP_TEST is not set @@ -870,7 +872,6 @@ CONFIG_FAIL_MAKE_REQUEST=y CONFIG_FAIL_IO_TIMEOUT=y CONFIG_FAIL_FUTEX=y CONFIG_FAULT_INJECTION_DEBUG_FS=y -CONFIG_FAIL_FUNCTION=y CONFIG_FAULT_INJECTION_STACKTRACE_FILTER=y CONFIG_LKDTM=m CONFIG_TEST_MIN_HEAP=y diff --git a/arch/s390/configs/defconfig b/arch/s390/configs/defconfig index 2bc2d0fe57743ccd8caa5ea8838c1a8b555bfcc6..cec71268e3bc2e22916ad2c59c7a6601bb91284b 100644 --- a/arch/s390/configs/defconfig +++ b/arch/s390/configs/defconfig @@ -181,7 +181,6 @@ CONFIG_NFT_CT=m CONFIG_NFT_LOG=m CONFIG_NFT_LIMIT=m CONFIG_NFT_NAT=m -CONFIG_NFT_OBJREF=m CONFIG_NFT_REJECT=m CONFIG_NFT_COMPAT=m CONFIG_NFT_HASH=m @@ -559,6 +558,7 @@ CONFIG_INPUT_EVDEV=y # CONFIG_INPUT_MOUSE is not set # CONFIG_SERIO is not set CONFIG_LEGACY_PTY_COUNT=0 +# CONFIG_LEGACY_TIOCSTI is not set CONFIG_VIRTIO_CONSOLE=m CONFIG_HW_RANDOM_VIRTIO=m CONFIG_HANGCHECK_TIMER=m @@ -645,6 +645,7 @@ CONFIG_CONFIGFS_FS=m CONFIG_ECRYPT_FS=m CONFIG_CRAMFS=m CONFIG_SQUASHFS=m +CONFIG_SQUASHFS_CHOICE_DECOMP_BY_MOUNT=y CONFIG_SQUASHFS_XATTR=y CONFIG_SQUASHFS_LZ4=y CONFIG_SQUASHFS_LZO=y @@ -688,6 +689,7 @@ CONFIG_SECURITY_LOCKDOWN_LSM_EARLY=y CONFIG_SECURITY_LANDLOCK=y CONFIG_INTEGRITY_SIGNATURE=y CONFIG_INTEGRITY_ASYMMETRIC_KEYS=y +CONFIG_INTEGRITY_PLATFORM_KEYRING=y CONFIG_IMA=y CONFIG_IMA_DEFAULT_HASH_SHA256=y CONFIG_IMA_WRITE_POLICY=y @@ -766,6 +768,7 @@ CONFIG_ZCRYPT=m CONFIG_PKEY=m CONFIG_CRYPTO_PAES_S390=m CONFIG_CRYPTO_DEV_VIRTIO=m +CONFIG_SYSTEM_BLACKLIST_KEYRING=y CONFIG_CORDIC=m CONFIG_PRIME_NUMBERS=m CONFIG_CRYPTO_LIB_CURVE25519=m @@ -798,7 +801,6 @@ CONFIG_STACK_TRACER=y CONFIG_SCHED_TRACER=y CONFIG_FTRACE_SYSCALLS=y CONFIG_BLK_DEV_IO_TRACE=y -CONFIG_BPF_KPROBE_OVERRIDE=y CONFIG_HIST_TRIGGERS=y CONFIG_SAMPLES=y CONFIG_SAMPLE_TRACE_PRINTK=m diff --git a/arch/s390/configs/zfcpdump_defconfig b/arch/s390/configs/zfcpdump_defconfig index ae14ab0b864d5946c277c2fabae7869223b3e87a..a9c0c81d1de992c8cfdadb7985e293856318681e 100644 --- a/arch/s390/configs/zfcpdump_defconfig +++ b/arch/s390/configs/zfcpdump_defconfig @@ -13,7 +13,6 @@ CONFIG_TUNE_ZEC12=y # CONFIG_COMPAT is not set CONFIG_NR_CPUS=2 CONFIG_HZ_100=y -# CONFIG_RELOCATABLE is not set # CONFIG_CHSC_SCH is not set # CONFIG_SCM_BUS is not set CONFIG_CRASH_DUMP=y @@ -50,6 +49,7 @@ CONFIG_ZFCP=y # CONFIG_INPUT_KEYBOARD is not set # CONFIG_INPUT_MOUSE is not set # CONFIG_SERIO is not set +# CONFIG_LEGACY_TIOCSTI is not set # CONFIG_HVC_IUCV is not set # CONFIG_HW_RANDOM_S390 is not set # CONFIG_HMC_DRV is not set diff --git a/arch/s390/include/asm/cpu_mf.h b/arch/s390/include/asm/cpu_mf.h index feaba12dbecb8d72d7e30128d3158c158f7b179d..efa103b52a1a190a19dd9d98bc98d81447274075 100644 --- a/arch/s390/include/asm/cpu_mf.h +++ b/arch/s390/include/asm/cpu_mf.h @@ -131,19 +131,21 @@ struct hws_combined_entry { struct hws_diag_entry diag; /* Diagnostic-sampling data entry */ } __packed; -struct hws_trailer_entry { - union { - struct { - unsigned int f:1; /* 0 - Block Full Indicator */ - unsigned int a:1; /* 1 - Alert request control */ - unsigned int t:1; /* 2 - Timestamp format */ - unsigned int :29; /* 3 - 31: Reserved */ - unsigned int bsdes:16; /* 32-47: size of basic SDE */ - unsigned int dsdes:16; /* 48-63: size of diagnostic SDE */ - }; - unsigned long long flags; /* 0 - 63: All indicators */ +union hws_trailer_header { + struct { + unsigned int f:1; /* 0 - Block Full Indicator */ + unsigned int a:1; /* 1 - Alert request control */ + unsigned int t:1; /* 2 - Timestamp format */ + unsigned int :29; /* 3 - 31: Reserved */ + unsigned int bsdes:16; /* 32-47: size of basic SDE */ + unsigned int dsdes:16; /* 48-63: size of diagnostic SDE */ + unsigned long long overflow; /* 64 - Overflow Count */ }; - unsigned long long overflow; /* 64 - sample Overflow count */ + __uint128_t val; +}; + +struct hws_trailer_entry { + union hws_trailer_header header; /* 0 - 15 Flags + Overflow Count */ unsigned char timestamp[16]; /* 16 - 31 timestamp */ unsigned long long reserved1; /* 32 -Reserved */ unsigned long long reserved2; /* */ @@ -290,14 +292,11 @@ static inline unsigned long sample_rate_to_freq(struct hws_qsi_info_block *qsi, return USEC_PER_SEC * qsi->cpu_speed / rate; } -#define SDB_TE_ALERT_REQ_MASK 0x4000000000000000UL -#define SDB_TE_BUFFER_FULL_MASK 0x8000000000000000UL - /* Return TOD timestamp contained in an trailer entry */ static inline unsigned long long trailer_timestamp(struct hws_trailer_entry *te) { /* TOD in STCKE format */ - if (te->t) + if (te->header.t) return *((unsigned long long *) &te->timestamp[1]); /* TOD in STCK format */ diff --git a/arch/s390/include/asm/debug.h b/arch/s390/include/asm/debug.h index 77f24262c25c18764f3c87f8e6a1b706db5714a1..ac665b9670c5dba3a62948459d187e80e5313afc 100644 --- a/arch/s390/include/asm/debug.h +++ b/arch/s390/include/asm/debug.h @@ -4,8 +4,8 @@ * * Copyright IBM Corp. 1999, 2020 */ -#ifndef DEBUG_H -#define DEBUG_H +#ifndef _ASM_S390_DEBUG_H +#define _ASM_S390_DEBUG_H #include #include @@ -487,4 +487,4 @@ void debug_register_static(debug_info_t *id, int pages_per_area, int nr_areas); #endif /* MODULE */ -#endif /* DEBUG_H */ +#endif /* _ASM_S390_DEBUG_H */ diff --git a/arch/s390/include/asm/percpu.h b/arch/s390/include/asm/percpu.h index cb5fc06904354a2e1cb523e826f02c1d171f7f55..081837b391e353a4467b194719671e4bd435e3a9 100644 --- a/arch/s390/include/asm/percpu.h +++ b/arch/s390/include/asm/percpu.h @@ -31,7 +31,7 @@ pcp_op_T__ *ptr__; \ preempt_disable_notrace(); \ ptr__ = raw_cpu_ptr(&(pcp)); \ - prev__ = *ptr__; \ + prev__ = READ_ONCE(*ptr__); \ do { \ old__ = prev__; \ new__ = old__ op (val); \ diff --git a/arch/s390/kernel/machine_kexec_file.c b/arch/s390/kernel/machine_kexec_file.c index fc6d5f58debeb4c94338c2c4d73d27427c253299..2df94d32140c407962aec49f5d3e5d75e54fa58a 100644 --- a/arch/s390/kernel/machine_kexec_file.c +++ b/arch/s390/kernel/machine_kexec_file.c @@ -187,8 +187,6 @@ static int kexec_file_add_ipl_report(struct kimage *image, data->memsz = ALIGN(data->memsz, PAGE_SIZE); buf.mem = data->memsz; - if (image->type == KEXEC_TYPE_CRASH) - buf.mem += crashk_res.start; ptr = (void *)ipl_cert_list_addr; end = ptr + ipl_cert_list_size; @@ -225,6 +223,9 @@ static int kexec_file_add_ipl_report(struct kimage *image, data->kernel_buf + offsetof(struct lowcore, ipl_parmblock_ptr); *lc_ipl_parmblock_ptr = (__u32)buf.mem; + if (image->type == KEXEC_TYPE_CRASH) + buf.mem += crashk_res.start; + ret = kexec_add_buffer(&buf); out: return ret; diff --git a/arch/s390/kernel/perf_cpum_sf.c b/arch/s390/kernel/perf_cpum_sf.c index 332a4996513087ddede85bfa797be9408abad063..ce886a03545ae1d195962bd102992371f4da6874 100644 --- a/arch/s390/kernel/perf_cpum_sf.c +++ b/arch/s390/kernel/perf_cpum_sf.c @@ -163,14 +163,15 @@ static void free_sampling_buffer(struct sf_buffer *sfb) static int alloc_sample_data_block(unsigned long *sdbt, gfp_t gfp_flags) { - unsigned long sdb, *trailer; + struct hws_trailer_entry *te; + unsigned long sdb; /* Allocate and initialize sample-data-block */ sdb = get_zeroed_page(gfp_flags); if (!sdb) return -ENOMEM; - trailer = trailer_entry_ptr(sdb); - *trailer = SDB_TE_ALERT_REQ_MASK; + te = (struct hws_trailer_entry *)trailer_entry_ptr(sdb); + te->header.a = 1; /* Link SDB into the sample-data-block-table */ *sdbt = sdb; @@ -1206,7 +1207,7 @@ static void hw_collect_samples(struct perf_event *event, unsigned long *sdbt, "%s: Found unknown" " sampling data entry: te->f %i" " basic.def %#4x (%p)\n", __func__, - te->f, sample->def, sample); + te->header.f, sample->def, sample); /* Sample slot is not yet written or other record. * * This condition can occur if the buffer was reused @@ -1217,7 +1218,7 @@ static void hw_collect_samples(struct perf_event *event, unsigned long *sdbt, * that are not full. Stop processing if the first * invalid format was detected. */ - if (!te->f) + if (!te->header.f) break; } @@ -1227,6 +1228,16 @@ static void hw_collect_samples(struct perf_event *event, unsigned long *sdbt, } } +static inline __uint128_t __cdsg(__uint128_t *ptr, __uint128_t old, __uint128_t new) +{ + asm volatile( + " cdsg %[old],%[new],%[ptr]\n" + : [old] "+d" (old), [ptr] "+QS" (*ptr) + : [new] "d" (new) + : "memory", "cc"); + return old; +} + /* hw_perf_event_update() - Process sampling buffer * @event: The perf event * @flush_all: Flag to also flush partially filled sample-data-blocks @@ -1243,10 +1254,11 @@ static void hw_collect_samples(struct perf_event *event, unsigned long *sdbt, */ static void hw_perf_event_update(struct perf_event *event, int flush_all) { + unsigned long long event_overflow, sampl_overflow, num_sdb; + union hws_trailer_header old, prev, new; struct hw_perf_event *hwc = &event->hw; struct hws_trailer_entry *te; unsigned long *sdbt; - unsigned long long event_overflow, sampl_overflow, num_sdb, te_flags; int done; /* @@ -1266,25 +1278,25 @@ static void hw_perf_event_update(struct perf_event *event, int flush_all) te = (struct hws_trailer_entry *) trailer_entry_ptr(*sdbt); /* Leave loop if no more work to do (block full indicator) */ - if (!te->f) { + if (!te->header.f) { done = 1; if (!flush_all) break; } /* Check the sample overflow count */ - if (te->overflow) + if (te->header.overflow) /* Account sample overflows and, if a particular limit * is reached, extend the sampling buffer. * For details, see sfb_account_overflows(). */ - sampl_overflow += te->overflow; + sampl_overflow += te->header.overflow; /* Timestamps are valid for full sample-data-blocks only */ debug_sprintf_event(sfdbg, 6, "%s: sdbt %#lx " "overflow %llu timestamp %#llx\n", - __func__, (unsigned long)sdbt, te->overflow, - (te->f) ? trailer_timestamp(te) : 0ULL); + __func__, (unsigned long)sdbt, te->header.overflow, + (te->header.f) ? trailer_timestamp(te) : 0ULL); /* Collect all samples from a single sample-data-block and * flag if an (perf) event overflow happened. If so, the PMU @@ -1294,12 +1306,16 @@ static void hw_perf_event_update(struct perf_event *event, int flush_all) num_sdb++; /* Reset trailer (using compare-double-and-swap) */ + /* READ_ONCE() 16 byte header */ + prev.val = __cdsg(&te->header.val, 0, 0); do { - te_flags = te->flags & ~SDB_TE_BUFFER_FULL_MASK; - te_flags |= SDB_TE_ALERT_REQ_MASK; - } while (!cmpxchg_double(&te->flags, &te->overflow, - te->flags, te->overflow, - te_flags, 0ULL)); + old.val = prev.val; + new.val = prev.val; + new.f = 0; + new.a = 1; + new.overflow = 0; + prev.val = __cdsg(&te->header.val, old.val, new.val); + } while (prev.val != old.val); /* Advance to next sample-data-block */ sdbt++; @@ -1384,7 +1400,7 @@ static void aux_output_end(struct perf_output_handle *handle) range_scan = AUX_SDB_NUM_ALERT(aux); for (i = 0, idx = aux->head; i < range_scan; i++, idx++) { te = aux_sdb_trailer(aux, idx); - if (!(te->flags & SDB_TE_BUFFER_FULL_MASK)) + if (!te->header.f) break; } /* i is num of SDBs which are full */ @@ -1392,7 +1408,7 @@ static void aux_output_end(struct perf_output_handle *handle) /* Remove alert indicators in the buffer */ te = aux_sdb_trailer(aux, aux->alert_mark); - te->flags &= ~SDB_TE_ALERT_REQ_MASK; + te->header.a = 0; debug_sprintf_event(sfdbg, 6, "%s: SDBs %ld range %ld head %ld\n", __func__, i, range_scan, aux->head); @@ -1437,9 +1453,9 @@ static int aux_output_begin(struct perf_output_handle *handle, idx = aux->empty_mark + 1; for (i = 0; i < range_scan; i++, idx++) { te = aux_sdb_trailer(aux, idx); - te->flags &= ~(SDB_TE_BUFFER_FULL_MASK | - SDB_TE_ALERT_REQ_MASK); - te->overflow = 0; + te->header.f = 0; + te->header.a = 0; + te->header.overflow = 0; } /* Save the position of empty SDBs */ aux->empty_mark = aux->head + range - 1; @@ -1448,7 +1464,7 @@ static int aux_output_begin(struct perf_output_handle *handle, /* Set alert indicator */ aux->alert_mark = aux->head + range/2 - 1; te = aux_sdb_trailer(aux, aux->alert_mark); - te->flags = te->flags | SDB_TE_ALERT_REQ_MASK; + te->header.a = 1; /* Reset hardware buffer head */ head = AUX_SDB_INDEX(aux, aux->head); @@ -1475,14 +1491,17 @@ static int aux_output_begin(struct perf_output_handle *handle, static bool aux_set_alert(struct aux_buffer *aux, unsigned long alert_index, unsigned long long *overflow) { - unsigned long long orig_overflow, orig_flags, new_flags; + union hws_trailer_header old, prev, new; struct hws_trailer_entry *te; te = aux_sdb_trailer(aux, alert_index); + /* READ_ONCE() 16 byte header */ + prev.val = __cdsg(&te->header.val, 0, 0); do { - orig_flags = te->flags; - *overflow = orig_overflow = te->overflow; - if (orig_flags & SDB_TE_BUFFER_FULL_MASK) { + old.val = prev.val; + new.val = prev.val; + *overflow = old.overflow; + if (old.f) { /* * SDB is already set by hardware. * Abort and try to set somewhere @@ -1490,10 +1509,10 @@ static bool aux_set_alert(struct aux_buffer *aux, unsigned long alert_index, */ return false; } - new_flags = orig_flags | SDB_TE_ALERT_REQ_MASK; - } while (!cmpxchg_double(&te->flags, &te->overflow, - orig_flags, orig_overflow, - new_flags, 0ULL)); + new.a = 1; + new.overflow = 0; + prev.val = __cdsg(&te->header.val, old.val, new.val); + } while (prev.val != old.val); return true; } @@ -1522,8 +1541,9 @@ static bool aux_set_alert(struct aux_buffer *aux, unsigned long alert_index, static bool aux_reset_buffer(struct aux_buffer *aux, unsigned long range, unsigned long long *overflow) { - unsigned long long orig_overflow, orig_flags, new_flags; unsigned long i, range_scan, idx, idx_old; + union hws_trailer_header old, prev, new; + unsigned long long orig_overflow; struct hws_trailer_entry *te; debug_sprintf_event(sfdbg, 6, "%s: range %ld head %ld alert %ld " @@ -1554,17 +1574,20 @@ static bool aux_reset_buffer(struct aux_buffer *aux, unsigned long range, idx_old = idx = aux->empty_mark + 1; for (i = 0; i < range_scan; i++, idx++) { te = aux_sdb_trailer(aux, idx); + /* READ_ONCE() 16 byte header */ + prev.val = __cdsg(&te->header.val, 0, 0); do { - orig_flags = te->flags; - orig_overflow = te->overflow; - new_flags = orig_flags & ~SDB_TE_BUFFER_FULL_MASK; + old.val = prev.val; + new.val = prev.val; + orig_overflow = old.overflow; + new.f = 0; + new.overflow = 0; if (idx == aux->alert_mark) - new_flags |= SDB_TE_ALERT_REQ_MASK; + new.a = 1; else - new_flags &= ~SDB_TE_ALERT_REQ_MASK; - } while (!cmpxchg_double(&te->flags, &te->overflow, - orig_flags, orig_overflow, - new_flags, 0ULL)); + new.a = 0; + prev.val = __cdsg(&te->header.val, old.val, new.val); + } while (prev.val != old.val); *overflow += orig_overflow; } diff --git a/arch/s390/kernel/setup.c b/arch/s390/kernel/setup.c index 2b6091349daa2553c504a362ed1e7339aff8bd62..696c9e007a369d8df883317bdde15c1717a5bfef 100644 --- a/arch/s390/kernel/setup.c +++ b/arch/s390/kernel/setup.c @@ -508,6 +508,7 @@ static void __init setup_lowcore_dat_on(void) { struct lowcore *abs_lc; unsigned long flags; + int i; __ctl_clear_bit(0, 28); S390_lowcore.external_new_psw.mask |= PSW_MASK_DAT; @@ -523,8 +524,8 @@ static void __init setup_lowcore_dat_on(void) abs_lc = get_abs_lowcore(&flags); abs_lc->restart_flags = RESTART_FLAG_CTLREGS; abs_lc->program_new_psw = S390_lowcore.program_new_psw; - memcpy(abs_lc->cregs_save_area, S390_lowcore.cregs_save_area, - sizeof(abs_lc->cregs_save_area)); + for (i = 0; i < 16; i++) + abs_lc->cregs_save_area[i] = S390_lowcore.cregs_save_area[i]; put_abs_lowcore(abs_lc, flags); } diff --git a/arch/s390/kernel/vmlinux.lds.S b/arch/s390/kernel/vmlinux.lds.S index 5ea3830af0ccff11c6a18a06a2392cf690742b34..cbf9c1b0beda4a8a82c5821fc97f23410d5ecd31 100644 --- a/arch/s390/kernel/vmlinux.lds.S +++ b/arch/s390/kernel/vmlinux.lds.S @@ -17,6 +17,8 @@ /* Handle ro_after_init data on our own. */ #define RO_AFTER_INIT_DATA +#define RUNTIME_DISCARD_EXIT + #define EMITS_PT_NOTE #include @@ -79,6 +81,7 @@ SECTIONS _end_amode31_refs = .; } + . = ALIGN(PAGE_SIZE); _edata = .; /* End of data section */ /* will be freed after init */ @@ -193,6 +196,7 @@ SECTIONS BSS_SECTION(PAGE_SIZE, 4 * PAGE_SIZE, PAGE_SIZE) + . = ALIGN(PAGE_SIZE); _end = . ; /* diff --git a/arch/s390/kvm/interrupt.c b/arch/s390/kvm/interrupt.c index 1dae78deddf28602d32d051dc5ad956be6dd5315..ab26aa53ee3713bbc54bb4f4d08e9b6689c8de34 100644 --- a/arch/s390/kvm/interrupt.c +++ b/arch/s390/kvm/interrupt.c @@ -83,8 +83,9 @@ static int sca_inject_ext_call(struct kvm_vcpu *vcpu, int src_id) struct esca_block *sca = vcpu->kvm->arch.sca; union esca_sigp_ctrl *sigp_ctrl = &(sca->cpu[vcpu->vcpu_id].sigp_ctrl); - union esca_sigp_ctrl new_val = {0}, old_val = *sigp_ctrl; + union esca_sigp_ctrl new_val = {0}, old_val; + old_val = READ_ONCE(*sigp_ctrl); new_val.scn = src_id; new_val.c = 1; old_val.c = 0; @@ -95,8 +96,9 @@ static int sca_inject_ext_call(struct kvm_vcpu *vcpu, int src_id) struct bsca_block *sca = vcpu->kvm->arch.sca; union bsca_sigp_ctrl *sigp_ctrl = &(sca->cpu[vcpu->vcpu_id].sigp_ctrl); - union bsca_sigp_ctrl new_val = {0}, old_val = *sigp_ctrl; + union bsca_sigp_ctrl new_val = {0}, old_val; + old_val = READ_ONCE(*sigp_ctrl); new_val.scn = src_id; new_val.c = 1; old_val.c = 0; @@ -126,16 +128,18 @@ static void sca_clear_ext_call(struct kvm_vcpu *vcpu) struct esca_block *sca = vcpu->kvm->arch.sca; union esca_sigp_ctrl *sigp_ctrl = &(sca->cpu[vcpu->vcpu_id].sigp_ctrl); - union esca_sigp_ctrl old = *sigp_ctrl; + union esca_sigp_ctrl old; + old = READ_ONCE(*sigp_ctrl); expect = old.value; rc = cmpxchg(&sigp_ctrl->value, old.value, 0); } else { struct bsca_block *sca = vcpu->kvm->arch.sca; union bsca_sigp_ctrl *sigp_ctrl = &(sca->cpu[vcpu->vcpu_id].sigp_ctrl); - union bsca_sigp_ctrl old = *sigp_ctrl; + union bsca_sigp_ctrl old; + old = READ_ONCE(*sigp_ctrl); expect = old.value; rc = cmpxchg(&sigp_ctrl->value, old.value, 0); } diff --git a/arch/sh/include/asm/pgtable-3level.h b/arch/sh/include/asm/pgtable-3level.h index a889a3a938bab85510dc136d73386c96615bb32d..d1ce73f3bd85ef18ba1d14d017744f812b2fe3ce 100644 --- a/arch/sh/include/asm/pgtable-3level.h +++ b/arch/sh/include/asm/pgtable-3level.h @@ -28,7 +28,7 @@ #define pmd_ERROR(e) \ printk("%s:%d: bad pmd %016llx.\n", __FILE__, __LINE__, pmd_val(e)) -typedef struct { +typedef union { struct { unsigned long pmd_low; unsigned long pmd_high; diff --git a/arch/x86/Makefile b/arch/x86/Makefile index 9cf07322875a42f914f16505a16003250f02cceb..73ed982d41000059e6d0f7e78c90e87d7f7d0390 100644 --- a/arch/x86/Makefile +++ b/arch/x86/Makefile @@ -14,13 +14,13 @@ endif ifdef CONFIG_CC_IS_GCC RETPOLINE_CFLAGS := $(call cc-option,-mindirect-branch=thunk-extern -mindirect-branch-register) -RETPOLINE_CFLAGS += $(call cc-option,-mindirect-branch-cs-prefix) RETPOLINE_VDSO_CFLAGS := $(call cc-option,-mindirect-branch=thunk-inline -mindirect-branch-register) endif ifdef CONFIG_CC_IS_CLANG RETPOLINE_CFLAGS := -mretpoline-external-thunk RETPOLINE_VDSO_CFLAGS := -mretpoline endif +RETPOLINE_CFLAGS += $(call cc-option,-mindirect-branch-cs-prefix) ifdef CONFIG_RETHUNK RETHUNK_CFLAGS := -mfunction-return=thunk-extern diff --git a/arch/x86/boot/bioscall.S b/arch/x86/boot/bioscall.S index 5521ea12f44e068281ef99c309a5f6ebf736ff71..aa9b9645758433b088eeb579a9553b693dfa381a 100644 --- a/arch/x86/boot/bioscall.S +++ b/arch/x86/boot/bioscall.S @@ -32,7 +32,7 @@ intcall: movw %dx, %si movw %sp, %di movw $11, %cx - rep; movsd + rep; movsl /* Pop full state from the stack */ popal @@ -67,7 +67,7 @@ intcall: jz 4f movw %sp, %si movw $11, %cx - rep; movsd + rep; movsl 4: addw $44, %sp /* Restore state and return */ diff --git a/arch/x86/boot/compressed/ident_map_64.c b/arch/x86/boot/compressed/ident_map_64.c index d4a314cc50d6ee6ecaa7268f59bbb946c71fb916..321a5011042d40f51120ff910786f6c17e80aaef 100644 --- a/arch/x86/boot/compressed/ident_map_64.c +++ b/arch/x86/boot/compressed/ident_map_64.c @@ -180,6 +180,12 @@ void initialize_identity_maps(void *rmode) /* Load the new page-table. */ write_cr3(top_level_pgt); + + /* + * Now that the required page table mappings are established and a + * GHCB can be used, check for SNP guest/HV feature compatibility. + */ + snp_check_features(); } static pte_t *split_large_pmd(struct x86_mapping_info *info, diff --git a/arch/x86/boot/compressed/misc.h b/arch/x86/boot/compressed/misc.h index 62208ec04ca4b800e7cf3eb8949f6cf6a228fee2..20118fb7c53bbbec630a87cac129096a311ebef0 100644 --- a/arch/x86/boot/compressed/misc.h +++ b/arch/x86/boot/compressed/misc.h @@ -126,6 +126,7 @@ static inline void console_init(void) #ifdef CONFIG_AMD_MEM_ENCRYPT void sev_enable(struct boot_params *bp); +void snp_check_features(void); void sev_es_shutdown_ghcb(void); extern bool sev_es_check_ghcb_fault(unsigned long address); void snp_set_page_private(unsigned long paddr); @@ -143,6 +144,7 @@ static inline void sev_enable(struct boot_params *bp) if (bp) bp->cc_blob_address = 0; } +static inline void snp_check_features(void) { } static inline void sev_es_shutdown_ghcb(void) { } static inline bool sev_es_check_ghcb_fault(unsigned long address) { diff --git a/arch/x86/boot/compressed/sev.c b/arch/x86/boot/compressed/sev.c index c93930d5ccbd0f1b4894012e09c885e20ee1ba0a..d63ad8f99f83a3c616f3560cd12f3ea03acbe049 100644 --- a/arch/x86/boot/compressed/sev.c +++ b/arch/x86/boot/compressed/sev.c @@ -208,6 +208,23 @@ void sev_es_shutdown_ghcb(void) error("Can't unmap GHCB page"); } +static void __noreturn sev_es_ghcb_terminate(struct ghcb *ghcb, unsigned int set, + unsigned int reason, u64 exit_info_2) +{ + u64 exit_info_1 = SVM_VMGEXIT_TERM_REASON(set, reason); + + vc_ghcb_invalidate(ghcb); + ghcb_set_sw_exit_code(ghcb, SVM_VMGEXIT_TERM_REQUEST); + ghcb_set_sw_exit_info_1(ghcb, exit_info_1); + ghcb_set_sw_exit_info_2(ghcb, exit_info_2); + + sev_es_wr_ghcb_msr(__pa(ghcb)); + VMGEXIT(); + + while (true) + asm volatile("hlt\n" : : : "memory"); +} + bool sev_es_check_ghcb_fault(unsigned long address) { /* Check whether the fault was on the GHCB page */ @@ -270,6 +287,59 @@ static void enforce_vmpl0(void) sev_es_terminate(SEV_TERM_SET_LINUX, GHCB_TERM_NOT_VMPL0); } +/* + * SNP_FEATURES_IMPL_REQ is the mask of SNP features that will need + * guest side implementation for proper functioning of the guest. If any + * of these features are enabled in the hypervisor but are lacking guest + * side implementation, the behavior of the guest will be undefined. The + * guest could fail in non-obvious way making it difficult to debug. + * + * As the behavior of reserved feature bits is unknown to be on the + * safe side add them to the required features mask. + */ +#define SNP_FEATURES_IMPL_REQ (MSR_AMD64_SNP_VTOM | \ + MSR_AMD64_SNP_REFLECT_VC | \ + MSR_AMD64_SNP_RESTRICTED_INJ | \ + MSR_AMD64_SNP_ALT_INJ | \ + MSR_AMD64_SNP_DEBUG_SWAP | \ + MSR_AMD64_SNP_VMPL_SSS | \ + MSR_AMD64_SNP_SECURE_TSC | \ + MSR_AMD64_SNP_VMGEXIT_PARAM | \ + MSR_AMD64_SNP_VMSA_REG_PROTECTION | \ + MSR_AMD64_SNP_RESERVED_BIT13 | \ + MSR_AMD64_SNP_RESERVED_BIT15 | \ + MSR_AMD64_SNP_RESERVED_MASK) + +/* + * SNP_FEATURES_PRESENT is the mask of SNP features that are implemented + * by the guest kernel. As and when a new feature is implemented in the + * guest kernel, a corresponding bit should be added to the mask. + */ +#define SNP_FEATURES_PRESENT (0) + +void snp_check_features(void) +{ + u64 unsupported; + + if (!(sev_status & MSR_AMD64_SEV_SNP_ENABLED)) + return; + + /* + * Terminate the boot if hypervisor has enabled any feature lacking + * guest side implementation. Pass on the unsupported features mask through + * EXIT_INFO_2 of the GHCB protocol so that those features can be reported + * as part of the guest boot failure. + */ + unsupported = sev_status & SNP_FEATURES_IMPL_REQ & ~SNP_FEATURES_PRESENT; + if (unsupported) { + if (ghcb_version < 2 || (!boot_ghcb && !early_setup_ghcb())) + sev_es_terminate(SEV_TERM_SET_GEN, GHCB_SNP_UNSUPPORTED); + + sev_es_ghcb_terminate(boot_ghcb, SEV_TERM_SET_GEN, + GHCB_SNP_UNSUPPORTED, unsupported); + } +} + void sev_enable(struct boot_params *bp) { unsigned int eax, ebx, ecx, edx; diff --git a/arch/x86/coco/tdx/tdx.c b/arch/x86/coco/tdx/tdx.c index cfd4c95b9f045e564ee48885aac4534421619b99..669d9e4f290152cc94b881e529a7239859491a67 100644 --- a/arch/x86/coco/tdx/tdx.c +++ b/arch/x86/coco/tdx/tdx.c @@ -386,8 +386,8 @@ static int handle_mmio(struct pt_regs *regs, struct ve_info *ve) { unsigned long *reg, val, vaddr; char buffer[MAX_INSN_SIZE]; + enum insn_mmio_type mmio; struct insn insn = {}; - enum mmio_type mmio; int size, extend_size; u8 extend_val = 0; @@ -402,10 +402,10 @@ static int handle_mmio(struct pt_regs *regs, struct ve_info *ve) return -EINVAL; mmio = insn_decode_mmio(&insn, &size); - if (WARN_ON_ONCE(mmio == MMIO_DECODE_FAILED)) + if (WARN_ON_ONCE(mmio == INSN_MMIO_DECODE_FAILED)) return -EINVAL; - if (mmio != MMIO_WRITE_IMM && mmio != MMIO_MOVS) { + if (mmio != INSN_MMIO_WRITE_IMM && mmio != INSN_MMIO_MOVS) { reg = insn_get_modrm_reg_ptr(&insn, regs); if (!reg) return -EINVAL; @@ -426,23 +426,23 @@ static int handle_mmio(struct pt_regs *regs, struct ve_info *ve) /* Handle writes first */ switch (mmio) { - case MMIO_WRITE: + case INSN_MMIO_WRITE: memcpy(&val, reg, size); if (!mmio_write(size, ve->gpa, val)) return -EIO; return insn.length; - case MMIO_WRITE_IMM: + case INSN_MMIO_WRITE_IMM: val = insn.immediate.value; if (!mmio_write(size, ve->gpa, val)) return -EIO; return insn.length; - case MMIO_READ: - case MMIO_READ_ZERO_EXTEND: - case MMIO_READ_SIGN_EXTEND: + case INSN_MMIO_READ: + case INSN_MMIO_READ_ZERO_EXTEND: + case INSN_MMIO_READ_SIGN_EXTEND: /* Reads are handled below */ break; - case MMIO_MOVS: - case MMIO_DECODE_FAILED: + case INSN_MMIO_MOVS: + case INSN_MMIO_DECODE_FAILED: /* * MMIO was accessed with an instruction that could not be * decoded or handled properly. It was likely not using io.h @@ -459,15 +459,15 @@ static int handle_mmio(struct pt_regs *regs, struct ve_info *ve) return -EIO; switch (mmio) { - case MMIO_READ: + case INSN_MMIO_READ: /* Zero-extend for 32-bit operation */ extend_size = size == 4 ? sizeof(*reg) : 0; break; - case MMIO_READ_ZERO_EXTEND: + case INSN_MMIO_READ_ZERO_EXTEND: /* Zero extend based on operand size */ extend_size = insn.opnd_bytes; break; - case MMIO_READ_SIGN_EXTEND: + case INSN_MMIO_READ_SIGN_EXTEND: /* Sign extend based on operand size */ extend_size = insn.opnd_bytes; if (size == 1 && val & BIT(7)) diff --git a/arch/x86/events/intel/core.c b/arch/x86/events/intel/core.c index dfd2c124cdf80a819570340caa2b886887d57890..bafdc2be479acedff68fac4fa9103cf83836ab36 100644 --- a/arch/x86/events/intel/core.c +++ b/arch/x86/events/intel/core.c @@ -6339,6 +6339,7 @@ __init int intel_pmu_init(void) break; case INTEL_FAM6_SAPPHIRERAPIDS_X: + case INTEL_FAM6_EMERALDRAPIDS_X: pmem = true; x86_pmu.late_ack = true; memcpy(hw_cache_event_ids, spr_hw_cache_event_ids, sizeof(hw_cache_event_ids)); diff --git a/arch/x86/events/intel/cstate.c b/arch/x86/events/intel/cstate.c index a2834bc93149a735295706e2599e5f4b5b667678..551741e79e038a20b2d7b519dd2a6a560b86d1f6 100644 --- a/arch/x86/events/intel/cstate.c +++ b/arch/x86/events/intel/cstate.c @@ -41,6 +41,7 @@ * MSR_CORE_C1_RES: CORE C1 Residency Counter * perf code: 0x00 * Available model: SLM,AMT,GLM,CNL,ICX,TNT,ADL,RPL + * MTL * Scope: Core (each processor core has a MSR) * MSR_CORE_C3_RESIDENCY: CORE C3 Residency Counter * perf code: 0x01 @@ -51,50 +52,50 @@ * perf code: 0x02 * Available model: SLM,AMT,NHM,WSM,SNB,IVB,HSW,BDW, * SKL,KNL,GLM,CNL,KBL,CML,ICL,ICX, - * TGL,TNT,RKL,ADL,RPL,SPR + * TGL,TNT,RKL,ADL,RPL,SPR,MTL * Scope: Core * MSR_CORE_C7_RESIDENCY: CORE C7 Residency Counter * perf code: 0x03 * Available model: SNB,IVB,HSW,BDW,SKL,CNL,KBL,CML, - * ICL,TGL,RKL,ADL,RPL + * ICL,TGL,RKL,ADL,RPL,MTL * Scope: Core * MSR_PKG_C2_RESIDENCY: Package C2 Residency Counter. * perf code: 0x00 * Available model: SNB,IVB,HSW,BDW,SKL,KNL,GLM,CNL, * KBL,CML,ICL,ICX,TGL,TNT,RKL,ADL, - * RPL,SPR + * RPL,SPR,MTL * Scope: Package (physical package) * MSR_PKG_C3_RESIDENCY: Package C3 Residency Counter. * perf code: 0x01 * Available model: NHM,WSM,SNB,IVB,HSW,BDW,SKL,KNL, * GLM,CNL,KBL,CML,ICL,TGL,TNT,RKL, - * ADL,RPL + * ADL,RPL,MTL * Scope: Package (physical package) * MSR_PKG_C6_RESIDENCY: Package C6 Residency Counter. * perf code: 0x02 * Available model: SLM,AMT,NHM,WSM,SNB,IVB,HSW,BDW, * SKL,KNL,GLM,CNL,KBL,CML,ICL,ICX, - * TGL,TNT,RKL,ADL,RPL,SPR + * TGL,TNT,RKL,ADL,RPL,SPR,MTL * Scope: Package (physical package) * MSR_PKG_C7_RESIDENCY: Package C7 Residency Counter. * perf code: 0x03 * Available model: NHM,WSM,SNB,IVB,HSW,BDW,SKL,CNL, - * KBL,CML,ICL,TGL,RKL,ADL,RPL + * KBL,CML,ICL,TGL,RKL,ADL,RPL,MTL * Scope: Package (physical package) * MSR_PKG_C8_RESIDENCY: Package C8 Residency Counter. * perf code: 0x04 * Available model: HSW ULT,KBL,CNL,CML,ICL,TGL,RKL, - * ADL,RPL + * ADL,RPL,MTL * Scope: Package (physical package) * MSR_PKG_C9_RESIDENCY: Package C9 Residency Counter. * perf code: 0x05 * Available model: HSW ULT,KBL,CNL,CML,ICL,TGL,RKL, - * ADL,RPL + * ADL,RPL,MTL * Scope: Package (physical package) * MSR_PKG_C10_RESIDENCY: Package C10 Residency Counter. * perf code: 0x06 * Available model: HSW ULT,KBL,GLM,CNL,CML,ICL,TGL, - * TNT,RKL,ADL,RPL + * TNT,RKL,ADL,RPL,MTL * Scope: Package (physical package) * */ @@ -676,6 +677,7 @@ static const struct x86_cpu_id intel_cstates_match[] __initconst = { X86_MATCH_INTEL_FAM6_MODEL(ICELAKE_X, &icx_cstates), X86_MATCH_INTEL_FAM6_MODEL(ICELAKE_D, &icx_cstates), X86_MATCH_INTEL_FAM6_MODEL(SAPPHIRERAPIDS_X, &icx_cstates), + X86_MATCH_INTEL_FAM6_MODEL(EMERALDRAPIDS_X, &icx_cstates), X86_MATCH_INTEL_FAM6_MODEL(TIGERLAKE_L, &icl_cstates), X86_MATCH_INTEL_FAM6_MODEL(TIGERLAKE, &icl_cstates), @@ -686,6 +688,8 @@ static const struct x86_cpu_id intel_cstates_match[] __initconst = { X86_MATCH_INTEL_FAM6_MODEL(RAPTORLAKE, &adl_cstates), X86_MATCH_INTEL_FAM6_MODEL(RAPTORLAKE_P, &adl_cstates), X86_MATCH_INTEL_FAM6_MODEL(RAPTORLAKE_S, &adl_cstates), + X86_MATCH_INTEL_FAM6_MODEL(METEORLAKE, &adl_cstates), + X86_MATCH_INTEL_FAM6_MODEL(METEORLAKE_L, &adl_cstates), { }, }; MODULE_DEVICE_TABLE(x86cpu, intel_cstates_match); diff --git a/arch/x86/events/intel/uncore.c b/arch/x86/events/intel/uncore.c index 6f1ccc57a6921fd0ffd2d7f924f96bf0e2907a6c..459b1aafd4d4a3d36fcce623575ba0beaa9aca3f 100644 --- a/arch/x86/events/intel/uncore.c +++ b/arch/x86/events/intel/uncore.c @@ -1833,6 +1833,7 @@ static const struct x86_cpu_id intel_uncore_match[] __initconst = { X86_MATCH_INTEL_FAM6_MODEL(RAPTORLAKE_P, &adl_uncore_init), X86_MATCH_INTEL_FAM6_MODEL(RAPTORLAKE_S, &adl_uncore_init), X86_MATCH_INTEL_FAM6_MODEL(SAPPHIRERAPIDS_X, &spr_uncore_init), + X86_MATCH_INTEL_FAM6_MODEL(EMERALDRAPIDS_X, &spr_uncore_init), X86_MATCH_INTEL_FAM6_MODEL(ATOM_TREMONT_D, &snr_uncore_init), {}, }; diff --git a/arch/x86/events/msr.c b/arch/x86/events/msr.c index ecced3a52668a129e3fbc38d38facaeff9c2ca61..c65d8906cbcf4de7e41f53a9a48d1625ca7cd4a2 100644 --- a/arch/x86/events/msr.c +++ b/arch/x86/events/msr.c @@ -69,6 +69,7 @@ static bool test_intel(int idx, void *data) case INTEL_FAM6_BROADWELL_G: case INTEL_FAM6_BROADWELL_X: case INTEL_FAM6_SAPPHIRERAPIDS_X: + case INTEL_FAM6_EMERALDRAPIDS_X: case INTEL_FAM6_ATOM_SILVERMONT: case INTEL_FAM6_ATOM_SILVERMONT_D: @@ -107,6 +108,8 @@ static bool test_intel(int idx, void *data) case INTEL_FAM6_RAPTORLAKE: case INTEL_FAM6_RAPTORLAKE_P: case INTEL_FAM6_RAPTORLAKE_S: + case INTEL_FAM6_METEORLAKE: + case INTEL_FAM6_METEORLAKE_L: if (idx == PERF_MSR_SMI || idx == PERF_MSR_PPERF) return true; break; diff --git a/arch/x86/events/rapl.c b/arch/x86/events/rapl.c index a829492bca4c193ab3ab30b85bd1fbb4e73b2b26..52e6e7ed4f78a655414ce1b7548c8f0b7d5d8822 100644 --- a/arch/x86/events/rapl.c +++ b/arch/x86/events/rapl.c @@ -800,13 +800,18 @@ static const struct x86_cpu_id rapl_model_match[] __initconst = { X86_MATCH_INTEL_FAM6_MODEL(ICELAKE_X, &model_hsx), X86_MATCH_INTEL_FAM6_MODEL(COMETLAKE_L, &model_skl), X86_MATCH_INTEL_FAM6_MODEL(COMETLAKE, &model_skl), + X86_MATCH_INTEL_FAM6_MODEL(TIGERLAKE_L, &model_skl), + X86_MATCH_INTEL_FAM6_MODEL(TIGERLAKE, &model_skl), X86_MATCH_INTEL_FAM6_MODEL(ALDERLAKE, &model_skl), X86_MATCH_INTEL_FAM6_MODEL(ALDERLAKE_L, &model_skl), X86_MATCH_INTEL_FAM6_MODEL(ALDERLAKE_N, &model_skl), X86_MATCH_INTEL_FAM6_MODEL(SAPPHIRERAPIDS_X, &model_spr), + X86_MATCH_INTEL_FAM6_MODEL(EMERALDRAPIDS_X, &model_spr), X86_MATCH_INTEL_FAM6_MODEL(RAPTORLAKE, &model_skl), X86_MATCH_INTEL_FAM6_MODEL(RAPTORLAKE_P, &model_skl), X86_MATCH_INTEL_FAM6_MODEL(RAPTORLAKE_S, &model_skl), + X86_MATCH_INTEL_FAM6_MODEL(METEORLAKE, &model_skl), + X86_MATCH_INTEL_FAM6_MODEL(METEORLAKE_L, &model_skl), {}, }; MODULE_DEVICE_TABLE(x86cpu, rapl_model_match); diff --git a/arch/x86/include/asm/acpi.h b/arch/x86/include/asm/acpi.h index 65064d9f7fa6ead3835194be565b10bf26cfb909..8eb74cf386dbe5cac24c25fca9422a99212ab9b7 100644 --- a/arch/x86/include/asm/acpi.h +++ b/arch/x86/include/asm/acpi.h @@ -14,6 +14,7 @@ #include #include #include +#include #ifdef CONFIG_ACPI_APEI # include @@ -63,6 +64,13 @@ extern int (*acpi_suspend_lowlevel)(void); /* Physical address to resume after wakeup */ unsigned long acpi_get_wakeup_address(void); +static inline bool acpi_skip_set_wakeup_address(void) +{ + return cpu_feature_enabled(X86_FEATURE_XENPV); +} + +#define acpi_skip_set_wakeup_address acpi_skip_set_wakeup_address + /* * Check if the CPU can handle C2 and deeper */ diff --git a/arch/x86/include/asm/insn-eval.h b/arch/x86/include/asm/insn-eval.h index f07faa61c7f3118e581a6f88114623f78a7dbde1..54368a43abf67e455dbea4ca800e51e3196759dc 100644 --- a/arch/x86/include/asm/insn-eval.h +++ b/arch/x86/include/asm/insn-eval.h @@ -32,16 +32,16 @@ int insn_fetch_from_user_inatomic(struct pt_regs *regs, bool insn_decode_from_regs(struct insn *insn, struct pt_regs *regs, unsigned char buf[MAX_INSN_SIZE], int buf_size); -enum mmio_type { - MMIO_DECODE_FAILED, - MMIO_WRITE, - MMIO_WRITE_IMM, - MMIO_READ, - MMIO_READ_ZERO_EXTEND, - MMIO_READ_SIGN_EXTEND, - MMIO_MOVS, +enum insn_mmio_type { + INSN_MMIO_DECODE_FAILED, + INSN_MMIO_WRITE, + INSN_MMIO_WRITE_IMM, + INSN_MMIO_READ, + INSN_MMIO_READ_ZERO_EXTEND, + INSN_MMIO_READ_SIGN_EXTEND, + INSN_MMIO_MOVS, }; -enum mmio_type insn_decode_mmio(struct insn *insn, int *bytes); +enum insn_mmio_type insn_decode_mmio(struct insn *insn, int *bytes); #endif /* _ASM_X86_INSN_EVAL_H */ diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index f35f1ff4427bb85273abab37fe40a9154bf11cd9..6aaae18f1854474d7811d0d41081c39fba27d3d8 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1111,6 +1111,7 @@ struct msr_bitmap_range { /* Xen emulation context */ struct kvm_xen { + struct mutex xen_lock; u32 xen_version; bool long_mode; bool runstate_update_flag; diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h index 37ff47552bcb7b57fdc75bc0246f2bb3876add4a..d3fe82c5d6b660d103788298d5f04a5a86a08104 100644 --- a/arch/x86/include/asm/msr-index.h +++ b/arch/x86/include/asm/msr-index.h @@ -566,6 +566,26 @@ #define MSR_AMD64_SEV_ES_ENABLED BIT_ULL(MSR_AMD64_SEV_ES_ENABLED_BIT) #define MSR_AMD64_SEV_SNP_ENABLED BIT_ULL(MSR_AMD64_SEV_SNP_ENABLED_BIT) +/* SNP feature bits enabled by the hypervisor */ +#define MSR_AMD64_SNP_VTOM BIT_ULL(3) +#define MSR_AMD64_SNP_REFLECT_VC BIT_ULL(4) +#define MSR_AMD64_SNP_RESTRICTED_INJ BIT_ULL(5) +#define MSR_AMD64_SNP_ALT_INJ BIT_ULL(6) +#define MSR_AMD64_SNP_DEBUG_SWAP BIT_ULL(7) +#define MSR_AMD64_SNP_PREVENT_HOST_IBS BIT_ULL(8) +#define MSR_AMD64_SNP_BTB_ISOLATION BIT_ULL(9) +#define MSR_AMD64_SNP_VMPL_SSS BIT_ULL(10) +#define MSR_AMD64_SNP_SECURE_TSC BIT_ULL(11) +#define MSR_AMD64_SNP_VMGEXIT_PARAM BIT_ULL(12) +#define MSR_AMD64_SNP_IBS_VIRT BIT_ULL(14) +#define MSR_AMD64_SNP_VMSA_REG_PROTECTION BIT_ULL(16) +#define MSR_AMD64_SNP_SMT_PROTECTION BIT_ULL(17) + +/* SNP feature bits reserved for future use. */ +#define MSR_AMD64_SNP_RESERVED_BIT13 BIT_ULL(13) +#define MSR_AMD64_SNP_RESERVED_BIT15 BIT_ULL(15) +#define MSR_AMD64_SNP_RESERVED_MASK GENMASK_ULL(63, 18) + #define MSR_AMD64_VIRT_SPEC_CTRL 0xc001011f /* AMD Collaborative Processor Performance Control MSRs */ diff --git a/arch/x86/include/uapi/asm/svm.h b/arch/x86/include/uapi/asm/svm.h index f69c168391aa58cf280ba6fb5aca6630e3593d1a..80e1df482337dfe064d809966a309cecd0930070 100644 --- a/arch/x86/include/uapi/asm/svm.h +++ b/arch/x86/include/uapi/asm/svm.h @@ -116,6 +116,12 @@ #define SVM_VMGEXIT_AP_CREATE 1 #define SVM_VMGEXIT_AP_DESTROY 2 #define SVM_VMGEXIT_HV_FEATURES 0x8000fffd +#define SVM_VMGEXIT_TERM_REQUEST 0x8000fffe +#define SVM_VMGEXIT_TERM_REASON(reason_set, reason_code) \ + /* SW_EXITINFO1[3:0] */ \ + (((((u64)reason_set) & 0xf)) | \ + /* SW_EXITINFO1[11:4] */ \ + ((((u64)reason_code) & 0xff) << 4)) #define SVM_VMGEXIT_UNSUPPORTED_EVENT 0x8000ffff /* Exit code reserved for hypervisor/software use */ diff --git a/arch/x86/kernel/cpu/aperfmperf.c b/arch/x86/kernel/cpu/aperfmperf.c index 1f60a2b279368e94c79f64ea43f736baf966f39f..fdbb5f07448fad0dd267868fdfb0a6394d9e45b9 100644 --- a/arch/x86/kernel/cpu/aperfmperf.c +++ b/arch/x86/kernel/cpu/aperfmperf.c @@ -330,7 +330,16 @@ static void __init bp_init_freq_invariance(void) static void disable_freq_invariance_workfn(struct work_struct *work) { + int cpu; + static_branch_disable(&arch_scale_freq_key); + + /* + * Set arch_freq_scale to a default value on all cpus + * This negates the effect of scaling + */ + for_each_possible_cpu(cpu) + per_cpu(arch_freq_scale, cpu) = SCHED_CAPACITY_SCALE; } static DECLARE_WORK(disable_freq_invariance_work, diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c index d970ddb0cc65bedf358c32321d202036cf792923..bca0bd8f4846413831da02ec7eee8ae9c12abd12 100644 --- a/arch/x86/kernel/cpu/bugs.c +++ b/arch/x86/kernel/cpu/bugs.c @@ -1981,6 +1981,8 @@ static int ib_prctl_set(struct task_struct *task, unsigned long ctrl) if (ctrl == PR_SPEC_FORCE_DISABLE) task_set_spec_ib_force_disable(task); task_update_spec_tif(task); + if (task == current) + indirect_branch_prediction_barrier(); break; default: return -ERANGE; diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c index efe0c30d3a12d5f126a129d81bcac921dda97252..77538abeb72afe7b99736efbe552d8293bdd8b60 100644 --- a/arch/x86/kernel/cpu/resctrl/monitor.c +++ b/arch/x86/kernel/cpu/resctrl/monitor.c @@ -146,6 +146,30 @@ static inline struct rmid_entry *__rmid_entry(u32 rmid) return entry; } +static int __rmid_read(u32 rmid, enum resctrl_event_id eventid, u64 *val) +{ + u64 msr_val; + + /* + * As per the SDM, when IA32_QM_EVTSEL.EvtID (bits 7:0) is configured + * with a valid event code for supported resource type and the bits + * IA32_QM_EVTSEL.RMID (bits 41:32) are configured with valid RMID, + * IA32_QM_CTR.data (bits 61:0) reports the monitored data. + * IA32_QM_CTR.Error (bit 63) and IA32_QM_CTR.Unavailable (bit 62) + * are error bits. + */ + wrmsr(MSR_IA32_QM_EVTSEL, eventid, rmid); + rdmsrl(MSR_IA32_QM_CTR, msr_val); + + if (msr_val & RMID_VAL_ERROR) + return -EIO; + if (msr_val & RMID_VAL_UNAVAIL) + return -EINVAL; + + *val = msr_val; + return 0; +} + static struct arch_mbm_state *get_arch_mbm_state(struct rdt_hw_domain *hw_dom, u32 rmid, enum resctrl_event_id eventid) @@ -172,8 +196,12 @@ void resctrl_arch_reset_rmid(struct rdt_resource *r, struct rdt_domain *d, struct arch_mbm_state *am; am = get_arch_mbm_state(hw_dom, rmid, eventid); - if (am) + if (am) { memset(am, 0, sizeof(*am)); + + /* Record any initial, non-zero count value. */ + __rmid_read(rmid, eventid, &am->prev_msr); + } } static u64 mbm_overflow_count(u64 prev_msr, u64 cur_msr, unsigned int width) @@ -191,25 +219,14 @@ int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain *d, struct rdt_hw_domain *hw_dom = resctrl_to_arch_dom(d); struct arch_mbm_state *am; u64 msr_val, chunks; + int ret; if (!cpumask_test_cpu(smp_processor_id(), &d->cpu_mask)) return -EINVAL; - /* - * As per the SDM, when IA32_QM_EVTSEL.EvtID (bits 7:0) is configured - * with a valid event code for supported resource type and the bits - * IA32_QM_EVTSEL.RMID (bits 41:32) are configured with valid RMID, - * IA32_QM_CTR.data (bits 61:0) reports the monitored data. - * IA32_QM_CTR.Error (bit 63) and IA32_QM_CTR.Unavailable (bit 62) - * are error bits. - */ - wrmsr(MSR_IA32_QM_EVTSEL, eventid, rmid); - rdmsrl(MSR_IA32_QM_CTR, msr_val); - - if (msr_val & RMID_VAL_ERROR) - return -EIO; - if (msr_val & RMID_VAL_UNAVAIL) - return -EINVAL; + ret = __rmid_read(rmid, eventid, &msr_val); + if (ret) + return ret; am = get_arch_mbm_state(hw_dom, rmid, eventid); if (am) { diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c index e5a48f05e7876adcfb942e48f930603c2375fd19..5993da21d82255737638c6684c74a83836463dd1 100644 --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c @@ -580,8 +580,10 @@ static int __rdtgroup_move_task(struct task_struct *tsk, /* * Ensure the task's closid and rmid are written before determining if * the task is current that will decide if it will be interrupted. + * This pairs with the full barrier between the rq->curr update and + * resctrl_sched_in() during context switch. */ - barrier(); + smp_mb(); /* * By now, the task's closid and rmid are set. If the task is current @@ -2401,6 +2403,14 @@ static void rdt_move_group_tasks(struct rdtgroup *from, struct rdtgroup *to, WRITE_ONCE(t->closid, to->closid); WRITE_ONCE(t->rmid, to->mon.rmid); + /* + * Order the closid/rmid stores above before the loads + * in task_curr(). This pairs with the full barrier + * between the rq->curr update and resctrl_sched_in() + * during context switch. + */ + smp_mb(); + /* * If the task is on a CPU, set the CPU in the mask. * The detection is inaccurate as tasks might move or diff --git a/arch/x86/kernel/crash.c b/arch/x86/kernel/crash.c index 9730c88530fc8b4c0a6c0bfc770acd087524ea10..305514431f26e03d9b50709cdaa4a2732f74e5c3 100644 --- a/arch/x86/kernel/crash.c +++ b/arch/x86/kernel/crash.c @@ -401,10 +401,8 @@ int crash_load_segments(struct kimage *image) kbuf.buf_align = ELF_CORE_HEADER_ALIGN; kbuf.mem = KEXEC_BUF_MEM_UNKNOWN; ret = kexec_add_buffer(&kbuf); - if (ret) { - vfree((void *)image->elf_headers); + if (ret) return ret; - } image->elf_load_addr = kbuf.mem; pr_debug("Loaded ELF headers at 0x%lx bufsz=0x%lx memsz=0x%lx\n", image->elf_load_addr, kbuf.bufsz, kbuf.memsz); diff --git a/arch/x86/kernel/i8259.c b/arch/x86/kernel/i8259.c index 3aa5304200c5073d380ddcbe240d49b09129fab2..4d8aff05a509ec4ca413b08e3d26ca812f5d94e7 100644 --- a/arch/x86/kernel/i8259.c +++ b/arch/x86/kernel/i8259.c @@ -114,6 +114,7 @@ static void make_8259A_irq(unsigned int irq) disable_irq_nosync(irq); io_apic_irqs &= ~(1<init(0); - for (i = 0; i < nr_legacy_irqs(); i++) + for (i = 0; i < nr_legacy_irqs(); i++) { irq_set_chip_and_handler(i, chip, handle_level_irq); + irq_set_status_flags(i, IRQ_LEVEL); + } } void __init init_IRQ(void) diff --git a/arch/x86/kernel/sev.c b/arch/x86/kernel/sev.c index a428c62330d371a6742ada0a2e9e578dd3f1def6..679026a640efd8e2e87a03d3b5e57cf18de1a6dd 100644 --- a/arch/x86/kernel/sev.c +++ b/arch/x86/kernel/sev.c @@ -1536,32 +1536,32 @@ static enum es_result vc_handle_mmio_movs(struct es_em_ctxt *ctxt, static enum es_result vc_handle_mmio(struct ghcb *ghcb, struct es_em_ctxt *ctxt) { struct insn *insn = &ctxt->insn; + enum insn_mmio_type mmio; unsigned int bytes = 0; - enum mmio_type mmio; enum es_result ret; u8 sign_byte; long *reg_data; mmio = insn_decode_mmio(insn, &bytes); - if (mmio == MMIO_DECODE_FAILED) + if (mmio == INSN_MMIO_DECODE_FAILED) return ES_DECODE_FAILED; - if (mmio != MMIO_WRITE_IMM && mmio != MMIO_MOVS) { + if (mmio != INSN_MMIO_WRITE_IMM && mmio != INSN_MMIO_MOVS) { reg_data = insn_get_modrm_reg_ptr(insn, ctxt->regs); if (!reg_data) return ES_DECODE_FAILED; } switch (mmio) { - case MMIO_WRITE: + case INSN_MMIO_WRITE: memcpy(ghcb->shared_buffer, reg_data, bytes); ret = vc_do_mmio(ghcb, ctxt, bytes, false); break; - case MMIO_WRITE_IMM: + case INSN_MMIO_WRITE_IMM: memcpy(ghcb->shared_buffer, insn->immediate1.bytes, bytes); ret = vc_do_mmio(ghcb, ctxt, bytes, false); break; - case MMIO_READ: + case INSN_MMIO_READ: ret = vc_do_mmio(ghcb, ctxt, bytes, true); if (ret) break; @@ -1572,7 +1572,7 @@ static enum es_result vc_handle_mmio(struct ghcb *ghcb, struct es_em_ctxt *ctxt) memcpy(reg_data, ghcb->shared_buffer, bytes); break; - case MMIO_READ_ZERO_EXTEND: + case INSN_MMIO_READ_ZERO_EXTEND: ret = vc_do_mmio(ghcb, ctxt, bytes, true); if (ret) break; @@ -1581,7 +1581,7 @@ static enum es_result vc_handle_mmio(struct ghcb *ghcb, struct es_em_ctxt *ctxt) memset(reg_data, 0, insn->opnd_bytes); memcpy(reg_data, ghcb->shared_buffer, bytes); break; - case MMIO_READ_SIGN_EXTEND: + case INSN_MMIO_READ_SIGN_EXTEND: ret = vc_do_mmio(ghcb, ctxt, bytes, true); if (ret) break; @@ -1600,7 +1600,7 @@ static enum es_result vc_handle_mmio(struct ghcb *ghcb, struct es_em_ctxt *ctxt) memset(reg_data, sign_byte, insn->opnd_bytes); memcpy(reg_data, ghcb->shared_buffer, bytes); break; - case MMIO_MOVS: + case INSN_MMIO_MOVS: ret = vc_handle_mmio_movs(ctxt, bytes); break; default: diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c index b14653b61470c2720267006f7e7166ebac159613..596061c1610e671885705baa3682af326bc93294 100644 --- a/arch/x86/kvm/cpuid.c +++ b/arch/x86/kvm/cpuid.c @@ -770,16 +770,22 @@ struct kvm_cpuid_array { int nent; }; +static struct kvm_cpuid_entry2 *get_next_cpuid(struct kvm_cpuid_array *array) +{ + if (array->nent >= array->maxnent) + return NULL; + + return &array->entries[array->nent++]; +} + static struct kvm_cpuid_entry2 *do_host_cpuid(struct kvm_cpuid_array *array, u32 function, u32 index) { - struct kvm_cpuid_entry2 *entry; + struct kvm_cpuid_entry2 *entry = get_next_cpuid(array); - if (array->nent >= array->maxnent) + if (!entry) return NULL; - entry = &array->entries[array->nent++]; - memset(entry, 0, sizeof(*entry)); entry->function = function; entry->index = index; @@ -956,22 +962,13 @@ static inline int __do_cpuid_func(struct kvm_cpuid_array *array, u32 function) entry->edx = edx.full; break; } - /* - * Per Intel's SDM, the 0x1f is a superset of 0xb, - * thus they can be handled by common code. - */ case 0x1f: case 0xb: /* - * Populate entries until the level type (ECX[15:8]) of the - * previous entry is zero. Note, CPUID EAX.{0x1f,0xb}.0 is - * the starting entry, filled by the primary do_host_cpuid(). + * No topology; a valid topology is indicated by the presence + * of subleaf 1. */ - for (i = 1; entry->ecx & 0xff00; ++i) { - entry = do_host_cpuid(array, function, i); - if (!entry) - goto out; - } + entry->eax = entry->ebx = entry->ecx = 0; break; case 0xd: { u64 permitted_xcr0 = kvm_caps.supported_xcr0 & xstate_get_guest_group_perm(); @@ -1202,6 +1199,9 @@ static inline int __do_cpuid_func(struct kvm_cpuid_array *array, u32 function) entry->ebx = entry->ecx = entry->edx = 0; break; case 0x8000001e: + /* Do not return host topology information. */ + entry->eax = entry->ebx = entry->ecx = 0; + entry->edx = 0; /* reserved */ break; case 0x8000001F: if (!kvm_cpu_cap_has(X86_FEATURE_SEV)) { diff --git a/arch/x86/kvm/svm/nested.c b/arch/x86/kvm/svm/nested.c index bc9cd7086fa972a8c9dd0600f7316278da4029ec..add65dd597569e70b484b18c9453d21723aeff5f 100644 --- a/arch/x86/kvm/svm/nested.c +++ b/arch/x86/kvm/svm/nested.c @@ -138,15 +138,13 @@ void recalc_intercepts(struct vcpu_svm *svm) c->intercepts[i] = h->intercepts[i]; if (g->int_ctl & V_INTR_MASKING_MASK) { - /* We only want the cr8 intercept bits of L1 */ - vmcb_clr_intercept(c, INTERCEPT_CR8_READ); - vmcb_clr_intercept(c, INTERCEPT_CR8_WRITE); - /* - * Once running L2 with HF_VINTR_MASK, EFLAGS.IF does not - * affect any interrupt we may want to inject; therefore, - * interrupt window vmexits are irrelevant to L0. + * Once running L2 with HF_VINTR_MASK, EFLAGS.IF and CR8 + * does not affect any interrupt we may want to inject; + * therefore, writes to CR8 are irrelevant to L0, as are + * interrupt window vmexits. */ + vmcb_clr_intercept(c, INTERCEPT_CR8_WRITE); vmcb_clr_intercept(c, INTERCEPT_VINTR); } diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index fc9008dbed33487b472e4966a20b56cbcc4c4580..7eec0226d56a246681fa04236d12ecb5bdf0fada 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -3440,18 +3440,15 @@ static u32 vmx_segment_access_rights(struct kvm_segment *var) { u32 ar; - if (var->unusable || !var->present) - ar = 1 << 16; - else { - ar = var->type & 15; - ar |= (var->s & 1) << 4; - ar |= (var->dpl & 3) << 5; - ar |= (var->present & 1) << 7; - ar |= (var->avl & 1) << 12; - ar |= (var->l & 1) << 13; - ar |= (var->db & 1) << 14; - ar |= (var->g & 1) << 15; - } + ar = var->type & 15; + ar |= (var->s & 1) << 4; + ar |= (var->dpl & 3) << 5; + ar |= (var->present & 1) << 7; + ar |= (var->avl & 1) << 12; + ar |= (var->l & 1) << 13; + ar |= (var->db & 1) << 14; + ar |= (var->g & 1) << 15; + ar |= (var->unusable || !var->present) << 16; return ar; } diff --git a/arch/x86/kvm/xen.c b/arch/x86/kvm/xen.c index 2e29bdc2949ca9fd456eb83993bd200c93c280cf..8fd41f5deae39f605b4865382a3c5b6e58dc0467 100644 --- a/arch/x86/kvm/xen.c +++ b/arch/x86/kvm/xen.c @@ -271,7 +271,15 @@ static void kvm_xen_update_runstate_guest(struct kvm_vcpu *v, bool atomic) * Attempt to obtain the GPC lock on *both* (if there are two) * gfn_to_pfn caches that cover the region. */ - read_lock_irqsave(&gpc1->lock, flags); + if (atomic) { + local_irq_save(flags); + if (!read_trylock(&gpc1->lock)) { + local_irq_restore(flags); + return; + } + } else { + read_lock_irqsave(&gpc1->lock, flags); + } while (!kvm_gpc_check(gpc1, user_len1)) { read_unlock_irqrestore(&gpc1->lock, flags); @@ -304,9 +312,18 @@ static void kvm_xen_update_runstate_guest(struct kvm_vcpu *v, bool atomic) * The guest's runstate_info is split across two pages and we * need to hold and validate both GPCs simultaneously. We can * declare a lock ordering GPC1 > GPC2 because nothing else - * takes them more than one at a time. + * takes them more than one at a time. Set a subclass on the + * gpc1 lock to make lockdep shut up about it. */ - read_lock(&gpc2->lock); + lock_set_subclass(&gpc1->lock.dep_map, 1, _THIS_IP_); + if (atomic) { + if (!read_trylock(&gpc2->lock)) { + read_unlock_irqrestore(&gpc1->lock, flags); + return; + } + } else { + read_lock(&gpc2->lock); + } if (!kvm_gpc_check(gpc2, user_len2)) { read_unlock(&gpc2->lock); @@ -590,26 +607,26 @@ int kvm_xen_hvm_set_attr(struct kvm *kvm, struct kvm_xen_hvm_attr *data) if (!IS_ENABLED(CONFIG_64BIT) && data->u.long_mode) { r = -EINVAL; } else { - mutex_lock(&kvm->lock); + mutex_lock(&kvm->arch.xen.xen_lock); kvm->arch.xen.long_mode = !!data->u.long_mode; - mutex_unlock(&kvm->lock); + mutex_unlock(&kvm->arch.xen.xen_lock); r = 0; } break; case KVM_XEN_ATTR_TYPE_SHARED_INFO: - mutex_lock(&kvm->lock); + mutex_lock(&kvm->arch.xen.xen_lock); r = kvm_xen_shared_info_init(kvm, data->u.shared_info.gfn); - mutex_unlock(&kvm->lock); + mutex_unlock(&kvm->arch.xen.xen_lock); break; case KVM_XEN_ATTR_TYPE_UPCALL_VECTOR: if (data->u.vector && data->u.vector < 0x10) r = -EINVAL; else { - mutex_lock(&kvm->lock); + mutex_lock(&kvm->arch.xen.xen_lock); kvm->arch.xen.upcall_vector = data->u.vector; - mutex_unlock(&kvm->lock); + mutex_unlock(&kvm->arch.xen.xen_lock); r = 0; } break; @@ -619,9 +636,9 @@ int kvm_xen_hvm_set_attr(struct kvm *kvm, struct kvm_xen_hvm_attr *data) break; case KVM_XEN_ATTR_TYPE_XEN_VERSION: - mutex_lock(&kvm->lock); + mutex_lock(&kvm->arch.xen.xen_lock); kvm->arch.xen.xen_version = data->u.xen_version; - mutex_unlock(&kvm->lock); + mutex_unlock(&kvm->arch.xen.xen_lock); r = 0; break; @@ -630,9 +647,9 @@ int kvm_xen_hvm_set_attr(struct kvm *kvm, struct kvm_xen_hvm_attr *data) r = -EOPNOTSUPP; break; } - mutex_lock(&kvm->lock); + mutex_lock(&kvm->arch.xen.xen_lock); kvm->arch.xen.runstate_update_flag = !!data->u.runstate_update_flag; - mutex_unlock(&kvm->lock); + mutex_unlock(&kvm->arch.xen.xen_lock); r = 0; break; @@ -647,7 +664,7 @@ int kvm_xen_hvm_get_attr(struct kvm *kvm, struct kvm_xen_hvm_attr *data) { int r = -ENOENT; - mutex_lock(&kvm->lock); + mutex_lock(&kvm->arch.xen.xen_lock); switch (data->type) { case KVM_XEN_ATTR_TYPE_LONG_MODE: @@ -686,7 +703,7 @@ int kvm_xen_hvm_get_attr(struct kvm *kvm, struct kvm_xen_hvm_attr *data) break; } - mutex_unlock(&kvm->lock); + mutex_unlock(&kvm->arch.xen.xen_lock); return r; } @@ -694,7 +711,7 @@ int kvm_xen_vcpu_set_attr(struct kvm_vcpu *vcpu, struct kvm_xen_vcpu_attr *data) { int idx, r = -ENOENT; - mutex_lock(&vcpu->kvm->lock); + mutex_lock(&vcpu->kvm->arch.xen.xen_lock); idx = srcu_read_lock(&vcpu->kvm->srcu); switch (data->type) { @@ -922,7 +939,7 @@ int kvm_xen_vcpu_set_attr(struct kvm_vcpu *vcpu, struct kvm_xen_vcpu_attr *data) } srcu_read_unlock(&vcpu->kvm->srcu, idx); - mutex_unlock(&vcpu->kvm->lock); + mutex_unlock(&vcpu->kvm->arch.xen.xen_lock); return r; } @@ -930,7 +947,7 @@ int kvm_xen_vcpu_get_attr(struct kvm_vcpu *vcpu, struct kvm_xen_vcpu_attr *data) { int r = -ENOENT; - mutex_lock(&vcpu->kvm->lock); + mutex_lock(&vcpu->kvm->arch.xen.xen_lock); switch (data->type) { case KVM_XEN_VCPU_ATTR_TYPE_VCPU_INFO: @@ -1013,7 +1030,7 @@ int kvm_xen_vcpu_get_attr(struct kvm_vcpu *vcpu, struct kvm_xen_vcpu_attr *data) break; } - mutex_unlock(&vcpu->kvm->lock); + mutex_unlock(&vcpu->kvm->arch.xen.xen_lock); return r; } @@ -1106,7 +1123,7 @@ int kvm_xen_hvm_config(struct kvm *kvm, struct kvm_xen_hvm_config *xhc) xhc->blob_size_32 || xhc->blob_size_64)) return -EINVAL; - mutex_lock(&kvm->lock); + mutex_lock(&kvm->arch.xen.xen_lock); if (xhc->msr && !kvm->arch.xen_hvm_config.msr) static_branch_inc(&kvm_xen_enabled.key); @@ -1115,7 +1132,7 @@ int kvm_xen_hvm_config(struct kvm *kvm, struct kvm_xen_hvm_config *xhc) memcpy(&kvm->arch.xen_hvm_config, xhc, sizeof(*xhc)); - mutex_unlock(&kvm->lock); + mutex_unlock(&kvm->arch.xen.xen_lock); return 0; } @@ -1658,15 +1675,7 @@ static int kvm_xen_set_evtchn(struct kvm_xen_evtchn *xe, struct kvm *kvm) mm_borrowed = true; } - /* - * For the irqfd workqueue, using the main kvm->lock mutex is - * fine since this function is invoked from kvm_set_irq() with - * no other lock held, no srcu. In future if it will be called - * directly from a vCPU thread (e.g. on hypercall for an IPI) - * then it may need to switch to using a leaf-node mutex for - * serializing the shared_info mapping. - */ - mutex_lock(&kvm->lock); + mutex_lock(&kvm->arch.xen.xen_lock); /* * It is theoretically possible for the page to be unmapped @@ -1695,7 +1704,7 @@ static int kvm_xen_set_evtchn(struct kvm_xen_evtchn *xe, struct kvm *kvm) srcu_read_unlock(&kvm->srcu, idx); } while(!rc); - mutex_unlock(&kvm->lock); + mutex_unlock(&kvm->arch.xen.xen_lock); if (mm_borrowed) kthread_unuse_mm(kvm->mm); @@ -1811,7 +1820,7 @@ static int kvm_xen_eventfd_update(struct kvm *kvm, int ret; /* Protect writes to evtchnfd as well as the idr lookup. */ - mutex_lock(&kvm->lock); + mutex_lock(&kvm->arch.xen.xen_lock); evtchnfd = idr_find(&kvm->arch.xen.evtchn_ports, port); ret = -ENOENT; @@ -1842,7 +1851,7 @@ static int kvm_xen_eventfd_update(struct kvm *kvm, } ret = 0; out_unlock: - mutex_unlock(&kvm->lock); + mutex_unlock(&kvm->arch.xen.xen_lock); return ret; } @@ -1905,10 +1914,10 @@ static int kvm_xen_eventfd_assign(struct kvm *kvm, evtchnfd->deliver.port.priority = data->u.evtchn.deliver.port.priority; } - mutex_lock(&kvm->lock); + mutex_lock(&kvm->arch.xen.xen_lock); ret = idr_alloc(&kvm->arch.xen.evtchn_ports, evtchnfd, port, port + 1, GFP_KERNEL); - mutex_unlock(&kvm->lock); + mutex_unlock(&kvm->arch.xen.xen_lock); if (ret >= 0) return 0; @@ -1926,9 +1935,9 @@ static int kvm_xen_eventfd_deassign(struct kvm *kvm, u32 port) { struct evtchnfd *evtchnfd; - mutex_lock(&kvm->lock); + mutex_lock(&kvm->arch.xen.xen_lock); evtchnfd = idr_remove(&kvm->arch.xen.evtchn_ports, port); - mutex_unlock(&kvm->lock); + mutex_unlock(&kvm->arch.xen.xen_lock); if (!evtchnfd) return -ENOENT; @@ -1946,7 +1955,7 @@ static int kvm_xen_eventfd_reset(struct kvm *kvm) int i; int n = 0; - mutex_lock(&kvm->lock); + mutex_lock(&kvm->arch.xen.xen_lock); /* * Because synchronize_srcu() cannot be called inside the @@ -1958,7 +1967,7 @@ static int kvm_xen_eventfd_reset(struct kvm *kvm) all_evtchnfds = kmalloc_array(n, sizeof(struct evtchnfd *), GFP_KERNEL); if (!all_evtchnfds) { - mutex_unlock(&kvm->lock); + mutex_unlock(&kvm->arch.xen.xen_lock); return -ENOMEM; } @@ -1967,7 +1976,7 @@ static int kvm_xen_eventfd_reset(struct kvm *kvm) all_evtchnfds[n++] = evtchnfd; idr_remove(&kvm->arch.xen.evtchn_ports, evtchnfd->send_port); } - mutex_unlock(&kvm->lock); + mutex_unlock(&kvm->arch.xen.xen_lock); synchronize_srcu(&kvm->srcu); @@ -2069,6 +2078,7 @@ void kvm_xen_destroy_vcpu(struct kvm_vcpu *vcpu) void kvm_xen_init_vm(struct kvm *kvm) { + mutex_init(&kvm->arch.xen.xen_lock); idr_init(&kvm->arch.xen.evtchn_ports); kvm_gpc_init(&kvm->arch.xen.shinfo_cache, kvm, NULL, KVM_HOST_USES_PFN); } diff --git a/arch/x86/lib/insn-eval.c b/arch/x86/lib/insn-eval.c index 21104c41cba04868e54d14f93a3592b9fa2541ba..558a605929db52f0bd05e8b547413d5d95e7eb90 100644 --- a/arch/x86/lib/insn-eval.c +++ b/arch/x86/lib/insn-eval.c @@ -1595,16 +1595,16 @@ bool insn_decode_from_regs(struct insn *insn, struct pt_regs *regs, * Returns: * * Type of the instruction. Size of the memory operand is stored in - * @bytes. If decode failed, MMIO_DECODE_FAILED returned. + * @bytes. If decode failed, INSN_MMIO_DECODE_FAILED returned. */ -enum mmio_type insn_decode_mmio(struct insn *insn, int *bytes) +enum insn_mmio_type insn_decode_mmio(struct insn *insn, int *bytes) { - enum mmio_type type = MMIO_DECODE_FAILED; + enum insn_mmio_type type = INSN_MMIO_DECODE_FAILED; *bytes = 0; if (insn_get_opcode(insn)) - return MMIO_DECODE_FAILED; + return INSN_MMIO_DECODE_FAILED; switch (insn->opcode.bytes[0]) { case 0x88: /* MOV m8,r8 */ @@ -1613,7 +1613,7 @@ enum mmio_type insn_decode_mmio(struct insn *insn, int *bytes) case 0x89: /* MOV m16/m32/m64, r16/m32/m64 */ if (!*bytes) *bytes = insn->opnd_bytes; - type = MMIO_WRITE; + type = INSN_MMIO_WRITE; break; case 0xc6: /* MOV m8, imm8 */ @@ -1622,7 +1622,7 @@ enum mmio_type insn_decode_mmio(struct insn *insn, int *bytes) case 0xc7: /* MOV m16/m32/m64, imm16/imm32/imm64 */ if (!*bytes) *bytes = insn->opnd_bytes; - type = MMIO_WRITE_IMM; + type = INSN_MMIO_WRITE_IMM; break; case 0x8a: /* MOV r8, m8 */ @@ -1631,7 +1631,7 @@ enum mmio_type insn_decode_mmio(struct insn *insn, int *bytes) case 0x8b: /* MOV r16/r32/r64, m16/m32/m64 */ if (!*bytes) *bytes = insn->opnd_bytes; - type = MMIO_READ; + type = INSN_MMIO_READ; break; case 0xa4: /* MOVS m8, m8 */ @@ -1640,7 +1640,7 @@ enum mmio_type insn_decode_mmio(struct insn *insn, int *bytes) case 0xa5: /* MOVS m16/m32/m64, m16/m32/m64 */ if (!*bytes) *bytes = insn->opnd_bytes; - type = MMIO_MOVS; + type = INSN_MMIO_MOVS; break; case 0x0f: /* Two-byte instruction */ @@ -1651,7 +1651,7 @@ enum mmio_type insn_decode_mmio(struct insn *insn, int *bytes) case 0xb7: /* MOVZX r32/r64, m16 */ if (!*bytes) *bytes = 2; - type = MMIO_READ_ZERO_EXTEND; + type = INSN_MMIO_READ_ZERO_EXTEND; break; case 0xbe: /* MOVSX r16/r32/r64, m8 */ @@ -1660,7 +1660,7 @@ enum mmio_type insn_decode_mmio(struct insn *insn, int *bytes) case 0xbf: /* MOVSX r32/r64, m16 */ if (!*bytes) *bytes = 2; - type = MMIO_READ_SIGN_EXTEND; + type = INSN_MMIO_READ_SIGN_EXTEND; break; } break; diff --git a/arch/x86/lib/iomap_copy_64.S b/arch/x86/lib/iomap_copy_64.S index a1f9416bf67a53df1f99f1315c48d78167f06060..6ff2f56cb0f71ae8a9066157c31babe112fc57aa 100644 --- a/arch/x86/lib/iomap_copy_64.S +++ b/arch/x86/lib/iomap_copy_64.S @@ -10,6 +10,6 @@ */ SYM_FUNC_START(__iowrite32_copy) movl %edx,%ecx - rep movsd + rep movsl RET SYM_FUNC_END(__iowrite32_copy) diff --git a/arch/x86/mm/init.c b/arch/x86/mm/init.c index d3987359d44140ae67c21bdc0798c210dc10fcf1..cb258f58fdc87935500c28053eef87ce1b02e206 100644 --- a/arch/x86/mm/init.c +++ b/arch/x86/mm/init.c @@ -26,6 +26,7 @@ #include #include #include +#include /* * We need to define the tracepoints somewhere, and tlb.c @@ -804,6 +805,9 @@ void __init poking_init(void) poking_mm = mm_alloc(); BUG_ON(!poking_mm); + /* Xen PV guests need the PGD to be pinned. */ + paravirt_arch_dup_mmap(NULL, poking_mm); + /* * Randomize the poking address, but make sure that the following page * will be mapped at the same PMD. We need 2 pages, so find space for 3, diff --git a/arch/x86/mm/pat/memtype.c b/arch/x86/mm/pat/memtype.c index 46de9cf5c91d2be26a72afdd4de86b87c2b2c3c2..fb4b1b5e0deab2fcbd1f1a2442431fe63adbbc24 100644 --- a/arch/x86/mm/pat/memtype.c +++ b/arch/x86/mm/pat/memtype.c @@ -387,7 +387,8 @@ static unsigned long pat_x_mtrr_type(u64 start, u64 end, u8 mtrr_type, uniform; mtrr_type = mtrr_type_lookup(start, end, &uniform); - if (mtrr_type != MTRR_TYPE_WRBACK) + if (mtrr_type != MTRR_TYPE_WRBACK && + mtrr_type != MTRR_TYPE_INVALID) return _PAGE_CACHE_MODE_UC_MINUS; return _PAGE_CACHE_MODE_WB; diff --git a/arch/x86/pci/mmconfig-shared.c b/arch/x86/pci/mmconfig-shared.c index 758cbfe55daa35519b0360c970cce25532598a9b..4b3efaa82ab7c14a747cea8dbe50ad6645cf1981 100644 --- a/arch/x86/pci/mmconfig-shared.c +++ b/arch/x86/pci/mmconfig-shared.c @@ -12,6 +12,7 @@ */ #include +#include #include #include #include @@ -442,17 +443,42 @@ static bool is_acpi_reserved(u64 start, u64 end, enum e820_type not_used) return mcfg_res.flags; } +static bool is_efi_mmio(u64 start, u64 end, enum e820_type not_used) +{ +#ifdef CONFIG_EFI + efi_memory_desc_t *md; + u64 size, mmio_start, mmio_end; + + for_each_efi_memory_desc(md) { + if (md->type == EFI_MEMORY_MAPPED_IO) { + size = md->num_pages << EFI_PAGE_SHIFT; + mmio_start = md->phys_addr; + mmio_end = mmio_start + size; + + /* + * N.B. Caller supplies (start, start + size), + * so to match, mmio_end is the first address + * *past* the EFI_MEMORY_MAPPED_IO area. + */ + if (mmio_start <= start && end <= mmio_end) + return true; + } + } +#endif + + return false; +} + typedef bool (*check_reserved_t)(u64 start, u64 end, enum e820_type type); static bool __ref is_mmconf_reserved(check_reserved_t is_reserved, struct pci_mmcfg_region *cfg, - struct device *dev, int with_e820) + struct device *dev, const char *method) { u64 addr = cfg->res.start; u64 size = resource_size(&cfg->res); u64 old_size = size; int num_buses; - char *method = with_e820 ? "E820" : "ACPI motherboard resources"; while (!is_reserved(addr, addr + size, E820_TYPE_RESERVED)) { size >>= 1; @@ -464,10 +490,10 @@ static bool __ref is_mmconf_reserved(check_reserved_t is_reserved, return false; if (dev) - dev_info(dev, "MMCONFIG at %pR reserved in %s\n", + dev_info(dev, "MMCONFIG at %pR reserved as %s\n", &cfg->res, method); else - pr_info(PREFIX "MMCONFIG at %pR reserved in %s\n", + pr_info(PREFIX "MMCONFIG at %pR reserved as %s\n", &cfg->res, method); if (old_size != size) { @@ -500,7 +526,8 @@ static bool __ref pci_mmcfg_check_reserved(struct device *dev, struct pci_mmcfg_region *cfg, int early) { if (!early && !acpi_disabled) { - if (is_mmconf_reserved(is_acpi_reserved, cfg, dev, 0)) + if (is_mmconf_reserved(is_acpi_reserved, cfg, dev, + "ACPI motherboard resource")) return true; if (dev) @@ -513,6 +540,10 @@ pci_mmcfg_check_reserved(struct device *dev, struct pci_mmcfg_region *cfg, int e "MMCONFIG at %pR not reserved in " "ACPI motherboard resources\n", &cfg->res); + + if (is_mmconf_reserved(is_efi_mmio, cfg, dev, + "EfiMemoryMappedIO")) + return true; } /* @@ -527,7 +558,8 @@ pci_mmcfg_check_reserved(struct device *dev, struct pci_mmcfg_region *cfg, int e /* Don't try to do this check unless configuration type 1 is available. how about type 2 ?*/ if (raw_pci_ops) - return is_mmconf_reserved(e820__mapped_all, cfg, dev, 1); + return is_mmconf_reserved(e820__mapped_all, cfg, dev, + "E820 entry"); return false; } diff --git a/arch/x86/pci/xen.c b/arch/x86/pci/xen.c index b94f727251b6436d3101cb6d0abe04c7fc13a660..8babce71915fe514aa24cfdb25f36c98ddc90369 100644 --- a/arch/x86/pci/xen.c +++ b/arch/x86/pci/xen.c @@ -392,6 +392,7 @@ static void xen_teardown_msi_irqs(struct pci_dev *dev) msi_for_each_desc(msidesc, &dev->dev, MSI_DESC_ASSOCIATED) { for (i = 0; i < msidesc->nvec_used; i++) xen_destroy_irq(msidesc->irq + i); + msidesc->irq = 0; } } @@ -433,6 +434,7 @@ static struct msi_domain_ops xen_pci_msi_domain_ops = { }; static struct msi_domain_info xen_pci_msi_domain_info = { + .flags = MSI_FLAG_PCI_MSIX | MSI_FLAG_FREE_MSI_DESCS | MSI_FLAG_DEV_SYSFS, .ops = &xen_pci_msi_domain_ops, }; diff --git a/arch/x86/um/elfcore.c b/arch/x86/um/elfcore.c index 48a3eb09d95165b4baea6e2cc8f00071e032ec76..650cdbbdaf45e5b2bb77605e60a9ccb52d397fee 100644 --- a/arch/x86/um/elfcore.c +++ b/arch/x86/um/elfcore.c @@ -7,7 +7,7 @@ #include -Elf32_Half elf_core_extra_phdrs(void) +Elf32_Half elf_core_extra_phdrs(struct coredump_params *cprm) { return vsyscall_ehdr ? (((struct elfhdr *)vsyscall_ehdr)->e_phnum) : 0; } @@ -60,7 +60,7 @@ int elf_core_write_extra_data(struct coredump_params *cprm) return 1; } -size_t elf_core_extra_data_size(void) +size_t elf_core_extra_data_size(struct coredump_params *cprm) { if ( vsyscall_ehdr ) { const struct elfhdr *const ehdrp = diff --git a/arch/x86/xen/p2m.c b/arch/x86/xen/p2m.c index 58db86f7b3846ef04b749154f0c9e9178654a806..9bdc3b656b2c49ac1bc1323b3e6ec4bf0ec418af 100644 --- a/arch/x86/xen/p2m.c +++ b/arch/x86/xen/p2m.c @@ -134,11 +134,6 @@ static inline unsigned p2m_mid_index(unsigned long pfn) return (pfn / P2M_PER_PAGE) % P2M_MID_PER_PAGE; } -static inline unsigned p2m_index(unsigned long pfn) -{ - return pfn % P2M_PER_PAGE; -} - static void p2m_top_mfn_init(unsigned long *top) { unsigned i; diff --git a/arch/xtensa/include/asm/processor.h b/arch/xtensa/include/asm/processor.h index 228e4dff5fb2d3b7fb08e59743915d8223940dc8..a6d09fe0483110e46aaa8c48333d52677e2c86c0 100644 --- a/arch/xtensa/include/asm/processor.h +++ b/arch/xtensa/include/asm/processor.h @@ -154,11 +154,6 @@ struct thread_struct { unsigned long ra; /* kernel's a0: return address and window call size */ unsigned long sp; /* kernel's a1: stack pointer */ - /* struct xtensa_cpuinfo info; */ - - unsigned long bad_vaddr; /* last user fault */ - unsigned long bad_uaddr; /* last kernel fault accessing user space */ - unsigned long error_code; #ifdef CONFIG_HAVE_HW_BREAKPOINT struct perf_event *ptrace_bp[XCHAL_NUM_IBREAK]; struct perf_event *ptrace_wp[XCHAL_NUM_DBREAK]; @@ -176,10 +171,6 @@ struct thread_struct { { \ ra: 0, \ sp: sizeof(init_stack) + (long) &init_stack, \ - /*info: {0}, */ \ - bad_vaddr: 0, \ - bad_uaddr: 0, \ - error_code: 0, \ } diff --git a/arch/xtensa/kernel/traps.c b/arch/xtensa/kernel/traps.c index 0c25e035ff107b83f40c659e4ee1d7caf419910e..cd98366a9b238841c7e4aaa021fb623d3eb52136 100644 --- a/arch/xtensa/kernel/traps.c +++ b/arch/xtensa/kernel/traps.c @@ -362,8 +362,6 @@ static void do_unaligned_user(struct pt_regs *regs) __die_if_kernel("Unhandled unaligned exception in kernel", regs, SIGKILL); - current->thread.bad_vaddr = regs->excvaddr; - current->thread.error_code = -3; pr_info_ratelimited("Unaligned memory access to %08lx in '%s' " "(pid = %d, pc = %#010lx)\n", regs->excvaddr, current->comm, diff --git a/arch/xtensa/mm/fault.c b/arch/xtensa/mm/fault.c index 8c781b05c0bdd9d55d62ff456c8150b94cf9d493..faf7cf35a0ee3d244f60332ad967da9e6d500142 100644 --- a/arch/xtensa/mm/fault.c +++ b/arch/xtensa/mm/fault.c @@ -206,8 +206,6 @@ void do_page_fault(struct pt_regs *regs) bad_area: mmap_read_unlock(mm); if (user_mode(regs)) { - current->thread.bad_vaddr = address; - current->thread.error_code = is_write; force_sig_fault(SIGSEGV, code, (void *) address); return; } @@ -232,7 +230,6 @@ void do_page_fault(struct pt_regs *regs) /* Send a sigbus, regardless of whether we were in kernel * or user mode. */ - current->thread.bad_vaddr = address; force_sig_fault(SIGBUS, BUS_ADRERR, (void *) address); /* Kernel mode? Handle exceptions or die */ @@ -252,7 +249,6 @@ bad_page_fault(struct pt_regs *regs, unsigned long address, int sig) if ((entry = search_exception_tables(regs->pc)) != NULL) { pr_debug("%s: Exception at pc=%#010lx (%lx)\n", current->comm, regs->pc, entry->fixup); - current->thread.bad_uaddr = address; regs->pc = entry->fixup; return; } diff --git a/block/Kconfig b/block/Kconfig index 444c5ab3b67e2fe1c3bed6989a819d0a02127604..5d9d9c84d51657f1c6d9e5b25bd3cac406308d76 100644 --- a/block/Kconfig +++ b/block/Kconfig @@ -6,7 +6,6 @@ menuconfig BLOCK bool "Enable the block layer" if EXPERT default y select SBITMAP - select SRCU help Provide block layer support for the kernel. diff --git a/block/bfq-cgroup.c b/block/bfq-cgroup.c index 1b2829e99dad02cab2b04a417d87f9481023053e..7d9b15f0dbd57917aa7d91f42b61ea9c305be2df 100644 --- a/block/bfq-cgroup.c +++ b/block/bfq-cgroup.c @@ -316,14 +316,12 @@ struct bfq_group *bfqq_group(struct bfq_queue *bfqq) static void bfqg_get(struct bfq_group *bfqg) { - bfqg->ref++; + refcount_inc(&bfqg->ref); } static void bfqg_put(struct bfq_group *bfqg) { - bfqg->ref--; - - if (bfqg->ref == 0) + if (refcount_dec_and_test(&bfqg->ref)) kfree(bfqg); } @@ -530,7 +528,7 @@ static struct blkg_policy_data *bfq_pd_alloc(gfp_t gfp, struct request_queue *q, } /* see comments in bfq_bic_update_cgroup for why refcounting */ - bfqg_get(bfqg); + refcount_set(&bfqg->ref, 1); return &bfqg->pd; } diff --git a/block/bfq-iosched.h b/block/bfq-iosched.h index 41aa151ccc223457621abb30e24ec8a02701346b..466e4865ace63d1f12f897686442652da6e615a2 100644 --- a/block/bfq-iosched.h +++ b/block/bfq-iosched.h @@ -928,7 +928,7 @@ struct bfq_group { char blkg_path[128]; /* reference counter (see comments in bfq_bic_update_cgroup) */ - int ref; + refcount_t ref; /* Is bfq_group still online? */ bool online; diff --git a/block/bio.c b/block/bio.c index 5f96fcae3f7549e99664bba9dedac6b43c3d7f96..ab59a491a883e3b657edc27b8f8cfd042843f9e0 100644 --- a/block/bio.c +++ b/block/bio.c @@ -1401,6 +1401,27 @@ void __bio_advance(struct bio *bio, unsigned bytes) } EXPORT_SYMBOL(__bio_advance); +void bio_copy_data_iter(struct bio *dst, struct bvec_iter *dst_iter, + struct bio *src, struct bvec_iter *src_iter) +{ + while (src_iter->bi_size && dst_iter->bi_size) { + struct bio_vec src_bv = bio_iter_iovec(src, *src_iter); + struct bio_vec dst_bv = bio_iter_iovec(dst, *dst_iter); + unsigned int bytes = min(src_bv.bv_len, dst_bv.bv_len); + void *src_buf = bvec_kmap_local(&src_bv); + void *dst_buf = bvec_kmap_local(&dst_bv); + + memcpy(dst_buf, src_buf, bytes); + + kunmap_local(dst_buf); + kunmap_local(src_buf); + + bio_advance_iter_single(src, src_iter, bytes); + bio_advance_iter_single(dst, dst_iter, bytes); + } +} +EXPORT_SYMBOL(bio_copy_data_iter); + /** * bio_copy_data - copy contents of data buffers from one bio to another * @src: source bio @@ -1414,21 +1435,7 @@ void bio_copy_data(struct bio *dst, struct bio *src) struct bvec_iter src_iter = src->bi_iter; struct bvec_iter dst_iter = dst->bi_iter; - while (src_iter.bi_size && dst_iter.bi_size) { - struct bio_vec src_bv = bio_iter_iovec(src, src_iter); - struct bio_vec dst_bv = bio_iter_iovec(dst, dst_iter); - unsigned int bytes = min(src_bv.bv_len, dst_bv.bv_len); - void *src_buf = bvec_kmap_local(&src_bv); - void *dst_buf = bvec_kmap_local(&dst_bv); - - memcpy(dst_buf, src_buf, bytes); - - kunmap_local(dst_buf); - kunmap_local(src_buf); - - bio_advance_iter_single(src, &src_iter, bytes); - bio_advance_iter_single(dst, &dst_iter, bytes); - } + bio_copy_data_iter(dst, &dst_iter, src, &src_iter); } EXPORT_SYMBOL(bio_copy_data); diff --git a/block/blk-cgroup.c b/block/blk-cgroup.c index ce6a2b7d3dfb2b405e013393ffe8e90e92be504e..4c94a6560f625ebde92cfc95fa26337c5b2c60ed 100644 --- a/block/blk-cgroup.c +++ b/block/blk-cgroup.c @@ -1455,6 +1455,10 @@ int blkcg_activate_policy(struct request_queue *q, list_for_each_entry_reverse(blkg, &q->blkg_list, q_node) pol->pd_init_fn(blkg->pd[pol->plid]); + if (pol->pd_online_fn) + list_for_each_entry_reverse(blkg, &q->blkg_list, q_node) + pol->pd_online_fn(blkg->pd[pol->plid]); + __set_bit(pol->plid, q->blkcg_pols); ret = 0; diff --git a/block/blk-core.c b/block/blk-core.c index 9321767470dc5f5b3bcfd2939851a15a6dffe2c2..b5098355d8b27349d3ac08b6728afbd392fe2725 100644 --- a/block/blk-core.c +++ b/block/blk-core.c @@ -283,12 +283,9 @@ static void blk_free_queue(struct request_queue *q) * * Decrements the refcount of the request_queue and free it when the refcount * reaches 0. - * - * Context: Can sleep. */ void blk_put_queue(struct request_queue *q) { - might_sleep(); if (refcount_dec_and_test(&q->refs)) blk_free_queue(q); } diff --git a/block/blk-merge.c b/block/blk-merge.c index 35a8f75cc45d1c7c8e63ecad419a19368a4e0260..b7c193d67185de9dc6b3a320fe49272229ebc32e 100644 --- a/block/blk-merge.c +++ b/block/blk-merge.c @@ -309,6 +309,16 @@ static struct bio *bio_split_rw(struct bio *bio, const struct queue_limits *lim, *segs = nsegs; return NULL; split: + /* + * We can't sanely support splitting for a REQ_NOWAIT bio. End it + * with EAGAIN if splitting is required and return an error pointer. + */ + if (bio->bi_opf & REQ_NOWAIT) { + bio->bi_status = BLK_STS_AGAIN; + bio_endio(bio); + return ERR_PTR(-EAGAIN); + } + *segs = nsegs; /* @@ -358,11 +368,13 @@ struct bio *__bio_split_to_limits(struct bio *bio, default: split = bio_split_rw(bio, lim, nr_segs, bs, get_max_io_size(bio, lim) << SECTOR_SHIFT); + if (IS_ERR(split)) + return NULL; break; } if (split) { - /* there isn't chance to merge the splitted bio */ + /* there isn't chance to merge the split bio */ split->bi_opf |= REQ_NOMERGE; blkcg_bio_issue_init(split); diff --git a/block/blk-mq.c b/block/blk-mq.c index c5cf0dbca1db8dcae3930451b0ba56161e91753c..9d463f7563bc5cbebcdd2a22114046c6b4306e15 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -2890,6 +2890,7 @@ static inline struct request *blk_mq_get_cached_request(struct request_queue *q, struct blk_plug *plug, struct bio **bio, unsigned int nsegs) { struct request *rq; + enum hctx_type type, hctx_type; if (!plug) return NULL; @@ -2902,7 +2903,10 @@ static inline struct request *blk_mq_get_cached_request(struct request_queue *q, return NULL; } - if (blk_mq_get_hctx_type((*bio)->bi_opf) != rq->mq_hctx->type) + type = blk_mq_get_hctx_type((*bio)->bi_opf); + hctx_type = rq->mq_hctx->type; + if (type != hctx_type && + !(type == HCTX_TYPE_READ && hctx_type == HCTX_TYPE_DEFAULT)) return NULL; if (op_is_flush(rq->cmd_flags) != op_is_flush((*bio)->bi_opf)) return NULL; @@ -2951,8 +2955,11 @@ void blk_mq_submit_bio(struct bio *bio) blk_status_t ret; bio = blk_queue_bounce(bio, q); - if (bio_may_exceed_limits(bio, &q->limits)) + if (bio_may_exceed_limits(bio, &q->limits)) { bio = __bio_split_to_limits(bio, &q->limits, &nr_segs); + if (!bio) + return; + } if (!bio_integrity_prep(bio)) return; diff --git a/block/genhd.c b/block/genhd.c index ab3cbe44196f77c1b69a6847813997d3396e96b0..23cf83b3331cdea5c916fbf01cb5b92aeb2f7cf8 100644 --- a/block/genhd.c +++ b/block/genhd.c @@ -1201,10 +1201,21 @@ struct class block_class = { .dev_uevent = block_uevent, }; +static char *block_devnode(struct device *dev, umode_t *mode, + kuid_t *uid, kgid_t *gid) +{ + struct gendisk *disk = dev_to_disk(dev); + + if (disk->fops->devnode) + return disk->fops->devnode(disk, mode); + return NULL; +} + const struct device_type disk_type = { .name = "disk", .groups = disk_attr_groups, .release = disk_release, + .devnode = block_devnode, }; #ifdef CONFIG_PROC_FS diff --git a/drivers/accel/Kconfig b/drivers/accel/Kconfig index 2d88f2afee8cee4fe634b2d1ecda70d68021f292..c437206aa3f11a9a7b8fb568bf72f6ef6db5c41d 100644 --- a/drivers/accel/Kconfig +++ b/drivers/accel/Kconfig @@ -24,6 +24,7 @@ menuconfig DRM_ACCEL different device files, called accel/accel* (in /dev, sysfs and debugfs). +source "drivers/accel/habanalabs/Kconfig" source "drivers/accel/ivpu/Kconfig" endif diff --git a/drivers/accel/Makefile b/drivers/accel/Makefile index b1036dbc0ba41b9e36cc160ed56383c984130db3..07aa77aed1c8dd27dd63b0ed658cb923c5cde5f4 100644 --- a/drivers/accel/Makefile +++ b/drivers/accel/Makefile @@ -1,3 +1,4 @@ # SPDX-License-Identifier: GPL-2.0-only +obj-y += habanalabs/ obj-y += ivpu/ diff --git a/drivers/misc/habanalabs/Kconfig b/drivers/accel/habanalabs/Kconfig similarity index 81% rename from drivers/misc/habanalabs/Kconfig rename to drivers/accel/habanalabs/Kconfig index bd01d0d940c018ec6e769b1d67364d37e005f62c..be85336107f9fad731dfe317a58dc0c449ae2060 100644 --- a/drivers/misc/habanalabs/Kconfig +++ b/drivers/accel/habanalabs/Kconfig @@ -3,8 +3,10 @@ # HabanaLabs AI accelerators driver # -config HABANA_AI - tristate "HabanaAI accelerators (habanalabs)" +config DRM_ACCEL_HABANALABS + tristate "HabanaLabs AI accelerators" + depends on DRM_ACCEL + depends on X86_64 depends on PCI && HAS_IOMEM select GENERIC_ALLOCATOR select HWMON @@ -19,7 +21,7 @@ config HABANA_AI the user to submit workloads to the devices. The user-space interface is described in - include/uapi/misc/habanalabs.h + include/uapi/drm/habanalabs_accel.h If unsure, say N. diff --git a/drivers/misc/habanalabs/Makefile b/drivers/accel/habanalabs/Makefile similarity index 88% rename from drivers/misc/habanalabs/Makefile rename to drivers/accel/habanalabs/Makefile index a48a9e0969ed2da61bbf20be752f80e57a0d7ace..98510cdd506643b59e3e8e4817f999697a9844e3 100644 --- a/drivers/misc/habanalabs/Makefile +++ b/drivers/accel/habanalabs/Makefile @@ -3,7 +3,7 @@ # Makefile for HabanaLabs AI accelerators driver # -obj-$(CONFIG_HABANA_AI) := habanalabs.o +obj-$(CONFIG_DRM_ACCEL_HABANALABS) := habanalabs.o include $(src)/common/Makefile habanalabs-y += $(HL_COMMON_FILES) diff --git a/drivers/misc/habanalabs/common/Makefile b/drivers/accel/habanalabs/common/Makefile similarity index 100% rename from drivers/misc/habanalabs/common/Makefile rename to drivers/accel/habanalabs/common/Makefile diff --git a/drivers/misc/habanalabs/common/asid.c b/drivers/accel/habanalabs/common/asid.c similarity index 100% rename from drivers/misc/habanalabs/common/asid.c rename to drivers/accel/habanalabs/common/asid.c diff --git a/drivers/misc/habanalabs/common/command_buffer.c b/drivers/accel/habanalabs/common/command_buffer.c similarity index 95% rename from drivers/misc/habanalabs/common/command_buffer.c rename to drivers/accel/habanalabs/common/command_buffer.c index 2b332991ac6a75f9a809eb55375ed579e6accb99..3a0535ac28b1ba4857e663319da5ac0ade86b9f3 100644 --- a/drivers/misc/habanalabs/common/command_buffer.c +++ b/drivers/accel/habanalabs/common/command_buffer.c @@ -5,7 +5,7 @@ * All Rights Reserved. */ -#include +#include #include "habanalabs.h" #include @@ -88,6 +88,7 @@ static void cb_fini(struct hl_device *hdev, struct hl_cb *cb) static void cb_do_release(struct hl_device *hdev, struct hl_cb *cb) { if (cb->is_pool) { + atomic_set(&cb->is_handle_destroyed, 0); spin_lock(&hdev->cb_pool_lock); list_add(&cb->pool_list, &hdev->cb_pool); spin_unlock(&hdev->cb_pool_lock); @@ -298,8 +299,25 @@ int hl_cb_create(struct hl_device *hdev, struct hl_mem_mgr *mmg, int hl_cb_destroy(struct hl_mem_mgr *mmg, u64 cb_handle) { + struct hl_cb *cb; int rc; + cb = hl_cb_get(mmg, cb_handle); + if (!cb) { + dev_dbg(mmg->dev, "CB destroy failed, no CB was found for handle %#llx\n", + cb_handle); + return -EINVAL; + } + + /* Make sure that CB handle isn't destroyed more than once */ + rc = atomic_cmpxchg(&cb->is_handle_destroyed, 0, 1); + hl_cb_put(cb); + if (rc) { + dev_dbg(mmg->dev, "CB destroy failed, handle %#llx was already destroyed\n", + cb_handle); + return -EINVAL; + } + rc = hl_mmap_mem_buf_put_handle(mmg, cb_handle); if (rc < 0) return rc; /* Invalid handle */ @@ -350,7 +368,7 @@ int hl_cb_ioctl(struct hl_fpriv *hpriv, void *data) int rc; if (!hl_device_operational(hdev, &status)) { - dev_warn_ratelimited(hdev->dev, + dev_dbg_ratelimited(hdev->dev, "Device is %s. Can't execute CB IOCTL\n", hdev->status[status]); return -EBUSY; diff --git a/drivers/misc/habanalabs/common/command_submission.c b/drivers/accel/habanalabs/common/command_submission.c similarity index 96% rename from drivers/misc/habanalabs/common/command_submission.c rename to drivers/accel/habanalabs/common/command_submission.c index ea0e5101c10ed6ea9f7e71e0c63ec09c941b55ae..8270db0a72a26b44bb9c7ee8bee88729559c554e 100644 --- a/drivers/misc/habanalabs/common/command_submission.c +++ b/drivers/accel/habanalabs/common/command_submission.c @@ -5,7 +5,7 @@ * All Rights Reserved. */ -#include +#include #include "habanalabs.h" #include @@ -13,7 +13,8 @@ #define HL_CS_FLAGS_TYPE_MASK (HL_CS_FLAGS_SIGNAL | HL_CS_FLAGS_WAIT | \ HL_CS_FLAGS_COLLECTIVE_WAIT | HL_CS_FLAGS_RESERVE_SIGNALS_ONLY | \ - HL_CS_FLAGS_UNRESERVE_SIGNALS_ONLY | HL_CS_FLAGS_ENGINE_CORE_COMMAND) + HL_CS_FLAGS_UNRESERVE_SIGNALS_ONLY | HL_CS_FLAGS_ENGINE_CORE_COMMAND | \ + HL_CS_FLAGS_FLUSH_PCI_HBW_WRITES) #define MAX_TS_ITER_NUM 10 @@ -397,8 +398,16 @@ static void hl_complete_job(struct hl_device *hdev, struct hl_cs_job *job) * flow by calling 'hl_hw_queue_update_ci'. */ if (cs_needs_completion(cs) && - (job->queue_type == QUEUE_TYPE_EXT || job->queue_type == QUEUE_TYPE_HW)) + (job->queue_type == QUEUE_TYPE_EXT || job->queue_type == QUEUE_TYPE_HW)) { + + /* In CS based completions, the timestamp is already available, + * so no need to extract it from job + */ + if (hdev->asic_prop.completion_mode == HL_COMPLETION_MODE_JOB) + cs->completion_timestamp = job->timestamp; + cs_put(cs); + } hl_cs_job_put(job); } @@ -775,7 +784,7 @@ static void cs_do_release(struct kref *ref) } if (cs->timestamp) { - cs->fence->timestamp = ktime_get(); + cs->fence->timestamp = cs->completion_timestamp; hl_push_cs_outcome(hdev, &cs->ctx->outcome_store, cs->sequence, cs->fence->timestamp, cs->fence->error); } @@ -1117,6 +1126,27 @@ void hl_release_pending_user_interrupts(struct hl_device *hdev) wake_pending_user_interrupt_threads(interrupt); } +static void force_complete_cs(struct hl_device *hdev) +{ + struct hl_cs *cs; + + spin_lock(&hdev->cs_mirror_lock); + + list_for_each_entry(cs, &hdev->cs_mirror_list, mirror_node) { + cs->fence->error = -EIO; + complete_all(&cs->fence->completion); + } + + spin_unlock(&hdev->cs_mirror_lock); +} + +void hl_abort_waitings_for_completion(struct hl_device *hdev) +{ + force_complete_cs(hdev); + force_complete_multi_cs(hdev); + hl_release_pending_user_interrupts(hdev); +} + static void job_wq_completion(struct work_struct *work) { struct hl_cs_job *job = container_of(work, struct hl_cs_job, @@ -1274,6 +1304,8 @@ static enum hl_cs_type hl_cs_get_cs_type(u32 cs_type_flags) return CS_UNRESERVE_SIGNALS; else if (cs_type_flags & HL_CS_FLAGS_ENGINE_CORE_COMMAND) return CS_TYPE_ENGINE_CORE; + else if (cs_type_flags & HL_CS_FLAGS_FLUSH_PCI_HBW_WRITES) + return CS_TYPE_FLUSH_PCI_HBW_WRITES; else return CS_TYPE_DEFAULT; } @@ -1286,6 +1318,13 @@ static int hl_cs_sanity_checks(struct hl_fpriv *hpriv, union hl_cs_args *args) enum hl_device_status status; enum hl_cs_type cs_type; bool is_sync_stream; + int i; + + for (i = 0 ; i < sizeof(args->in.pad) ; i++) + if (args->in.pad[i]) { + dev_dbg(hdev->dev, "Padding bytes must be 0\n"); + return -EINVAL; + } if (!hl_device_operational(hdev, &status)) { return -EBUSY; @@ -2422,6 +2461,21 @@ static int cs_ioctl_engine_cores(struct hl_fpriv *hpriv, u64 engine_cores, return rc; } +static int cs_ioctl_flush_pci_hbw_writes(struct hl_fpriv *hpriv) +{ + struct hl_device *hdev = hpriv->hdev; + struct asic_fixed_properties *prop = &hdev->asic_prop; + + if (!prop->hbw_flush_reg) { + dev_dbg(hdev->dev, "HBW flush is not supported\n"); + return -EOPNOTSUPP; + } + + RREG32(prop->hbw_flush_reg); + + return 0; +} + int hl_cs_ioctl(struct hl_fpriv *hpriv, void *data) { union hl_cs_args *args = data; @@ -2478,6 +2532,9 @@ int hl_cs_ioctl(struct hl_fpriv *hpriv, void *data) rc = cs_ioctl_engine_cores(hpriv, args->in.engine_cores, args->in.num_engine_cores, args->in.core_command); break; + case CS_TYPE_FLUSH_PCI_HBW_WRITES: + rc = cs_ioctl_flush_pci_hbw_writes(hpriv); + break; default: rc = cs_ioctl_default(hpriv, chunks, num_chunks, &cs_seq, args->in.cs_flags, @@ -2569,7 +2626,9 @@ static int hl_wait_for_fence(struct hl_ctx *ctx, u64 seq, struct hl_fence *fence *status = CS_WAIT_STATUS_BUSY; } - if (error == -ETIMEDOUT || error == -EIO) + if (completion_rc == -ERESTARTSYS) + rc = completion_rc; + else if (error == -ETIMEDOUT || error == -EIO) rc = error; return rc; @@ -2699,7 +2758,8 @@ static int hl_cs_poll_fences(struct multi_cs_data *mcs_data, struct multi_cs_com break; default: dev_err(hdev->dev, "Invalid fence status\n"); - return -EINVAL; + rc = -EINVAL; + break; } } @@ -2828,6 +2888,9 @@ static int hl_wait_multi_cs_completion(struct multi_cs_data *mcs_data, if (completion_rc > 0) mcs_data->timestamp = mcs_compl->timestamp; + if (completion_rc == -ERESTARTSYS) + return completion_rc; + mcs_data->wait_status = completion_rc; return 0; @@ -2870,7 +2933,13 @@ static int hl_multi_cs_wait_ioctl(struct hl_fpriv *hpriv, void *data) u32 size_to_copy; u64 *cs_seq_arr; u8 seq_arr_len; - int rc; + int rc, i; + + for (i = 0 ; i < sizeof(args->in.pad) ; i++) + if (args->in.pad[i]) { + dev_dbg(hdev->dev, "Padding bytes must be 0\n"); + return -EINVAL; + } if (!hdev->supports_wait_for_multi_cs) { dev_err(hdev->dev, "Wait for multi CS is not supported\n"); @@ -2973,15 +3042,15 @@ static int hl_multi_cs_wait_ioctl(struct hl_fpriv *hpriv, void *data) free_seq_arr: kfree(cs_seq_arr); - if (rc) - return rc; - - if (mcs_data.wait_status == -ERESTARTSYS) { + if (rc == -ERESTARTSYS) { dev_err_ratelimited(hdev->dev, "user process got signal while waiting for Multi-CS\n"); - return -EINTR; + rc = -EINTR; } + if (rc) + return rc; + /* update output args */ memset(args, 0, sizeof(*args)); @@ -3119,19 +3188,18 @@ static int ts_buff_get_kernel_ts_record(struct hl_mmap_mem_buf *buf, goto start_over; } } else { + /* Fill up the new registration node info */ + requested_offset_record->ts_reg_info.buf = buf; + requested_offset_record->ts_reg_info.cq_cb = cq_cb; + requested_offset_record->ts_reg_info.timestamp_kernel_addr = + (u64 *) ts_buff->user_buff_address + ts_offset; + requested_offset_record->cq_kernel_addr = + (u64 *) cq_cb->kernel_address + cq_offset; + requested_offset_record->cq_target_value = target_value; + spin_unlock_irqrestore(wait_list_lock, flags); } - /* Fill up the new registration node info */ - requested_offset_record->ts_reg_info.in_use = 1; - requested_offset_record->ts_reg_info.buf = buf; - requested_offset_record->ts_reg_info.cq_cb = cq_cb; - requested_offset_record->ts_reg_info.timestamp_kernel_addr = - (u64 *) ts_buff->user_buff_address + ts_offset; - requested_offset_record->cq_kernel_addr = - (u64 *) cq_cb->kernel_address + cq_offset; - requested_offset_record->cq_target_value = target_value; - *pend = requested_offset_record; dev_dbg(buf->mmg->dev, "Found available node in TS kernel CB %p\n", @@ -3179,7 +3247,7 @@ static int _hl_interrupt_wait_ioctl(struct hl_device *hdev, struct hl_ctx *ctx, goto put_cq_cb; } - /* Find first available record */ + /* get ts buffer record */ rc = ts_buff_get_kernel_ts_record(buf, cq_cb, ts_offset, cq_counters_offset, target_value, &interrupt->wait_list_lock, &pend); @@ -3227,7 +3295,19 @@ static int _hl_interrupt_wait_ioctl(struct hl_device *hdev, struct hl_ctx *ctx, * Note that we cannot have sorted list by target value, * in order to shorten the list pass loop, since * same list could have nodes for different cq counter handle. + * Note: + * Mark ts buff offset as in use here in the spinlock protection area + * to avoid getting in the re-use section in ts_buff_get_kernel_ts_record + * before adding the node to the list. this scenario might happen when + * multiple threads are racing on same offset and one thread could + * set the ts buff in ts_buff_get_kernel_ts_record then the other thread + * takes over and get to ts_buff_get_kernel_ts_record and then we will try + * to re-use the same ts buff offset, and will try to delete a non existing + * node from the list. */ + if (register_ts_record) + pend->ts_reg_info.in_use = 1; + list_add_tail(&pend->wait_list_node, &interrupt->wait_list_head); spin_unlock_irqrestore(&interrupt->wait_list_lock, flags); @@ -3489,14 +3569,15 @@ static int hl_interrupt_wait_ioctl(struct hl_fpriv *hpriv, void *data) int hl_wait_ioctl(struct hl_fpriv *hpriv, void *data) { + struct hl_device *hdev = hpriv->hdev; union hl_wait_cs_args *args = data; u32 flags = args->in.flags; int rc; - /* If the device is not operational, no point in waiting for any command submission or - * user interrupt + /* If the device is not operational, or if an error has happened and user should release the + * device, there is no point in waiting for any command submission or user interrupt. */ - if (!hl_device_operational(hpriv->hdev, NULL)) + if (!hl_device_operational(hpriv->hdev, NULL) || hdev->reset_info.watchdog_active) return -EBUSY; if (flags & HL_WAIT_CS_FLAGS_INTERRUPT) diff --git a/drivers/misc/habanalabs/common/context.c b/drivers/accel/habanalabs/common/context.c similarity index 100% rename from drivers/misc/habanalabs/common/context.c rename to drivers/accel/habanalabs/common/context.c diff --git a/drivers/misc/habanalabs/common/debugfs.c b/drivers/accel/habanalabs/common/debugfs.c similarity index 100% rename from drivers/misc/habanalabs/common/debugfs.c rename to drivers/accel/habanalabs/common/debugfs.c diff --git a/drivers/misc/habanalabs/common/decoder.c b/drivers/accel/habanalabs/common/decoder.c similarity index 100% rename from drivers/misc/habanalabs/common/decoder.c rename to drivers/accel/habanalabs/common/decoder.c diff --git a/drivers/misc/habanalabs/common/device.c b/drivers/accel/habanalabs/common/device.c similarity index 95% rename from drivers/misc/habanalabs/common/device.c rename to drivers/accel/habanalabs/common/device.c index 87ab329e65d49dd5fbb8ae03bfed1c695546a702..9933e5858a36330c172a89d39bf3134efb1c80a5 100644 --- a/drivers/misc/habanalabs/common/device.c +++ b/drivers/accel/habanalabs/common/device.c @@ -7,7 +7,7 @@ #define pr_fmt(fmt) "habanalabs: " fmt -#include +#include #include "habanalabs.h" #include @@ -428,8 +428,10 @@ static void hpriv_release(struct kref *ref) */ reset_device = hdev->reset_upon_device_release || hdev->reset_info.watchdog_active; - /* Unless device is reset in any case, check idle status and reset if device is not idle */ - if (!reset_device && hdev->pdev && !hdev->pldm) + /* Check the device idle status and reset if not idle. + * Skip it if already in reset, or if device is going to be reset in any case. + */ + if (!hdev->reset_info.in_reset && !reset_device && hdev->pdev && !hdev->pldm) device_is_idle = hdev->asic_funcs->is_device_idle(hdev, idle_mask, HL_BUSY_ENGINES_MASK_EXT_SIZE, NULL); if (!device_is_idle) { @@ -511,11 +513,6 @@ static int hl_device_release(struct inode *inode, struct file *filp) return 0; } - /* Each pending user interrupt holds the user's context, hence we - * must release them all before calling hl_ctx_mgr_fini(). - */ - hl_release_pending_user_interrupts(hpriv->hdev); - hl_ctx_mgr_fini(hdev, &hpriv->ctx_mgr); hl_mem_mgr_fini(&hpriv->mem_mgr); @@ -1428,8 +1425,8 @@ static void handle_reset_trigger(struct hl_device *hdev, u32 flags) int hl_device_reset(struct hl_device *hdev, u32 flags) { bool hard_reset, from_hard_reset_thread, fw_reset, hard_instead_soft = false, - reset_upon_device_release = false, schedule_hard_reset = false, delay_reset, - from_dev_release, from_watchdog_thread; + reset_upon_device_release = false, schedule_hard_reset = false, + delay_reset, from_dev_release, from_watchdog_thread; u64 idle_mask[HL_BUSY_ENGINES_MASK_EXT_SIZE] = {0}; struct hl_ctx *ctx; int i, rc; @@ -1446,12 +1443,17 @@ int hl_device_reset(struct hl_device *hdev, u32 flags) delay_reset = !!(flags & HL_DRV_RESET_DELAY); from_watchdog_thread = !!(flags & HL_DRV_RESET_FROM_WD_THR); + if (!hard_reset && (hl_device_status(hdev) == HL_DEVICE_STATUS_MALFUNCTION)) { + dev_dbg(hdev->dev, "soft-reset isn't supported on a malfunctioning device\n"); + return 0; + } + if (!hard_reset && !hdev->asic_prop.supports_compute_reset) { hard_instead_soft = true; hard_reset = true; } - if (hdev->reset_upon_device_release && (flags & HL_DRV_RESET_DEV_RELEASE)) { + if (hdev->reset_upon_device_release && from_dev_release) { if (hard_reset) { dev_crit(hdev->dev, "Aborting reset because hard-reset is mutually exclusive with reset-on-device-release\n"); @@ -1512,6 +1514,7 @@ int hl_device_reset(struct hl_device *hdev, u32 flags) &hdev->device_release_watchdog_work.reset_work); if (from_dev_release) { + hdev->reset_info.in_compute_reset = 0; flags |= HL_DRV_RESET_HARD; flags &= ~HL_DRV_RESET_DEV_RELEASE; hard_reset = true; @@ -1566,7 +1569,8 @@ int hl_device_reset(struct hl_device *hdev, u32 flags) if (rc == -EBUSY) { if (hdev->device_fini_pending) { dev_crit(hdev->dev, - "Failed to kill all open processes, stopping hard reset\n"); + "%s Failed to kill all open processes, stopping hard reset\n", + dev_name(&(hdev)->pdev->dev)); goto out_err; } @@ -1576,7 +1580,8 @@ int hl_device_reset(struct hl_device *hdev, u32 flags) if (rc) { dev_crit(hdev->dev, - "Failed to kill all open processes, stopping hard reset\n"); + "%s Failed to kill all open processes, stopping hard reset\n", + dev_name(&(hdev)->pdev->dev)); goto out_err; } @@ -1627,14 +1632,16 @@ int hl_device_reset(struct hl_device *hdev, u32 flags) * ensure driver puts the driver in a unusable state */ dev_crit(hdev->dev, - "Consecutive FW fatal errors received, stopping hard reset\n"); + "%s Consecutive FW fatal errors received, stopping hard reset\n", + dev_name(&(hdev)->pdev->dev)); rc = -EIO; goto out_err; } if (hdev->kernel_ctx) { dev_crit(hdev->dev, - "kernel ctx was alive during hard reset, something is terribly wrong\n"); + "%s kernel ctx was alive during hard reset, something is terribly wrong\n", + dev_name(&(hdev)->pdev->dev)); rc = -EBUSY; goto out_err; } @@ -1732,7 +1739,7 @@ int hl_device_reset(struct hl_device *hdev, u32 flags) rc = hdev->asic_funcs->scrub_device_mem(hdev); if (rc) { dev_err(hdev->dev, "scrub mem failed from device reset (%d)\n", rc); - return rc; + goto out_err; } spin_lock(&hdev->reset_info.lock); @@ -1752,9 +1759,13 @@ int hl_device_reset(struct hl_device *hdev, u32 flags) hdev->reset_info.needs_reset = false; if (hard_reset) - dev_info(hdev->dev, "Successfully finished resetting the device\n"); + dev_info(hdev->dev, + "Successfully finished resetting the %s device\n", + dev_name(&(hdev)->pdev->dev)); else - dev_dbg(hdev->dev, "Successfully finished resetting the device\n"); + dev_dbg(hdev->dev, + "Successfully finished resetting the %s device\n", + dev_name(&(hdev)->pdev->dev)); if (hard_reset) { hdev->reset_info.hard_reset_cnt++; @@ -1789,7 +1800,9 @@ int hl_device_reset(struct hl_device *hdev, u32 flags) hdev->reset_info.in_compute_reset = 0; if (hard_reset) { - dev_err(hdev->dev, "Failed to reset! Device is NOT usable\n"); + dev_err(hdev->dev, + "%s Failed to reset! Device is NOT usable\n", + dev_name(&(hdev)->pdev->dev)); hdev->reset_info.hard_reset_cnt++; } else if (reset_upon_device_release) { spin_unlock(&hdev->reset_info.lock); @@ -1870,6 +1883,8 @@ int hl_device_cond_reset(struct hl_device *hdev, u32 flags, u64 event_mask) hl_ctx_put(ctx); + hl_abort_waitings_for_completion(hdev); + return 0; device_reset: @@ -2186,7 +2201,8 @@ int hl_device_init(struct hl_device *hdev, struct class *hclass) } dev_notice(hdev->dev, - "Successfully added device to habanalabs driver\n"); + "Successfully added device %s to habanalabs driver\n", + dev_name(&(hdev)->pdev->dev)); hdev->init_done = true; @@ -2235,11 +2251,11 @@ int hl_device_init(struct hl_device *hdev, struct class *hclass) device_cdev_sysfs_add(hdev); if (hdev->pdev) dev_err(&hdev->pdev->dev, - "Failed to initialize hl%d. Device is NOT usable !\n", - hdev->cdev_idx); + "Failed to initialize hl%d. Device %s is NOT usable !\n", + hdev->cdev_idx, dev_name(&(hdev)->pdev->dev)); else - pr_err("Failed to initialize hl%d. Device is NOT usable !\n", - hdev->cdev_idx); + pr_err("Failed to initialize hl%d. Device %s is NOT usable !\n", + hdev->cdev_idx, dev_name(&(hdev)->pdev->dev)); return rc; } @@ -2295,7 +2311,8 @@ void hl_device_fini(struct hl_device *hdev) if (ktime_compare(ktime_get(), timeout) > 0) { dev_crit(hdev->dev, - "Failed to remove device because reset function did not finish\n"); + "%s Failed to remove device because reset function did not finish\n", + dev_name(&(hdev)->pdev->dev)); return; } } @@ -2363,7 +2380,7 @@ void hl_device_fini(struct hl_device *hdev) hl_mmu_fini(hdev); - vfree(hdev->captured_err_info.pgf_info.user_mappings); + vfree(hdev->captured_err_info.page_fault_info.user_mappings); hl_eq_fini(hdev, &hdev->event_queue); @@ -2402,7 +2419,12 @@ void hl_device_fini(struct hl_device *hdev) */ inline u32 hl_rreg(struct hl_device *hdev, u32 reg) { - return readl(hdev->rmmio + reg); + u32 val = readl(hdev->rmmio + reg); + + if (unlikely(trace_habanalabs_rreg32_enabled())) + trace_habanalabs_rreg32(hdev->dev, reg, val); + + return val; } /* @@ -2417,12 +2439,17 @@ inline u32 hl_rreg(struct hl_device *hdev, u32 reg) */ inline void hl_wreg(struct hl_device *hdev, u32 reg, u32 val) { + if (unlikely(trace_habanalabs_wreg32_enabled())) + trace_habanalabs_wreg32(hdev->dev, reg, val); + writel(val, hdev->rmmio + reg); } void hl_capture_razwi(struct hl_device *hdev, u64 addr, u16 *engine_id, u16 num_of_engines, u8 flags) { + struct razwi_info *razwi_info = &hdev->captured_err_info.razwi_info; + if (num_of_engines > HL_RAZWI_MAX_NUM_OF_ENGINES_PER_RTR) { dev_err(hdev->dev, "Number of possible razwi initiators (%u) exceeded limit (%u)\n", @@ -2431,15 +2458,17 @@ void hl_capture_razwi(struct hl_device *hdev, u64 addr, u16 *engine_id, u16 num_ } /* In case it's the first razwi since the device was opened, capture its parameters */ - if (atomic_cmpxchg(&hdev->captured_err_info.razwi_info_recorded, 0, 1)) + if (atomic_cmpxchg(&hdev->captured_err_info.razwi_info.razwi_detected, 0, 1)) return; - hdev->captured_err_info.razwi.timestamp = ktime_to_ns(ktime_get()); - hdev->captured_err_info.razwi.addr = addr; - hdev->captured_err_info.razwi.num_of_possible_engines = num_of_engines; - memcpy(&hdev->captured_err_info.razwi.engine_id[0], &engine_id[0], + razwi_info->razwi.timestamp = ktime_to_ns(ktime_get()); + razwi_info->razwi.addr = addr; + razwi_info->razwi.num_of_possible_engines = num_of_engines; + memcpy(&razwi_info->razwi.engine_id[0], &engine_id[0], num_of_engines * sizeof(u16)); - hdev->captured_err_info.razwi.flags = flags; + razwi_info->razwi.flags = flags; + + razwi_info->razwi_info_available = true; } void hl_handle_razwi(struct hl_device *hdev, u64 addr, u16 *engine_id, u16 num_of_engines, @@ -2453,7 +2482,7 @@ void hl_handle_razwi(struct hl_device *hdev, u64 addr, u16 *engine_id, u16 num_o static void hl_capture_user_mappings(struct hl_device *hdev, bool is_pmmu) { - struct page_fault_info *pgf_info = &hdev->captured_err_info.pgf_info; + struct page_fault_info *pgf_info = &hdev->captured_err_info.page_fault_info; struct hl_vm_phys_pg_pack *phys_pg_pack = NULL; struct hl_vm_hash_node *hnode; struct hl_userptr *userptr; @@ -2515,14 +2544,18 @@ static void hl_capture_user_mappings(struct hl_device *hdev, bool is_pmmu) void hl_capture_page_fault(struct hl_device *hdev, u64 addr, u16 eng_id, bool is_pmmu) { + struct page_fault_info *pgf_info = &hdev->captured_err_info.page_fault_info; + /* Capture only the first page fault */ - if (atomic_cmpxchg(&hdev->captured_err_info.pgf_info_recorded, 0, 1)) + if (atomic_cmpxchg(&pgf_info->page_fault_detected, 0, 1)) return; - hdev->captured_err_info.pgf_info.pgf.timestamp = ktime_to_ns(ktime_get()); - hdev->captured_err_info.pgf_info.pgf.addr = addr; - hdev->captured_err_info.pgf_info.pgf.engine_id = eng_id; + pgf_info->page_fault.timestamp = ktime_to_ns(ktime_get()); + pgf_info->page_fault.addr = addr; + pgf_info->page_fault.engine_id = eng_id; hl_capture_user_mappings(hdev, is_pmmu); + + pgf_info->page_fault_info_available = true; } void hl_handle_page_fault(struct hl_device *hdev, u64 addr, u16 eng_id, bool is_pmmu, diff --git a/drivers/misc/habanalabs/common/firmware_if.c b/drivers/accel/habanalabs/common/firmware_if.c similarity index 93% rename from drivers/misc/habanalabs/common/firmware_if.c rename to drivers/accel/habanalabs/common/firmware_if.c index 228b92278e480ba904faebd9b3093cf1c5709415..da892d8fb3d6dbe296e99ae203f98005d8b28010 100644 --- a/drivers/misc/habanalabs/common/firmware_if.c +++ b/drivers/accel/habanalabs/common/firmware_if.c @@ -14,8 +14,32 @@ #include #include +#include + #define FW_FILE_MAX_SIZE 0x1400000 /* maximum size of 20MB */ +static char *comms_cmd_str_arr[COMMS_INVLD_LAST] = { + [COMMS_NOOP] = __stringify(COMMS_NOOP), + [COMMS_CLR_STS] = __stringify(COMMS_CLR_STS), + [COMMS_RST_STATE] = __stringify(COMMS_RST_STATE), + [COMMS_PREP_DESC] = __stringify(COMMS_PREP_DESC), + [COMMS_DATA_RDY] = __stringify(COMMS_DATA_RDY), + [COMMS_EXEC] = __stringify(COMMS_EXEC), + [COMMS_RST_DEV] = __stringify(COMMS_RST_DEV), + [COMMS_GOTO_WFE] = __stringify(COMMS_GOTO_WFE), + [COMMS_SKIP_BMC] = __stringify(COMMS_SKIP_BMC), + [COMMS_PREP_DESC_ELBI] = __stringify(COMMS_PREP_DESC_ELBI), +}; + +static char *comms_sts_str_arr[COMMS_STS_INVLD_LAST] = { + [COMMS_STS_NOOP] = __stringify(COMMS_STS_NOOP), + [COMMS_STS_ACK] = __stringify(COMMS_STS_ACK), + [COMMS_STS_OK] = __stringify(COMMS_STS_OK), + [COMMS_STS_ERR] = __stringify(COMMS_STS_ERR), + [COMMS_STS_VALID_ERR] = __stringify(COMMS_STS_VALID_ERR), + [COMMS_STS_TIMEOUT_ERR] = __stringify(COMMS_STS_TIMEOUT_ERR), +}; + static char *extract_fw_ver_from_str(const char *fw_str) { char *str, *fw_ver, *whitespace; @@ -311,7 +335,7 @@ int hl_fw_send_cpu_message(struct hl_device *hdev, u32 hw_queue_id, u32 *msg, dev_dbg(hdev->dev, "Device CPU packet timeout (0x%x) due to FW reset\n", tmp); else - dev_err(hdev->dev, "Device CPU packet timeout (0x%x)\n", tmp); + dev_err(hdev->dev, "Device CPU packet timeout (status = 0x%x)\n", tmp); hdev->device_cpu_disabled = true; goto out; } @@ -1322,13 +1346,12 @@ static void detect_cpu_boot_status(struct hl_device *hdev, u32 status) break; default: dev_err(hdev->dev, - "Device boot progress - Invalid status code %d\n", - status); + "Device boot progress - Invalid or unexpected status code %d\n", status); break; } } -static int hl_fw_wait_preboot_ready(struct hl_device *hdev) +int hl_fw_wait_preboot_ready(struct hl_device *hdev) { struct pre_fw_load_props *pre_fw_load = &hdev->fw_loader.pre_fw_load; u32 status; @@ -1353,8 +1376,8 @@ static int hl_fw_wait_preboot_ready(struct hl_device *hdev) pre_fw_load->wait_for_preboot_timeout); if (rc) { - dev_err(hdev->dev, "CPU boot ready status timeout\n"); detect_cpu_boot_status(hdev, status); + dev_err(hdev->dev, "CPU boot ready timeout (status = %d)\n", status); /* If we read all FF, then something is totally wrong, no point * of reading specific errors @@ -1634,6 +1657,7 @@ static void hl_fw_dynamic_send_cmd(struct hl_device *hdev, val = FIELD_PREP(COMMS_COMMAND_CMD_MASK, cmd); val |= FIELD_PREP(COMMS_COMMAND_SIZE_MASK, size); + trace_habanalabs_comms_send_cmd(hdev->dev, comms_cmd_str_arr[cmd]); WREG32(le32_to_cpu(dyn_regs->kmd_msg_to_cpu), val); } @@ -1691,6 +1715,8 @@ static int hl_fw_dynamic_wait_for_status(struct hl_device *hdev, dyn_regs = &fw_loader->dynamic_loader.comm_desc.cpu_dyn_regs; + trace_habanalabs_comms_wait_status(hdev->dev, comms_sts_str_arr[expected_status]); + /* Wait for expected status */ rc = hl_poll_timeout( hdev, @@ -1706,6 +1732,8 @@ static int hl_fw_dynamic_wait_for_status(struct hl_device *hdev, return -EIO; } + trace_habanalabs_comms_wait_status_done(hdev->dev, comms_sts_str_arr[expected_status]); + /* * skip storing FW response for NOOP to preserve the actual desired * FW status @@ -1778,6 +1806,8 @@ int hl_fw_dynamic_send_protocol_cmd(struct hl_device *hdev, { int rc; + trace_habanalabs_comms_protocol_cmd(hdev->dev, comms_cmd_str_arr[cmd]); + /* first send clear command to clean former commands */ rc = hl_fw_dynamic_send_clear_cmd(hdev, fw_loader); if (rc) @@ -1884,7 +1914,7 @@ static int hl_fw_dynamic_validate_memory_bound(struct hl_device *hdev, * * @hdev: pointer to the habanalabs device structure * @fw_loader: managing structure for loading device's FW - * @fw_desc: the descriptor form FW + * @fw_desc: the descriptor from FW * * @return 0 on success, otherwise non-zero error code */ @@ -1901,11 +1931,11 @@ static int hl_fw_dynamic_validate_descriptor(struct hl_device *hdev, int rc; if (le32_to_cpu(fw_desc->header.magic) != HL_COMMS_DESC_MAGIC) - dev_warn(hdev->dev, "Invalid magic for dynamic FW descriptor (%x)\n", + dev_dbg(hdev->dev, "Invalid magic for dynamic FW descriptor (%x)\n", fw_desc->header.magic); if (fw_desc->header.version != HL_COMMS_DESC_VER) - dev_warn(hdev->dev, "Invalid version for dynamic FW descriptor (%x)\n", + dev_dbg(hdev->dev, "Invalid version for dynamic FW descriptor (%x)\n", fw_desc->header.version); /* @@ -1976,6 +2006,43 @@ static int hl_fw_dynamic_validate_response(struct hl_device *hdev, return rc; } +/* + * hl_fw_dynamic_read_descriptor_msg - read and show the ascii msg that sent by fw + * + * @hdev: pointer to the habanalabs device structure + * @fw_desc: the descriptor from FW + */ +static void hl_fw_dynamic_read_descriptor_msg(struct hl_device *hdev, + struct lkd_fw_comms_desc *fw_desc) +{ + int i; + char *msg; + + for (i = 0 ; i < LKD_FW_ASCII_MSG_MAX ; i++) { + if (!fw_desc->ascii_msg[i].valid) + return; + + /* force NULL termination */ + msg = fw_desc->ascii_msg[i].msg; + msg[LKD_FW_ASCII_MSG_MAX_LEN - 1] = '\0'; + + switch (fw_desc->ascii_msg[i].msg_lvl) { + case LKD_FW_ASCII_MSG_ERR: + dev_err(hdev->dev, "fw: %s", fw_desc->ascii_msg[i].msg); + break; + case LKD_FW_ASCII_MSG_WRN: + dev_warn(hdev->dev, "fw: %s", fw_desc->ascii_msg[i].msg); + break; + case LKD_FW_ASCII_MSG_INF: + dev_info(hdev->dev, "fw: %s", fw_desc->ascii_msg[i].msg); + break; + default: + dev_dbg(hdev->dev, "fw: %s", fw_desc->ascii_msg[i].msg); + break; + } + } +} + /** * hl_fw_dynamic_read_and_validate_descriptor - read and validate FW descriptor * @@ -1988,9 +2055,10 @@ static int hl_fw_dynamic_read_and_validate_descriptor(struct hl_device *hdev, struct fw_load_mgr *fw_loader) { struct lkd_fw_comms_desc *fw_desc; - void __iomem *src, *temp_fw_desc; struct pci_mem_region *region; struct fw_response *response; + void *temp_fw_desc; + void __iomem *src; u16 fw_data_size; enum pci_region region_id; int rc; @@ -2039,6 +2107,10 @@ static int hl_fw_dynamic_read_and_validate_descriptor(struct hl_device *hdev, rc = hl_fw_dynamic_validate_descriptor(hdev, fw_loader, (struct lkd_fw_comms_desc *) temp_fw_desc); + + if (!rc) + hl_fw_dynamic_read_descriptor_msg(hdev, temp_fw_desc); + vfree(temp_fw_desc); return rc; @@ -2354,7 +2426,7 @@ static int hl_fw_dynamic_wait_for_boot_fit_active(struct hl_device *hdev, hdev->fw_poll_interval_usec, dyn_loader->wait_for_bl_timeout); if (rc) { - dev_err(hdev->dev, "failed to wait for boot\n"); + dev_err(hdev->dev, "failed to wait for boot (status = %d)\n", status); return rc; } @@ -2381,7 +2453,7 @@ static int hl_fw_dynamic_wait_for_linux_active(struct hl_device *hdev, hdev->fw_poll_interval_usec, fw_loader->cpu_timeout); if (rc) { - dev_err(hdev->dev, "failed to wait for Linux\n"); + dev_err(hdev->dev, "failed to wait for Linux (status = %d)\n", status); return rc; } @@ -2459,51 +2531,54 @@ static void hl_fw_linux_update_state(struct hl_device *hdev, static int hl_fw_dynamic_send_msg(struct hl_device *hdev, struct fw_load_mgr *fw_loader, u8 msg_type, void *data) { - struct lkd_msg_comms msg; + struct lkd_msg_comms *msg; int rc; - memset(&msg, 0, sizeof(msg)); + msg = kzalloc(sizeof(*msg), GFP_KERNEL); + if (!msg) + return -ENOMEM; /* create message to be sent */ - msg.header.type = msg_type; - msg.header.size = cpu_to_le16(sizeof(struct comms_msg_header)); - msg.header.magic = cpu_to_le32(HL_COMMS_MSG_MAGIC); + msg->header.type = msg_type; + msg->header.size = cpu_to_le16(sizeof(struct comms_msg_header)); + msg->header.magic = cpu_to_le32(HL_COMMS_MSG_MAGIC); switch (msg_type) { case HL_COMMS_RESET_CAUSE_TYPE: - msg.reset_cause = *(__u8 *) data; + msg->reset_cause = *(__u8 *) data; break; default: dev_err(hdev->dev, "Send COMMS message - invalid message type %u\n", msg_type); - return -EINVAL; + rc = -EINVAL; + goto out; } rc = hl_fw_dynamic_request_descriptor(hdev, fw_loader, sizeof(struct lkd_msg_comms)); if (rc) - return rc; + goto out; /* copy message to space allocated by FW */ - rc = hl_fw_dynamic_copy_msg(hdev, &msg, fw_loader); + rc = hl_fw_dynamic_copy_msg(hdev, msg, fw_loader); if (rc) - return rc; + goto out; rc = hl_fw_dynamic_send_protocol_cmd(hdev, fw_loader, COMMS_DATA_RDY, 0, true, fw_loader->cpu_timeout); if (rc) - return rc; + goto out; rc = hl_fw_dynamic_send_protocol_cmd(hdev, fw_loader, COMMS_EXEC, 0, true, fw_loader->cpu_timeout); - if (rc) - return rc; - return 0; +out: + kfree(msg); + return rc; } /** @@ -2560,13 +2635,43 @@ static int hl_fw_dynamic_init_cpu(struct hl_device *hdev, } if (!(hdev->fw_components & FW_TYPE_BOOT_CPU)) { + struct lkd_fw_binning_info *binning_info; + rc = hl_fw_dynamic_request_descriptor(hdev, fw_loader, 0); if (rc) goto protocol_err; /* read preboot version */ - return hl_fw_dynamic_read_device_fw_version(hdev, FW_COMP_PREBOOT, + rc = hl_fw_dynamic_read_device_fw_version(hdev, FW_COMP_PREBOOT, fw_loader->dynamic_loader.comm_desc.cur_fw_ver); + + if (rc) + return rc; + + /* read binning info from preboot */ + if (hdev->support_preboot_binning) { + binning_info = &fw_loader->dynamic_loader.comm_desc.binning_info; + hdev->tpc_binning = le64_to_cpu(binning_info->tpc_mask_l); + hdev->dram_binning = le32_to_cpu(binning_info->dram_mask); + hdev->edma_binning = le32_to_cpu(binning_info->edma_mask); + hdev->decoder_binning = le32_to_cpu(binning_info->dec_mask); + hdev->rotator_binning = le32_to_cpu(binning_info->rot_mask); + + rc = hdev->asic_funcs->set_dram_properties(hdev); + if (rc) + return rc; + + rc = hdev->asic_funcs->set_binning_masks(hdev); + if (rc) + return rc; + + dev_dbg(hdev->dev, + "Read binning masks: tpc: 0x%llx, dram: 0x%llx, edma: 0x%x, dec: 0x%x, rot:0x%x\n", + hdev->tpc_binning, hdev->dram_binning, hdev->edma_binning, + hdev->decoder_binning, hdev->rotator_binning); + } + + return 0; } /* load boot fit to FW */ @@ -2687,7 +2792,7 @@ static int hl_fw_static_init_cpu(struct hl_device *hdev, if (rc) { dev_dbg(hdev->dev, - "No boot fit request received, resuming boot\n"); + "No boot fit request received (status = %d), resuming boot\n", status); } else { rc = hdev->asic_funcs->load_boot_fit_to_device(hdev); if (rc) @@ -2710,7 +2815,7 @@ static int hl_fw_static_init_cpu(struct hl_device *hdev, if (rc) { dev_err(hdev->dev, - "Timeout waiting for boot fit load ack\n"); + "Timeout waiting for boot fit load ack (status = %d)\n", status); goto out; } @@ -2788,7 +2893,7 @@ static int hl_fw_static_init_cpu(struct hl_device *hdev, if (rc) { dev_err(hdev->dev, - "Failed to get ACK on skipping BMC, %d\n", + "Failed to get ACK on skipping BMC (status = %d)\n", status); WREG32(msg_to_cpu_reg, KMD_MSG_NA); rc = -EIO; @@ -2815,7 +2920,7 @@ static int hl_fw_static_init_cpu(struct hl_device *hdev, "Device reports FIT image is corrupted\n"); else dev_err(hdev->dev, - "Failed to load firmware to device, %d\n", + "Failed to load firmware to device (status = %d)\n", status); rc = -EIO; @@ -3043,3 +3148,27 @@ int hl_fw_get_sec_attest_info(struct hl_device *hdev, struct cpucp_sec_attest_in sizeof(struct cpucp_sec_attest_info), nonce, HL_CPUCP_SEC_ATTEST_INFO_TINEOUT_USEC); } + +int hl_fw_send_generic_request(struct hl_device *hdev, enum hl_passthrough_type sub_opcode, + dma_addr_t buff, u32 *size) +{ + struct cpucp_packet pkt = {0}; + u64 result; + int rc = 0; + + pkt.ctl = cpu_to_le32(CPUCP_PACKET_GENERIC_PASSTHROUGH << CPUCP_PKT_CTL_OPCODE_SHIFT); + pkt.addr = cpu_to_le64(buff); + pkt.data_max_size = cpu_to_le32(*size); + pkt.pkt_subidx = cpu_to_le32(sub_opcode); + + rc = hdev->asic_funcs->send_cpu_message(hdev, (u32 *)&pkt, sizeof(pkt), + HL_CPUCP_INFO_TIMEOUT_USEC, &result); + if (rc) + dev_err(hdev->dev, "failed to send CPUCP data of generic fw pkt\n"); + else + dev_dbg(hdev->dev, "generic pkt was successful, result: 0x%llx\n", result); + + *size = (u32)result; + + return rc; +} diff --git a/drivers/misc/habanalabs/common/habanalabs.h b/drivers/accel/habanalabs/common/habanalabs.h similarity index 97% rename from drivers/misc/habanalabs/common/habanalabs.h rename to drivers/accel/habanalabs/common/habanalabs.h index e2527d976ee052b4eb523900688ac398485a85b4..fa05e76d3d21aa64cf0ec940f6068018d84ea8a6 100644 --- a/drivers/misc/habanalabs/common/habanalabs.h +++ b/drivers/accel/habanalabs/common/habanalabs.h @@ -11,7 +11,7 @@ #include "../include/common/cpucp_if.h" #include "../include/common/qman_if.h" #include "../include/hw_ip/mmu/mmu_general.h" -#include +#include #include #include @@ -29,6 +29,8 @@ #include #include +#include "security.h" + #define HL_NAME "habanalabs" struct hl_device; @@ -375,7 +377,8 @@ enum hl_cs_type { CS_TYPE_COLLECTIVE_WAIT, CS_RESERVE_SIGNALS, CS_UNRESERVE_SIGNALS, - CS_TYPE_ENGINE_CORE + CS_TYPE_ENGINE_CORE, + CS_TYPE_FLUSH_PCI_HBW_WRITES, }; /* @@ -545,6 +548,8 @@ struct hl_hints_range { /** * struct asic_fixed_properties - ASIC specific immutable properties. * @hw_queues_props: H/W queues properties. + * @special_blocks: points to an array containing special blocks info. + * @skip_special_blocks_cfg: special blocks skip configs. * @cpucp_info: received various information from CPU-CP regarding the H/W, e.g. * available sensors. * @uboot_ver: F/W U-boot version. @@ -644,6 +649,10 @@ struct hl_hints_range { * (i.e. the DRAM supports multiple page sizes), otherwise * it will shall be equal to dram_page_size. * @num_engine_cores: number of engine cpu cores + * @num_of_special_blocks: special_blocks array size. + * @glbl_err_cause_num: global err cause number. + * @hbw_flush_reg: register to read to generate HBW flush. value of 0 means HBW flush is + * not supported. * @collective_first_sob: first sync object available for collective use * @collective_first_mon: first monitor available for collective use * @sync_stream_first_sob: first sync object available for sync stream use @@ -692,6 +701,8 @@ struct hl_hints_range { */ struct asic_fixed_properties { struct hw_queue_properties *hw_queues_props; + struct hl_special_block_info *special_blocks; + struct hl_skip_blocks_cfg skip_special_blocks_cfg; struct cpucp_info cpucp_info; char uboot_ver[VERSION_MAX_LEN]; char preboot_ver[VERSION_MAX_LEN]; @@ -764,6 +775,9 @@ struct asic_fixed_properties { u32 xbar_edge_enabled_mask; u32 device_mem_alloc_default_page_size; u32 num_engine_cores; + u32 num_of_special_blocks; + u32 glbl_err_cause_num; + u32 hbw_flush_reg; u16 collective_first_sob; u16 collective_first_mon; u16 sync_stream_first_sob; @@ -935,6 +949,7 @@ struct hl_mmap_mem_buf { * @size: holds the CB's size. * @roundup_size: holds the cb size after roundup to page size. * @cs_cnt: holds number of CS that this CB participates in. + * @is_handle_destroyed: atomic boolean indicating whether or not the CB handle was destroyed. * @is_pool: true if CB was acquired from the pool, false otherwise. * @is_internal: internally allocated * @is_mmu_mapped: true if the CB is mapped to the device's MMU. @@ -951,6 +966,7 @@ struct hl_cb { u32 size; u32 roundup_size; atomic_t cs_cnt; + atomic_t is_handle_destroyed; u8 is_pool; u8 is_internal; u8 is_mmu_mapped; @@ -1077,20 +1093,25 @@ struct hl_cq { atomic_t free_slots_cnt; }; +enum hl_user_interrupt_type { + HL_USR_INTERRUPT_CQ = 0, + HL_USR_INTERRUPT_DECODER, +}; + /** * struct hl_user_interrupt - holds user interrupt information * @hdev: pointer to the device structure + * @type: user interrupt type * @wait_list_head: head to the list of user threads pending on this interrupt * @wait_list_lock: protects wait_list_head * @interrupt_id: msix interrupt id - * @is_decoder: whether this entry represents a decoder interrupt */ struct hl_user_interrupt { - struct hl_device *hdev; - struct list_head wait_list_head; - spinlock_t wait_list_lock; - u32 interrupt_id; - bool is_decoder; + struct hl_device *hdev; + enum hl_user_interrupt_type type; + struct list_head wait_list_head; + spinlock_t wait_list_lock; + u32 interrupt_id; }; /** @@ -1540,8 +1561,10 @@ struct engines_data { * @check_if_razwi_happened: check if there was a razwi due to RR violation. * @access_dev_mem: access device memory * @set_dram_bar_base: set the base of the DRAM BAR - * @set_engine_cores: set a config command to enigne cores + * @set_engine_cores: set a config command to engine cores * @send_device_activity: indication to FW about device availability + * @set_dram_properties: set DRAM related properties. + * @set_binning_masks: set binning/enable masks for all relevant components. */ struct hl_asic_funcs { int (*early_init)(struct hl_device *hdev); @@ -1679,6 +1702,8 @@ struct hl_asic_funcs { int (*set_engine_cores)(struct hl_device *hdev, u32 *core_ids, u32 num_cores, u32 core_command); int (*send_device_activity)(struct hl_device *hdev, bool open); + int (*set_dram_properties)(struct hl_device *hdev); + int (*set_binning_masks)(struct hl_device *hdev); }; @@ -1739,8 +1764,9 @@ struct hl_cs_counters_atomic { * struct hl_dmabuf_priv - a dma-buf private object. * @dmabuf: pointer to dma-buf object. * @ctx: pointer to the dma-buf owner's context. - * @phys_pg_pack: pointer to physical page pack if the dma-buf was exported for - * memory allocation handle. + * @phys_pg_pack: pointer to physical page pack if the dma-buf was exported + * where virtual memory is supported. + * @memhash_hnode: pointer to the memhash node. this object holds the export count. * @device_address: physical address of the device's memory. Relevant only * if phys_pg_pack is NULL (dma-buf was exported from address). * The total size can be taken from the dmabuf object. @@ -1749,6 +1775,7 @@ struct hl_dmabuf_priv { struct dma_buf *dmabuf; struct hl_ctx *ctx; struct hl_vm_phys_pg_pack *phys_pg_pack; + struct hl_vm_hash_node *memhash_hnode; uint64_t device_address; }; @@ -1923,6 +1950,7 @@ struct hl_userptr { * @type: CS_TYPE_*. * @jobs_cnt: counter of submitted jobs on all queues. * @encaps_sig_hdl_id: encaps signals handle id, set for the first staged cs. + * @completion_timestamp: timestamp of the last completed cs job. * @sob_addr_offset: sob offset from the configuration base address. * @initial_sob_count: count of completed signals in SOB before current submission of signal or * cs with encaps signals. @@ -1955,6 +1983,7 @@ struct hl_cs { struct list_head staged_cs_node; struct list_head debugfs_list; struct hl_cs_encaps_sig_handle *encaps_sig_hdl; + ktime_t completion_timestamp; u64 sequence; u64 staged_sequence; u64 timeout_jiffies; @@ -1990,6 +2019,7 @@ struct hl_cs { * @debugfs_list: node in debugfs list of command submission jobs. * @refcount: reference counter for usage of the CS job. * @queue_type: the type of the H/W queue this job is submitted to. + * @timestamp: timestamp upon job completion * @id: the id of this job inside a CS. * @hw_queue_id: the id of the H/W queue this job is submitted to. * @user_cb_size: the actual size of the CB we got from the user. @@ -2016,6 +2046,7 @@ struct hl_cs_job { struct list_head debugfs_list; struct kref refcount; enum hl_queue_type queue_type; + ktime_t timestamp; u32 id; u32 hw_queue_id; u32 user_cb_size; @@ -2076,12 +2107,16 @@ struct hl_cs_parser { * hl_userptr). * @node: node to hang on the hash table in context object. * @vaddr: key virtual address. + * @handle: memory handle for device memory allocation. * @ptr: value pointer (hl_vm_phys_pg_list or hl_userptr). + * @export_cnt: number of exports from within the VA block. */ struct hl_vm_hash_node { struct hlist_node node; u64 vaddr; + u64 handle; void *ptr; + int export_cnt; }; /** @@ -2109,10 +2144,10 @@ struct hl_vm_hw_block_list_node { * @pages: the physical page array. * @npages: num physical pages in the pack. * @total_size: total size of all the pages in this list. + * @exported_size: buffer exported size. * @node: used to attach to deletion list that is used when all the allocations are cleared * at the teardown of the context. * @mapping_cnt: number of shared mappings. - * @exporting_cnt: number of dma-buf exporting. * @asid: the context related to this list. * @page_size: size of each page in the pack. * @flags: HL_MEM_* flags related to this list. @@ -2126,9 +2161,9 @@ struct hl_vm_phys_pg_pack { u64 *pages; u64 npages; u64 total_size; + u64 exported_size; struct list_head node; atomic_t mapping_cnt; - u32 exporting_cnt; u32 asid; u32 page_size; u32 flags; @@ -2675,11 +2710,11 @@ void hl_wreg(struct hl_device *hdev, u32 reg, u32 val); p->size = sz; \ }) -#define HL_USR_INTR_STRUCT_INIT(usr_intr, hdev, intr_id, decoder) \ +#define HL_USR_INTR_STRUCT_INIT(usr_intr, hdev, intr_id, intr_type) \ ({ \ usr_intr.hdev = hdev; \ usr_intr.interrupt_id = intr_id; \ - usr_intr.is_decoder = decoder; \ + usr_intr.type = intr_type; \ INIT_LIST_HEAD(&usr_intr.wait_list_head); \ spin_lock_init(&usr_intr.wait_list_lock); \ }) @@ -2961,37 +2996,53 @@ struct undefined_opcode_info { }; /** - * struct page_fault_info - info about page fault - * @pgf_info: page fault information. + * struct page_fault_info - page fault information. + * @page_fault: holds information collected during a page fault. * @user_mappings: buffer containing user mappings. * @num_of_user_mappings: number of user mappings. + * @page_fault_detected: if set as 1, then a page-fault was discovered for the + * first time after the driver has finished booting-up. + * Since we're looking for the page-fault's root cause, + * we don't care of the others that might follow it- + * so once changed to 1, it will remain that way. + * @page_fault_info_available: indicates that a page fault info is now available. */ struct page_fault_info { - struct hl_page_fault_info pgf; + struct hl_page_fault_info page_fault; struct hl_user_mapping *user_mappings; u64 num_of_user_mappings; + atomic_t page_fault_detected; + bool page_fault_info_available; +}; + +/** + * struct razwi_info - RAZWI information. + * @razwi: holds information collected during a RAZWI + * @razwi_detected: if set as 1, then a RAZWI was discovered for the + * first time after the driver has finished booting-up. + * Since we're looking for the RAZWI's root cause, + * we don't care of the others that might follow it- + * so once changed to 1, it will remain that way. + * @razwi_info_available: indicates that a RAZWI info is now available. + */ +struct razwi_info { + struct hl_info_razwi_event razwi; + atomic_t razwi_detected; + bool razwi_info_available; }; /** * struct hl_error_info - holds information collected during an error. * @cs_timeout: CS timeout error information. - * @razwi: razwi information. - * @razwi_info_recorded: if set writing to razwi information is enabled. - * otherwise - disabled, so the first (root cause) razwi will not be - * overwritten. - * @undef_opcode: undefined opcode information - * @pgf_info: page fault information. - * @pgf_info_recorded: if set writing to page fault information is enabled. - * otherwise - disabled, so the first (root cause) page fault will not be - * overwritten. + * @razwi_info: RAZWI information. + * @undef_opcode: undefined opcode information. + * @page_fault_info: page fault information. */ struct hl_error_info { struct cs_timeout_info cs_timeout; - struct hl_info_razwi_event razwi; - atomic_t razwi_info_recorded; + struct razwi_info razwi_info; struct undefined_opcode_info undef_opcode; - struct page_fault_info pgf_info; - atomic_t pgf_info_recorded; + struct page_fault_info page_fault_info; }; /** @@ -3157,6 +3208,8 @@ struct hl_reset_info { * @edma_binning: contains mask of edma engines that is received from the f/w which * indicates which edma engines are binned-out * @device_release_watchdog_timeout_sec: device release watchdog timeout value in seconds. + * @rotator_binning: contains mask of rotators engines that is received from the f/w + * which indicates which rotator engines are binned-out(Gaudi3 and above). * @id: device minor. * @id_control: minor of the control device. * @cdev_idx: char device index. Used for setting its name. @@ -3214,6 +3267,7 @@ struct hl_reset_info { * @heartbeat: Controls if we want to enable the heartbeat mechanism vs. the f/w, which verifies * that the f/w is always alive. Used only for testing. * @supports_ctx_switch: true if a ctx switch is required upon first submission. + * @support_preboot_binning: true if we support read binning info from preboot. */ struct hl_device { struct pci_dev *pdev; @@ -3322,6 +3376,7 @@ struct hl_device { u32 decoder_binning; u32 edma_binning; u32 device_release_watchdog_timeout_sec; + u32 rotator_binning; u16 id; u16 id_control; u16 cdev_idx; @@ -3355,6 +3410,7 @@ struct hl_device { u8 supports_mmu_prefetch; u8 reset_upon_device_release; u8 supports_ctx_switch; + u8 support_preboot_binning; /* Parameters for bring-up */ u64 nic_ports_mask; @@ -3729,6 +3785,7 @@ int hl_fw_cpucp_power_get(struct hl_device *hdev, u64 *power); void hl_fw_ask_hard_reset_without_linux(struct hl_device *hdev); void hl_fw_ask_halt_machine_without_linux(struct hl_device *hdev); int hl_fw_init_cpu(struct hl_device *hdev); +int hl_fw_wait_preboot_ready(struct hl_device *hdev); int hl_fw_read_preboot_status(struct hl_device *hdev); int hl_fw_dynamic_send_protocol_cmd(struct hl_device *hdev, struct fw_load_mgr *fw_loader, @@ -3772,6 +3829,8 @@ int hl_fw_get_clk_rate(struct hl_device *hdev, u32 *cur_clk, u32 *max_clk); void hl_fw_set_pll_profile(struct hl_device *hdev); void hl_sysfs_add_dev_clk_attr(struct hl_device *hdev, struct attribute_group *dev_clk_attr_grp); void hl_sysfs_add_dev_vrm_attr(struct hl_device *hdev, struct attribute_group *dev_vrm_attr_grp); +int hl_fw_send_generic_request(struct hl_device *hdev, enum hl_passthrough_type sub_opcode, + dma_addr_t buff, u32 *size); void hw_sob_get(struct hl_hw_sob *hw_sob); void hw_sob_put(struct hl_hw_sob *hw_sob); @@ -3786,6 +3845,7 @@ void hl_dec_fini(struct hl_device *hdev); void hl_dec_ctx_fini(struct hl_ctx *ctx); void hl_release_pending_user_interrupts(struct hl_device *hdev); +void hl_abort_waitings_for_completion(struct hl_device *hdev); int hl_cs_signal_sob_wraparound_handler(struct hl_device *hdev, u32 q_idx, struct hl_hw_sob **hw_sob, u32 count, bool encaps_sig); diff --git a/drivers/misc/habanalabs/common/habanalabs_drv.c b/drivers/accel/habanalabs/common/habanalabs_drv.c similarity index 98% rename from drivers/misc/habanalabs/common/habanalabs_drv.c rename to drivers/accel/habanalabs/common/habanalabs_drv.c index 7815c60df54e23bcf141bc6cf9d40b2675bf758c..03dae57dc8386306b3ea34b7ec8337f87c365cba 100644 --- a/drivers/misc/habanalabs/common/habanalabs_drv.c +++ b/drivers/accel/habanalabs/common/habanalabs_drv.c @@ -222,9 +222,11 @@ int hl_device_open(struct inode *inode, struct file *filp) hl_debugfs_add_file(hpriv); atomic_set(&hdev->captured_err_info.cs_timeout.write_enable, 1); - atomic_set(&hdev->captured_err_info.razwi_info_recorded, 0); - atomic_set(&hdev->captured_err_info.pgf_info_recorded, 0); + atomic_set(&hdev->captured_err_info.razwi_info.razwi_detected, 0); + atomic_set(&hdev->captured_err_info.page_fault_info.page_fault_detected, 0); hdev->captured_err_info.undef_opcode.write_enable = true; + hdev->captured_err_info.razwi_info.razwi_info_available = false; + hdev->captured_err_info.page_fault_info.page_fault_info_available = false; hdev->open_counter++; hdev->last_successful_open_jif = jiffies; diff --git a/drivers/misc/habanalabs/common/habanalabs_ioctl.c b/drivers/accel/habanalabs/common/habanalabs_ioctl.c similarity index 93% rename from drivers/misc/habanalabs/common/habanalabs_ioctl.c rename to drivers/accel/habanalabs/common/habanalabs_ioctl.c index b6abfa7761a70e296dfc243d3827f22fa7bcfbed..5005e6fca691234ded305a45c2255e4d3a465dd7 100644 --- a/drivers/misc/habanalabs/common/habanalabs_ioctl.c +++ b/drivers/accel/habanalabs/common/habanalabs_ioctl.c @@ -7,7 +7,7 @@ #define pr_fmt(fmt) "habanalabs: " fmt -#include +#include #include "habanalabs.h" #include @@ -607,16 +607,20 @@ static int cs_timeout_info(struct hl_fpriv *hpriv, struct hl_info_args *args) static int razwi_info(struct hl_fpriv *hpriv, struct hl_info_args *args) { + void __user *out = (void __user *) (uintptr_t) args->return_pointer; struct hl_device *hdev = hpriv->hdev; u32 max_size = args->return_size; - struct hl_info_razwi_event *info = &hdev->captured_err_info.razwi; - void __user *out = (void __user *) (uintptr_t) args->return_pointer; + struct razwi_info *razwi_info; if ((!max_size) || (!out)) return -EINVAL; - return copy_to_user(out, info, min_t(size_t, max_size, sizeof(struct hl_info_razwi_event))) - ? -EFAULT : 0; + razwi_info = &hdev->captured_err_info.razwi_info; + if (!razwi_info->razwi_info_available) + return 0; + + return copy_to_user(out, &razwi_info->razwi, + min_t(size_t, max_size, sizeof(struct hl_info_razwi_event))) ? -EFAULT : 0; } static int undefined_opcode_info(struct hl_fpriv *hpriv, struct hl_info_args *args) @@ -786,16 +790,20 @@ static int engine_status_info(struct hl_fpriv *hpriv, struct hl_info_args *args) static int page_fault_info(struct hl_fpriv *hpriv, struct hl_info_args *args) { + void __user *out = (void __user *) (uintptr_t) args->return_pointer; struct hl_device *hdev = hpriv->hdev; u32 max_size = args->return_size; - struct hl_page_fault_info *info = &hdev->captured_err_info.pgf_info.pgf; - void __user *out = (void __user *) (uintptr_t) args->return_pointer; + struct page_fault_info *pgf_info; if ((!max_size) || (!out)) return -EINVAL; - return copy_to_user(out, info, min_t(size_t, max_size, sizeof(struct hl_page_fault_info))) - ? -EFAULT : 0; + pgf_info = &hdev->captured_err_info.page_fault_info; + if (!pgf_info->page_fault_info_available) + return 0; + + return copy_to_user(out, &pgf_info->page_fault, + min_t(size_t, max_size, sizeof(struct hl_page_fault_info))) ? -EFAULT : 0; } static int user_mappings_info(struct hl_fpriv *hpriv, struct hl_info_args *args) @@ -806,18 +814,68 @@ static int user_mappings_info(struct hl_fpriv *hpriv, struct hl_info_args *args) struct page_fault_info *pgf_info; u64 actual_size; - pgf_info = &hdev->captured_err_info.pgf_info; - args->array_size = pgf_info->num_of_user_mappings; - if (!out) return -EINVAL; + pgf_info = &hdev->captured_err_info.page_fault_info; + if (!pgf_info->page_fault_info_available) + return 0; + + args->array_size = pgf_info->num_of_user_mappings; + actual_size = pgf_info->num_of_user_mappings * sizeof(struct hl_user_mapping); if (user_buf_size < actual_size) return -ENOMEM; - return copy_to_user(out, pgf_info->user_mappings, min_t(size_t, user_buf_size, actual_size)) - ? -EFAULT : 0; + return copy_to_user(out, pgf_info->user_mappings, actual_size) ? -EFAULT : 0; +} + +static int send_fw_generic_request(struct hl_device *hdev, struct hl_info_args *info_args) +{ + void __user *buff = (void __user *) (uintptr_t) info_args->return_pointer; + u32 size = info_args->return_size; + dma_addr_t dma_handle; + bool need_input_buff; + void *fw_buff; + int rc = 0; + + switch (info_args->fw_sub_opcode) { + case HL_PASSTHROUGH_VERSIONS: + need_input_buff = false; + break; + default: + return -EINVAL; + } + + if (size > SZ_1M) { + dev_err(hdev->dev, "buffer size cannot exceed 1MB\n"); + return -EINVAL; + } + + fw_buff = hl_cpu_accessible_dma_pool_alloc(hdev, size, &dma_handle); + if (!fw_buff) + return -ENOMEM; + + + if (need_input_buff && copy_from_user(fw_buff, buff, size)) { + dev_dbg(hdev->dev, "Failed to copy from user FW buff\n"); + rc = -EFAULT; + goto free_buff; + } + + rc = hl_fw_send_generic_request(hdev, info_args->fw_sub_opcode, dma_handle, &size); + if (rc) + goto free_buff; + + if (copy_to_user(buff, fw_buff, min(size, info_args->return_size))) { + dev_dbg(hdev->dev, "Failed to copy to user FW generic req output\n"); + rc = -EFAULT; + } + +free_buff: + hl_cpu_accessible_dma_pool_free(hdev, info_args->return_size, fw_buff); + + return rc; } static int _hl_info_ioctl(struct hl_fpriv *hpriv, void *data, @@ -826,9 +884,13 @@ static int _hl_info_ioctl(struct hl_fpriv *hpriv, void *data, enum hl_device_status status; struct hl_info_args *args = data; struct hl_device *hdev = hpriv->hdev; - int rc; + if (args->pad) { + dev_dbg(hdev->dev, "Padding bytes must be 0\n"); + return -EINVAL; + } + /* * Information is returned for the following opcodes even if the device * is disabled or in reset. @@ -893,7 +955,7 @@ static int _hl_info_ioctl(struct hl_fpriv *hpriv, void *data, } if (!hl_device_operational(hdev, &status)) { - dev_warn_ratelimited(dev, + dev_dbg_ratelimited(dev, "Device is %s. Can't execute INFO IOCTL\n", hdev->status[status]); return -EBUSY; @@ -947,6 +1009,9 @@ static int _hl_info_ioctl(struct hl_fpriv *hpriv, void *data, case HL_INFO_ENGINE_STATUS: return engine_status_info(hpriv, args); + case HL_INFO_FW_GENERIC_REQ: + return send_fw_generic_request(hdev, args); + default: dev_err(dev, "Invalid request %d\n", args->op); rc = -EINVAL; @@ -975,7 +1040,7 @@ static int hl_debug_ioctl(struct hl_fpriv *hpriv, void *data) int rc = 0; if (!hl_device_operational(hdev, &status)) { - dev_warn_ratelimited(hdev->dev, + dev_dbg_ratelimited(hdev->dev, "Device is %s. Can't execute DEBUG IOCTL\n", hdev->status[status]); return -EBUSY; @@ -1072,8 +1137,6 @@ static long _hl_ioctl(struct file *filep, unsigned int cmd, unsigned long arg, retcode = -EFAULT; goto out_err; } - } else if (cmd & IOC_OUT) { - memset(kdata, 0, usize); } retcode = func(hpriv, kdata); diff --git a/drivers/misc/habanalabs/common/hw_queue.c b/drivers/accel/habanalabs/common/hw_queue.c similarity index 100% rename from drivers/misc/habanalabs/common/hw_queue.c rename to drivers/accel/habanalabs/common/hw_queue.c diff --git a/drivers/misc/habanalabs/common/hwmon.c b/drivers/accel/habanalabs/common/hwmon.c similarity index 100% rename from drivers/misc/habanalabs/common/hwmon.c rename to drivers/accel/habanalabs/common/hwmon.c diff --git a/drivers/misc/habanalabs/common/irq.c b/drivers/accel/habanalabs/common/irq.c similarity index 94% rename from drivers/misc/habanalabs/common/irq.c rename to drivers/accel/habanalabs/common/irq.c index 94d537fd4fde524b3cbea52178d8bfda5cbd3a7c..04844e843a7b08a74aeb247c5833db4d0288ecfd 100644 --- a/drivers/misc/habanalabs/common/irq.c +++ b/drivers/accel/habanalabs/common/irq.c @@ -72,15 +72,17 @@ static void irq_handle_eqe(struct work_struct *work) * @hdev: pointer to device structure * @cs_seq: command submission sequence * @cq: completion queue + * @timestamp: interrupt timestamp * */ -static void job_finish(struct hl_device *hdev, u32 cs_seq, struct hl_cq *cq) +static void job_finish(struct hl_device *hdev, u32 cs_seq, struct hl_cq *cq, ktime_t timestamp) { struct hl_hw_queue *queue; struct hl_cs_job *job; queue = &hdev->kernel_queues[cq->hw_queue_id]; job = queue->shadow_queue[hl_pi_2_offset(cs_seq)]; + job->timestamp = timestamp; queue_work(hdev->cq_wq[cq->cq_idx], &job->finish_work); atomic_inc(&queue->ci); @@ -91,9 +93,10 @@ static void job_finish(struct hl_device *hdev, u32 cs_seq, struct hl_cq *cq) * * @hdev: pointer to device structure * @cs_seq: command submission sequence + * @timestamp: interrupt timestamp * */ -static void cs_finish(struct hl_device *hdev, u16 cs_seq) +static void cs_finish(struct hl_device *hdev, u16 cs_seq, ktime_t timestamp) { struct asic_fixed_properties *prop = &hdev->asic_prop; struct hl_hw_queue *queue; @@ -113,6 +116,7 @@ static void cs_finish(struct hl_device *hdev, u16 cs_seq) atomic_inc(&queue->ci); } + cs->completion_timestamp = timestamp; queue_work(hdev->cs_cmplt_wq, &cs->finish_work); } @@ -130,6 +134,7 @@ irqreturn_t hl_irq_handler_cq(int irq, void *arg) bool shadow_index_valid, entry_ready; u16 shadow_index; struct hl_cq_entry *cq_entry, *cq_base; + ktime_t timestamp = ktime_get(); if (hdev->disabled) { dev_dbg(hdev->dev, @@ -171,9 +176,9 @@ irqreturn_t hl_irq_handler_cq(int irq, void *arg) if (shadow_index_valid && !hdev->disabled) { if (hdev->asic_prop.completion_mode == HL_COMPLETION_MODE_CS) - cs_finish(hdev, shadow_index); + cs_finish(hdev, shadow_index, timestamp); else - job_finish(hdev, shadow_index, cq); + job_finish(hdev, shadow_index, cq, timestamp); } /* Clear CQ entry ready bit */ @@ -228,7 +233,7 @@ static void hl_ts_free_objects(struct work_struct *work) * list to a dedicated workqueue to do the actual put. */ static int handle_registration_node(struct hl_device *hdev, struct hl_user_pending_interrupt *pend, - struct list_head **free_list) + struct list_head **free_list, ktime_t now) { struct timestamp_reg_free_node *free_node; u64 timestamp; @@ -246,7 +251,7 @@ static int handle_registration_node(struct hl_device *hdev, struct hl_user_pendi if (!free_node) return -ENOMEM; - timestamp = ktime_get_ns(); + timestamp = ktime_to_ns(now); *pend->ts_reg_info.timestamp_kernel_addr = timestamp; @@ -298,7 +303,7 @@ static void handle_user_interrupt(struct hl_device *hdev, struct hl_user_interru if (pend->ts_reg_info.buf) { if (!reg_node_handle_fail) { rc = handle_registration_node(hdev, pend, - &ts_reg_free_list_head); + &ts_reg_free_list_head, now); if (rc) reg_node_handle_fail = true; } @@ -333,13 +338,22 @@ irqreturn_t hl_irq_handler_user_interrupt(int irq, void *arg) struct hl_user_interrupt *user_int = arg; struct hl_device *hdev = user_int->hdev; - if (user_int->is_decoder) - handle_user_interrupt(hdev, &hdev->common_decoder_interrupt); - else + switch (user_int->type) { + case HL_USR_INTERRUPT_CQ: handle_user_interrupt(hdev, &hdev->common_user_cq_interrupt); - /* Handle user cq or decoder interrupts registered on this specific irq */ - handle_user_interrupt(hdev, user_int); + /* Handle user cq interrupt registered on this specific irq */ + handle_user_interrupt(hdev, user_int); + break; + case HL_USR_INTERRUPT_DECODER: + handle_user_interrupt(hdev, &hdev->common_decoder_interrupt); + + /* Handle decoder interrupt registered on this specific irq */ + handle_user_interrupt(hdev, user_int); + break; + default: + break; + } return IRQ_HANDLED; } diff --git a/drivers/misc/habanalabs/common/memory.c b/drivers/accel/habanalabs/common/memory.c similarity index 92% rename from drivers/misc/habanalabs/common/memory.c rename to drivers/accel/habanalabs/common/memory.c index 5e9ae7600d75ee2865755f8f9c057f0c76547975..e6474d38afc49187f92e041f8fdff6c592e9e558 100644 --- a/drivers/misc/habanalabs/common/memory.c +++ b/drivers/accel/habanalabs/common/memory.c @@ -5,7 +5,7 @@ * All Rights Reserved. */ -#include +#include #include "habanalabs.h" #include "../include/hw_ip/mmu/mmu_general.h" @@ -19,7 +19,9 @@ MODULE_IMPORT_NS(DMA_BUF); #define HL_MMU_DEBUG 0 /* use small pages for supporting non-pow2 (32M/40M/48M) DRAM phys page sizes */ -#define DRAM_POOL_PAGE_SIZE SZ_8M +#define DRAM_POOL_PAGE_SIZE SZ_8M + +#define MEM_HANDLE_INVALID ULONG_MAX static int allocate_timestamps_buffers(struct hl_fpriv *hpriv, struct hl_mem_in *args, u64 *handle); @@ -371,12 +373,6 @@ static int free_device_memory(struct hl_ctx *ctx, struct hl_mem_in *args) return -EINVAL; } - if (phys_pg_pack->exporting_cnt) { - spin_unlock(&vm->idr_lock); - dev_dbg(hdev->dev, "handle %u is exported, cannot free\n", handle); - return -EINVAL; - } - /* must remove from idr before the freeing of the physical pages as the refcount of the pool * is also the trigger of the idr destroy */ @@ -1240,6 +1236,7 @@ static int map_device_va(struct hl_ctx *ctx, struct hl_mem_in *args, u64 *device hnode->ptr = vm_type; hnode->vaddr = ret_vaddr; + hnode->handle = is_userptr ? MEM_HANDLE_INVALID : handle; mutex_lock(&ctx->mem_hash_lock); hash_add(ctx->mem_hash, &hnode->node, ret_vaddr); @@ -1313,6 +1310,12 @@ static int unmap_device_va(struct hl_ctx *ctx, struct hl_mem_in *args, return -EINVAL; } + if (hnode->export_cnt) { + mutex_unlock(&ctx->mem_hash_lock); + dev_err(hdev->dev, "failed to unmap %#llx, memory is exported\n", vaddr); + return -EINVAL; + } + hash_del(&hnode->node); mutex_unlock(&ctx->mem_hash_lock); @@ -1545,10 +1548,10 @@ static int set_dma_sg(struct scatterlist *sg, u64 bar_address, u64 chunk_size, } static struct sg_table *alloc_sgt_from_device_pages(struct hl_device *hdev, u64 *pages, u64 npages, - u64 page_size, struct device *dev, - enum dma_data_direction dir) + u64 page_size, u64 exported_size, + struct device *dev, enum dma_data_direction dir) { - u64 chunk_size, bar_address, dma_max_seg_size; + u64 chunk_size, bar_address, dma_max_seg_size, cur_size_to_export, cur_npages; struct asic_fixed_properties *prop; int rc, i, j, nents, cur_page; struct scatterlist *sg; @@ -1574,16 +1577,23 @@ static struct sg_table *alloc_sgt_from_device_pages(struct hl_device *hdev, u64 if (!sgt) return ERR_PTR(-ENOMEM); + /* remove export size restrictions in case not explicitly defined */ + cur_size_to_export = exported_size ? exported_size : (npages * page_size); + /* If the size of each page is larger than the dma max segment size, * then we can't combine pages and the number of entries in the SGL * will just be the * * */ - if (page_size > dma_max_seg_size) - nents = npages * DIV_ROUND_UP_ULL(page_size, dma_max_seg_size); - else + if (page_size > dma_max_seg_size) { + /* we should limit number of pages according to the exported size */ + cur_npages = DIV_ROUND_UP_SECTOR_T(cur_size_to_export, page_size); + nents = cur_npages * DIV_ROUND_UP_SECTOR_T(page_size, dma_max_seg_size); + } else { + cur_npages = npages; + /* Get number of non-contiguous chunks */ - for (i = 1, nents = 1, chunk_size = page_size ; i < npages ; i++) { + for (i = 1, nents = 1, chunk_size = page_size ; i < cur_npages ; i++) { if (pages[i - 1] + page_size != pages[i] || chunk_size + page_size > dma_max_seg_size) { nents++; @@ -1593,6 +1603,7 @@ static struct sg_table *alloc_sgt_from_device_pages(struct hl_device *hdev, u64 chunk_size += page_size; } + } rc = sg_alloc_table(sgt, nents, GFP_KERNEL | __GFP_ZERO); if (rc) @@ -1615,7 +1626,8 @@ static struct sg_table *alloc_sgt_from_device_pages(struct hl_device *hdev, u64 else cur_device_address += dma_max_seg_size; - chunk_size = min(size_left, dma_max_seg_size); + /* make sure not to export over exported size */ + chunk_size = min3(size_left, dma_max_seg_size, cur_size_to_export); bar_address = hdev->dram_pci_bar_start + cur_device_address; @@ -1623,6 +1635,8 @@ static struct sg_table *alloc_sgt_from_device_pages(struct hl_device *hdev, u64 if (rc) goto error_unmap; + cur_size_to_export -= chunk_size; + if (size_left > dma_max_seg_size) { size_left -= dma_max_seg_size; } else { @@ -1634,7 +1648,7 @@ static struct sg_table *alloc_sgt_from_device_pages(struct hl_device *hdev, u64 /* Merge pages and put them into the scatterlist */ for_each_sgtable_dma_sg(sgt, sg, i) { chunk_size = page_size; - for (j = cur_page + 1 ; j < npages ; j++) { + for (j = cur_page + 1 ; j < cur_npages ; j++) { if (pages[j - 1] + page_size != pages[j] || chunk_size + page_size > dma_max_seg_size) break; @@ -1645,10 +1659,13 @@ static struct sg_table *alloc_sgt_from_device_pages(struct hl_device *hdev, u64 bar_address = hdev->dram_pci_bar_start + (pages[cur_page] - prop->dram_base_address); + /* make sure not to export over exported size */ + chunk_size = min(chunk_size, cur_size_to_export); rc = set_dma_sg(sg, bar_address, chunk_size, dev, dir); if (rc) goto error_unmap; + cur_size_to_export -= chunk_size; cur_page = j; } } @@ -1719,6 +1736,7 @@ static struct sg_table *hl_map_dmabuf(struct dma_buf_attachment *attachment, phys_pg_pack->pages, phys_pg_pack->npages, phys_pg_pack->page_size, + phys_pg_pack->exported_size, attachment->dev, dir); else @@ -1726,6 +1744,7 @@ static struct sg_table *hl_map_dmabuf(struct dma_buf_attachment *attachment, &hl_dmabuf->device_address, 1, hl_dmabuf->dmabuf->size, + 0, attachment->dev, dir); @@ -1763,18 +1782,20 @@ static void hl_unmap_dmabuf(struct dma_buf_attachment *attachment, static void hl_release_dmabuf(struct dma_buf *dmabuf) { struct hl_dmabuf_priv *hl_dmabuf = dmabuf->priv; - struct hl_ctx *ctx = hl_dmabuf->ctx; - struct hl_device *hdev = ctx->hdev; - struct hl_vm *vm = &hdev->vm; + struct hl_ctx *ctx; - if (hl_dmabuf->phys_pg_pack) { - spin_lock(&vm->idr_lock); - hl_dmabuf->phys_pg_pack->exporting_cnt--; - spin_unlock(&vm->idr_lock); - } + if (!hl_dmabuf) + return; - hl_ctx_put(hl_dmabuf->ctx); + ctx = hl_dmabuf->ctx; + if (hl_dmabuf->memhash_hnode) { + mutex_lock(&ctx->mem_hash_lock); + hl_dmabuf->memhash_hnode->export_cnt--; + mutex_unlock(&ctx->mem_hash_lock); + } + + hl_ctx_put(ctx); kfree(hl_dmabuf); } @@ -1785,7 +1806,7 @@ static const struct dma_buf_ops habanalabs_dmabuf_ops = { .release = hl_release_dmabuf, }; -static int export_dmabuf_common(struct hl_ctx *ctx, +static int export_dmabuf(struct hl_ctx *ctx, struct hl_dmabuf_priv *hl_dmabuf, u64 total_size, int flags, int *dmabuf_fd) { @@ -1806,7 +1827,7 @@ static int export_dmabuf_common(struct hl_ctx *ctx, fd = dma_buf_fd(hl_dmabuf->dmabuf, flags); if (fd < 0) { - dev_err(hdev->dev, "failed to get a file descriptor for a dma-buf\n"); + dev_err(hdev->dev, "failed to get a file descriptor for a dma-buf, %d\n", fd); rc = fd; goto err_dma_buf_put; } @@ -1819,36 +1840,13 @@ static int export_dmabuf_common(struct hl_ctx *ctx, return 0; err_dma_buf_put: + hl_dmabuf->dmabuf->priv = NULL; dma_buf_put(hl_dmabuf->dmabuf); return rc; } -/** - * export_dmabuf_from_addr() - export a dma-buf object for the given memory - * address and size. - * @ctx: pointer to the context structure. - * @device_addr: device memory physical address. - * @size: size of device memory. - * @flags: DMA-BUF file/FD flags. - * @dmabuf_fd: pointer to result FD that represents the dma-buf object. - * - * Create and export a dma-buf object for an existing memory allocation inside - * the device memory, and return a FD which is associated with the dma-buf - * object. - * - * Return: 0 on success, non-zero for failure. - */ -static int export_dmabuf_from_addr(struct hl_ctx *ctx, u64 device_addr, - u64 size, int flags, int *dmabuf_fd) +static int validate_export_params_common(struct hl_device *hdev, u64 device_addr, u64 size) { - struct hl_dmabuf_priv *hl_dmabuf; - struct hl_device *hdev = ctx->hdev; - struct asic_fixed_properties *prop; - u64 bar_address; - int rc; - - prop = &hdev->asic_prop; - if (!IS_ALIGNED(device_addr, PAGE_SIZE)) { dev_dbg(hdev->dev, "exported device memory address 0x%llx should be aligned to 0x%lx\n", @@ -1863,49 +1861,150 @@ static int export_dmabuf_from_addr(struct hl_ctx *ctx, u64 device_addr, return -EINVAL; } + return 0; +} + +static int validate_export_params_no_mmu(struct hl_device *hdev, u64 device_addr, u64 size) +{ + struct asic_fixed_properties *prop = &hdev->asic_prop; + u64 bar_address; + int rc; + + rc = validate_export_params_common(hdev, device_addr, size); + if (rc) + return rc; + if (device_addr < prop->dram_user_base_address || - device_addr + size > prop->dram_end_address || - device_addr + size < device_addr) { + (device_addr + size) > prop->dram_end_address || + (device_addr + size) < device_addr) { dev_dbg(hdev->dev, "DRAM memory range 0x%llx (+0x%llx) is outside of DRAM boundaries\n", device_addr, size); return -EINVAL; } - bar_address = hdev->dram_pci_bar_start + - (device_addr - prop->dram_base_address); + bar_address = hdev->dram_pci_bar_start + (device_addr - prop->dram_base_address); - if (bar_address + size > - hdev->dram_pci_bar_start + prop->dram_pci_bar_size || - bar_address + size < bar_address) { + if ((bar_address + size) > (hdev->dram_pci_bar_start + prop->dram_pci_bar_size) || + (bar_address + size) < bar_address) { dev_dbg(hdev->dev, "DRAM memory range 0x%llx (+0x%llx) is outside of PCI BAR boundaries\n", device_addr, size); return -EINVAL; } - hl_dmabuf = kzalloc(sizeof(*hl_dmabuf), GFP_KERNEL); - if (!hl_dmabuf) - return -ENOMEM; + return 0; +} - hl_dmabuf->device_address = device_addr; +static int validate_export_params(struct hl_device *hdev, u64 device_addr, u64 size, u64 offset, + struct hl_vm_phys_pg_pack *phys_pg_pack) +{ + struct asic_fixed_properties *prop = &hdev->asic_prop; + u64 bar_address; + int i, rc; - rc = export_dmabuf_common(ctx, hl_dmabuf, size, flags, dmabuf_fd); + rc = validate_export_params_common(hdev, device_addr, size); if (rc) - goto err_free_dmabuf_wrapper; + return rc; + + if ((offset + size) > phys_pg_pack->total_size) { + dev_dbg(hdev->dev, "offset %#llx and size %#llx exceed total map size %#llx\n", + offset, size, phys_pg_pack->total_size); + return -EINVAL; + } + + for (i = 0 ; i < phys_pg_pack->npages ; i++) { + + bar_address = hdev->dram_pci_bar_start + + (phys_pg_pack->pages[i] - prop->dram_base_address); + + if ((bar_address + phys_pg_pack->page_size) > + (hdev->dram_pci_bar_start + prop->dram_pci_bar_size) || + (bar_address + phys_pg_pack->page_size) < bar_address) { + dev_dbg(hdev->dev, + "DRAM memory range 0x%llx (+0x%x) is outside of PCI BAR boundaries\n", + phys_pg_pack->pages[i], + phys_pg_pack->page_size); + + return -EINVAL; + } + } return 0; +} -err_free_dmabuf_wrapper: - kfree(hl_dmabuf); - return rc; +static struct hl_vm_hash_node *memhash_node_export_get(struct hl_ctx *ctx, u64 addr) +{ + struct hl_device *hdev = ctx->hdev; + struct hl_vm_hash_node *hnode; + + /* get the memory handle */ + mutex_lock(&ctx->mem_hash_lock); + hash_for_each_possible(ctx->mem_hash, hnode, node, (unsigned long)addr) + if (addr == hnode->vaddr) + break; + + if (!hnode) { + mutex_unlock(&ctx->mem_hash_lock); + dev_dbg(hdev->dev, "map address %#llx not found\n", addr); + return ERR_PTR(-EINVAL); + } + + if (upper_32_bits(hnode->handle)) { + mutex_unlock(&ctx->mem_hash_lock); + dev_dbg(hdev->dev, "invalid handle %#llx for map address %#llx\n", + hnode->handle, addr); + return ERR_PTR(-EINVAL); + } + + /* + * node found, increase export count so this memory cannot be unmapped + * and the hash node cannot be deleted. + */ + hnode->export_cnt++; + mutex_unlock(&ctx->mem_hash_lock); + + return hnode; +} + +static void memhash_node_export_put(struct hl_ctx *ctx, struct hl_vm_hash_node *hnode) +{ + mutex_lock(&ctx->mem_hash_lock); + hnode->export_cnt--; + mutex_unlock(&ctx->mem_hash_lock); +} + +static struct hl_vm_phys_pg_pack *get_phys_pg_pack_from_hash_node(struct hl_device *hdev, + struct hl_vm_hash_node *hnode) +{ + struct hl_vm_phys_pg_pack *phys_pg_pack; + struct hl_vm *vm = &hdev->vm; + + spin_lock(&vm->idr_lock); + phys_pg_pack = idr_find(&vm->phys_pg_pack_handles, (u32) hnode->handle); + if (!phys_pg_pack) { + spin_unlock(&vm->idr_lock); + dev_dbg(hdev->dev, "no match for handle 0x%x\n", (u32) hnode->handle); + return ERR_PTR(-EINVAL); + } + + spin_unlock(&vm->idr_lock); + + if (phys_pg_pack->vm_type != VM_TYPE_PHYS_PACK) { + dev_dbg(hdev->dev, "handle 0x%llx does not represent DRAM memory\n", hnode->handle); + return ERR_PTR(-EINVAL); + } + + return phys_pg_pack; } /** - * export_dmabuf_from_handle() - export a dma-buf object for the given memory - * handle. + * export_dmabuf_from_addr() - export a dma-buf object for the given memory + * address and size. * @ctx: pointer to the context structure. - * @handle: device memory allocation handle. + * @addr: device address. + * @size: size of device memory to export. + * @offset: the offset into the buffer from which to start exporting * @flags: DMA-BUF file/FD flags. * @dmabuf_fd: pointer to result FD that represents the dma-buf object. * @@ -1915,87 +2014,69 @@ static int export_dmabuf_from_addr(struct hl_ctx *ctx, u64 device_addr, * * Return: 0 on success, non-zero for failure. */ -static int export_dmabuf_from_handle(struct hl_ctx *ctx, u64 handle, int flags, - int *dmabuf_fd) +static int export_dmabuf_from_addr(struct hl_ctx *ctx, u64 addr, u64 size, u64 offset, + int flags, int *dmabuf_fd) { - struct hl_vm_phys_pg_pack *phys_pg_pack; - struct hl_dmabuf_priv *hl_dmabuf; - struct hl_device *hdev = ctx->hdev; + struct hl_vm_phys_pg_pack *phys_pg_pack = NULL; + struct hl_vm_hash_node *hnode = NULL; struct asic_fixed_properties *prop; - struct hl_vm *vm = &hdev->vm; - u64 bar_address; - int rc, i; + struct hl_dmabuf_priv *hl_dmabuf; + struct hl_device *hdev; + u64 export_addr; + int rc; + hdev = ctx->hdev; prop = &hdev->asic_prop; - if (upper_32_bits(handle)) { - dev_dbg(hdev->dev, "no match for handle 0x%llx\n", handle); + /* offset must be 0 in devices without virtual memory support */ + if (!prop->dram_supports_virtual_memory && offset) { + dev_dbg(hdev->dev, "offset is not allowed in device without virtual memory\n"); return -EINVAL; } - spin_lock(&vm->idr_lock); + export_addr = addr + offset; - phys_pg_pack = idr_find(&vm->phys_pg_pack_handles, (u32) handle); - if (!phys_pg_pack) { - spin_unlock(&vm->idr_lock); - dev_dbg(hdev->dev, "no match for handle 0x%x\n", (u32) handle); - return -EINVAL; - } - - /* increment now to avoid freeing device memory while exporting */ - phys_pg_pack->exporting_cnt++; - - spin_unlock(&vm->idr_lock); - - if (phys_pg_pack->vm_type != VM_TYPE_PHYS_PACK) { - dev_dbg(hdev->dev, "handle 0x%llx does not represent DRAM memory\n", handle); - rc = -EINVAL; - goto err_dec_exporting_cnt; - } - - for (i = 0 ; i < phys_pg_pack->npages ; i++) { - - bar_address = hdev->dram_pci_bar_start + - (phys_pg_pack->pages[i] - - prop->dram_base_address); - - if (bar_address + phys_pg_pack->page_size > - hdev->dram_pci_bar_start + prop->dram_pci_bar_size || - bar_address + phys_pg_pack->page_size < bar_address) { - - dev_dbg(hdev->dev, - "DRAM memory range 0x%llx (+0x%x) is outside of PCI BAR boundaries\n", - phys_pg_pack->pages[i], - phys_pg_pack->page_size); + hl_dmabuf = kzalloc(sizeof(*hl_dmabuf), GFP_KERNEL); + if (!hl_dmabuf) + return -ENOMEM; - rc = -EINVAL; - goto err_dec_exporting_cnt; + if (prop->dram_supports_virtual_memory) { + hnode = memhash_node_export_get(ctx, addr); + if (IS_ERR(hnode)) { + rc = PTR_ERR(hnode); + goto err_free_dmabuf_wrapper; } - } + phys_pg_pack = get_phys_pg_pack_from_hash_node(hdev, hnode); + if (IS_ERR(phys_pg_pack)) { + rc = PTR_ERR(phys_pg_pack); + goto dec_memhash_export_cnt; + } + rc = validate_export_params(hdev, export_addr, size, offset, phys_pg_pack); + if (rc) + goto dec_memhash_export_cnt; - hl_dmabuf = kzalloc(sizeof(*hl_dmabuf), GFP_KERNEL); - if (!hl_dmabuf) { - rc = -ENOMEM; - goto err_dec_exporting_cnt; + phys_pg_pack->exported_size = size; + hl_dmabuf->phys_pg_pack = phys_pg_pack; + hl_dmabuf->memhash_hnode = hnode; + } else { + rc = validate_export_params_no_mmu(hdev, export_addr, size); + if (rc) + goto err_free_dmabuf_wrapper; } - hl_dmabuf->phys_pg_pack = phys_pg_pack; + hl_dmabuf->device_address = export_addr; - rc = export_dmabuf_common(ctx, hl_dmabuf, phys_pg_pack->total_size, - flags, dmabuf_fd); + rc = export_dmabuf(ctx, hl_dmabuf, size, flags, dmabuf_fd); if (rc) - goto err_free_dmabuf_wrapper; + goto dec_memhash_export_cnt; return 0; +dec_memhash_export_cnt: + if (prop->dram_supports_virtual_memory) + memhash_node_export_put(ctx, hnode); err_free_dmabuf_wrapper: kfree(hl_dmabuf); - -err_dec_exporting_cnt: - spin_lock(&vm->idr_lock); - phys_pg_pack->exporting_cnt--; - spin_unlock(&vm->idr_lock); - return rc; } @@ -2089,12 +2170,13 @@ static int hl_ts_mmap(struct hl_mmap_mem_buf *buf, struct vm_area_struct *vma, v static int hl_ts_alloc_buf(struct hl_mmap_mem_buf *buf, gfp_t gfp, void *args) { struct hl_ts_buff *ts_buff = NULL; - u32 size, num_elements; + u32 num_elements; + size_t size; void *p; num_elements = *(u32 *)args; - ts_buff = kzalloc(sizeof(*ts_buff), GFP_KERNEL); + ts_buff = kzalloc(sizeof(*ts_buff), gfp); if (!ts_buff) return -ENOMEM; @@ -2180,7 +2262,7 @@ int hl_mem_ioctl(struct hl_fpriv *hpriv, void *data) int rc, dmabuf_fd = -EBADF; if (!hl_device_operational(hdev, &status)) { - dev_warn_ratelimited(hdev->dev, + dev_dbg_ratelimited(hdev->dev, "Device is %s. Can't execute MEMORY IOCTL\n", hdev->status[status]); return -EBUSY; @@ -2269,17 +2351,12 @@ int hl_mem_ioctl(struct hl_fpriv *hpriv, void *data) break; case HL_MEM_OP_EXPORT_DMABUF_FD: - if (hdev->asic_prop.dram_supports_virtual_memory) - rc = export_dmabuf_from_handle(ctx, - args->in.export_dmabuf_fd.handle, - args->in.flags, - &dmabuf_fd); - else - rc = export_dmabuf_from_addr(ctx, - args->in.export_dmabuf_fd.handle, - args->in.export_dmabuf_fd.mem_size, - args->in.flags, - &dmabuf_fd); + rc = export_dmabuf_from_addr(ctx, + args->in.export_dmabuf_fd.addr, + args->in.export_dmabuf_fd.mem_size, + args->in.export_dmabuf_fd.offset, + args->in.flags, + &dmabuf_fd); memset(args, 0, sizeof(*args)); args->out.fd = dmabuf_fd; break; diff --git a/drivers/misc/habanalabs/common/memory_mgr.c b/drivers/accel/habanalabs/common/memory_mgr.c similarity index 99% rename from drivers/misc/habanalabs/common/memory_mgr.c rename to drivers/accel/habanalabs/common/memory_mgr.c index 1936d653699ed0e0df3d725b45ffc5de335b083d..0f2759e265477df8fc7e4500f1f7b590607e2c62 100644 --- a/drivers/misc/habanalabs/common/memory_mgr.c +++ b/drivers/accel/habanalabs/common/memory_mgr.c @@ -25,8 +25,7 @@ struct hl_mmap_mem_buf *hl_mmap_mem_buf_get(struct hl_mem_mgr *mmg, u64 handle) buf = idr_find(&mmg->handles, lower_32_bits(handle >> PAGE_SHIFT)); if (!buf) { spin_unlock(&mmg->lock); - dev_warn(mmg->dev, - "Buff get failed, no match to handle %#llx\n", handle); + dev_dbg(mmg->dev, "Buff get failed, no match to handle %#llx\n", handle); return NULL; } kref_get(&buf->refcount); diff --git a/drivers/misc/habanalabs/common/mmu/Makefile b/drivers/accel/habanalabs/common/mmu/Makefile similarity index 100% rename from drivers/misc/habanalabs/common/mmu/Makefile rename to drivers/accel/habanalabs/common/mmu/Makefile diff --git a/drivers/misc/habanalabs/common/mmu/mmu.c b/drivers/accel/habanalabs/common/mmu/mmu.c similarity index 99% rename from drivers/misc/habanalabs/common/mmu/mmu.c rename to drivers/accel/habanalabs/common/mmu/mmu.c index 2c1005f74cf440eb59175862439452a6e0292ad4..a42ae8bc61e8d0ade9e071cf78325f042bdce19e 100644 --- a/drivers/misc/habanalabs/common/mmu/mmu.c +++ b/drivers/accel/habanalabs/common/mmu/mmu.c @@ -781,7 +781,7 @@ static void mmu_dma_mem_free_from_chunk(struct gen_pool *pool, struct gen_pool_chunk *chunk, void *data) { - struct hl_device *hdev = (struct hl_device *)data; + struct hl_device *hdev = data; hl_asic_dma_free_coherent(hdev, (chunk->end_addr - chunk->start_addr) + 1, (void *)chunk->start_addr, chunk->phys_addr); diff --git a/drivers/misc/habanalabs/common/mmu/mmu_v1.c b/drivers/accel/habanalabs/common/mmu/mmu_v1.c similarity index 99% rename from drivers/misc/habanalabs/common/mmu/mmu_v1.c rename to drivers/accel/habanalabs/common/mmu/mmu_v1.c index 8a40de4a47617c46ce6deb6dceb7334bf425cb22..d925dc4dd0972553fedf0231b919851aa047721b 100644 --- a/drivers/misc/habanalabs/common/mmu/mmu_v1.c +++ b/drivers/accel/habanalabs/common/mmu/mmu_v1.c @@ -344,7 +344,6 @@ static void dram_default_mapping_fini(struct hl_ctx *ctx) } } - hop2_pte_addr = hop2_addr; hop2_pte_addr = hop2_addr; for (i = 0 ; i < num_of_hop3 ; i++) { clear_pte(ctx, hop2_pte_addr); diff --git a/drivers/misc/habanalabs/common/mmu/mmu_v2_hr.c b/drivers/accel/habanalabs/common/mmu/mmu_v2_hr.c similarity index 100% rename from drivers/misc/habanalabs/common/mmu/mmu_v2_hr.c rename to drivers/accel/habanalabs/common/mmu/mmu_v2_hr.c diff --git a/drivers/misc/habanalabs/common/pci/Makefile b/drivers/accel/habanalabs/common/pci/Makefile similarity index 100% rename from drivers/misc/habanalabs/common/pci/Makefile rename to drivers/accel/habanalabs/common/pci/Makefile diff --git a/drivers/misc/habanalabs/common/pci/pci.c b/drivers/accel/habanalabs/common/pci/pci.c similarity index 97% rename from drivers/misc/habanalabs/common/pci/pci.c rename to drivers/accel/habanalabs/common/pci/pci.c index 5fe3da5fba306acfd9b9a8679ae749690a67d07c..d1f4c695baf22d63a18f97cda3efb1e751025105 100644 --- a/drivers/misc/habanalabs/common/pci/pci.c +++ b/drivers/accel/habanalabs/common/pci/pci.c @@ -10,6 +10,8 @@ #include +#include + #define HL_PLDM_PCI_ELBI_TIMEOUT_MSEC (HL_PCI_ELBI_TIMEOUT_MSEC * 100) #define IATU_REGION_CTRL_REGION_EN_MASK BIT(31) @@ -120,6 +122,9 @@ int hl_pci_elbi_read(struct hl_device *hdev, u64 addr, u32 *data) if ((val & PCI_CONFIG_ELBI_STS_MASK) == PCI_CONFIG_ELBI_STS_DONE) { pci_read_config_dword(pdev, mmPCI_CONFIG_ELBI_DATA, data); + if (unlikely(trace_habanalabs_elbi_read_enabled())) + trace_habanalabs_elbi_read(hdev->dev, (u32) addr, val); + return 0; } @@ -179,8 +184,11 @@ static int hl_pci_elbi_write(struct hl_device *hdev, u64 addr, u32 data) usleep_range(300, 500); } - if ((val & PCI_CONFIG_ELBI_STS_MASK) == PCI_CONFIG_ELBI_STS_DONE) + if ((val & PCI_CONFIG_ELBI_STS_MASK) == PCI_CONFIG_ELBI_STS_DONE) { + if (unlikely(trace_habanalabs_elbi_write_enabled())) + trace_habanalabs_elbi_write(hdev->dev, (u32) addr, val); return 0; + } if (val & PCI_CONFIG_ELBI_STS_ERR) return -EIO; diff --git a/drivers/misc/habanalabs/common/security.c b/drivers/accel/habanalabs/common/security.c similarity index 76% rename from drivers/misc/habanalabs/common/security.c rename to drivers/accel/habanalabs/common/security.c index 6196c0487c8b0e35bc66a47c9b33ec3cbb7a95a2..5f03ade07ead59c85b2db4fee3e186dc47e0ec99 100644 --- a/drivers/misc/habanalabs/common/security.c +++ b/drivers/accel/habanalabs/common/security.c @@ -7,6 +7,19 @@ #include "habanalabs.h" +static const char * const hl_glbl_error_cause[HL_MAX_NUM_OF_GLBL_ERR_CAUSE] = { + "Error due to un-priv read", + "Error due to un-secure read", + "Error due to read from unmapped reg", + "Error due to un-priv write", + "Error due to un-secure write", + "Error due to write to unmapped reg", + "External I/F write sec violation", + "External I/F write to un-mapped reg", + "Read to write only", + "Write to read only" +}; + /** * hl_get_pb_block - return the relevant block within the block array * @@ -598,3 +611,164 @@ void hl_ack_pb_single_dcore(struct hl_device *hdev, u32 dcore_offset, blocks_array_size); } + +static u32 hl_automated_get_block_base_addr(struct hl_device *hdev, + struct hl_special_block_info *block_info, + u32 major, u32 minor, u32 sub_minor) +{ + u32 fw_block_base_address = block_info->base_addr + + major * block_info->major_offset + + minor * block_info->minor_offset + + sub_minor * block_info->sub_minor_offset; + struct asic_fixed_properties *prop = &hdev->asic_prop; + + /* Calculation above returns an address for FW use, and therefore should + * be casted for driver use. + */ + return (fw_block_base_address - lower_32_bits(prop->cfg_base_address)); +} + +static bool hl_check_block_type_exclusion(struct hl_skip_blocks_cfg *skip_blocks_cfg, + int block_type) +{ + int i; + + /* Check if block type is listed in the exclusion list of block types */ + for (i = 0 ; i < skip_blocks_cfg->block_types_len ; i++) + if (block_type == skip_blocks_cfg->block_types[i]) + return true; + + return false; +} + +static bool hl_check_block_range_exclusion(struct hl_device *hdev, + struct hl_skip_blocks_cfg *skip_blocks_cfg, + struct hl_special_block_info *block_info, + u32 major, u32 minor, u32 sub_minor) +{ + u32 blocks_in_range, block_base_addr_in_range, block_base_addr; + int i, j; + + block_base_addr = hl_automated_get_block_base_addr(hdev, block_info, + major, minor, sub_minor); + + for (i = 0 ; i < skip_blocks_cfg->block_ranges_len ; i++) { + blocks_in_range = (skip_blocks_cfg->block_ranges[i].end - + skip_blocks_cfg->block_ranges[i].start) / + HL_BLOCK_SIZE + 1; + for (j = 0 ; j < blocks_in_range ; j++) { + block_base_addr_in_range = skip_blocks_cfg->block_ranges[i].start + + j * HL_BLOCK_SIZE; + if (block_base_addr == block_base_addr_in_range) + return true; + } + } + + return false; +} + +static int hl_read_glbl_errors(struct hl_device *hdev, + u32 blk_idx, u32 major, u32 minor, u32 sub_minor, void *data) +{ + struct hl_special_block_info *special_blocks = hdev->asic_prop.special_blocks; + struct hl_special_block_info *current_block = &special_blocks[blk_idx]; + u32 glbl_err_addr, glbl_err_cause, addr_val, cause_val, block_base, + base = current_block->base_addr - lower_32_bits(hdev->asic_prop.cfg_base_address); + int i; + + block_base = base + major * current_block->major_offset + + minor * current_block->minor_offset + + sub_minor * current_block->sub_minor_offset; + + glbl_err_cause = block_base + HL_GLBL_ERR_CAUSE_OFFSET; + cause_val = RREG32(glbl_err_cause); + if (!cause_val) + return 0; + + glbl_err_addr = block_base + HL_GLBL_ERR_ADDR_OFFSET; + addr_val = RREG32(glbl_err_addr); + + for (i = 0 ; i < hdev->asic_prop.glbl_err_cause_num ; i++) { + if (cause_val & BIT(i)) + dev_err_ratelimited(hdev->dev, + "%s, addr %#llx\n", + hl_glbl_error_cause[i], + hdev->asic_prop.cfg_base_address + block_base + + FIELD_GET(HL_GLBL_ERR_ADDRESS_MASK, addr_val)); + } + + WREG32(glbl_err_cause, cause_val); + + return 0; +} + +void hl_check_for_glbl_errors(struct hl_device *hdev) +{ + struct asic_fixed_properties *prop = &hdev->asic_prop; + struct hl_special_blocks_cfg special_blocks_cfg; + struct iterate_special_ctx glbl_err_iter; + int rc; + + memset(&special_blocks_cfg, 0, sizeof(special_blocks_cfg)); + special_blocks_cfg.skip_blocks_cfg = &prop->skip_special_blocks_cfg; + + glbl_err_iter.fn = &hl_read_glbl_errors; + glbl_err_iter.data = &special_blocks_cfg; + + rc = hl_iterate_special_blocks(hdev, &glbl_err_iter); + if (rc) + dev_err_ratelimited(hdev->dev, + "Could not iterate special blocks, glbl error check failed\n"); +} + +int hl_iterate_special_blocks(struct hl_device *hdev, struct iterate_special_ctx *ctx) +{ + struct hl_special_blocks_cfg *special_blocks_cfg = + (struct hl_special_blocks_cfg *)ctx->data; + struct hl_skip_blocks_cfg *skip_blocks_cfg = + special_blocks_cfg->skip_blocks_cfg; + u32 major, minor, sub_minor, blk_idx, num_blocks; + struct hl_special_block_info *block_info_arr; + int rc; + + block_info_arr = hdev->asic_prop.special_blocks; + if (!block_info_arr) + return -EINVAL; + + num_blocks = hdev->asic_prop.num_of_special_blocks; + + for (blk_idx = 0 ; blk_idx < num_blocks ; blk_idx++, block_info_arr++) { + if (hl_check_block_type_exclusion(skip_blocks_cfg, block_info_arr->block_type)) + continue; + + for (major = 0 ; major < block_info_arr->major ; major++) { + minor = 0; + do { + sub_minor = 0; + do { + if ((hl_check_block_range_exclusion(hdev, + skip_blocks_cfg, block_info_arr, + major, minor, sub_minor)) || + (skip_blocks_cfg->skip_block_hook && + skip_blocks_cfg->skip_block_hook(hdev, + special_blocks_cfg, + blk_idx, major, minor, sub_minor))) { + sub_minor++; + continue; + } + + rc = ctx->fn(hdev, blk_idx, major, minor, + sub_minor, ctx->data); + if (rc) + return rc; + + sub_minor++; + } while (sub_minor < block_info_arr->sub_minor); + + minor++; + } while (minor < block_info_arr->minor); + } + } + + return 0; +} diff --git a/drivers/accel/habanalabs/common/security.h b/drivers/accel/habanalabs/common/security.h new file mode 100644 index 0000000000000000000000000000000000000000..234b4a6ed8bc1c86003802ba1579bb25a08720a4 --- /dev/null +++ b/drivers/accel/habanalabs/common/security.h @@ -0,0 +1,163 @@ +/* SPDX-License-Identifier: GPL-2.0 + * + * Copyright 2016-2022 HabanaLabs, Ltd. + * All Rights Reserved. + * + */ + +#ifndef SECURITY_H_ +#define SECURITY_H_ + +#include + +extern struct hl_device *hdev; + +/* special blocks */ +#define HL_MAX_NUM_OF_GLBL_ERR_CAUSE 10 +#define HL_GLBL_ERR_ADDRESS_MASK GENMASK(11, 0) +/* GLBL_ERR_ADDR register offset from the start of the block */ +#define HL_GLBL_ERR_ADDR_OFFSET 0xF44 +/* GLBL_ERR_CAUSE register offset from the start of the block */ +#define HL_GLBL_ERR_CAUSE_OFFSET 0xF48 + +/* + * struct hl_special_block_info - stores address details of a particular type of + * IP block which has a SPECIAL part. + * + * @block_type: block type as described in every ASIC's block_types enum. + * @base_addr: base address of the first block of particular type, + * e.g., address of NIC0_UMR0_0 of 'NIC_UMR' block. + * @major: number of major blocks of particular type. + * @minor: number of minor blocks of particular type. + * @sub_minor: number of sub minor blocks of particular type. + * @major_offset: address gap between 2 consecutive major blocks of particular type, + * e.g., offset between NIC0_UMR0_0 and NIC1_UMR0_0 is 0x80000. + * @minor_offset: address gap between 2 consecutive minor blocks of particular type, + * e.g., offset between NIC0_UMR0_0 and NIC0_UMR1_0 is 0x20000. + * @sub_minor_offset: address gap between 2 consecutive sub_minor blocks of particular + * type, e.g., offset between NIC0_UMR0_0 and NIC0_UMR0_1 is 0x1000. + * + * e.g., in Gaudi2, NIC_UMR blocks can be interpreted as: + * NIC_UMR_ where major=12, minor=2, sub_minor=15. + * In other words, for each of 12 major numbers (i.e 0 to 11) there are + * 2 blocks with different minor numbers (i.e. 0 to 1). Again, for each minor + * number there are 15 blocks with different sub_minor numbers (i.e. 0 to 14). + * So different blocks are NIC0_UMR0_0, NIC0_UMR0_1, ..., NIC0_UMR1_0, ...., + * NIC11_UMR1_14. + * + * Struct's formatted data is located in the SOL-based auto-generated protbits headers. + */ +struct hl_special_block_info { + int block_type; + u32 base_addr; + u32 major; + u32 minor; + u32 sub_minor; + u32 major_offset; + u32 minor_offset; + u32 sub_minor_offset; +}; + +/* + * struct hl_automated_pb_cfg - represents configurations of a particular type + * of IP block which has protection bits. + * + * @addr: address details as described in hl_automation_pb_addr struct. + * @prot_map: each bit corresponds to one among 32 protection configuration regs + * (e.g., SPECIAL_GLBL_PRIV). '1' means 0xffffffff and '0' means 0x0 + * to be written into the corresponding protection configuration reg. + * This bit is meaningful if same bit in data_map is 0, otherwise ignored. + * @data_map: each bit corresponds to one among 32 protection configuration regs + * (e.g., SPECIAL_GLBL_PRIV). '1' means corresponding protection + * configuration reg is to be written with a value in array pointed + * by 'data', otherwise the value is decided by 'prot_map'. + * @data: pointer to data array which stores the config value(s) to be written + * to corresponding protection configuration reg(s). + * @data_size: size of the data array. + * + * Each bit of 'data_map' and 'prot_map' fields corresponds to one among 32 + * protection configuration registers e.g., SPECIAL GLBL PRIV regs (starting at + * offset 0xE80). '1' in 'data_map' means protection configuration to be done + * using configuration in data array. '0' in 'data_map" means protection + * configuration to be done as per the value of corresponding bit in 'prot_map'. + * '1' in 'prot_map' means the register to be programmed with 0xFFFFFFFF + * (all non-protected). '0' in 'prot_map' means the register to be programmed + * with 0x0 (all protected). + * + * e.g., prot_map = 0x00000001, data_map = 0xC0000000 , data = {0xff, 0x12} + * SPECIAL_GLBL_PRIV[0] = 0xFFFFFFFF + * SPECIAL_GLBL_PRIV[1..29] = 0x0 + * SPECIAL_GLBL_PRIV[30] = 0xFF + * SPECIAL_GLBL_PRIV[31] = 0x12 + */ +struct hl_automated_pb_cfg { + struct hl_special_block_info addr; + u32 prot_map; + u32 data_map; + const u32 *data; + u8 data_size; +}; + +/* struct hl_special_blocks_cfg - holds special blocks cfg data. + * + * @priv_automated_pb_cfg: points to the main privileged PB array. + * @sec_automated_pb_cfg: points to the main secured PB array. + * @skip_blocks_cfg: holds arrays of block types & block ranges to be excluded. + * @priv_cfg_size: size of the main privileged PB array. + * @sec_cfg_size: size of the main secured PB array. + * @prot_lvl_priv: indication if it's a privileged/secured PB configurations. + */ +struct hl_special_blocks_cfg { + struct hl_automated_pb_cfg *priv_automated_pb_cfg; + struct hl_automated_pb_cfg *sec_automated_pb_cfg; + struct hl_skip_blocks_cfg *skip_blocks_cfg; + u32 priv_cfg_size; + u32 sec_cfg_size; + u8 prot_lvl_priv; +}; + +/* Automated security */ + +/* struct hl_skip_blocks_cfg - holds arrays of block types & block ranges to be + * excluded from special blocks configurations. + * + * @block_types: an array of block types NOT to be configured. + * @block_types_len: len of an array of block types not to be configured. + * @block_ranges: an array of block ranges not to be configured. + * @block_ranges_len: len of an array of block ranges not to be configured. + * @skip_block_hook: hook that will be called before initializing special blocks. + */ +struct hl_skip_blocks_cfg { + int *block_types; + size_t block_types_len; + struct range *block_ranges; + size_t block_ranges_len; + bool (*skip_block_hook)(struct hl_device *hdev, + struct hl_special_blocks_cfg *special_blocks_cfg, + u32 blk_idx, u32 major, u32 minor, u32 sub_minor); +}; + +/** + * struct iterate_special_ctx - HW module special block iterator + * @fn: function to apply to each HW module special block instance + * @data: optional internal data to the function iterator + */ +struct iterate_special_ctx { + /* + * callback for the HW module special block iterator + * @hdev: pointer to the habanalabs device structure + * @block_id: block (ASIC specific definition can be dcore/hdcore) + * @major: major block index within block_id + * @minor: minor block index within the major block + * @sub_minor: sub_minor block index within the minor block + * @data: function specific data + */ + int (*fn)(struct hl_device *hdev, u32 block_id, u32 major, u32 minor, + u32 sub_minor, void *data); + void *data; +}; + +int hl_iterate_special_blocks(struct hl_device *hdev, struct iterate_special_ctx *ctx); +void hl_check_for_glbl_errors(struct hl_device *hdev); + +#endif /* SECURITY_H_ */ diff --git a/drivers/misc/habanalabs/common/state_dump.c b/drivers/accel/habanalabs/common/state_dump.c similarity index 99% rename from drivers/misc/habanalabs/common/state_dump.c rename to drivers/accel/habanalabs/common/state_dump.c index 74726907c95edf6ffae92af5578f7eb2bf1d4672..3a9931f24259928ad5748fe4a9d9f3d2b78ca1b0 100644 --- a/drivers/misc/habanalabs/common/state_dump.c +++ b/drivers/accel/habanalabs/common/state_dump.c @@ -6,7 +6,7 @@ */ #include -#include +#include #include "habanalabs.h" /** diff --git a/drivers/misc/habanalabs/common/sysfs.c b/drivers/accel/habanalabs/common/sysfs.c similarity index 100% rename from drivers/misc/habanalabs/common/sysfs.c rename to drivers/accel/habanalabs/common/sysfs.c diff --git a/drivers/misc/habanalabs/gaudi/Makefile b/drivers/accel/habanalabs/gaudi/Makefile similarity index 100% rename from drivers/misc/habanalabs/gaudi/Makefile rename to drivers/accel/habanalabs/gaudi/Makefile diff --git a/drivers/misc/habanalabs/gaudi/gaudi.c b/drivers/accel/habanalabs/gaudi/gaudi.c similarity index 99% rename from drivers/misc/habanalabs/gaudi/gaudi.c rename to drivers/accel/habanalabs/gaudi/gaudi.c index 9f5e208701bad1186088113fb4129c7da4ccd821..71debe862c865fd1e4440036db32a63933ba8583 100644 --- a/drivers/misc/habanalabs/gaudi/gaudi.c +++ b/drivers/accel/habanalabs/gaudi/gaudi.c @@ -701,6 +701,8 @@ static int gaudi_set_fixed_properties(struct hl_device *hdev) prop->dma_mask = 48; + prop->hbw_flush_reg = mmPCIE_WRAP_RR_ELBI_RD_SEC_REG_CTRL; + return 0; } @@ -6432,12 +6434,6 @@ static int gaudi_send_job_on_qman0(struct hl_device *hdev, else timeout = HL_DEVICE_TIMEOUT_USEC; - if (!hdev->asic_funcs->is_device_idle(hdev, NULL, 0, NULL)) { - dev_err_ratelimited(hdev->dev, - "Can't send driver job on QMAN0 because the device is not idle\n"); - return -EBUSY; - } - fence_ptr = hl_asic_dma_pool_zalloc(hdev, 4, GFP_KERNEL, &fence_dma_addr); if (!fence_ptr) { dev_err(hdev->dev, @@ -7584,7 +7580,7 @@ static int tpc_krn_event_to_tpc_id(u16 tpc_dec_event_type) return (tpc_dec_event_type - GAUDI_EVENT_TPC0_KRN_ERR) / 6; } -static void gaudi_print_clk_change_info(struct hl_device *hdev, u16 event_type) +static void gaudi_print_clk_change_info(struct hl_device *hdev, u16 event_type, u64 *event_mask) { ktime_t zero_time = ktime_set(0, 0); @@ -7612,6 +7608,7 @@ static void gaudi_print_clk_change_info(struct hl_device *hdev, u16 event_type) hdev->clk_throttling.aggregated_reason |= HL_CLK_THROTTLE_THERMAL; hdev->clk_throttling.timestamp[HL_CLK_THROTTLE_TYPE_THERMAL].start = ktime_get(); hdev->clk_throttling.timestamp[HL_CLK_THROTTLE_TYPE_THERMAL].end = zero_time; + *event_mask |= HL_NOTIFIER_EVENT_USER_ENGINE_ERR; dev_info_ratelimited(hdev->dev, "Clock throttling due to overheating\n"); break; @@ -7619,6 +7616,7 @@ static void gaudi_print_clk_change_info(struct hl_device *hdev, u16 event_type) case GAUDI_EVENT_FIX_THERMAL_ENV_E: hdev->clk_throttling.current_reason &= ~HL_CLK_THROTTLE_THERMAL; hdev->clk_throttling.timestamp[HL_CLK_THROTTLE_TYPE_THERMAL].end = ktime_get(); + *event_mask |= HL_NOTIFIER_EVENT_USER_ENGINE_ERR; dev_info_ratelimited(hdev->dev, "Thermal envelop is safe, back to optimal clock\n"); break; @@ -7887,8 +7885,7 @@ static void gaudi_handle_eqe(struct hl_device *hdev, struct hl_eq_entry *eq_entr break; case GAUDI_EVENT_FIX_POWER_ENV_S ... GAUDI_EVENT_FIX_THERMAL_ENV_E: - event_mask |= HL_NOTIFIER_EVENT_USER_ENGINE_ERR; - gaudi_print_clk_change_info(hdev, event_type); + gaudi_print_clk_change_info(hdev, event_type, &event_mask); hl_fw_unmask_irq(hdev, event_type); break; @@ -9133,6 +9130,16 @@ static u32 *gaudi_get_stream_master_qid_arr(void) return gaudi_stream_master; } +static int gaudi_set_dram_properties(struct hl_device *hdev) +{ + return 0; +} + +static int gaudi_set_binning_masks(struct hl_device *hdev) +{ + return 0; +} + static void gaudi_check_if_razwi_happened(struct hl_device *hdev) { } @@ -9259,6 +9266,8 @@ static const struct hl_asic_funcs gaudi_funcs = { .access_dev_mem = hl_access_dev_mem, .set_dram_bar_base = gaudi_set_hbm_bar_base, .send_device_activity = gaudi_send_device_activity, + .set_dram_properties = gaudi_set_dram_properties, + .set_binning_masks = gaudi_set_binning_masks, }; /** diff --git a/drivers/misc/habanalabs/gaudi/gaudiP.h b/drivers/accel/habanalabs/gaudi/gaudiP.h similarity index 99% rename from drivers/misc/habanalabs/gaudi/gaudiP.h rename to drivers/accel/habanalabs/gaudi/gaudiP.h index 4fbcf3f0afe58952b3698c3845f1db7c440ab465..3d88d56c8eb3a34428345d7d936545c70db5c3bb 100644 --- a/drivers/misc/habanalabs/gaudi/gaudiP.h +++ b/drivers/accel/habanalabs/gaudi/gaudiP.h @@ -8,7 +8,7 @@ #ifndef GAUDIP_H_ #define GAUDIP_H_ -#include +#include #include "../common/habanalabs.h" #include "../include/common/hl_boot_if.h" #include "../include/gaudi/gaudi_packets.h" diff --git a/drivers/misc/habanalabs/gaudi/gaudi_coresight.c b/drivers/accel/habanalabs/gaudi/gaudi_coresight.c similarity index 99% rename from drivers/misc/habanalabs/gaudi/gaudi_coresight.c rename to drivers/accel/habanalabs/gaudi/gaudi_coresight.c index 08108f5fed674872846132ace7f4f3fd5940be03..3455b14554c677e2f8396a328081ef3facb8db0b 100644 --- a/drivers/misc/habanalabs/gaudi/gaudi_coresight.c +++ b/drivers/accel/habanalabs/gaudi/gaudi_coresight.c @@ -11,7 +11,8 @@ #include "../include/gaudi/gaudi_masks.h" #include "../include/gaudi/gaudi_reg_map.h" -#include +#include + #define SPMU_SECTION_SIZE MME0_ACC_SPMU_MAX_OFFSET #define SPMU_EVENT_TYPES_OFFSET 0x400 #define SPMU_MAX_COUNTERS 6 diff --git a/drivers/misc/habanalabs/gaudi/gaudi_security.c b/drivers/accel/habanalabs/gaudi/gaudi_security.c similarity index 100% rename from drivers/misc/habanalabs/gaudi/gaudi_security.c rename to drivers/accel/habanalabs/gaudi/gaudi_security.c diff --git a/drivers/misc/habanalabs/gaudi2/Makefile b/drivers/accel/habanalabs/gaudi2/Makefile similarity index 100% rename from drivers/misc/habanalabs/gaudi2/Makefile rename to drivers/accel/habanalabs/gaudi2/Makefile diff --git a/drivers/misc/habanalabs/gaudi2/gaudi2.c b/drivers/accel/habanalabs/gaudi2/gaudi2.c similarity index 92% rename from drivers/misc/habanalabs/gaudi2/gaudi2.c rename to drivers/accel/habanalabs/gaudi2/gaudi2.c index e793fb2bdcbe6c9d55765420c3e9dfca3e6ed225..f1f2a58ee68c2aaae77b77fc956172fbef501e93 100644 --- a/drivers/misc/habanalabs/gaudi2/gaudi2.c +++ b/drivers/accel/habanalabs/gaudi2/gaudi2.c @@ -7,6 +7,7 @@ #include "gaudi2P.h" #include "gaudi2_masks.h" +#include "../include/gaudi2/gaudi2_special_blocks.h" #include "../include/hw_ip/mmu/mmu_general.h" #include "../include/hw_ip/mmu/mmu_v2_0.h" #include "../include/gaudi2/gaudi2_packets.h" @@ -53,6 +54,7 @@ #define GAUDI2_HIF_HMMU_FULL_MASK 0xFFFF #define GAUDI2_DECODER_FULL_MASK 0x3FF +#define GAUDI2_NA_EVENT_CAUSE 0xFF #define GAUDI2_NUM_OF_QM_ERR_CAUSE 18 #define GAUDI2_NUM_OF_QM_LCP_ERR_CAUSE 25 #define GAUDI2_NUM_OF_QM_ARB_ERR_CAUSE 3 @@ -675,14 +677,13 @@ static const char * const gaudi2_kdma_core_interrupts_cause[GAUDI2_NUM_OF_DMA_CO struct gaudi2_sm_sei_cause_data { const char *cause_name; const char *log_name; - u32 log_mask; }; static const struct gaudi2_sm_sei_cause_data gaudi2_sm_sei_cause[GAUDI2_NUM_OF_SM_SEI_ERR_CAUSE] = { - {"calculated SO value overflow/underflow", "SOB group ID", 0x7FF}, - {"payload address of monitor is not aligned to 4B", "monitor addr", 0xFFFF}, - {"armed monitor write got BRESP (SLVERR or DECERR)", "AXI id", 0xFFFF}, + {"calculated SO value overflow/underflow", "SOB ID"}, + {"payload address of monitor is not aligned to 4B", "monitor addr"}, + {"armed monitor write got BRESP (SLVERR or DECERR)", "AXI id"}, }; static const char * const @@ -1568,7 +1569,7 @@ enum rtr_id { DCORE3_RTR7, }; -static const u32 gaudi2_tpc_initiator_rtr_id[NUM_OF_TPC_PER_DCORE * NUM_OF_DCORES + 1] = { +static const u32 gaudi2_tpc_initiator_hbw_rtr_id[NUM_OF_TPC_PER_DCORE * NUM_OF_DCORES + 1] = { DCORE0_RTR1, DCORE0_RTR1, DCORE0_RTR2, DCORE0_RTR2, DCORE0_RTR3, DCORE0_RTR3, DCORE1_RTR6, DCORE1_RTR6, DCORE1_RTR5, DCORE1_RTR5, DCORE1_RTR4, DCORE1_RTR4, DCORE2_RTR3, DCORE2_RTR3, DCORE2_RTR2, DCORE2_RTR2, DCORE2_RTR1, DCORE2_RTR1, @@ -1576,33 +1577,61 @@ static const u32 gaudi2_tpc_initiator_rtr_id[NUM_OF_TPC_PER_DCORE * NUM_OF_DCORE DCORE0_RTR0 }; -static const u32 gaudi2_dec_initiator_rtr_id[NUMBER_OF_DEC] = { +static const u32 gaudi2_tpc_initiator_lbw_rtr_id[NUM_OF_TPC_PER_DCORE * NUM_OF_DCORES + 1] = { + DCORE0_RTR1, DCORE0_RTR1, DCORE0_RTR1, DCORE0_RTR1, DCORE0_RTR2, DCORE0_RTR2, + DCORE1_RTR7, DCORE1_RTR7, DCORE1_RTR6, DCORE1_RTR6, DCORE1_RTR5, DCORE1_RTR5, + DCORE2_RTR2, DCORE2_RTR2, DCORE2_RTR1, DCORE2_RTR1, DCORE2_RTR0, DCORE2_RTR0, + DCORE3_RTR5, DCORE3_RTR5, DCORE3_RTR6, DCORE3_RTR6, DCORE3_RTR7, DCORE3_RTR7, + DCORE0_RTR0 +}; + +static const u32 gaudi2_dec_initiator_hbw_rtr_id[NUMBER_OF_DEC] = { DCORE0_RTR0, DCORE0_RTR0, DCORE1_RTR7, DCORE1_RTR7, DCORE2_RTR0, DCORE2_RTR0, DCORE3_RTR7, DCORE3_RTR7, DCORE0_RTR0, DCORE0_RTR0 }; -static const u32 gaudi2_nic_initiator_rtr_id[NIC_NUMBER_OF_MACROS] = { +static const u32 gaudi2_dec_initiator_lbw_rtr_id[NUMBER_OF_DEC] = { + DCORE0_RTR1, DCORE0_RTR1, DCORE1_RTR6, DCORE1_RTR6, DCORE2_RTR1, DCORE2_RTR1, + DCORE3_RTR6, DCORE3_RTR6, DCORE0_RTR0, DCORE0_RTR0 +}; + +static const u32 gaudi2_nic_initiator_hbw_rtr_id[NIC_NUMBER_OF_MACROS] = { DCORE1_RTR7, DCORE1_RTR7, DCORE1_RTR7, DCORE1_RTR7, DCORE1_RTR7, DCORE2_RTR0, DCORE2_RTR0, DCORE2_RTR0, DCORE2_RTR0, DCORE3_RTR7, DCORE3_RTR7, DCORE3_RTR7 }; -struct sft_info { - u8 interface_id; - u8 dcore_id; +static const u32 gaudi2_nic_initiator_lbw_rtr_id[NIC_NUMBER_OF_MACROS] = { + DCORE1_RTR7, DCORE1_RTR7, DCORE1_RTR7, DCORE1_RTR7, DCORE1_RTR7, DCORE2_RTR0, + DCORE2_RTR0, DCORE2_RTR0, DCORE2_RTR0, DCORE3_RTR7, DCORE3_RTR7, DCORE3_RTR7 }; -static const struct sft_info gaudi2_edma_initiator_sft_id[NUM_OF_EDMA_PER_DCORE * NUM_OF_DCORES] = { - {0, 0}, {1, 0}, {0, 1}, {1, 1}, {1, 2}, {1, 3}, {0, 2}, {0, 3}, +static const u32 gaudi2_edma_initiator_hbw_sft[NUM_OF_EDMA_PER_DCORE * NUM_OF_DCORES] = { + mmSFT0_HBW_RTR_IF1_MSTR_IF_RR_SHRD_HBW_BASE, + mmSFT0_HBW_RTR_IF0_MSTR_IF_RR_SHRD_HBW_BASE, + mmSFT1_HBW_RTR_IF1_MSTR_IF_RR_SHRD_HBW_BASE, + mmSFT1_HBW_RTR_IF0_MSTR_IF_RR_SHRD_HBW_BASE, + mmSFT2_HBW_RTR_IF0_MSTR_IF_RR_SHRD_HBW_BASE, + mmSFT2_HBW_RTR_IF1_MSTR_IF_RR_SHRD_HBW_BASE, + mmSFT3_HBW_RTR_IF0_MSTR_IF_RR_SHRD_HBW_BASE, + mmSFT3_HBW_RTR_IF1_MSTR_IF_RR_SHRD_HBW_BASE }; -static const u32 gaudi2_pdma_initiator_rtr_id[NUM_OF_PDMA] = { +static const u32 gaudi2_pdma_initiator_hbw_rtr_id[NUM_OF_PDMA] = { DCORE0_RTR0, DCORE0_RTR0 }; -static const u32 gaudi2_rot_initiator_rtr_id[NUM_OF_ROT] = { +static const u32 gaudi2_pdma_initiator_lbw_rtr_id[NUM_OF_PDMA] = { + DCORE0_RTR2, DCORE0_RTR2 +}; + +static const u32 gaudi2_rot_initiator_hbw_rtr_id[NUM_OF_ROT] = { DCORE2_RTR0, DCORE3_RTR7 }; +static const u32 gaudi2_rot_initiator_lbw_rtr_id[NUM_OF_ROT] = { + DCORE2_RTR2, DCORE3_RTR5 +}; + struct mme_initiators_rtr_id { u32 wap0; u32 wap1; @@ -1655,6 +1684,30 @@ struct hbm_mc_error_causes { char cause[50]; }; +static struct hl_special_block_info gaudi2_special_blocks[] = GAUDI2_SPECIAL_BLOCKS; + +/* Special blocks iterator is currently used to configure security protection bits, + * and read global errors. Most HW blocks are addressable and those who aren't (N/A)- + * must be skipped. Following configurations are commonly used for both PB config + * and global error reading, since currently they both share the same settings. + * Once it changes, we must remember to use separate configurations for either one. + */ +static int gaudi2_iterator_skip_block_types[] = { + GAUDI2_BLOCK_TYPE_PLL, + GAUDI2_BLOCK_TYPE_EU_BIST, + GAUDI2_BLOCK_TYPE_HBM, + GAUDI2_BLOCK_TYPE_XFT +}; + +static struct range gaudi2_iterator_skip_block_ranges[] = { + /* Skip all PSOC blocks except for PSOC_GLOBAL_CONF */ + {mmPSOC_I2C_M0_BASE, mmPSOC_EFUSE_BASE}, + {mmPSOC_BTL_BASE, mmPSOC_MSTR_IF_RR_SHRD_HBW_BASE}, + /* Skip all CPU blocks except for CPU_IF */ + {mmCPU_CA53_CFG_BASE, mmCPU_CA53_CFG_BASE}, + {mmCPU_TIMESTAMP_BASE, mmCPU_MSTR_IF_RR_SHRD_HBW_BASE} +}; + static struct hbm_mc_error_causes hbm_mc_spi[GAUDI2_NUM_OF_HBM_MC_SPI_CAUSE] = { {HBM_MC_SPI_TEMP_PIN_CHG_MASK, "temperature pins changed"}, {HBM_MC_SPI_THR_ENG_MASK, "temperature-based throttling engaged"}, @@ -2070,6 +2123,8 @@ static int gaudi2_set_fixed_properties(struct hl_device *hdev) prop->dma_mask = 64; + prop->hbw_flush_reg = mmPCIE_WRAP_SPECIAL_GLBL_SPARE_0; + return 0; } @@ -2434,6 +2489,25 @@ static int gaudi2_set_cluster_binning_masks(struct hl_device *hdev) return 0; } +static int gaudi2_set_binning_masks(struct hl_device *hdev) +{ + int rc; + + rc = gaudi2_set_cluster_binning_masks(hdev); + if (rc) + return rc; + + rc = gaudi2_set_tpc_binning_masks(hdev); + if (rc) + return rc; + + rc = gaudi2_set_dec_binning_masks(hdev); + if (rc) + return rc; + + return 0; +} + static int gaudi2_cpucp_info_get(struct hl_device *hdev) { struct gaudi2_device *gaudi2 = hdev->asic_specific; @@ -2485,19 +2559,11 @@ static int gaudi2_cpucp_info_get(struct hl_device *hdev) * at this point the DRAM parameters need to be updated according to data obtained * from the FW */ - rc = gaudi2_set_dram_properties(hdev); + rc = hdev->asic_funcs->set_dram_properties(hdev); if (rc) return rc; - rc = gaudi2_set_cluster_binning_masks(hdev); - if (rc) - return rc; - - rc = gaudi2_set_tpc_binning_masks(hdev); - if (rc) - return rc; - - rc = gaudi2_set_dec_binning_masks(hdev); + rc = hdev->asic_funcs->set_binning_masks(hdev); if (rc) return rc; @@ -2925,11 +2991,11 @@ static void gaudi2_user_interrupt_setup(struct hl_device *hdev) /* Initialize common user CQ interrupt */ HL_USR_INTR_STRUCT_INIT(hdev->common_user_cq_interrupt, hdev, - HL_COMMON_USER_CQ_INTERRUPT_ID, false); + HL_COMMON_USER_CQ_INTERRUPT_ID, HL_USR_INTERRUPT_CQ); /* Initialize common decoder interrupt */ HL_USR_INTR_STRUCT_INIT(hdev->common_decoder_interrupt, hdev, - HL_COMMON_DEC_INTERRUPT_ID, true); + HL_COMMON_DEC_INTERRUPT_ID, HL_USR_INTERRUPT_DECODER); /* User interrupts structure holds both decoder and user interrupts from various engines. * We first initialize the decoder interrupts and then we add the user interrupts. @@ -2942,10 +3008,11 @@ static void gaudi2_user_interrupt_setup(struct hl_device *hdev) */ for (i = GAUDI2_IRQ_NUM_DCORE0_DEC0_NRM, j = 0 ; i <= GAUDI2_IRQ_NUM_SHARED_DEC1_NRM; i += 2, j++) - HL_USR_INTR_STRUCT_INIT(hdev->user_interrupt[j], hdev, i, true); + HL_USR_INTR_STRUCT_INIT(hdev->user_interrupt[j], hdev, i, + HL_USR_INTERRUPT_DECODER); for (i = GAUDI2_IRQ_NUM_USER_FIRST, k = 0 ; k < prop->user_interrupt_count; i++, j++, k++) - HL_USR_INTR_STRUCT_INIT(hdev->user_interrupt[j], hdev, i, false); + HL_USR_INTR_STRUCT_INIT(hdev->user_interrupt[j], hdev, i, HL_USR_INTERRUPT_CQ); } static inline int gaudi2_get_non_zero_random_int(void) @@ -2955,6 +3022,99 @@ static inline int gaudi2_get_non_zero_random_int(void) return rand ? rand : 1; } +static void gaudi2_special_blocks_free(struct hl_device *hdev) +{ + struct asic_fixed_properties *prop = &hdev->asic_prop; + struct hl_skip_blocks_cfg *skip_special_blocks_cfg = + &prop->skip_special_blocks_cfg; + + kfree(prop->special_blocks); + kfree(skip_special_blocks_cfg->block_types); + kfree(skip_special_blocks_cfg->block_ranges); +} + +static void gaudi2_special_blocks_iterator_free(struct hl_device *hdev) +{ + gaudi2_special_blocks_free(hdev); +} + +static bool gaudi2_special_block_skip(struct hl_device *hdev, + struct hl_special_blocks_cfg *special_blocks_cfg, + u32 blk_idx, u32 major, u32 minor, u32 sub_minor) +{ + return false; +} + +static int gaudi2_special_blocks_config(struct hl_device *hdev) +{ + struct asic_fixed_properties *prop = &hdev->asic_prop; + int i, rc; + + /* Configure Special blocks */ + prop->glbl_err_cause_num = GAUDI2_NUM_OF_GLBL_ERR_CAUSE; + prop->num_of_special_blocks = ARRAY_SIZE(gaudi2_special_blocks); + prop->special_blocks = kmalloc_array(prop->num_of_special_blocks, + sizeof(*prop->special_blocks), GFP_KERNEL); + if (!prop->special_blocks) + return -ENOMEM; + + for (i = 0 ; i < prop->num_of_special_blocks ; i++) + memcpy(&prop->special_blocks[i], &gaudi2_special_blocks[i], + sizeof(*prop->special_blocks)); + + /* Configure when to skip Special blocks */ + memset(&prop->skip_special_blocks_cfg, 0, sizeof(prop->skip_special_blocks_cfg)); + prop->skip_special_blocks_cfg.skip_block_hook = gaudi2_special_block_skip; + + if (ARRAY_SIZE(gaudi2_iterator_skip_block_types)) { + prop->skip_special_blocks_cfg.block_types = + kmalloc_array(ARRAY_SIZE(gaudi2_iterator_skip_block_types), + sizeof(gaudi2_iterator_skip_block_types[0]), GFP_KERNEL); + if (!prop->skip_special_blocks_cfg.block_types) { + rc = -ENOMEM; + goto free_special_blocks; + } + + memcpy(prop->skip_special_blocks_cfg.block_types, gaudi2_iterator_skip_block_types, + sizeof(gaudi2_iterator_skip_block_types)); + + prop->skip_special_blocks_cfg.block_types_len = + ARRAY_SIZE(gaudi2_iterator_skip_block_types); + } + + if (ARRAY_SIZE(gaudi2_iterator_skip_block_ranges)) { + prop->skip_special_blocks_cfg.block_ranges = + kmalloc_array(ARRAY_SIZE(gaudi2_iterator_skip_block_ranges), + sizeof(gaudi2_iterator_skip_block_ranges[0]), GFP_KERNEL); + if (!prop->skip_special_blocks_cfg.block_ranges) { + rc = -ENOMEM; + goto free_skip_special_blocks_types; + } + + for (i = 0 ; i < ARRAY_SIZE(gaudi2_iterator_skip_block_ranges) ; i++) + memcpy(&prop->skip_special_blocks_cfg.block_ranges[i], + &gaudi2_iterator_skip_block_ranges[i], + sizeof(struct range)); + + prop->skip_special_blocks_cfg.block_ranges_len = + ARRAY_SIZE(gaudi2_iterator_skip_block_ranges); + } + + return 0; + +free_skip_special_blocks_types: + kfree(prop->skip_special_blocks_cfg.block_types); +free_special_blocks: + kfree(prop->special_blocks); + + return rc; +} + +static int gaudi2_special_blocks_iterator_config(struct hl_device *hdev) +{ + return gaudi2_special_blocks_config(hdev); +} + static int gaudi2_sw_init(struct hl_device *hdev) { struct asic_fixed_properties *prop = &hdev->asic_prop; @@ -3050,8 +3210,15 @@ static int gaudi2_sw_init(struct hl_device *hdev) hdev->asic_funcs->set_pci_memory_regions(hdev); + rc = gaudi2_special_blocks_iterator_config(hdev); + if (rc) + goto free_scratchpad_mem; + return 0; +free_scratchpad_mem: + hl_asic_dma_pool_free(hdev, gaudi2->scratchpad_kernel_address, + gaudi2->scratchpad_bus_address); free_virt_msix_db_mem: hl_cpu_accessible_dma_pool_free(hdev, prop->pmmu.page_size, gaudi2->virt_msix_db_cpu_addr); free_cpu_accessible_dma_pool: @@ -3071,6 +3238,8 @@ static int gaudi2_sw_fini(struct hl_device *hdev) struct asic_fixed_properties *prop = &hdev->asic_prop; struct gaudi2_device *gaudi2 = hdev->asic_specific; + gaudi2_special_blocks_iterator_free(hdev); + hl_cpu_accessible_dma_pool_free(hdev, prop->pmmu.page_size, gaudi2->virt_msix_db_cpu_addr); gen_pool_destroy(hdev->cpu_accessible_dma_pool); @@ -5483,7 +5652,31 @@ static void gaudi2_hw_fini(struct hl_device *hdev, bool hard_reset, bool fw_rese skip_reset: if (driver_performs_reset || hard_reset) - gaudi2_poll_btm_indication(hdev, reset_sleep_ms, poll_timeout_us); + /* + * Instead of waiting for BTM indication we should wait for preboot ready: + * Consider the below scenario: + * 1. FW update is being triggered + * - setting the dirty bit + * 2. hard reset will be triggered due to the dirty bit + * 3. FW initiates the reset: + * - dirty bit cleared + * - BTM indication cleared + * - preboot ready indication cleared + * 4. during hard reset: + * - BTM indication will be set + * - BIST test performed and another reset triggered + * 5. only after this reset the preboot will set the preboot ready + * + * when polling on BTM indication alone we can lose sync with FW while trying to + * communicate with FW that is during reset. + * to overcome this we will always wait to preboot ready indication + */ + if ((hdev->fw_components & FW_TYPE_PREBOOT_CPU)) { + msleep(reset_sleep_ms); + hl_fw_wait_preboot_ready(hdev); + } else { + gaudi2_poll_btm_indication(hdev, reset_sleep_ms, poll_timeout_us); + } else gaudi2_get_soft_rst_done_indication(hdev, poll_timeout_us); @@ -6803,38 +6996,37 @@ static inline bool is_info_event(u32 event) switch (event) { case GAUDI2_EVENT_CPU_CPLD_SHUTDOWN_CAUSE: case GAUDI2_EVENT_CPU_FIX_POWER_ENV_S ... GAUDI2_EVENT_CPU_FIX_THERMAL_ENV_E: + + /* return in case of NIC status event - these events are received periodically and not as + * an indication to an error. + */ + case GAUDI2_EVENT_CPU0_STATUS_NIC0_ENG0 ... GAUDI2_EVENT_CPU11_STATUS_NIC11_ENG1: return true; default: return false; } } -static void gaudi2_print_irq_info(struct hl_device *hdev, u16 event_type) +static void gaudi2_print_event(struct hl_device *hdev, u16 event_type, + bool ratelimited, const char *fmt, ...) { - char desc[64] = ""; - bool event_valid = false; + struct va_format vaf; + va_list args; - /* return in case of NIC status event - these events are received periodically and not as - * an indication to an error, thus not printed. - */ - if (event_type >= GAUDI2_EVENT_CPU0_STATUS_NIC0_ENG0 && - event_type <= GAUDI2_EVENT_CPU11_STATUS_NIC11_ENG1) - return; + va_start(args, fmt); + vaf.fmt = fmt; + vaf.va = &args; - if (gaudi2_irq_map_table[event_type].valid) { - snprintf(desc, sizeof(desc), gaudi2_irq_map_table[event_type].name); - event_valid = true; - } - - if (!event_valid) - snprintf(desc, sizeof(desc), "N/A"); - - if (is_info_event(event_type)) - dev_info_ratelimited(hdev->dev, "Received H/W interrupt %d [\"%s\"]\n", - event_type, desc); + if (ratelimited) + dev_err_ratelimited(hdev->dev, "%s: %pV\n", + gaudi2_irq_map_table[event_type].valid ? + gaudi2_irq_map_table[event_type].name : "N/A Event", &vaf); else - dev_err_ratelimited(hdev->dev, "Received H/W interrupt %d [\"%s\"]\n", - event_type, desc); + dev_err(hdev->dev, "%s: %pV\n", + gaudi2_irq_map_table[event_type].valid ? + gaudi2_irq_map_table[event_type].name : "N/A Event", &vaf); + + va_end(args); } static bool gaudi2_handle_ecc_event(struct hl_device *hdev, u16 event_type, @@ -6847,7 +7039,7 @@ static bool gaudi2_handle_ecc_event(struct hl_device *hdev, u16 event_type, ecc_syndrom = le64_to_cpu(ecc_data->ecc_syndrom); memory_wrapper_idx = ecc_data->memory_wrapper_idx; - dev_err(hdev->dev, + gaudi2_print_event(hdev, event_type, !ecc_data->is_critical, "ECC error detected. address: %#llx. Syndrom: %#llx. block id %u. critical %u.\n", ecc_address, ecc_syndrom, memory_wrapper_idx, ecc_data->is_critical); @@ -6987,10 +7179,10 @@ static void print_qman_data_on_err(struct hl_device *hdev, u32 qid_base, u32 str gaudi2_print_last_pqes_on_err(hdev, qid_base, i, qman_base, false); } -static void gaudi2_handle_qman_err_generic(struct hl_device *hdev, const char *qm_name, - u64 qman_base, u32 qid_base) +static int gaudi2_handle_qman_err_generic(struct hl_device *hdev, u16 event_type, + u64 qman_base, u32 qid_base) { - u32 i, j, glbl_sts_val, arb_err_val, num_error_causes; + u32 i, j, glbl_sts_val, arb_err_val, num_error_causes, error_count = 0; u64 glbl_sts_addr, arb_err_addr; char reg_desc[32]; @@ -7013,12 +7205,14 @@ static void gaudi2_handle_qman_err_generic(struct hl_device *hdev, const char *q } for (j = 0 ; j < num_error_causes ; j++) - if (glbl_sts_val & BIT(j)) - dev_err_ratelimited(hdev->dev, "%s %s. err cause: %s\n", - qm_name, reg_desc, - i == QMAN_STREAMS ? - gaudi2_qman_lower_cp_error_cause[j] : - gaudi2_qman_error_cause[j]); + if (glbl_sts_val & BIT(j)) { + gaudi2_print_event(hdev, event_type, true, + "%s. err cause: %s", reg_desc, + i == QMAN_STREAMS ? + gaudi2_qman_lower_cp_error_cause[j] : + gaudi2_qman_error_cause[j]); + error_count++; + } print_qman_data_on_err(hdev, qid_base, i, qman_base); } @@ -7026,18 +7220,23 @@ static void gaudi2_handle_qman_err_generic(struct hl_device *hdev, const char *q arb_err_val = RREG32(arb_err_addr); if (!arb_err_val) - return; + goto out; for (j = 0 ; j < GAUDI2_NUM_OF_QM_ARB_ERR_CAUSE ; j++) { - if (arb_err_val & BIT(j)) - dev_err_ratelimited(hdev->dev, "%s ARB_ERR. err cause: %s\n", - qm_name, gaudi2_qman_arb_error_cause[j]); + if (arb_err_val & BIT(j)) { + gaudi2_print_event(hdev, event_type, true, + "ARB_ERR. err cause: %s", + gaudi2_qman_arb_error_cause[j]); + error_count++; + } } + +out: + return error_count; } static void gaudi2_razwi_rr_hbw_shared_printf_info(struct hl_device *hdev, u64 rtr_mstr_if_base_addr, bool is_write, char *name, - bool read_razwi_regs, struct hl_eq_razwi_info *razwi_info, enum gaudi2_engine_id id, u64 *event_mask) { u32 razwi_hi, razwi_lo, razwi_xy; @@ -7045,26 +7244,14 @@ static void gaudi2_razwi_rr_hbw_shared_printf_info(struct hl_device *hdev, u8 rd_wr_flag; if (is_write) { - if (read_razwi_regs) { - razwi_hi = RREG32(rtr_mstr_if_base_addr + RR_SHRD_HBW_AW_RAZWI_HI); - razwi_lo = RREG32(rtr_mstr_if_base_addr + RR_SHRD_HBW_AW_RAZWI_LO); - razwi_xy = RREG32(rtr_mstr_if_base_addr + RR_SHRD_HBW_AW_RAZWI_XY); - } else { - razwi_hi = le32_to_cpu(razwi_info->hbw.rr_aw_razwi_hi_reg); - razwi_lo = le32_to_cpu(razwi_info->hbw.rr_aw_razwi_lo_reg); - razwi_xy = le32_to_cpu(razwi_info->hbw.rr_aw_razwi_id_reg); - } + razwi_hi = RREG32(rtr_mstr_if_base_addr + RR_SHRD_HBW_AW_RAZWI_HI); + razwi_lo = RREG32(rtr_mstr_if_base_addr + RR_SHRD_HBW_AW_RAZWI_LO); + razwi_xy = RREG32(rtr_mstr_if_base_addr + RR_SHRD_HBW_AW_RAZWI_XY); rd_wr_flag = HL_RAZWI_WRITE; } else { - if (read_razwi_regs) { - razwi_hi = RREG32(rtr_mstr_if_base_addr + RR_SHRD_HBW_AR_RAZWI_HI); - razwi_lo = RREG32(rtr_mstr_if_base_addr + RR_SHRD_HBW_AR_RAZWI_LO); - razwi_xy = RREG32(rtr_mstr_if_base_addr + RR_SHRD_HBW_AR_RAZWI_XY); - } else { - razwi_hi = le32_to_cpu(razwi_info->hbw.rr_ar_razwi_hi_reg); - razwi_lo = le32_to_cpu(razwi_info->hbw.rr_ar_razwi_lo_reg); - razwi_xy = le32_to_cpu(razwi_info->hbw.rr_ar_razwi_id_reg); - } + razwi_hi = RREG32(rtr_mstr_if_base_addr + RR_SHRD_HBW_AR_RAZWI_HI); + razwi_lo = RREG32(rtr_mstr_if_base_addr + RR_SHRD_HBW_AR_RAZWI_LO); + razwi_xy = RREG32(rtr_mstr_if_base_addr + RR_SHRD_HBW_AR_RAZWI_XY); rd_wr_flag = HL_RAZWI_READ; } @@ -7078,38 +7265,26 @@ static void gaudi2_razwi_rr_hbw_shared_printf_info(struct hl_device *hdev, static void gaudi2_razwi_rr_lbw_shared_printf_info(struct hl_device *hdev, u64 rtr_mstr_if_base_addr, bool is_write, char *name, - bool read_razwi_regs, struct hl_eq_razwi_info *razwi_info, enum gaudi2_engine_id id, u64 *event_mask) { - u32 razwi_addr, razwi_xy; + u64 razwi_addr = CFG_BASE; + u32 razwi_xy; u16 eng_id = id; u8 rd_wr_flag; if (is_write) { - if (read_razwi_regs) { - razwi_addr = RREG32(rtr_mstr_if_base_addr + RR_SHRD_LBW_AW_RAZWI); - razwi_xy = RREG32(rtr_mstr_if_base_addr + RR_SHRD_LBW_AW_RAZWI_XY); - } else { - razwi_addr = le32_to_cpu(razwi_info->lbw.rr_aw_razwi_reg); - razwi_xy = le32_to_cpu(razwi_info->lbw.rr_aw_razwi_id_reg); - } - + razwi_addr += RREG32(rtr_mstr_if_base_addr + RR_SHRD_LBW_AW_RAZWI); + razwi_xy = RREG32(rtr_mstr_if_base_addr + RR_SHRD_LBW_AW_RAZWI_XY); rd_wr_flag = HL_RAZWI_WRITE; } else { - if (read_razwi_regs) { - razwi_addr = RREG32(rtr_mstr_if_base_addr + RR_SHRD_LBW_AR_RAZWI); - razwi_xy = RREG32(rtr_mstr_if_base_addr + RR_SHRD_LBW_AR_RAZWI_XY); - } else { - razwi_addr = le32_to_cpu(razwi_info->lbw.rr_ar_razwi_reg); - razwi_xy = le32_to_cpu(razwi_info->lbw.rr_ar_razwi_id_reg); - } - + razwi_addr += RREG32(rtr_mstr_if_base_addr + RR_SHRD_LBW_AR_RAZWI); + razwi_xy = RREG32(rtr_mstr_if_base_addr + RR_SHRD_LBW_AR_RAZWI_XY); rd_wr_flag = HL_RAZWI_READ; } hl_handle_razwi(hdev, razwi_addr, &eng_id, 1, rd_wr_flag | HL_RAZWI_LBW, event_mask); dev_err_ratelimited(hdev->dev, - "%s-RAZWI SHARED RR LBW %s error, mstr_if 0x%llx, captured address 0x%x Initiator coordinates 0x%x\n", + "%s-RAZWI SHARED RR LBW %s error, mstr_if 0x%llx, captured address 0x%llX Initiator coordinates 0x%x\n", name, is_write ? "WR" : "RD", rtr_mstr_if_base_addr, razwi_addr, razwi_xy); } @@ -7164,183 +7339,148 @@ static enum gaudi2_engine_id gaudi2_razwi_calc_engine_id(struct hl_device *hdev, */ static void gaudi2_ack_module_razwi_event_handler(struct hl_device *hdev, enum razwi_event_sources module, u8 module_idx, - u8 module_sub_idx, struct hl_eq_razwi_info *razwi_info, - u64 *event_mask) + u8 module_sub_idx, u64 *event_mask) { - bool via_sft = false, read_razwi_regs = false; - u32 rtr_id, dcore_id, dcore_rtr_id, sft_id, eng_id; - u64 rtr_mstr_if_base_addr; + bool via_sft = false; + u32 hbw_rtr_id, lbw_rtr_id, dcore_id, dcore_rtr_id, eng_id; + u64 hbw_rtr_mstr_if_base_addr, lbw_rtr_mstr_if_base_addr; u32 hbw_shrd_aw = 0, hbw_shrd_ar = 0; u32 lbw_shrd_aw = 0, lbw_shrd_ar = 0; char initiator_name[64]; - if (hdev->pldm || !(hdev->fw_components & FW_TYPE_LINUX) || !razwi_info) - read_razwi_regs = true; - switch (module) { case RAZWI_TPC: - rtr_id = gaudi2_tpc_initiator_rtr_id[module_idx]; + hbw_rtr_id = gaudi2_tpc_initiator_hbw_rtr_id[module_idx]; + + /* TODO : remove this check and depend only on tpc routers table + * when SW-118828 is resolved + */ + if (!hdev->asic_prop.fw_security_enabled && + ((module_idx == 0) || (module_idx == 1))) + lbw_rtr_id = DCORE0_RTR0; + else + lbw_rtr_id = gaudi2_tpc_initiator_lbw_rtr_id[module_idx]; sprintf(initiator_name, "TPC_%u", module_idx); break; case RAZWI_MME: sprintf(initiator_name, "MME_%u", module_idx); switch (module_sub_idx) { case MME_WAP0: - rtr_id = gaudi2_mme_initiator_rtr_id[module_idx].wap0; + hbw_rtr_id = gaudi2_mme_initiator_rtr_id[module_idx].wap0; break; case MME_WAP1: - rtr_id = gaudi2_mme_initiator_rtr_id[module_idx].wap1; + hbw_rtr_id = gaudi2_mme_initiator_rtr_id[module_idx].wap1; break; case MME_WRITE: - rtr_id = gaudi2_mme_initiator_rtr_id[module_idx].write; + hbw_rtr_id = gaudi2_mme_initiator_rtr_id[module_idx].write; break; case MME_READ: - rtr_id = gaudi2_mme_initiator_rtr_id[module_idx].read; + hbw_rtr_id = gaudi2_mme_initiator_rtr_id[module_idx].read; break; case MME_SBTE0: - rtr_id = gaudi2_mme_initiator_rtr_id[module_idx].sbte0; + hbw_rtr_id = gaudi2_mme_initiator_rtr_id[module_idx].sbte0; break; case MME_SBTE1: - rtr_id = gaudi2_mme_initiator_rtr_id[module_idx].sbte1; + hbw_rtr_id = gaudi2_mme_initiator_rtr_id[module_idx].sbte1; break; case MME_SBTE2: - rtr_id = gaudi2_mme_initiator_rtr_id[module_idx].sbte2; + hbw_rtr_id = gaudi2_mme_initiator_rtr_id[module_idx].sbte2; break; case MME_SBTE3: - rtr_id = gaudi2_mme_initiator_rtr_id[module_idx].sbte3; + hbw_rtr_id = gaudi2_mme_initiator_rtr_id[module_idx].sbte3; break; case MME_SBTE4: - rtr_id = gaudi2_mme_initiator_rtr_id[module_idx].sbte4; + hbw_rtr_id = gaudi2_mme_initiator_rtr_id[module_idx].sbte4; break; default: return; } + lbw_rtr_id = hbw_rtr_id; break; case RAZWI_EDMA: - sft_id = gaudi2_edma_initiator_sft_id[module_idx].interface_id; - dcore_id = gaudi2_edma_initiator_sft_id[module_idx].dcore_id; + hbw_rtr_mstr_if_base_addr = gaudi2_edma_initiator_hbw_sft[module_idx]; + dcore_id = module_idx / NUM_OF_EDMA_PER_DCORE; + /* SFT has separate MSTR_IF for LBW, only there we can + * read the LBW razwi related registers + */ + lbw_rtr_mstr_if_base_addr = mmSFT0_LBW_RTR_IF_MSTR_IF_RR_SHRD_HBW_BASE + + dcore_id * SFT_DCORE_OFFSET; via_sft = true; sprintf(initiator_name, "EDMA_%u", module_idx); break; case RAZWI_PDMA: - rtr_id = gaudi2_pdma_initiator_rtr_id[module_idx]; + hbw_rtr_id = gaudi2_pdma_initiator_hbw_rtr_id[module_idx]; + lbw_rtr_id = gaudi2_pdma_initiator_lbw_rtr_id[module_idx]; sprintf(initiator_name, "PDMA_%u", module_idx); break; case RAZWI_NIC: - rtr_id = gaudi2_nic_initiator_rtr_id[module_idx]; + hbw_rtr_id = gaudi2_nic_initiator_hbw_rtr_id[module_idx]; + lbw_rtr_id = gaudi2_nic_initiator_lbw_rtr_id[module_idx]; sprintf(initiator_name, "NIC_%u", module_idx); break; case RAZWI_DEC: - rtr_id = gaudi2_dec_initiator_rtr_id[module_idx]; + hbw_rtr_id = gaudi2_dec_initiator_hbw_rtr_id[module_idx]; + lbw_rtr_id = gaudi2_dec_initiator_lbw_rtr_id[module_idx]; sprintf(initiator_name, "DEC_%u", module_idx); break; case RAZWI_ROT: - rtr_id = gaudi2_rot_initiator_rtr_id[module_idx]; + hbw_rtr_id = gaudi2_rot_initiator_hbw_rtr_id[module_idx]; + lbw_rtr_id = gaudi2_rot_initiator_lbw_rtr_id[module_idx]; sprintf(initiator_name, "ROT_%u", module_idx); break; default: return; } - if (!read_razwi_regs) { - if (le32_to_cpu(razwi_info->razwi_happened_mask) & RAZWI_HAPPENED_HBW) { - hbw_shrd_aw = le32_to_cpu(razwi_info->razwi_happened_mask) & - RAZWI_HAPPENED_AW; - hbw_shrd_ar = le32_to_cpu(razwi_info->razwi_happened_mask) & - RAZWI_HAPPENED_AR; - } else if (le32_to_cpu(razwi_info->razwi_happened_mask) & RAZWI_HAPPENED_LBW) { - lbw_shrd_aw = le32_to_cpu(razwi_info->razwi_happened_mask) & - RAZWI_HAPPENED_AW; - lbw_shrd_ar = le32_to_cpu(razwi_info->razwi_happened_mask) & - RAZWI_HAPPENED_AR; - } - rtr_mstr_if_base_addr = 0; - - goto dump_info; - } - /* Find router mstr_if register base */ - if (via_sft) { - rtr_mstr_if_base_addr = mmSFT0_HBW_RTR_IF0_RTR_CTRL_BASE + - dcore_id * SFT_DCORE_OFFSET + - sft_id * SFT_IF_OFFSET + - RTR_MSTR_IF_OFFSET; - } else { - dcore_id = rtr_id / NUM_OF_RTR_PER_DCORE; - dcore_rtr_id = rtr_id % NUM_OF_RTR_PER_DCORE; - rtr_mstr_if_base_addr = mmDCORE0_RTR0_CTRL_BASE + + if (!via_sft) { + dcore_id = hbw_rtr_id / NUM_OF_RTR_PER_DCORE; + dcore_rtr_id = hbw_rtr_id % NUM_OF_RTR_PER_DCORE; + hbw_rtr_mstr_if_base_addr = mmDCORE0_RTR0_CTRL_BASE + dcore_id * DCORE_OFFSET + dcore_rtr_id * DCORE_RTR_OFFSET + RTR_MSTR_IF_OFFSET; + lbw_rtr_mstr_if_base_addr = hbw_rtr_mstr_if_base_addr + + (((s32)lbw_rtr_id - hbw_rtr_id) * DCORE_RTR_OFFSET); } /* Find out event cause by reading "RAZWI_HAPPENED" registers */ - hbw_shrd_aw = RREG32(rtr_mstr_if_base_addr + RR_SHRD_HBW_AW_RAZWI_HAPPENED); - - hbw_shrd_ar = RREG32(rtr_mstr_if_base_addr + RR_SHRD_HBW_AR_RAZWI_HAPPENED); - - if (via_sft) { - /* SFT has separate MSTR_IF for LBW, only there we can - * read the LBW razwi related registers - */ - u64 base; - - base = mmSFT0_HBW_RTR_IF0_RTR_CTRL_BASE + dcore_id * SFT_DCORE_OFFSET + - RTR_LBW_MSTR_IF_OFFSET; - - lbw_shrd_aw = RREG32(base + RR_SHRD_LBW_AW_RAZWI_HAPPENED); - - lbw_shrd_ar = RREG32(base + RR_SHRD_LBW_AR_RAZWI_HAPPENED); - } else { - lbw_shrd_aw = RREG32(rtr_mstr_if_base_addr + RR_SHRD_LBW_AW_RAZWI_HAPPENED); - - lbw_shrd_ar = RREG32(rtr_mstr_if_base_addr + RR_SHRD_LBW_AR_RAZWI_HAPPENED); - } - -dump_info: - /* check if there is no RR razwi indication at all */ - if (!hbw_shrd_aw && !hbw_shrd_ar && !lbw_shrd_aw && !lbw_shrd_ar) - return; + hbw_shrd_aw = RREG32(hbw_rtr_mstr_if_base_addr + RR_SHRD_HBW_AW_RAZWI_HAPPENED); + hbw_shrd_ar = RREG32(hbw_rtr_mstr_if_base_addr + RR_SHRD_HBW_AR_RAZWI_HAPPENED); + lbw_shrd_aw = RREG32(lbw_rtr_mstr_if_base_addr + RR_SHRD_LBW_AW_RAZWI_HAPPENED); + lbw_shrd_ar = RREG32(lbw_rtr_mstr_if_base_addr + RR_SHRD_LBW_AR_RAZWI_HAPPENED); eng_id = gaudi2_razwi_calc_engine_id(hdev, module, module_idx); if (hbw_shrd_aw) { - gaudi2_razwi_rr_hbw_shared_printf_info(hdev, rtr_mstr_if_base_addr, true, - initiator_name, read_razwi_regs, razwi_info, - eng_id, event_mask); + gaudi2_razwi_rr_hbw_shared_printf_info(hdev, hbw_rtr_mstr_if_base_addr, true, + initiator_name, eng_id, event_mask); /* Clear event indication */ - if (read_razwi_regs) - WREG32(rtr_mstr_if_base_addr + RR_SHRD_HBW_AW_RAZWI_HAPPENED, hbw_shrd_aw); + WREG32(hbw_rtr_mstr_if_base_addr + RR_SHRD_HBW_AW_RAZWI_HAPPENED, hbw_shrd_aw); } if (hbw_shrd_ar) { - gaudi2_razwi_rr_hbw_shared_printf_info(hdev, rtr_mstr_if_base_addr, false, - initiator_name, read_razwi_regs, razwi_info, - eng_id, event_mask); + gaudi2_razwi_rr_hbw_shared_printf_info(hdev, hbw_rtr_mstr_if_base_addr, false, + initiator_name, eng_id, event_mask); /* Clear event indication */ - if (read_razwi_regs) - WREG32(rtr_mstr_if_base_addr + RR_SHRD_HBW_AR_RAZWI_HAPPENED, hbw_shrd_ar); + WREG32(hbw_rtr_mstr_if_base_addr + RR_SHRD_HBW_AR_RAZWI_HAPPENED, hbw_shrd_ar); } if (lbw_shrd_aw) { - gaudi2_razwi_rr_lbw_shared_printf_info(hdev, rtr_mstr_if_base_addr, true, - initiator_name, read_razwi_regs, razwi_info, - eng_id, event_mask); + gaudi2_razwi_rr_lbw_shared_printf_info(hdev, lbw_rtr_mstr_if_base_addr, true, + initiator_name, eng_id, event_mask); /* Clear event indication */ - if (read_razwi_regs) - WREG32(rtr_mstr_if_base_addr + RR_SHRD_LBW_AW_RAZWI_HAPPENED, lbw_shrd_aw); + WREG32(lbw_rtr_mstr_if_base_addr + RR_SHRD_LBW_AW_RAZWI_HAPPENED, lbw_shrd_aw); } if (lbw_shrd_ar) { - gaudi2_razwi_rr_lbw_shared_printf_info(hdev, rtr_mstr_if_base_addr, false, - initiator_name, read_razwi_regs, razwi_info, - eng_id, event_mask); + gaudi2_razwi_rr_lbw_shared_printf_info(hdev, lbw_rtr_mstr_if_base_addr, false, + initiator_name, eng_id, event_mask); /* Clear event indication */ - if (read_razwi_regs) - WREG32(rtr_mstr_if_base_addr + RR_SHRD_LBW_AR_RAZWI_HAPPENED, lbw_shrd_ar); + WREG32(lbw_rtr_mstr_if_base_addr + RR_SHRD_LBW_AR_RAZWI_HAPPENED, lbw_shrd_ar); } } @@ -7352,42 +7492,38 @@ static void gaudi2_check_if_razwi_happened(struct hl_device *hdev) /* check all TPCs */ for (mod_idx = 0 ; mod_idx < (NUM_OF_TPC_PER_DCORE * NUM_OF_DCORES + 1) ; mod_idx++) { if (prop->tpc_enabled_mask & BIT(mod_idx)) - gaudi2_ack_module_razwi_event_handler(hdev, RAZWI_TPC, mod_idx, 0, NULL, - NULL); + gaudi2_ack_module_razwi_event_handler(hdev, RAZWI_TPC, mod_idx, 0, NULL); } /* check all MMEs */ for (mod_idx = 0 ; mod_idx < (NUM_OF_MME_PER_DCORE * NUM_OF_DCORES) ; mod_idx++) for (sub_mod = MME_WAP0 ; sub_mod < MME_INITIATORS_MAX ; sub_mod++) gaudi2_ack_module_razwi_event_handler(hdev, RAZWI_MME, mod_idx, - sub_mod, NULL, NULL); + sub_mod, NULL); /* check all EDMAs */ for (mod_idx = 0 ; mod_idx < (NUM_OF_EDMA_PER_DCORE * NUM_OF_DCORES) ; mod_idx++) if (prop->edma_enabled_mask & BIT(mod_idx)) - gaudi2_ack_module_razwi_event_handler(hdev, RAZWI_EDMA, mod_idx, 0, NULL, - NULL); + gaudi2_ack_module_razwi_event_handler(hdev, RAZWI_EDMA, mod_idx, 0, NULL); /* check all PDMAs */ for (mod_idx = 0 ; mod_idx < NUM_OF_PDMA ; mod_idx++) - gaudi2_ack_module_razwi_event_handler(hdev, RAZWI_PDMA, mod_idx, 0, NULL, - NULL); + gaudi2_ack_module_razwi_event_handler(hdev, RAZWI_PDMA, mod_idx, 0, NULL); /* check all NICs */ for (mod_idx = 0 ; mod_idx < NIC_NUMBER_OF_PORTS ; mod_idx++) if (hdev->nic_ports_mask & BIT(mod_idx)) gaudi2_ack_module_razwi_event_handler(hdev, RAZWI_NIC, mod_idx >> 1, 0, - NULL, NULL); + NULL); /* check all DECs */ for (mod_idx = 0 ; mod_idx < NUMBER_OF_DEC ; mod_idx++) if (prop->decoder_enabled_mask & BIT(mod_idx)) - gaudi2_ack_module_razwi_event_handler(hdev, RAZWI_DEC, mod_idx, 0, NULL, - NULL); + gaudi2_ack_module_razwi_event_handler(hdev, RAZWI_DEC, mod_idx, 0, NULL); /* check all ROTs */ for (mod_idx = 0 ; mod_idx < NUM_OF_ROT ; mod_idx++) - gaudi2_ack_module_razwi_event_handler(hdev, RAZWI_ROT, mod_idx, 0, NULL, NULL); + gaudi2_ack_module_razwi_event_handler(hdev, RAZWI_ROT, mod_idx, 0, NULL); } static const char *gaudi2_get_initiators_name(u32 rtr_id) @@ -7645,19 +7781,19 @@ static void gaudi2_razwi_unmapped_addr_lbw_printf_info(struct hl_device *hdev, u u64 *event_mask) { u16 engines[HL_RAZWI_MAX_NUM_OF_ENGINES_PER_RTR], num_of_eng; - u32 razwi_addr; + u64 razwi_addr = CFG_BASE; u8 rd_wr_flag; num_of_eng = gaudi2_get_razwi_initiators(rtr_id, &engines[0]); if (is_write) { - razwi_addr = RREG32(rtr_ctrl_base_addr + DEC_RAZWI_LBW_AW_ADDR); + razwi_addr += RREG32(rtr_ctrl_base_addr + DEC_RAZWI_LBW_AW_ADDR); rd_wr_flag = HL_RAZWI_WRITE; /* Clear set indication */ WREG32(rtr_ctrl_base_addr + DEC_RAZWI_LBW_AW_SET, 0x1); } else { - razwi_addr = RREG32(rtr_ctrl_base_addr + DEC_RAZWI_LBW_AR_ADDR); + razwi_addr += RREG32(rtr_ctrl_base_addr + DEC_RAZWI_LBW_AR_ADDR); rd_wr_flag = HL_RAZWI_READ; /* Clear set indication */ @@ -7667,7 +7803,7 @@ static void gaudi2_razwi_unmapped_addr_lbw_printf_info(struct hl_device *hdev, u hl_handle_razwi(hdev, razwi_addr, &engines[0], num_of_eng, rd_wr_flag | HL_RAZWI_LBW, event_mask); dev_err_ratelimited(hdev->dev, - "RAZWI PSOC unmapped LBW %s error, rtr id %u, address %#x\n", + "RAZWI PSOC unmapped LBW %s error, rtr id %u, address 0x%llX\n", is_write ? "WR" : "RD", rtr_id, razwi_addr); dev_err_ratelimited(hdev->dev, @@ -7675,17 +7811,17 @@ static void gaudi2_razwi_unmapped_addr_lbw_printf_info(struct hl_device *hdev, u } /* PSOC RAZWI interrupt occurs only when trying to access a bad address */ -static void gaudi2_ack_psoc_razwi_event_handler(struct hl_device *hdev, u64 *event_mask) +static int gaudi2_ack_psoc_razwi_event_handler(struct hl_device *hdev, u64 *event_mask) { u32 hbw_aw_set, hbw_ar_set, lbw_aw_set, lbw_ar_set, rtr_id, dcore_id, dcore_rtr_id, xy, - razwi_mask_info, razwi_intr = 0; + razwi_mask_info, razwi_intr = 0, error_count = 0; int rtr_map_arr_len = NUM_OF_RTR_PER_DCORE * NUM_OF_DCORES; u64 rtr_ctrl_base_addr; if (hdev->pldm || !(hdev->fw_components & FW_TYPE_LINUX)) { razwi_intr = RREG32(mmPSOC_GLOBAL_CONF_RAZWI_INTERRUPT); if (!razwi_intr) - return; + return 0; } razwi_mask_info = RREG32(mmPSOC_GLOBAL_CONF_RAZWI_MASK_INFO); @@ -7743,33 +7879,41 @@ static void gaudi2_ack_psoc_razwi_event_handler(struct hl_device *hdev, u64 *eve gaudi2_razwi_unmapped_addr_lbw_printf_info(hdev, rtr_id, rtr_ctrl_base_addr, false, event_mask); + error_count++; + clear: /* Clear Interrupts only on pldm or if f/w doesn't handle interrupts */ if (hdev->pldm || !(hdev->fw_components & FW_TYPE_LINUX)) WREG32(mmPSOC_GLOBAL_CONF_RAZWI_INTERRUPT, razwi_intr); + + return error_count; } -static void _gaudi2_handle_qm_sei_err(struct hl_device *hdev, u64 qman_base) +static int _gaudi2_handle_qm_sei_err(struct hl_device *hdev, u64 qman_base, u16 event_type) { - u32 i, sts_val, sts_clr_val = 0; + u32 i, sts_val, sts_clr_val = 0, error_count = 0; sts_val = RREG32(qman_base + QM_SEI_STATUS_OFFSET); for (i = 0 ; i < GAUDI2_NUM_OF_QM_SEI_ERR_CAUSE ; i++) { if (sts_val & BIT(i)) { - dev_err_ratelimited(hdev->dev, "QM SEI. err cause: %s\n", - gaudi2_qm_sei_error_cause[i]); + gaudi2_print_event(hdev, event_type, true, + "err cause: %s", gaudi2_qm_sei_error_cause[i]); sts_clr_val |= BIT(i); + error_count++; } } WREG32(qman_base + QM_SEI_STATUS_OFFSET, sts_clr_val); + + return error_count; } -static void gaudi2_handle_qm_sei_err(struct hl_device *hdev, u16 event_type, - struct hl_eq_razwi_info *razwi_info, u64 *event_mask) +static int gaudi2_handle_qm_sei_err(struct hl_device *hdev, u16 event_type, + bool extended_err_check, u64 *event_mask) { enum razwi_event_sources module; + u32 error_count = 0; u64 qman_base; u8 index; @@ -7808,26 +7952,30 @@ static void gaudi2_handle_qm_sei_err(struct hl_device *hdev, u16 event_type, module = RAZWI_ROT; break; default: - return; + return 0; } - _gaudi2_handle_qm_sei_err(hdev, qman_base); + error_count = _gaudi2_handle_qm_sei_err(hdev, qman_base, event_type); /* There is a single event per NIC macro, so should check its both QMAN blocks */ if (event_type >= GAUDI2_EVENT_NIC0_AXI_ERROR_RESPONSE && event_type <= GAUDI2_EVENT_NIC11_AXI_ERROR_RESPONSE) - _gaudi2_handle_qm_sei_err(hdev, qman_base + NIC_QM_OFFSET); + error_count += _gaudi2_handle_qm_sei_err(hdev, + qman_base + NIC_QM_OFFSET, event_type); - /* check if RAZWI happened */ - if (razwi_info) - gaudi2_ack_module_razwi_event_handler(hdev, module, 0, 0, razwi_info, event_mask); + if (extended_err_check) { + /* check if RAZWI happened */ + gaudi2_ack_module_razwi_event_handler(hdev, module, 0, 0, event_mask); + hl_check_for_glbl_errors(hdev); + } + + return error_count; } -static void gaudi2_handle_qman_err(struct hl_device *hdev, u16 event_type) +static int gaudi2_handle_qman_err(struct hl_device *hdev, u16 event_type, u64 *event_mask) { - u32 qid_base; + u32 qid_base, error_count = 0; u64 qman_base; - char desc[32]; u8 index; switch (event_type) { @@ -7835,194 +7983,207 @@ static void gaudi2_handle_qman_err(struct hl_device *hdev, u16 event_type) index = event_type - GAUDI2_EVENT_TPC0_QM; qid_base = GAUDI2_QUEUE_ID_DCORE0_TPC_0_0 + index * QMAN_STREAMS; qman_base = mmDCORE0_TPC0_QM_BASE + index * DCORE_TPC_OFFSET; - snprintf(desc, ARRAY_SIZE(desc), "DCORE0_TPC%d_QM", index); break; case GAUDI2_EVENT_TPC6_QM ... GAUDI2_EVENT_TPC11_QM: index = event_type - GAUDI2_EVENT_TPC6_QM; qid_base = GAUDI2_QUEUE_ID_DCORE1_TPC_0_0 + index * QMAN_STREAMS; qman_base = mmDCORE1_TPC0_QM_BASE + index * DCORE_TPC_OFFSET; - snprintf(desc, ARRAY_SIZE(desc), "DCORE1_TPC%d_QM", index); break; case GAUDI2_EVENT_TPC12_QM ... GAUDI2_EVENT_TPC17_QM: index = event_type - GAUDI2_EVENT_TPC12_QM; qid_base = GAUDI2_QUEUE_ID_DCORE2_TPC_0_0 + index * QMAN_STREAMS; qman_base = mmDCORE2_TPC0_QM_BASE + index * DCORE_TPC_OFFSET; - snprintf(desc, ARRAY_SIZE(desc), "DCORE2_TPC%d_QM", index); break; case GAUDI2_EVENT_TPC18_QM ... GAUDI2_EVENT_TPC23_QM: index = event_type - GAUDI2_EVENT_TPC18_QM; qid_base = GAUDI2_QUEUE_ID_DCORE3_TPC_0_0 + index * QMAN_STREAMS; qman_base = mmDCORE3_TPC0_QM_BASE + index * DCORE_TPC_OFFSET; - snprintf(desc, ARRAY_SIZE(desc), "DCORE3_TPC%d_QM", index); break; case GAUDI2_EVENT_TPC24_QM: qid_base = GAUDI2_QUEUE_ID_DCORE0_TPC_6_0; qman_base = mmDCORE0_TPC6_QM_BASE; - snprintf(desc, ARRAY_SIZE(desc), "DCORE0_TPC6_QM"); break; case GAUDI2_EVENT_MME0_QM: qid_base = GAUDI2_QUEUE_ID_DCORE0_MME_0_0; qman_base = mmDCORE0_MME_QM_BASE; - snprintf(desc, ARRAY_SIZE(desc), "DCORE0_MME_QM"); break; case GAUDI2_EVENT_MME1_QM: qid_base = GAUDI2_QUEUE_ID_DCORE1_MME_0_0; qman_base = mmDCORE1_MME_QM_BASE; - snprintf(desc, ARRAY_SIZE(desc), "DCORE1_MME_QM"); break; case GAUDI2_EVENT_MME2_QM: qid_base = GAUDI2_QUEUE_ID_DCORE2_MME_0_0; qman_base = mmDCORE2_MME_QM_BASE; - snprintf(desc, ARRAY_SIZE(desc), "DCORE2_MME_QM"); break; case GAUDI2_EVENT_MME3_QM: qid_base = GAUDI2_QUEUE_ID_DCORE3_MME_0_0; qman_base = mmDCORE3_MME_QM_BASE; - snprintf(desc, ARRAY_SIZE(desc), "DCORE3_MME_QM"); break; case GAUDI2_EVENT_HDMA0_QM: + index = 0; qid_base = GAUDI2_QUEUE_ID_DCORE0_EDMA_0_0; qman_base = mmDCORE0_EDMA0_QM_BASE; - snprintf(desc, ARRAY_SIZE(desc), "DCORE0_EDMA0_QM"); break; case GAUDI2_EVENT_HDMA1_QM: + index = 1; qid_base = GAUDI2_QUEUE_ID_DCORE0_EDMA_1_0; qman_base = mmDCORE0_EDMA1_QM_BASE; - snprintf(desc, ARRAY_SIZE(desc), "DCORE0_EDMA1_QM"); break; case GAUDI2_EVENT_HDMA2_QM: + index = 2; qid_base = GAUDI2_QUEUE_ID_DCORE1_EDMA_0_0; qman_base = mmDCORE1_EDMA0_QM_BASE; - snprintf(desc, ARRAY_SIZE(desc), "DCORE1_EDMA0_QM"); break; case GAUDI2_EVENT_HDMA3_QM: + index = 3; qid_base = GAUDI2_QUEUE_ID_DCORE1_EDMA_1_0; qman_base = mmDCORE1_EDMA1_QM_BASE; - snprintf(desc, ARRAY_SIZE(desc), "DCORE1_EDMA1_QM"); break; case GAUDI2_EVENT_HDMA4_QM: + index = 4; qid_base = GAUDI2_QUEUE_ID_DCORE2_EDMA_0_0; qman_base = mmDCORE2_EDMA0_QM_BASE; - snprintf(desc, ARRAY_SIZE(desc), "DCORE2_EDMA0_QM"); break; case GAUDI2_EVENT_HDMA5_QM: + index = 5; qid_base = GAUDI2_QUEUE_ID_DCORE2_EDMA_1_0; qman_base = mmDCORE2_EDMA1_QM_BASE; - snprintf(desc, ARRAY_SIZE(desc), "DCORE2_EDMA1_QM"); break; case GAUDI2_EVENT_HDMA6_QM: + index = 6; qid_base = GAUDI2_QUEUE_ID_DCORE3_EDMA_0_0; qman_base = mmDCORE3_EDMA0_QM_BASE; - snprintf(desc, ARRAY_SIZE(desc), "DCORE3_EDMA0_QM"); break; case GAUDI2_EVENT_HDMA7_QM: + index = 7; qid_base = GAUDI2_QUEUE_ID_DCORE3_EDMA_1_0; qman_base = mmDCORE3_EDMA1_QM_BASE; - snprintf(desc, ARRAY_SIZE(desc), "DCORE3_EDMA1_QM"); break; case GAUDI2_EVENT_PDMA0_QM: qid_base = GAUDI2_QUEUE_ID_PDMA_0_0; qman_base = mmPDMA0_QM_BASE; - snprintf(desc, ARRAY_SIZE(desc), "PDMA0_QM"); break; case GAUDI2_EVENT_PDMA1_QM: qid_base = GAUDI2_QUEUE_ID_PDMA_1_0; qman_base = mmPDMA1_QM_BASE; - snprintf(desc, ARRAY_SIZE(desc), "PDMA1_QM"); break; case GAUDI2_EVENT_ROTATOR0_ROT0_QM: qid_base = GAUDI2_QUEUE_ID_ROT_0_0; qman_base = mmROT0_QM_BASE; - snprintf(desc, ARRAY_SIZE(desc), "ROTATOR0_QM"); break; case GAUDI2_EVENT_ROTATOR1_ROT1_QM: qid_base = GAUDI2_QUEUE_ID_ROT_1_0; qman_base = mmROT1_QM_BASE; - snprintf(desc, ARRAY_SIZE(desc), "ROTATOR1_QM"); break; default: - return; + return 0; } - gaudi2_handle_qman_err_generic(hdev, desc, qman_base, qid_base); + error_count = gaudi2_handle_qman_err_generic(hdev, event_type, qman_base, qid_base); /* Handle EDMA QM SEI here because there is no AXI error response event for EDMA */ - if (event_type >= GAUDI2_EVENT_HDMA2_QM && event_type <= GAUDI2_EVENT_HDMA5_QM) - _gaudi2_handle_qm_sei_err(hdev, qman_base); + if (event_type >= GAUDI2_EVENT_HDMA2_QM && event_type <= GAUDI2_EVENT_HDMA5_QM) { + error_count += _gaudi2_handle_qm_sei_err(hdev, qman_base, event_type); + gaudi2_ack_module_razwi_event_handler(hdev, RAZWI_EDMA, index, 0, event_mask); + } + + hl_check_for_glbl_errors(hdev); + + return error_count; } -static void gaudi2_handle_arc_farm_sei_err(struct hl_device *hdev) +static int gaudi2_handle_arc_farm_sei_err(struct hl_device *hdev, u16 event_type) { - u32 i, sts_val, sts_clr_val = 0; + u32 i, sts_val, sts_clr_val = 0, error_count = 0; sts_val = RREG32(mmARC_FARM_ARC0_AUX_ARC_SEI_INTR_STS); for (i = 0 ; i < GAUDI2_NUM_OF_ARC_SEI_ERR_CAUSE ; i++) { if (sts_val & BIT(i)) { - dev_err_ratelimited(hdev->dev, "ARC SEI. err cause: %s\n", - gaudi2_arc_sei_error_cause[i]); + gaudi2_print_event(hdev, event_type, true, + "err cause: %s", gaudi2_arc_sei_error_cause[i]); sts_clr_val |= BIT(i); + error_count++; } } + hl_check_for_glbl_errors(hdev); + WREG32(mmARC_FARM_ARC0_AUX_ARC_SEI_INTR_CLR, sts_clr_val); + + return error_count; } -static void gaudi2_handle_cpu_sei_err(struct hl_device *hdev) +static int gaudi2_handle_cpu_sei_err(struct hl_device *hdev, u16 event_type) { - u32 i, sts_val, sts_clr_val = 0; + u32 i, sts_val, sts_clr_val = 0, error_count = 0; sts_val = RREG32(mmCPU_IF_CPU_SEI_INTR_STS); for (i = 0 ; i < GAUDI2_NUM_OF_CPU_SEI_ERR_CAUSE ; i++) { if (sts_val & BIT(i)) { - dev_err_ratelimited(hdev->dev, "CPU SEI. err cause: %s\n", - gaudi2_cpu_sei_error_cause[i]); + gaudi2_print_event(hdev, event_type, true, + "err cause: %s", gaudi2_cpu_sei_error_cause[i]); sts_clr_val |= BIT(i); + error_count++; } } + hl_check_for_glbl_errors(hdev); + WREG32(mmCPU_IF_CPU_SEI_INTR_CLR, sts_clr_val); + + return error_count; } -static void gaudi2_handle_rot_err(struct hl_device *hdev, u8 rot_index, +static int gaudi2_handle_rot_err(struct hl_device *hdev, u8 rot_index, u16 event_type, struct hl_eq_razwi_with_intr_cause *razwi_with_intr_cause, u64 *event_mask) { u64 intr_cause_data = le64_to_cpu(razwi_with_intr_cause->intr_cause.intr_cause_data); + u32 error_count = 0; int i; for (i = 0 ; i < GAUDI2_NUM_OF_ROT_ERR_CAUSE ; i++) - if (intr_cause_data & BIT(i)) - dev_err_ratelimited(hdev->dev, "ROT%u. err cause: %s\n", - rot_index, guadi2_rot_error_cause[i]); + if (intr_cause_data & BIT(i)) { + gaudi2_print_event(hdev, event_type, true, + "err cause: %s", guadi2_rot_error_cause[i]); + error_count++; + } /* check if RAZWI happened */ - gaudi2_ack_module_razwi_event_handler(hdev, RAZWI_ROT, rot_index, 0, - &razwi_with_intr_cause->razwi_info, event_mask); + gaudi2_ack_module_razwi_event_handler(hdev, RAZWI_ROT, rot_index, 0, event_mask); + hl_check_for_glbl_errors(hdev); + + return error_count; } -static void gaudi2_tpc_ack_interrupts(struct hl_device *hdev, u8 tpc_index, char *interrupt_name, +static int gaudi2_tpc_ack_interrupts(struct hl_device *hdev, u8 tpc_index, u16 event_type, struct hl_eq_razwi_with_intr_cause *razwi_with_intr_cause, u64 *event_mask) { u64 intr_cause_data = le64_to_cpu(razwi_with_intr_cause->intr_cause.intr_cause_data); + u32 error_count = 0; int i; for (i = 0 ; i < GAUDI2_NUM_OF_TPC_INTR_CAUSE ; i++) - if (intr_cause_data & BIT(i)) - dev_err_ratelimited(hdev->dev, "TPC%d_%s interrupt cause: %s\n", - tpc_index, interrupt_name, gaudi2_tpc_interrupts_cause[i]); + if (intr_cause_data & BIT(i)) { + gaudi2_print_event(hdev, event_type, true, + "interrupt cause: %s", gaudi2_tpc_interrupts_cause[i]); + error_count++; + } /* check if RAZWI happened */ - gaudi2_ack_module_razwi_event_handler(hdev, RAZWI_TPC, tpc_index, 0, - &razwi_with_intr_cause->razwi_info, event_mask); + gaudi2_ack_module_razwi_event_handler(hdev, RAZWI_TPC, tpc_index, 0, event_mask); + hl_check_for_glbl_errors(hdev); + + return error_count; } -static void gaudi2_handle_dec_err(struct hl_device *hdev, u8 dec_index, const char *interrupt_name, - struct hl_eq_razwi_info *razwi_info, u64 *event_mask) +static int gaudi2_handle_dec_err(struct hl_device *hdev, u8 dec_index, u16 event_type, + u64 *event_mask) { - u32 sts_addr, sts_val, sts_clr_val = 0; + u32 sts_addr, sts_val, sts_clr_val = 0, error_count = 0; int i; if (dec_index < NUM_OF_VDEC_PER_DCORE * NUM_OF_DCORES) @@ -8039,24 +8200,27 @@ static void gaudi2_handle_dec_err(struct hl_device *hdev, u8 dec_index, const ch for (i = 0 ; i < GAUDI2_NUM_OF_DEC_ERR_CAUSE ; i++) { if (sts_val & BIT(i)) { - dev_err_ratelimited(hdev->dev, "DEC%u_%s err cause: %s\n", - dec_index, interrupt_name, gaudi2_dec_error_cause[i]); + gaudi2_print_event(hdev, event_type, true, + "err cause: %s", gaudi2_dec_error_cause[i]); sts_clr_val |= BIT(i); + error_count++; } } /* check if RAZWI happened */ - gaudi2_ack_module_razwi_event_handler(hdev, RAZWI_DEC, dec_index, 0, razwi_info, - event_mask); + gaudi2_ack_module_razwi_event_handler(hdev, RAZWI_DEC, dec_index, 0, event_mask); + hl_check_for_glbl_errors(hdev); /* Write 1 clear errors */ WREG32(sts_addr, sts_clr_val); + + return error_count; } -static void gaudi2_handle_mme_err(struct hl_device *hdev, u8 mme_index, const char *interrupt_name, - struct hl_eq_razwi_info *razwi_info, u64 *event_mask) +static int gaudi2_handle_mme_err(struct hl_device *hdev, u8 mme_index, u16 event_type, + u64 *event_mask) { - u32 sts_addr, sts_val, sts_clr_addr, sts_clr_val = 0; + u32 sts_addr, sts_val, sts_clr_addr, sts_clr_val = 0, error_count = 0; int i; sts_addr = mmDCORE0_MME_CTRL_LO_INTR_CAUSE + DCORE_OFFSET * mme_index; @@ -8066,35 +8230,45 @@ static void gaudi2_handle_mme_err(struct hl_device *hdev, u8 mme_index, const ch for (i = 0 ; i < GAUDI2_NUM_OF_MME_ERR_CAUSE ; i++) { if (sts_val & BIT(i)) { - dev_err_ratelimited(hdev->dev, "MME%u_%s err cause: %s\n", - mme_index, interrupt_name, guadi2_mme_error_cause[i]); + gaudi2_print_event(hdev, event_type, true, + "err cause: %s", guadi2_mme_error_cause[i]); sts_clr_val |= BIT(i); + error_count++; } } /* check if RAZWI happened */ for (i = MME_WRITE ; i < MME_INITIATORS_MAX ; i++) - gaudi2_ack_module_razwi_event_handler(hdev, RAZWI_MME, mme_index, i, razwi_info, - event_mask); + gaudi2_ack_module_razwi_event_handler(hdev, RAZWI_MME, mme_index, i, event_mask); + + hl_check_for_glbl_errors(hdev); WREG32(sts_clr_addr, sts_clr_val); + + return error_count; } -static void gaudi2_handle_mme_sbte_err(struct hl_device *hdev, u8 mme_index, u8 sbte_index, +static int gaudi2_handle_mme_sbte_err(struct hl_device *hdev, u16 event_type, u64 intr_cause_data) { - int i; + int i, error_count = 0; for (i = 0 ; i < GAUDI2_NUM_OF_MME_SBTE_ERR_CAUSE ; i++) - if (intr_cause_data & BIT(i)) - dev_err_ratelimited(hdev->dev, "MME%uSBTE%u_AXI_ERR_RSP err cause: %s\n", - mme_index, sbte_index, guadi2_mme_sbte_error_cause[i]); + if (intr_cause_data & BIT(i)) { + gaudi2_print_event(hdev, event_type, true, + "err cause: %s", guadi2_mme_sbte_error_cause[i]); + error_count++; + } + + hl_check_for_glbl_errors(hdev); + + return error_count; } -static void gaudi2_handle_mme_wap_err(struct hl_device *hdev, u8 mme_index, - struct hl_eq_razwi_info *razwi_info, u64 *event_mask) +static int gaudi2_handle_mme_wap_err(struct hl_device *hdev, u8 mme_index, u16 event_type, + u64 *event_mask) { - u32 sts_addr, sts_val, sts_clr_addr, sts_clr_val = 0; + u32 sts_addr, sts_val, sts_clr_addr, sts_clr_val = 0, error_count = 0; int i; sts_addr = mmDCORE0_MME_ACC_INTR_CAUSE + DCORE_OFFSET * mme_index; @@ -8104,24 +8278,27 @@ static void gaudi2_handle_mme_wap_err(struct hl_device *hdev, u8 mme_index, for (i = 0 ; i < GAUDI2_NUM_OF_MME_WAP_ERR_CAUSE ; i++) { if (sts_val & BIT(i)) { - dev_err_ratelimited(hdev->dev, - "MME%u_WAP_SOURCE_RESULT_INVALID err cause: %s\n", - mme_index, guadi2_mme_wap_error_cause[i]); + gaudi2_print_event(hdev, event_type, true, + "err cause: %s", guadi2_mme_wap_error_cause[i]); sts_clr_val |= BIT(i); + error_count++; } } /* check if RAZWI happened on WAP0/1 */ - gaudi2_ack_module_razwi_event_handler(hdev, RAZWI_MME, mme_index, MME_WAP0, razwi_info, - event_mask); - gaudi2_ack_module_razwi_event_handler(hdev, RAZWI_MME, mme_index, MME_WAP1, razwi_info, - event_mask); + gaudi2_ack_module_razwi_event_handler(hdev, RAZWI_MME, mme_index, MME_WAP0, event_mask); + gaudi2_ack_module_razwi_event_handler(hdev, RAZWI_MME, mme_index, MME_WAP1, event_mask); + hl_check_for_glbl_errors(hdev); WREG32(sts_clr_addr, sts_clr_val); + + return error_count; } -static void gaudi2_handle_kdma_core_event(struct hl_device *hdev, u64 intr_cause_data) +static int gaudi2_handle_kdma_core_event(struct hl_device *hdev, u16 event_type, + u64 intr_cause_data) { + u32 error_count = 0; int i; /* If an AXI read or write error is received, an error is reported and @@ -8130,19 +8307,33 @@ static void gaudi2_handle_kdma_core_event(struct hl_device *hdev, u64 intr_cause * the actual error caused by a LBW KDMA transaction. */ for (i = 0 ; i < GAUDI2_NUM_OF_DMA_CORE_INTR_CAUSE ; i++) - if (intr_cause_data & BIT(i)) - dev_err_ratelimited(hdev->dev, "kdma core err cause: %s\n", - gaudi2_kdma_core_interrupts_cause[i]); + if (intr_cause_data & BIT(i)) { + gaudi2_print_event(hdev, event_type, true, + "err cause: %s", gaudi2_kdma_core_interrupts_cause[i]); + error_count++; + } + + hl_check_for_glbl_errors(hdev); + + return error_count; } -static void gaudi2_handle_dma_core_event(struct hl_device *hdev, u64 intr_cause_data) +static int gaudi2_handle_dma_core_event(struct hl_device *hdev, u16 event_type, + u64 intr_cause_data) { + u32 error_count = 0; int i; for (i = 0 ; i < GAUDI2_NUM_OF_DMA_CORE_INTR_CAUSE ; i++) - if (intr_cause_data & BIT(i)) - dev_err_ratelimited(hdev->dev, "dma core err cause: %s\n", - gaudi2_dma_core_interrupts_cause[i]); + if (intr_cause_data & BIT(i)) { + gaudi2_print_event(hdev, event_type, true, + "err cause: %s", gaudi2_dma_core_interrupts_cause[i]); + error_count++; + } + + hl_check_for_glbl_errors(hdev); + + return error_count; } static void gaudi2_print_pcie_mstr_rr_mstr_if_razwi_info(struct hl_device *hdev, u64 *event_mask) @@ -8151,86 +8342,98 @@ static void gaudi2_print_pcie_mstr_rr_mstr_if_razwi_info(struct hl_device *hdev, razwi_happened_addr = mstr_if_base_addr + RR_SHRD_HBW_AW_RAZWI_HAPPENED; if (RREG32(razwi_happened_addr)) { - gaudi2_razwi_rr_hbw_shared_printf_info(hdev, mstr_if_base_addr, true, "PCIE", true, - NULL, GAUDI2_ENGINE_ID_PCIE, event_mask); + gaudi2_razwi_rr_hbw_shared_printf_info(hdev, mstr_if_base_addr, true, "PCIE", + GAUDI2_ENGINE_ID_PCIE, event_mask); WREG32(razwi_happened_addr, 0x1); } razwi_happened_addr = mstr_if_base_addr + RR_SHRD_HBW_AR_RAZWI_HAPPENED; if (RREG32(razwi_happened_addr)) { - gaudi2_razwi_rr_hbw_shared_printf_info(hdev, mstr_if_base_addr, false, "PCIE", true, - NULL, GAUDI2_ENGINE_ID_PCIE, event_mask); + gaudi2_razwi_rr_hbw_shared_printf_info(hdev, mstr_if_base_addr, false, "PCIE", + GAUDI2_ENGINE_ID_PCIE, event_mask); WREG32(razwi_happened_addr, 0x1); } razwi_happened_addr = mstr_if_base_addr + RR_SHRD_LBW_AW_RAZWI_HAPPENED; if (RREG32(razwi_happened_addr)) { - gaudi2_razwi_rr_lbw_shared_printf_info(hdev, mstr_if_base_addr, true, "PCIE", true, - NULL, GAUDI2_ENGINE_ID_PCIE, event_mask); + gaudi2_razwi_rr_lbw_shared_printf_info(hdev, mstr_if_base_addr, true, "PCIE", + GAUDI2_ENGINE_ID_PCIE, event_mask); WREG32(razwi_happened_addr, 0x1); } razwi_happened_addr = mstr_if_base_addr + RR_SHRD_LBW_AR_RAZWI_HAPPENED; if (RREG32(razwi_happened_addr)) { - gaudi2_razwi_rr_lbw_shared_printf_info(hdev, mstr_if_base_addr, false, "PCIE", true, - NULL, GAUDI2_ENGINE_ID_PCIE, event_mask); + gaudi2_razwi_rr_lbw_shared_printf_info(hdev, mstr_if_base_addr, false, "PCIE", + GAUDI2_ENGINE_ID_PCIE, event_mask); WREG32(razwi_happened_addr, 0x1); } } -static void gaudi2_print_pcie_addr_dec_info(struct hl_device *hdev, u64 intr_cause_data, - u64 *event_mask) +static int gaudi2_print_pcie_addr_dec_info(struct hl_device *hdev, u16 event_type, + u64 intr_cause_data, u64 *event_mask) { + u32 error_count = 0; int i; for (i = 0 ; i < GAUDI2_NUM_OF_PCIE_ADDR_DEC_ERR_CAUSE ; i++) { if (!(intr_cause_data & BIT_ULL(i))) continue; - dev_err_ratelimited(hdev->dev, "PCIE ADDR DEC Error: %s\n", - gaudi2_pcie_addr_dec_error_cause[i]); + gaudi2_print_event(hdev, event_type, true, + "err cause: %s", gaudi2_pcie_addr_dec_error_cause[i]); + error_count++; switch (intr_cause_data & BIT_ULL(i)) { case PCIE_WRAP_PCIE_IC_SEI_INTR_IND_AXI_LBW_ERR_INTR_MASK: + hl_check_for_glbl_errors(hdev); break; case PCIE_WRAP_PCIE_IC_SEI_INTR_IND_BAD_ACCESS_INTR_MASK: gaudi2_print_pcie_mstr_rr_mstr_if_razwi_info(hdev, event_mask); break; } } + + return error_count; } -static void gaudi2_handle_pif_fatal(struct hl_device *hdev, u64 intr_cause_data) +static int gaudi2_handle_pif_fatal(struct hl_device *hdev, u16 event_type, + u64 intr_cause_data) { + u32 error_count = 0; int i; for (i = 0 ; i < GAUDI2_NUM_OF_PMMU_FATAL_ERR_CAUSE ; i++) { - if (intr_cause_data & BIT_ULL(i)) - dev_err_ratelimited(hdev->dev, "PMMU PIF err cause: %s\n", - gaudi2_pmmu_fatal_interrupts_cause[i]); + if (intr_cause_data & BIT_ULL(i)) { + gaudi2_print_event(hdev, event_type, true, + "err cause: %s", gaudi2_pmmu_fatal_interrupts_cause[i]); + error_count++; + } } + + return error_count; } -static void gaudi2_handle_hif_fatal(struct hl_device *hdev, u16 event_type, u64 intr_cause_data) +static int gaudi2_handle_hif_fatal(struct hl_device *hdev, u16 event_type, u64 intr_cause_data) { - u32 dcore_id, hif_id; + u32 error_count = 0; int i; - dcore_id = (event_type - GAUDI2_EVENT_HIF0_FATAL) / 4; - hif_id = (event_type - GAUDI2_EVENT_HIF0_FATAL) % 4; - for (i = 0 ; i < GAUDI2_NUM_OF_HIF_FATAL_ERR_CAUSE ; i++) { - if (intr_cause_data & BIT_ULL(i)) - dev_err_ratelimited(hdev->dev, "DCORE%u_HIF%u: %s\n", dcore_id, hif_id, - gaudi2_hif_fatal_interrupts_cause[i]); + if (intr_cause_data & BIT_ULL(i)) { + gaudi2_print_event(hdev, event_type, true, + "err cause: %s", gaudi2_hif_fatal_interrupts_cause[i]); + error_count++; + } } + + return error_count; } static void gaudi2_handle_page_error(struct hl_device *hdev, u64 mmu_base, bool is_pmmu, u64 *event_mask) { - u32 valid, val; + u32 valid, val, axid_l, axid_h; u64 addr; valid = RREG32(mmu_base + MMU_OFFSET(mmDCORE0_HMMU0_MMU_ACCESS_PAGE_ERROR_VALID)); @@ -8243,8 +8446,11 @@ static void gaudi2_handle_page_error(struct hl_device *hdev, u64 mmu_base, bool addr <<= 32; addr |= RREG32(mmu_base + MMU_OFFSET(mmDCORE0_HMMU0_MMU_PAGE_ERROR_CAPTURE_VA)); - dev_err_ratelimited(hdev->dev, "%s page fault on va 0x%llx\n", - is_pmmu ? "PMMU" : "HMMU", addr); + axid_l = RREG32(mmu_base + MMU_OFFSET(mmDCORE0_HMMU0_MMU_PAGE_FAULT_ID_LSB)); + axid_h = RREG32(mmu_base + MMU_OFFSET(mmDCORE0_HMMU0_MMU_PAGE_FAULT_ID_MSB)); + + dev_err_ratelimited(hdev->dev, "%s page fault on va 0x%llx, transaction id 0x%llX\n", + is_pmmu ? "PMMU" : "HMMU", addr, ((u64)axid_h << 32) + axid_l); hl_handle_page_fault(hdev, addr, 0, is_pmmu, event_mask); WREG32(mmu_base + MMU_OFFSET(mmDCORE0_HMMU0_MMU_PAGE_ERROR_CAPTURE), 0); @@ -8270,18 +8476,18 @@ static void gaudi2_handle_access_error(struct hl_device *hdev, u64 mmu_base, boo WREG32(mmu_base + MMU_OFFSET(mmDCORE0_HMMU0_MMU_ACCESS_ERROR_CAPTURE), 0); } -static void gaudi2_handle_mmu_spi_sei_generic(struct hl_device *hdev, const char *mmu_name, +static int gaudi2_handle_mmu_spi_sei_generic(struct hl_device *hdev, u16 event_type, u64 mmu_base, bool is_pmmu, u64 *event_mask) { - u32 spi_sei_cause, interrupt_clr = 0x0; + u32 spi_sei_cause, interrupt_clr = 0x0, error_count = 0; int i; spi_sei_cause = RREG32(mmu_base + MMU_SPI_SEI_CAUSE_OFFSET); for (i = 0 ; i < GAUDI2_NUM_OF_MMU_SPI_SEI_CAUSE ; i++) { if (spi_sei_cause & BIT(i)) { - dev_err_ratelimited(hdev->dev, "%s SPI_SEI ERR. err cause: %s\n", - mmu_name, gaudi2_mmu_spi_sei[i].cause); + gaudi2_print_event(hdev, event_type, true, + "err cause: %s", gaudi2_mmu_spi_sei[i].cause); if (i == 0) gaudi2_handle_page_error(hdev, mmu_base, is_pmmu, event_mask); @@ -8290,6 +8496,8 @@ static void gaudi2_handle_mmu_spi_sei_generic(struct hl_device *hdev, const char if (gaudi2_mmu_spi_sei[i].clear_bit >= 0) interrupt_clr |= BIT(gaudi2_mmu_spi_sei[i].clear_bit); + + error_count++; } } @@ -8298,12 +8506,14 @@ static void gaudi2_handle_mmu_spi_sei_generic(struct hl_device *hdev, const char /* Clear interrupt */ WREG32(mmu_base + MMU_INTERRUPT_CLR_OFFSET, interrupt_clr); + + return error_count; } -static void gaudi2_handle_sm_err(struct hl_device *hdev, u8 sm_index) +static int gaudi2_handle_sm_err(struct hl_device *hdev, u16 event_type, u8 sm_index) { - u32 sei_cause_addr, sei_cause_val, sei_cause_cause, sei_cause_log; - u32 cq_intr_addr, cq_intr_val, cq_intr_queue_index; + u32 sei_cause_addr, sei_cause_val, sei_cause_cause, sei_cause_log, + cq_intr_addr, cq_intr_val, cq_intr_queue_index, error_count = 0; int i; sei_cause_addr = mmDCORE0_SYNC_MNGR_GLBL_SM_SEI_CAUSE + DCORE_OFFSET * sm_index; @@ -8323,11 +8533,12 @@ static void gaudi2_handle_sm_err(struct hl_device *hdev, u8 sm_index) if (!(sei_cause_cause & BIT(i))) continue; - dev_err_ratelimited(hdev->dev, "SM%u SEI ERR. err cause: %s. %s: 0x%X\n", - sm_index, - gaudi2_sm_sei_cause[i].cause_name, - gaudi2_sm_sei_cause[i].log_name, - sei_cause_log & gaudi2_sm_sei_cause[i].log_mask); + gaudi2_print_event(hdev, event_type, true, + "err cause: %s. %s: 0x%X\n", + gaudi2_sm_sei_cause[i].cause_name, + gaudi2_sm_sei_cause[i].log_name, + sei_cause_log); + error_count++; break; } @@ -8343,16 +8554,21 @@ static void gaudi2_handle_sm_err(struct hl_device *hdev, u8 sm_index) dev_err_ratelimited(hdev->dev, "SM%u err. err cause: CQ_INTR. queue index: %u\n", sm_index, cq_intr_queue_index); + error_count++; /* Clear CQ_INTR */ WREG32(cq_intr_addr, 0); } + + hl_check_for_glbl_errors(hdev); + + return error_count; } -static void gaudi2_handle_mmu_spi_sei_err(struct hl_device *hdev, u16 event_type, u64 *event_mask) +static int gaudi2_handle_mmu_spi_sei_err(struct hl_device *hdev, u16 event_type, u64 *event_mask) { bool is_pmmu = false; - char desc[32]; + u32 error_count = 0; u64 mmu_base; u8 index; @@ -8360,54 +8576,49 @@ static void gaudi2_handle_mmu_spi_sei_err(struct hl_device *hdev, u16 event_type case GAUDI2_EVENT_HMMU0_PAGE_FAULT_OR_WR_PERM ... GAUDI2_EVENT_HMMU3_SECURITY_ERROR: index = (event_type - GAUDI2_EVENT_HMMU0_PAGE_FAULT_OR_WR_PERM) / 3; mmu_base = mmDCORE0_HMMU0_MMU_BASE + index * DCORE_HMMU_OFFSET; - snprintf(desc, ARRAY_SIZE(desc), "DCORE0_HMMU%d", index); break; case GAUDI2_EVENT_HMMU_0_AXI_ERR_RSP ... GAUDI2_EVENT_HMMU_3_AXI_ERR_RSP: index = (event_type - GAUDI2_EVENT_HMMU_0_AXI_ERR_RSP); mmu_base = mmDCORE0_HMMU0_MMU_BASE + index * DCORE_HMMU_OFFSET; - snprintf(desc, ARRAY_SIZE(desc), "DCORE0_HMMU%d", index); break; case GAUDI2_EVENT_HMMU8_PAGE_FAULT_WR_PERM ... GAUDI2_EVENT_HMMU11_SECURITY_ERROR: index = (event_type - GAUDI2_EVENT_HMMU8_PAGE_FAULT_WR_PERM) / 3; mmu_base = mmDCORE1_HMMU0_MMU_BASE + index * DCORE_HMMU_OFFSET; - snprintf(desc, ARRAY_SIZE(desc), "DCORE1_HMMU%d", index); break; case GAUDI2_EVENT_HMMU_8_AXI_ERR_RSP ... GAUDI2_EVENT_HMMU_11_AXI_ERR_RSP: index = (event_type - GAUDI2_EVENT_HMMU_8_AXI_ERR_RSP); mmu_base = mmDCORE1_HMMU0_MMU_BASE + index * DCORE_HMMU_OFFSET; - snprintf(desc, ARRAY_SIZE(desc), "DCORE1_HMMU%d", index); break; case GAUDI2_EVENT_HMMU7_PAGE_FAULT_WR_PERM ... GAUDI2_EVENT_HMMU4_SECURITY_ERROR: index = (event_type - GAUDI2_EVENT_HMMU7_PAGE_FAULT_WR_PERM) / 3; mmu_base = mmDCORE2_HMMU0_MMU_BASE + index * DCORE_HMMU_OFFSET; - snprintf(desc, ARRAY_SIZE(desc), "DCORE2_HMMU%d", index); break; case GAUDI2_EVENT_HMMU_7_AXI_ERR_RSP ... GAUDI2_EVENT_HMMU_4_AXI_ERR_RSP: index = (event_type - GAUDI2_EVENT_HMMU_7_AXI_ERR_RSP); mmu_base = mmDCORE2_HMMU0_MMU_BASE + index * DCORE_HMMU_OFFSET; - snprintf(desc, ARRAY_SIZE(desc), "DCORE2_HMMU%d", index); break; case GAUDI2_EVENT_HMMU15_PAGE_FAULT_WR_PERM ... GAUDI2_EVENT_HMMU12_SECURITY_ERROR: index = (event_type - GAUDI2_EVENT_HMMU15_PAGE_FAULT_WR_PERM) / 3; mmu_base = mmDCORE3_HMMU0_MMU_BASE + index * DCORE_HMMU_OFFSET; - snprintf(desc, ARRAY_SIZE(desc), "DCORE3_HMMU%d", index); break; case GAUDI2_EVENT_HMMU_15_AXI_ERR_RSP ... GAUDI2_EVENT_HMMU_12_AXI_ERR_RSP: index = (event_type - GAUDI2_EVENT_HMMU_15_AXI_ERR_RSP); mmu_base = mmDCORE3_HMMU0_MMU_BASE + index * DCORE_HMMU_OFFSET; - snprintf(desc, ARRAY_SIZE(desc), "DCORE3_HMMU%d", index); break; case GAUDI2_EVENT_PMMU0_PAGE_FAULT_WR_PERM ... GAUDI2_EVENT_PMMU0_SECURITY_ERROR: case GAUDI2_EVENT_PMMU_AXI_ERR_RSP_0: is_pmmu = true; mmu_base = mmPMMU_HBW_MMU_BASE; - snprintf(desc, ARRAY_SIZE(desc), "PMMU"); break; default: - return; + return 0; } - gaudi2_handle_mmu_spi_sei_generic(hdev, desc, mmu_base, is_pmmu, event_mask); + error_count = gaudi2_handle_mmu_spi_sei_generic(hdev, event_type, mmu_base, + is_pmmu, event_mask); + hl_check_for_glbl_errors(hdev); + + return error_count; } @@ -8527,22 +8738,17 @@ static bool gaudi2_handle_hbm_mc_sei_err(struct hl_device *hdev, u16 event_type, cause_idx = sei_data->hdr.sei_cause; if (cause_idx > GAUDI2_NUM_OF_HBM_SEI_CAUSE - 1) { - dev_err_ratelimited(hdev->dev, "Invalid HBM SEI event cause (%d) provided by FW\n", - cause_idx); + gaudi2_print_event(hdev, event_type, true, + "err cause: %s", + "Invalid HBM SEI event cause (%d) provided by FW\n", cause_idx); return true; } - if (sei_data->hdr.is_critical) - dev_err(hdev->dev, - "System Critical Error Interrupt - HBM(%u) MC(%u) MC_CH(%u) MC_PC(%u). Error cause: %s\n", - hbm_id, mc_id, sei_data->hdr.mc_channel, sei_data->hdr.mc_pseudo_channel, - hbm_mc_sei_cause[cause_idx]); - - else - dev_err_ratelimited(hdev->dev, - "System Non-Critical Error Interrupt - HBM(%u) MC(%u) MC_CH(%u) MC_PC(%u). Error cause: %s\n", - hbm_id, mc_id, sei_data->hdr.mc_channel, sei_data->hdr.mc_pseudo_channel, - hbm_mc_sei_cause[cause_idx]); + gaudi2_print_event(hdev, event_type, !sei_data->hdr.is_critical, + "System %s Error Interrupt - HBM(%u) MC(%u) MC_CH(%u) MC_PC(%u). Error cause: %s\n", + sei_data->hdr.is_critical ? "Critical" : "Non-critical", + hbm_id, mc_id, sei_data->hdr.mc_channel, sei_data->hdr.mc_pseudo_channel, + hbm_mc_sei_cause[cause_idx]); /* Print error-specific info */ switch (cause_idx) { @@ -8586,24 +8792,33 @@ static bool gaudi2_handle_hbm_mc_sei_err(struct hl_device *hdev, u16 event_type, return require_hard_reset; } -static void gaudi2_handle_hbm_cattrip(struct hl_device *hdev, u64 intr_cause_data) +static int gaudi2_handle_hbm_cattrip(struct hl_device *hdev, u16 event_type, + u64 intr_cause_data) { - dev_err(hdev->dev, - "HBM catastrophic temperature error (CATTRIP) cause %#llx\n", - intr_cause_data); + if (intr_cause_data) { + gaudi2_print_event(hdev, event_type, true, + "temperature error cause: %#llx", intr_cause_data); + return 1; + } + + return 0; } -static void gaudi2_handle_hbm_mc_spi(struct hl_device *hdev, u64 intr_cause_data) +static int gaudi2_handle_hbm_mc_spi(struct hl_device *hdev, u64 intr_cause_data) { - u32 i; + u32 i, error_count = 0; for (i = 0 ; i < GAUDI2_NUM_OF_HBM_MC_SPI_CAUSE ; i++) - if (intr_cause_data & hbm_mc_spi[i].mask) + if (intr_cause_data & hbm_mc_spi[i].mask) { dev_dbg(hdev->dev, "HBM spi event: notification cause(%s)\n", hbm_mc_spi[i].cause); + error_count++; + } + + return error_count; } -static void gaudi2_print_clk_change_info(struct hl_device *hdev, u16 event_type) +static void gaudi2_print_clk_change_info(struct hl_device *hdev, u16 event_type, u64 *event_mask) { ktime_t zero_time = ktime_set(0, 0); @@ -8615,13 +8830,13 @@ static void gaudi2_print_clk_change_info(struct hl_device *hdev, u16 event_type) hdev->clk_throttling.aggregated_reason |= HL_CLK_THROTTLE_POWER; hdev->clk_throttling.timestamp[HL_CLK_THROTTLE_TYPE_POWER].start = ktime_get(); hdev->clk_throttling.timestamp[HL_CLK_THROTTLE_TYPE_POWER].end = zero_time; - dev_info_ratelimited(hdev->dev, "Clock throttling due to power consumption\n"); + dev_dbg_ratelimited(hdev->dev, "Clock throttling due to power consumption\n"); break; case GAUDI2_EVENT_CPU_FIX_POWER_ENV_E: hdev->clk_throttling.current_reason &= ~HL_CLK_THROTTLE_POWER; hdev->clk_throttling.timestamp[HL_CLK_THROTTLE_TYPE_POWER].end = ktime_get(); - dev_info_ratelimited(hdev->dev, "Power envelop is safe, back to optimal clock\n"); + dev_dbg_ratelimited(hdev->dev, "Power envelop is safe, back to optimal clock\n"); break; case GAUDI2_EVENT_CPU_FIX_THERMAL_ENV_S: @@ -8629,12 +8844,14 @@ static void gaudi2_print_clk_change_info(struct hl_device *hdev, u16 event_type) hdev->clk_throttling.aggregated_reason |= HL_CLK_THROTTLE_THERMAL; hdev->clk_throttling.timestamp[HL_CLK_THROTTLE_TYPE_THERMAL].start = ktime_get(); hdev->clk_throttling.timestamp[HL_CLK_THROTTLE_TYPE_THERMAL].end = zero_time; + *event_mask |= HL_NOTIFIER_EVENT_USER_ENGINE_ERR; dev_info_ratelimited(hdev->dev, "Clock throttling due to overheating\n"); break; case GAUDI2_EVENT_CPU_FIX_THERMAL_ENV_E: hdev->clk_throttling.current_reason &= ~HL_CLK_THROTTLE_THERMAL; hdev->clk_throttling.timestamp[HL_CLK_THROTTLE_TYPE_THERMAL].end = ktime_get(); + *event_mask |= HL_NOTIFIER_EVENT_USER_ENGINE_ERR; dev_info_ratelimited(hdev->dev, "Thermal envelop is safe, back to optimal clock\n"); break; @@ -8646,43 +8863,49 @@ static void gaudi2_print_clk_change_info(struct hl_device *hdev, u16 event_type) mutex_unlock(&hdev->clk_throttling.lock); } -static void gaudi2_print_out_of_sync_info(struct hl_device *hdev, +static void gaudi2_print_out_of_sync_info(struct hl_device *hdev, u16 event_type, struct cpucp_pkt_sync_err *sync_err) { struct hl_hw_queue *q = &hdev->kernel_queues[GAUDI2_QUEUE_ID_CPU_PQ]; - dev_err(hdev->dev, "Out of sync with FW, FW: pi=%u, ci=%u, LKD: pi=%u, ci=%d\n", - le32_to_cpu(sync_err->pi), le32_to_cpu(sync_err->ci), q->pi, atomic_read(&q->ci)); + gaudi2_print_event(hdev, event_type, false, + "FW: pi=%u, ci=%u, LKD: pi=%u, ci=%d\n", + le32_to_cpu(sync_err->pi), le32_to_cpu(sync_err->ci), + q->pi, atomic_read(&q->ci)); } -static void gaudi2_handle_pcie_p2p_msix(struct hl_device *hdev) +static int gaudi2_handle_pcie_p2p_msix(struct hl_device *hdev, u16 event_type) { - u32 p2p_intr, msix_gw_intr; + u32 p2p_intr, msix_gw_intr, error_count = 0; p2p_intr = RREG32(mmPCIE_WRAP_P2P_INTR); msix_gw_intr = RREG32(mmPCIE_WRAP_MSIX_GW_INTR); if (p2p_intr) { - dev_err_ratelimited(hdev->dev, + gaudi2_print_event(hdev, event_type, true, "pcie p2p transaction terminated due to security, req_id(0x%x)\n", RREG32(mmPCIE_WRAP_P2P_REQ_ID)); WREG32(mmPCIE_WRAP_P2P_INTR, 0x1); + error_count++; } if (msix_gw_intr) { - dev_err_ratelimited(hdev->dev, + gaudi2_print_event(hdev, event_type, true, "pcie msi-x gen denied due to vector num check failure, vec(0x%X)\n", RREG32(mmPCIE_WRAP_MSIX_GW_VEC)); WREG32(mmPCIE_WRAP_MSIX_GW_INTR, 0x1); + error_count++; } + + return error_count; } -static void gaudi2_handle_pcie_drain(struct hl_device *hdev, +static int gaudi2_handle_pcie_drain(struct hl_device *hdev, struct hl_eq_pcie_drain_ind_data *drain_data) { - u64 lbw_rd, lbw_wr, hbw_rd, hbw_wr, cause; + u64 lbw_rd, lbw_wr, hbw_rd, hbw_wr, cause, error_count = 0; cause = le64_to_cpu(drain_data->intr_cause.intr_cause_data); lbw_rd = le64_to_cpu(drain_data->drain_rd_addr_lbw); @@ -8690,39 +8913,52 @@ static void gaudi2_handle_pcie_drain(struct hl_device *hdev, hbw_rd = le64_to_cpu(drain_data->drain_rd_addr_hbw); hbw_wr = le64_to_cpu(drain_data->drain_wr_addr_hbw); - if (cause & BIT_ULL(0)) + if (cause & BIT_ULL(0)) { dev_err_ratelimited(hdev->dev, "PCIE AXI drain LBW completed, read_err %u, write_err %u\n", !!lbw_rd, !!lbw_wr); + error_count++; + } - if (cause & BIT_ULL(1)) + if (cause & BIT_ULL(1)) { dev_err_ratelimited(hdev->dev, "PCIE AXI drain HBW completed, raddr %#llx, waddr %#llx\n", hbw_rd, hbw_wr); + error_count++; + } + + return error_count; } -static void gaudi2_handle_psoc_drain(struct hl_device *hdev, u64 intr_cause_data) +static int gaudi2_handle_psoc_drain(struct hl_device *hdev, u64 intr_cause_data) { + u32 error_count = 0; int i; for (i = 0 ; i < GAUDI2_NUM_OF_AXI_DRAIN_ERR_CAUSE ; i++) { - if (intr_cause_data & BIT_ULL(i)) + if (intr_cause_data & BIT_ULL(i)) { dev_err_ratelimited(hdev->dev, "PSOC %s completed\n", gaudi2_psoc_axi_drain_interrupts_cause[i]); + error_count++; + } } + + hl_check_for_glbl_errors(hdev); + + return error_count; } -static void gaudi2_print_cpu_pkt_failure_info(struct hl_device *hdev, +static void gaudi2_print_cpu_pkt_failure_info(struct hl_device *hdev, u16 event_type, struct cpucp_pkt_sync_err *sync_err) { struct hl_hw_queue *q = &hdev->kernel_queues[GAUDI2_QUEUE_ID_CPU_PQ]; - dev_warn(hdev->dev, + gaudi2_print_event(hdev, event_type, false, "FW reported sanity check failure, FW: pi=%u, ci=%u, LKD: pi=%u, ci=%d\n", le32_to_cpu(sync_err->pi), le32_to_cpu(sync_err->ci), q->pi, atomic_read(&q->ci)); } -static void hl_arc_event_handle(struct hl_device *hdev, +static int hl_arc_event_handle(struct hl_device *hdev, u16 event_type, struct hl_eq_engine_arc_intr_data *data) { struct hl_engine_arc_dccm_queue_full_irq *q; @@ -8737,12 +8973,13 @@ static void hl_arc_event_handle(struct hl_device *hdev, case ENGINE_ARC_DCCM_QUEUE_FULL_IRQ: q = (struct hl_engine_arc_dccm_queue_full_irq *) &payload; - dev_err_ratelimited(hdev->dev, + gaudi2_print_event(hdev, event_type, true, "ARC DCCM Full event: EngId: %u, Intr_type: %u, Qidx: %u\n", engine_id, intr_type, q->queue_index); - break; + return 1; default: - dev_err_ratelimited(hdev->dev, "Unknown ARC event type\n"); + gaudi2_print_event(hdev, event_type, true, "Unknown ARC event type\n"); + return 0; } } @@ -8750,8 +8987,7 @@ static void gaudi2_handle_eqe(struct hl_device *hdev, struct hl_eq_entry *eq_ent { struct gaudi2_device *gaudi2 = hdev->asic_specific; bool reset_required = false, is_critical = false; - u32 ctl, reset_flags = HL_DRV_RESET_HARD; - int index, sbte_index; + u32 index, ctl, reset_flags = HL_DRV_RESET_HARD, error_count = 0; u64 event_mask = 0; u16 event_type; @@ -8767,8 +9003,6 @@ static void gaudi2_handle_eqe(struct hl_device *hdev, struct hl_eq_entry *eq_ent gaudi2->events_stat[event_type]++; gaudi2->events_stat_aggregate[event_type]++; - gaudi2_print_irq_info(hdev, event_type); - switch (event_type) { case GAUDI2_EVENT_PCIE_CORE_SERR ... GAUDI2_EVENT_ARC0_ECC_DERR: fallthrough; @@ -8777,6 +9011,7 @@ static void gaudi2_handle_eqe(struct hl_device *hdev, struct hl_eq_entry *eq_ent event_mask |= HL_NOTIFIER_EVENT_GENERAL_HW_ERR; reset_required = gaudi2_handle_ecc_event(hdev, event_type, &eq_entry->ecc_data); is_critical = eq_entry->ecc_data.is_critical; + error_count++; break; case GAUDI2_EVENT_TPC0_QM ... GAUDI2_EVENT_PDMA1_QM: @@ -8784,48 +9019,48 @@ static void gaudi2_handle_eqe(struct hl_device *hdev, struct hl_eq_entry *eq_ent case GAUDI2_EVENT_ROTATOR0_ROT0_QM ... GAUDI2_EVENT_ROTATOR1_ROT1_QM: fallthrough; case GAUDI2_EVENT_NIC0_QM0 ... GAUDI2_EVENT_NIC11_QM1: - gaudi2_handle_qman_err(hdev, event_type); + error_count = gaudi2_handle_qman_err(hdev, event_type, &event_mask); event_mask |= HL_NOTIFIER_EVENT_USER_ENGINE_ERR; break; case GAUDI2_EVENT_ARC_AXI_ERROR_RESPONSE_0: reset_flags |= HL_DRV_RESET_FW_FATAL_ERR; - gaudi2_handle_arc_farm_sei_err(hdev); + error_count = gaudi2_handle_arc_farm_sei_err(hdev, event_type); event_mask |= HL_NOTIFIER_EVENT_GENERAL_HW_ERR; break; case GAUDI2_EVENT_CPU_AXI_ERR_RSP: - gaudi2_handle_cpu_sei_err(hdev); + error_count = gaudi2_handle_cpu_sei_err(hdev, event_type); event_mask |= HL_NOTIFIER_EVENT_GENERAL_HW_ERR; break; case GAUDI2_EVENT_PDMA_CH0_AXI_ERR_RSP: case GAUDI2_EVENT_PDMA_CH1_AXI_ERR_RSP: reset_flags |= HL_DRV_RESET_FW_FATAL_ERR; - gaudi2_handle_qm_sei_err(hdev, event_type, &eq_entry->razwi_info, &event_mask); + error_count = gaudi2_handle_qm_sei_err(hdev, event_type, true, &event_mask); event_mask |= HL_NOTIFIER_EVENT_USER_ENGINE_ERR; break; case GAUDI2_EVENT_ROTATOR0_AXI_ERROR_RESPONSE: case GAUDI2_EVENT_ROTATOR1_AXI_ERROR_RESPONSE: index = event_type - GAUDI2_EVENT_ROTATOR0_AXI_ERROR_RESPONSE; - gaudi2_handle_rot_err(hdev, index, &eq_entry->razwi_with_intr_cause, &event_mask); - gaudi2_handle_qm_sei_err(hdev, event_type, NULL, &event_mask); + error_count = gaudi2_handle_rot_err(hdev, index, event_type, + &eq_entry->razwi_with_intr_cause, &event_mask); + error_count += gaudi2_handle_qm_sei_err(hdev, event_type, false, &event_mask); event_mask |= HL_NOTIFIER_EVENT_USER_ENGINE_ERR; break; case GAUDI2_EVENT_TPC0_AXI_ERR_RSP ... GAUDI2_EVENT_TPC24_AXI_ERR_RSP: index = event_type - GAUDI2_EVENT_TPC0_AXI_ERR_RSP; - gaudi2_tpc_ack_interrupts(hdev, index, "AXI_ERR_RSP", + error_count = gaudi2_tpc_ack_interrupts(hdev, index, event_type, &eq_entry->razwi_with_intr_cause, &event_mask); - gaudi2_handle_qm_sei_err(hdev, event_type, NULL, &event_mask); + error_count += gaudi2_handle_qm_sei_err(hdev, event_type, false, &event_mask); event_mask |= HL_NOTIFIER_EVENT_USER_ENGINE_ERR; break; case GAUDI2_EVENT_DEC0_AXI_ERR_RSPONSE ... GAUDI2_EVENT_DEC9_AXI_ERR_RSPONSE: index = event_type - GAUDI2_EVENT_DEC0_AXI_ERR_RSPONSE; - gaudi2_handle_dec_err(hdev, index, "AXI_ERR_RESPONSE", &eq_entry->razwi_info, - &event_mask); + error_count = gaudi2_handle_dec_err(hdev, index, event_type, &event_mask); event_mask |= HL_NOTIFIER_EVENT_USER_ENGINE_ERR; break; @@ -8856,8 +9091,8 @@ static void gaudi2_handle_eqe(struct hl_device *hdev, struct hl_eq_entry *eq_ent case GAUDI2_EVENT_TPC24_KERNEL_ERR: index = (event_type - GAUDI2_EVENT_TPC0_KERNEL_ERR) / (GAUDI2_EVENT_TPC1_KERNEL_ERR - GAUDI2_EVENT_TPC0_KERNEL_ERR); - gaudi2_tpc_ack_interrupts(hdev, index, "KRN_ERR", &eq_entry->razwi_with_intr_cause, - &event_mask); + error_count = gaudi2_tpc_ack_interrupts(hdev, index, event_type, + &eq_entry->razwi_with_intr_cause, &event_mask); event_mask |= HL_NOTIFIER_EVENT_USER_ENGINE_ERR; break; @@ -8873,7 +9108,7 @@ static void gaudi2_handle_eqe(struct hl_device *hdev, struct hl_eq_entry *eq_ent case GAUDI2_EVENT_DEC9_SPI: index = (event_type - GAUDI2_EVENT_DEC0_SPI) / (GAUDI2_EVENT_DEC1_SPI - GAUDI2_EVENT_DEC0_SPI); - gaudi2_handle_dec_err(hdev, index, "SPI", &eq_entry->razwi_info, &event_mask); + error_count = gaudi2_handle_dec_err(hdev, index, event_type, &event_mask); event_mask |= HL_NOTIFIER_EVENT_USER_ENGINE_ERR; break; @@ -8884,9 +9119,8 @@ static void gaudi2_handle_eqe(struct hl_device *hdev, struct hl_eq_entry *eq_ent index = (event_type - GAUDI2_EVENT_MME0_CTRL_AXI_ERROR_RESPONSE) / (GAUDI2_EVENT_MME1_CTRL_AXI_ERROR_RESPONSE - GAUDI2_EVENT_MME0_CTRL_AXI_ERROR_RESPONSE); - gaudi2_handle_mme_err(hdev, index, - "CTRL_AXI_ERROR_RESPONSE", &eq_entry->razwi_info, &event_mask); - gaudi2_handle_qm_sei_err(hdev, event_type, NULL, &event_mask); + error_count = gaudi2_handle_mme_err(hdev, index, event_type, &event_mask); + error_count += gaudi2_handle_qm_sei_err(hdev, event_type, false, &event_mask); event_mask |= HL_NOTIFIER_EVENT_USER_ENGINE_ERR; break; @@ -8897,8 +9131,7 @@ static void gaudi2_handle_eqe(struct hl_device *hdev, struct hl_eq_entry *eq_ent index = (event_type - GAUDI2_EVENT_MME0_QMAN_SW_ERROR) / (GAUDI2_EVENT_MME1_QMAN_SW_ERROR - GAUDI2_EVENT_MME0_QMAN_SW_ERROR); - gaudi2_handle_mme_err(hdev, index, "QMAN_SW_ERROR", &eq_entry->razwi_info, - &event_mask); + error_count = gaudi2_handle_mme_err(hdev, index, event_type, &event_mask); event_mask |= HL_NOTIFIER_EVENT_USER_ENGINE_ERR; break; @@ -8909,25 +9142,25 @@ static void gaudi2_handle_eqe(struct hl_device *hdev, struct hl_eq_entry *eq_ent index = (event_type - GAUDI2_EVENT_MME0_WAP_SOURCE_RESULT_INVALID) / (GAUDI2_EVENT_MME1_WAP_SOURCE_RESULT_INVALID - GAUDI2_EVENT_MME0_WAP_SOURCE_RESULT_INVALID); - gaudi2_handle_mme_wap_err(hdev, index, &eq_entry->razwi_info, &event_mask); + error_count = gaudi2_handle_mme_wap_err(hdev, index, event_type, &event_mask); event_mask |= HL_NOTIFIER_EVENT_USER_ENGINE_ERR; break; case GAUDI2_EVENT_KDMA_CH0_AXI_ERR_RSP: case GAUDI2_EVENT_KDMA0_CORE: - gaudi2_handle_kdma_core_event(hdev, + error_count = gaudi2_handle_kdma_core_event(hdev, event_type, le64_to_cpu(eq_entry->intr_cause.intr_cause_data)); event_mask |= HL_NOTIFIER_EVENT_GENERAL_HW_ERR; break; case GAUDI2_EVENT_HDMA2_CORE ... GAUDI2_EVENT_PDMA1_CORE: - gaudi2_handle_dma_core_event(hdev, + error_count = gaudi2_handle_dma_core_event(hdev, event_type, le64_to_cpu(eq_entry->intr_cause.intr_cause_data)); event_mask |= HL_NOTIFIER_EVENT_USER_ENGINE_ERR; break; case GAUDI2_EVENT_PCIE_ADDR_DEC_ERR: - gaudi2_print_pcie_addr_dec_info(hdev, + error_count = gaudi2_print_pcie_addr_dec_info(hdev, event_type, le64_to_cpu(eq_entry->intr_cause.intr_cause_data), &event_mask); reset_flags |= HL_DRV_RESET_FW_FATAL_ERR; event_mask |= HL_NOTIFIER_EVENT_GENERAL_HW_ERR; @@ -8937,27 +9170,27 @@ static void gaudi2_handle_eqe(struct hl_device *hdev, struct hl_eq_entry *eq_ent case GAUDI2_EVENT_HMMU_0_AXI_ERR_RSP ... GAUDI2_EVENT_HMMU_12_AXI_ERR_RSP: case GAUDI2_EVENT_PMMU0_PAGE_FAULT_WR_PERM ... GAUDI2_EVENT_PMMU0_SECURITY_ERROR: case GAUDI2_EVENT_PMMU_AXI_ERR_RSP_0: - gaudi2_handle_mmu_spi_sei_err(hdev, event_type, &event_mask); + error_count = gaudi2_handle_mmu_spi_sei_err(hdev, event_type, &event_mask); reset_flags |= HL_DRV_RESET_FW_FATAL_ERR; event_mask |= HL_NOTIFIER_EVENT_USER_ENGINE_ERR; break; case GAUDI2_EVENT_HIF0_FATAL ... GAUDI2_EVENT_HIF12_FATAL: - gaudi2_handle_hif_fatal(hdev, event_type, + error_count = gaudi2_handle_hif_fatal(hdev, event_type, le64_to_cpu(eq_entry->intr_cause.intr_cause_data)); reset_flags |= HL_DRV_RESET_FW_FATAL_ERR; event_mask |= HL_NOTIFIER_EVENT_GENERAL_HW_ERR; break; case GAUDI2_EVENT_PMMU_FATAL_0: - gaudi2_handle_pif_fatal(hdev, + error_count = gaudi2_handle_pif_fatal(hdev, event_type, le64_to_cpu(eq_entry->intr_cause.intr_cause_data)); reset_flags |= HL_DRV_RESET_FW_FATAL_ERR; event_mask |= HL_NOTIFIER_EVENT_GENERAL_HW_ERR; break; case GAUDI2_EVENT_PSOC63_RAZWI_OR_PID_MIN_MAX_INTERRUPT: - gaudi2_ack_psoc_razwi_event_handler(hdev, &event_mask); + error_count = gaudi2_ack_psoc_razwi_event_handler(hdev, &event_mask); event_mask |= HL_NOTIFIER_EVENT_USER_ENGINE_ERR; break; @@ -8967,33 +9200,39 @@ static void gaudi2_handle_eqe(struct hl_device *hdev, struct hl_eq_entry *eq_ent reset_flags |= HL_DRV_RESET_FW_FATAL_ERR; reset_required = true; } + error_count++; break; case GAUDI2_EVENT_HBM_CATTRIP_0 ... GAUDI2_EVENT_HBM_CATTRIP_5: - gaudi2_handle_hbm_cattrip(hdev, le64_to_cpu(eq_entry->intr_cause.intr_cause_data)); + error_count = gaudi2_handle_hbm_cattrip(hdev, event_type, + le64_to_cpu(eq_entry->intr_cause.intr_cause_data)); event_mask |= HL_NOTIFIER_EVENT_GENERAL_HW_ERR; break; case GAUDI2_EVENT_HBM0_MC0_SPI ... GAUDI2_EVENT_HBM5_MC1_SPI: - gaudi2_handle_hbm_mc_spi(hdev, le64_to_cpu(eq_entry->intr_cause.intr_cause_data)); + error_count = gaudi2_handle_hbm_mc_spi(hdev, + le64_to_cpu(eq_entry->intr_cause.intr_cause_data)); event_mask |= HL_NOTIFIER_EVENT_GENERAL_HW_ERR; break; case GAUDI2_EVENT_PCIE_DRAIN_COMPLETE: - gaudi2_handle_pcie_drain(hdev, &eq_entry->pcie_drain_ind_data); + error_count = gaudi2_handle_pcie_drain(hdev, &eq_entry->pcie_drain_ind_data); event_mask |= HL_NOTIFIER_EVENT_GENERAL_HW_ERR; break; case GAUDI2_EVENT_PSOC59_RPM_ERROR_OR_DRAIN: - gaudi2_handle_psoc_drain(hdev, le64_to_cpu(eq_entry->intr_cause.intr_cause_data)); + error_count = gaudi2_handle_psoc_drain(hdev, + le64_to_cpu(eq_entry->intr_cause.intr_cause_data)); event_mask |= HL_NOTIFIER_EVENT_GENERAL_HW_ERR; break; case GAUDI2_EVENT_CPU_AXI_ECC: + error_count = GAUDI2_NA_EVENT_CAUSE; reset_flags |= HL_DRV_RESET_FW_FATAL_ERR; event_mask |= HL_NOTIFIER_EVENT_GENERAL_HW_ERR; break; case GAUDI2_EVENT_CPU_L2_RAM_ECC: + error_count = GAUDI2_NA_EVENT_CAUSE; reset_flags |= HL_DRV_RESET_FW_FATAL_ERR; event_mask |= HL_NOTIFIER_EVENT_GENERAL_HW_ERR; break; @@ -9001,31 +9240,30 @@ static void gaudi2_handle_eqe(struct hl_device *hdev, struct hl_eq_entry *eq_ent case GAUDI2_EVENT_MME1_SBTE0_AXI_ERR_RSP ... GAUDI2_EVENT_MME1_SBTE4_AXI_ERR_RSP: case GAUDI2_EVENT_MME2_SBTE0_AXI_ERR_RSP ... GAUDI2_EVENT_MME2_SBTE4_AXI_ERR_RSP: case GAUDI2_EVENT_MME3_SBTE0_AXI_ERR_RSP ... GAUDI2_EVENT_MME3_SBTE4_AXI_ERR_RSP: - index = (event_type - GAUDI2_EVENT_MME0_SBTE0_AXI_ERR_RSP) / - (GAUDI2_EVENT_MME1_SBTE0_AXI_ERR_RSP - - GAUDI2_EVENT_MME0_SBTE0_AXI_ERR_RSP); - sbte_index = (event_type - GAUDI2_EVENT_MME0_SBTE0_AXI_ERR_RSP) % - (GAUDI2_EVENT_MME1_SBTE0_AXI_ERR_RSP - - GAUDI2_EVENT_MME0_SBTE0_AXI_ERR_RSP); - gaudi2_handle_mme_sbte_err(hdev, index, sbte_index, + error_count = gaudi2_handle_mme_sbte_err(hdev, event_type, le64_to_cpu(eq_entry->intr_cause.intr_cause_data)); event_mask |= HL_NOTIFIER_EVENT_USER_ENGINE_ERR; break; case GAUDI2_EVENT_VM0_ALARM_A ... GAUDI2_EVENT_VM3_ALARM_B: + error_count = GAUDI2_NA_EVENT_CAUSE; reset_flags |= HL_DRV_RESET_FW_FATAL_ERR; event_mask |= HL_NOTIFIER_EVENT_GENERAL_HW_ERR; break; case GAUDI2_EVENT_PSOC_AXI_ERR_RSP: + error_count = GAUDI2_NA_EVENT_CAUSE; event_mask |= HL_NOTIFIER_EVENT_GENERAL_HW_ERR; break; case GAUDI2_EVENT_PSOC_PRSTN_FALL: + error_count = GAUDI2_NA_EVENT_CAUSE; event_mask |= HL_NOTIFIER_EVENT_GENERAL_HW_ERR; break; case GAUDI2_EVENT_PCIE_APB_TIMEOUT: + error_count = GAUDI2_NA_EVENT_CAUSE; reset_flags |= HL_DRV_RESET_FW_FATAL_ERR; event_mask |= HL_NOTIFIER_EVENT_GENERAL_HW_ERR; break; case GAUDI2_EVENT_PCIE_FATAL_ERR: + error_count = GAUDI2_NA_EVENT_CAUSE; event_mask |= HL_NOTIFIER_EVENT_GENERAL_HW_ERR; break; case GAUDI2_EVENT_TPC0_BMON_SPMU: @@ -9078,6 +9316,7 @@ static void gaudi2_handle_eqe(struct hl_device *hdev, struct hl_eq_entry *eq_ent case GAUDI2_EVENT_DEC8_BMON_SPMU: case GAUDI2_EVENT_DEC9_BMON_SPMU: case GAUDI2_EVENT_ROTATOR0_BMON_SPMU ... GAUDI2_EVENT_SM3_BMON_SPMU: + error_count = GAUDI2_NA_EVENT_CAUSE; event_mask |= HL_NOTIFIER_EVENT_USER_ENGINE_ERR; break; @@ -9085,67 +9324,87 @@ static void gaudi2_handle_eqe(struct hl_device *hdev, struct hl_eq_entry *eq_ent case GAUDI2_EVENT_CPU_FIX_POWER_ENV_E: case GAUDI2_EVENT_CPU_FIX_THERMAL_ENV_S: case GAUDI2_EVENT_CPU_FIX_THERMAL_ENV_E: - gaudi2_print_clk_change_info(hdev, event_type); - event_mask |= HL_NOTIFIER_EVENT_USER_ENGINE_ERR; + gaudi2_print_clk_change_info(hdev, event_type, &event_mask); + error_count = GAUDI2_NA_EVENT_CAUSE; break; case GAUDI2_EVENT_CPU_PKT_QUEUE_OUT_SYNC: - gaudi2_print_out_of_sync_info(hdev, &eq_entry->pkt_sync_err); + gaudi2_print_out_of_sync_info(hdev, event_type, &eq_entry->pkt_sync_err); + error_count = GAUDI2_NA_EVENT_CAUSE; event_mask |= HL_NOTIFIER_EVENT_GENERAL_HW_ERR; break; case GAUDI2_EVENT_PCIE_FLR_REQUESTED: event_mask |= HL_NOTIFIER_EVENT_GENERAL_HW_ERR; + error_count = GAUDI2_NA_EVENT_CAUSE; /* Do nothing- FW will handle it */ break; case GAUDI2_EVENT_PCIE_P2P_MSIX: - gaudi2_handle_pcie_p2p_msix(hdev); + error_count = gaudi2_handle_pcie_p2p_msix(hdev, event_type); event_mask |= HL_NOTIFIER_EVENT_USER_ENGINE_ERR; break; case GAUDI2_EVENT_SM0_AXI_ERROR_RESPONSE ... GAUDI2_EVENT_SM3_AXI_ERROR_RESPONSE: index = event_type - GAUDI2_EVENT_SM0_AXI_ERROR_RESPONSE; - gaudi2_handle_sm_err(hdev, index); + error_count = gaudi2_handle_sm_err(hdev, event_type, index); event_mask |= HL_NOTIFIER_EVENT_USER_ENGINE_ERR; break; case GAUDI2_EVENT_PSOC_MME_PLL_LOCK_ERR ... GAUDI2_EVENT_DCORE2_HBM_PLL_LOCK_ERR: + error_count = GAUDI2_NA_EVENT_CAUSE; event_mask |= HL_NOTIFIER_EVENT_GENERAL_HW_ERR; break; case GAUDI2_EVENT_CPU_CPLD_SHUTDOWN_CAUSE: dev_info(hdev->dev, "CPLD shutdown cause, reset reason: 0x%llx\n", le64_to_cpu(eq_entry->data[0])); + error_count = GAUDI2_NA_EVENT_CAUSE; event_mask |= HL_NOTIFIER_EVENT_GENERAL_HW_ERR; break; case GAUDI2_EVENT_CPU_CPLD_SHUTDOWN_EVENT: dev_err(hdev->dev, "CPLD shutdown event, reset reason: 0x%llx\n", le64_to_cpu(eq_entry->data[0])); + error_count = GAUDI2_NA_EVENT_CAUSE; event_mask |= HL_NOTIFIER_EVENT_GENERAL_HW_ERR; break; case GAUDI2_EVENT_CPU_PKT_SANITY_FAILED: - gaudi2_print_cpu_pkt_failure_info(hdev, &eq_entry->pkt_sync_err); + gaudi2_print_cpu_pkt_failure_info(hdev, event_type, &eq_entry->pkt_sync_err); + error_count = GAUDI2_NA_EVENT_CAUSE; event_mask |= HL_NOTIFIER_EVENT_GENERAL_HW_ERR; break; case GAUDI2_EVENT_ARC_DCCM_FULL: - hl_arc_event_handle(hdev, &eq_entry->arc_data); + error_count = hl_arc_event_handle(hdev, event_type, &eq_entry->arc_data); event_mask |= HL_NOTIFIER_EVENT_USER_ENGINE_ERR; break; case GAUDI2_EVENT_CPU_FP32_NOT_SUPPORTED: + case GAUDI2_EVENT_DEV_RESET_REQ: event_mask |= HL_NOTIFIER_EVENT_GENERAL_HW_ERR; + error_count = GAUDI2_NA_EVENT_CAUSE; is_critical = true; break; default: - if (gaudi2_irq_map_table[event_type].valid) + if (gaudi2_irq_map_table[event_type].valid) { dev_err_ratelimited(hdev->dev, "Cannot find handler for event %d\n", event_type); + error_count = GAUDI2_NA_EVENT_CAUSE; + } } + /* Make sure to dump an error in case no error cause was printed so far. + * Note that although we have counted the errors, we use this number as + * a boolean. + */ + if (error_count == GAUDI2_NA_EVENT_CAUSE && !is_info_event(event_type)) + gaudi2_print_event(hdev, event_type, true, "%d", event_type); + else if (error_count == 0) + gaudi2_print_event(hdev, event_type, true, + "No error cause for H/W event %u\n", event_type); + if ((gaudi2_irq_map_table[event_type].reset || reset_required) && (hdev->hard_reset_on_fw_events || (hdev->asic_prop.fw_security_enabled && is_critical))) @@ -10466,6 +10725,8 @@ static const struct hl_asic_funcs gaudi2_funcs = { .set_dram_bar_base = gaudi2_set_hbm_bar_base, .set_engine_cores = gaudi2_set_engine_cores, .send_device_activity = gaudi2_send_device_activity, + .set_dram_properties = gaudi2_set_dram_properties, + .set_binning_masks = gaudi2_set_binning_masks, }; void gaudi2_set_asic_funcs(struct hl_device *hdev) diff --git a/drivers/misc/habanalabs/gaudi2/gaudi2P.h b/drivers/accel/habanalabs/gaudi2/gaudi2P.h similarity index 95% rename from drivers/misc/habanalabs/gaudi2/gaudi2P.h rename to drivers/accel/habanalabs/gaudi2/gaudi2P.h index b4383c199bbbab066f3e55b301e3eb80993bbd15..2687404d9d213299e8ac173647729f3fa3d2ea20 100644 --- a/drivers/misc/habanalabs/gaudi2/gaudi2P.h +++ b/drivers/accel/habanalabs/gaudi2/gaudi2P.h @@ -8,7 +8,7 @@ #ifndef GAUDI2P_H_ #define GAUDI2P_H_ -#include +#include #include "../common/habanalabs.h" #include "../include/common/hl_boot_if.h" #include "../include/gaudi2/gaudi2.h" @@ -240,6 +240,8 @@ #define GAUDI2_SOB_INCREMENT_BY_ONE (FIELD_PREP(DCORE0_SYNC_MNGR_OBJS_SOB_OBJ_VAL_MASK, 1) | \ FIELD_PREP(DCORE0_SYNC_MNGR_OBJS_SOB_OBJ_INC_MASK, 1)) +#define GAUDI2_NUM_OF_GLBL_ERR_CAUSE 8 + enum gaudi2_reserved_sob_id { GAUDI2_RESERVED_SOB_CS_COMPLETION_FIRST, GAUDI2_RESERVED_SOB_CS_COMPLETION_LAST = @@ -532,6 +534,41 @@ struct gaudi2_device { u32 num_of_valid_hw_events; }; +/* + * Types of the Gaudi2 IP blocks, used by special blocks iterator. + * Required for scenarios where only particular block types can be + * addressed (e.g., special PLDM images). + */ +enum gaudi2_block_types { + GAUDI2_BLOCK_TYPE_PLL, + GAUDI2_BLOCK_TYPE_RTR, + GAUDI2_BLOCK_TYPE_CPU, + GAUDI2_BLOCK_TYPE_HIF, + GAUDI2_BLOCK_TYPE_HBM, + GAUDI2_BLOCK_TYPE_NIC, + GAUDI2_BLOCK_TYPE_PCIE, + GAUDI2_BLOCK_TYPE_PCIE_PMA, + GAUDI2_BLOCK_TYPE_PDMA, + GAUDI2_BLOCK_TYPE_EDMA, + GAUDI2_BLOCK_TYPE_PMMU, + GAUDI2_BLOCK_TYPE_PSOC, + GAUDI2_BLOCK_TYPE_ROT, + GAUDI2_BLOCK_TYPE_ARC_FARM, + GAUDI2_BLOCK_TYPE_DEC, + GAUDI2_BLOCK_TYPE_MME, + GAUDI2_BLOCK_TYPE_EU_BIST, + GAUDI2_BLOCK_TYPE_SYNC_MNGR, + GAUDI2_BLOCK_TYPE_STLB, + GAUDI2_BLOCK_TYPE_TPC, + GAUDI2_BLOCK_TYPE_HMMU, + GAUDI2_BLOCK_TYPE_SRAM, + GAUDI2_BLOCK_TYPE_XBAR, + GAUDI2_BLOCK_TYPE_KDMA, + GAUDI2_BLOCK_TYPE_XDMA, + GAUDI2_BLOCK_TYPE_XFT, + GAUDI2_BLOCK_TYPE_MAX +}; + extern const u32 gaudi2_dma_core_blocks_bases[DMA_CORE_ID_SIZE]; extern const u32 gaudi2_qm_blocks_bases[GAUDI2_QUEUE_ID_SIZE]; extern const u32 gaudi2_mme_acc_blocks_bases[MME_ID_SIZE]; diff --git a/drivers/misc/habanalabs/gaudi2/gaudi2_coresight.c b/drivers/accel/habanalabs/gaudi2/gaudi2_coresight.c similarity index 99% rename from drivers/misc/habanalabs/gaudi2/gaudi2_coresight.c rename to drivers/accel/habanalabs/gaudi2/gaudi2_coresight.c index 56c6ab692482f975b277b1838f8d81212d5d4f29..1dfbe293ececf96c8456ead5499a3fdb553a890d 100644 --- a/drivers/misc/habanalabs/gaudi2/gaudi2_coresight.c +++ b/drivers/accel/habanalabs/gaudi2/gaudi2_coresight.c @@ -5,7 +5,7 @@ * All Rights Reserved. */ #include "gaudi2_coresight_regs.h" -#include +#include #define GAUDI2_PLDM_CORESIGHT_TIMEOUT_USEC (CORESIGHT_TIMEOUT_USEC * 2000) #define SPMU_MAX_COUNTERS 6 @@ -2376,10 +2376,10 @@ static int gaudi2_config_bmon(struct hl_device *hdev, struct hl_debug_params *pa WREG32(base_reg + mmBMON_ADDRH_S2_OFFSET, upper_32_bits(input->start_addr2)); WREG32(base_reg + mmBMON_ADDRL_E2_OFFSET, lower_32_bits(input->end_addr2)); WREG32(base_reg + mmBMON_ADDRH_E2_OFFSET, upper_32_bits(input->end_addr2)); - WREG32(base_reg + mmBMON_ADDRL_S3_OFFSET, lower_32_bits(input->start_addr2)); - WREG32(base_reg + mmBMON_ADDRH_S3_OFFSET, upper_32_bits(input->start_addr2)); - WREG32(base_reg + mmBMON_ADDRL_E3_OFFSET, lower_32_bits(input->end_addr2)); - WREG32(base_reg + mmBMON_ADDRH_E3_OFFSET, upper_32_bits(input->end_addr2)); + WREG32(base_reg + mmBMON_ADDRL_S3_OFFSET, lower_32_bits(input->start_addr3)); + WREG32(base_reg + mmBMON_ADDRH_S3_OFFSET, upper_32_bits(input->start_addr3)); + WREG32(base_reg + mmBMON_ADDRL_E3_OFFSET, lower_32_bits(input->end_addr3)); + WREG32(base_reg + mmBMON_ADDRH_E3_OFFSET, upper_32_bits(input->end_addr3)); WREG32(base_reg + mmBMON_IDL_OFFSET, 0x0); WREG32(base_reg + mmBMON_IDH_OFFSET, 0x0); diff --git a/drivers/misc/habanalabs/gaudi2/gaudi2_coresight_regs.h b/drivers/accel/habanalabs/gaudi2/gaudi2_coresight_regs.h similarity index 100% rename from drivers/misc/habanalabs/gaudi2/gaudi2_coresight_regs.h rename to drivers/accel/habanalabs/gaudi2/gaudi2_coresight_regs.h diff --git a/drivers/misc/habanalabs/gaudi2/gaudi2_masks.h b/drivers/accel/habanalabs/gaudi2/gaudi2_masks.h similarity index 100% rename from drivers/misc/habanalabs/gaudi2/gaudi2_masks.h rename to drivers/accel/habanalabs/gaudi2/gaudi2_masks.h diff --git a/drivers/misc/habanalabs/gaudi2/gaudi2_security.c b/drivers/accel/habanalabs/gaudi2/gaudi2_security.c similarity index 99% rename from drivers/misc/habanalabs/gaudi2/gaudi2_security.c rename to drivers/accel/habanalabs/gaudi2/gaudi2_security.c index 768c2f3dc9003946879fe91d712a1f85e1fedfc0..a212f82e660483615bd8fd3081b1ed822c73976b 100644 --- a/drivers/misc/habanalabs/gaudi2/gaudi2_security.c +++ b/drivers/accel/habanalabs/gaudi2/gaudi2_security.c @@ -1561,6 +1561,7 @@ static const u32 gaudi2_pb_dcr0_tpc0_unsecured_regs[] = { mmDCORE0_TPC0_CFG_LUT_FUNC128_BASE_ADDR_HI, mmDCORE0_TPC0_CFG_LUT_FUNC256_BASE_ADDR_LO, mmDCORE0_TPC0_CFG_LUT_FUNC256_BASE_ADDR_HI, + mmDCORE0_TPC0_CFG_KERNEL_KERNEL_CONFIG, mmDCORE0_TPC0_CFG_KERNEL_SRF_0, mmDCORE0_TPC0_CFG_KERNEL_SRF_1, mmDCORE0_TPC0_CFG_KERNEL_SRF_2, @@ -1666,6 +1667,10 @@ static const u32 gaudi2_pb_dcr0_sm_glbl[] = { mmDCORE0_SYNC_MNGR_GLBL_BASE, }; +static const u32 gaudi2_pb_dcr1_sm_glbl[] = { + mmDCORE1_SYNC_MNGR_GLBL_BASE, +}; + static const struct range gaudi2_pb_dcr0_sm_glbl_unsecured_regs[] = { {mmDCORE0_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_1, mmDCORE0_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_63}, {mmDCORE0_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_1, mmDCORE0_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_63}, @@ -1678,14 +1683,14 @@ static const struct range gaudi2_pb_dcr0_sm_glbl_unsecured_regs[] = { }; static const struct range gaudi2_pb_dcr_x_sm_glbl_unsecured_regs[] = { - {mmDCORE0_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_0, mmDCORE0_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_63}, - {mmDCORE0_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_0, mmDCORE0_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_63}, - {mmDCORE0_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_0, mmDCORE0_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_63}, - {mmDCORE0_SYNC_MNGR_GLBL_CQ_PI_0, mmDCORE0_SYNC_MNGR_GLBL_CQ_PI_63}, - {mmDCORE0_SYNC_MNGR_GLBL_LBW_ADDR_L_0, mmDCORE0_SYNC_MNGR_GLBL_LBW_ADDR_L_63}, - {mmDCORE0_SYNC_MNGR_GLBL_LBW_ADDR_H_0, mmDCORE0_SYNC_MNGR_GLBL_LBW_ADDR_H_63}, - {mmDCORE0_SYNC_MNGR_GLBL_LBW_DATA_0, mmDCORE0_SYNC_MNGR_GLBL_LBW_DATA_63}, - {mmDCORE0_SYNC_MNGR_GLBL_CQ_INC_MODE_0, mmDCORE0_SYNC_MNGR_GLBL_CQ_INC_MODE_63}, + {mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_0, mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_63}, + {mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_0, mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_63}, + {mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_0, mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_63}, + {mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_0, mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_63}, + {mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_0, mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_63}, + {mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_0, mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_63}, + {mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_0, mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_63}, + {mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_0, mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_63}, }; static const u32 gaudi2_pb_arc_sched[] = { @@ -3358,14 +3363,6 @@ static int gaudi2_init_protection_bits(struct hl_device *hdev) /* Sync Manager GLBL */ - /* Unsecure all CQ registers */ - rc |= hl_init_pb_ranges(hdev, NUM_OF_DCORES, DCORE_OFFSET, - HL_PB_SINGLE_INSTANCE, HL_PB_NA, - gaudi2_pb_dcr0_sm_glbl, - ARRAY_SIZE(gaudi2_pb_dcr0_sm_glbl), - gaudi2_pb_dcr_x_sm_glbl_unsecured_regs, - ARRAY_SIZE(gaudi2_pb_dcr_x_sm_glbl_unsecured_regs)); - /* Secure Dcore0 CQ0 registers */ rc |= hl_init_pb_ranges(hdev, HL_PB_SHARED, HL_PB_NA, HL_PB_SINGLE_INSTANCE, HL_PB_NA, @@ -3374,6 +3371,14 @@ static int gaudi2_init_protection_bits(struct hl_device *hdev) gaudi2_pb_dcr0_sm_glbl_unsecured_regs, ARRAY_SIZE(gaudi2_pb_dcr0_sm_glbl_unsecured_regs)); + /* Unsecure all other CQ registers */ + rc |= hl_init_pb_ranges(hdev, NUM_OF_DCORES - 1, DCORE_OFFSET, + HL_PB_SINGLE_INSTANCE, HL_PB_NA, + gaudi2_pb_dcr1_sm_glbl, + ARRAY_SIZE(gaudi2_pb_dcr1_sm_glbl), + gaudi2_pb_dcr_x_sm_glbl_unsecured_regs, + ARRAY_SIZE(gaudi2_pb_dcr_x_sm_glbl_unsecured_regs)); + /* PSOC. * Except for PSOC_GLOBAL_CONF, skip when security is enabled in F/W, because the blocks are * protected by privileged RR. diff --git a/drivers/misc/habanalabs/goya/Makefile b/drivers/accel/habanalabs/goya/Makefile similarity index 100% rename from drivers/misc/habanalabs/goya/Makefile rename to drivers/accel/habanalabs/goya/Makefile diff --git a/drivers/misc/habanalabs/goya/goya.c b/drivers/accel/habanalabs/goya/goya.c similarity index 99% rename from drivers/misc/habanalabs/goya/goya.c rename to drivers/accel/habanalabs/goya/goya.c index 0f083fcf81a6b11bd1a4f12cf614052ed6cd165a..2b135e856607da737187cea7d796f56de5b6413a 100644 --- a/drivers/misc/habanalabs/goya/goya.c +++ b/drivers/accel/habanalabs/goya/goya.c @@ -5420,6 +5420,16 @@ static int goya_scrub_device_dram(struct hl_device *hdev, u64 val) return -EOPNOTSUPP; } +static int goya_set_dram_properties(struct hl_device *hdev) +{ + return 0; +} + +static int goya_set_binning_masks(struct hl_device *hdev) +{ + return 0; +} + static int goya_send_device_activity(struct hl_device *hdev, bool open) { return 0; @@ -5518,6 +5528,8 @@ static const struct hl_asic_funcs goya_funcs = { .access_dev_mem = hl_access_dev_mem, .set_dram_bar_base = goya_set_ddr_bar_base, .send_device_activity = goya_send_device_activity, + .set_dram_properties = goya_set_dram_properties, + .set_binning_masks = goya_set_binning_masks, }; /* diff --git a/drivers/misc/habanalabs/goya/goyaP.h b/drivers/accel/habanalabs/goya/goyaP.h similarity index 99% rename from drivers/misc/habanalabs/goya/goyaP.h rename to drivers/accel/habanalabs/goya/goyaP.h index d6ec43d6f6b0d4fd79fcdca85cdb3dc5c7e89775..5df3d30b91fdd18252e28b04be329411e23256bf 100644 --- a/drivers/misc/habanalabs/goya/goyaP.h +++ b/drivers/accel/habanalabs/goya/goyaP.h @@ -8,7 +8,7 @@ #ifndef GOYAP_H_ #define GOYAP_H_ -#include +#include #include "../common/habanalabs.h" #include "../include/common/hl_boot_if.h" #include "../include/goya/goya_packets.h" diff --git a/drivers/misc/habanalabs/goya/goya_coresight.c b/drivers/accel/habanalabs/goya/goya_coresight.c similarity index 99% rename from drivers/misc/habanalabs/goya/goya_coresight.c rename to drivers/accel/habanalabs/goya/goya_coresight.c index 2c5133cfae6597bb75931e4e823f77c27c7a05a0..e7ac3046cfaae1196555be3c4f1968a9011a694c 100644 --- a/drivers/misc/habanalabs/goya/goya_coresight.c +++ b/drivers/accel/habanalabs/goya/goya_coresight.c @@ -10,7 +10,7 @@ #include "../include/goya/asic_reg/goya_regs.h" #include "../include/goya/asic_reg/goya_masks.h" -#include +#include #define GOYA_PLDM_CORESIGHT_TIMEOUT_USEC (CORESIGHT_TIMEOUT_USEC * 100) diff --git a/drivers/misc/habanalabs/goya/goya_hwmgr.c b/drivers/accel/habanalabs/goya/goya_hwmgr.c similarity index 100% rename from drivers/misc/habanalabs/goya/goya_hwmgr.c rename to drivers/accel/habanalabs/goya/goya_hwmgr.c diff --git a/drivers/misc/habanalabs/goya/goya_security.c b/drivers/accel/habanalabs/goya/goya_security.c similarity index 100% rename from drivers/misc/habanalabs/goya/goya_security.c rename to drivers/accel/habanalabs/goya/goya_security.c diff --git a/drivers/misc/habanalabs/include/common/cpucp_if.h b/drivers/accel/habanalabs/include/common/cpucp_if.h similarity index 94% rename from drivers/misc/habanalabs/include/common/cpucp_if.h rename to drivers/accel/habanalabs/include/common/cpucp_if.h index baa5aa43b6f490acfffb6458e8061e39bc300106..d713252a4f1334841d3b189f12f15ab89dc917a0 100644 --- a/drivers/misc/habanalabs/include/common/cpucp_if.h +++ b/drivers/accel/habanalabs/include/common/cpucp_if.h @@ -344,6 +344,16 @@ struct hl_eq_engine_arc_intr_data { __le64 pad[5]; }; +#define ADDR_DEC_ADDRESS_COUNT_MAX 4 + +/* Data structure specifies details of ADDR_DEC interrupt */ +struct hl_eq_addr_dec_intr_data { + struct hl_eq_intr_cause intr_cause; + __le64 addr[ADDR_DEC_ADDRESS_COUNT_MAX]; + __u8 addr_cnt; + __u8 pad[7]; +}; + struct hl_eq_entry { struct hl_eq_header hdr; union { @@ -358,6 +368,7 @@ struct hl_eq_entry { struct hl_eq_razwi_with_intr_cause razwi_with_intr_cause; struct hl_eq_hbm_sei_data sei_data; /* Gaudi2 HBM */ struct hl_eq_engine_arc_intr_data arc_data; + struct hl_eq_addr_dec_intr_data addr_dec; __le64 data[7]; }; }; @@ -643,6 +654,10 @@ enum pq_init_status { * data corruption in case of mismatched driver/FW versions. * Relevant only to Gaudi. * + * CPUCP_PACKET_GENERIC_PASSTHROUGH - + * Generic opcode for all firmware info that is only passed to host + * through the LKD, without getting parsed there. + * * CPUCP_PACKET_ACTIVE_STATUS_SET - * LKD sends FW indication whether device is free or in use, this indication is reported * also to the BMC. @@ -704,9 +719,12 @@ enum cpucp_packet_id { CPUCP_PACKET_RESERVED5, /* not used */ CPUCP_PACKET_RESERVED6, /* not used */ CPUCP_PACKET_RESERVED7, /* not used */ + CPUCP_PACKET_GENERIC_PASSTHROUGH, /* IOCTL */ CPUCP_PACKET_RESERVED8, /* not used */ - CPUCP_PACKET_RESERVED9, /* not used */ CPUCP_PACKET_ACTIVE_STATUS_SET, /* internal */ + CPUCP_PACKET_RESERVED9, /* not used */ + CPUCP_PACKET_RESERVED10, /* not used */ + CPUCP_PACKET_RESERVED11, /* not used */ CPUCP_PACKET_ID_MAX /* must be last */ }; @@ -727,6 +745,11 @@ enum cpucp_packet_id { #define CPUCP_PKT_RES_PLL_OUT3_SHIFT 48 #define CPUCP_PKT_RES_PLL_OUT3_MASK 0xFFFF000000000000ull +#define CPUCP_PKT_RES_EEPROM_OUT0_SHIFT 0 +#define CPUCP_PKT_RES_EEPROM_OUT0_MASK 0x000000000000FFFFull +#define CPUCP_PKT_RES_EEPROM_OUT1_SHIFT 16 +#define CPUCP_PKT_RES_EEPROM_OUT1_MASK 0x0000000000FF0000ull + #define CPUCP_PKT_VAL_PFC_IN1_SHIFT 0 #define CPUCP_PKT_VAL_PFC_IN1_MASK 0x0000000000000001ull #define CPUCP_PKT_VAL_PFC_IN2_SHIFT 1 @@ -805,8 +828,13 @@ struct cpucp_packet { __le32 nonce; }; - /* For NIC requests */ - __le32 port_index; + union { + /* For NIC requests */ + __le32 port_index; + + /* For Generic packet sub index */ + __le32 pkt_subidx; + }; }; struct cpucp_unmask_irq_arr_packet { @@ -881,7 +909,9 @@ enum cpucp_in_attributes { cpucp_in_max, cpucp_in_lowest = 6, cpucp_in_highest = 7, - cpucp_in_reset_history + cpucp_in_reset_history, + cpucp_in_intr_alarm_a, + cpucp_in_intr_alarm_b, }; enum cpucp_curr_attributes { @@ -976,6 +1006,11 @@ enum pll_index { IC_PLL = 16, MC_PLL = 17, EMMC_PLL = 18, + D2D_PLL = 19, + CS_PLL = 20, + C2C_PLL = 21, + NCH_PLL = 22, + C2M_PLL = 23, PLL_MAX }; @@ -1135,8 +1170,9 @@ enum cpucp_serdes_type { HLS1_SERDES_TYPE, HLS1H_SERDES_TYPE, HLS2_SERDES_TYPE, - UNKNOWN_SERDES_TYPE, - MAX_NUM_SERDES_TYPE = UNKNOWN_SERDES_TYPE + HLS2_TYPE_1_SERDES_TYPE, + MAX_NUM_SERDES_TYPE, /* number of types */ + UNKNOWN_SERDES_TYPE = 0xFFFF /* serdes_type is u16 */ }; struct cpucp_nic_info { @@ -1160,6 +1196,21 @@ struct page_discard_info { __le32 mmu_page_idx[PAGE_DISCARD_MAX]; }; +/* + * struct frac_val - fracture value represented by "integer.frac". + * @integer: the integer part of the fracture value; + * @frac: the fracture part of the fracture value. + */ +struct frac_val { + union { + struct { + __le16 integer; + __le16 frac; + }; + __le32 val; + }; +}; + /* * struct ser_val - the SER (symbol error rate) value is represented by "integer * 10 ^ -exp". * @integer: the integer part of the SER value; @@ -1183,8 +1234,12 @@ struct ser_val { * @pcs_link: has PCS link. * @phy_ready: is PHY ready. * @auto_neg: is Autoneg enabled. - * @timeout_retransmission_cnt: timeout retransmission events - * @high_ber_cnt: high ber events + * @timeout_retransmission_cnt: timeout retransmission events. + * @high_ber_cnt: high ber events. + * @pre_fec_ser: pre FEC SER value. + * @post_fec_ser: post FEC SER value. + * @throughput: measured throughput. + * @latency: measured latency. */ struct cpucp_nic_status { __le32 port; @@ -1200,6 +1255,10 @@ struct cpucp_nic_status { __u8 auto_neg; __le32 timeout_retransmission_cnt; __le32 high_ber_cnt; + struct ser_val pre_fec_ser; + struct ser_val post_fec_ser; + struct frac_val bandwidth; + struct frac_val lat; }; enum cpucp_hbm_row_replace_cause { @@ -1292,6 +1351,7 @@ struct cpucp_dev_info_signed { __u8 certificate[SEC_CERTIFICATE_BUF_SZ]; }; +#define DCORE_MON_REGS_SZ 512 /* * struct dcore_monitor_regs_data - DCORE monitor regs data. * the structure follows sync manager block layout. relevant only to Gaudi. @@ -1302,11 +1362,11 @@ struct cpucp_dev_info_signed { * @mon_status: array of monitor status. */ struct dcore_monitor_regs_data { - __le32 mon_pay_addrl[512]; - __le32 mon_pay_addrh[512]; - __le32 mon_pay_data[512]; - __le32 mon_arm[512]; - __le32 mon_status[512]; + __le32 mon_pay_addrl[DCORE_MON_REGS_SZ]; + __le32 mon_pay_addrh[DCORE_MON_REGS_SZ]; + __le32 mon_pay_data[DCORE_MON_REGS_SZ]; + __le32 mon_arm[DCORE_MON_REGS_SZ]; + __le32 mon_status[DCORE_MON_REGS_SZ]; }; /* contains SM data for each SYNC_MNGR (relevant only to Gaudi) */ @@ -1317,4 +1377,14 @@ struct cpucp_monitor_dump { struct dcore_monitor_regs_data sync_mngr_e_n; }; +/* + * The Type of the generic request (and other input arguments) will be fetched from user by reading + * from "pkt_subidx" field in struct cpucp_packet. + * + * HL_PASSTHROUGHT_VERSIONS - Fetch all firmware versions. + */ +enum hl_passthrough_type { + HL_PASSTHROUGH_VERSIONS, +}; + #endif /* CPUCP_IF_H */ diff --git a/drivers/misc/habanalabs/include/common/hl_boot_if.h b/drivers/accel/habanalabs/include/common/hl_boot_if.h similarity index 83% rename from drivers/misc/habanalabs/include/common/hl_boot_if.h rename to drivers/accel/habanalabs/include/common/hl_boot_if.h index e0ea51cc74756c050343864b384cefd8a7436b6c..2256add057c51493a55ccd4952737b1d23c41b9d 100644 --- a/drivers/misc/habanalabs/include/common/hl_boot_if.h +++ b/drivers/accel/habanalabs/include/common/hl_boot_if.h @@ -40,6 +40,19 @@ enum cpu_boot_err { CPU_BOOT_ERR_LAST = 64 /* we have 2 registers of 32 bits */ }; +/* + * Mask for fatal failures + * This mask contains all possible fatal failures, and a dynamic code + * will clear the non-relevant ones. + */ +#define CPU_BOOT_ERR_FATAL_MASK \ + ((1 << CPU_BOOT_ERR_DRAM_INIT_FAIL) | \ + (1 << CPU_BOOT_ERR_PLL_FAIL) | \ + (1 << CPU_BOOT_ERR_DEVICE_UNUSABLE_FAIL) | \ + (1 << CPU_BOOT_ERR_BINNING_FAIL) | \ + (1 << CPU_BOOT_ERR_DRAM_SKIPPED) | \ + (1 << CPU_BOOT_ERR_EEPROM_FAIL)) + /* * CPU error bits in BOOT_ERROR registers * @@ -439,7 +452,7 @@ struct cpu_dyn_regs { /* TODO: remove the desc magic after the code is updated to use message */ /* HCDM - Habana Communications Descriptor Magic */ #define HL_COMMS_DESC_MAGIC 0x4843444D -#define HL_COMMS_DESC_VER 1 +#define HL_COMMS_DESC_VER 3 /* HCMv - Habana Communications Message + header version */ #define HL_COMMS_MSG_MAGIC_VALUE 0x48434D00 @@ -450,8 +463,10 @@ struct cpu_dyn_regs { ((ver) & HL_COMMS_MSG_MAGIC_VER_MASK)) #define HL_COMMS_MSG_MAGIC_V0 HL_COMMS_DESC_MAGIC #define HL_COMMS_MSG_MAGIC_V1 HL_COMMS_MSG_MAGIC_VER(1) +#define HL_COMMS_MSG_MAGIC_V2 HL_COMMS_MSG_MAGIC_VER(2) +#define HL_COMMS_MSG_MAGIC_V3 HL_COMMS_MSG_MAGIC_VER(3) -#define HL_COMMS_MSG_MAGIC HL_COMMS_MSG_MAGIC_V1 +#define HL_COMMS_MSG_MAGIC HL_COMMS_MSG_MAGIC_V3 #define HL_COMMS_MSG_MAGIC_VALIDATE_MAGIC(magic) \ (((magic) & HL_COMMS_MSG_MAGIC_MASK) == \ @@ -474,22 +489,31 @@ enum comms_msg_type { /* * Binning information shared between LKD and FW - * @tpc_mask - TPC binning information + * @tpc_mask_l - TPC binning information lower 64 bit * @dec_mask - Decoder binning information - * @hbm_mask - HBM binning information + * @dram_mask - DRAM binning information * @edma_mask - EDMA binning information * @mme_mask_l - MME binning information lower 32 * @mme_mask_h - MME binning information upper 32 - * @reserved - reserved field for 64 bit alignment + * @rot_mask - Rotator binning information + * @xbar_mask - xBAR binning information + * @reserved - reserved field for future binning info w/o ABI change + * @tpc_mask_h - TPC binning information upper 64 bit + * @nic_mask - NIC binning information */ struct lkd_fw_binning_info { - __le64 tpc_mask; + __le64 tpc_mask_l; __le32 dec_mask; - __le32 hbm_mask; + __le32 dram_mask; __le32 edma_mask; __le32 mme_mask_l; __le32 mme_mask_h; - __le32 reserved; + __le32 rot_mask; + __le32 xbar_mask; + __le32 reserved0; + __le64 tpc_mask_h; + __le64 nic_mask; + __le32 reserved1[8]; }; /* TODO: remove this struct after the code is updated to use message */ @@ -512,6 +536,23 @@ struct comms_msg_header { __u8 reserved[4]; /* pad to 64 bit */ }; +enum lkd_fw_ascii_msg_lvls { + LKD_FW_ASCII_MSG_ERR = 0, + LKD_FW_ASCII_MSG_WRN = 1, + LKD_FW_ASCII_MSG_INF = 2, + LKD_FW_ASCII_MSG_DBG = 3, +}; + +#define LKD_FW_ASCII_MSG_MAX_LEN 128 +#define LKD_FW_ASCII_MSG_MAX 4 /* consider ABI when changing */ + +struct lkd_fw_ascii_msg { + __u8 valid; + __u8 msg_lvl; + __u8 reserved[6]; + char msg[LKD_FW_ASCII_MSG_MAX_LEN]; +}; + /* this is the main FW descriptor - consider ABI when changing */ struct lkd_fw_comms_desc { struct comms_desc_header header; @@ -521,6 +562,8 @@ struct lkd_fw_comms_desc { /* can be used for 1 more version w/o ABI change */ char reserved0[VERSION_MAX_LEN]; __le64 img_addr; /* address for next FW component load */ + struct lkd_fw_binning_info binning_info; + struct lkd_fw_ascii_msg ascii_msg[LKD_FW_ASCII_MSG_MAX]; }; enum comms_reset_cause { @@ -545,6 +588,8 @@ struct lkd_fw_comms_msg { char reserved0[VERSION_MAX_LEN]; /* address for next FW component load */ __le64 img_addr; + struct lkd_fw_binning_info binning_info; + struct lkd_fw_ascii_msg ascii_msg[LKD_FW_ASCII_MSG_MAX]; }; struct { __u8 reset_cause; @@ -552,7 +597,7 @@ struct lkd_fw_comms_msg { struct { __u8 fw_cfg_skip; /* 1 - skip, 0 - don't skip */ }; - struct lkd_fw_binning_info binning_info; + struct lkd_fw_binning_info binning_conf; }; }; @@ -699,4 +744,92 @@ struct comms_status { }; }; +/** + * HL_MODULES_MAX_NUM is determined by the size of modules_mask in struct + * hl_component_versions + */ +enum hl_modules { + HL_MODULES_BOOT_INFO = 0, + HL_MODULES_EEPROM, + HL_MODULES_FDT, + HL_MODULES_I2C, + HL_MODULES_LZ4, + HL_MODULES_MBEDTLS, + HL_MODULES_MAX_NUM = 16 +}; + +/** + * HL_COMPONENTS_MAX_NUM is determined by the size of components_mask in + * struct cpucp_versions + */ +enum hl_components { + HL_COMPONENTS_PID = 0, + HL_COMPONENTS_MGMT, + HL_COMPONENTS_PREBOOT, + HL_COMPONENTS_PPBOOT, + HL_COMPONENTS_ARMCP, + HL_COMPONENTS_CPLD, + HL_COMPONENTS_UBOOT, + HL_COMPONENTS_MAX_NUM = 16 +}; + +/** + * struct hl_component_versions - versions associated with hl component. + * @struct_size: size of all the struct (including dynamic size of modules). + * @modules_offset: offset of the modules field in this struct. + * @component: version of the component itself. + * @fw_os: Firmware OS Version. + * @modules_mask: i'th bit (from LSB) is a flag - on if module i in enum + * hl_modules is used. + * @modules_counter: number of set bits in modules_mask. + * @reserved: reserved for future use. + * @modules: versions of the component's modules. Elborated explanation in + * struct cpucp_versions. + */ +struct hl_component_versions { + __le16 struct_size; + __le16 modules_offset; + __u8 component[VERSION_MAX_LEN]; + __u8 fw_os[VERSION_MAX_LEN]; + __le16 modules_mask; + __u8 modules_counter; + __u8 reserved[1]; + __u8 modules[][VERSION_MAX_LEN]; +}; + +/** + * struct hl_fw_versions - all versions (fuse, cpucp's components with their + * modules) + * @struct_size: size of all the struct (including dynamic size of components). + * @components_offset: offset of the components field in this struct. + * @fuse: silicon production FUSE information. + * @components_mask: i'th bit (from LSB) is a flag - on if component i in enum + * hl_components is used. + * @components_counter: number of set bits in components_mask. + * @reserved: reserved for future use. + * @components: versions of hl components. Index i corresponds to the i'th bit + * that is *on* in components_mask. For example, if + * components_mask=0b101, then *components represents arcpid and + * *(hl_component_versions*)((char*)components + 1') represents + * preboot, where 1' = components[0].struct_size. + */ +struct hl_fw_versions { + __le16 struct_size; + __le16 components_offset; + __u8 fuse[VERSION_MAX_LEN]; + __le16 components_mask; + __u8 components_counter; + __u8 reserved[1]; + struct hl_component_versions components[]; +}; + +/* Max size of struct hl_component_versions */ +#define HL_COMPONENT_VERSIONS_MAX_SIZE \ + (sizeof(struct hl_component_versions) + HL_MODULES_MAX_NUM * \ + VERSION_MAX_LEN) + +/* Max size of struct hl_fw_versions */ +#define HL_FW_VERSIONS_MAX_SIZE (sizeof(struct hl_fw_versions) + \ + HL_COMPONENTS_MAX_NUM * HL_COMPONENT_VERSIONS_MAX_SIZE) + #endif /* HL_BOOT_IF_H */ diff --git a/drivers/misc/habanalabs/include/common/qman_if.h b/drivers/accel/habanalabs/include/common/qman_if.h similarity index 100% rename from drivers/misc/habanalabs/include/common/qman_if.h rename to drivers/accel/habanalabs/include/common/qman_if.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/cpu_if_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/cpu_if_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/cpu_if_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/cpu_if_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/dma0_core_masks.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/dma0_core_masks.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/dma0_core_masks.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/dma0_core_masks.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/dma0_core_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/dma0_core_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/dma0_core_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/dma0_core_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/dma0_qm_masks.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/dma0_qm_masks.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/dma0_qm_masks.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/dma0_qm_masks.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/dma0_qm_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/dma0_qm_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/dma0_qm_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/dma0_qm_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/dma1_core_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/dma1_core_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/dma1_core_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/dma1_core_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/dma1_qm_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/dma1_qm_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/dma1_qm_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/dma1_qm_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/dma2_core_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/dma2_core_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/dma2_core_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/dma2_core_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/dma2_qm_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/dma2_qm_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/dma2_qm_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/dma2_qm_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/dma3_core_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/dma3_core_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/dma3_core_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/dma3_core_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/dma3_qm_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/dma3_qm_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/dma3_qm_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/dma3_qm_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/dma4_core_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/dma4_core_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/dma4_core_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/dma4_core_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/dma4_qm_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/dma4_qm_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/dma4_qm_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/dma4_qm_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/dma5_core_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/dma5_core_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/dma5_core_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/dma5_core_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/dma5_qm_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/dma5_qm_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/dma5_qm_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/dma5_qm_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/dma6_core_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/dma6_core_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/dma6_core_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/dma6_core_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/dma6_qm_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/dma6_qm_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/dma6_qm_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/dma6_qm_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/dma7_core_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/dma7_core_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/dma7_core_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/dma7_core_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/dma7_qm_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/dma7_qm_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/dma7_qm_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/dma7_qm_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/dma_if_e_n_down_ch0_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/dma_if_e_n_down_ch0_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/dma_if_e_n_down_ch0_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/dma_if_e_n_down_ch0_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/dma_if_e_n_down_ch1_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/dma_if_e_n_down_ch1_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/dma_if_e_n_down_ch1_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/dma_if_e_n_down_ch1_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/dma_if_e_n_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/dma_if_e_n_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/dma_if_e_n_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/dma_if_e_n_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/dma_if_e_s_down_ch0_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/dma_if_e_s_down_ch0_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/dma_if_e_s_down_ch0_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/dma_if_e_s_down_ch0_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/dma_if_e_s_down_ch1_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/dma_if_e_s_down_ch1_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/dma_if_e_s_down_ch1_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/dma_if_e_s_down_ch1_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/dma_if_e_s_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/dma_if_e_s_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/dma_if_e_s_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/dma_if_e_s_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/dma_if_w_n_down_ch0_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/dma_if_w_n_down_ch0_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/dma_if_w_n_down_ch0_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/dma_if_w_n_down_ch0_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/dma_if_w_n_down_ch1_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/dma_if_w_n_down_ch1_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/dma_if_w_n_down_ch1_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/dma_if_w_n_down_ch1_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/dma_if_w_n_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/dma_if_w_n_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/dma_if_w_n_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/dma_if_w_n_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/dma_if_w_s_down_ch0_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/dma_if_w_s_down_ch0_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/dma_if_w_s_down_ch0_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/dma_if_w_s_down_ch0_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/dma_if_w_s_down_ch1_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/dma_if_w_s_down_ch1_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/dma_if_w_s_down_ch1_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/dma_if_w_s_down_ch1_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/dma_if_w_s_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/dma_if_w_s_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/dma_if_w_s_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/dma_if_w_s_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/gaudi_blocks.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/gaudi_blocks.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/gaudi_blocks.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/gaudi_blocks.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/gaudi_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/gaudi_regs.h similarity index 99% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/gaudi_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/gaudi_regs.h index 1a6576666794538e2e750c2643418d15c4bdb851..23ee8691db4615f5ccb82be3062dde3a901d8980 100644 --- a/drivers/misc/habanalabs/include/gaudi/asic_reg/gaudi_regs.h +++ b/drivers/accel/habanalabs/include/gaudi/asic_reg/gaudi_regs.h @@ -320,4 +320,6 @@ #define mmPSOC_TPC_PLL_NR 0xC73100 #define mmIF_W_PLL_NR 0x488100 +#define mmPCIE_WRAP_RR_ELBI_RD_SEC_REG_CTRL 0xC01208 + #endif /* ASIC_REG_GAUDI_REGS_H_ */ diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/mme0_ctrl_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/mme0_ctrl_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/mme0_ctrl_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/mme0_ctrl_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/mme0_qm_masks.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/mme0_qm_masks.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/mme0_qm_masks.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/mme0_qm_masks.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/mme0_qm_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/mme0_qm_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/mme0_qm_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/mme0_qm_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/mme1_ctrl_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/mme1_ctrl_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/mme1_ctrl_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/mme1_ctrl_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/mme2_ctrl_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/mme2_ctrl_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/mme2_ctrl_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/mme2_ctrl_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/mme2_qm_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/mme2_qm_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/mme2_qm_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/mme2_qm_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/mme3_ctrl_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/mme3_ctrl_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/mme3_ctrl_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/mme3_ctrl_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/mmu_up_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/mmu_up_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/mmu_up_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/mmu_up_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/nic0_qm0_masks.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/nic0_qm0_masks.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/nic0_qm0_masks.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/nic0_qm0_masks.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/nic0_qm0_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/nic0_qm0_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/nic0_qm0_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/nic0_qm0_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/nic0_qm1_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/nic0_qm1_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/nic0_qm1_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/nic0_qm1_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/nic1_qm0_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/nic1_qm0_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/nic1_qm0_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/nic1_qm0_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/nic1_qm1_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/nic1_qm1_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/nic1_qm1_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/nic1_qm1_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/nic2_qm0_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/nic2_qm0_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/nic2_qm0_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/nic2_qm0_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/nic2_qm1_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/nic2_qm1_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/nic2_qm1_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/nic2_qm1_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/nic3_qm0_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/nic3_qm0_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/nic3_qm0_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/nic3_qm0_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/nic3_qm1_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/nic3_qm1_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/nic3_qm1_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/nic3_qm1_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/nic4_qm0_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/nic4_qm0_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/nic4_qm0_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/nic4_qm0_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/nic4_qm1_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/nic4_qm1_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/nic4_qm1_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/nic4_qm1_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/nif_rtr_ctrl_0_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/nif_rtr_ctrl_0_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/nif_rtr_ctrl_0_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/nif_rtr_ctrl_0_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/nif_rtr_ctrl_1_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/nif_rtr_ctrl_1_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/nif_rtr_ctrl_1_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/nif_rtr_ctrl_1_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/nif_rtr_ctrl_2_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/nif_rtr_ctrl_2_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/nif_rtr_ctrl_2_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/nif_rtr_ctrl_2_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/nif_rtr_ctrl_3_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/nif_rtr_ctrl_3_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/nif_rtr_ctrl_3_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/nif_rtr_ctrl_3_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/nif_rtr_ctrl_4_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/nif_rtr_ctrl_4_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/nif_rtr_ctrl_4_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/nif_rtr_ctrl_4_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/nif_rtr_ctrl_5_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/nif_rtr_ctrl_5_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/nif_rtr_ctrl_5_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/nif_rtr_ctrl_5_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/nif_rtr_ctrl_6_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/nif_rtr_ctrl_6_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/nif_rtr_ctrl_6_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/nif_rtr_ctrl_6_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/nif_rtr_ctrl_7_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/nif_rtr_ctrl_7_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/nif_rtr_ctrl_7_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/nif_rtr_ctrl_7_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/psoc_cpu_pll_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/psoc_cpu_pll_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/psoc_cpu_pll_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/psoc_cpu_pll_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/psoc_etr_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/psoc_etr_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/psoc_etr_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/psoc_etr_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/psoc_global_conf_masks.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/psoc_global_conf_masks.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/psoc_global_conf_masks.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/psoc_global_conf_masks.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/psoc_global_conf_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/psoc_global_conf_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/psoc_global_conf_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/psoc_global_conf_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/psoc_timestamp_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/psoc_timestamp_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/psoc_timestamp_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/psoc_timestamp_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/sif_rtr_ctrl_0_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/sif_rtr_ctrl_0_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/sif_rtr_ctrl_0_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/sif_rtr_ctrl_0_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/sif_rtr_ctrl_1_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/sif_rtr_ctrl_1_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/sif_rtr_ctrl_1_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/sif_rtr_ctrl_1_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/sif_rtr_ctrl_2_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/sif_rtr_ctrl_2_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/sif_rtr_ctrl_2_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/sif_rtr_ctrl_2_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/sif_rtr_ctrl_3_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/sif_rtr_ctrl_3_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/sif_rtr_ctrl_3_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/sif_rtr_ctrl_3_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/sif_rtr_ctrl_4_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/sif_rtr_ctrl_4_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/sif_rtr_ctrl_4_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/sif_rtr_ctrl_4_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/sif_rtr_ctrl_5_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/sif_rtr_ctrl_5_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/sif_rtr_ctrl_5_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/sif_rtr_ctrl_5_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/sif_rtr_ctrl_6_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/sif_rtr_ctrl_6_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/sif_rtr_ctrl_6_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/sif_rtr_ctrl_6_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/sif_rtr_ctrl_7_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/sif_rtr_ctrl_7_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/sif_rtr_ctrl_7_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/sif_rtr_ctrl_7_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/stlb_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/stlb_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/stlb_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/stlb_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/tpc0_cfg_masks.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/tpc0_cfg_masks.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/tpc0_cfg_masks.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/tpc0_cfg_masks.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/tpc0_cfg_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/tpc0_cfg_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/tpc0_cfg_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/tpc0_cfg_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/tpc0_qm_masks.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/tpc0_qm_masks.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/tpc0_qm_masks.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/tpc0_qm_masks.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/tpc0_qm_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/tpc0_qm_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/tpc0_qm_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/tpc0_qm_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/tpc1_cfg_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/tpc1_cfg_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/tpc1_cfg_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/tpc1_cfg_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/tpc1_qm_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/tpc1_qm_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/tpc1_qm_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/tpc1_qm_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/tpc2_cfg_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/tpc2_cfg_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/tpc2_cfg_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/tpc2_cfg_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/tpc2_qm_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/tpc2_qm_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/tpc2_qm_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/tpc2_qm_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/tpc3_cfg_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/tpc3_cfg_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/tpc3_cfg_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/tpc3_cfg_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/tpc3_qm_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/tpc3_qm_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/tpc3_qm_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/tpc3_qm_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/tpc4_cfg_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/tpc4_cfg_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/tpc4_cfg_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/tpc4_cfg_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/tpc4_qm_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/tpc4_qm_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/tpc4_qm_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/tpc4_qm_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/tpc5_cfg_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/tpc5_cfg_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/tpc5_cfg_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/tpc5_cfg_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/tpc5_qm_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/tpc5_qm_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/tpc5_qm_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/tpc5_qm_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/tpc6_cfg_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/tpc6_cfg_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/tpc6_cfg_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/tpc6_cfg_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/tpc6_qm_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/tpc6_qm_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/tpc6_qm_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/tpc6_qm_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/tpc7_cfg_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/tpc7_cfg_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/tpc7_cfg_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/tpc7_cfg_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/asic_reg/tpc7_qm_regs.h b/drivers/accel/habanalabs/include/gaudi/asic_reg/tpc7_qm_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/asic_reg/tpc7_qm_regs.h rename to drivers/accel/habanalabs/include/gaudi/asic_reg/tpc7_qm_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi/gaudi.h b/drivers/accel/habanalabs/include/gaudi/gaudi.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/gaudi.h rename to drivers/accel/habanalabs/include/gaudi/gaudi.h diff --git a/drivers/misc/habanalabs/include/gaudi/gaudi_async_events.h b/drivers/accel/habanalabs/include/gaudi/gaudi_async_events.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/gaudi_async_events.h rename to drivers/accel/habanalabs/include/gaudi/gaudi_async_events.h diff --git a/drivers/misc/habanalabs/include/gaudi/gaudi_async_ids_map_extended.h b/drivers/accel/habanalabs/include/gaudi/gaudi_async_ids_map_extended.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/gaudi_async_ids_map_extended.h rename to drivers/accel/habanalabs/include/gaudi/gaudi_async_ids_map_extended.h diff --git a/drivers/misc/habanalabs/include/gaudi/gaudi_coresight.h b/drivers/accel/habanalabs/include/gaudi/gaudi_coresight.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/gaudi_coresight.h rename to drivers/accel/habanalabs/include/gaudi/gaudi_coresight.h diff --git a/drivers/misc/habanalabs/include/gaudi/gaudi_fw_if.h b/drivers/accel/habanalabs/include/gaudi/gaudi_fw_if.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/gaudi_fw_if.h rename to drivers/accel/habanalabs/include/gaudi/gaudi_fw_if.h diff --git a/drivers/misc/habanalabs/include/gaudi/gaudi_masks.h b/drivers/accel/habanalabs/include/gaudi/gaudi_masks.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/gaudi_masks.h rename to drivers/accel/habanalabs/include/gaudi/gaudi_masks.h diff --git a/drivers/misc/habanalabs/include/gaudi/gaudi_packets.h b/drivers/accel/habanalabs/include/gaudi/gaudi_packets.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/gaudi_packets.h rename to drivers/accel/habanalabs/include/gaudi/gaudi_packets.h diff --git a/drivers/misc/habanalabs/include/gaudi/gaudi_reg_map.h b/drivers/accel/habanalabs/include/gaudi/gaudi_reg_map.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi/gaudi_reg_map.h rename to drivers/accel/habanalabs/include/gaudi/gaudi_reg_map.h diff --git a/drivers/accel/habanalabs/include/gaudi2/arc/gaudi2_arc_common_packets.h b/drivers/accel/habanalabs/include/gaudi2/arc/gaudi2_arc_common_packets.h new file mode 100644 index 0000000000000000000000000000000000000000..22a6ab9a7f47a9c351c30e7e69d67e7c18ed364a --- /dev/null +++ b/drivers/accel/habanalabs/include/gaudi2/arc/gaudi2_arc_common_packets.h @@ -0,0 +1,211 @@ +/* SPDX-License-Identifier: GPL-2.0 + * + * Copyright (C) 2020 HabanaLabs Ltd. + * All Rights Reserved. + */ + +#ifndef __GAUDI2_ARC_COMMON_PACKETS_H__ +#define __GAUDI2_ARC_COMMON_PACKETS_H__ + +enum { + CPU_ID_SCHED_ARC0 = 0, /* FARM_ARC0 */ + CPU_ID_SCHED_ARC1 = 1, /* FARM_ARC1 */ + CPU_ID_SCHED_ARC2 = 2, /* FARM_ARC2 */ + CPU_ID_SCHED_ARC3 = 3, /* FARM_ARC3 */ + /* Dcore1 MME Engine ARC instance used as scheduler */ + CPU_ID_SCHED_ARC4 = 4, /* DCORE1_MME0 */ + /* Dcore3 MME Engine ARC instance used as scheduler */ + CPU_ID_SCHED_ARC5 = 5, /* DCORE3_MME0 */ + + CPU_ID_TPC_QMAN_ARC0 = 6, /* DCORE0_TPC0 */ + CPU_ID_TPC_QMAN_ARC1 = 7, /* DCORE0_TPC1 */ + CPU_ID_TPC_QMAN_ARC2 = 8, /* DCORE0_TPC2 */ + CPU_ID_TPC_QMAN_ARC3 = 9, /* DCORE0_TPC3 */ + CPU_ID_TPC_QMAN_ARC4 = 10, /* DCORE0_TPC4 */ + CPU_ID_TPC_QMAN_ARC5 = 11, /* DCORE0_TPC5 */ + CPU_ID_TPC_QMAN_ARC6 = 12, /* DCORE1_TPC0 */ + CPU_ID_TPC_QMAN_ARC7 = 13, /* DCORE1_TPC1 */ + CPU_ID_TPC_QMAN_ARC8 = 14, /* DCORE1_TPC2 */ + CPU_ID_TPC_QMAN_ARC9 = 15, /* DCORE1_TPC3 */ + CPU_ID_TPC_QMAN_ARC10 = 16, /* DCORE1_TPC4 */ + CPU_ID_TPC_QMAN_ARC11 = 17, /* DCORE1_TPC5 */ + CPU_ID_TPC_QMAN_ARC12 = 18, /* DCORE2_TPC0 */ + CPU_ID_TPC_QMAN_ARC13 = 19, /* DCORE2_TPC1 */ + CPU_ID_TPC_QMAN_ARC14 = 20, /* DCORE2_TPC2 */ + CPU_ID_TPC_QMAN_ARC15 = 21, /* DCORE2_TPC3 */ + CPU_ID_TPC_QMAN_ARC16 = 22, /* DCORE2_TPC4 */ + CPU_ID_TPC_QMAN_ARC17 = 23, /* DCORE2_TPC5 */ + CPU_ID_TPC_QMAN_ARC18 = 24, /* DCORE3_TPC0 */ + CPU_ID_TPC_QMAN_ARC19 = 25, /* DCORE3_TPC1 */ + CPU_ID_TPC_QMAN_ARC20 = 26, /* DCORE3_TPC2 */ + CPU_ID_TPC_QMAN_ARC21 = 27, /* DCORE3_TPC3 */ + CPU_ID_TPC_QMAN_ARC22 = 28, /* DCORE3_TPC4 */ + CPU_ID_TPC_QMAN_ARC23 = 29, /* DCORE3_TPC5 */ + CPU_ID_TPC_QMAN_ARC24 = 30, /* DCORE0_TPC6 - Never present */ + + CPU_ID_MME_QMAN_ARC0 = 31, /* DCORE0_MME0 */ + CPU_ID_MME_QMAN_ARC1 = 32, /* DCORE2_MME0 */ + + CPU_ID_EDMA_QMAN_ARC0 = 33, /* DCORE0_EDMA0 */ + CPU_ID_EDMA_QMAN_ARC1 = 34, /* DCORE0_EDMA1 */ + CPU_ID_EDMA_QMAN_ARC2 = 35, /* DCORE1_EDMA0 */ + CPU_ID_EDMA_QMAN_ARC3 = 36, /* DCORE1_EDMA1 */ + CPU_ID_EDMA_QMAN_ARC4 = 37, /* DCORE2_EDMA0 */ + CPU_ID_EDMA_QMAN_ARC5 = 38, /* DCORE2_EDMA1 */ + CPU_ID_EDMA_QMAN_ARC6 = 39, /* DCORE3_EDMA0 */ + CPU_ID_EDMA_QMAN_ARC7 = 40, /* DCORE3_EDMA1 */ + + CPU_ID_PDMA_QMAN_ARC0 = 41, /* DCORE0_PDMA0 */ + CPU_ID_PDMA_QMAN_ARC1 = 42, /* DCORE0_PDMA1 */ + + CPU_ID_ROT_QMAN_ARC0 = 43, /* ROT0 */ + CPU_ID_ROT_QMAN_ARC1 = 44, /* ROT1 */ + + CPU_ID_NIC_QMAN_ARC0 = 45, /* NIC0_0 */ + CPU_ID_NIC_QMAN_ARC1 = 46, /* NIC0_1 */ + CPU_ID_NIC_QMAN_ARC2 = 47, /* NIC1_0 */ + CPU_ID_NIC_QMAN_ARC3 = 48, /* NIC1_1 */ + CPU_ID_NIC_QMAN_ARC4 = 49, /* NIC2_0 */ + CPU_ID_NIC_QMAN_ARC5 = 50, /* NIC2_1 */ + CPU_ID_NIC_QMAN_ARC6 = 51, /* NIC3_0 */ + CPU_ID_NIC_QMAN_ARC7 = 52, /* NIC3_1 */ + CPU_ID_NIC_QMAN_ARC8 = 53, /* NIC4_0 */ + CPU_ID_NIC_QMAN_ARC9 = 54, /* NIC4_1 */ + CPU_ID_NIC_QMAN_ARC10 = 55, /* NIC5_0 */ + CPU_ID_NIC_QMAN_ARC11 = 56, /* NIC5_1 */ + CPU_ID_NIC_QMAN_ARC12 = 57, /* NIC6_0 */ + CPU_ID_NIC_QMAN_ARC13 = 58, /* NIC6_1 */ + CPU_ID_NIC_QMAN_ARC14 = 59, /* NIC7_0 */ + CPU_ID_NIC_QMAN_ARC15 = 60, /* NIC7_1 */ + CPU_ID_NIC_QMAN_ARC16 = 61, /* NIC8_0 */ + CPU_ID_NIC_QMAN_ARC17 = 62, /* NIC8_1 */ + CPU_ID_NIC_QMAN_ARC18 = 63, /* NIC9_0 */ + CPU_ID_NIC_QMAN_ARC19 = 64, /* NIC9_1 */ + CPU_ID_NIC_QMAN_ARC20 = 65, /* NIC10_0 */ + CPU_ID_NIC_QMAN_ARC21 = 66, /* NIC10_1 */ + CPU_ID_NIC_QMAN_ARC22 = 67, /* NIC11_0 */ + CPU_ID_NIC_QMAN_ARC23 = 68, /* NIC11_1 */ + + CPU_ID_MAX = 69, + CPU_ID_SCHED_MAX = 6, + + CPU_ID_ALL = 0xFE, + CPU_ID_INVALID = 0xFF, +}; + +enum arc_regions_t { + ARC_REGION0_UNSED = 0, + /* + * Extension registers + * None + */ + ARC_REGION1_SRAM = 1, + /* + * Extension registers + * AUX_SRAM_LSB_ADDR + * AUX_SRAM_MSB_ADDR + * ARC Address: 0x1000_0000 + */ + ARC_REGION2_CFG = 2, + /* + * Extension registers + * AUX_CFG_LSB_ADDR + * AUX_CFG_MSB_ADDR + * ARC Address: 0x2000_0000 + */ + ARC_REGION3_GENERAL = 3, + /* + * Extension registers + * AUX_GENERAL_PURPOSE_LSB_ADDR_0 + * AUX_GENERAL_PURPOSE_MSB_ADDR_0 + * ARC Address: 0x3000_0000 + */ + ARC_REGION4_HBM0_FW = 4, + /* + * Extension registers + * AUX_HBM0_LSB_ADDR + * AUX_HBM0_MSB_ADDR + * AUX_HBM0_OFFSET + * ARC Address: 0x4000_0000 + */ + ARC_REGION5_HBM1_GC_DATA = 5, + /* + * Extension registers + * AUX_HBM1_LSB_ADDR + * AUX_HBM1_MSB_ADDR + * AUX_HBM1_OFFSET + * ARC Address: 0x5000_0000 + */ + ARC_REGION6_HBM2_GC_DATA = 6, + /* + * Extension registers + * AUX_HBM2_LSB_ADDR + * AUX_HBM2_MSB_ADDR + * AUX_HBM2_OFFSET + * ARC Address: 0x6000_0000 + */ + ARC_REGION7_HBM3_GC_DATA = 7, + /* + * Extension registers + * AUX_HBM3_LSB_ADDR + * AUX_HBM3_MSB_ADDR + * AUX_HBM3_OFFSET + * ARC Address: 0x7000_0000 + */ + ARC_REGION8_DCCM = 8, + /* + * Extension registers + * None + * ARC Address: 0x8000_0000 + */ + ARC_REGION9_PCIE = 9, + /* + * Extension registers + * AUX_PCIE_LSB_ADDR + * AUX_PCIE_MSB_ADDR + * ARC Address: 0x9000_0000 + */ + ARC_REGION10_GENERAL = 10, + /* + * Extension registers + * AUX_GENERAL_PURPOSE_LSB_ADDR_1 + * AUX_GENERAL_PURPOSE_MSB_ADDR_1 + * ARC Address: 0xA000_0000 + */ + ARC_REGION11_GENERAL = 11, + /* + * Extension registers + * AUX_GENERAL_PURPOSE_LSB_ADDR_2 + * AUX_GENERAL_PURPOSE_MSB_ADDR_2 + * ARC Address: 0xB000_0000 + */ + ARC_REGION12_GENERAL = 12, + /* + * Extension registers + * AUX_GENERAL_PURPOSE_LSB_ADDR_3 + * AUX_GENERAL_PURPOSE_MSB_ADDR_3 + * ARC Address: 0xC000_0000 + */ + ARC_REGION13_GENERAL = 13, + /* + * Extension registers + * AUX_GENERAL_PURPOSE_LSB_ADDR_4 + * AUX_GENERAL_PURPOSE_MSB_ADDR_4 + * ARC Address: 0xD000_0000 + */ + ARC_REGION14_GENERAL = 14, + /* + * Extension registers + * AUX_GENERAL_PURPOSE_LSB_ADDR_5 + * AUX_GENERAL_PURPOSE_MSB_ADDR_5 + * ARC Address: 0xE000_0000 + */ + ARC_REGION15_LBU = 15 + /* + * Extension registers + * None + * ARC Address: 0xF000_0000 + */ +}; + +#endif /* __GAUDI2_ARC_COMMON_PACKETS_H__ */ diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/arc_farm_arc0_acp_eng_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/arc_farm_arc0_acp_eng_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/arc_farm_arc0_acp_eng_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/arc_farm_arc0_acp_eng_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/arc_farm_arc0_aux_masks.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/arc_farm_arc0_aux_masks.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/arc_farm_arc0_aux_masks.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/arc_farm_arc0_aux_masks.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/arc_farm_arc0_aux_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/arc_farm_arc0_aux_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/arc_farm_arc0_aux_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/arc_farm_arc0_aux_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/arc_farm_arc0_dup_eng_axuser_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/arc_farm_arc0_dup_eng_axuser_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/arc_farm_arc0_dup_eng_axuser_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/arc_farm_arc0_dup_eng_axuser_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/arc_farm_arc0_dup_eng_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/arc_farm_arc0_dup_eng_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/arc_farm_arc0_dup_eng_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/arc_farm_arc0_dup_eng_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/arc_farm_kdma_ctx_axuser_masks.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/arc_farm_kdma_ctx_axuser_masks.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/arc_farm_kdma_ctx_axuser_masks.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/arc_farm_kdma_ctx_axuser_masks.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/arc_farm_kdma_ctx_axuser_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/arc_farm_kdma_ctx_axuser_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/arc_farm_kdma_ctx_axuser_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/arc_farm_kdma_ctx_axuser_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/arc_farm_kdma_ctx_masks.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/arc_farm_kdma_ctx_masks.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/arc_farm_kdma_ctx_masks.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/arc_farm_kdma_ctx_masks.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/arc_farm_kdma_ctx_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/arc_farm_kdma_ctx_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/arc_farm_kdma_ctx_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/arc_farm_kdma_ctx_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/arc_farm_kdma_kdma_cgm_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/arc_farm_kdma_kdma_cgm_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/arc_farm_kdma_kdma_cgm_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/arc_farm_kdma_kdma_cgm_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/arc_farm_kdma_masks.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/arc_farm_kdma_masks.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/arc_farm_kdma_masks.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/arc_farm_kdma_masks.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/arc_farm_kdma_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/arc_farm_kdma_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/arc_farm_kdma_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/arc_farm_kdma_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/cpu_if_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/cpu_if_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/cpu_if_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/cpu_if_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_dec0_cmd_masks.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_dec0_cmd_masks.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_dec0_cmd_masks.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_dec0_cmd_masks.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_dec0_cmd_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_dec0_cmd_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_dec0_cmd_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_dec0_cmd_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_edma0_core_ctx_axuser_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_edma0_core_ctx_axuser_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_edma0_core_ctx_axuser_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_edma0_core_ctx_axuser_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_edma0_core_ctx_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_edma0_core_ctx_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_edma0_core_ctx_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_edma0_core_ctx_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_edma0_core_masks.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_edma0_core_masks.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_edma0_core_masks.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_edma0_core_masks.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_edma0_core_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_edma0_core_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_edma0_core_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_edma0_core_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_edma0_qm_arc_aux_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_edma0_qm_arc_aux_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_edma0_qm_arc_aux_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_edma0_qm_arc_aux_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_edma0_qm_axuser_nonsecured_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_edma0_qm_axuser_nonsecured_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_edma0_qm_axuser_nonsecured_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_edma0_qm_axuser_nonsecured_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_edma0_qm_cgm_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_edma0_qm_cgm_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_edma0_qm_cgm_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_edma0_qm_cgm_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_edma0_qm_masks.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_edma0_qm_masks.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_edma0_qm_masks.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_edma0_qm_masks.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_edma0_qm_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_edma0_qm_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_edma0_qm_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_edma0_qm_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_edma1_core_ctx_axuser_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_edma1_core_ctx_axuser_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_edma1_core_ctx_axuser_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_edma1_core_ctx_axuser_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_edma1_qm_axuser_nonsecured_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_edma1_qm_axuser_nonsecured_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_edma1_qm_axuser_nonsecured_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_edma1_qm_axuser_nonsecured_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_hmmu0_mmu_masks.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_hmmu0_mmu_masks.h similarity index 99% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_hmmu0_mmu_masks.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_hmmu0_mmu_masks.h index df51eac10dd7352b7a59960832d8664e56c929a3..2965b6a3b4232770641ad8085a19753e8605bc1c 100644 --- a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_hmmu0_mmu_masks.h +++ b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_hmmu0_mmu_masks.h @@ -150,8 +150,7 @@ #define DCORE0_HMMU0_MMU_STATIC_MULTI_PAGE_SIZE_HOP1_PAGE_SIZE_SHIFT 16 #define DCORE0_HMMU0_MMU_STATIC_MULTI_PAGE_SIZE_HOP1_PAGE_SIZE_MASK 0xF0000 #define DCORE0_HMMU0_MMU_STATIC_MULTI_PAGE_SIZE_CFG_8_BITS_HOP_MODE_EN_SHIFT 20 -#define DCORE0_HMMU0_MMU_STATIC_MULTI_PAGE_SIZE_CFG_8_BITS_HOP_MODE_EN_MASK \ -0x100000 +#define DCORE0_HMMU0_MMU_STATIC_MULTI_PAGE_SIZE_CFG_8_BITS_HOP_MODE_EN_MASK 0x100000 /* DCORE0_HMMU0_MMU_CORE_SEP_CACHE_RNG */ #define DCORE0_HMMU0_MMU_CORE_SEP_CACHE_RNG_CORE_SET_MASK_SHIFT 0 @@ -235,23 +234,19 @@ /* DCORE0_HMMU0_MMU_ILLEGAL_ADDR_WRITE_63_32 */ #define DCORE0_HMMU0_MMU_ILLEGAL_ADDR_WRITE_63_32_ILLEGAL_ADDR_63_32_SHIFT 0 -#define DCORE0_HMMU0_MMU_ILLEGAL_ADDR_WRITE_63_32_ILLEGAL_ADDR_63_32_MASK \ -0xFFFFFFFF +#define DCORE0_HMMU0_MMU_ILLEGAL_ADDR_WRITE_63_32_ILLEGAL_ADDR_63_32_MASK 0xFFFFFFFF /* DCORE0_HMMU0_MMU_ILLEGAL_ADDR_WRITE_31_0 */ #define DCORE0_HMMU0_MMU_ILLEGAL_ADDR_WRITE_31_0_ILLEGAL_ADDR_31_0_SHIFT 0 -#define DCORE0_HMMU0_MMU_ILLEGAL_ADDR_WRITE_31_0_ILLEGAL_ADDR_31_0_MASK \ -0xFFFFFFFF +#define DCORE0_HMMU0_MMU_ILLEGAL_ADDR_WRITE_31_0_ILLEGAL_ADDR_31_0_MASK 0xFFFFFFFF /* DCORE0_HMMU0_MMU_ILLEGAL_ADDR_READ_63_32 */ #define DCORE0_HMMU0_MMU_ILLEGAL_ADDR_READ_63_32_ILLEGAL_ADDR_63_32_SHIFT 0 -#define DCORE0_HMMU0_MMU_ILLEGAL_ADDR_READ_63_32_ILLEGAL_ADDR_63_32_MASK \ -0xFFFFFFFF +#define DCORE0_HMMU0_MMU_ILLEGAL_ADDR_READ_63_32_ILLEGAL_ADDR_63_32_MASK 0xFFFFFFFF /* DCORE0_HMMU0_MMU_ILLEGAL_ADDR_READ_31_0 */ #define DCORE0_HMMU0_MMU_ILLEGAL_ADDR_READ_31_0_ILLEGAL_ADDR_31_0_SHIFT 0 -#define DCORE0_HMMU0_MMU_ILLEGAL_ADDR_READ_31_0_ILLEGAL_ADDR_31_0_MASK \ -0xFFFFFFFF +#define DCORE0_HMMU0_MMU_ILLEGAL_ADDR_READ_31_0_ILLEGAL_ADDR_31_0_MASK 0xFFFFFFFF /* DCORE0_HMMU0_MMU_RAZWI_WRITE_VLD */ #define DCORE0_HMMU0_MMU_RAZWI_WRITE_VLD_R_SHIFT 0 diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_hmmu0_mmu_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_hmmu0_mmu_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_hmmu0_mmu_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_hmmu0_mmu_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_hmmu0_stlb_masks.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_hmmu0_stlb_masks.h similarity index 97% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_hmmu0_stlb_masks.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_hmmu0_stlb_masks.h index 192eba5f07bb61151f602ae8b3bb891eb249a8a1..a311778b21e7a649733ccf1cf8f392b5fc4ca71f 100644 --- a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_hmmu0_stlb_masks.h +++ b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_hmmu0_stlb_masks.h @@ -92,8 +92,7 @@ #define DCORE0_HMMU0_STLB_HOP_CONFIGURATION_ONLY_LARGE_PAGE_SHIFT 20 #define DCORE0_HMMU0_STLB_HOP_CONFIGURATION_ONLY_LARGE_PAGE_MASK 0x100000 #define DCORE0_HMMU0_STLB_HOP_CONFIGURATION_LARGE_PAGE_INDICATION_BIT_SHIFT 21 -#define DCORE0_HMMU0_STLB_HOP_CONFIGURATION_LARGE_PAGE_INDICATION_BIT_MASK \ -0x7E00000 +#define DCORE0_HMMU0_STLB_HOP_CONFIGURATION_LARGE_PAGE_INDICATION_BIT_MASK 0x7E00000 /* DCORE0_HMMU0_STLB_LINK_LIST_LOOKUP_MASK_63_32 */ #define DCORE0_HMMU0_STLB_LINK_LIST_LOOKUP_MASK_63_32_R_SHIFT 0 @@ -228,12 +227,8 @@ #define DCORE0_HMMU0_STLB_MEM_READ_ARPROT_R_MASK 0x7 /* DCORE0_HMMU0_STLB_RANGE_CACHE_INVALIDATION */ -#define \ -DCORE0_HMMU0_STLB_RANGE_CACHE_INVALIDATION_RANGE_INVALIDATION_ENABLE_SHIFT \ -0 -#define \ -DCORE0_HMMU0_STLB_RANGE_CACHE_INVALIDATION_RANGE_INVALIDATION_ENABLE_MASK \ -0x1 +#define DCORE0_HMMU0_STLB_RANGE_CACHE_INVALIDATION_RANGE_INVALIDATION_ENABLE_SHIFT 0 +#define DCORE0_HMMU0_STLB_RANGE_CACHE_INVALIDATION_RANGE_INVALIDATION_ENABLE_MASK 0x1 #define DCORE0_HMMU0_STLB_RANGE_CACHE_INVALIDATION_INVALIDATION_ASID_EN_SHIFT 1 #define DCORE0_HMMU0_STLB_RANGE_CACHE_INVALIDATION_INVALIDATION_ASID_EN_MASK 0x2 #define DCORE0_HMMU0_STLB_RANGE_CACHE_INVALIDATION_INVALIDATION_ASID_SHIFT 2 @@ -261,53 +256,43 @@ DCORE0_HMMU0_STLB_RANGE_CACHE_INVALIDATION_RANGE_INVALIDATION_ENABLE_MASK \ /* DCORE0_HMMU0_STLB_ASID_SCR_POLY_MATRIX_H3_0 */ #define DCORE0_HMMU0_STLB_ASID_SCR_POLY_MATRIX_H3_0_ASID_POLY_MATRIX_H3_SHIFT 0 -#define DCORE0_HMMU0_STLB_ASID_SCR_POLY_MATRIX_H3_0_ASID_POLY_MATRIX_H3_MASK \ -0x1FF +#define DCORE0_HMMU0_STLB_ASID_SCR_POLY_MATRIX_H3_0_ASID_POLY_MATRIX_H3_MASK 0x1FF /* DCORE0_HMMU0_STLB_ASID_SCR_POLY_MATRIX_H3_1 */ #define DCORE0_HMMU0_STLB_ASID_SCR_POLY_MATRIX_H3_1_ASID_POLY_MATRIX_H3_SHIFT 0 -#define DCORE0_HMMU0_STLB_ASID_SCR_POLY_MATRIX_H3_1_ASID_POLY_MATRIX_H3_MASK \ -0x1FF +#define DCORE0_HMMU0_STLB_ASID_SCR_POLY_MATRIX_H3_1_ASID_POLY_MATRIX_H3_MASK 0x1FF /* DCORE0_HMMU0_STLB_ASID_SCR_POLY_MATRIX_H3_2 */ #define DCORE0_HMMU0_STLB_ASID_SCR_POLY_MATRIX_H3_2_ASID_POLY_MATRIX_H3_SHIFT 0 -#define DCORE0_HMMU0_STLB_ASID_SCR_POLY_MATRIX_H3_2_ASID_POLY_MATRIX_H3_MASK \ -0x1FF +#define DCORE0_HMMU0_STLB_ASID_SCR_POLY_MATRIX_H3_2_ASID_POLY_MATRIX_H3_MASK 0x1FF /* DCORE0_HMMU0_STLB_ASID_SCR_POLY_MATRIX_H3_3 */ #define DCORE0_HMMU0_STLB_ASID_SCR_POLY_MATRIX_H3_3_ASID_POLY_MATRIX_H3_SHIFT 0 -#define DCORE0_HMMU0_STLB_ASID_SCR_POLY_MATRIX_H3_3_ASID_POLY_MATRIX_H3_MASK \ -0x1FF +#define DCORE0_HMMU0_STLB_ASID_SCR_POLY_MATRIX_H3_3_ASID_POLY_MATRIX_H3_MASK 0x1FF /* DCORE0_HMMU0_STLB_ASID_SCR_POLY_MATRIX_H3_4 */ #define DCORE0_HMMU0_STLB_ASID_SCR_POLY_MATRIX_H3_4_ASID_POLY_MATRIX_H3_SHIFT 0 -#define DCORE0_HMMU0_STLB_ASID_SCR_POLY_MATRIX_H3_4_ASID_POLY_MATRIX_H3_MASK \ -0x1FF +#define DCORE0_HMMU0_STLB_ASID_SCR_POLY_MATRIX_H3_4_ASID_POLY_MATRIX_H3_MASK 0x1FF /* DCORE0_HMMU0_STLB_ASID_SCR_POLY_MATRIX_H3_5 */ #define DCORE0_HMMU0_STLB_ASID_SCR_POLY_MATRIX_H3_5_ASID_POLY_MATRIX_H3_SHIFT 0 -#define DCORE0_HMMU0_STLB_ASID_SCR_POLY_MATRIX_H3_5_ASID_POLY_MATRIX_H3_MASK \ -0x1FF +#define DCORE0_HMMU0_STLB_ASID_SCR_POLY_MATRIX_H3_5_ASID_POLY_MATRIX_H3_MASK 0x1FF /* DCORE0_HMMU0_STLB_ASID_SCR_POLY_MATRIX_H3_6 */ #define DCORE0_HMMU0_STLB_ASID_SCR_POLY_MATRIX_H3_6_ASID_POLY_MATRIX_H3_SHIFT 0 -#define DCORE0_HMMU0_STLB_ASID_SCR_POLY_MATRIX_H3_6_ASID_POLY_MATRIX_H3_MASK \ -0x1FF +#define DCORE0_HMMU0_STLB_ASID_SCR_POLY_MATRIX_H3_6_ASID_POLY_MATRIX_H3_MASK 0x1FF /* DCORE0_HMMU0_STLB_ASID_SCR_POLY_MATRIX_H3_7 */ #define DCORE0_HMMU0_STLB_ASID_SCR_POLY_MATRIX_H3_7_ASID_POLY_MATRIX_H3_SHIFT 0 -#define DCORE0_HMMU0_STLB_ASID_SCR_POLY_MATRIX_H3_7_ASID_POLY_MATRIX_H3_MASK \ -0x1FF +#define DCORE0_HMMU0_STLB_ASID_SCR_POLY_MATRIX_H3_7_ASID_POLY_MATRIX_H3_MASK 0x1FF /* DCORE0_HMMU0_STLB_ASID_SCR_POLY_MATRIX_H3_8 */ #define DCORE0_HMMU0_STLB_ASID_SCR_POLY_MATRIX_H3_8_ASID_POLY_MATRIX_H3_SHIFT 0 -#define DCORE0_HMMU0_STLB_ASID_SCR_POLY_MATRIX_H3_8_ASID_POLY_MATRIX_H3_MASK \ -0x1FF +#define DCORE0_HMMU0_STLB_ASID_SCR_POLY_MATRIX_H3_8_ASID_POLY_MATRIX_H3_MASK 0x1FF /* DCORE0_HMMU0_STLB_ASID_SCR_POLY_MATRIX_H3_9 */ #define DCORE0_HMMU0_STLB_ASID_SCR_POLY_MATRIX_H3_9_ASID_POLY_MATRIX_H3_SHIFT 0 -#define DCORE0_HMMU0_STLB_ASID_SCR_POLY_MATRIX_H3_9_ASID_POLY_MATRIX_H3_MASK \ -0x1FF +#define DCORE0_HMMU0_STLB_ASID_SCR_POLY_MATRIX_H3_9_ASID_POLY_MATRIX_H3_MASK 0x1FF /* DCORE0_HMMU0_STLB_ASID_SCR_POLY_MAT_H3_10 */ #define DCORE0_HMMU0_STLB_ASID_SCR_POLY_MAT_H3_10_ASID_POLY_MATRIX_H3_SHIFT 0 diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_hmmu0_stlb_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_hmmu0_stlb_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_hmmu0_stlb_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_hmmu0_stlb_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_acc_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_acc_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_acc_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_acc_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_agu_cout0_master_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_agu_cout0_master_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_agu_cout0_master_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_agu_cout0_master_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_agu_cout0_slave_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_agu_cout0_slave_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_agu_cout0_slave_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_agu_cout0_slave_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_agu_cout1_master_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_agu_cout1_master_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_agu_cout1_master_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_agu_cout1_master_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_agu_cout1_slave_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_agu_cout1_slave_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_agu_cout1_slave_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_agu_cout1_slave_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_agu_in0_master_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_agu_in0_master_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_agu_in0_master_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_agu_in0_master_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_agu_in0_slave_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_agu_in0_slave_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_agu_in0_slave_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_agu_in0_slave_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_agu_in1_master_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_agu_in1_master_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_agu_in1_master_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_agu_in1_master_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_agu_in1_slave_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_agu_in1_slave_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_agu_in1_slave_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_agu_in1_slave_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_agu_in2_master_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_agu_in2_master_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_agu_in2_master_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_agu_in2_master_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_agu_in2_slave_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_agu_in2_slave_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_agu_in2_slave_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_agu_in2_slave_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_agu_in3_master_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_agu_in3_master_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_agu_in3_master_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_agu_in3_master_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_agu_in3_slave_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_agu_in3_slave_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_agu_in3_slave_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_agu_in3_slave_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_agu_in4_master_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_agu_in4_master_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_agu_in4_master_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_agu_in4_master_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_agu_in4_slave_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_agu_in4_slave_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_agu_in4_slave_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_agu_in4_slave_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_base_addr_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_base_addr_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_base_addr_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_base_addr_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_non_tensor_end_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_non_tensor_end_regs.h similarity index 98% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_non_tensor_end_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_non_tensor_end_regs.h index 7c22b9383f3cb0700fe3db6bb27466043a770091..fb53feb0a1a6ff8a8b76de39a9362a63dd68289c 100644 --- a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_non_tensor_end_regs.h +++ b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_non_tensor_end_regs.h @@ -20,8 +20,7 @@ ***************************************** */ -#define mmDCORE0_MME_CTRL_LO_ARCH_NON_TENSOR_END_CONV_KERNEL_SIZE_MINUS_1 \ -0x40CB280 +#define mmDCORE0_MME_CTRL_LO_ARCH_NON_TENSOR_END_CONV_KERNEL_SIZE_MINUS_1 0x40CB280 #define mmDCORE0_MME_CTRL_LO_ARCH_NON_TENSOR_END_CONV_LOW 0x40CB284 @@ -29,8 +28,7 @@ #define mmDCORE0_MME_CTRL_LO_ARCH_NON_TENSOR_END_OUTER_LOOP 0x40CB28C -#define mmDCORE0_MME_CTRL_LO_ARCH_NON_TENSOR_END_NUM_ITERATIONS_MINUS_1 \ -0x40CB290 +#define mmDCORE0_MME_CTRL_LO_ARCH_NON_TENSOR_END_NUM_ITERATIONS_MINUS_1 0x40CB290 #define mmDCORE0_MME_CTRL_LO_ARCH_NON_TENSOR_END_SB_REPEAT 0x40CB294 diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_non_tensor_start_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_non_tensor_start_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_non_tensor_start_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_non_tensor_start_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_tensor_a_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_tensor_a_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_tensor_a_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_tensor_a_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_tensor_b_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_tensor_b_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_tensor_b_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_tensor_b_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_tensor_cout_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_tensor_cout_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_tensor_cout_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_arch_tensor_cout_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_masks.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_masks.h similarity index 99% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_masks.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_masks.h index f699661d76aacbb9d180d3b43dc3e72cb17270be..da0c94075e64f03640a46c4446370904a908517c 100644 --- a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_masks.h +++ b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_masks.h @@ -78,8 +78,7 @@ #define DCORE0_MME_CTRL_LO_ARCH_SYNC_OBJ_DW0_MASTER_WAIT_SLAVE_FENCE_SHIFT 15 #define DCORE0_MME_CTRL_LO_ARCH_SYNC_OBJ_DW0_MASTER_WAIT_SLAVE_FENCE_MASK 0x8000 #define DCORE0_MME_CTRL_LO_ARCH_SYNC_OBJ_DW0_SLAVE_SEND_FENCE2MASTER_SHIFT 16 -#define DCORE0_MME_CTRL_LO_ARCH_SYNC_OBJ_DW0_SLAVE_SEND_FENCE2MASTER_MASK \ -0x10000 +#define DCORE0_MME_CTRL_LO_ARCH_SYNC_OBJ_DW0_SLAVE_SEND_FENCE2MASTER_MASK 0x10000 #define DCORE0_MME_CTRL_LO_ARCH_SYNC_OBJ_DW0_SLAVE_SIGNAL_EN_SHIFT 17 #define DCORE0_MME_CTRL_LO_ARCH_SYNC_OBJ_DW0_SLAVE_SIGNAL_EN_MASK 0x20000 #define DCORE0_MME_CTRL_LO_ARCH_SYNC_OBJ_DW0_SLAVE0_USE_SLV_ADR_SHIFT 18 @@ -87,11 +86,9 @@ #define DCORE0_MME_CTRL_LO_ARCH_SYNC_OBJ_DW0_SLAVE1_USE_SLV_ADR_SHIFT 19 #define DCORE0_MME_CTRL_LO_ARCH_SYNC_OBJ_DW0_SLAVE1_USE_SLV_ADR_MASK 0x80000 #define DCORE0_MME_CTRL_LO_ARCH_SYNC_OBJ_DW0_SLAVE0_USE_MSTR_ADR_PLUS4_SHIFT 20 -#define DCORE0_MME_CTRL_LO_ARCH_SYNC_OBJ_DW0_SLAVE0_USE_MSTR_ADR_PLUS4_MASK \ -0x100000 +#define DCORE0_MME_CTRL_LO_ARCH_SYNC_OBJ_DW0_SLAVE0_USE_MSTR_ADR_PLUS4_MASK 0x100000 #define DCORE0_MME_CTRL_LO_ARCH_SYNC_OBJ_DW0_SLAVE1_USE_MSTR_ADR_PLUS4_SHIFT 21 -#define DCORE0_MME_CTRL_LO_ARCH_SYNC_OBJ_DW0_SLAVE1_USE_MSTR_ADR_PLUS4_MASK \ -0x200000 +#define DCORE0_MME_CTRL_LO_ARCH_SYNC_OBJ_DW0_SLAVE1_USE_MSTR_ADR_PLUS4_MASK 0x200000 /* DCORE0_MME_CTRL_LO_ARCH_SYNC_OBJ_ADDR0 */ #define DCORE0_MME_CTRL_LO_ARCH_SYNC_OBJ_ADDR0_V_SHIFT 0 diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_mme_axuser_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_mme_axuser_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_mme_axuser_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_mme_axuser_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_ctrl_lo_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_qm_arc_acp_eng_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_qm_arc_acp_eng_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_qm_arc_acp_eng_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_qm_arc_acp_eng_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_qm_arc_aux_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_qm_arc_aux_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_qm_arc_aux_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_qm_arc_aux_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_qm_arc_dup_eng_axuser_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_qm_arc_dup_eng_axuser_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_qm_arc_dup_eng_axuser_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_qm_arc_dup_eng_axuser_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_qm_arc_dup_eng_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_qm_arc_dup_eng_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_qm_arc_dup_eng_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_qm_arc_dup_eng_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_qm_axuser_nonsecured_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_qm_axuser_nonsecured_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_qm_axuser_nonsecured_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_qm_axuser_nonsecured_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_qm_axuser_secured_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_qm_axuser_secured_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_qm_axuser_secured_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_qm_axuser_secured_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_qm_cgm_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_qm_cgm_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_qm_cgm_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_qm_cgm_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_qm_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_qm_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_qm_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_qm_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_sbte0_masks.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_sbte0_masks.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_sbte0_masks.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_sbte0_masks.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_sbte0_mstr_if_axuser_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_sbte0_mstr_if_axuser_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_sbte0_mstr_if_axuser_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_sbte0_mstr_if_axuser_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_wb0_mstr_if_axuser_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_wb0_mstr_if_axuser_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_mme_wb0_mstr_if_axuser_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_mme_wb0_mstr_if_axuser_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_rtr0_ctrl_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_rtr0_ctrl_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_rtr0_ctrl_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_rtr0_ctrl_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_rtr0_mstr_if_rr_prvt_hbw_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_rtr0_mstr_if_rr_prvt_hbw_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_rtr0_mstr_if_rr_prvt_hbw_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_rtr0_mstr_if_rr_prvt_hbw_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_rtr0_mstr_if_rr_prvt_lbw_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_rtr0_mstr_if_rr_prvt_lbw_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_rtr0_mstr_if_rr_prvt_lbw_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_rtr0_mstr_if_rr_prvt_lbw_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_rtr0_mstr_if_rr_shrd_hbw_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_rtr0_mstr_if_rr_shrd_hbw_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_rtr0_mstr_if_rr_shrd_hbw_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_rtr0_mstr_if_rr_shrd_hbw_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_rtr0_mstr_if_rr_shrd_lbw_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_rtr0_mstr_if_rr_shrd_lbw_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_rtr0_mstr_if_rr_shrd_lbw_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_rtr0_mstr_if_rr_shrd_lbw_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_sync_mngr_glbl_masks.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_sync_mngr_glbl_masks.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_sync_mngr_glbl_masks.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_sync_mngr_glbl_masks.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_sync_mngr_glbl_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_sync_mngr_glbl_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_sync_mngr_glbl_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_sync_mngr_glbl_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_sync_mngr_mstr_if_axuser_masks.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_sync_mngr_mstr_if_axuser_masks.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_sync_mngr_mstr_if_axuser_masks.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_sync_mngr_mstr_if_axuser_masks.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_sync_mngr_mstr_if_axuser_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_sync_mngr_mstr_if_axuser_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_sync_mngr_mstr_if_axuser_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_sync_mngr_mstr_if_axuser_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_sync_mngr_objs_masks.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_sync_mngr_objs_masks.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_sync_mngr_objs_masks.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_sync_mngr_objs_masks.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_sync_mngr_objs_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_sync_mngr_objs_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_sync_mngr_objs_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_sync_mngr_objs_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_cfg_axuser_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_cfg_axuser_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_cfg_axuser_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_cfg_axuser_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_cfg_kernel_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_cfg_kernel_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_cfg_kernel_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_cfg_kernel_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_cfg_kernel_tensor_0_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_cfg_kernel_tensor_0_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_cfg_kernel_tensor_0_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_cfg_kernel_tensor_0_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_cfg_masks.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_cfg_masks.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_cfg_masks.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_cfg_masks.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_cfg_qm_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_cfg_qm_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_cfg_qm_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_cfg_qm_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_cfg_qm_sync_object_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_cfg_qm_sync_object_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_cfg_qm_sync_object_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_cfg_qm_sync_object_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_cfg_qm_tensor_0_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_cfg_qm_tensor_0_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_cfg_qm_tensor_0_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_cfg_qm_tensor_0_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_cfg_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_cfg_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_cfg_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_cfg_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_cfg_special_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_cfg_special_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_cfg_special_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_cfg_special_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_eml_busmon_0_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_eml_busmon_0_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_eml_busmon_0_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_eml_busmon_0_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_eml_etf_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_eml_etf_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_eml_etf_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_eml_etf_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_eml_funnel_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_eml_funnel_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_eml_funnel_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_eml_funnel_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_eml_spmu_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_eml_spmu_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_eml_spmu_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_eml_spmu_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_eml_stm_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_eml_stm_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_eml_stm_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_eml_stm_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_qm_arc_aux_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_qm_arc_aux_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_qm_arc_aux_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_qm_arc_aux_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_qm_axuser_nonsecured_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_qm_axuser_nonsecured_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_qm_axuser_nonsecured_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_qm_axuser_nonsecured_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_qm_cgm_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_qm_cgm_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_qm_cgm_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_qm_cgm_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_qm_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_qm_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_qm_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_tpc0_qm_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_vdec0_brdg_ctrl_axuser_dec_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_vdec0_brdg_ctrl_axuser_dec_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_vdec0_brdg_ctrl_axuser_dec_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_vdec0_brdg_ctrl_axuser_dec_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_vdec0_brdg_ctrl_axuser_msix_abnrm_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_vdec0_brdg_ctrl_axuser_msix_abnrm_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_vdec0_brdg_ctrl_axuser_msix_abnrm_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_vdec0_brdg_ctrl_axuser_msix_abnrm_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_vdec0_brdg_ctrl_axuser_msix_l2c_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_vdec0_brdg_ctrl_axuser_msix_l2c_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_vdec0_brdg_ctrl_axuser_msix_l2c_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_vdec0_brdg_ctrl_axuser_msix_l2c_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_vdec0_brdg_ctrl_axuser_msix_nrm_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_vdec0_brdg_ctrl_axuser_msix_nrm_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_vdec0_brdg_ctrl_axuser_msix_nrm_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_vdec0_brdg_ctrl_axuser_msix_nrm_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_vdec0_brdg_ctrl_axuser_msix_vcd_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_vdec0_brdg_ctrl_axuser_msix_vcd_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_vdec0_brdg_ctrl_axuser_msix_vcd_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_vdec0_brdg_ctrl_axuser_msix_vcd_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_vdec0_brdg_ctrl_masks.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_vdec0_brdg_ctrl_masks.h similarity index 99% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_vdec0_brdg_ctrl_masks.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_vdec0_brdg_ctrl_masks.h index 68dd98459c86d4616bd931389d42536f1ae9beb5..1c02f3dfdb6e8b9b7389c3e5d489cd0dc6bd422c 100644 --- a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_vdec0_brdg_ctrl_masks.h +++ b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_vdec0_brdg_ctrl_masks.h @@ -106,8 +106,7 @@ #define DCORE0_VDEC0_BRDG_CTRL_HBW_AXI_VIOL_CAUSE_AWBURST_VIOL_SHIFT 2 #define DCORE0_VDEC0_BRDG_CTRL_HBW_AXI_VIOL_CAUSE_AWBURST_VIOL_MASK 0x4 #define DCORE0_VDEC0_BRDG_CTRL_HBW_AXI_VIOL_CAUSE_AWADDR_SIZE_ALIGN_VIOL_SHIFT 3 -#define DCORE0_VDEC0_BRDG_CTRL_HBW_AXI_VIOL_CAUSE_AWADDR_SIZE_ALIGN_VIOL_MASK \ -0x8 +#define DCORE0_VDEC0_BRDG_CTRL_HBW_AXI_VIOL_CAUSE_AWADDR_SIZE_ALIGN_VIOL_MASK 0x8 #define DCORE0_VDEC0_BRDG_CTRL_HBW_AXI_VIOL_CAUSE_AWSIZE_VIOL_SHIFT 4 #define DCORE0_VDEC0_BRDG_CTRL_HBW_AXI_VIOL_CAUSE_AWSIZE_VIOL_MASK 0x10 #define DCORE0_VDEC0_BRDG_CTRL_HBW_AXI_VIOL_CAUSE_ARLEN_GT_31_SHIFT 5 @@ -117,8 +116,7 @@ #define DCORE0_VDEC0_BRDG_CTRL_HBW_AXI_VIOL_CAUSE_ARBURST_VIOL_SHIFT 7 #define DCORE0_VDEC0_BRDG_CTRL_HBW_AXI_VIOL_CAUSE_ARBURST_VIOL_MASK 0x80 #define DCORE0_VDEC0_BRDG_CTRL_HBW_AXI_VIOL_CAUSE_ARADDR_SIZE_ALIGN_VIOL_SHIFT 8 -#define DCORE0_VDEC0_BRDG_CTRL_HBW_AXI_VIOL_CAUSE_ARADDR_SIZE_ALIGN_VIOL_MASK \ -0x100 +#define DCORE0_VDEC0_BRDG_CTRL_HBW_AXI_VIOL_CAUSE_ARADDR_SIZE_ALIGN_VIOL_MASK 0x100 #define DCORE0_VDEC0_BRDG_CTRL_HBW_AXI_VIOL_CAUSE_ARSIZE_VIOL_SHIFT 9 #define DCORE0_VDEC0_BRDG_CTRL_HBW_AXI_VIOL_CAUSE_ARSIZE_VIOL_MASK 0x200 diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_vdec0_brdg_ctrl_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_vdec0_brdg_ctrl_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_vdec0_brdg_ctrl_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_vdec0_brdg_ctrl_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_vdec0_ctrl_special_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_vdec0_ctrl_special_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore0_vdec0_ctrl_special_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore0_vdec0_ctrl_special_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore1_mme_ctrl_lo_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore1_mme_ctrl_lo_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore1_mme_ctrl_lo_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore1_mme_ctrl_lo_regs.h diff --git a/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore1_sync_mngr_glbl_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore1_sync_mngr_glbl_regs.h new file mode 100644 index 0000000000000000000000000000000000000000..3d38027558148ad68c327ede249105536f7cbb23 --- /dev/null +++ b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore1_sync_mngr_glbl_regs.h @@ -0,0 +1,1203 @@ +/* SPDX-License-Identifier: GPL-2.0 + * + * Copyright 2016-2020 HabanaLabs, Ltd. + * All Rights Reserved. + * + */ + +/************************************ + ** This is an auto-generated file ** + ** DO NOT EDIT BELOW ** + ************************************/ + +#ifndef ASIC_REG_DCORE1_SYNC_MNGR_GLBL_REGS_H_ +#define ASIC_REG_DCORE1_SYNC_MNGR_GLBL_REGS_H_ + +/* + ***************************************** + * DCORE1_SYNC_MNGR_GLBL + * (Prototype: SOB_GLBL) + ***************************************** + */ + +#define mmDCORE1_SYNC_MNGR_GLBL_SM_SEI_MASK 0x431E000 + +#define mmDCORE1_SYNC_MNGR_GLBL_SM_SEI_CAUSE 0x431E004 + +#define mmDCORE1_SYNC_MNGR_GLBL_L2H_CPMR_L 0x431E008 + +#define mmDCORE1_SYNC_MNGR_GLBL_L2H_CPMR_H 0x431E00C + +#define mmDCORE1_SYNC_MNGR_GLBL_L2H_MASK_L 0x431E020 + +#define mmDCORE1_SYNC_MNGR_GLBL_L2H_MASK_H 0x431E024 + +#define mmDCORE1_SYNC_MNGR_GLBL_ASID_SEC 0x431E030 + +#define mmDCORE1_SYNC_MNGR_GLBL_ASID_PRIV_ONLY 0x431E034 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DELAY 0x431E038 + +#define mmDCORE1_SYNC_MNGR_GLBL_PI_SIZE 0x431E03C + +#define mmDCORE1_SYNC_MNGR_GLBL_SOB_ONLY 0x431E040 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INTR 0x431E044 + +#define mmDCORE1_SYNC_MNGR_GLBL_ASID_NONE_SEC_PRIV 0x431E048 + +#define mmDCORE1_SYNC_MNGR_GLBL_PI_INC_MODE_SIZE 0x431E04C + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_0 0x431E050 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_1 0x431E054 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_2 0x431E058 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_3 0x431E05C + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_4 0x431E060 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_5 0x431E064 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_6 0x431E068 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_7 0x431E06C + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_8 0x431E070 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_9 0x431E074 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_10 0x431E078 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_11 0x431E07C + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_12 0x431E080 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_13 0x431E084 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_14 0x431E088 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_15 0x431E08C + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_16 0x431E090 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_17 0x431E094 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_18 0x431E098 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_19 0x431E09C + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_20 0x431E0A0 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_21 0x431E0A4 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_22 0x431E0A8 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_23 0x431E0AC + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_24 0x431E0B0 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_25 0x431E0B4 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_26 0x431E0B8 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_27 0x431E0BC + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_28 0x431E0C0 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_29 0x431E0C4 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_30 0x431E0C8 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_31 0x431E0CC + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_32 0x431E0D0 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_33 0x431E0D4 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_34 0x431E0D8 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_35 0x431E0DC + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_36 0x431E0E0 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_37 0x431E0E4 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_38 0x431E0E8 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_39 0x431E0EC + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_40 0x431E0F0 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_41 0x431E0F4 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_42 0x431E0F8 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_43 0x431E0FC + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_44 0x431E100 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_45 0x431E104 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_46 0x431E108 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_47 0x431E10C + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_48 0x431E110 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_49 0x431E114 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_50 0x431E118 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_51 0x431E11C + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_52 0x431E120 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_53 0x431E124 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_54 0x431E128 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_55 0x431E12C + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_56 0x431E130 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_57 0x431E134 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_58 0x431E138 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_59 0x431E13C + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_60 0x431E140 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_61 0x431E144 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_62 0x431E148 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_L_63 0x431E14C + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_0 0x431E150 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_1 0x431E154 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_2 0x431E158 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_3 0x431E15C + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_4 0x431E160 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_5 0x431E164 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_6 0x431E168 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_7 0x431E16C + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_8 0x431E170 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_9 0x431E174 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_10 0x431E178 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_11 0x431E17C + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_12 0x431E180 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_13 0x431E184 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_14 0x431E188 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_15 0x431E18C + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_16 0x431E190 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_17 0x431E194 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_18 0x431E198 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_19 0x431E19C + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_20 0x431E1A0 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_21 0x431E1A4 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_22 0x431E1A8 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_23 0x431E1AC + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_24 0x431E1B0 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_25 0x431E1B4 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_26 0x431E1B8 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_27 0x431E1BC + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_28 0x431E1C0 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_29 0x431E1C4 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_30 0x431E1C8 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_31 0x431E1CC + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_32 0x431E1D0 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_33 0x431E1D4 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_34 0x431E1D8 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_35 0x431E1DC + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_36 0x431E1E0 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_37 0x431E1E4 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_38 0x431E1E8 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_39 0x431E1EC + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_40 0x431E1F0 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_41 0x431E1F4 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_42 0x431E1F8 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_43 0x431E1FC + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_44 0x431E200 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_45 0x431E204 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_46 0x431E208 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_47 0x431E20C + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_48 0x431E210 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_49 0x431E214 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_50 0x431E218 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_51 0x431E21C + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_52 0x431E220 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_53 0x431E224 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_54 0x431E228 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_55 0x431E22C + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_56 0x431E230 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_57 0x431E234 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_58 0x431E238 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_59 0x431E23C + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_60 0x431E240 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_61 0x431E244 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_62 0x431E248 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_BASE_ADDR_H_63 0x431E24C + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_0 0x431E250 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_1 0x431E254 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_2 0x431E258 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_3 0x431E25C + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_4 0x431E260 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_5 0x431E264 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_6 0x431E268 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_7 0x431E26C + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_8 0x431E270 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_9 0x431E274 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_10 0x431E278 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_11 0x431E27C + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_12 0x431E280 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_13 0x431E284 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_14 0x431E288 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_15 0x431E28C + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_16 0x431E290 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_17 0x431E294 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_18 0x431E298 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_19 0x431E29C + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_20 0x431E2A0 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_21 0x431E2A4 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_22 0x431E2A8 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_23 0x431E2AC + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_24 0x431E2B0 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_25 0x431E2B4 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_26 0x431E2B8 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_27 0x431E2BC + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_28 0x431E2C0 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_29 0x431E2C4 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_30 0x431E2C8 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_31 0x431E2CC + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_32 0x431E2D0 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_33 0x431E2D4 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_34 0x431E2D8 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_35 0x431E2DC + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_36 0x431E2E0 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_37 0x431E2E4 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_38 0x431E2E8 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_39 0x431E2EC + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_40 0x431E2F0 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_41 0x431E2F4 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_42 0x431E2F8 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_43 0x431E2FC + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_44 0x431E300 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_45 0x431E304 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_46 0x431E308 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_47 0x431E30C + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_48 0x431E310 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_49 0x431E314 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_50 0x431E318 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_51 0x431E31C + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_52 0x431E320 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_53 0x431E324 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_54 0x431E328 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_55 0x431E32C + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_56 0x431E330 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_57 0x431E334 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_58 0x431E338 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_59 0x431E33C + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_60 0x431E340 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_61 0x431E344 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_62 0x431E348 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SIZE_LOG2_63 0x431E34C + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_0 0x431E350 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_1 0x431E354 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_2 0x431E358 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_3 0x431E35C + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_4 0x431E360 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_5 0x431E364 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_6 0x431E368 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_7 0x431E36C + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_8 0x431E370 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_9 0x431E374 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_10 0x431E378 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_11 0x431E37C + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_12 0x431E380 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_13 0x431E384 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_14 0x431E388 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_15 0x431E38C + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_16 0x431E390 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_17 0x431E394 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_18 0x431E398 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_19 0x431E39C + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_20 0x431E3A0 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_21 0x431E3A4 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_22 0x431E3A8 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_23 0x431E3AC + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_24 0x431E3B0 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_25 0x431E3B4 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_26 0x431E3B8 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_27 0x431E3BC + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_28 0x431E3C0 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_29 0x431E3C4 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_30 0x431E3C8 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_31 0x431E3CC + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_32 0x431E3D0 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_33 0x431E3D4 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_34 0x431E3D8 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_35 0x431E3DC + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_36 0x431E3E0 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_37 0x431E3E4 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_38 0x431E3E8 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_39 0x431E3EC + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_40 0x431E3F0 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_41 0x431E3F4 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_42 0x431E3F8 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_43 0x431E3FC + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_44 0x431E400 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_45 0x431E404 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_46 0x431E408 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_47 0x431E40C + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_48 0x431E410 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_49 0x431E414 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_50 0x431E418 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_51 0x431E41C + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_52 0x431E420 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_53 0x431E424 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_54 0x431E428 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_55 0x431E42C + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_56 0x431E430 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_57 0x431E434 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_58 0x431E438 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_59 0x431E43C + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_60 0x431E440 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_61 0x431E444 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_62 0x431E448 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_PI_63 0x431E44C + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SEC_0 0x431E450 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SEC_1 0x431E454 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SEC_2 0x431E458 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SEC_3 0x431E45C + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SEC_4 0x431E460 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SEC_5 0x431E464 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SEC_6 0x431E468 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SEC_7 0x431E46C + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SEC_8 0x431E470 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SEC_9 0x431E474 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SEC_10 0x431E478 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SEC_11 0x431E47C + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SEC_12 0x431E480 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SEC_13 0x431E484 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SEC_14 0x431E488 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SEC_15 0x431E48C + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SEC_16 0x431E490 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SEC_17 0x431E494 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SEC_18 0x431E498 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SEC_19 0x431E49C + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SEC_20 0x431E4A0 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SEC_21 0x431E4A4 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SEC_22 0x431E4A8 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SEC_23 0x431E4AC + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SEC_24 0x431E4B0 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SEC_25 0x431E4B4 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SEC_26 0x431E4B8 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SEC_27 0x431E4BC + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SEC_28 0x431E4C0 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SEC_29 0x431E4C4 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SEC_30 0x431E4C8 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SEC_31 0x431E4CC + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SEC_32 0x431E4D0 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SEC_33 0x431E4D4 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SEC_34 0x431E4D8 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SEC_35 0x431E4DC + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SEC_36 0x431E4E0 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SEC_37 0x431E4E4 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SEC_38 0x431E4E8 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SEC_39 0x431E4EC + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SEC_40 0x431E4F0 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SEC_41 0x431E4F4 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SEC_42 0x431E4F8 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SEC_43 0x431E4FC + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SEC_44 0x431E500 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SEC_45 0x431E504 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SEC_46 0x431E508 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SEC_47 0x431E50C + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SEC_48 0x431E510 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SEC_49 0x431E514 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SEC_50 0x431E518 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SEC_51 0x431E51C + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SEC_52 0x431E520 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SEC_53 0x431E524 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SEC_54 0x431E528 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SEC_55 0x431E52C + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SEC_56 0x431E530 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SEC_57 0x431E534 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SEC_58 0x431E538 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SEC_59 0x431E53C + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SEC_60 0x431E540 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SEC_61 0x431E544 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SEC_62 0x431E548 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_SEC_63 0x431E54C + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_0 0x431E550 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_1 0x431E554 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_2 0x431E558 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_3 0x431E55C + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_4 0x431E560 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_5 0x431E564 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_6 0x431E568 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_7 0x431E56C + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_8 0x431E570 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_9 0x431E574 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_10 0x431E578 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_11 0x431E57C + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_12 0x431E580 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_13 0x431E584 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_14 0x431E588 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_15 0x431E58C + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_16 0x431E590 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_17 0x431E594 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_18 0x431E598 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_19 0x431E59C + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_20 0x431E5A0 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_21 0x431E5A4 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_22 0x431E5A8 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_23 0x431E5AC + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_24 0x431E5B0 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_25 0x431E5B4 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_26 0x431E5B8 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_27 0x431E5BC + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_28 0x431E5C0 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_29 0x431E5C4 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_30 0x431E5C8 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_31 0x431E5CC + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_32 0x431E5D0 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_33 0x431E5D4 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_34 0x431E5D8 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_35 0x431E5DC + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_36 0x431E5E0 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_37 0x431E5E4 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_38 0x431E5E8 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_39 0x431E5EC + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_40 0x431E5F0 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_41 0x431E5F4 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_42 0x431E5F8 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_43 0x431E5FC + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_44 0x431E600 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_45 0x431E604 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_46 0x431E608 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_47 0x431E60C + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_48 0x431E610 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_49 0x431E614 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_50 0x431E618 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_51 0x431E61C + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_52 0x431E620 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_53 0x431E624 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_54 0x431E628 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_55 0x431E62C + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_56 0x431E630 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_57 0x431E634 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_58 0x431E638 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_59 0x431E63C + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_60 0x431E640 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_61 0x431E644 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_62 0x431E648 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_L_63 0x431E64C + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_0 0x431E650 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_1 0x431E654 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_2 0x431E658 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_3 0x431E65C + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_4 0x431E660 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_5 0x431E664 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_6 0x431E668 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_7 0x431E66C + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_8 0x431E670 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_9 0x431E674 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_10 0x431E678 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_11 0x431E67C + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_12 0x431E680 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_13 0x431E684 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_14 0x431E688 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_15 0x431E68C + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_16 0x431E690 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_17 0x431E694 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_18 0x431E698 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_19 0x431E69C + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_20 0x431E6A0 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_21 0x431E6A4 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_22 0x431E6A8 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_23 0x431E6AC + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_24 0x431E6B0 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_25 0x431E6B4 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_26 0x431E6B8 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_27 0x431E6BC + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_28 0x431E6C0 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_29 0x431E6C4 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_30 0x431E6C8 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_31 0x431E6CC + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_32 0x431E6D0 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_33 0x431E6D4 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_34 0x431E6D8 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_35 0x431E6DC + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_36 0x431E6E0 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_37 0x431E6E4 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_38 0x431E6E8 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_39 0x431E6EC + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_40 0x431E6F0 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_41 0x431E6F4 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_42 0x431E6F8 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_43 0x431E6FC + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_44 0x431E700 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_45 0x431E704 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_46 0x431E708 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_47 0x431E70C + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_48 0x431E710 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_49 0x431E714 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_50 0x431E718 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_51 0x431E71C + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_52 0x431E720 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_53 0x431E724 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_54 0x431E728 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_55 0x431E72C + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_56 0x431E730 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_57 0x431E734 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_58 0x431E738 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_59 0x431E73C + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_60 0x431E740 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_61 0x431E744 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_62 0x431E748 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_ADDR_H_63 0x431E74C + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_0 0x431E750 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_1 0x431E754 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_2 0x431E758 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_3 0x431E75C + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_4 0x431E760 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_5 0x431E764 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_6 0x431E768 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_7 0x431E76C + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_8 0x431E770 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_9 0x431E774 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_10 0x431E778 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_11 0x431E77C + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_12 0x431E780 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_13 0x431E784 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_14 0x431E788 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_15 0x431E78C + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_16 0x431E790 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_17 0x431E794 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_18 0x431E798 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_19 0x431E79C + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_20 0x431E7A0 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_21 0x431E7A4 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_22 0x431E7A8 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_23 0x431E7AC + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_24 0x431E7B0 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_25 0x431E7B4 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_26 0x431E7B8 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_27 0x431E7BC + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_28 0x431E7C0 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_29 0x431E7C4 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_30 0x431E7C8 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_31 0x431E7CC + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_32 0x431E7D0 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_33 0x431E7D4 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_34 0x431E7D8 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_35 0x431E7DC + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_36 0x431E7E0 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_37 0x431E7E4 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_38 0x431E7E8 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_39 0x431E7EC + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_40 0x431E7F0 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_41 0x431E7F4 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_42 0x431E7F8 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_43 0x431E7FC + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_44 0x431E800 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_45 0x431E804 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_46 0x431E808 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_47 0x431E80C + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_48 0x431E810 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_49 0x431E814 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_50 0x431E818 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_51 0x431E81C + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_52 0x431E820 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_53 0x431E824 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_54 0x431E828 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_55 0x431E82C + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_56 0x431E830 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_57 0x431E834 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_58 0x431E838 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_59 0x431E83C + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_60 0x431E840 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_61 0x431E844 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_62 0x431E848 + +#define mmDCORE1_SYNC_MNGR_GLBL_LBW_DATA_63 0x431E84C + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_0 0x431E850 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_1 0x431E854 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_2 0x431E858 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_3 0x431E85C + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_4 0x431E860 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_5 0x431E864 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_6 0x431E868 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_7 0x431E86C + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_8 0x431E870 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_9 0x431E874 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_10 0x431E878 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_11 0x431E87C + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_12 0x431E880 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_13 0x431E884 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_14 0x431E888 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_15 0x431E88C + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_16 0x431E890 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_17 0x431E894 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_18 0x431E898 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_19 0x431E89C + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_20 0x431E8A0 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_21 0x431E8A4 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_22 0x431E8A8 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_23 0x431E8AC + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_24 0x431E8B0 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_25 0x431E8B4 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_26 0x431E8B8 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_27 0x431E8BC + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_28 0x431E8C0 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_29 0x431E8C4 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_30 0x431E8C8 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_31 0x431E8CC + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_32 0x431E8D0 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_33 0x431E8D4 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_34 0x431E8D8 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_35 0x431E8DC + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_36 0x431E8E0 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_37 0x431E8E4 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_38 0x431E8E8 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_39 0x431E8EC + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_40 0x431E8F0 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_41 0x431E8F4 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_42 0x431E8F8 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_43 0x431E8FC + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_44 0x431E900 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_45 0x431E904 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_46 0x431E908 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_47 0x431E90C + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_48 0x431E910 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_49 0x431E914 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_50 0x431E918 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_51 0x431E91C + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_52 0x431E920 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_53 0x431E924 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_54 0x431E928 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_55 0x431E92C + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_56 0x431E930 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_57 0x431E934 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_58 0x431E938 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_59 0x431E93C + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_60 0x431E940 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_61 0x431E944 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_62 0x431E948 + +#define mmDCORE1_SYNC_MNGR_GLBL_CQ_INC_MODE_63 0x431E94C + +#endif /* ASIC_REG_DCORE1_SYNC_MNGR_GLBL_REGS_H_ */ diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore3_mme_ctrl_lo_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore3_mme_ctrl_lo_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/dcore3_mme_ctrl_lo_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/dcore3_mme_ctrl_lo_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/gaudi2_blocks_linux_driver.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/gaudi2_blocks_linux_driver.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/gaudi2_blocks_linux_driver.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/gaudi2_blocks_linux_driver.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/gaudi2_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/gaudi2_regs.h similarity index 99% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/gaudi2_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/gaudi2_regs.h index 6aa1b14124629e6fa27980e779fbfdc79f2494d8..0bf3092bfeea321fe1c64e504b907a82a193d70c 100644 --- a/drivers/misc/habanalabs/include/gaudi2/asic_reg/gaudi2_regs.h +++ b/drivers/accel/habanalabs/include/gaudi2/asic_reg/gaudi2_regs.h @@ -31,6 +31,7 @@ #include "dcore0_sync_mngr_objs_regs.h" #include "dcore0_sync_mngr_glbl_regs.h" #include "dcore0_sync_mngr_mstr_if_axuser_regs.h" +#include "dcore1_sync_mngr_glbl_regs.h" #include "pdma0_qm_arc_aux_regs.h" #include "pdma0_core_ctx_regs.h" #include "pdma0_core_regs.h" diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/nic0_qm0_cgm_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/nic0_qm0_cgm_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/nic0_qm0_cgm_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/nic0_qm0_cgm_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/nic0_qm0_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/nic0_qm0_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/nic0_qm0_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/nic0_qm0_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/nic0_qm_arc_aux0_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/nic0_qm_arc_aux0_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/nic0_qm_arc_aux0_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/nic0_qm_arc_aux0_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/nic0_qpc0_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/nic0_qpc0_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/nic0_qpc0_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/nic0_qpc0_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/nic0_umr0_0_completion_queue_ci_1_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/nic0_umr0_0_completion_queue_ci_1_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/nic0_umr0_0_completion_queue_ci_1_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/nic0_umr0_0_completion_queue_ci_1_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/nic0_umr0_0_unsecure_doorbell0_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/nic0_umr0_0_unsecure_doorbell0_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/nic0_umr0_0_unsecure_doorbell0_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/nic0_umr0_0_unsecure_doorbell0_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/pcie_aux_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/pcie_aux_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/pcie_aux_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/pcie_aux_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/pcie_dbi_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/pcie_dbi_regs.h similarity index 99% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/pcie_dbi_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/pcie_dbi_regs.h index cc5842ec6ceb54b91ccec0c37f73a00eb4e21e29..2ee79d8e62d09f8e55f841ef52fa114271a3d6b7 100644 --- a/drivers/misc/habanalabs/include/gaudi2/asic_reg/pcie_dbi_regs.h +++ b/drivers/accel/habanalabs/include/gaudi2/asic_reg/pcie_dbi_regs.h @@ -48,8 +48,7 @@ #define mmPCIE_DBI_PCI_CAP_PTR_REG 0x4C02034 -#define mmPCIE_DBI_MAX_LATENCY_MIN_GRANT_INTERRUPT_PIN_INTERRUPT_LINE_REG \ -0x4C0203C +#define mmPCIE_DBI_MAX_LATENCY_MIN_GRANT_INTERRUPT_PIN_INTERRUPT_LINE_REG 0x4C0203C #define mmPCIE_DBI_CAP_ID_NXT_PTR_REG 0x4C02040 diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/pcie_dec0_cmd_masks.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/pcie_dec0_cmd_masks.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/pcie_dec0_cmd_masks.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/pcie_dec0_cmd_masks.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/pcie_dec0_cmd_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/pcie_dec0_cmd_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/pcie_dec0_cmd_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/pcie_dec0_cmd_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/pcie_vdec0_brdg_ctrl_axuser_dec_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/pcie_vdec0_brdg_ctrl_axuser_dec_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/pcie_vdec0_brdg_ctrl_axuser_dec_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/pcie_vdec0_brdg_ctrl_axuser_dec_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/pcie_vdec0_brdg_ctrl_axuser_msix_abnrm_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/pcie_vdec0_brdg_ctrl_axuser_msix_abnrm_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/pcie_vdec0_brdg_ctrl_axuser_msix_abnrm_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/pcie_vdec0_brdg_ctrl_axuser_msix_abnrm_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/pcie_vdec0_brdg_ctrl_axuser_msix_l2c_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/pcie_vdec0_brdg_ctrl_axuser_msix_l2c_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/pcie_vdec0_brdg_ctrl_axuser_msix_l2c_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/pcie_vdec0_brdg_ctrl_axuser_msix_l2c_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/pcie_vdec0_brdg_ctrl_axuser_msix_nrm_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/pcie_vdec0_brdg_ctrl_axuser_msix_nrm_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/pcie_vdec0_brdg_ctrl_axuser_msix_nrm_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/pcie_vdec0_brdg_ctrl_axuser_msix_nrm_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/pcie_vdec0_brdg_ctrl_axuser_msix_vcd_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/pcie_vdec0_brdg_ctrl_axuser_msix_vcd_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/pcie_vdec0_brdg_ctrl_axuser_msix_vcd_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/pcie_vdec0_brdg_ctrl_axuser_msix_vcd_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/pcie_vdec0_brdg_ctrl_masks.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/pcie_vdec0_brdg_ctrl_masks.h similarity index 99% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/pcie_vdec0_brdg_ctrl_masks.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/pcie_vdec0_brdg_ctrl_masks.h index d29837883216a054b95a4fdce0f6020947b09015..7a96aebf08b33ce65c23ac313eb1cef95a779792 100644 --- a/drivers/misc/habanalabs/include/gaudi2/asic_reg/pcie_vdec0_brdg_ctrl_masks.h +++ b/drivers/accel/habanalabs/include/gaudi2/asic_reg/pcie_vdec0_brdg_ctrl_masks.h @@ -116,8 +116,7 @@ #define PCIE_VDEC0_BRDG_CTRL_HBW_AXI_VIOL_CAUSE_ARBURST_VIOL_SHIFT 7 #define PCIE_VDEC0_BRDG_CTRL_HBW_AXI_VIOL_CAUSE_ARBURST_VIOL_MASK 0x80 #define PCIE_VDEC0_BRDG_CTRL_HBW_AXI_VIOL_CAUSE_ARADDR_SIZE_ALIGN_VIOL_SHIFT 8 -#define PCIE_VDEC0_BRDG_CTRL_HBW_AXI_VIOL_CAUSE_ARADDR_SIZE_ALIGN_VIOL_MASK \ -0x100 +#define PCIE_VDEC0_BRDG_CTRL_HBW_AXI_VIOL_CAUSE_ARADDR_SIZE_ALIGN_VIOL_MASK 0x100 #define PCIE_VDEC0_BRDG_CTRL_HBW_AXI_VIOL_CAUSE_ARSIZE_VIOL_SHIFT 9 #define PCIE_VDEC0_BRDG_CTRL_HBW_AXI_VIOL_CAUSE_ARSIZE_VIOL_MASK 0x200 diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/pcie_vdec0_brdg_ctrl_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/pcie_vdec0_brdg_ctrl_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/pcie_vdec0_brdg_ctrl_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/pcie_vdec0_brdg_ctrl_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/pcie_vdec0_ctrl_special_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/pcie_vdec0_ctrl_special_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/pcie_vdec0_ctrl_special_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/pcie_vdec0_ctrl_special_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/pcie_wrap_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/pcie_wrap_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/pcie_wrap_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/pcie_wrap_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/pcie_wrap_special_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/pcie_wrap_special_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/pcie_wrap_special_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/pcie_wrap_special_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/pdma0_core_ctx_axuser_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/pdma0_core_ctx_axuser_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/pdma0_core_ctx_axuser_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/pdma0_core_ctx_axuser_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/pdma0_core_ctx_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/pdma0_core_ctx_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/pdma0_core_ctx_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/pdma0_core_ctx_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/pdma0_core_masks.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/pdma0_core_masks.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/pdma0_core_masks.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/pdma0_core_masks.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/pdma0_core_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/pdma0_core_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/pdma0_core_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/pdma0_core_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/pdma0_core_special_masks.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/pdma0_core_special_masks.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/pdma0_core_special_masks.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/pdma0_core_special_masks.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/pdma0_qm_arc_aux_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/pdma0_qm_arc_aux_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/pdma0_qm_arc_aux_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/pdma0_qm_arc_aux_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/pdma0_qm_axuser_nonsecured_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/pdma0_qm_axuser_nonsecured_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/pdma0_qm_axuser_nonsecured_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/pdma0_qm_axuser_nonsecured_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/pdma0_qm_axuser_secured_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/pdma0_qm_axuser_secured_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/pdma0_qm_axuser_secured_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/pdma0_qm_axuser_secured_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/pdma0_qm_cgm_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/pdma0_qm_cgm_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/pdma0_qm_cgm_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/pdma0_qm_cgm_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/pdma0_qm_masks.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/pdma0_qm_masks.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/pdma0_qm_masks.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/pdma0_qm_masks.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/pdma0_qm_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/pdma0_qm_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/pdma0_qm_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/pdma0_qm_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/pdma1_core_ctx_axuser_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/pdma1_core_ctx_axuser_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/pdma1_core_ctx_axuser_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/pdma1_core_ctx_axuser_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/pdma1_qm_axuser_nonsecured_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/pdma1_qm_axuser_nonsecured_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/pdma1_qm_axuser_nonsecured_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/pdma1_qm_axuser_nonsecured_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/pmmu_hbw_stlb_masks.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/pmmu_hbw_stlb_masks.h similarity index 99% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/pmmu_hbw_stlb_masks.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/pmmu_hbw_stlb_masks.h index 0276506ea52399aa272927755539040efc0178fb..b4f32632cd362d5845dff5c812f415250bc29989 100644 --- a/drivers/misc/habanalabs/include/gaudi2/asic_reg/pmmu_hbw_stlb_masks.h +++ b/drivers/accel/habanalabs/include/gaudi2/asic_reg/pmmu_hbw_stlb_masks.h @@ -228,8 +228,7 @@ /* PMMU_HBW_STLB_RANGE_CACHE_INVALIDATION */ #define PMMU_HBW_STLB_RANGE_CACHE_INVALIDATION_RANGE_INVALIDATION_ENABLE_SHIFT 0 -#define PMMU_HBW_STLB_RANGE_CACHE_INVALIDATION_RANGE_INVALIDATION_ENABLE_MASK \ -0x1 +#define PMMU_HBW_STLB_RANGE_CACHE_INVALIDATION_RANGE_INVALIDATION_ENABLE_MASK 0x1 #define PMMU_HBW_STLB_RANGE_CACHE_INVALIDATION_INVALIDATION_ASID_EN_SHIFT 1 #define PMMU_HBW_STLB_RANGE_CACHE_INVALIDATION_INVALIDATION_ASID_EN_MASK 0x2 #define PMMU_HBW_STLB_RANGE_CACHE_INVALIDATION_INVALIDATION_ASID_SHIFT 2 diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/pmmu_hbw_stlb_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/pmmu_hbw_stlb_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/pmmu_hbw_stlb_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/pmmu_hbw_stlb_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/pmmu_pif_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/pmmu_pif_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/pmmu_pif_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/pmmu_pif_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/psoc_etr_masks.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/psoc_etr_masks.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/psoc_etr_masks.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/psoc_etr_masks.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/psoc_etr_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/psoc_etr_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/psoc_etr_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/psoc_etr_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/psoc_global_conf_masks.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/psoc_global_conf_masks.h similarity index 99% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/psoc_global_conf_masks.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/psoc_global_conf_masks.h index 9be3d656da3a8d3b505b9ccea6e80fb6a316a10f..85a81e2cb546399ea87e58f5bf704d573ebe21ec 100644 --- a/drivers/misc/habanalabs/include/gaudi2/asic_reg/psoc_global_conf_masks.h +++ b/drivers/accel/habanalabs/include/gaudi2/asic_reg/psoc_global_conf_masks.h @@ -1306,11 +1306,9 @@ #define PSOC_GLOBAL_CONF_AXI_DRAIN_NL_SRC_CTRL0_ADDR_EXTMEM_PC_LOC2_SHIFT 12 #define PSOC_GLOBAL_CONF_AXI_DRAIN_NL_SRC_CTRL0_ADDR_EXTMEM_PC_LOC2_MASK 0x3F000 #define PSOC_GLOBAL_CONF_AXI_DRAIN_NL_SRC_CTRL0_ADDR_EXTMEM_PC_LOC3_SHIFT 18 -#define PSOC_GLOBAL_CONF_AXI_DRAIN_NL_SRC_CTRL0_ADDR_EXTMEM_PC_LOC3_MASK \ -0xFC0000 +#define PSOC_GLOBAL_CONF_AXI_DRAIN_NL_SRC_CTRL0_ADDR_EXTMEM_PC_LOC3_MASK 0xFC0000 #define PSOC_GLOBAL_CONF_AXI_DRAIN_NL_SRC_CTRL0_ADDR_EXTMEM_HBM_LOC0_SHIFT 24 -#define PSOC_GLOBAL_CONF_AXI_DRAIN_NL_SRC_CTRL0_ADDR_EXTMEM_HBM_LOC0_MASK \ -0x3F000000 +#define PSOC_GLOBAL_CONF_AXI_DRAIN_NL_SRC_CTRL0_ADDR_EXTMEM_HBM_LOC0_MASK 0x3F000000 /* PSOC_GLOBAL_CONF_AXI_DRAIN_NL_SRC_CTRL1 */ #define PSOC_GLOBAL_CONF_AXI_DRAIN_NL_SRC_CTRL1_ADDR_EXTMEM_HBM_LOC1_SHIFT 0 @@ -1322,24 +1320,17 @@ #define PSOC_GLOBAL_CONF_AXI_DRAIN_NL_SRC_CTRL1_NON_LIN_HBM_CNT_EN_SHIFT 13 #define PSOC_GLOBAL_CONF_AXI_DRAIN_NL_SRC_CTRL1_NON_LIN_HBM_CNT_EN_MASK 0x2000 #define PSOC_GLOBAL_CONF_AXI_DRAIN_NL_SRC_CTRL1_NON_LIN_HBM_ALL_ADDR_EN_SHIFT 14 -#define PSOC_GLOBAL_CONF_AXI_DRAIN_NL_SRC_CTRL1_NON_LIN_HBM_ALL_ADDR_EN_MASK \ -0x4000 -#define PSOC_GLOBAL_CONF_AXI_DRAIN_NL_SRC_CTRL1_NON_LIN_HBM_ALL_ADDR_MASK_SHIFT \ -16 -#define PSOC_GLOBAL_CONF_AXI_DRAIN_NL_SRC_CTRL1_NON_LIN_HBM_ALL_ADDR_MASK_MASK \ -0xFF0000 +#define PSOC_GLOBAL_CONF_AXI_DRAIN_NL_SRC_CTRL1_NON_LIN_HBM_ALL_ADDR_EN_MASK 0x4000 +#define PSOC_GLOBAL_CONF_AXI_DRAIN_NL_SRC_CTRL1_NON_LIN_HBM_ALL_ADDR_MASK_SHIFT 16 +#define PSOC_GLOBAL_CONF_AXI_DRAIN_NL_SRC_CTRL1_NON_LIN_HBM_ALL_ADDR_MASK_MASK 0xFF0000 #define PSOC_GLOBAL_CONF_AXI_DRAIN_NL_SRC_CTRL1_HBM_NUM_SHIFT 24 #define PSOC_GLOBAL_CONF_AXI_DRAIN_NL_SRC_CTRL1_HBM_NUM_MASK 0x7000000 /* PSOC_GLOBAL_CONF_AXI_DRAIN_NL_SRC_CTRL2 */ -#define PSOC_GLOBAL_CONF_AXI_DRAIN_NL_SRC_CTRL2_SCRAM_NONLIN_HBM_CNT_MASK_SHIFT \ -0 -#define PSOC_GLOBAL_CONF_AXI_DRAIN_NL_SRC_CTRL2_SCRAM_NONLIN_HBM_CNT_MASK_MASK \ -0xFFFF -#define PSOC_GLOBAL_CONF_AXI_DRAIN_NL_SRC_CTRL2_SCRAM_NONLIN_EXTM_PC_MASK_SHIFT \ -16 -#define PSOC_GLOBAL_CONF_AXI_DRAIN_NL_SRC_CTRL2_SCRAM_NONLIN_EXTM_PC_MASK_MASK \ -0xFFFF0000 +#define PSOC_GLOBAL_CONF_AXI_DRAIN_NL_SRC_CTRL2_SCRAM_NONLIN_HBM_CNT_MASK_SHIFT 0 +#define PSOC_GLOBAL_CONF_AXI_DRAIN_NL_SRC_CTRL2_SCRAM_NONLIN_HBM_CNT_MASK_MASK 0xFFFF +#define PSOC_GLOBAL_CONF_AXI_DRAIN_NL_SRC_CTRL2_SCRAM_NONLIN_EXTM_PC_MASK_SHIFT 16 +#define PSOC_GLOBAL_CONF_AXI_DRAIN_NL_SRC_CTRL2_SCRAM_NONLIN_EXTM_PC_MASK_MASK 0xFFFF0000 /* PSOC_GLOBAL_CONF_AXI_DRAIN_NL_SRC_CTRL3 */ #define PSOC_GLOBAL_CONF_AXI_DRAIN_NL_SRC_CTRL3_HBM_MAP0_SHIFT 0 diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/psoc_global_conf_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/psoc_global_conf_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/psoc_global_conf_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/psoc_global_conf_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/psoc_reset_conf_masks.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/psoc_reset_conf_masks.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/psoc_reset_conf_masks.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/psoc_reset_conf_masks.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/psoc_reset_conf_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/psoc_reset_conf_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/psoc_reset_conf_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/psoc_reset_conf_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/psoc_timestamp_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/psoc_timestamp_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/psoc_timestamp_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/psoc_timestamp_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/rot0_desc_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/rot0_desc_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/rot0_desc_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/rot0_desc_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/rot0_masks.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/rot0_masks.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/rot0_masks.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/rot0_masks.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/rot0_qm_arc_aux_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/rot0_qm_arc_aux_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/rot0_qm_arc_aux_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/rot0_qm_arc_aux_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/rot0_qm_axuser_nonsecured_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/rot0_qm_axuser_nonsecured_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/rot0_qm_axuser_nonsecured_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/rot0_qm_axuser_nonsecured_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/rot0_qm_cgm_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/rot0_qm_cgm_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/rot0_qm_cgm_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/rot0_qm_cgm_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/rot0_qm_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/rot0_qm_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/rot0_qm_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/rot0_qm_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/rot0_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/rot0_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/rot0_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/rot0_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/xbar_edge_0_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/xbar_edge_0_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/xbar_edge_0_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/xbar_edge_0_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/asic_reg/xbar_mid_0_regs.h b/drivers/accel/habanalabs/include/gaudi2/asic_reg/xbar_mid_0_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/asic_reg/xbar_mid_0_regs.h rename to drivers/accel/habanalabs/include/gaudi2/asic_reg/xbar_mid_0_regs.h diff --git a/drivers/misc/habanalabs/include/gaudi2/gaudi2.h b/drivers/accel/habanalabs/include/gaudi2/gaudi2.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/gaudi2.h rename to drivers/accel/habanalabs/include/gaudi2/gaudi2.h diff --git a/drivers/misc/habanalabs/include/gaudi2/gaudi2_async_events.h b/drivers/accel/habanalabs/include/gaudi2/gaudi2_async_events.h similarity index 99% rename from drivers/misc/habanalabs/include/gaudi2/gaudi2_async_events.h rename to drivers/accel/habanalabs/include/gaudi2/gaudi2_async_events.h index 305b576222e6f39b26c14120a7a8e60019088fa7..50852cc803739ccfab9650a9d2c1efe8cfdd6051 100644 --- a/drivers/misc/habanalabs/include/gaudi2/gaudi2_async_events.h +++ b/drivers/accel/habanalabs/include/gaudi2/gaudi2_async_events.h @@ -958,6 +958,7 @@ enum gaudi2_async_event_id { GAUDI2_EVENT_CPU11_STATUS_NIC11_ENG1 = 1318, GAUDI2_EVENT_ARC_DCCM_FULL = 1319, GAUDI2_EVENT_CPU_FP32_NOT_SUPPORTED = 1320, + GAUDI2_EVENT_DEV_RESET_REQ = 1321, GAUDI2_EVENT_SIZE, }; diff --git a/drivers/misc/habanalabs/include/gaudi2/gaudi2_async_ids_map_extended.h b/drivers/accel/habanalabs/include/gaudi2/gaudi2_async_ids_map_extended.h similarity index 99% rename from drivers/misc/habanalabs/include/gaudi2/gaudi2_async_ids_map_extended.h rename to drivers/accel/habanalabs/include/gaudi2/gaudi2_async_ids_map_extended.h index d510cb10c883138de9a22c37d9bfddd14202d4c3..82be01bea98e549b79737cf74141bf41755b8b43 100644 --- a/drivers/misc/habanalabs/include/gaudi2/gaudi2_async_ids_map_extended.h +++ b/drivers/accel/habanalabs/include/gaudi2/gaudi2_async_ids_map_extended.h @@ -2665,6 +2665,8 @@ static struct gaudi2_async_events_ids_map gaudi2_irq_map_table[] = { .msg = 1, .reset = 0, .name = "ARC_DCCM_FULL" }, { .fc_id = 1320, .cpu_id = 626, .valid = 1, .msg = 1, .reset = 1, .name = "FP32_NOT_SUPPORTED" }, + { .fc_id = 1321, .cpu_id = 627, .valid = 1, + .msg = 1, .reset = 1, .name = "DEV_RESET_REQ" }, }; #endif /* __GAUDI2_ASYNC_IDS_MAP_EVENTS_EXT_H_ */ diff --git a/drivers/misc/habanalabs/include/gaudi2/gaudi2_coresight.h b/drivers/accel/habanalabs/include/gaudi2/gaudi2_coresight.h similarity index 100% rename from drivers/misc/habanalabs/include/gaudi2/gaudi2_coresight.h rename to drivers/accel/habanalabs/include/gaudi2/gaudi2_coresight.h diff --git a/drivers/misc/habanalabs/include/gaudi2/gaudi2_fw_if.h b/drivers/accel/habanalabs/include/gaudi2/gaudi2_fw_if.h similarity index 82% rename from drivers/misc/habanalabs/include/gaudi2/gaudi2_fw_if.h rename to drivers/accel/habanalabs/include/gaudi2/gaudi2_fw_if.h index e4a7d572509616cb0f4fced3b8c0b5c6152160dc..82f3ca2a3966e22dac64b535ba374712836c35cf 100644 --- a/drivers/misc/habanalabs/include/gaudi2/gaudi2_fw_if.h +++ b/drivers/accel/habanalabs/include/gaudi2/gaudi2_fw_if.h @@ -20,22 +20,25 @@ #define GAUDI2_NUM_MME 4 +#define NUM_OF_GPIOS_PER_PORT 16 +#define GAUDI2_WD_GPIO (62 % NUM_OF_GPIOS_PER_PORT) + #define GAUDI2_ARCPID_TX_MB_SIZE 0x1000 #define GAUDI2_ARCPID_RX_MB_SIZE 0x400 #define GAUDI2_ARM_TX_MB_SIZE 0x400 #define GAUDI2_ARM_RX_MB_SIZE 0x1800 #define GAUDI2_DCCM_BASE_ADDR 0x27020000 -#define GAUDI2_ARCPID_TX_MB_ADDR GAUDI2_DCCM_BASE_ADDR - -#define GAUDI2_ARCPID_RX_MB_ADDR (GAUDI2_ARCPID_TX_MB_ADDR + \ - GAUDI2_ARCPID_TX_MB_SIZE) #define GAUDI2_ARM_TX_MB_ADDR GAUDI2_MAILBOX_BASE_ADDR #define GAUDI2_ARM_RX_MB_ADDR (GAUDI2_ARM_TX_MB_ADDR + \ GAUDI2_ARM_TX_MB_SIZE) +#define GAUDI2_ARCPID_TX_MB_ADDR (GAUDI2_ARM_RX_MB_ADDR + GAUDI2_ARM_RX_MB_SIZE) + +#define GAUDI2_ARCPID_RX_MB_ADDR (GAUDI2_ARCPID_TX_MB_ADDR + GAUDI2_ARCPID_TX_MB_SIZE) + #define GAUDI2_ARM_TX_MB_OFFSET (GAUDI2_ARM_TX_MB_ADDR - \ GAUDI2_SP_SRAM_BASE_ADDR) @@ -58,7 +61,9 @@ struct gaudi2_cold_rst_data { u32 spsram_init_done : 1; u32 fake_security_enable : 1; u32 fake_sig_validation_en : 1; - u32 reserved : 26; + u32 bist_skip_enable : 1; + u32 bist_need_iatu_config : 1; + u32 reserved : 24; }; __le32 data; }; @@ -77,10 +82,10 @@ enum gaudi2_rst_src { }; struct gaudi2_redundancy_ctx { - int redundant_hbm; - int redundant_edma; - int redundant_tpc; - int redundant_vdec; + __le32 redundant_hbm; + __le32 redundant_edma; + __le32 redundant_tpc; + __le32 redundant_vdec; __le64 hbm_mask; __le64 edma_mask; __le64 tpc_mask; diff --git a/drivers/misc/habanalabs/include/gaudi2/gaudi2_packets.h b/drivers/accel/habanalabs/include/gaudi2/gaudi2_packets.h similarity index 98% rename from drivers/misc/habanalabs/include/gaudi2/gaudi2_packets.h rename to drivers/accel/habanalabs/include/gaudi2/gaudi2_packets.h index 8bf90fc18bf5273e1070f26c73210e26514502a7..a812f8503f90d3cdcfbe59607b37314d09f41507 100644 --- a/drivers/misc/habanalabs/include/gaudi2/gaudi2_packets.h +++ b/drivers/accel/habanalabs/include/gaudi2/gaudi2_packets.h @@ -59,7 +59,7 @@ struct gaudi2_packet { /* The rest of the packet data follows. Use the corresponding * packet_XXX struct to deference the data, based on packet type */ - u8 contents[0]; + u8 contents[]; }; struct packet_nop { @@ -80,7 +80,7 @@ struct packet_wreg32 { struct packet_wreg_bulk { __le32 size64; __le32 ctl; - __le64 values[0]; /* data starts here */ + __le64 values[]; /* data starts here */ }; struct packet_msg_long { diff --git a/drivers/misc/habanalabs/include/gaudi2/gaudi2_reg_map.h b/drivers/accel/habanalabs/include/gaudi2/gaudi2_reg_map.h similarity index 83% rename from drivers/misc/habanalabs/include/gaudi2/gaudi2_reg_map.h rename to drivers/accel/habanalabs/include/gaudi2/gaudi2_reg_map.h index ae7feb388f638b15ed82f8e0868fb86c90fbbd48..f3eaeb6d9b7ecb531b942cb4555690f759e84e05 100644 --- a/drivers/misc/habanalabs/include/gaudi2/gaudi2_reg_map.h +++ b/drivers/accel/habanalabs/include/gaudi2/gaudi2_reg_map.h @@ -24,14 +24,14 @@ #define mmGIC_HOST_HALT_IRQ_POLL_REG mmPSOC_GLOBAL_CONF_SCRATCHPAD_10 #define mmGIC_HOST_INTS_IRQ_POLL_REG mmPSOC_GLOBAL_CONF_SCRATCHPAD_11 #define mmGIC_HOST_SOFT_RST_IRQ_POLL_REG mmPSOC_GLOBAL_CONF_SCRATCHPAD_12 -#define mmEEPROM_COPY_LOCATION_REG mmPSOC_GLOBAL_CONF_SCRATCHPAD_13 #define mmCPU_RST_STATUS_TO_HOST mmPSOC_GLOBAL_CONF_SCRATCHPAD_14 -#define mmENGINE_ARC_IRQ_CTRL_POLL_REG mmPSOC_GLOBAL_CONF_SCRATCHPAD_15 -#define mmPID_CFG_REG mmPSOC_GLOBAL_CONF_SCRATCHPAD_18 /* - * TODO: mmGIC_RAZWI_STATUS_REG is temporary - * macro and to be removed after GAUDI2 PO + * Single scratchpad register used for all ARCs to notify dccm queue full event to FW. + * So a new event would overwrite any unhandled previous event. In other words, incase + * of multiple events before previous ones are handled, last one would be considered. */ +#define mmENGINE_ARC_IRQ_CTRL_POLL_REG mmPSOC_GLOBAL_CONF_SCRATCHPAD_15 +#define mmPID_CFG_REG mmPSOC_GLOBAL_CONF_SCRATCHPAD_18 #define mmGIC_RAZWI_STATUS_REG mmPSOC_GLOBAL_CONF_SCRATCHPAD_19 #define mmCPU_BOOT_DEV_STS0 mmPSOC_GLOBAL_CONF_SCRATCHPAD_20 #define mmCPU_BOOT_DEV_STS1 mmPSOC_GLOBAL_CONF_SCRATCHPAD_21 @@ -40,11 +40,10 @@ #define mmCPU_BOOT_ERR1 mmPSOC_GLOBAL_CONF_SCRATCHPAD_25 #define mmUPD_STS mmPSOC_GLOBAL_CONF_SCRATCHPAD_26 #define mmUPD_CMD mmPSOC_GLOBAL_CONF_SCRATCHPAD_27 -#define mmUBOOT_VER_OFFSET mmPSOC_GLOBAL_CONF_SCRATCHPAD_29 +#define mmPPBOOT_VER_OFFSET mmPSOC_GLOBAL_CONF_SCRATCHPAD_28 #define mmRDWR_TEST mmPSOC_GLOBAL_CONF_SCRATCHPAD_30 #define mmBTL_ID mmPSOC_GLOBAL_CONF_SCRATCHPAD_31 #define mmRST_SRC mmPSOC_GLOBAL_CONF_COLD_RST_FLOPS_0 -#define mmPREBOOT_PCIE_EN mmPSOC_GLOBAL_CONF_COLD_RST_FLOPS_1 #define mmCOLD_RST_DATA mmPSOC_GLOBAL_CONF_COLD_RST_FLOPS_2 #define mmUPD_PENDING_STS mmPSOC_GLOBAL_CONF_COLD_RST_FLOPS_3 #define mmPID_CMD_REQ_REG mmPSOC_PID_PID_CMD_0 @@ -55,5 +54,8 @@ #define mmPID_CMD_TELEMETRY_REG_0_HI mmPSOC_PID_PID_CMD_5 #define mmPID_CMD_TELEMETRY_REG_1 mmPSOC_PID_PID_CMD_6 #define mmPID_CMD_TELEMETRY_REG_1_HI mmPSOC_PID_PID_CMD_7 +#define mmWD_GPIO_OUTSET_REG mmPSOC_GPIO3_OUTENSET +#define mmWD_GPIO_DATAOUT_REG mmPSOC_GPIO3_DATAOUT +#define mmSTM_PROFILER_SPE_REG mmPSOC_STM_STMSPER #endif /* GAUDI2_REG_MAP_H_ */ diff --git a/drivers/accel/habanalabs/include/gaudi2/gaudi2_special_blocks.h b/drivers/accel/habanalabs/include/gaudi2/gaudi2_special_blocks.h new file mode 100644 index 0000000000000000000000000000000000000000..a55668f92dd133ec1332971a64ff5996e36601ed --- /dev/null +++ b/drivers/accel/habanalabs/include/gaudi2/gaudi2_special_blocks.h @@ -0,0 +1,157 @@ +/* SPDX-License-Identifier: GPL-2.0 + * + * Copyright 2022 HabanaLabs, Ltd. + * All Rights Reserved. + */ + +/* + * This file was generated automatically. + * DON'T EDIT THIS FILE. + */ + +#ifndef GAUDI2_SPECIAL_BLOCKS_H +#define GAUDI2_SPECIAL_BLOCKS_H + +#define GAUDI2_SPECIAL_BLOCKS { \ + { GAUDI2_BLOCK_TYPE_TPC, 0xfc008000, 4, 6, 0, 0x200000, 0x10000, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_TPC, 0xfc00a000, 4, 6, 0, 0x200000, 0x10000, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_TPC, 0xfc00b000, 4, 6, 0, 0x200000, 0x10000, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_TPC, 0xfc00c000, 4, 6, 0, 0x200000, 0x10000, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_HMMU, 0xfc080000, 4, 4, 0, 0x200000, 0x10000, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_HMMU, 0xfc081000, 4, 4, 0, 0x200000, 0x10000, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_HMMU, 0xfc083000, 4, 4, 0, 0x200000, 0x10000, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_HMMU, 0xfc084000, 4, 4, 0, 0x200000, 0x10000, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_MME, 0xfc0c8000, 4, 0, 0, 0x200000, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_MME, 0xfc0c9000, 4, 0, 0, 0x200000, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_MME, 0xfc0ca000, 4, 0, 0, 0x200000, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_MME, 0xfc0cb000, 4, 0, 0, 0x200000, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_MME, 0xfc0cc000, 4, 0, 0, 0x200000, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_EU_BIST, 0xfc0cd000, 4, 0, 0, 0x200000, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_MME, 0xfc0ce000, 4, 0, 0, 0x200000, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_MME, 0xfc0cf000, 4, 0, 0, 0x200000, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_MME, 0xfc0d0000, 4, 5, 0, 0x200000, 0x8000, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_MME, 0xfc0d1000, 4, 5, 0, 0x200000, 0x8000, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_MME, 0xfc0f8000, 4, 0, 0, 0x200000, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_MME, 0xfc0f9000, 4, 2, 0, 0x200000, 0x1000, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_SYNC_MNGR, 0xfc11e000, 4, 0, 0, 0x200000, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_SYNC_MNGR, 0xfc11f000, 4, 0, 0, 0x200000, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_HIF, 0xfc120000, 4, 4, 0, 0x200000, 0x4000, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_RTR, 0xfc140000, 4, 8, 0, 0x200000, 0x8000, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_RTR, 0xfc141000, 4, 8, 0, 0x200000, 0x8000, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_RTR, 0xfc142000, 4, 8, 0, 0x200000, 0x8000, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_RTR, 0xfc143000, 4, 8, 0, 0x200000, 0x8000, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_RTR, 0xfc144000, 4, 8, 0, 0x200000, 0x8000, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_RTR, 0xfc145000, 4, 8, 0, 0x200000, 0x8000, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_SRAM, 0xfc180000, 4, 8, 0, 0x200000, 0x8000, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_RTR, 0xfc181000, 4, 8, 0, 0x200000, 0x8000, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_SRAM, 0xfc182000, 4, 8, 0, 0x200000, 0x8000, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_EDMA, 0xfc1c8000, 4, 2, 0, 0x200000, 0x10000, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_EDMA, 0xfc1ca000, 4, 2, 0, 0x200000, 0x10000, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_EDMA, 0xfc1cb000, 4, 2, 0, 0x200000, 0x10000, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_EDMA, 0xfc1cc000, 4, 2, 0, 0x200000, 0x10000, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_DEC, 0xfc1e3000, 4, 2, 0, 0x200000, 0x10000, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_DEC, 0xfc1e4000, 4, 2, 0, 0x200000, 0x10000, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_DEC, 0xfc1e5000, 4, 2, 0, 0x200000, 0x10000, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_PCIE, 0xfcc01000, 1, 0, 0, 0x0, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_PCIE, 0xfcc04000, 1, 0, 0, 0x0, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_PCIE, 0xfcc07000, 1, 0, 0, 0x0, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_PCIE, 0xfcc10000, 1, 0, 0, 0x0, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_PCIE, 0xfcc14000, 1, 0, 0, 0x0, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_PCIE, 0xfcc15000, 1, 0, 0, 0x0, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_PCIE, 0xfcc16000, 1, 0, 0, 0x0, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_PSOC, 0xfcc4a000, 1, 0, 0, 0x0, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_PSOC, 0xfcc4b000, 1, 0, 0, 0x0, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_PSOC, 0xfcc4e000, 1, 0, 0, 0x0, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_PSOC, 0xfcc4f000, 1, 0, 0, 0x0, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_PSOC, 0xfcc53000, 1, 0, 0, 0x0, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_PSOC, 0xfcc54000, 2, 0, 0, 0x1000, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_PSOC, 0xfcc58000, 1, 0, 0, 0x0, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_PSOC, 0xfcc59000, 2, 0, 0, 0x3000, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_PSOC, 0xfcc5a000, 2, 0, 0, 0x3000, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_PSOC, 0xfcc5b000, 2, 0, 0, 0x3000, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_PSOC, 0xfcc60000, 1, 0, 0, 0x0, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_PSOC, 0xfcc61000, 1, 0, 0, 0x0, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_PSOC, 0xfcc62000, 1, 0, 0, 0x0, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_PSOC, 0xfcc63000, 1, 0, 0, 0x0, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_PSOC, 0xfcc64000, 3, 0, 0, 0x1000, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_PLL, 0xfcc6c000, 1, 0, 0, 0x0, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_PLL, 0xfcc6d000, 1, 0, 0, 0x0, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_PLL, 0xfcc6e000, 1, 0, 0, 0x0, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_PSOC, 0xfcc74000, 1, 0, 0, 0x0, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_PSOC, 0xfcc76000, 3, 0, 0, 0x1000, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_PSOC, 0xfcc79000, 2, 0, 0, 0x1000, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_PSOC, 0xfcc7b000, 3, 0, 0, 0x1000, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_PSOC, 0xfcc7f000, 1, 0, 0, 0x0, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_PDMA, 0xfcc88000, 2, 0, 0, 0x10000, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_PDMA, 0xfcc8a000, 2, 0, 0, 0x10000, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_PDMA, 0xfcc8b000, 2, 0, 0, 0x10000, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_PDMA, 0xfcc8c000, 2, 0, 0, 0x10000, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_CPU, 0xfccc0000, 1, 0, 0, 0x0, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_CPU, 0xfccc1000, 1, 0, 0, 0x0, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_CPU, 0xfccc3000, 1, 0, 0, 0x0, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_PMMU, 0xfcd00000, 1, 0, 0, 0x0, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_PMMU, 0xfcd01000, 1, 0, 0, 0x0, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_PMMU, 0xfcd02000, 1, 0, 0, 0x0, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_PMMU, 0xfcd03000, 1, 0, 0, 0x0, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_PLL, 0xfcd04000, 1, 0, 0, 0x0, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_PLL, 0xfcd05000, 1, 0, 0, 0x0, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_XBAR, 0xfcd40000, 4, 0, 0, 0x10000, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_PLL, 0xfcd41000, 4, 0, 0, 0x10000, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_PLL, 0xfcd42000, 4, 0, 0, 0x10000, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_PLL, 0xfcd43000, 4, 0, 0, 0x10000, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_PLL, 0xfcd44000, 2, 0, 0, 0x10000, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_XBAR, 0xfcd48000, 4, 0, 0, 0x10000, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_PLL, 0xfcd55000, 1, 0, 0, 0x0, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_PLL, 0xfcd64000, 1, 0, 0, 0x0, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_PLL, 0xfcd65000, 1, 0, 0, 0x0, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_PLL, 0xfcd74000, 1, 0, 0, 0x0, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_ROT, 0xfce08000, 2, 0, 0, 0x10000, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_ROT, 0xfce0a000, 2, 0, 0, 0x10000, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_ROT, 0xfce0b000, 2, 0, 0, 0x10000, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_ROT, 0xfce0c000, 2, 0, 0, 0x10000, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_RTR, 0xfce40000, 4, 2, 0, 0x10000, 0x4000, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_RTR, 0xfce41000, 4, 2, 0, 0x10000, 0x4000, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_RTR, 0xfce42000, 4, 2, 0, 0x10000, 0x4000, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_RTR, 0xfce43000, 4, 2, 0, 0x10000, 0x4000, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_RTR, 0xfce48000, 4, 0, 0, 0x10000, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_RTR, 0xfce49000, 4, 0, 0, 0x10000, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_RTR, 0xfce4a000, 4, 0, 0, 0x10000, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_RTR, 0xfce4b000, 4, 0, 0, 0x10000, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_RTR, 0xfce4c000, 4, 0, 0, 0x10000, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_ARC_FARM, 0xfce81000, 1, 0, 0, 0x0, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_ARC_FARM, 0xfce82000, 1, 0, 0, 0x0, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_ARC_FARM, 0xfce88000, 4, 0, 0, 0x20000, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_ARC_FARM, 0xfce89000, 4, 0, 0, 0x20000, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_ARC_FARM, 0xfce8b000, 1, 0, 0, 0x0, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_ARC_FARM, 0xfce8c000, 1, 0, 0, 0x0, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_ARC_FARM, 0xfce8f000, 4, 0, 0, 0x20000, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_DEC, 0xfcf03000, 1, 0, 0, 0x0, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_DEC, 0xfcf04000, 1, 0, 0, 0x0, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_DEC, 0xfcf05000, 1, 0, 0, 0x0, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_XFT, 0xfcf40000, 4, 0, 0, 0x10000, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_PLL, 0xfcf41000, 4, 0, 0, 0x10000, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_PLL, 0xfcf42000, 4, 0, 0, 0x10000, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_PLL, 0xfcf43000, 1, 0, 0, 0x0, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_PLL, 0xfcf53000, 1, 0, 0, 0x0, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_PLL, 0xfcf73000, 1, 0, 0, 0x0, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_HBM, 0xfd000000, 6, 2, 0, 0x80000, 0x20000, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_HBM, 0xfd001000, 6, 2, 8, 0x80000, 0x20000, 0x1000 }, \ + { GAUDI2_BLOCK_TYPE_HBM, 0xfd009000, 6, 2, 0, 0x80000, 0x20000, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_NIC, 0xfd400000, 12, 2, 15, 0x80000, 0x20000, 0x1000 }, \ + { GAUDI2_BLOCK_TYPE_NIC, 0xfd418000, 12, 2, 0, 0x80000, 0x20000, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_NIC, 0xfd41a000, 12, 2, 0, 0x80000, 0x20000, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_NIC, 0xfd41f000, 12, 2, 0, 0x80000, 0x20000, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_NIC, 0xfd448000, 12, 0, 0, 0x80000, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_NIC, 0xfd449000, 12, 0, 0, 0x80000, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_NIC, 0xfd44a000, 12, 2, 0, 0x80000, 0x1000, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_NIC, 0xfd44c000, 12, 2, 0, 0x80000, 0x1000, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_NIC, 0xfd450000, 12, 2, 0, 0x80000, 0x1000, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_NIC, 0xfd452000, 12, 2, 0, 0x80000, 0x1000, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_NIC, 0xfd454000, 12, 0, 0, 0x80000, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_NIC, 0xfd455000, 12, 0, 0, 0x80000, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_NIC, 0xfd460000, 12, 0, 0, 0x80000, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_NIC, 0xfd468000, 12, 0, 0, 0x80000, 0x0, 0x0 }, \ + { GAUDI2_BLOCK_TYPE_NIC, 0xfd469000, 12, 0, 0, 0x80000, 0x0, 0x0 }, \ +} + +#endif diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/cpu_ca53_cfg_masks.h b/drivers/accel/habanalabs/include/goya/asic_reg/cpu_ca53_cfg_masks.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/cpu_ca53_cfg_masks.h rename to drivers/accel/habanalabs/include/goya/asic_reg/cpu_ca53_cfg_masks.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/cpu_ca53_cfg_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/cpu_ca53_cfg_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/cpu_ca53_cfg_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/cpu_ca53_cfg_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/cpu_if_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/cpu_if_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/cpu_if_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/cpu_if_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/cpu_pll_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/cpu_pll_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/cpu_pll_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/cpu_pll_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/dma_ch_0_masks.h b/drivers/accel/habanalabs/include/goya/asic_reg/dma_ch_0_masks.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/dma_ch_0_masks.h rename to drivers/accel/habanalabs/include/goya/asic_reg/dma_ch_0_masks.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/dma_ch_0_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/dma_ch_0_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/dma_ch_0_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/dma_ch_0_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/dma_ch_1_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/dma_ch_1_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/dma_ch_1_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/dma_ch_1_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/dma_ch_2_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/dma_ch_2_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/dma_ch_2_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/dma_ch_2_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/dma_ch_3_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/dma_ch_3_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/dma_ch_3_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/dma_ch_3_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/dma_ch_4_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/dma_ch_4_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/dma_ch_4_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/dma_ch_4_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/dma_macro_masks.h b/drivers/accel/habanalabs/include/goya/asic_reg/dma_macro_masks.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/dma_macro_masks.h rename to drivers/accel/habanalabs/include/goya/asic_reg/dma_macro_masks.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/dma_macro_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/dma_macro_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/dma_macro_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/dma_macro_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/dma_nrtr_masks.h b/drivers/accel/habanalabs/include/goya/asic_reg/dma_nrtr_masks.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/dma_nrtr_masks.h rename to drivers/accel/habanalabs/include/goya/asic_reg/dma_nrtr_masks.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/dma_nrtr_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/dma_nrtr_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/dma_nrtr_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/dma_nrtr_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/dma_qm_0_masks.h b/drivers/accel/habanalabs/include/goya/asic_reg/dma_qm_0_masks.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/dma_qm_0_masks.h rename to drivers/accel/habanalabs/include/goya/asic_reg/dma_qm_0_masks.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/dma_qm_0_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/dma_qm_0_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/dma_qm_0_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/dma_qm_0_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/dma_qm_1_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/dma_qm_1_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/dma_qm_1_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/dma_qm_1_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/dma_qm_2_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/dma_qm_2_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/dma_qm_2_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/dma_qm_2_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/dma_qm_3_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/dma_qm_3_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/dma_qm_3_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/dma_qm_3_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/dma_qm_4_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/dma_qm_4_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/dma_qm_4_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/dma_qm_4_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/goya_blocks.h b/drivers/accel/habanalabs/include/goya/asic_reg/goya_blocks.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/goya_blocks.h rename to drivers/accel/habanalabs/include/goya/asic_reg/goya_blocks.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/goya_masks.h b/drivers/accel/habanalabs/include/goya/asic_reg/goya_masks.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/goya_masks.h rename to drivers/accel/habanalabs/include/goya/asic_reg/goya_masks.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/goya_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/goya_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/goya_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/goya_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/ic_pll_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/ic_pll_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/ic_pll_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/ic_pll_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/mc_pll_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/mc_pll_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/mc_pll_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/mc_pll_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/mme1_rtr_masks.h b/drivers/accel/habanalabs/include/goya/asic_reg/mme1_rtr_masks.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/mme1_rtr_masks.h rename to drivers/accel/habanalabs/include/goya/asic_reg/mme1_rtr_masks.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/mme1_rtr_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/mme1_rtr_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/mme1_rtr_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/mme1_rtr_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/mme2_rtr_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/mme2_rtr_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/mme2_rtr_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/mme2_rtr_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/mme3_rtr_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/mme3_rtr_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/mme3_rtr_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/mme3_rtr_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/mme4_rtr_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/mme4_rtr_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/mme4_rtr_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/mme4_rtr_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/mme5_rtr_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/mme5_rtr_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/mme5_rtr_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/mme5_rtr_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/mme6_rtr_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/mme6_rtr_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/mme6_rtr_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/mme6_rtr_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/mme_cmdq_masks.h b/drivers/accel/habanalabs/include/goya/asic_reg/mme_cmdq_masks.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/mme_cmdq_masks.h rename to drivers/accel/habanalabs/include/goya/asic_reg/mme_cmdq_masks.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/mme_cmdq_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/mme_cmdq_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/mme_cmdq_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/mme_cmdq_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/mme_masks.h b/drivers/accel/habanalabs/include/goya/asic_reg/mme_masks.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/mme_masks.h rename to drivers/accel/habanalabs/include/goya/asic_reg/mme_masks.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/mme_qm_masks.h b/drivers/accel/habanalabs/include/goya/asic_reg/mme_qm_masks.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/mme_qm_masks.h rename to drivers/accel/habanalabs/include/goya/asic_reg/mme_qm_masks.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/mme_qm_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/mme_qm_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/mme_qm_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/mme_qm_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/mme_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/mme_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/mme_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/mme_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/mmu_masks.h b/drivers/accel/habanalabs/include/goya/asic_reg/mmu_masks.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/mmu_masks.h rename to drivers/accel/habanalabs/include/goya/asic_reg/mmu_masks.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/mmu_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/mmu_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/mmu_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/mmu_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/pci_nrtr_masks.h b/drivers/accel/habanalabs/include/goya/asic_reg/pci_nrtr_masks.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/pci_nrtr_masks.h rename to drivers/accel/habanalabs/include/goya/asic_reg/pci_nrtr_masks.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/pci_nrtr_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/pci_nrtr_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/pci_nrtr_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/pci_nrtr_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/pcie_aux_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/pcie_aux_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/pcie_aux_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/pcie_aux_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/pcie_wrap_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/pcie_wrap_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/pcie_wrap_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/pcie_wrap_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/psoc_emmc_pll_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/psoc_emmc_pll_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/psoc_emmc_pll_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/psoc_emmc_pll_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/psoc_etr_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/psoc_etr_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/psoc_etr_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/psoc_etr_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/psoc_global_conf_masks.h b/drivers/accel/habanalabs/include/goya/asic_reg/psoc_global_conf_masks.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/psoc_global_conf_masks.h rename to drivers/accel/habanalabs/include/goya/asic_reg/psoc_global_conf_masks.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/psoc_global_conf_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/psoc_global_conf_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/psoc_global_conf_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/psoc_global_conf_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/psoc_mme_pll_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/psoc_mme_pll_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/psoc_mme_pll_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/psoc_mme_pll_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/psoc_pci_pll_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/psoc_pci_pll_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/psoc_pci_pll_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/psoc_pci_pll_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/psoc_spi_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/psoc_spi_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/psoc_spi_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/psoc_spi_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/psoc_timestamp_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/psoc_timestamp_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/psoc_timestamp_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/psoc_timestamp_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/sram_y0_x0_rtr_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/sram_y0_x0_rtr_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/sram_y0_x0_rtr_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/sram_y0_x0_rtr_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/sram_y0_x1_rtr_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/sram_y0_x1_rtr_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/sram_y0_x1_rtr_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/sram_y0_x1_rtr_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/sram_y0_x2_rtr_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/sram_y0_x2_rtr_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/sram_y0_x2_rtr_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/sram_y0_x2_rtr_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/sram_y0_x3_rtr_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/sram_y0_x3_rtr_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/sram_y0_x3_rtr_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/sram_y0_x3_rtr_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/sram_y0_x4_rtr_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/sram_y0_x4_rtr_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/sram_y0_x4_rtr_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/sram_y0_x4_rtr_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/stlb_masks.h b/drivers/accel/habanalabs/include/goya/asic_reg/stlb_masks.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/stlb_masks.h rename to drivers/accel/habanalabs/include/goya/asic_reg/stlb_masks.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/stlb_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/stlb_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/stlb_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/stlb_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/tpc0_cfg_masks.h b/drivers/accel/habanalabs/include/goya/asic_reg/tpc0_cfg_masks.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/tpc0_cfg_masks.h rename to drivers/accel/habanalabs/include/goya/asic_reg/tpc0_cfg_masks.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/tpc0_cfg_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/tpc0_cfg_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/tpc0_cfg_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/tpc0_cfg_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/tpc0_cmdq_masks.h b/drivers/accel/habanalabs/include/goya/asic_reg/tpc0_cmdq_masks.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/tpc0_cmdq_masks.h rename to drivers/accel/habanalabs/include/goya/asic_reg/tpc0_cmdq_masks.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/tpc0_cmdq_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/tpc0_cmdq_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/tpc0_cmdq_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/tpc0_cmdq_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/tpc0_eml_cfg_masks.h b/drivers/accel/habanalabs/include/goya/asic_reg/tpc0_eml_cfg_masks.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/tpc0_eml_cfg_masks.h rename to drivers/accel/habanalabs/include/goya/asic_reg/tpc0_eml_cfg_masks.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/tpc0_eml_cfg_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/tpc0_eml_cfg_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/tpc0_eml_cfg_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/tpc0_eml_cfg_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/tpc0_nrtr_masks.h b/drivers/accel/habanalabs/include/goya/asic_reg/tpc0_nrtr_masks.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/tpc0_nrtr_masks.h rename to drivers/accel/habanalabs/include/goya/asic_reg/tpc0_nrtr_masks.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/tpc0_nrtr_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/tpc0_nrtr_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/tpc0_nrtr_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/tpc0_nrtr_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/tpc0_qm_masks.h b/drivers/accel/habanalabs/include/goya/asic_reg/tpc0_qm_masks.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/tpc0_qm_masks.h rename to drivers/accel/habanalabs/include/goya/asic_reg/tpc0_qm_masks.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/tpc0_qm_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/tpc0_qm_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/tpc0_qm_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/tpc0_qm_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/tpc1_cfg_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/tpc1_cfg_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/tpc1_cfg_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/tpc1_cfg_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/tpc1_cmdq_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/tpc1_cmdq_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/tpc1_cmdq_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/tpc1_cmdq_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/tpc1_qm_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/tpc1_qm_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/tpc1_qm_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/tpc1_qm_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/tpc1_rtr_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/tpc1_rtr_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/tpc1_rtr_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/tpc1_rtr_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/tpc2_cfg_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/tpc2_cfg_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/tpc2_cfg_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/tpc2_cfg_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/tpc2_cmdq_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/tpc2_cmdq_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/tpc2_cmdq_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/tpc2_cmdq_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/tpc2_qm_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/tpc2_qm_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/tpc2_qm_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/tpc2_qm_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/tpc2_rtr_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/tpc2_rtr_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/tpc2_rtr_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/tpc2_rtr_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/tpc3_cfg_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/tpc3_cfg_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/tpc3_cfg_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/tpc3_cfg_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/tpc3_cmdq_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/tpc3_cmdq_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/tpc3_cmdq_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/tpc3_cmdq_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/tpc3_qm_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/tpc3_qm_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/tpc3_qm_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/tpc3_qm_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/tpc3_rtr_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/tpc3_rtr_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/tpc3_rtr_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/tpc3_rtr_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/tpc4_cfg_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/tpc4_cfg_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/tpc4_cfg_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/tpc4_cfg_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/tpc4_cmdq_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/tpc4_cmdq_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/tpc4_cmdq_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/tpc4_cmdq_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/tpc4_qm_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/tpc4_qm_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/tpc4_qm_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/tpc4_qm_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/tpc4_rtr_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/tpc4_rtr_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/tpc4_rtr_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/tpc4_rtr_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/tpc5_cfg_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/tpc5_cfg_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/tpc5_cfg_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/tpc5_cfg_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/tpc5_cmdq_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/tpc5_cmdq_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/tpc5_cmdq_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/tpc5_cmdq_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/tpc5_qm_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/tpc5_qm_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/tpc5_qm_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/tpc5_qm_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/tpc5_rtr_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/tpc5_rtr_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/tpc5_rtr_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/tpc5_rtr_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/tpc6_cfg_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/tpc6_cfg_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/tpc6_cfg_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/tpc6_cfg_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/tpc6_cmdq_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/tpc6_cmdq_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/tpc6_cmdq_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/tpc6_cmdq_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/tpc6_qm_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/tpc6_qm_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/tpc6_qm_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/tpc6_qm_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/tpc6_rtr_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/tpc6_rtr_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/tpc6_rtr_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/tpc6_rtr_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/tpc7_cfg_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/tpc7_cfg_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/tpc7_cfg_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/tpc7_cfg_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/tpc7_cmdq_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/tpc7_cmdq_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/tpc7_cmdq_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/tpc7_cmdq_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/tpc7_nrtr_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/tpc7_nrtr_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/tpc7_nrtr_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/tpc7_nrtr_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/tpc7_qm_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/tpc7_qm_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/tpc7_qm_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/tpc7_qm_regs.h diff --git a/drivers/misc/habanalabs/include/goya/asic_reg/tpc_pll_regs.h b/drivers/accel/habanalabs/include/goya/asic_reg/tpc_pll_regs.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/asic_reg/tpc_pll_regs.h rename to drivers/accel/habanalabs/include/goya/asic_reg/tpc_pll_regs.h diff --git a/drivers/misc/habanalabs/include/goya/goya.h b/drivers/accel/habanalabs/include/goya/goya.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/goya.h rename to drivers/accel/habanalabs/include/goya/goya.h diff --git a/drivers/misc/habanalabs/include/goya/goya_async_events.h b/drivers/accel/habanalabs/include/goya/goya_async_events.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/goya_async_events.h rename to drivers/accel/habanalabs/include/goya/goya_async_events.h diff --git a/drivers/misc/habanalabs/include/goya/goya_coresight.h b/drivers/accel/habanalabs/include/goya/goya_coresight.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/goya_coresight.h rename to drivers/accel/habanalabs/include/goya/goya_coresight.h diff --git a/drivers/misc/habanalabs/include/goya/goya_fw_if.h b/drivers/accel/habanalabs/include/goya/goya_fw_if.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/goya_fw_if.h rename to drivers/accel/habanalabs/include/goya/goya_fw_if.h diff --git a/drivers/misc/habanalabs/include/goya/goya_packets.h b/drivers/accel/habanalabs/include/goya/goya_packets.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/goya_packets.h rename to drivers/accel/habanalabs/include/goya/goya_packets.h diff --git a/drivers/misc/habanalabs/include/goya/goya_reg_map.h b/drivers/accel/habanalabs/include/goya/goya_reg_map.h similarity index 100% rename from drivers/misc/habanalabs/include/goya/goya_reg_map.h rename to drivers/accel/habanalabs/include/goya/goya_reg_map.h diff --git a/drivers/misc/habanalabs/include/hw_ip/mmu/mmu_general.h b/drivers/accel/habanalabs/include/hw_ip/mmu/mmu_general.h similarity index 100% rename from drivers/misc/habanalabs/include/hw_ip/mmu/mmu_general.h rename to drivers/accel/habanalabs/include/hw_ip/mmu/mmu_general.h diff --git a/drivers/misc/habanalabs/include/hw_ip/mmu/mmu_v1_0.h b/drivers/accel/habanalabs/include/hw_ip/mmu/mmu_v1_0.h similarity index 100% rename from drivers/misc/habanalabs/include/hw_ip/mmu/mmu_v1_0.h rename to drivers/accel/habanalabs/include/hw_ip/mmu/mmu_v1_0.h diff --git a/drivers/misc/habanalabs/include/hw_ip/mmu/mmu_v1_1.h b/drivers/accel/habanalabs/include/hw_ip/mmu/mmu_v1_1.h similarity index 100% rename from drivers/misc/habanalabs/include/hw_ip/mmu/mmu_v1_1.h rename to drivers/accel/habanalabs/include/hw_ip/mmu/mmu_v1_1.h diff --git a/drivers/misc/habanalabs/include/hw_ip/mmu/mmu_v2_0.h b/drivers/accel/habanalabs/include/hw_ip/mmu/mmu_v2_0.h similarity index 100% rename from drivers/misc/habanalabs/include/hw_ip/mmu/mmu_v2_0.h rename to drivers/accel/habanalabs/include/hw_ip/mmu/mmu_v2_0.h diff --git a/drivers/misc/habanalabs/include/hw_ip/pci/pci_general.h b/drivers/accel/habanalabs/include/hw_ip/pci/pci_general.h similarity index 100% rename from drivers/misc/habanalabs/include/hw_ip/pci/pci_general.h rename to drivers/accel/habanalabs/include/hw_ip/pci/pci_general.h diff --git a/drivers/accessibility/speakup/spk_ttyio.c b/drivers/accessibility/speakup/spk_ttyio.c index 08cf8a17754bb6147e85d5932a063efd446ed29a..07373b3debd1e61efd293d51598545769503f29f 100644 --- a/drivers/accessibility/speakup/spk_ttyio.c +++ b/drivers/accessibility/speakup/spk_ttyio.c @@ -354,6 +354,9 @@ void spk_ttyio_release(struct spk_synth *in_synth) { struct tty_struct *tty = in_synth->dev; + if (tty == NULL) + return; + tty_lock(tty); if (tty->ops->close) diff --git a/drivers/acpi/glue.c b/drivers/acpi/glue.c index 204fe94c7e458c725627422f4eb7bea9a110cf1e..a194f30876c59a9f8d09fca5aec15b40e062546d 100644 --- a/drivers/acpi/glue.c +++ b/drivers/acpi/glue.c @@ -75,7 +75,8 @@ static struct acpi_bus_type *acpi_get_bus_type(struct device *dev) } #define FIND_CHILD_MIN_SCORE 1 -#define FIND_CHILD_MAX_SCORE 2 +#define FIND_CHILD_MID_SCORE 2 +#define FIND_CHILD_MAX_SCORE 3 static int match_any(struct acpi_device *adev, void *not_used) { @@ -96,8 +97,17 @@ static int find_child_checks(struct acpi_device *adev, bool check_children) return -ENODEV; status = acpi_evaluate_integer(adev->handle, "_STA", NULL, &sta); - if (status == AE_NOT_FOUND) + if (status == AE_NOT_FOUND) { + /* + * Special case: backlight device objects without _STA are + * preferred to other objects with the same _ADR value, because + * it is more likely that they are actually useful. + */ + if (adev->pnp.type.backlight) + return FIND_CHILD_MID_SCORE; + return FIND_CHILD_MIN_SCORE; + } if (ACPI_FAILURE(status) || !(sta & ACPI_STA_DEVICE_ENABLED)) return -ENODEV; diff --git a/drivers/acpi/prmt.c b/drivers/acpi/prmt.c index 998101cf16e4714552d3921681a7fe0c53338926..3d4c4620f9f953096dcf265ccf9c36c8c1cf15da 100644 --- a/drivers/acpi/prmt.c +++ b/drivers/acpi/prmt.c @@ -236,6 +236,11 @@ static acpi_status acpi_platformrt_space_handler(u32 function, efi_status_t status; struct prm_context_buffer context; + if (!efi_enabled(EFI_RUNTIME_SERVICES)) { + pr_err_ratelimited("PRM: EFI runtime services no longer available\n"); + return AE_NO_HANDLER; + } + /* * The returned acpi_status will always be AE_OK. Error values will be * saved in the first byte of the PRM message buffer to be used by ASL. @@ -325,6 +330,11 @@ void __init init_prmt(void) pr_info("PRM: found %u modules\n", mc); + if (!efi_enabled(EFI_RUNTIME_SERVICES)) { + pr_err("PRM: EFI runtime services unavailable\n"); + return; + } + status = acpi_install_address_space_handler(ACPI_ROOT_OBJECT, ACPI_ADR_SPACE_PLATFORM_RT, &acpi_platformrt_space_handler, diff --git a/drivers/acpi/resource.c b/drivers/acpi/resource.c index 16dcd31d124fe5370f3b2f83bdb8c3a12a3b3760..192d1784e409b49295a322d4068f5637225a1fef 100644 --- a/drivers/acpi/resource.c +++ b/drivers/acpi/resource.c @@ -432,6 +432,13 @@ static const struct dmi_system_id asus_laptop[] = { DMI_MATCH(DMI_BOARD_NAME, "S5602ZA"), }, }, + { + .ident = "Asus ExpertBook B2402CBA", + .matches = { + DMI_MATCH(DMI_SYS_VENDOR, "ASUSTeK COMPUTER INC."), + DMI_MATCH(DMI_BOARD_NAME, "B2402CBA"), + }, + }, { .ident = "Asus ExpertBook B2502", .matches = { diff --git a/drivers/acpi/scan.c b/drivers/acpi/scan.c index 274344434282ee8e2cc60d91130729270550f536..0c6f06abe3f47f2b77879b0fd051ffb5a043be58 100644 --- a/drivers/acpi/scan.c +++ b/drivers/acpi/scan.c @@ -1370,9 +1370,12 @@ static void acpi_set_pnp_ids(acpi_handle handle, struct acpi_device_pnp *pnp, * Some devices don't reliably have _HIDs & _CIDs, so add * synthetic HIDs to make sure drivers can find them. */ - if (acpi_is_video_device(handle)) + if (acpi_is_video_device(handle)) { acpi_add_id(pnp, ACPI_VIDEO_HID); - else if (acpi_bay_match(handle)) + pnp->type.backlight = 1; + break; + } + if (acpi_bay_match(handle)) acpi_add_id(pnp, ACPI_BAY_HID); else if (acpi_dock_match(handle)) acpi_add_id(pnp, ACPI_DOCK_HID); diff --git a/drivers/acpi/sleep.c b/drivers/acpi/sleep.c index 0b557c0d405ef1945024374556ddaf7e831ea4a4..4ca667251272210ab1afe05b37f7caa43eec9fbe 100644 --- a/drivers/acpi/sleep.c +++ b/drivers/acpi/sleep.c @@ -60,13 +60,17 @@ static struct notifier_block tts_notifier = { .priority = 0, }; +#ifndef acpi_skip_set_wakeup_address +#define acpi_skip_set_wakeup_address() false +#endif + static int acpi_sleep_prepare(u32 acpi_state) { #ifdef CONFIG_ACPI_SLEEP unsigned long acpi_wakeup_address; /* do we have a wakeup address for S2 and S3? */ - if (acpi_state == ACPI_STATE_S3) { + if (acpi_state == ACPI_STATE_S3 && !acpi_skip_set_wakeup_address()) { acpi_wakeup_address = acpi_get_wakeup_address(); if (!acpi_wakeup_address) return -EFAULT; diff --git a/drivers/acpi/video_detect.c b/drivers/acpi/video_detect.c index 1b78c743449282958cbeb2a02157cb1001243538..a8c02608dde4530ff3df98d610aba3a9927f7ecb 100644 --- a/drivers/acpi/video_detect.c +++ b/drivers/acpi/video_detect.c @@ -50,6 +50,10 @@ static void acpi_video_parse_cmdline(void) acpi_backlight_cmdline = acpi_backlight_video; if (!strcmp("native", acpi_video_backlight_string)) acpi_backlight_cmdline = acpi_backlight_native; + if (!strcmp("nvidia_wmi_ec", acpi_video_backlight_string)) + acpi_backlight_cmdline = acpi_backlight_nvidia_wmi_ec; + if (!strcmp("apple_gmux", acpi_video_backlight_string)) + acpi_backlight_cmdline = acpi_backlight_apple_gmux; if (!strcmp("none", acpi_video_backlight_string)) acpi_backlight_cmdline = acpi_backlight_none; } @@ -106,26 +110,6 @@ static bool nvidia_wmi_ec_supported(void) } #endif -static bool apple_gmux_backlight_present(void) -{ - struct acpi_device *adev; - struct device *dev; - - adev = acpi_dev_get_first_match_dev(GMUX_ACPI_HID, NULL, -1); - if (!adev) - return false; - - dev = acpi_get_first_physical_node(adev); - if (!dev) - return false; - - /* - * drivers/platform/x86/apple-gmux.c only supports old style - * Apple GMUX with an IO-resource. - */ - return pnp_get_resource(to_pnp_dev(dev), IORESOURCE_IO, 0) != NULL; -} - /* Force to use vendor driver when the ACPI device is known to be * buggy */ static int video_detect_force_vendor(const struct dmi_system_id *d) @@ -511,6 +495,14 @@ static const struct dmi_system_id video_detect_dmi_table[] = { DMI_MATCH(DMI_PRODUCT_NAME, "Precision 7510"), }, }, + { + .callback = video_detect_force_native, + /* Acer Aspire 4810T */ + .matches = { + DMI_MATCH(DMI_SYS_VENDOR, "Acer"), + DMI_MATCH(DMI_PRODUCT_NAME, "Aspire 4810T"), + }, + }, { .callback = video_detect_force_native, /* Acer Aspire 5738z */ @@ -598,6 +590,14 @@ static const struct dmi_system_id video_detect_dmi_table[] = { DMI_MATCH(DMI_PRODUCT_NAME, "GA503"), }, }, + { + .callback = video_detect_force_native, + /* Asus U46E */ + .matches = { + DMI_MATCH(DMI_SYS_VENDOR, "ASUSTeK Computer Inc."), + DMI_MATCH(DMI_PRODUCT_NAME, "U46E"), + }, + }, { .callback = video_detect_force_native, /* Asus UX303UB */ @@ -606,6 +606,23 @@ static const struct dmi_system_id video_detect_dmi_table[] = { DMI_MATCH(DMI_PRODUCT_NAME, "UX303UB"), }, }, + { + .callback = video_detect_force_native, + /* HP EliteBook 8460p */ + .matches = { + DMI_MATCH(DMI_SYS_VENDOR, "Hewlett-Packard"), + DMI_MATCH(DMI_PRODUCT_NAME, "HP EliteBook 8460p"), + }, + }, + { + .callback = video_detect_force_native, + /* HP Pavilion g6-1d80nr / B4U19UA */ + .matches = { + DMI_MATCH(DMI_SYS_VENDOR, "Hewlett-Packard"), + DMI_MATCH(DMI_PRODUCT_NAME, "HP Pavilion g6 Notebook PC"), + DMI_MATCH(DMI_PRODUCT_SKU, "B4U19UA"), + }, + }, { .callback = video_detect_force_native, /* Samsung N150P */ @@ -754,6 +771,7 @@ static enum acpi_backlight_type __acpi_video_get_backlight_type(bool native) { static DEFINE_MUTEX(init_mutex); static bool nvidia_wmi_ec_present; + static bool apple_gmux_present; static bool native_available; static bool init_done; static long video_caps; @@ -767,6 +785,7 @@ static enum acpi_backlight_type __acpi_video_get_backlight_type(bool native) ACPI_UINT32_MAX, find_video, NULL, &video_caps, NULL); nvidia_wmi_ec_present = nvidia_wmi_ec_supported(); + apple_gmux_present = apple_gmux_detect(NULL, NULL); init_done = true; } if (native) @@ -788,7 +807,7 @@ static enum acpi_backlight_type __acpi_video_get_backlight_type(bool native) if (nvidia_wmi_ec_present) return acpi_backlight_nvidia_wmi_ec; - if (apple_gmux_backlight_present()) + if (apple_gmux_present) return acpi_backlight_apple_gmux; /* Use ACPI video if available, except when native should be preferred. */ diff --git a/drivers/ata/Kconfig b/drivers/ata/Kconfig index eceaec33af65b479f7c0d942fa4ac7e6e7085efb..9695c4404e26a17698d9b4f807377e630d47018d 100644 --- a/drivers/ata/Kconfig +++ b/drivers/ata/Kconfig @@ -640,6 +640,7 @@ config PATA_CS5530 config PATA_CS5535 tristate "CS5535 PATA support (Experimental)" depends on PCI && (X86_32 || (X86_64 && COMPILE_TEST)) + depends on !UML help This option enables support for the NatSemi/AMD CS5535 companion chip used with the Geode processor family. diff --git a/drivers/base/property.c b/drivers/base/property.c index bbb3e499ff4a5db3f7238959668f8642429b6487..083a95791d3b3ca0444bfa9390a6d66c3117e64a 100644 --- a/drivers/base/property.c +++ b/drivers/base/property.c @@ -997,26 +997,32 @@ struct fwnode_handle * fwnode_graph_get_next_endpoint(const struct fwnode_handle *fwnode, struct fwnode_handle *prev) { + struct fwnode_handle *ep, *port_parent = NULL; const struct fwnode_handle *parent; - struct fwnode_handle *ep; /* * If this function is in a loop and the previous iteration returned * an endpoint from fwnode->secondary, then we need to use the secondary * as parent rather than @fwnode. */ - if (prev) - parent = fwnode_graph_get_port_parent(prev); - else + if (prev) { + port_parent = fwnode_graph_get_port_parent(prev); + parent = port_parent; + } else { parent = fwnode; + } if (IS_ERR_OR_NULL(parent)) return NULL; ep = fwnode_call_ptr_op(parent, graph_get_next_endpoint, prev); if (ep) - return ep; + goto out_put_port_parent; + + ep = fwnode_graph_get_next_endpoint(parent->secondary, NULL); - return fwnode_graph_get_next_endpoint(parent->secondary, NULL); +out_put_port_parent: + fwnode_handle_put(port_parent); + return ep; } EXPORT_SYMBOL_GPL(fwnode_graph_get_next_endpoint); diff --git a/drivers/base/test/test_async_driver_probe.c b/drivers/base/test/test_async_driver_probe.c index 4d1976ca507271487235107b17a74226af6164b8..929410d0dd6fe2a726eff7094b7b63ff5cbd516c 100644 --- a/drivers/base/test/test_async_driver_probe.c +++ b/drivers/base/test/test_async_driver_probe.c @@ -145,7 +145,7 @@ static int __init test_async_probe_init(void) calltime = ktime_get(); for_each_online_cpu(cpu) { nid = cpu_to_node(cpu); - pdev = &sync_dev[sync_id]; + pdev = &async_dev[async_id]; *pdev = test_platform_device_register_node("test_async_driver", async_id, diff --git a/drivers/block/Kconfig b/drivers/block/Kconfig index a2184b42849361791f4d7d027dc3c46bf9a2e569..a41145d52de9426b0b6c9ac0ba97c0f7e4c79b5d 100644 --- a/drivers/block/Kconfig +++ b/drivers/block/Kconfig @@ -285,6 +285,49 @@ config BLK_DEV_RAM_SIZE The default value is 4096 kilobytes. Only change this if you know what you are doing. +config CDROM_PKTCDVD + tristate "Packet writing on CD/DVD media (DEPRECATED)" + depends on !UML + depends on SCSI + select CDROM + help + Note: This driver is deprecated and will be removed from the + kernel in the near future! + + If you have a CDROM/DVD drive that supports packet writing, say + Y to include support. It should work with any MMC/Mt Fuji + compliant ATAPI or SCSI drive, which is just about any newer + DVD/CD writer. + + Currently only writing to CD-RW, DVD-RW, DVD+RW and DVDRAM discs + is possible. + DVD-RW disks must be in restricted overwrite mode. + + See the file + for further information on the use of this driver. + + To compile this driver as a module, choose M here: the + module will be called pktcdvd. + +config CDROM_PKTCDVD_BUFFERS + int "Free buffers for data gathering" + depends on CDROM_PKTCDVD + default "8" + help + This controls the maximum number of active concurrent packets. More + concurrent packets can increase write performance, but also require + more memory. Each concurrent packet will require approximately 64Kb + of non-swappable kernel memory, memory which will be allocated when + a disc is opened for writing. + +config CDROM_PKTCDVD_WCACHE + bool "Enable write caching" + depends on CDROM_PKTCDVD + help + If enabled, write caching will be set for the CD-R/W device. For now + this option is dangerous unless the CD-RW media is known good, as we + don't do deferred write error handling yet. + config ATA_OVER_ETH tristate "ATA over Ethernet support" depends on NET diff --git a/drivers/block/Makefile b/drivers/block/Makefile index 962ee65d8ca303a608a1e780328dda47bbe4e293..101612cba303a39336cd290035c854ab3df0ff72 100644 --- a/drivers/block/Makefile +++ b/drivers/block/Makefile @@ -20,6 +20,7 @@ obj-$(CONFIG_AMIGA_Z2RAM) += z2ram.o obj-$(CONFIG_N64CART) += n64cart.o obj-$(CONFIG_BLK_DEV_RAM) += brd.o obj-$(CONFIG_BLK_DEV_LOOP) += loop.o +obj-$(CONFIG_CDROM_PKTCDVD) += pktcdvd.o obj-$(CONFIG_SUNVDC) += sunvdc.o obj-$(CONFIG_BLK_DEV_NBD) += nbd.o diff --git a/drivers/block/drbd/drbd_req.c b/drivers/block/drbd/drbd_req.c index eb14ec8ec04cfcaf46011809fae0c8fb9f6e8428..e36216d50753c83843e3222632872ae76c685ccd 100644 --- a/drivers/block/drbd/drbd_req.c +++ b/drivers/block/drbd/drbd_req.c @@ -1607,6 +1607,8 @@ void drbd_submit_bio(struct bio *bio) struct drbd_device *device = bio->bi_bdev->bd_disk->private_data; bio = bio_split_to_limits(bio); + if (!bio) + return; /* * what we "blindly" assume: diff --git a/drivers/block/pktcdvd.c b/drivers/block/pktcdvd.c new file mode 100644 index 0000000000000000000000000000000000000000..2f1a92509271c406f59fd2bb4325737574285be6 --- /dev/null +++ b/drivers/block/pktcdvd.c @@ -0,0 +1,2946 @@ +/* + * Copyright (C) 2000 Jens Axboe + * Copyright (C) 2001-2004 Peter Osterlund + * Copyright (C) 2006 Thomas Maier + * + * May be copied or modified under the terms of the GNU General Public + * License. See linux/COPYING for more information. + * + * Packet writing layer for ATAPI and SCSI CD-RW, DVD+RW, DVD-RW and + * DVD-RAM devices. + * + * Theory of operation: + * + * At the lowest level, there is the standard driver for the CD/DVD device, + * such as drivers/scsi/sr.c. This driver can handle read and write requests, + * but it doesn't know anything about the special restrictions that apply to + * packet writing. One restriction is that write requests must be aligned to + * packet boundaries on the physical media, and the size of a write request + * must be equal to the packet size. Another restriction is that a + * GPCMD_FLUSH_CACHE command has to be issued to the drive before a read + * command, if the previous command was a write. + * + * The purpose of the packet writing driver is to hide these restrictions from + * higher layers, such as file systems, and present a block device that can be + * randomly read and written using 2kB-sized blocks. + * + * The lowest layer in the packet writing driver is the packet I/O scheduler. + * Its data is defined by the struct packet_iosched and includes two bio + * queues with pending read and write requests. These queues are processed + * by the pkt_iosched_process_queue() function. The write requests in this + * queue are already properly aligned and sized. This layer is responsible for + * issuing the flush cache commands and scheduling the I/O in a good order. + * + * The next layer transforms unaligned write requests to aligned writes. This + * transformation requires reading missing pieces of data from the underlying + * block device, assembling the pieces to full packets and queuing them to the + * packet I/O scheduler. + * + * At the top layer there is a custom ->submit_bio function that forwards + * read requests directly to the iosched queue and puts write requests in the + * unaligned write queue. A kernel thread performs the necessary read + * gathering to convert the unaligned writes to aligned writes and then feeds + * them to the packet I/O scheduler. + * + *************************************************************************/ + +#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#define DRIVER_NAME "pktcdvd" + +#define pkt_err(pd, fmt, ...) \ + pr_err("%s: " fmt, pd->name, ##__VA_ARGS__) +#define pkt_notice(pd, fmt, ...) \ + pr_notice("%s: " fmt, pd->name, ##__VA_ARGS__) +#define pkt_info(pd, fmt, ...) \ + pr_info("%s: " fmt, pd->name, ##__VA_ARGS__) + +#define pkt_dbg(level, pd, fmt, ...) \ +do { \ + if (level == 2 && PACKET_DEBUG >= 2) \ + pr_notice("%s: %s():" fmt, \ + pd->name, __func__, ##__VA_ARGS__); \ + else if (level == 1 && PACKET_DEBUG >= 1) \ + pr_notice("%s: " fmt, pd->name, ##__VA_ARGS__); \ +} while (0) + +#define MAX_SPEED 0xffff + +static DEFINE_MUTEX(pktcdvd_mutex); +static struct pktcdvd_device *pkt_devs[MAX_WRITERS]; +static struct proc_dir_entry *pkt_proc; +static int pktdev_major; +static int write_congestion_on = PKT_WRITE_CONGESTION_ON; +static int write_congestion_off = PKT_WRITE_CONGESTION_OFF; +static struct mutex ctl_mutex; /* Serialize open/close/setup/teardown */ +static mempool_t psd_pool; +static struct bio_set pkt_bio_set; + +static struct class *class_pktcdvd = NULL; /* /sys/class/pktcdvd */ +static struct dentry *pkt_debugfs_root = NULL; /* /sys/kernel/debug/pktcdvd */ + +/* forward declaration */ +static int pkt_setup_dev(dev_t dev, dev_t* pkt_dev); +static int pkt_remove_dev(dev_t pkt_dev); +static int pkt_seq_show(struct seq_file *m, void *p); + +static sector_t get_zone(sector_t sector, struct pktcdvd_device *pd) +{ + return (sector + pd->offset) & ~(sector_t)(pd->settings.size - 1); +} + +/********************************************************** + * sysfs interface for pktcdvd + * by (C) 2006 Thomas Maier + + /sys/class/pktcdvd/pktcdvd[0-7]/ + stat/reset + stat/packets_started + stat/packets_finished + stat/kb_written + stat/kb_read + stat/kb_read_gather + write_queue/size + write_queue/congestion_off + write_queue/congestion_on + **********************************************************/ + +static ssize_t packets_started_show(struct device *dev, + struct device_attribute *attr, char *buf) +{ + struct pktcdvd_device *pd = dev_get_drvdata(dev); + + return sysfs_emit(buf, "%lu\n", pd->stats.pkt_started); +} +static DEVICE_ATTR_RO(packets_started); + +static ssize_t packets_finished_show(struct device *dev, + struct device_attribute *attr, char *buf) +{ + struct pktcdvd_device *pd = dev_get_drvdata(dev); + + return sysfs_emit(buf, "%lu\n", pd->stats.pkt_ended); +} +static DEVICE_ATTR_RO(packets_finished); + +static ssize_t kb_written_show(struct device *dev, + struct device_attribute *attr, char *buf) +{ + struct pktcdvd_device *pd = dev_get_drvdata(dev); + + return sysfs_emit(buf, "%lu\n", pd->stats.secs_w >> 1); +} +static DEVICE_ATTR_RO(kb_written); + +static ssize_t kb_read_show(struct device *dev, + struct device_attribute *attr, char *buf) +{ + struct pktcdvd_device *pd = dev_get_drvdata(dev); + + return sysfs_emit(buf, "%lu\n", pd->stats.secs_r >> 1); +} +static DEVICE_ATTR_RO(kb_read); + +static ssize_t kb_read_gather_show(struct device *dev, + struct device_attribute *attr, char *buf) +{ + struct pktcdvd_device *pd = dev_get_drvdata(dev); + + return sysfs_emit(buf, "%lu\n", pd->stats.secs_rg >> 1); +} +static DEVICE_ATTR_RO(kb_read_gather); + +static ssize_t reset_store(struct device *dev, struct device_attribute *attr, + const char *buf, size_t len) +{ + struct pktcdvd_device *pd = dev_get_drvdata(dev); + + if (len > 0) { + pd->stats.pkt_started = 0; + pd->stats.pkt_ended = 0; + pd->stats.secs_w = 0; + pd->stats.secs_rg = 0; + pd->stats.secs_r = 0; + } + return len; +} +static DEVICE_ATTR_WO(reset); + +static struct attribute *pkt_stat_attrs[] = { + &dev_attr_packets_finished.attr, + &dev_attr_packets_started.attr, + &dev_attr_kb_read.attr, + &dev_attr_kb_written.attr, + &dev_attr_kb_read_gather.attr, + &dev_attr_reset.attr, + NULL, +}; + +static const struct attribute_group pkt_stat_group = { + .name = "stat", + .attrs = pkt_stat_attrs, +}; + +static ssize_t size_show(struct device *dev, + struct device_attribute *attr, char *buf) +{ + struct pktcdvd_device *pd = dev_get_drvdata(dev); + int n; + + spin_lock(&pd->lock); + n = sysfs_emit(buf, "%d\n", pd->bio_queue_size); + spin_unlock(&pd->lock); + return n; +} +static DEVICE_ATTR_RO(size); + +static void init_write_congestion_marks(int* lo, int* hi) +{ + if (*hi > 0) { + *hi = max(*hi, 500); + *hi = min(*hi, 1000000); + if (*lo <= 0) + *lo = *hi - 100; + else { + *lo = min(*lo, *hi - 100); + *lo = max(*lo, 100); + } + } else { + *hi = -1; + *lo = -1; + } +} + +static ssize_t congestion_off_show(struct device *dev, + struct device_attribute *attr, char *buf) +{ + struct pktcdvd_device *pd = dev_get_drvdata(dev); + int n; + + spin_lock(&pd->lock); + n = sysfs_emit(buf, "%d\n", pd->write_congestion_off); + spin_unlock(&pd->lock); + return n; +} + +static ssize_t congestion_off_store(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t len) +{ + struct pktcdvd_device *pd = dev_get_drvdata(dev); + int val; + + if (sscanf(buf, "%d", &val) == 1) { + spin_lock(&pd->lock); + pd->write_congestion_off = val; + init_write_congestion_marks(&pd->write_congestion_off, + &pd->write_congestion_on); + spin_unlock(&pd->lock); + } + return len; +} +static DEVICE_ATTR_RW(congestion_off); + +static ssize_t congestion_on_show(struct device *dev, + struct device_attribute *attr, char *buf) +{ + struct pktcdvd_device *pd = dev_get_drvdata(dev); + int n; + + spin_lock(&pd->lock); + n = sysfs_emit(buf, "%d\n", pd->write_congestion_on); + spin_unlock(&pd->lock); + return n; +} + +static ssize_t congestion_on_store(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t len) +{ + struct pktcdvd_device *pd = dev_get_drvdata(dev); + int val; + + if (sscanf(buf, "%d", &val) == 1) { + spin_lock(&pd->lock); + pd->write_congestion_on = val; + init_write_congestion_marks(&pd->write_congestion_off, + &pd->write_congestion_on); + spin_unlock(&pd->lock); + } + return len; +} +static DEVICE_ATTR_RW(congestion_on); + +static struct attribute *pkt_wq_attrs[] = { + &dev_attr_congestion_on.attr, + &dev_attr_congestion_off.attr, + &dev_attr_size.attr, + NULL, +}; + +static const struct attribute_group pkt_wq_group = { + .name = "write_queue", + .attrs = pkt_wq_attrs, +}; + +static const struct attribute_group *pkt_groups[] = { + &pkt_stat_group, + &pkt_wq_group, + NULL, +}; + +static void pkt_sysfs_dev_new(struct pktcdvd_device *pd) +{ + if (class_pktcdvd) { + pd->dev = device_create_with_groups(class_pktcdvd, NULL, + MKDEV(0, 0), pd, pkt_groups, + "%s", pd->name); + if (IS_ERR(pd->dev)) + pd->dev = NULL; + } +} + +static void pkt_sysfs_dev_remove(struct pktcdvd_device *pd) +{ + if (class_pktcdvd) + device_unregister(pd->dev); +} + + +/******************************************************************** + /sys/class/pktcdvd/ + add map block device + remove unmap packet dev + device_map show mappings + *******************************************************************/ + +static void class_pktcdvd_release(struct class *cls) +{ + kfree(cls); +} + +static ssize_t device_map_show(struct class *c, struct class_attribute *attr, + char *data) +{ + int n = 0; + int idx; + mutex_lock_nested(&ctl_mutex, SINGLE_DEPTH_NESTING); + for (idx = 0; idx < MAX_WRITERS; idx++) { + struct pktcdvd_device *pd = pkt_devs[idx]; + if (!pd) + continue; + n += sprintf(data+n, "%s %u:%u %u:%u\n", + pd->name, + MAJOR(pd->pkt_dev), MINOR(pd->pkt_dev), + MAJOR(pd->bdev->bd_dev), + MINOR(pd->bdev->bd_dev)); + } + mutex_unlock(&ctl_mutex); + return n; +} +static CLASS_ATTR_RO(device_map); + +static ssize_t add_store(struct class *c, struct class_attribute *attr, + const char *buf, size_t count) +{ + unsigned int major, minor; + + if (sscanf(buf, "%u:%u", &major, &minor) == 2) { + /* pkt_setup_dev() expects caller to hold reference to self */ + if (!try_module_get(THIS_MODULE)) + return -ENODEV; + + pkt_setup_dev(MKDEV(major, minor), NULL); + + module_put(THIS_MODULE); + + return count; + } + + return -EINVAL; +} +static CLASS_ATTR_WO(add); + +static ssize_t remove_store(struct class *c, struct class_attribute *attr, + const char *buf, size_t count) +{ + unsigned int major, minor; + if (sscanf(buf, "%u:%u", &major, &minor) == 2) { + pkt_remove_dev(MKDEV(major, minor)); + return count; + } + return -EINVAL; +} +static CLASS_ATTR_WO(remove); + +static struct attribute *class_pktcdvd_attrs[] = { + &class_attr_add.attr, + &class_attr_remove.attr, + &class_attr_device_map.attr, + NULL, +}; +ATTRIBUTE_GROUPS(class_pktcdvd); + +static int pkt_sysfs_init(void) +{ + int ret = 0; + + /* + * create control files in sysfs + * /sys/class/pktcdvd/... + */ + class_pktcdvd = kzalloc(sizeof(*class_pktcdvd), GFP_KERNEL); + if (!class_pktcdvd) + return -ENOMEM; + class_pktcdvd->name = DRIVER_NAME; + class_pktcdvd->owner = THIS_MODULE; + class_pktcdvd->class_release = class_pktcdvd_release; + class_pktcdvd->class_groups = class_pktcdvd_groups; + ret = class_register(class_pktcdvd); + if (ret) { + kfree(class_pktcdvd); + class_pktcdvd = NULL; + pr_err("failed to create class pktcdvd\n"); + return ret; + } + return 0; +} + +static void pkt_sysfs_cleanup(void) +{ + if (class_pktcdvd) + class_destroy(class_pktcdvd); + class_pktcdvd = NULL; +} + +/******************************************************************** + entries in debugfs + + /sys/kernel/debug/pktcdvd[0-7]/ + info + + *******************************************************************/ + +static int pkt_debugfs_seq_show(struct seq_file *m, void *p) +{ + return pkt_seq_show(m, p); +} + +static int pkt_debugfs_fops_open(struct inode *inode, struct file *file) +{ + return single_open(file, pkt_debugfs_seq_show, inode->i_private); +} + +static const struct file_operations debug_fops = { + .open = pkt_debugfs_fops_open, + .read = seq_read, + .llseek = seq_lseek, + .release = single_release, + .owner = THIS_MODULE, +}; + +static void pkt_debugfs_dev_new(struct pktcdvd_device *pd) +{ + if (!pkt_debugfs_root) + return; + pd->dfs_d_root = debugfs_create_dir(pd->name, pkt_debugfs_root); + if (!pd->dfs_d_root) + return; + + pd->dfs_f_info = debugfs_create_file("info", 0444, + pd->dfs_d_root, pd, &debug_fops); +} + +static void pkt_debugfs_dev_remove(struct pktcdvd_device *pd) +{ + if (!pkt_debugfs_root) + return; + debugfs_remove(pd->dfs_f_info); + debugfs_remove(pd->dfs_d_root); + pd->dfs_f_info = NULL; + pd->dfs_d_root = NULL; +} + +static void pkt_debugfs_init(void) +{ + pkt_debugfs_root = debugfs_create_dir(DRIVER_NAME, NULL); +} + +static void pkt_debugfs_cleanup(void) +{ + debugfs_remove(pkt_debugfs_root); + pkt_debugfs_root = NULL; +} + +/* ----------------------------------------------------------*/ + + +static void pkt_bio_finished(struct pktcdvd_device *pd) +{ + BUG_ON(atomic_read(&pd->cdrw.pending_bios) <= 0); + if (atomic_dec_and_test(&pd->cdrw.pending_bios)) { + pkt_dbg(2, pd, "queue empty\n"); + atomic_set(&pd->iosched.attention, 1); + wake_up(&pd->wqueue); + } +} + +/* + * Allocate a packet_data struct + */ +static struct packet_data *pkt_alloc_packet_data(int frames) +{ + int i; + struct packet_data *pkt; + + pkt = kzalloc(sizeof(struct packet_data), GFP_KERNEL); + if (!pkt) + goto no_pkt; + + pkt->frames = frames; + pkt->w_bio = bio_kmalloc(frames, GFP_KERNEL); + if (!pkt->w_bio) + goto no_bio; + + for (i = 0; i < frames / FRAMES_PER_PAGE; i++) { + pkt->pages[i] = alloc_page(GFP_KERNEL|__GFP_ZERO); + if (!pkt->pages[i]) + goto no_page; + } + + spin_lock_init(&pkt->lock); + bio_list_init(&pkt->orig_bios); + + for (i = 0; i < frames; i++) { + pkt->r_bios[i] = bio_kmalloc(1, GFP_KERNEL); + if (!pkt->r_bios[i]) + goto no_rd_bio; + } + + return pkt; + +no_rd_bio: + for (i = 0; i < frames; i++) + kfree(pkt->r_bios[i]); +no_page: + for (i = 0; i < frames / FRAMES_PER_PAGE; i++) + if (pkt->pages[i]) + __free_page(pkt->pages[i]); + kfree(pkt->w_bio); +no_bio: + kfree(pkt); +no_pkt: + return NULL; +} + +/* + * Free a packet_data struct + */ +static void pkt_free_packet_data(struct packet_data *pkt) +{ + int i; + + for (i = 0; i < pkt->frames; i++) + kfree(pkt->r_bios[i]); + for (i = 0; i < pkt->frames / FRAMES_PER_PAGE; i++) + __free_page(pkt->pages[i]); + kfree(pkt->w_bio); + kfree(pkt); +} + +static void pkt_shrink_pktlist(struct pktcdvd_device *pd) +{ + struct packet_data *pkt, *next; + + BUG_ON(!list_empty(&pd->cdrw.pkt_active_list)); + + list_for_each_entry_safe(pkt, next, &pd->cdrw.pkt_free_list, list) { + pkt_free_packet_data(pkt); + } + INIT_LIST_HEAD(&pd->cdrw.pkt_free_list); +} + +static int pkt_grow_pktlist(struct pktcdvd_device *pd, int nr_packets) +{ + struct packet_data *pkt; + + BUG_ON(!list_empty(&pd->cdrw.pkt_free_list)); + + while (nr_packets > 0) { + pkt = pkt_alloc_packet_data(pd->settings.size >> 2); + if (!pkt) { + pkt_shrink_pktlist(pd); + return 0; + } + pkt->id = nr_packets; + pkt->pd = pd; + list_add(&pkt->list, &pd->cdrw.pkt_free_list); + nr_packets--; + } + return 1; +} + +static inline struct pkt_rb_node *pkt_rbtree_next(struct pkt_rb_node *node) +{ + struct rb_node *n = rb_next(&node->rb_node); + if (!n) + return NULL; + return rb_entry(n, struct pkt_rb_node, rb_node); +} + +static void pkt_rbtree_erase(struct pktcdvd_device *pd, struct pkt_rb_node *node) +{ + rb_erase(&node->rb_node, &pd->bio_queue); + mempool_free(node, &pd->rb_pool); + pd->bio_queue_size--; + BUG_ON(pd->bio_queue_size < 0); +} + +/* + * Find the first node in the pd->bio_queue rb tree with a starting sector >= s. + */ +static struct pkt_rb_node *pkt_rbtree_find(struct pktcdvd_device *pd, sector_t s) +{ + struct rb_node *n = pd->bio_queue.rb_node; + struct rb_node *next; + struct pkt_rb_node *tmp; + + if (!n) { + BUG_ON(pd->bio_queue_size > 0); + return NULL; + } + + for (;;) { + tmp = rb_entry(n, struct pkt_rb_node, rb_node); + if (s <= tmp->bio->bi_iter.bi_sector) + next = n->rb_left; + else + next = n->rb_right; + if (!next) + break; + n = next; + } + + if (s > tmp->bio->bi_iter.bi_sector) { + tmp = pkt_rbtree_next(tmp); + if (!tmp) + return NULL; + } + BUG_ON(s > tmp->bio->bi_iter.bi_sector); + return tmp; +} + +/* + * Insert a node into the pd->bio_queue rb tree. + */ +static void pkt_rbtree_insert(struct pktcdvd_device *pd, struct pkt_rb_node *node) +{ + struct rb_node **p = &pd->bio_queue.rb_node; + struct rb_node *parent = NULL; + sector_t s = node->bio->bi_iter.bi_sector; + struct pkt_rb_node *tmp; + + while (*p) { + parent = *p; + tmp = rb_entry(parent, struct pkt_rb_node, rb_node); + if (s < tmp->bio->bi_iter.bi_sector) + p = &(*p)->rb_left; + else + p = &(*p)->rb_right; + } + rb_link_node(&node->rb_node, parent, p); + rb_insert_color(&node->rb_node, &pd->bio_queue); + pd->bio_queue_size++; +} + +/* + * Send a packet_command to the underlying block device and + * wait for completion. + */ +static int pkt_generic_packet(struct pktcdvd_device *pd, struct packet_command *cgc) +{ + struct request_queue *q = bdev_get_queue(pd->bdev); + struct scsi_cmnd *scmd; + struct request *rq; + int ret = 0; + + rq = scsi_alloc_request(q, (cgc->data_direction == CGC_DATA_WRITE) ? + REQ_OP_DRV_OUT : REQ_OP_DRV_IN, 0); + if (IS_ERR(rq)) + return PTR_ERR(rq); + scmd = blk_mq_rq_to_pdu(rq); + + if (cgc->buflen) { + ret = blk_rq_map_kern(q, rq, cgc->buffer, cgc->buflen, + GFP_NOIO); + if (ret) + goto out; + } + + scmd->cmd_len = COMMAND_SIZE(cgc->cmd[0]); + memcpy(scmd->cmnd, cgc->cmd, CDROM_PACKET_SIZE); + + rq->timeout = 60*HZ; + if (cgc->quiet) + rq->rq_flags |= RQF_QUIET; + + blk_execute_rq(rq, false); + if (scmd->result) + ret = -EIO; +out: + blk_mq_free_request(rq); + return ret; +} + +static const char *sense_key_string(__u8 index) +{ + static const char * const info[] = { + "No sense", "Recovered error", "Not ready", + "Medium error", "Hardware error", "Illegal request", + "Unit attention", "Data protect", "Blank check", + }; + + return index < ARRAY_SIZE(info) ? info[index] : "INVALID"; +} + +/* + * A generic sense dump / resolve mechanism should be implemented across + * all ATAPI + SCSI devices. + */ +static void pkt_dump_sense(struct pktcdvd_device *pd, + struct packet_command *cgc) +{ + struct scsi_sense_hdr *sshdr = cgc->sshdr; + + if (sshdr) + pkt_err(pd, "%*ph - sense %02x.%02x.%02x (%s)\n", + CDROM_PACKET_SIZE, cgc->cmd, + sshdr->sense_key, sshdr->asc, sshdr->ascq, + sense_key_string(sshdr->sense_key)); + else + pkt_err(pd, "%*ph - no sense\n", CDROM_PACKET_SIZE, cgc->cmd); +} + +/* + * flush the drive cache to media + */ +static int pkt_flush_cache(struct pktcdvd_device *pd) +{ + struct packet_command cgc; + + init_cdrom_command(&cgc, NULL, 0, CGC_DATA_NONE); + cgc.cmd[0] = GPCMD_FLUSH_CACHE; + cgc.quiet = 1; + + /* + * the IMMED bit -- we default to not setting it, although that + * would allow a much faster close, this is safer + */ +#if 0 + cgc.cmd[1] = 1 << 1; +#endif + return pkt_generic_packet(pd, &cgc); +} + +/* + * speed is given as the normal factor, e.g. 4 for 4x + */ +static noinline_for_stack int pkt_set_speed(struct pktcdvd_device *pd, + unsigned write_speed, unsigned read_speed) +{ + struct packet_command cgc; + struct scsi_sense_hdr sshdr; + int ret; + + init_cdrom_command(&cgc, NULL, 0, CGC_DATA_NONE); + cgc.sshdr = &sshdr; + cgc.cmd[0] = GPCMD_SET_SPEED; + cgc.cmd[2] = (read_speed >> 8) & 0xff; + cgc.cmd[3] = read_speed & 0xff; + cgc.cmd[4] = (write_speed >> 8) & 0xff; + cgc.cmd[5] = write_speed & 0xff; + + ret = pkt_generic_packet(pd, &cgc); + if (ret) + pkt_dump_sense(pd, &cgc); + + return ret; +} + +/* + * Queue a bio for processing by the low-level CD device. Must be called + * from process context. + */ +static void pkt_queue_bio(struct pktcdvd_device *pd, struct bio *bio) +{ + spin_lock(&pd->iosched.lock); + if (bio_data_dir(bio) == READ) + bio_list_add(&pd->iosched.read_queue, bio); + else + bio_list_add(&pd->iosched.write_queue, bio); + spin_unlock(&pd->iosched.lock); + + atomic_set(&pd->iosched.attention, 1); + wake_up(&pd->wqueue); +} + +/* + * Process the queued read/write requests. This function handles special + * requirements for CDRW drives: + * - A cache flush command must be inserted before a read request if the + * previous request was a write. + * - Switching between reading and writing is slow, so don't do it more often + * than necessary. + * - Optimize for throughput at the expense of latency. This means that streaming + * writes will never be interrupted by a read, but if the drive has to seek + * before the next write, switch to reading instead if there are any pending + * read requests. + * - Set the read speed according to current usage pattern. When only reading + * from the device, it's best to use the highest possible read speed, but + * when switching often between reading and writing, it's better to have the + * same read and write speeds. + */ +static void pkt_iosched_process_queue(struct pktcdvd_device *pd) +{ + + if (atomic_read(&pd->iosched.attention) == 0) + return; + atomic_set(&pd->iosched.attention, 0); + + for (;;) { + struct bio *bio; + int reads_queued, writes_queued; + + spin_lock(&pd->iosched.lock); + reads_queued = !bio_list_empty(&pd->iosched.read_queue); + writes_queued = !bio_list_empty(&pd->iosched.write_queue); + spin_unlock(&pd->iosched.lock); + + if (!reads_queued && !writes_queued) + break; + + if (pd->iosched.writing) { + int need_write_seek = 1; + spin_lock(&pd->iosched.lock); + bio = bio_list_peek(&pd->iosched.write_queue); + spin_unlock(&pd->iosched.lock); + if (bio && (bio->bi_iter.bi_sector == + pd->iosched.last_write)) + need_write_seek = 0; + if (need_write_seek && reads_queued) { + if (atomic_read(&pd->cdrw.pending_bios) > 0) { + pkt_dbg(2, pd, "write, waiting\n"); + break; + } + pkt_flush_cache(pd); + pd->iosched.writing = 0; + } + } else { + if (!reads_queued && writes_queued) { + if (atomic_read(&pd->cdrw.pending_bios) > 0) { + pkt_dbg(2, pd, "read, waiting\n"); + break; + } + pd->iosched.writing = 1; + } + } + + spin_lock(&pd->iosched.lock); + if (pd->iosched.writing) + bio = bio_list_pop(&pd->iosched.write_queue); + else + bio = bio_list_pop(&pd->iosched.read_queue); + spin_unlock(&pd->iosched.lock); + + if (!bio) + continue; + + if (bio_data_dir(bio) == READ) + pd->iosched.successive_reads += + bio->bi_iter.bi_size >> 10; + else { + pd->iosched.successive_reads = 0; + pd->iosched.last_write = bio_end_sector(bio); + } + if (pd->iosched.successive_reads >= HI_SPEED_SWITCH) { + if (pd->read_speed == pd->write_speed) { + pd->read_speed = MAX_SPEED; + pkt_set_speed(pd, pd->write_speed, pd->read_speed); + } + } else { + if (pd->read_speed != pd->write_speed) { + pd->read_speed = pd->write_speed; + pkt_set_speed(pd, pd->write_speed, pd->read_speed); + } + } + + atomic_inc(&pd->cdrw.pending_bios); + submit_bio_noacct(bio); + } +} + +/* + * Special care is needed if the underlying block device has a small + * max_phys_segments value. + */ +static int pkt_set_segment_merging(struct pktcdvd_device *pd, struct request_queue *q) +{ + if ((pd->settings.size << 9) / CD_FRAMESIZE + <= queue_max_segments(q)) { + /* + * The cdrom device can handle one segment/frame + */ + clear_bit(PACKET_MERGE_SEGS, &pd->flags); + return 0; + } else if ((pd->settings.size << 9) / PAGE_SIZE + <= queue_max_segments(q)) { + /* + * We can handle this case at the expense of some extra memory + * copies during write operations + */ + set_bit(PACKET_MERGE_SEGS, &pd->flags); + return 0; + } else { + pkt_err(pd, "cdrom max_phys_segments too small\n"); + return -EIO; + } +} + +static void pkt_end_io_read(struct bio *bio) +{ + struct packet_data *pkt = bio->bi_private; + struct pktcdvd_device *pd = pkt->pd; + BUG_ON(!pd); + + pkt_dbg(2, pd, "bio=%p sec0=%llx sec=%llx err=%d\n", + bio, (unsigned long long)pkt->sector, + (unsigned long long)bio->bi_iter.bi_sector, bio->bi_status); + + if (bio->bi_status) + atomic_inc(&pkt->io_errors); + bio_uninit(bio); + if (atomic_dec_and_test(&pkt->io_wait)) { + atomic_inc(&pkt->run_sm); + wake_up(&pd->wqueue); + } + pkt_bio_finished(pd); +} + +static void pkt_end_io_packet_write(struct bio *bio) +{ + struct packet_data *pkt = bio->bi_private; + struct pktcdvd_device *pd = pkt->pd; + BUG_ON(!pd); + + pkt_dbg(2, pd, "id=%d, err=%d\n", pkt->id, bio->bi_status); + + pd->stats.pkt_ended++; + + bio_uninit(bio); + pkt_bio_finished(pd); + atomic_dec(&pkt->io_wait); + atomic_inc(&pkt->run_sm); + wake_up(&pd->wqueue); +} + +/* + * Schedule reads for the holes in a packet + */ +static void pkt_gather_data(struct pktcdvd_device *pd, struct packet_data *pkt) +{ + int frames_read = 0; + struct bio *bio; + int f; + char written[PACKET_MAX_SIZE]; + + BUG_ON(bio_list_empty(&pkt->orig_bios)); + + atomic_set(&pkt->io_wait, 0); + atomic_set(&pkt->io_errors, 0); + + /* + * Figure out which frames we need to read before we can write. + */ + memset(written, 0, sizeof(written)); + spin_lock(&pkt->lock); + bio_list_for_each(bio, &pkt->orig_bios) { + int first_frame = (bio->bi_iter.bi_sector - pkt->sector) / + (CD_FRAMESIZE >> 9); + int num_frames = bio->bi_iter.bi_size / CD_FRAMESIZE; + pd->stats.secs_w += num_frames * (CD_FRAMESIZE >> 9); + BUG_ON(first_frame < 0); + BUG_ON(first_frame + num_frames > pkt->frames); + for (f = first_frame; f < first_frame + num_frames; f++) + written[f] = 1; + } + spin_unlock(&pkt->lock); + + if (pkt->cache_valid) { + pkt_dbg(2, pd, "zone %llx cached\n", + (unsigned long long)pkt->sector); + goto out_account; + } + + /* + * Schedule reads for missing parts of the packet. + */ + for (f = 0; f < pkt->frames; f++) { + int p, offset; + + if (written[f]) + continue; + + bio = pkt->r_bios[f]; + bio_init(bio, pd->bdev, bio->bi_inline_vecs, 1, REQ_OP_READ); + bio->bi_iter.bi_sector = pkt->sector + f * (CD_FRAMESIZE >> 9); + bio->bi_end_io = pkt_end_io_read; + bio->bi_private = pkt; + + p = (f * CD_FRAMESIZE) / PAGE_SIZE; + offset = (f * CD_FRAMESIZE) % PAGE_SIZE; + pkt_dbg(2, pd, "Adding frame %d, page:%p offs:%d\n", + f, pkt->pages[p], offset); + if (!bio_add_page(bio, pkt->pages[p], CD_FRAMESIZE, offset)) + BUG(); + + atomic_inc(&pkt->io_wait); + pkt_queue_bio(pd, bio); + frames_read++; + } + +out_account: + pkt_dbg(2, pd, "need %d frames for zone %llx\n", + frames_read, (unsigned long long)pkt->sector); + pd->stats.pkt_started++; + pd->stats.secs_rg += frames_read * (CD_FRAMESIZE >> 9); +} + +/* + * Find a packet matching zone, or the least recently used packet if + * there is no match. + */ +static struct packet_data *pkt_get_packet_data(struct pktcdvd_device *pd, int zone) +{ + struct packet_data *pkt; + + list_for_each_entry(pkt, &pd->cdrw.pkt_free_list, list) { + if (pkt->sector == zone || pkt->list.next == &pd->cdrw.pkt_free_list) { + list_del_init(&pkt->list); + if (pkt->sector != zone) + pkt->cache_valid = 0; + return pkt; + } + } + BUG(); + return NULL; +} + +static void pkt_put_packet_data(struct pktcdvd_device *pd, struct packet_data *pkt) +{ + if (pkt->cache_valid) { + list_add(&pkt->list, &pd->cdrw.pkt_free_list); + } else { + list_add_tail(&pkt->list, &pd->cdrw.pkt_free_list); + } +} + +static inline void pkt_set_state(struct packet_data *pkt, enum packet_data_state state) +{ +#if PACKET_DEBUG > 1 + static const char *state_name[] = { + "IDLE", "WAITING", "READ_WAIT", "WRITE_WAIT", "RECOVERY", "FINISHED" + }; + enum packet_data_state old_state = pkt->state; + pkt_dbg(2, pd, "pkt %2d : s=%6llx %s -> %s\n", + pkt->id, (unsigned long long)pkt->sector, + state_name[old_state], state_name[state]); +#endif + pkt->state = state; +} + +/* + * Scan the work queue to see if we can start a new packet. + * returns non-zero if any work was done. + */ +static int pkt_handle_queue(struct pktcdvd_device *pd) +{ + struct packet_data *pkt, *p; + struct bio *bio = NULL; + sector_t zone = 0; /* Suppress gcc warning */ + struct pkt_rb_node *node, *first_node; + struct rb_node *n; + + atomic_set(&pd->scan_queue, 0); + + if (list_empty(&pd->cdrw.pkt_free_list)) { + pkt_dbg(2, pd, "no pkt\n"); + return 0; + } + + /* + * Try to find a zone we are not already working on. + */ + spin_lock(&pd->lock); + first_node = pkt_rbtree_find(pd, pd->current_sector); + if (!first_node) { + n = rb_first(&pd->bio_queue); + if (n) + first_node = rb_entry(n, struct pkt_rb_node, rb_node); + } + node = first_node; + while (node) { + bio = node->bio; + zone = get_zone(bio->bi_iter.bi_sector, pd); + list_for_each_entry(p, &pd->cdrw.pkt_active_list, list) { + if (p->sector == zone) { + bio = NULL; + goto try_next_bio; + } + } + break; +try_next_bio: + node = pkt_rbtree_next(node); + if (!node) { + n = rb_first(&pd->bio_queue); + if (n) + node = rb_entry(n, struct pkt_rb_node, rb_node); + } + if (node == first_node) + node = NULL; + } + spin_unlock(&pd->lock); + if (!bio) { + pkt_dbg(2, pd, "no bio\n"); + return 0; + } + + pkt = pkt_get_packet_data(pd, zone); + + pd->current_sector = zone + pd->settings.size; + pkt->sector = zone; + BUG_ON(pkt->frames != pd->settings.size >> 2); + pkt->write_size = 0; + + /* + * Scan work queue for bios in the same zone and link them + * to this packet. + */ + spin_lock(&pd->lock); + pkt_dbg(2, pd, "looking for zone %llx\n", (unsigned long long)zone); + while ((node = pkt_rbtree_find(pd, zone)) != NULL) { + bio = node->bio; + pkt_dbg(2, pd, "found zone=%llx\n", (unsigned long long) + get_zone(bio->bi_iter.bi_sector, pd)); + if (get_zone(bio->bi_iter.bi_sector, pd) != zone) + break; + pkt_rbtree_erase(pd, node); + spin_lock(&pkt->lock); + bio_list_add(&pkt->orig_bios, bio); + pkt->write_size += bio->bi_iter.bi_size / CD_FRAMESIZE; + spin_unlock(&pkt->lock); + } + /* check write congestion marks, and if bio_queue_size is + * below, wake up any waiters + */ + if (pd->congested && + pd->bio_queue_size <= pd->write_congestion_off) { + pd->congested = false; + wake_up_var(&pd->congested); + } + spin_unlock(&pd->lock); + + pkt->sleep_time = max(PACKET_WAIT_TIME, 1); + pkt_set_state(pkt, PACKET_WAITING_STATE); + atomic_set(&pkt->run_sm, 1); + + spin_lock(&pd->cdrw.active_list_lock); + list_add(&pkt->list, &pd->cdrw.pkt_active_list); + spin_unlock(&pd->cdrw.active_list_lock); + + return 1; +} + +/** + * bio_list_copy_data - copy contents of data buffers from one chain of bios to + * another + * @src: source bio list + * @dst: destination bio list + * + * Stops when it reaches the end of either the @src list or @dst list - that is, + * copies min(src->bi_size, dst->bi_size) bytes (or the equivalent for lists of + * bios). + */ +static void bio_list_copy_data(struct bio *dst, struct bio *src) +{ + struct bvec_iter src_iter = src->bi_iter; + struct bvec_iter dst_iter = dst->bi_iter; + + while (1) { + if (!src_iter.bi_size) { + src = src->bi_next; + if (!src) + break; + + src_iter = src->bi_iter; + } + + if (!dst_iter.bi_size) { + dst = dst->bi_next; + if (!dst) + break; + + dst_iter = dst->bi_iter; + } + + bio_copy_data_iter(dst, &dst_iter, src, &src_iter); + } +} + +/* + * Assemble a bio to write one packet and queue the bio for processing + * by the underlying block device. + */ +static void pkt_start_write(struct pktcdvd_device *pd, struct packet_data *pkt) +{ + int f; + + bio_init(pkt->w_bio, pd->bdev, pkt->w_bio->bi_inline_vecs, pkt->frames, + REQ_OP_WRITE); + pkt->w_bio->bi_iter.bi_sector = pkt->sector; + pkt->w_bio->bi_end_io = pkt_end_io_packet_write; + pkt->w_bio->bi_private = pkt; + + /* XXX: locking? */ + for (f = 0; f < pkt->frames; f++) { + struct page *page = pkt->pages[(f * CD_FRAMESIZE) / PAGE_SIZE]; + unsigned offset = (f * CD_FRAMESIZE) % PAGE_SIZE; + + if (!bio_add_page(pkt->w_bio, page, CD_FRAMESIZE, offset)) + BUG(); + } + pkt_dbg(2, pd, "vcnt=%d\n", pkt->w_bio->bi_vcnt); + + /* + * Fill-in bvec with data from orig_bios. + */ + spin_lock(&pkt->lock); + bio_list_copy_data(pkt->w_bio, pkt->orig_bios.head); + + pkt_set_state(pkt, PACKET_WRITE_WAIT_STATE); + spin_unlock(&pkt->lock); + + pkt_dbg(2, pd, "Writing %d frames for zone %llx\n", + pkt->write_size, (unsigned long long)pkt->sector); + + if (test_bit(PACKET_MERGE_SEGS, &pd->flags) || (pkt->write_size < pkt->frames)) + pkt->cache_valid = 1; + else + pkt->cache_valid = 0; + + /* Start the write request */ + atomic_set(&pkt->io_wait, 1); + pkt_queue_bio(pd, pkt->w_bio); +} + +static void pkt_finish_packet(struct packet_data *pkt, blk_status_t status) +{ + struct bio *bio; + + if (status) + pkt->cache_valid = 0; + + /* Finish all bios corresponding to this packet */ + while ((bio = bio_list_pop(&pkt->orig_bios))) { + bio->bi_status = status; + bio_endio(bio); + } +} + +static void pkt_run_state_machine(struct pktcdvd_device *pd, struct packet_data *pkt) +{ + pkt_dbg(2, pd, "pkt %d\n", pkt->id); + + for (;;) { + switch (pkt->state) { + case PACKET_WAITING_STATE: + if ((pkt->write_size < pkt->frames) && (pkt->sleep_time > 0)) + return; + + pkt->sleep_time = 0; + pkt_gather_data(pd, pkt); + pkt_set_state(pkt, PACKET_READ_WAIT_STATE); + break; + + case PACKET_READ_WAIT_STATE: + if (atomic_read(&pkt->io_wait) > 0) + return; + + if (atomic_read(&pkt->io_errors) > 0) { + pkt_set_state(pkt, PACKET_RECOVERY_STATE); + } else { + pkt_start_write(pd, pkt); + } + break; + + case PACKET_WRITE_WAIT_STATE: + if (atomic_read(&pkt->io_wait) > 0) + return; + + if (!pkt->w_bio->bi_status) { + pkt_set_state(pkt, PACKET_FINISHED_STATE); + } else { + pkt_set_state(pkt, PACKET_RECOVERY_STATE); + } + break; + + case PACKET_RECOVERY_STATE: + pkt_dbg(2, pd, "No recovery possible\n"); + pkt_set_state(pkt, PACKET_FINISHED_STATE); + break; + + case PACKET_FINISHED_STATE: + pkt_finish_packet(pkt, pkt->w_bio->bi_status); + return; + + default: + BUG(); + break; + } + } +} + +static void pkt_handle_packets(struct pktcdvd_device *pd) +{ + struct packet_data *pkt, *next; + + /* + * Run state machine for active packets + */ + list_for_each_entry(pkt, &pd->cdrw.pkt_active_list, list) { + if (atomic_read(&pkt->run_sm) > 0) { + atomic_set(&pkt->run_sm, 0); + pkt_run_state_machine(pd, pkt); + } + } + + /* + * Move no longer active packets to the free list + */ + spin_lock(&pd->cdrw.active_list_lock); + list_for_each_entry_safe(pkt, next, &pd->cdrw.pkt_active_list, list) { + if (pkt->state == PACKET_FINISHED_STATE) { + list_del(&pkt->list); + pkt_put_packet_data(pd, pkt); + pkt_set_state(pkt, PACKET_IDLE_STATE); + atomic_set(&pd->scan_queue, 1); + } + } + spin_unlock(&pd->cdrw.active_list_lock); +} + +static void pkt_count_states(struct pktcdvd_device *pd, int *states) +{ + struct packet_data *pkt; + int i; + + for (i = 0; i < PACKET_NUM_STATES; i++) + states[i] = 0; + + spin_lock(&pd->cdrw.active_list_lock); + list_for_each_entry(pkt, &pd->cdrw.pkt_active_list, list) { + states[pkt->state]++; + } + spin_unlock(&pd->cdrw.active_list_lock); +} + +/* + * kcdrwd is woken up when writes have been queued for one of our + * registered devices + */ +static int kcdrwd(void *foobar) +{ + struct pktcdvd_device *pd = foobar; + struct packet_data *pkt; + long min_sleep_time, residue; + + set_user_nice(current, MIN_NICE); + set_freezable(); + + for (;;) { + DECLARE_WAITQUEUE(wait, current); + + /* + * Wait until there is something to do + */ + add_wait_queue(&pd->wqueue, &wait); + for (;;) { + set_current_state(TASK_INTERRUPTIBLE); + + /* Check if we need to run pkt_handle_queue */ + if (atomic_read(&pd->scan_queue) > 0) + goto work_to_do; + + /* Check if we need to run the state machine for some packet */ + list_for_each_entry(pkt, &pd->cdrw.pkt_active_list, list) { + if (atomic_read(&pkt->run_sm) > 0) + goto work_to_do; + } + + /* Check if we need to process the iosched queues */ + if (atomic_read(&pd->iosched.attention) != 0) + goto work_to_do; + + /* Otherwise, go to sleep */ + if (PACKET_DEBUG > 1) { + int states[PACKET_NUM_STATES]; + pkt_count_states(pd, states); + pkt_dbg(2, pd, "i:%d ow:%d rw:%d ww:%d rec:%d fin:%d\n", + states[0], states[1], states[2], + states[3], states[4], states[5]); + } + + min_sleep_time = MAX_SCHEDULE_TIMEOUT; + list_for_each_entry(pkt, &pd->cdrw.pkt_active_list, list) { + if (pkt->sleep_time && pkt->sleep_time < min_sleep_time) + min_sleep_time = pkt->sleep_time; + } + + pkt_dbg(2, pd, "sleeping\n"); + residue = schedule_timeout(min_sleep_time); + pkt_dbg(2, pd, "wake up\n"); + + /* make swsusp happy with our thread */ + try_to_freeze(); + + list_for_each_entry(pkt, &pd->cdrw.pkt_active_list, list) { + if (!pkt->sleep_time) + continue; + pkt->sleep_time -= min_sleep_time - residue; + if (pkt->sleep_time <= 0) { + pkt->sleep_time = 0; + atomic_inc(&pkt->run_sm); + } + } + + if (kthread_should_stop()) + break; + } +work_to_do: + set_current_state(TASK_RUNNING); + remove_wait_queue(&pd->wqueue, &wait); + + if (kthread_should_stop()) + break; + + /* + * if pkt_handle_queue returns true, we can queue + * another request. + */ + while (pkt_handle_queue(pd)) + ; + + /* + * Handle packet state machine + */ + pkt_handle_packets(pd); + + /* + * Handle iosched queues + */ + pkt_iosched_process_queue(pd); + } + + return 0; +} + +static void pkt_print_settings(struct pktcdvd_device *pd) +{ + pkt_info(pd, "%s packets, %u blocks, Mode-%c disc\n", + pd->settings.fp ? "Fixed" : "Variable", + pd->settings.size >> 2, + pd->settings.block_mode == 8 ? '1' : '2'); +} + +static int pkt_mode_sense(struct pktcdvd_device *pd, struct packet_command *cgc, int page_code, int page_control) +{ + memset(cgc->cmd, 0, sizeof(cgc->cmd)); + + cgc->cmd[0] = GPCMD_MODE_SENSE_10; + cgc->cmd[2] = page_code | (page_control << 6); + cgc->cmd[7] = cgc->buflen >> 8; + cgc->cmd[8] = cgc->buflen & 0xff; + cgc->data_direction = CGC_DATA_READ; + return pkt_generic_packet(pd, cgc); +} + +static int pkt_mode_select(struct pktcdvd_device *pd, struct packet_command *cgc) +{ + memset(cgc->cmd, 0, sizeof(cgc->cmd)); + memset(cgc->buffer, 0, 2); + cgc->cmd[0] = GPCMD_MODE_SELECT_10; + cgc->cmd[1] = 0x10; /* PF */ + cgc->cmd[7] = cgc->buflen >> 8; + cgc->cmd[8] = cgc->buflen & 0xff; + cgc->data_direction = CGC_DATA_WRITE; + return pkt_generic_packet(pd, cgc); +} + +static int pkt_get_disc_info(struct pktcdvd_device *pd, disc_information *di) +{ + struct packet_command cgc; + int ret; + + /* set up command and get the disc info */ + init_cdrom_command(&cgc, di, sizeof(*di), CGC_DATA_READ); + cgc.cmd[0] = GPCMD_READ_DISC_INFO; + cgc.cmd[8] = cgc.buflen = 2; + cgc.quiet = 1; + + ret = pkt_generic_packet(pd, &cgc); + if (ret) + return ret; + + /* not all drives have the same disc_info length, so requeue + * packet with the length the drive tells us it can supply + */ + cgc.buflen = be16_to_cpu(di->disc_information_length) + + sizeof(di->disc_information_length); + + if (cgc.buflen > sizeof(disc_information)) + cgc.buflen = sizeof(disc_information); + + cgc.cmd[8] = cgc.buflen; + return pkt_generic_packet(pd, &cgc); +} + +static int pkt_get_track_info(struct pktcdvd_device *pd, __u16 track, __u8 type, track_information *ti) +{ + struct packet_command cgc; + int ret; + + init_cdrom_command(&cgc, ti, 8, CGC_DATA_READ); + cgc.cmd[0] = GPCMD_READ_TRACK_RZONE_INFO; + cgc.cmd[1] = type & 3; + cgc.cmd[4] = (track & 0xff00) >> 8; + cgc.cmd[5] = track & 0xff; + cgc.cmd[8] = 8; + cgc.quiet = 1; + + ret = pkt_generic_packet(pd, &cgc); + if (ret) + return ret; + + cgc.buflen = be16_to_cpu(ti->track_information_length) + + sizeof(ti->track_information_length); + + if (cgc.buflen > sizeof(track_information)) + cgc.buflen = sizeof(track_information); + + cgc.cmd[8] = cgc.buflen; + return pkt_generic_packet(pd, &cgc); +} + +static noinline_for_stack int pkt_get_last_written(struct pktcdvd_device *pd, + long *last_written) +{ + disc_information di; + track_information ti; + __u32 last_track; + int ret; + + ret = pkt_get_disc_info(pd, &di); + if (ret) + return ret; + + last_track = (di.last_track_msb << 8) | di.last_track_lsb; + ret = pkt_get_track_info(pd, last_track, 1, &ti); + if (ret) + return ret; + + /* if this track is blank, try the previous. */ + if (ti.blank) { + last_track--; + ret = pkt_get_track_info(pd, last_track, 1, &ti); + if (ret) + return ret; + } + + /* if last recorded field is valid, return it. */ + if (ti.lra_v) { + *last_written = be32_to_cpu(ti.last_rec_address); + } else { + /* make it up instead */ + *last_written = be32_to_cpu(ti.track_start) + + be32_to_cpu(ti.track_size); + if (ti.free_blocks) + *last_written -= (be32_to_cpu(ti.free_blocks) + 7); + } + return 0; +} + +/* + * write mode select package based on pd->settings + */ +static noinline_for_stack int pkt_set_write_settings(struct pktcdvd_device *pd) +{ + struct packet_command cgc; + struct scsi_sense_hdr sshdr; + write_param_page *wp; + char buffer[128]; + int ret, size; + + /* doesn't apply to DVD+RW or DVD-RAM */ + if ((pd->mmc3_profile == 0x1a) || (pd->mmc3_profile == 0x12)) + return 0; + + memset(buffer, 0, sizeof(buffer)); + init_cdrom_command(&cgc, buffer, sizeof(*wp), CGC_DATA_READ); + cgc.sshdr = &sshdr; + ret = pkt_mode_sense(pd, &cgc, GPMODE_WRITE_PARMS_PAGE, 0); + if (ret) { + pkt_dump_sense(pd, &cgc); + return ret; + } + + size = 2 + ((buffer[0] << 8) | (buffer[1] & 0xff)); + pd->mode_offset = (buffer[6] << 8) | (buffer[7] & 0xff); + if (size > sizeof(buffer)) + size = sizeof(buffer); + + /* + * now get it all + */ + init_cdrom_command(&cgc, buffer, size, CGC_DATA_READ); + cgc.sshdr = &sshdr; + ret = pkt_mode_sense(pd, &cgc, GPMODE_WRITE_PARMS_PAGE, 0); + if (ret) { + pkt_dump_sense(pd, &cgc); + return ret; + } + + /* + * write page is offset header + block descriptor length + */ + wp = (write_param_page *) &buffer[sizeof(struct mode_page_header) + pd->mode_offset]; + + wp->fp = pd->settings.fp; + wp->track_mode = pd->settings.track_mode; + wp->write_type = pd->settings.write_type; + wp->data_block_type = pd->settings.block_mode; + + wp->multi_session = 0; + +#ifdef PACKET_USE_LS + wp->link_size = 7; + wp->ls_v = 1; +#endif + + if (wp->data_block_type == PACKET_BLOCK_MODE1) { + wp->session_format = 0; + wp->subhdr2 = 0x20; + } else if (wp->data_block_type == PACKET_BLOCK_MODE2) { + wp->session_format = 0x20; + wp->subhdr2 = 8; +#if 0 + wp->mcn[0] = 0x80; + memcpy(&wp->mcn[1], PACKET_MCN, sizeof(wp->mcn) - 1); +#endif + } else { + /* + * paranoia + */ + pkt_err(pd, "write mode wrong %d\n", wp->data_block_type); + return 1; + } + wp->packet_size = cpu_to_be32(pd->settings.size >> 2); + + cgc.buflen = cgc.cmd[8] = size; + ret = pkt_mode_select(pd, &cgc); + if (ret) { + pkt_dump_sense(pd, &cgc); + return ret; + } + + pkt_print_settings(pd); + return 0; +} + +/* + * 1 -- we can write to this track, 0 -- we can't + */ +static int pkt_writable_track(struct pktcdvd_device *pd, track_information *ti) +{ + switch (pd->mmc3_profile) { + case 0x1a: /* DVD+RW */ + case 0x12: /* DVD-RAM */ + /* The track is always writable on DVD+RW/DVD-RAM */ + return 1; + default: + break; + } + + if (!ti->packet || !ti->fp) + return 0; + + /* + * "good" settings as per Mt Fuji. + */ + if (ti->rt == 0 && ti->blank == 0) + return 1; + + if (ti->rt == 0 && ti->blank == 1) + return 1; + + if (ti->rt == 1 && ti->blank == 0) + return 1; + + pkt_err(pd, "bad state %d-%d-%d\n", ti->rt, ti->blank, ti->packet); + return 0; +} + +/* + * 1 -- we can write to this disc, 0 -- we can't + */ +static int pkt_writable_disc(struct pktcdvd_device *pd, disc_information *di) +{ + switch (pd->mmc3_profile) { + case 0x0a: /* CD-RW */ + case 0xffff: /* MMC3 not supported */ + break; + case 0x1a: /* DVD+RW */ + case 0x13: /* DVD-RW */ + case 0x12: /* DVD-RAM */ + return 1; + default: + pkt_dbg(2, pd, "Wrong disc profile (%x)\n", + pd->mmc3_profile); + return 0; + } + + /* + * for disc type 0xff we should probably reserve a new track. + * but i'm not sure, should we leave this to user apps? probably. + */ + if (di->disc_type == 0xff) { + pkt_notice(pd, "unknown disc - no track?\n"); + return 0; + } + + if (di->disc_type != 0x20 && di->disc_type != 0) { + pkt_err(pd, "wrong disc type (%x)\n", di->disc_type); + return 0; + } + + if (di->erasable == 0) { + pkt_notice(pd, "disc not erasable\n"); + return 0; + } + + if (di->border_status == PACKET_SESSION_RESERVED) { + pkt_err(pd, "can't write to last track (reserved)\n"); + return 0; + } + + return 1; +} + +static noinline_for_stack int pkt_probe_settings(struct pktcdvd_device *pd) +{ + struct packet_command cgc; + unsigned char buf[12]; + disc_information di; + track_information ti; + int ret, track; + + init_cdrom_command(&cgc, buf, sizeof(buf), CGC_DATA_READ); + cgc.cmd[0] = GPCMD_GET_CONFIGURATION; + cgc.cmd[8] = 8; + ret = pkt_generic_packet(pd, &cgc); + pd->mmc3_profile = ret ? 0xffff : buf[6] << 8 | buf[7]; + + memset(&di, 0, sizeof(disc_information)); + memset(&ti, 0, sizeof(track_information)); + + ret = pkt_get_disc_info(pd, &di); + if (ret) { + pkt_err(pd, "failed get_disc\n"); + return ret; + } + + if (!pkt_writable_disc(pd, &di)) + return -EROFS; + + pd->type = di.erasable ? PACKET_CDRW : PACKET_CDR; + + track = 1; /* (di.last_track_msb << 8) | di.last_track_lsb; */ + ret = pkt_get_track_info(pd, track, 1, &ti); + if (ret) { + pkt_err(pd, "failed get_track\n"); + return ret; + } + + if (!pkt_writable_track(pd, &ti)) { + pkt_err(pd, "can't write to this track\n"); + return -EROFS; + } + + /* + * we keep packet size in 512 byte units, makes it easier to + * deal with request calculations. + */ + pd->settings.size = be32_to_cpu(ti.fixed_packet_size) << 2; + if (pd->settings.size == 0) { + pkt_notice(pd, "detected zero packet size!\n"); + return -ENXIO; + } + if (pd->settings.size > PACKET_MAX_SECTORS) { + pkt_err(pd, "packet size is too big\n"); + return -EROFS; + } + pd->settings.fp = ti.fp; + pd->offset = (be32_to_cpu(ti.track_start) << 2) & (pd->settings.size - 1); + + if (ti.nwa_v) { + pd->nwa = be32_to_cpu(ti.next_writable); + set_bit(PACKET_NWA_VALID, &pd->flags); + } + + /* + * in theory we could use lra on -RW media as well and just zero + * blocks that haven't been written yet, but in practice that + * is just a no-go. we'll use that for -R, naturally. + */ + if (ti.lra_v) { + pd->lra = be32_to_cpu(ti.last_rec_address); + set_bit(PACKET_LRA_VALID, &pd->flags); + } else { + pd->lra = 0xffffffff; + set_bit(PACKET_LRA_VALID, &pd->flags); + } + + /* + * fine for now + */ + pd->settings.link_loss = 7; + pd->settings.write_type = 0; /* packet */ + pd->settings.track_mode = ti.track_mode; + + /* + * mode1 or mode2 disc + */ + switch (ti.data_mode) { + case PACKET_MODE1: + pd->settings.block_mode = PACKET_BLOCK_MODE1; + break; + case PACKET_MODE2: + pd->settings.block_mode = PACKET_BLOCK_MODE2; + break; + default: + pkt_err(pd, "unknown data mode\n"); + return -EROFS; + } + return 0; +} + +/* + * enable/disable write caching on drive + */ +static noinline_for_stack int pkt_write_caching(struct pktcdvd_device *pd, + int set) +{ + struct packet_command cgc; + struct scsi_sense_hdr sshdr; + unsigned char buf[64]; + int ret; + + init_cdrom_command(&cgc, buf, sizeof(buf), CGC_DATA_READ); + cgc.sshdr = &sshdr; + cgc.buflen = pd->mode_offset + 12; + + /* + * caching mode page might not be there, so quiet this command + */ + cgc.quiet = 1; + + ret = pkt_mode_sense(pd, &cgc, GPMODE_WCACHING_PAGE, 0); + if (ret) + return ret; + + buf[pd->mode_offset + 10] |= (!!set << 2); + + cgc.buflen = cgc.cmd[8] = 2 + ((buf[0] << 8) | (buf[1] & 0xff)); + ret = pkt_mode_select(pd, &cgc); + if (ret) { + pkt_err(pd, "write caching control failed\n"); + pkt_dump_sense(pd, &cgc); + } else if (!ret && set) + pkt_notice(pd, "enabled write caching\n"); + return ret; +} + +static int pkt_lock_door(struct pktcdvd_device *pd, int lockflag) +{ + struct packet_command cgc; + + init_cdrom_command(&cgc, NULL, 0, CGC_DATA_NONE); + cgc.cmd[0] = GPCMD_PREVENT_ALLOW_MEDIUM_REMOVAL; + cgc.cmd[4] = lockflag ? 1 : 0; + return pkt_generic_packet(pd, &cgc); +} + +/* + * Returns drive maximum write speed + */ +static noinline_for_stack int pkt_get_max_speed(struct pktcdvd_device *pd, + unsigned *write_speed) +{ + struct packet_command cgc; + struct scsi_sense_hdr sshdr; + unsigned char buf[256+18]; + unsigned char *cap_buf; + int ret, offset; + + cap_buf = &buf[sizeof(struct mode_page_header) + pd->mode_offset]; + init_cdrom_command(&cgc, buf, sizeof(buf), CGC_DATA_UNKNOWN); + cgc.sshdr = &sshdr; + + ret = pkt_mode_sense(pd, &cgc, GPMODE_CAPABILITIES_PAGE, 0); + if (ret) { + cgc.buflen = pd->mode_offset + cap_buf[1] + 2 + + sizeof(struct mode_page_header); + ret = pkt_mode_sense(pd, &cgc, GPMODE_CAPABILITIES_PAGE, 0); + if (ret) { + pkt_dump_sense(pd, &cgc); + return ret; + } + } + + offset = 20; /* Obsoleted field, used by older drives */ + if (cap_buf[1] >= 28) + offset = 28; /* Current write speed selected */ + if (cap_buf[1] >= 30) { + /* If the drive reports at least one "Logical Unit Write + * Speed Performance Descriptor Block", use the information + * in the first block. (contains the highest speed) + */ + int num_spdb = (cap_buf[30] << 8) + cap_buf[31]; + if (num_spdb > 0) + offset = 34; + } + + *write_speed = (cap_buf[offset] << 8) | cap_buf[offset + 1]; + return 0; +} + +/* These tables from cdrecord - I don't have orange book */ +/* standard speed CD-RW (1-4x) */ +static char clv_to_speed[16] = { + /* 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 */ + 0, 2, 4, 6, 8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 +}; +/* high speed CD-RW (-10x) */ +static char hs_clv_to_speed[16] = { + /* 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 */ + 0, 2, 4, 6, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 +}; +/* ultra high speed CD-RW */ +static char us_clv_to_speed[16] = { + /* 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 */ + 0, 2, 4, 8, 0, 0,16, 0,24,32,40,48, 0, 0, 0, 0 +}; + +/* + * reads the maximum media speed from ATIP + */ +static noinline_for_stack int pkt_media_speed(struct pktcdvd_device *pd, + unsigned *speed) +{ + struct packet_command cgc; + struct scsi_sense_hdr sshdr; + unsigned char buf[64]; + unsigned int size, st, sp; + int ret; + + init_cdrom_command(&cgc, buf, 2, CGC_DATA_READ); + cgc.sshdr = &sshdr; + cgc.cmd[0] = GPCMD_READ_TOC_PMA_ATIP; + cgc.cmd[1] = 2; + cgc.cmd[2] = 4; /* READ ATIP */ + cgc.cmd[8] = 2; + ret = pkt_generic_packet(pd, &cgc); + if (ret) { + pkt_dump_sense(pd, &cgc); + return ret; + } + size = ((unsigned int) buf[0]<<8) + buf[1] + 2; + if (size > sizeof(buf)) + size = sizeof(buf); + + init_cdrom_command(&cgc, buf, size, CGC_DATA_READ); + cgc.sshdr = &sshdr; + cgc.cmd[0] = GPCMD_READ_TOC_PMA_ATIP; + cgc.cmd[1] = 2; + cgc.cmd[2] = 4; + cgc.cmd[8] = size; + ret = pkt_generic_packet(pd, &cgc); + if (ret) { + pkt_dump_sense(pd, &cgc); + return ret; + } + + if (!(buf[6] & 0x40)) { + pkt_notice(pd, "disc type is not CD-RW\n"); + return 1; + } + if (!(buf[6] & 0x4)) { + pkt_notice(pd, "A1 values on media are not valid, maybe not CDRW?\n"); + return 1; + } + + st = (buf[6] >> 3) & 0x7; /* disc sub-type */ + + sp = buf[16] & 0xf; /* max speed from ATIP A1 field */ + + /* Info from cdrecord */ + switch (st) { + case 0: /* standard speed */ + *speed = clv_to_speed[sp]; + break; + case 1: /* high speed */ + *speed = hs_clv_to_speed[sp]; + break; + case 2: /* ultra high speed */ + *speed = us_clv_to_speed[sp]; + break; + default: + pkt_notice(pd, "unknown disc sub-type %d\n", st); + return 1; + } + if (*speed) { + pkt_info(pd, "maximum media speed: %d\n", *speed); + return 0; + } else { + pkt_notice(pd, "unknown speed %d for sub-type %d\n", sp, st); + return 1; + } +} + +static noinline_for_stack int pkt_perform_opc(struct pktcdvd_device *pd) +{ + struct packet_command cgc; + struct scsi_sense_hdr sshdr; + int ret; + + pkt_dbg(2, pd, "Performing OPC\n"); + + init_cdrom_command(&cgc, NULL, 0, CGC_DATA_NONE); + cgc.sshdr = &sshdr; + cgc.timeout = 60*HZ; + cgc.cmd[0] = GPCMD_SEND_OPC; + cgc.cmd[1] = 1; + ret = pkt_generic_packet(pd, &cgc); + if (ret) + pkt_dump_sense(pd, &cgc); + return ret; +} + +static int pkt_open_write(struct pktcdvd_device *pd) +{ + int ret; + unsigned int write_speed, media_write_speed, read_speed; + + ret = pkt_probe_settings(pd); + if (ret) { + pkt_dbg(2, pd, "failed probe\n"); + return ret; + } + + ret = pkt_set_write_settings(pd); + if (ret) { + pkt_dbg(1, pd, "failed saving write settings\n"); + return -EIO; + } + + pkt_write_caching(pd, USE_WCACHING); + + ret = pkt_get_max_speed(pd, &write_speed); + if (ret) + write_speed = 16 * 177; + switch (pd->mmc3_profile) { + case 0x13: /* DVD-RW */ + case 0x1a: /* DVD+RW */ + case 0x12: /* DVD-RAM */ + pkt_dbg(1, pd, "write speed %ukB/s\n", write_speed); + break; + default: + ret = pkt_media_speed(pd, &media_write_speed); + if (ret) + media_write_speed = 16; + write_speed = min(write_speed, media_write_speed * 177); + pkt_dbg(1, pd, "write speed %ux\n", write_speed / 176); + break; + } + read_speed = write_speed; + + ret = pkt_set_speed(pd, write_speed, read_speed); + if (ret) { + pkt_dbg(1, pd, "couldn't set write speed\n"); + return -EIO; + } + pd->write_speed = write_speed; + pd->read_speed = read_speed; + + ret = pkt_perform_opc(pd); + if (ret) { + pkt_dbg(1, pd, "Optimum Power Calibration failed\n"); + } + + return 0; +} + +/* + * called at open time. + */ +static int pkt_open_dev(struct pktcdvd_device *pd, fmode_t write) +{ + int ret; + long lba; + struct request_queue *q; + struct block_device *bdev; + + /* + * We need to re-open the cdrom device without O_NONBLOCK to be able + * to read/write from/to it. It is already opened in O_NONBLOCK mode + * so open should not fail. + */ + bdev = blkdev_get_by_dev(pd->bdev->bd_dev, FMODE_READ | FMODE_EXCL, pd); + if (IS_ERR(bdev)) { + ret = PTR_ERR(bdev); + goto out; + } + + ret = pkt_get_last_written(pd, &lba); + if (ret) { + pkt_err(pd, "pkt_get_last_written failed\n"); + goto out_putdev; + } + + set_capacity(pd->disk, lba << 2); + set_capacity_and_notify(pd->bdev->bd_disk, lba << 2); + + q = bdev_get_queue(pd->bdev); + if (write) { + ret = pkt_open_write(pd); + if (ret) + goto out_putdev; + /* + * Some CDRW drives can not handle writes larger than one packet, + * even if the size is a multiple of the packet size. + */ + blk_queue_max_hw_sectors(q, pd->settings.size); + set_bit(PACKET_WRITABLE, &pd->flags); + } else { + pkt_set_speed(pd, MAX_SPEED, MAX_SPEED); + clear_bit(PACKET_WRITABLE, &pd->flags); + } + + ret = pkt_set_segment_merging(pd, q); + if (ret) + goto out_putdev; + + if (write) { + if (!pkt_grow_pktlist(pd, CONFIG_CDROM_PKTCDVD_BUFFERS)) { + pkt_err(pd, "not enough memory for buffers\n"); + ret = -ENOMEM; + goto out_putdev; + } + pkt_info(pd, "%lukB available on disc\n", lba << 1); + } + + return 0; + +out_putdev: + blkdev_put(bdev, FMODE_READ | FMODE_EXCL); +out: + return ret; +} + +/* + * called when the device is closed. makes sure that the device flushes + * the internal cache before we close. + */ +static void pkt_release_dev(struct pktcdvd_device *pd, int flush) +{ + if (flush && pkt_flush_cache(pd)) + pkt_dbg(1, pd, "not flushing cache\n"); + + pkt_lock_door(pd, 0); + + pkt_set_speed(pd, MAX_SPEED, MAX_SPEED); + blkdev_put(pd->bdev, FMODE_READ | FMODE_EXCL); + + pkt_shrink_pktlist(pd); +} + +static struct pktcdvd_device *pkt_find_dev_from_minor(unsigned int dev_minor) +{ + if (dev_minor >= MAX_WRITERS) + return NULL; + + dev_minor = array_index_nospec(dev_minor, MAX_WRITERS); + return pkt_devs[dev_minor]; +} + +static int pkt_open(struct block_device *bdev, fmode_t mode) +{ + struct pktcdvd_device *pd = NULL; + int ret; + + mutex_lock(&pktcdvd_mutex); + mutex_lock(&ctl_mutex); + pd = pkt_find_dev_from_minor(MINOR(bdev->bd_dev)); + if (!pd) { + ret = -ENODEV; + goto out; + } + BUG_ON(pd->refcnt < 0); + + pd->refcnt++; + if (pd->refcnt > 1) { + if ((mode & FMODE_WRITE) && + !test_bit(PACKET_WRITABLE, &pd->flags)) { + ret = -EBUSY; + goto out_dec; + } + } else { + ret = pkt_open_dev(pd, mode & FMODE_WRITE); + if (ret) + goto out_dec; + /* + * needed here as well, since ext2 (among others) may change + * the blocksize at mount time + */ + set_blocksize(bdev, CD_FRAMESIZE); + } + + mutex_unlock(&ctl_mutex); + mutex_unlock(&pktcdvd_mutex); + return 0; + +out_dec: + pd->refcnt--; +out: + mutex_unlock(&ctl_mutex); + mutex_unlock(&pktcdvd_mutex); + return ret; +} + +static void pkt_close(struct gendisk *disk, fmode_t mode) +{ + struct pktcdvd_device *pd = disk->private_data; + + mutex_lock(&pktcdvd_mutex); + mutex_lock(&ctl_mutex); + pd->refcnt--; + BUG_ON(pd->refcnt < 0); + if (pd->refcnt == 0) { + int flush = test_bit(PACKET_WRITABLE, &pd->flags); + pkt_release_dev(pd, flush); + } + mutex_unlock(&ctl_mutex); + mutex_unlock(&pktcdvd_mutex); +} + + +static void pkt_end_io_read_cloned(struct bio *bio) +{ + struct packet_stacked_data *psd = bio->bi_private; + struct pktcdvd_device *pd = psd->pd; + + psd->bio->bi_status = bio->bi_status; + bio_put(bio); + bio_endio(psd->bio); + mempool_free(psd, &psd_pool); + pkt_bio_finished(pd); +} + +static void pkt_make_request_read(struct pktcdvd_device *pd, struct bio *bio) +{ + struct bio *cloned_bio = + bio_alloc_clone(pd->bdev, bio, GFP_NOIO, &pkt_bio_set); + struct packet_stacked_data *psd = mempool_alloc(&psd_pool, GFP_NOIO); + + psd->pd = pd; + psd->bio = bio; + cloned_bio->bi_private = psd; + cloned_bio->bi_end_io = pkt_end_io_read_cloned; + pd->stats.secs_r += bio_sectors(bio); + pkt_queue_bio(pd, cloned_bio); +} + +static void pkt_make_request_write(struct request_queue *q, struct bio *bio) +{ + struct pktcdvd_device *pd = q->queuedata; + sector_t zone; + struct packet_data *pkt; + int was_empty, blocked_bio; + struct pkt_rb_node *node; + + zone = get_zone(bio->bi_iter.bi_sector, pd); + + /* + * If we find a matching packet in state WAITING or READ_WAIT, we can + * just append this bio to that packet. + */ + spin_lock(&pd->cdrw.active_list_lock); + blocked_bio = 0; + list_for_each_entry(pkt, &pd->cdrw.pkt_active_list, list) { + if (pkt->sector == zone) { + spin_lock(&pkt->lock); + if ((pkt->state == PACKET_WAITING_STATE) || + (pkt->state == PACKET_READ_WAIT_STATE)) { + bio_list_add(&pkt->orig_bios, bio); + pkt->write_size += + bio->bi_iter.bi_size / CD_FRAMESIZE; + if ((pkt->write_size >= pkt->frames) && + (pkt->state == PACKET_WAITING_STATE)) { + atomic_inc(&pkt->run_sm); + wake_up(&pd->wqueue); + } + spin_unlock(&pkt->lock); + spin_unlock(&pd->cdrw.active_list_lock); + return; + } else { + blocked_bio = 1; + } + spin_unlock(&pkt->lock); + } + } + spin_unlock(&pd->cdrw.active_list_lock); + + /* + * Test if there is enough room left in the bio work queue + * (queue size >= congestion on mark). + * If not, wait till the work queue size is below the congestion off mark. + */ + spin_lock(&pd->lock); + if (pd->write_congestion_on > 0 + && pd->bio_queue_size >= pd->write_congestion_on) { + struct wait_bit_queue_entry wqe; + + init_wait_var_entry(&wqe, &pd->congested, 0); + for (;;) { + prepare_to_wait_event(__var_waitqueue(&pd->congested), + &wqe.wq_entry, + TASK_UNINTERRUPTIBLE); + if (pd->bio_queue_size <= pd->write_congestion_off) + break; + pd->congested = true; + spin_unlock(&pd->lock); + schedule(); + spin_lock(&pd->lock); + } + } + spin_unlock(&pd->lock); + + /* + * No matching packet found. Store the bio in the work queue. + */ + node = mempool_alloc(&pd->rb_pool, GFP_NOIO); + node->bio = bio; + spin_lock(&pd->lock); + BUG_ON(pd->bio_queue_size < 0); + was_empty = (pd->bio_queue_size == 0); + pkt_rbtree_insert(pd, node); + spin_unlock(&pd->lock); + + /* + * Wake up the worker thread. + */ + atomic_set(&pd->scan_queue, 1); + if (was_empty) { + /* This wake_up is required for correct operation */ + wake_up(&pd->wqueue); + } else if (!list_empty(&pd->cdrw.pkt_free_list) && !blocked_bio) { + /* + * This wake up is not required for correct operation, + * but improves performance in some cases. + */ + wake_up(&pd->wqueue); + } +} + +static void pkt_submit_bio(struct bio *bio) +{ + struct pktcdvd_device *pd = bio->bi_bdev->bd_disk->queue->queuedata; + struct bio *split; + + bio = bio_split_to_limits(bio); + if (!bio) + return; + + pkt_dbg(2, pd, "start = %6llx stop = %6llx\n", + (unsigned long long)bio->bi_iter.bi_sector, + (unsigned long long)bio_end_sector(bio)); + + /* + * Clone READ bios so we can have our own bi_end_io callback. + */ + if (bio_data_dir(bio) == READ) { + pkt_make_request_read(pd, bio); + return; + } + + if (!test_bit(PACKET_WRITABLE, &pd->flags)) { + pkt_notice(pd, "WRITE for ro device (%llu)\n", + (unsigned long long)bio->bi_iter.bi_sector); + goto end_io; + } + + if (!bio->bi_iter.bi_size || (bio->bi_iter.bi_size % CD_FRAMESIZE)) { + pkt_err(pd, "wrong bio size\n"); + goto end_io; + } + + do { + sector_t zone = get_zone(bio->bi_iter.bi_sector, pd); + sector_t last_zone = get_zone(bio_end_sector(bio) - 1, pd); + + if (last_zone != zone) { + BUG_ON(last_zone != zone + pd->settings.size); + + split = bio_split(bio, last_zone - + bio->bi_iter.bi_sector, + GFP_NOIO, &pkt_bio_set); + bio_chain(split, bio); + } else { + split = bio; + } + + pkt_make_request_write(bio->bi_bdev->bd_disk->queue, split); + } while (split != bio); + + return; +end_io: + bio_io_error(bio); +} + +static void pkt_init_queue(struct pktcdvd_device *pd) +{ + struct request_queue *q = pd->disk->queue; + + blk_queue_logical_block_size(q, CD_FRAMESIZE); + blk_queue_max_hw_sectors(q, PACKET_MAX_SECTORS); + q->queuedata = pd; +} + +static int pkt_seq_show(struct seq_file *m, void *p) +{ + struct pktcdvd_device *pd = m->private; + char *msg; + int states[PACKET_NUM_STATES]; + + seq_printf(m, "Writer %s mapped to %pg:\n", pd->name, pd->bdev); + + seq_printf(m, "\nSettings:\n"); + seq_printf(m, "\tpacket size:\t\t%dkB\n", pd->settings.size / 2); + + if (pd->settings.write_type == 0) + msg = "Packet"; + else + msg = "Unknown"; + seq_printf(m, "\twrite type:\t\t%s\n", msg); + + seq_printf(m, "\tpacket type:\t\t%s\n", pd->settings.fp ? "Fixed" : "Variable"); + seq_printf(m, "\tlink loss:\t\t%d\n", pd->settings.link_loss); + + seq_printf(m, "\ttrack mode:\t\t%d\n", pd->settings.track_mode); + + if (pd->settings.block_mode == PACKET_BLOCK_MODE1) + msg = "Mode 1"; + else if (pd->settings.block_mode == PACKET_BLOCK_MODE2) + msg = "Mode 2"; + else + msg = "Unknown"; + seq_printf(m, "\tblock mode:\t\t%s\n", msg); + + seq_printf(m, "\nStatistics:\n"); + seq_printf(m, "\tpackets started:\t%lu\n", pd->stats.pkt_started); + seq_printf(m, "\tpackets ended:\t\t%lu\n", pd->stats.pkt_ended); + seq_printf(m, "\twritten:\t\t%lukB\n", pd->stats.secs_w >> 1); + seq_printf(m, "\tread gather:\t\t%lukB\n", pd->stats.secs_rg >> 1); + seq_printf(m, "\tread:\t\t\t%lukB\n", pd->stats.secs_r >> 1); + + seq_printf(m, "\nMisc:\n"); + seq_printf(m, "\treference count:\t%d\n", pd->refcnt); + seq_printf(m, "\tflags:\t\t\t0x%lx\n", pd->flags); + seq_printf(m, "\tread speed:\t\t%ukB/s\n", pd->read_speed); + seq_printf(m, "\twrite speed:\t\t%ukB/s\n", pd->write_speed); + seq_printf(m, "\tstart offset:\t\t%lu\n", pd->offset); + seq_printf(m, "\tmode page offset:\t%u\n", pd->mode_offset); + + seq_printf(m, "\nQueue state:\n"); + seq_printf(m, "\tbios queued:\t\t%d\n", pd->bio_queue_size); + seq_printf(m, "\tbios pending:\t\t%d\n", atomic_read(&pd->cdrw.pending_bios)); + seq_printf(m, "\tcurrent sector:\t\t0x%llx\n", (unsigned long long)pd->current_sector); + + pkt_count_states(pd, states); + seq_printf(m, "\tstate:\t\t\ti:%d ow:%d rw:%d ww:%d rec:%d fin:%d\n", + states[0], states[1], states[2], states[3], states[4], states[5]); + + seq_printf(m, "\twrite congestion marks:\toff=%d on=%d\n", + pd->write_congestion_off, + pd->write_congestion_on); + return 0; +} + +static int pkt_new_dev(struct pktcdvd_device *pd, dev_t dev) +{ + int i; + struct block_device *bdev; + struct scsi_device *sdev; + + if (pd->pkt_dev == dev) { + pkt_err(pd, "recursive setup not allowed\n"); + return -EBUSY; + } + for (i = 0; i < MAX_WRITERS; i++) { + struct pktcdvd_device *pd2 = pkt_devs[i]; + if (!pd2) + continue; + if (pd2->bdev->bd_dev == dev) { + pkt_err(pd, "%pg already setup\n", pd2->bdev); + return -EBUSY; + } + if (pd2->pkt_dev == dev) { + pkt_err(pd, "can't chain pktcdvd devices\n"); + return -EBUSY; + } + } + + bdev = blkdev_get_by_dev(dev, FMODE_READ | FMODE_NDELAY, NULL); + if (IS_ERR(bdev)) + return PTR_ERR(bdev); + sdev = scsi_device_from_queue(bdev->bd_disk->queue); + if (!sdev) { + blkdev_put(bdev, FMODE_READ | FMODE_NDELAY); + return -EINVAL; + } + put_device(&sdev->sdev_gendev); + + /* This is safe, since we have a reference from open(). */ + __module_get(THIS_MODULE); + + pd->bdev = bdev; + set_blocksize(bdev, CD_FRAMESIZE); + + pkt_init_queue(pd); + + atomic_set(&pd->cdrw.pending_bios, 0); + pd->cdrw.thread = kthread_run(kcdrwd, pd, "%s", pd->name); + if (IS_ERR(pd->cdrw.thread)) { + pkt_err(pd, "can't start kernel thread\n"); + goto out_mem; + } + + proc_create_single_data(pd->name, 0, pkt_proc, pkt_seq_show, pd); + pkt_dbg(1, pd, "writer mapped to %pg\n", bdev); + return 0; + +out_mem: + blkdev_put(bdev, FMODE_READ | FMODE_NDELAY); + /* This is safe: open() is still holding a reference. */ + module_put(THIS_MODULE); + return -ENOMEM; +} + +static int pkt_ioctl(struct block_device *bdev, fmode_t mode, unsigned int cmd, unsigned long arg) +{ + struct pktcdvd_device *pd = bdev->bd_disk->private_data; + int ret; + + pkt_dbg(2, pd, "cmd %x, dev %d:%d\n", + cmd, MAJOR(bdev->bd_dev), MINOR(bdev->bd_dev)); + + mutex_lock(&pktcdvd_mutex); + switch (cmd) { + case CDROMEJECT: + /* + * The door gets locked when the device is opened, so we + * have to unlock it or else the eject command fails. + */ + if (pd->refcnt == 1) + pkt_lock_door(pd, 0); + fallthrough; + /* + * forward selected CDROM ioctls to CD-ROM, for UDF + */ + case CDROMMULTISESSION: + case CDROMREADTOCENTRY: + case CDROM_LAST_WRITTEN: + case CDROM_SEND_PACKET: + case SCSI_IOCTL_SEND_COMMAND: + if (!bdev->bd_disk->fops->ioctl) + ret = -ENOTTY; + else + ret = bdev->bd_disk->fops->ioctl(bdev, mode, cmd, arg); + break; + default: + pkt_dbg(2, pd, "Unknown ioctl (%x)\n", cmd); + ret = -ENOTTY; + } + mutex_unlock(&pktcdvd_mutex); + + return ret; +} + +static unsigned int pkt_check_events(struct gendisk *disk, + unsigned int clearing) +{ + struct pktcdvd_device *pd = disk->private_data; + struct gendisk *attached_disk; + + if (!pd) + return 0; + if (!pd->bdev) + return 0; + attached_disk = pd->bdev->bd_disk; + if (!attached_disk || !attached_disk->fops->check_events) + return 0; + return attached_disk->fops->check_events(attached_disk, clearing); +} + +static char *pkt_devnode(struct gendisk *disk, umode_t *mode) +{ + return kasprintf(GFP_KERNEL, "pktcdvd/%s", disk->disk_name); +} + +static const struct block_device_operations pktcdvd_ops = { + .owner = THIS_MODULE, + .submit_bio = pkt_submit_bio, + .open = pkt_open, + .release = pkt_close, + .ioctl = pkt_ioctl, + .compat_ioctl = blkdev_compat_ptr_ioctl, + .check_events = pkt_check_events, + .devnode = pkt_devnode, +}; + +/* + * Set up mapping from pktcdvd device to CD-ROM device. + */ +static int pkt_setup_dev(dev_t dev, dev_t* pkt_dev) +{ + int idx; + int ret = -ENOMEM; + struct pktcdvd_device *pd; + struct gendisk *disk; + + mutex_lock_nested(&ctl_mutex, SINGLE_DEPTH_NESTING); + + for (idx = 0; idx < MAX_WRITERS; idx++) + if (!pkt_devs[idx]) + break; + if (idx == MAX_WRITERS) { + pr_err("max %d writers supported\n", MAX_WRITERS); + ret = -EBUSY; + goto out_mutex; + } + + pd = kzalloc(sizeof(struct pktcdvd_device), GFP_KERNEL); + if (!pd) + goto out_mutex; + + ret = mempool_init_kmalloc_pool(&pd->rb_pool, PKT_RB_POOL_SIZE, + sizeof(struct pkt_rb_node)); + if (ret) + goto out_mem; + + INIT_LIST_HEAD(&pd->cdrw.pkt_free_list); + INIT_LIST_HEAD(&pd->cdrw.pkt_active_list); + spin_lock_init(&pd->cdrw.active_list_lock); + + spin_lock_init(&pd->lock); + spin_lock_init(&pd->iosched.lock); + bio_list_init(&pd->iosched.read_queue); + bio_list_init(&pd->iosched.write_queue); + sprintf(pd->name, DRIVER_NAME"%d", idx); + init_waitqueue_head(&pd->wqueue); + pd->bio_queue = RB_ROOT; + + pd->write_congestion_on = write_congestion_on; + pd->write_congestion_off = write_congestion_off; + + ret = -ENOMEM; + disk = blk_alloc_disk(NUMA_NO_NODE); + if (!disk) + goto out_mem; + pd->disk = disk; + disk->major = pktdev_major; + disk->first_minor = idx; + disk->minors = 1; + disk->fops = &pktcdvd_ops; + disk->flags = GENHD_FL_REMOVABLE | GENHD_FL_NO_PART; + strcpy(disk->disk_name, pd->name); + disk->private_data = pd; + + pd->pkt_dev = MKDEV(pktdev_major, idx); + ret = pkt_new_dev(pd, dev); + if (ret) + goto out_mem2; + + /* inherit events of the host device */ + disk->events = pd->bdev->bd_disk->events; + + ret = add_disk(disk); + if (ret) + goto out_mem2; + + pkt_sysfs_dev_new(pd); + pkt_debugfs_dev_new(pd); + + pkt_devs[idx] = pd; + if (pkt_dev) + *pkt_dev = pd->pkt_dev; + + mutex_unlock(&ctl_mutex); + return 0; + +out_mem2: + put_disk(disk); +out_mem: + mempool_exit(&pd->rb_pool); + kfree(pd); +out_mutex: + mutex_unlock(&ctl_mutex); + pr_err("setup of pktcdvd device failed\n"); + return ret; +} + +/* + * Tear down mapping from pktcdvd device to CD-ROM device. + */ +static int pkt_remove_dev(dev_t pkt_dev) +{ + struct pktcdvd_device *pd; + int idx; + int ret = 0; + + mutex_lock_nested(&ctl_mutex, SINGLE_DEPTH_NESTING); + + for (idx = 0; idx < MAX_WRITERS; idx++) { + pd = pkt_devs[idx]; + if (pd && (pd->pkt_dev == pkt_dev)) + break; + } + if (idx == MAX_WRITERS) { + pr_debug("dev not setup\n"); + ret = -ENXIO; + goto out; + } + + if (pd->refcnt > 0) { + ret = -EBUSY; + goto out; + } + if (!IS_ERR(pd->cdrw.thread)) + kthread_stop(pd->cdrw.thread); + + pkt_devs[idx] = NULL; + + pkt_debugfs_dev_remove(pd); + pkt_sysfs_dev_remove(pd); + + blkdev_put(pd->bdev, FMODE_READ | FMODE_NDELAY); + + remove_proc_entry(pd->name, pkt_proc); + pkt_dbg(1, pd, "writer unmapped\n"); + + del_gendisk(pd->disk); + put_disk(pd->disk); + + mempool_exit(&pd->rb_pool); + kfree(pd); + + /* This is safe: open() is still holding a reference. */ + module_put(THIS_MODULE); + +out: + mutex_unlock(&ctl_mutex); + return ret; +} + +static void pkt_get_status(struct pkt_ctrl_command *ctrl_cmd) +{ + struct pktcdvd_device *pd; + + mutex_lock_nested(&ctl_mutex, SINGLE_DEPTH_NESTING); + + pd = pkt_find_dev_from_minor(ctrl_cmd->dev_index); + if (pd) { + ctrl_cmd->dev = new_encode_dev(pd->bdev->bd_dev); + ctrl_cmd->pkt_dev = new_encode_dev(pd->pkt_dev); + } else { + ctrl_cmd->dev = 0; + ctrl_cmd->pkt_dev = 0; + } + ctrl_cmd->num_devices = MAX_WRITERS; + + mutex_unlock(&ctl_mutex); +} + +static long pkt_ctl_ioctl(struct file *file, unsigned int cmd, unsigned long arg) +{ + void __user *argp = (void __user *)arg; + struct pkt_ctrl_command ctrl_cmd; + int ret = 0; + dev_t pkt_dev = 0; + + if (cmd != PACKET_CTRL_CMD) + return -ENOTTY; + + if (copy_from_user(&ctrl_cmd, argp, sizeof(struct pkt_ctrl_command))) + return -EFAULT; + + switch (ctrl_cmd.command) { + case PKT_CTRL_CMD_SETUP: + if (!capable(CAP_SYS_ADMIN)) + return -EPERM; + ret = pkt_setup_dev(new_decode_dev(ctrl_cmd.dev), &pkt_dev); + ctrl_cmd.pkt_dev = new_encode_dev(pkt_dev); + break; + case PKT_CTRL_CMD_TEARDOWN: + if (!capable(CAP_SYS_ADMIN)) + return -EPERM; + ret = pkt_remove_dev(new_decode_dev(ctrl_cmd.pkt_dev)); + break; + case PKT_CTRL_CMD_STATUS: + pkt_get_status(&ctrl_cmd); + break; + default: + return -ENOTTY; + } + + if (copy_to_user(argp, &ctrl_cmd, sizeof(struct pkt_ctrl_command))) + return -EFAULT; + return ret; +} + +#ifdef CONFIG_COMPAT +static long pkt_ctl_compat_ioctl(struct file *file, unsigned int cmd, unsigned long arg) +{ + return pkt_ctl_ioctl(file, cmd, (unsigned long)compat_ptr(arg)); +} +#endif + +static const struct file_operations pkt_ctl_fops = { + .open = nonseekable_open, + .unlocked_ioctl = pkt_ctl_ioctl, +#ifdef CONFIG_COMPAT + .compat_ioctl = pkt_ctl_compat_ioctl, +#endif + .owner = THIS_MODULE, + .llseek = no_llseek, +}; + +static struct miscdevice pkt_misc = { + .minor = MISC_DYNAMIC_MINOR, + .name = DRIVER_NAME, + .nodename = "pktcdvd/control", + .fops = &pkt_ctl_fops +}; + +static int __init pkt_init(void) +{ + int ret; + + mutex_init(&ctl_mutex); + + ret = mempool_init_kmalloc_pool(&psd_pool, PSD_POOL_SIZE, + sizeof(struct packet_stacked_data)); + if (ret) + return ret; + ret = bioset_init(&pkt_bio_set, BIO_POOL_SIZE, 0, 0); + if (ret) { + mempool_exit(&psd_pool); + return ret; + } + + ret = register_blkdev(pktdev_major, DRIVER_NAME); + if (ret < 0) { + pr_err("unable to register block device\n"); + goto out2; + } + if (!pktdev_major) + pktdev_major = ret; + + ret = pkt_sysfs_init(); + if (ret) + goto out; + + pkt_debugfs_init(); + + ret = misc_register(&pkt_misc); + if (ret) { + pr_err("unable to register misc device\n"); + goto out_misc; + } + + pkt_proc = proc_mkdir("driver/"DRIVER_NAME, NULL); + + return 0; + +out_misc: + pkt_debugfs_cleanup(); + pkt_sysfs_cleanup(); +out: + unregister_blkdev(pktdev_major, DRIVER_NAME); +out2: + mempool_exit(&psd_pool); + bioset_exit(&pkt_bio_set); + return ret; +} + +static void __exit pkt_exit(void) +{ + remove_proc_entry("driver/"DRIVER_NAME, NULL); + misc_deregister(&pkt_misc); + + pkt_debugfs_cleanup(); + pkt_sysfs_cleanup(); + + unregister_blkdev(pktdev_major, DRIVER_NAME); + mempool_exit(&psd_pool); + bioset_exit(&pkt_bio_set); +} + +MODULE_DESCRIPTION("Packet writing layer for CD/DVD drives"); +MODULE_AUTHOR("Jens Axboe "); +MODULE_LICENSE("GPL"); + +module_init(pkt_init); +module_exit(pkt_exit); diff --git a/drivers/block/ps3vram.c b/drivers/block/ps3vram.c index c76e0148eada3302a04b81a886fa7cdb34df2de1..574e470b220b09bf3e032f14a60998cfe5d833ce 100644 --- a/drivers/block/ps3vram.c +++ b/drivers/block/ps3vram.c @@ -587,6 +587,8 @@ static void ps3vram_submit_bio(struct bio *bio) dev_dbg(&dev->core, "%s\n", __func__); bio = bio_split_to_limits(bio); + if (!bio) + return; spin_lock_irq(&priv->lock); busy = !bio_list_empty(&priv->list); diff --git a/drivers/block/rnbd/rnbd-clt.c b/drivers/block/rnbd/rnbd-clt.c index 78334da74d8bf90dcfd1f51d5ccd26f71a289a0c..5eb8c7855970d1ab277798056b0fb3db2ba151be 100644 --- a/drivers/block/rnbd/rnbd-clt.c +++ b/drivers/block/rnbd/rnbd-clt.c @@ -1440,7 +1440,7 @@ static struct rnbd_clt_dev *init_dev(struct rnbd_clt_session *sess, goto out_alloc; } - ret = ida_alloc_max(&index_ida, 1 << (MINORBITS - RNBD_PART_BITS), + ret = ida_alloc_max(&index_ida, (1 << (MINORBITS - RNBD_PART_BITS)) - 1, GFP_KERNEL); if (ret < 0) { pr_err("Failed to initialize device '%s' from session %s, allocating idr failed, err: %d\n", diff --git a/drivers/block/ublk_drv.c b/drivers/block/ublk_drv.c index e9de9d846b730359623d5b451015c169937f9685..e546932046309a66f63e2fcaf771e9d41c20ffc3 100644 --- a/drivers/block/ublk_drv.c +++ b/drivers/block/ublk_drv.c @@ -1992,6 +1992,9 @@ static int ublk_ctrl_uring_cmd(struct io_uring_cmd *cmd, struct ublksrv_ctrl_cmd *header = (struct ublksrv_ctrl_cmd *)cmd->cmd; int ret = -EINVAL; + if (issue_flags & IO_URING_F_NONBLOCK) + return -EAGAIN; + ublk_ctrl_cmd_dump(cmd); if (!(issue_flags & IO_URING_F_SQE128)) @@ -2089,13 +2092,12 @@ static void __exit ublk_exit(void) struct ublk_device *ub; int id; - class_destroy(ublk_chr_class); - - misc_deregister(&ublk_misc); - idr_for_each_entry(&ublk_index_idr, ub, id) ublk_remove(ub); + class_destroy(ublk_chr_class); + misc_deregister(&ublk_misc); + idr_destroy(&ublk_index_idr); unregister_chrdev_region(ublk_chr_devt, UBLK_MINORS); } diff --git a/drivers/block/virtio_blk.c b/drivers/block/virtio_blk.c index 68bd2f7961b3f5ad7d64843f2fbc659984cd3c92..6a77fa9174288089f803e3a698c0382e2d128381 100644 --- a/drivers/block/virtio_blk.c +++ b/drivers/block/virtio_blk.c @@ -315,22 +315,35 @@ static void virtio_commit_rqs(struct blk_mq_hw_ctx *hctx) virtqueue_notify(vq->vq); } +static blk_status_t virtblk_fail_to_queue(struct request *req, int rc) +{ + virtblk_cleanup_cmd(req); + switch (rc) { + case -ENOSPC: + return BLK_STS_DEV_RESOURCE; + case -ENOMEM: + return BLK_STS_RESOURCE; + default: + return BLK_STS_IOERR; + } +} + static blk_status_t virtblk_prep_rq(struct blk_mq_hw_ctx *hctx, struct virtio_blk *vblk, struct request *req, struct virtblk_req *vbr) { blk_status_t status; + int num; status = virtblk_setup_cmd(vblk->vdev, req, vbr); if (unlikely(status)) return status; - vbr->sg_table.nents = virtblk_map_data(hctx, req, vbr); - if (unlikely(vbr->sg_table.nents < 0)) { - virtblk_cleanup_cmd(req); - return BLK_STS_RESOURCE; - } + num = virtblk_map_data(hctx, req, vbr); + if (unlikely(num < 0)) + return virtblk_fail_to_queue(req, -ENOMEM); + vbr->sg_table.nents = num; blk_mq_start_request(req); @@ -364,15 +377,7 @@ static blk_status_t virtio_queue_rq(struct blk_mq_hw_ctx *hctx, blk_mq_stop_hw_queue(hctx); spin_unlock_irqrestore(&vblk->vqs[qid].lock, flags); virtblk_unmap_data(req, vbr); - virtblk_cleanup_cmd(req); - switch (err) { - case -ENOSPC: - return BLK_STS_DEV_RESOURCE; - case -ENOMEM: - return BLK_STS_RESOURCE; - default: - return BLK_STS_IOERR; - } + return virtblk_fail_to_queue(req, err); } if (bd->last && virtqueue_kick_prepare(vblk->vqs[qid].vq)) @@ -991,7 +996,7 @@ static int virtblk_probe(struct virtio_device *vdev) blk_queue_max_segments(q, sg_elems); /* No real sector limit. */ - blk_queue_max_hw_sectors(q, -1U); + blk_queue_max_hw_sectors(q, UINT_MAX); max_size = virtio_max_dma_size(vdev); diff --git a/drivers/block/xen-blkback/xenbus.c b/drivers/block/xen-blkback/xenbus.c index c0227dfa468879cb3af300caba282b620f3ac824..4807af1d58059394d7a992335dabaf2bc3901721 100644 --- a/drivers/block/xen-blkback/xenbus.c +++ b/drivers/block/xen-blkback/xenbus.c @@ -524,7 +524,7 @@ static int xen_vbd_create(struct xen_blkif *blkif, blkif_vdev_t handle, return 0; } -static int xen_blkbk_remove(struct xenbus_device *dev) +static void xen_blkbk_remove(struct xenbus_device *dev) { struct backend_info *be = dev_get_drvdata(&dev->dev); @@ -547,8 +547,6 @@ static int xen_blkbk_remove(struct xenbus_device *dev) /* Put the reference we set in xen_blkif_alloc(). */ xen_blkif_put(be->blkif); } - - return 0; } int xen_blkbk_flush_diskcache(struct xenbus_transaction xbt, diff --git a/drivers/block/xen-blkfront.c b/drivers/block/xen-blkfront.c index b28489290323faeeb74302570d4a971493c92c14..23ed258b57f0e585e122a60d6d4dba7cc2972f02 100644 --- a/drivers/block/xen-blkfront.c +++ b/drivers/block/xen-blkfront.c @@ -2467,7 +2467,7 @@ static void blkback_changed(struct xenbus_device *dev, } } -static int blkfront_remove(struct xenbus_device *xbdev) +static void blkfront_remove(struct xenbus_device *xbdev) { struct blkfront_info *info = dev_get_drvdata(&xbdev->dev); @@ -2488,7 +2488,6 @@ static int blkfront_remove(struct xenbus_device *xbdev) } kfree(info); - return 0; } static int blkfront_is_ready(struct xenbus_device *dev) diff --git a/drivers/bluetooth/hci_qca.c b/drivers/bluetooth/hci_qca.c index 6eddc23e49d9a7dc30eb24d6614e0402baa6bd91..bbe9cf1cae27f6088cea5cdd5cd71983fdded295 100644 --- a/drivers/bluetooth/hci_qca.c +++ b/drivers/bluetooth/hci_qca.c @@ -2164,10 +2164,17 @@ static void qca_serdev_shutdown(struct device *dev) int timeout = msecs_to_jiffies(CMD_TRANS_TIMEOUT_MS); struct serdev_device *serdev = to_serdev_device(dev); struct qca_serdev *qcadev = serdev_device_get_drvdata(serdev); + struct hci_uart *hu = &qcadev->serdev_hu; + struct hci_dev *hdev = hu->hdev; + struct qca_data *qca = hu->priv; const u8 ibs_wake_cmd[] = { 0xFD }; const u8 edl_reset_soc_cmd[] = { 0x01, 0x00, 0xFC, 0x01, 0x05 }; if (qcadev->btsoc_type == QCA_QCA6390) { + if (test_bit(QCA_BT_OFF, &qca->flags) || + !test_bit(HCI_RUNNING, &hdev->flags)) + return; + serdev_device_write_flush(serdev); ret = serdev_device_write_buf(serdev, ibs_wake_cmd, sizeof(ibs_wake_cmd)); diff --git a/drivers/char/tpm/tpm-interface.c b/drivers/char/tpm/tpm-interface.c index d69905233aff2da31bfd55d6f53a62e5a5cb90b4..7e513b7718320c0392385a7d322e97b05e868f11 100644 --- a/drivers/char/tpm/tpm-interface.c +++ b/drivers/char/tpm/tpm-interface.c @@ -412,7 +412,9 @@ int tpm_pm_suspend(struct device *dev) } suspended: - return rc; + if (rc) + dev_err(dev, "Ignoring error %d while suspending\n", rc); + return 0; } EXPORT_SYMBOL_GPL(tpm_pm_suspend); diff --git a/drivers/char/tpm/xen-tpmfront.c b/drivers/char/tpm/xen-tpmfront.c index 3792918262617983cd21d424a736955c395400b8..80cca3b83b226aa10918fc6e7a7468a43bd91996 100644 --- a/drivers/char/tpm/xen-tpmfront.c +++ b/drivers/char/tpm/xen-tpmfront.c @@ -360,14 +360,13 @@ static int tpmfront_probe(struct xenbus_device *dev, return tpm_chip_register(priv->chip); } -static int tpmfront_remove(struct xenbus_device *dev) +static void tpmfront_remove(struct xenbus_device *dev) { struct tpm_chip *chip = dev_get_drvdata(&dev->dev); struct tpm_private *priv = dev_get_drvdata(&chip->dev); tpm_chip_unregister(chip); ring_free(priv); dev_set_drvdata(&chip->dev, NULL); - return 0; } static int tpmfront_resume(struct xenbus_device *dev) diff --git a/drivers/comedi/drivers/adv_pci1760.c b/drivers/comedi/drivers/adv_pci1760.c index fcfc2e299110eddbb009969af8829bb859a49690..27f3890f471df7a78e7b8e27d0755a295ab8caad 100644 --- a/drivers/comedi/drivers/adv_pci1760.c +++ b/drivers/comedi/drivers/adv_pci1760.c @@ -58,7 +58,7 @@ #define PCI1760_CMD_CLR_IMB2 0x00 /* Clears IMB2 */ #define PCI1760_CMD_SET_DO 0x01 /* Set output state */ #define PCI1760_CMD_GET_DO 0x02 /* Read output status */ -#define PCI1760_CMD_GET_STATUS 0x03 /* Read current status */ +#define PCI1760_CMD_GET_STATUS 0x07 /* Read current status */ #define PCI1760_CMD_GET_FW_VER 0x0e /* Read firmware version */ #define PCI1760_CMD_GET_HW_VER 0x0f /* Read hardware version */ #define PCI1760_CMD_SET_PWM_HI(x) (0x10 + (x) * 2) /* Set "hi" period */ diff --git a/drivers/cpufreq/amd-pstate.c b/drivers/cpufreq/amd-pstate.c index 204e39006dda8f102643ecd90eb45fbb3df70218..c17bd845f5fcb8afa9c64662f6a13903015ac815 100644 --- a/drivers/cpufreq/amd-pstate.c +++ b/drivers/cpufreq/amd-pstate.c @@ -307,6 +307,7 @@ static void amd_pstate_adjust_perf(unsigned int cpu, max_perf = min_perf; amd_pstate_update(cpudata, min_perf, des_perf, max_perf, true); + cpufreq_cpu_put(policy); } static int amd_get_min_freq(struct amd_cpudata *cpudata) diff --git a/drivers/cpufreq/apple-soc-cpufreq.c b/drivers/cpufreq/apple-soc-cpufreq.c index d1801281cdd992106006250a4a395e828e36fcc5..c11d22fd84c3738584144377d2934b83d27c8f6a 100644 --- a/drivers/cpufreq/apple-soc-cpufreq.c +++ b/drivers/cpufreq/apple-soc-cpufreq.c @@ -280,6 +280,7 @@ static int apple_soc_cpufreq_init(struct cpufreq_policy *policy) policy->cpuinfo.transition_latency = transition_latency; policy->dvfs_possible_from_any_cpu = true; policy->fast_switch_possible = true; + policy->suspend_freq = freq_table[0].frequency; if (policy_has_boost_freq(policy)) { ret = cpufreq_enable_boost_support(); @@ -321,7 +322,6 @@ static struct cpufreq_driver apple_soc_cpufreq_driver = { .flags = CPUFREQ_HAVE_GOVERNOR_PER_POLICY | CPUFREQ_NEED_INITIAL_FREQ_CHECK | CPUFREQ_IS_COOLING_DEV, .verify = cpufreq_generic_frequency_table_verify, - .attr = cpufreq_generic_attr, .get = apple_soc_cpufreq_get_rate, .init = apple_soc_cpufreq_init, .exit = apple_soc_cpufreq_exit, @@ -329,6 +329,7 @@ static struct cpufreq_driver apple_soc_cpufreq_driver = { .fast_switch = apple_soc_cpufreq_fast_switch, .register_em = cpufreq_register_em_with_opp, .attr = apple_soc_cpufreq_hw_attr, + .suspend = cpufreq_generic_suspend, }; static int __init apple_soc_cpufreq_module_init(void) diff --git a/drivers/cpufreq/armada-37xx-cpufreq.c b/drivers/cpufreq/armada-37xx-cpufreq.c index c10fc33b29b181aa725c202ca734b95227836533..b74289a95a17178b7e2177e1bf3ad0b12bb307b0 100644 --- a/drivers/cpufreq/armada-37xx-cpufreq.c +++ b/drivers/cpufreq/armada-37xx-cpufreq.c @@ -445,7 +445,7 @@ static int __init armada37xx_cpufreq_driver_init(void) return -ENODEV; } - clk = clk_get(cpu_dev, 0); + clk = clk_get(cpu_dev, NULL); if (IS_ERR(clk)) { dev_err(cpu_dev, "Cannot get clock for CPU0\n"); return PTR_ERR(clk); diff --git a/drivers/cpufreq/cppc_cpufreq.c b/drivers/cpufreq/cppc_cpufreq.c index 432dfb4e8027e96b908c420cd40f0d0eb911bfef..022e3555407c867bd4d0d2508571fedd0f22a0f7 100644 --- a/drivers/cpufreq/cppc_cpufreq.c +++ b/drivers/cpufreq/cppc_cpufreq.c @@ -487,7 +487,8 @@ static unsigned int get_perf_level_count(struct cpufreq_policy *policy) cpu_data = policy->driver_data; perf_caps = &cpu_data->perf_caps; max_cap = arch_scale_cpu_capacity(cpu); - min_cap = div_u64(max_cap * perf_caps->lowest_perf, perf_caps->highest_perf); + min_cap = div_u64((u64)max_cap * perf_caps->lowest_perf, + perf_caps->highest_perf); if ((min_cap == 0) || (max_cap < min_cap)) return 0; return 1 + max_cap / CPPC_EM_CAP_STEP - min_cap / CPPC_EM_CAP_STEP; @@ -519,10 +520,10 @@ static int cppc_get_cpu_power(struct device *cpu_dev, cpu_data = policy->driver_data; perf_caps = &cpu_data->perf_caps; max_cap = arch_scale_cpu_capacity(cpu_dev->id); - min_cap = div_u64(max_cap * perf_caps->lowest_perf, - perf_caps->highest_perf); - - perf_step = CPPC_EM_CAP_STEP * perf_caps->highest_perf / max_cap; + min_cap = div_u64((u64)max_cap * perf_caps->lowest_perf, + perf_caps->highest_perf); + perf_step = div_u64((u64)CPPC_EM_CAP_STEP * perf_caps->highest_perf, + max_cap); min_step = min_cap / CPPC_EM_CAP_STEP; max_step = max_cap / CPPC_EM_CAP_STEP; diff --git a/drivers/cpufreq/cpufreq-dt-platdev.c b/drivers/cpufreq/cpufreq-dt-platdev.c index 8ab67288304369d5c3e644844a412bdd7fd8b629..e8570365109869d4063339d877b3c3110b75b9b3 100644 --- a/drivers/cpufreq/cpufreq-dt-platdev.c +++ b/drivers/cpufreq/cpufreq-dt-platdev.c @@ -137,6 +137,7 @@ static const struct of_device_id blocklist[] __initconst = { { .compatible = "nvidia,tegra30", }, { .compatible = "nvidia,tegra124", }, { .compatible = "nvidia,tegra210", }, + { .compatible = "nvidia,tegra234", }, { .compatible = "qcom,apq8096", }, { .compatible = "qcom,msm8996", }, @@ -150,6 +151,7 @@ static const struct of_device_id blocklist[] __initconst = { { .compatible = "qcom,sdm845", }, { .compatible = "qcom,sm6115", }, { .compatible = "qcom,sm6350", }, + { .compatible = "qcom,sm6375", }, { .compatible = "qcom,sm8150", }, { .compatible = "qcom,sm8250", }, { .compatible = "qcom,sm8350", }, diff --git a/drivers/cpufreq/qcom-cpufreq-hw.c b/drivers/cpufreq/qcom-cpufreq-hw.c index 340fed35e45dde64081837df046ae729459c9745..9505a812d6a1d230f1fdf23b736a3f910268990a 100644 --- a/drivers/cpufreq/qcom-cpufreq-hw.c +++ b/drivers/cpufreq/qcom-cpufreq-hw.c @@ -649,9 +649,10 @@ static int qcom_cpufreq_hw_driver_probe(struct platform_device *pdev) { struct clk_hw_onecell_data *clk_data; struct device *dev = &pdev->dev; + struct device_node *soc_node; struct device *cpu_dev; struct clk *clk; - int ret, i, num_domains; + int ret, i, num_domains, reg_sz; clk = clk_get(dev, "xo"); if (IS_ERR(clk)) @@ -679,7 +680,21 @@ static int qcom_cpufreq_hw_driver_probe(struct platform_device *pdev) return ret; /* Allocate qcom_cpufreq_data based on the available frequency domains in DT */ - num_domains = of_property_count_elems_of_size(dev->of_node, "reg", sizeof(u32) * 4); + soc_node = of_get_parent(dev->of_node); + if (!soc_node) + return -EINVAL; + + ret = of_property_read_u32(soc_node, "#address-cells", ®_sz); + if (ret) + goto of_exit; + + ret = of_property_read_u32(soc_node, "#size-cells", &i); + if (ret) + goto of_exit; + + reg_sz += i; + + num_domains = of_property_count_elems_of_size(dev->of_node, "reg", sizeof(u32) * reg_sz); if (num_domains <= 0) return num_domains; @@ -743,6 +758,9 @@ static int qcom_cpufreq_hw_driver_probe(struct platform_device *pdev) else dev_dbg(dev, "QCOM CPUFreq HW driver initialized\n"); +of_exit: + of_node_put(soc_node); + return ret; } diff --git a/drivers/crypto/atmel-ecc.c b/drivers/crypto/atmel-ecc.c index 53100fb9b07bdb9fe360916f93d76c46c46cb4cc..12205e2b53b4574851ced620eeda4f4c174b231b 100644 --- a/drivers/crypto/atmel-ecc.c +++ b/drivers/crypto/atmel-ecc.c @@ -3,7 +3,7 @@ * Microchip / Atmel ECC (I2C) driver. * * Copyright (c) 2017, Microchip Technology Inc. - * Author: Tudor Ambarus + * Author: Tudor Ambarus */ #include @@ -411,6 +411,6 @@ static void __exit atmel_ecc_exit(void) module_init(atmel_ecc_init); module_exit(atmel_ecc_exit); -MODULE_AUTHOR("Tudor Ambarus "); +MODULE_AUTHOR("Tudor Ambarus"); MODULE_DESCRIPTION("Microchip / Atmel ECC (I2C) driver"); MODULE_LICENSE("GPL v2"); diff --git a/drivers/crypto/atmel-i2c.c b/drivers/crypto/atmel-i2c.c index 81ce09bedda8f4348ba25ca8a676a92f83d2c96b..55bff1e13142606e650635f1b5345e75f16b061f 100644 --- a/drivers/crypto/atmel-i2c.c +++ b/drivers/crypto/atmel-i2c.c @@ -3,7 +3,7 @@ * Microchip / Atmel ECC (I2C) driver. * * Copyright (c) 2017, Microchip Technology Inc. - * Author: Tudor Ambarus + * Author: Tudor Ambarus */ #include @@ -390,6 +390,6 @@ static void __exit atmel_i2c_exit(void) module_init(atmel_i2c_init); module_exit(atmel_i2c_exit); -MODULE_AUTHOR("Tudor Ambarus "); +MODULE_AUTHOR("Tudor Ambarus"); MODULE_DESCRIPTION("Microchip / Atmel ECC (I2C) driver"); MODULE_LICENSE("GPL v2"); diff --git a/drivers/crypto/atmel-i2c.h b/drivers/crypto/atmel-i2c.h index 48929efe2a5bf20112c1e0cc2ec178b7fb1ad107..35f7857a7f7cc1c3fff0866a83dbfa5663df6086 100644 --- a/drivers/crypto/atmel-i2c.h +++ b/drivers/crypto/atmel-i2c.h @@ -1,7 +1,7 @@ /* SPDX-License-Identifier: GPL-2.0 */ /* * Copyright (c) 2017, Microchip Technology Inc. - * Author: Tudor Ambarus + * Author: Tudor Ambarus */ #ifndef __ATMEL_I2C_H__ diff --git a/drivers/crypto/caam/blob_gen.c b/drivers/crypto/caam/blob_gen.c index 1f65df48984783a992d8e6d77dcdd7e6d803ff0d..f46b161d2cda65c03446505f32234bc20ab8057f 100644 --- a/drivers/crypto/caam/blob_gen.c +++ b/drivers/crypto/caam/blob_gen.c @@ -104,7 +104,7 @@ int caam_process_blob(struct caam_blob_priv *priv, } ctrlpriv = dev_get_drvdata(jrdev->parent); - moo = FIELD_GET(CSTA_MOO, ioread32(&ctrlpriv->ctrl->perfmon.status)); + moo = FIELD_GET(CSTA_MOO, rd_reg32(&ctrlpriv->ctrl->perfmon.status)); if (moo != CSTA_MOO_SECURE && moo != CSTA_MOO_TRUSTED) dev_warn(jrdev, "using insecure test key, enable HAB to use unique device key!\n"); diff --git a/drivers/crypto/virtio/virtio_crypto_skcipher_algs.c b/drivers/crypto/virtio/virtio_crypto_skcipher_algs.c index e553ccadbcbc859a38ce112ca6ef8df2407f10a6..e5876286828b8e53e0dba6fa705090b299dffc66 100644 --- a/drivers/crypto/virtio/virtio_crypto_skcipher_algs.c +++ b/drivers/crypto/virtio/virtio_crypto_skcipher_algs.c @@ -239,7 +239,8 @@ static int virtio_crypto_alg_skcipher_close_session( pr_err("virtio_crypto: Close session failed status: %u, session_id: 0x%llx\n", ctrl_status->status, destroy_session->session_id); - return -EINVAL; + err = -EINVAL; + goto out; } err = 0; diff --git a/drivers/cxl/acpi.c b/drivers/cxl/acpi.c index ad0849af42d790a528f9c4b2dec4b91c97e400ed..13cde44c608602268780f2e0af5763a4bc62a682 100644 --- a/drivers/cxl/acpi.c +++ b/drivers/cxl/acpi.c @@ -736,4 +736,3 @@ module_exit(cxl_acpi_exit); MODULE_LICENSE("GPL v2"); MODULE_IMPORT_NS(CXL); MODULE_IMPORT_NS(ACPI); -MODULE_SOFTDEP("pre: cxl_pmem"); diff --git a/drivers/cxl/core/pmem.c b/drivers/cxl/core/pmem.c index f3d2169b673118acb729b010809a110929ee4888..c2e4b10937884a7ec3cc1266f395ce2dedcbe209 100644 --- a/drivers/cxl/core/pmem.c +++ b/drivers/cxl/core/pmem.c @@ -227,34 +227,16 @@ static struct cxl_nvdimm *cxl_nvdimm_alloc(struct cxl_nvdimm_bridge *cxl_nvb, return cxl_nvd; } -static void cxl_nvd_unregister(void *_cxl_nvd) +static void cxlmd_release_nvdimm(void *_cxlmd) { - struct cxl_nvdimm *cxl_nvd = _cxl_nvd; - struct cxl_memdev *cxlmd = cxl_nvd->cxlmd; + struct cxl_memdev *cxlmd = _cxlmd; + struct cxl_nvdimm *cxl_nvd = cxlmd->cxl_nvd; struct cxl_nvdimm_bridge *cxl_nvb = cxlmd->cxl_nvb; - /* - * Either the bridge is in ->remove() context under the device_lock(), - * or cxlmd_release_nvdimm() is cancelling the bridge's release action - * for @cxl_nvd and doing it itself (while manually holding the bridge - * lock). - */ - device_lock_assert(&cxl_nvb->dev); cxl_nvd->cxlmd = NULL; cxlmd->cxl_nvd = NULL; + cxlmd->cxl_nvb = NULL; device_unregister(&cxl_nvd->dev); -} - -static void cxlmd_release_nvdimm(void *_cxlmd) -{ - struct cxl_memdev *cxlmd = _cxlmd; - struct cxl_nvdimm_bridge *cxl_nvb = cxlmd->cxl_nvb; - - device_lock(&cxl_nvb->dev); - if (cxlmd->cxl_nvd) - devm_release_action(&cxl_nvb->dev, cxl_nvd_unregister, - cxlmd->cxl_nvd); - device_unlock(&cxl_nvb->dev); put_device(&cxl_nvb->dev); } @@ -293,22 +275,6 @@ int devm_cxl_add_nvdimm(struct cxl_memdev *cxlmd) dev_dbg(&cxlmd->dev, "register %s\n", dev_name(dev)); - /* - * The two actions below arrange for @cxl_nvd to be deleted when either - * the top-level PMEM bridge goes down, or the endpoint device goes - * through ->remove(). - */ - device_lock(&cxl_nvb->dev); - if (cxl_nvb->dev.driver) - rc = devm_add_action_or_reset(&cxl_nvb->dev, cxl_nvd_unregister, - cxl_nvd); - else - rc = -ENXIO; - device_unlock(&cxl_nvb->dev); - - if (rc) - goto err_alloc; - /* @cxlmd carries a reference on @cxl_nvb until cxlmd_release_nvdimm */ return devm_add_action_or_reset(&cxlmd->dev, cxlmd_release_nvdimm, cxlmd); diff --git a/drivers/cxl/pci.c b/drivers/cxl/pci.c index 33083a522fd1cc121b7b1a117364e19c88c04f53..258004f34281aea2384f712f7e9ee4f7313f1559 100644 --- a/drivers/cxl/pci.c +++ b/drivers/cxl/pci.c @@ -554,8 +554,11 @@ static bool cxl_report_and_clear(struct cxl_dev_state *cxlds) /* If multiple errors, log header points to first error from ctrl reg */ if (hweight32(status) > 1) { - addr = cxlds->regs.ras + CXL_RAS_CAP_CONTROL_OFFSET; - fe = BIT(FIELD_GET(CXL_RAS_CAP_CONTROL_FE_MASK, readl(addr))); + void __iomem *rcc_addr = + cxlds->regs.ras + CXL_RAS_CAP_CONTROL_OFFSET; + + fe = BIT(FIELD_GET(CXL_RAS_CAP_CONTROL_FE_MASK, + readl(rcc_addr))); } else { fe = status; } diff --git a/drivers/cxl/pmem.c b/drivers/cxl/pmem.c index eedefebc42839115869038ad64d31fc26e9abdec..08bbbac9a6d08a4388121e543b7e4e252cf458a9 100644 --- a/drivers/cxl/pmem.c +++ b/drivers/cxl/pmem.c @@ -225,11 +225,35 @@ static int cxl_pmem_ctl(struct nvdimm_bus_descriptor *nd_desc, return cxl_pmem_nvdimm_ctl(nvdimm, cmd, buf, buf_len); } +static int detach_nvdimm(struct device *dev, void *data) +{ + struct cxl_nvdimm *cxl_nvd; + bool release = false; + + if (!is_cxl_nvdimm(dev)) + return 0; + + device_lock(dev); + if (!dev->driver) + goto out; + + cxl_nvd = to_cxl_nvdimm(dev); + if (cxl_nvd->cxlmd && cxl_nvd->cxlmd->cxl_nvb == data) + release = true; +out: + device_unlock(dev); + if (release) + device_release_driver(dev); + return 0; +} + static void unregister_nvdimm_bus(void *_cxl_nvb) { struct cxl_nvdimm_bridge *cxl_nvb = _cxl_nvb; struct nvdimm_bus *nvdimm_bus = cxl_nvb->nvdimm_bus; + bus_for_each_dev(&cxl_bus_type, NULL, cxl_nvb, detach_nvdimm); + cxl_nvb->nvdimm_bus = NULL; nvdimm_bus_unregister(nvdimm_bus); } diff --git a/drivers/dma-buf/dma-buf-sysfs-stats.c b/drivers/dma-buf/dma-buf-sysfs-stats.c index f69d68122b9b1313ebc6f94cfb12d7c707cebe2e..fbf725fae7c1c39480d59c2ce5e610fcf5b1874c 100644 --- a/drivers/dma-buf/dma-buf-sysfs-stats.c +++ b/drivers/dma-buf/dma-buf-sysfs-stats.c @@ -168,14 +168,11 @@ void dma_buf_uninit_sysfs_statistics(void) kset_unregister(dma_buf_stats_kset); } -int dma_buf_stats_setup(struct dma_buf *dmabuf) +int dma_buf_stats_setup(struct dma_buf *dmabuf, struct file *file) { struct dma_buf_sysfs_entry *sysfs_entry; int ret; - if (!dmabuf || !dmabuf->file) - return -EINVAL; - if (!dmabuf->exp_name) { pr_err("exporter name must not be empty if stats needed\n"); return -EINVAL; @@ -192,7 +189,7 @@ int dma_buf_stats_setup(struct dma_buf *dmabuf) /* create the directory for buffer stats */ ret = kobject_init_and_add(&sysfs_entry->kobj, &dma_buf_ktype, NULL, - "%lu", file_inode(dmabuf->file)->i_ino); + "%lu", file_inode(file)->i_ino); if (ret) goto err_sysfs_dmabuf; diff --git a/drivers/dma-buf/dma-buf-sysfs-stats.h b/drivers/dma-buf/dma-buf-sysfs-stats.h index a49c6e2650cccddb5eddf639bca14df89a36050b..7a8a995b75bae10621a783f583d0f46a1feb792c 100644 --- a/drivers/dma-buf/dma-buf-sysfs-stats.h +++ b/drivers/dma-buf/dma-buf-sysfs-stats.h @@ -13,7 +13,7 @@ int dma_buf_init_sysfs_statistics(void); void dma_buf_uninit_sysfs_statistics(void); -int dma_buf_stats_setup(struct dma_buf *dmabuf); +int dma_buf_stats_setup(struct dma_buf *dmabuf, struct file *file); void dma_buf_stats_teardown(struct dma_buf *dmabuf); #else @@ -25,7 +25,7 @@ static inline int dma_buf_init_sysfs_statistics(void) static inline void dma_buf_uninit_sysfs_statistics(void) {} -static inline int dma_buf_stats_setup(struct dma_buf *dmabuf) +static inline int dma_buf_stats_setup(struct dma_buf *dmabuf, struct file *file) { return 0; } diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c index 24e656a3ee7458738ee815207ccb155edcea2ce4..757c0fb77a6cb33aba4fc6fa28f529ae6f507201 100644 --- a/drivers/dma-buf/dma-buf.c +++ b/drivers/dma-buf/dma-buf.c @@ -95,10 +95,11 @@ static int dma_buf_file_release(struct inode *inode, struct file *file) return -EINVAL; dmabuf = file->private_data; - - mutex_lock(&db_list.lock); - list_del(&dmabuf->list_node); - mutex_unlock(&db_list.lock); + if (dmabuf) { + mutex_lock(&db_list.lock); + list_del(&dmabuf->list_node); + mutex_unlock(&db_list.lock); + } return 0; } @@ -528,17 +529,17 @@ static inline int is_dma_buf_file(struct file *file) return file->f_op == &dma_buf_fops; } -static struct file *dma_buf_getfile(struct dma_buf *dmabuf, int flags) +static struct file *dma_buf_getfile(size_t size, int flags) { static atomic64_t dmabuf_inode = ATOMIC64_INIT(0); - struct file *file; struct inode *inode = alloc_anon_inode(dma_buf_mnt->mnt_sb); + struct file *file; if (IS_ERR(inode)) return ERR_CAST(inode); - inode->i_size = dmabuf->size; - inode_set_bytes(inode, dmabuf->size); + inode->i_size = size; + inode_set_bytes(inode, size); /* * The ->i_ino acquired from get_next_ino() is not unique thus @@ -552,8 +553,6 @@ static struct file *dma_buf_getfile(struct dma_buf *dmabuf, int flags) flags, &dma_buf_fops); if (IS_ERR(file)) goto err_alloc_file; - file->private_data = dmabuf; - file->f_path.dentry->d_fsdata = dmabuf; return file; @@ -619,19 +618,11 @@ struct dma_buf *dma_buf_export(const struct dma_buf_export_info *exp_info) size_t alloc_size = sizeof(struct dma_buf); int ret; - if (!exp_info->resv) - alloc_size += sizeof(struct dma_resv); - else - /* prevent &dma_buf[1] == dma_buf->resv */ - alloc_size += 1; - - if (WARN_ON(!exp_info->priv - || !exp_info->ops - || !exp_info->ops->map_dma_buf - || !exp_info->ops->unmap_dma_buf - || !exp_info->ops->release)) { + if (WARN_ON(!exp_info->priv || !exp_info->ops + || !exp_info->ops->map_dma_buf + || !exp_info->ops->unmap_dma_buf + || !exp_info->ops->release)) return ERR_PTR(-EINVAL); - } if (WARN_ON(exp_info->ops->cache_sgt_mapping && (exp_info->ops->pin || exp_info->ops->unpin))) @@ -643,10 +634,21 @@ struct dma_buf *dma_buf_export(const struct dma_buf_export_info *exp_info) if (!try_module_get(exp_info->owner)) return ERR_PTR(-ENOENT); + file = dma_buf_getfile(exp_info->size, exp_info->flags); + if (IS_ERR(file)) { + ret = PTR_ERR(file); + goto err_module; + } + + if (!exp_info->resv) + alloc_size += sizeof(struct dma_resv); + else + /* prevent &dma_buf[1] == dma_buf->resv */ + alloc_size += 1; dmabuf = kzalloc(alloc_size, GFP_KERNEL); if (!dmabuf) { ret = -ENOMEM; - goto err_module; + goto err_file; } dmabuf->priv = exp_info->priv; @@ -658,43 +660,35 @@ struct dma_buf *dma_buf_export(const struct dma_buf_export_info *exp_info) init_waitqueue_head(&dmabuf->poll); dmabuf->cb_in.poll = dmabuf->cb_out.poll = &dmabuf->poll; dmabuf->cb_in.active = dmabuf->cb_out.active = 0; + INIT_LIST_HEAD(&dmabuf->attachments); if (!resv) { - resv = (struct dma_resv *)&dmabuf[1]; - dma_resv_init(resv); + dmabuf->resv = (struct dma_resv *)&dmabuf[1]; + dma_resv_init(dmabuf->resv); + } else { + dmabuf->resv = resv; } - dmabuf->resv = resv; - file = dma_buf_getfile(dmabuf, exp_info->flags); - if (IS_ERR(file)) { - ret = PTR_ERR(file); + ret = dma_buf_stats_setup(dmabuf, file); + if (ret) goto err_dmabuf; - } + file->private_data = dmabuf; + file->f_path.dentry->d_fsdata = dmabuf; dmabuf->file = file; - INIT_LIST_HEAD(&dmabuf->attachments); - mutex_lock(&db_list.lock); list_add(&dmabuf->list_node, &db_list.head); mutex_unlock(&db_list.lock); - ret = dma_buf_stats_setup(dmabuf); - if (ret) - goto err_sysfs; - return dmabuf; -err_sysfs: - /* - * Set file->f_path.dentry->d_fsdata to NULL so that when - * dma_buf_release() gets invoked by dentry_ops, it exits - * early before calling the release() dma_buf op. - */ - file->f_path.dentry->d_fsdata = NULL; - fput(file); err_dmabuf: + if (!resv) + dma_resv_fini(dmabuf->resv); kfree(dmabuf); +err_file: + fput(file); err_module: module_put(exp_info->owner); return ERR_PTR(ret); diff --git a/drivers/dma/dmaengine.c b/drivers/dma/dmaengine.c index c741b6431958c20c5d46eb118fdbd46a42672e7a..8a6e6b60d66f377de23f570288c417f17f19aad2 100644 --- a/drivers/dma/dmaengine.c +++ b/drivers/dma/dmaengine.c @@ -451,7 +451,8 @@ static int dma_chan_get(struct dma_chan *chan) /* The channel is already in use, update client count */ if (chan->client_count) { __module_get(owner); - goto out; + chan->client_count++; + return 0; } if (!try_module_get(owner)) @@ -470,11 +471,11 @@ static int dma_chan_get(struct dma_chan *chan) goto err_out; } + chan->client_count++; + if (!dma_has_cap(DMA_PRIVATE, chan->device->cap_mask)) balance_ref_count(chan); -out: - chan->client_count++; return 0; err_out: diff --git a/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c b/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c index a183d93bd7e2986e6a4f385d62395cde2a5af834..bf85aa0979ecb8ecf62b83291c5bdbb19b24c2d7 100644 --- a/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c +++ b/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c @@ -1018,6 +1018,11 @@ static noinline void axi_chan_handle_err(struct axi_dma_chan *chan, u32 status) /* The bad descriptor currently is in the head of vc list */ vd = vchan_next_desc(&chan->vc); + if (!vd) { + dev_err(chan2dev(chan), "BUG: %s, IRQ with no descriptors\n", + axi_chan_name(chan)); + goto out; + } /* Remove the completed descriptor from issued list */ list_del(&vd->node); @@ -1032,6 +1037,7 @@ static noinline void axi_chan_handle_err(struct axi_dma_chan *chan, u32 status) /* Try to restart the controller */ axi_chan_start_first_queued(chan); +out: spin_unlock_irqrestore(&chan->vc.lock, flags); } diff --git a/drivers/dma/idxd/device.c b/drivers/dma/idxd/device.c index 06f5d3783d7719e01ad896818f5fa48a75cf945e..29dbb0f52e186c4ff5cb9948c3cfb0224b47d486 100644 --- a/drivers/dma/idxd/device.c +++ b/drivers/dma/idxd/device.c @@ -1172,8 +1172,19 @@ static void idxd_flush_pending_descs(struct idxd_irq_entry *ie) spin_unlock(&ie->list_lock); list_for_each_entry_safe(desc, itr, &flist, list) { + struct dma_async_tx_descriptor *tx; + list_del(&desc->list); ctype = desc->completion->status ? IDXD_COMPLETE_NORMAL : IDXD_COMPLETE_ABORT; + /* + * wq is being disabled. Any remaining descriptors are + * likely to be stuck and can be dropped. callback could + * point to code that is no longer accessible, for example + * if dmatest module has been unloaded. + */ + tx = &desc->txd; + tx->callback = NULL; + tx->callback_result = NULL; idxd_dma_complete_txd(desc, ctype, true); } } @@ -1390,8 +1401,7 @@ int drv_enable_wq(struct idxd_wq *wq) err_irq: idxd_wq_unmap_portal(wq); err_map_portal: - rc = idxd_wq_disable(wq, false); - if (rc < 0) + if (idxd_wq_disable(wq, false)) dev_dbg(dev, "wq %s disable failed\n", dev_name(wq_confdev(wq))); err: return rc; @@ -1408,11 +1418,11 @@ void drv_disable_wq(struct idxd_wq *wq) dev_warn(dev, "Clients has claim on wq %d: %d\n", wq->id, idxd_wq_refcount(wq)); - idxd_wq_free_resources(wq); idxd_wq_unmap_portal(wq); idxd_wq_drain(wq); idxd_wq_free_irq(wq); idxd_wq_reset(wq); + idxd_wq_free_resources(wq); percpu_ref_exit(&wq->wq_active); wq->type = IDXD_WQT_NONE; wq->client_count = 0; diff --git a/drivers/dma/imx-sdma.c b/drivers/dma/imx-sdma.c index fbea5f62dd98b82f09be1c2a23fc36b54fc99db9..b926abe4fa43aca6ef00f6c1ec3405be824e6edb 100644 --- a/drivers/dma/imx-sdma.c +++ b/drivers/dma/imx-sdma.c @@ -1521,10 +1521,12 @@ static struct sdma_desc *sdma_transfer_init(struct sdma_channel *sdmac, sdma_config_ownership(sdmac, false, true, false); if (sdma_load_context(sdmac)) - goto err_desc_out; + goto err_bd_out; return desc; +err_bd_out: + sdma_free_bd(desc); err_desc_out: kfree(desc); err_out: diff --git a/drivers/dma/lgm/lgm-dma.c b/drivers/dma/lgm/lgm-dma.c index 9b9184f964be397a4f5f5b8a7a1c2410032d6456..1709d159af7e019e04fe015fa38690c55ef71fe4 100644 --- a/drivers/dma/lgm/lgm-dma.c +++ b/drivers/dma/lgm/lgm-dma.c @@ -914,7 +914,7 @@ static void ldma_dev_init(struct ldma_dev *d) } } -static int ldma_cfg_init(struct ldma_dev *d) +static int ldma_parse_dt(struct ldma_dev *d) { struct fwnode_handle *fwnode = dev_fwnode(d->dev); struct ldma_port *p; @@ -1661,10 +1661,6 @@ static int intel_ldma_probe(struct platform_device *pdev) p->ldev = d; } - ret = ldma_cfg_init(d); - if (ret) - return ret; - dma_dev->dev = &pdev->dev; ch_mask = (unsigned long)d->channels_mask; @@ -1675,6 +1671,10 @@ static int intel_ldma_probe(struct platform_device *pdev) ldma_dma_init_v3X(j, d); } + ret = ldma_parse_dt(d); + if (ret) + return ret; + dma_dev->device_alloc_chan_resources = ldma_alloc_chan_resources; dma_dev->device_free_chan_resources = ldma_free_chan_resources; dma_dev->device_terminate_all = ldma_terminate_all; diff --git a/drivers/dma/ptdma/ptdma-dev.c b/drivers/dma/ptdma/ptdma-dev.c index 377da23012ac23e826234673c1095f8dc70aefa3..a2bf13ff18b6d2904b489f880c54563c8e12410d 100644 --- a/drivers/dma/ptdma/ptdma-dev.c +++ b/drivers/dma/ptdma/ptdma-dev.c @@ -71,12 +71,13 @@ static int pt_core_execute_cmd(struct ptdma_desc *desc, struct pt_cmd_queue *cmd bool soc = FIELD_GET(DWORD0_SOC, desc->dw0); u8 *q_desc = (u8 *)&cmd_q->qbase[cmd_q->qidx]; u32 tail; + unsigned long flags; if (soc) { desc->dw0 |= FIELD_PREP(DWORD0_IOC, desc->dw0); desc->dw0 &= ~DWORD0_SOC; } - mutex_lock(&cmd_q->q_mutex); + spin_lock_irqsave(&cmd_q->q_lock, flags); /* Copy 32-byte command descriptor to hw queue. */ memcpy(q_desc, desc, 32); @@ -91,7 +92,7 @@ static int pt_core_execute_cmd(struct ptdma_desc *desc, struct pt_cmd_queue *cmd /* Turn the queue back on using our cached control register */ pt_start_queue(cmd_q); - mutex_unlock(&cmd_q->q_mutex); + spin_unlock_irqrestore(&cmd_q->q_lock, flags); return 0; } @@ -199,7 +200,7 @@ int pt_core_init(struct pt_device *pt) cmd_q->pt = pt; cmd_q->dma_pool = dma_pool; - mutex_init(&cmd_q->q_mutex); + spin_lock_init(&cmd_q->q_lock); /* Page alignment satisfies our needs for N <= 128 */ cmd_q->qsize = Q_SIZE(Q_DESC_SIZE); diff --git a/drivers/dma/ptdma/ptdma.h b/drivers/dma/ptdma/ptdma.h index d093c43b7d1343b11227e14b815149085a32696f..21b4bf895200b5512ca91a54a19e2ce93623d46e 100644 --- a/drivers/dma/ptdma/ptdma.h +++ b/drivers/dma/ptdma/ptdma.h @@ -196,7 +196,7 @@ struct pt_cmd_queue { struct ptdma_desc *qbase; /* Aligned queue start address (per requirement) */ - struct mutex q_mutex ____cacheline_aligned; + spinlock_t q_lock ____cacheline_aligned; unsigned int qidx; unsigned int qsize; diff --git a/drivers/dma/qcom/gpi.c b/drivers/dma/qcom/gpi.c index 061add83229512e07f6ed90701789e395f02522d..59a36cbf9b5f7f146d3d98b1d0f0eb127eede8e3 100644 --- a/drivers/dma/qcom/gpi.c +++ b/drivers/dma/qcom/gpi.c @@ -1756,6 +1756,7 @@ static int gpi_create_spi_tre(struct gchan *chan, struct gpi_desc *desc, tre->dword[3] = u32_encode_bits(TRE_TYPE_GO, TRE_FLAGS_TYPE); if (spi->cmd == SPI_RX) { tre->dword[3] |= u32_encode_bits(1, TRE_FLAGS_IEOB); + tre->dword[3] |= u32_encode_bits(1, TRE_FLAGS_LINK); } else if (spi->cmd == SPI_TX) { tre->dword[3] |= u32_encode_bits(1, TRE_FLAGS_CHAIN); } else { /* SPI_DUPLEX */ diff --git a/drivers/dma/tegra186-gpc-dma.c b/drivers/dma/tegra186-gpc-dma.c index 1d1180db6d4ecdca4e540da718fcf733aa42438f..8f67f453a4922c2434cbfc84d378f84db7d45557 100644 --- a/drivers/dma/tegra186-gpc-dma.c +++ b/drivers/dma/tegra186-gpc-dma.c @@ -711,6 +711,7 @@ static int tegra_dma_terminate_all(struct dma_chan *dc) return err; } + vchan_terminate_vdesc(&tdc->dma_desc->vd); tegra_dma_disable(tdc); tdc->dma_desc = NULL; } diff --git a/drivers/dma/tegra210-adma.c b/drivers/dma/tegra210-adma.c index ae39b52012b2fb74e723637652b2e07ea718defe..79da93cc77b64b0f0893332f92448a53950db0f1 100644 --- a/drivers/dma/tegra210-adma.c +++ b/drivers/dma/tegra210-adma.c @@ -221,7 +221,7 @@ static int tegra_adma_init(struct tegra_adma *tdma) int ret; /* Clear any interrupts */ - tdma_write(tdma, tdma->cdata->global_int_clear, 0x1); + tdma_write(tdma, tdma->cdata->ch_base_offset + tdma->cdata->global_int_clear, 0x1); /* Assert soft reset */ tdma_write(tdma, ADMA_GLOBAL_SOFT_RESET, 0x1); diff --git a/drivers/dma/ti/k3-udma.c b/drivers/dma/ti/k3-udma.c index ce8b80bb34d7d1f2f0d1c537d776cda691590193..4c62274e0b33264e7246cff5468eadcc4f948692 100644 --- a/drivers/dma/ti/k3-udma.c +++ b/drivers/dma/ti/k3-udma.c @@ -762,11 +762,12 @@ static void udma_decrement_byte_counters(struct udma_chan *uc, u32 val) if (uc->desc->dir == DMA_DEV_TO_MEM) { udma_rchanrt_write(uc, UDMA_CHAN_RT_BCNT_REG, val); udma_rchanrt_write(uc, UDMA_CHAN_RT_SBCNT_REG, val); - udma_rchanrt_write(uc, UDMA_CHAN_RT_PEER_BCNT_REG, val); + if (uc->config.ep_type != PSIL_EP_NATIVE) + udma_rchanrt_write(uc, UDMA_CHAN_RT_PEER_BCNT_REG, val); } else { udma_tchanrt_write(uc, UDMA_CHAN_RT_BCNT_REG, val); udma_tchanrt_write(uc, UDMA_CHAN_RT_SBCNT_REG, val); - if (!uc->bchan) + if (!uc->bchan && uc->config.ep_type != PSIL_EP_NATIVE) udma_tchanrt_write(uc, UDMA_CHAN_RT_PEER_BCNT_REG, val); } } diff --git a/drivers/dma/xilinx/xilinx_dma.c b/drivers/dma/xilinx/xilinx_dma.c index a8d23cdf883e5b825a3a66576903b1043280deae..ac09f0e5f58d88929a7dd9452b9044ce82e083d3 100644 --- a/drivers/dma/xilinx/xilinx_dma.c +++ b/drivers/dma/xilinx/xilinx_dma.c @@ -3143,8 +3143,10 @@ static int xilinx_dma_probe(struct platform_device *pdev) /* Initialize the channels */ for_each_child_of_node(node, child) { err = xilinx_dma_child_probe(xdev, child); - if (err < 0) + if (err < 0) { + of_node_put(child); goto error; + } } if (xdev->dma_config->dmatype == XDMA_TYPE_VDMA) { diff --git a/drivers/edac/edac_device.c b/drivers/edac/edac_device.c index 19522c568aa5dea572f5d2c34eac44905ed1f142..0689e15107213c6a11b70b3460a78db17c3663b1 100644 --- a/drivers/edac/edac_device.c +++ b/drivers/edac/edac_device.c @@ -34,6 +34,9 @@ static DEFINE_MUTEX(device_ctls_mutex); static LIST_HEAD(edac_device_list); +/* Default workqueue processing interval on this instance, in msecs */ +#define DEFAULT_POLL_INTERVAL 1000 + #ifdef CONFIG_EDAC_DEBUG static void edac_device_dump_device(struct edac_device_ctl_info *edac_dev) { @@ -336,7 +339,7 @@ static void edac_device_workq_function(struct work_struct *work_req) * whole one second to save timers firing all over the period * between integral seconds */ - if (edac_dev->poll_msec == 1000) + if (edac_dev->poll_msec == DEFAULT_POLL_INTERVAL) edac_queue_work(&edac_dev->work, round_jiffies_relative(edac_dev->delay)); else edac_queue_work(&edac_dev->work, edac_dev->delay); @@ -366,7 +369,7 @@ static void edac_device_workq_setup(struct edac_device_ctl_info *edac_dev, * timers firing on sub-second basis, while they are happy * to fire together on the 1 second exactly */ - if (edac_dev->poll_msec == 1000) + if (edac_dev->poll_msec == DEFAULT_POLL_INTERVAL) edac_queue_work(&edac_dev->work, round_jiffies_relative(edac_dev->delay)); else edac_queue_work(&edac_dev->work, edac_dev->delay); @@ -394,17 +397,16 @@ static void edac_device_workq_teardown(struct edac_device_ctl_info *edac_dev) * Then restart the workq on the new delay */ void edac_device_reset_delay_period(struct edac_device_ctl_info *edac_dev, - unsigned long value) + unsigned long msec) { - unsigned long jiffs = msecs_to_jiffies(value); - - if (value == 1000) - jiffs = round_jiffies_relative(value); - - edac_dev->poll_msec = value; - edac_dev->delay = jiffs; + edac_dev->poll_msec = msec; + edac_dev->delay = msecs_to_jiffies(msec); - edac_mod_work(&edac_dev->work, jiffs); + /* See comment in edac_device_workq_setup() above */ + if (edac_dev->poll_msec == DEFAULT_POLL_INTERVAL) + edac_mod_work(&edac_dev->work, round_jiffies_relative(edac_dev->delay)); + else + edac_mod_work(&edac_dev->work, edac_dev->delay); } int edac_device_alloc_index(void) @@ -443,11 +445,7 @@ int edac_device_add_device(struct edac_device_ctl_info *edac_dev) /* This instance is NOW RUNNING */ edac_dev->op_state = OP_RUNNING_POLL; - /* - * enable workq processing on this instance, - * default = 1000 msec - */ - edac_device_workq_setup(edac_dev, 1000); + edac_device_workq_setup(edac_dev, edac_dev->poll_msec ?: DEFAULT_POLL_INTERVAL); } else { edac_dev->op_state = OP_RUNNING_INTERRUPT; } diff --git a/drivers/edac/edac_module.h b/drivers/edac/edac_module.h index 763c076d96f21d368931e9d6cc68e10ab0a29ea3..47593afdc2348ff71a6600813cca8c11d01a1010 100644 --- a/drivers/edac/edac_module.h +++ b/drivers/edac/edac_module.h @@ -53,7 +53,7 @@ bool edac_stop_work(struct delayed_work *work); bool edac_mod_work(struct delayed_work *work, unsigned long delay); extern void edac_device_reset_delay_period(struct edac_device_ctl_info - *edac_dev, unsigned long value); + *edac_dev, unsigned long msec); extern void edac_mc_reset_delay_period(unsigned long value); /* diff --git a/drivers/edac/highbank_mc_edac.c b/drivers/edac/highbank_mc_edac.c index 61b76ec226af11dcefb8a8c91585295d37fcc610..19fba258ae108876773bd9563ff4f21038e170be 100644 --- a/drivers/edac/highbank_mc_edac.c +++ b/drivers/edac/highbank_mc_edac.c @@ -174,8 +174,10 @@ static int highbank_mc_probe(struct platform_device *pdev) drvdata = mci->pvt_info; platform_set_drvdata(pdev, mci); - if (!devres_open_group(&pdev->dev, NULL, GFP_KERNEL)) - return -ENOMEM; + if (!devres_open_group(&pdev->dev, NULL, GFP_KERNEL)) { + res = -ENOMEM; + goto free; + } r = platform_get_resource(pdev, IORESOURCE_MEM, 0); if (!r) { @@ -243,6 +245,7 @@ static int highbank_mc_probe(struct platform_device *pdev) edac_mc_del_mc(&pdev->dev); err: devres_release_group(&pdev->dev, NULL); +free: edac_mc_free(mci); return res; } diff --git a/drivers/edac/qcom_edac.c b/drivers/edac/qcom_edac.c index 97a27e42dd610db0d3cf656357bf7421e57b56ff..c45519f59dc11b5d4954a04390c321d28b57d114 100644 --- a/drivers/edac/qcom_edac.c +++ b/drivers/edac/qcom_edac.c @@ -252,7 +252,7 @@ dump_syn_reg_values(struct llcc_drv_data *drv, u32 bank, int err_type) static int dump_syn_reg(struct edac_device_ctl_info *edev_ctl, int err_type, u32 bank) { - struct llcc_drv_data *drv = edev_ctl->pvt_info; + struct llcc_drv_data *drv = edev_ctl->dev->platform_data; int ret; ret = dump_syn_reg_values(drv, bank, err_type); @@ -289,7 +289,7 @@ static irqreturn_t llcc_ecc_irq_handler(int irq, void *edev_ctl) { struct edac_device_ctl_info *edac_dev_ctl = edev_ctl; - struct llcc_drv_data *drv = edac_dev_ctl->pvt_info; + struct llcc_drv_data *drv = edac_dev_ctl->dev->platform_data; irqreturn_t irq_rc = IRQ_NONE; u32 drp_error, trp_error, i; int ret; @@ -358,7 +358,6 @@ static int qcom_llcc_edac_probe(struct platform_device *pdev) edev_ctl->dev_name = dev_name(dev); edev_ctl->ctl_name = "llcc"; edev_ctl->panic_on_ue = LLCC_ERP_PANIC_ON_UE; - edev_ctl->pvt_info = llcc_driv_data; rc = edac_device_add_device(edev_ctl); if (rc) diff --git a/drivers/firmware/arm_scmi/driver.c b/drivers/firmware/arm_scmi/driver.c index f818d00bb2c69e700dc0ce5e70f1cf84df55953e..ffdad59ec81fcb5e438061aba98accede1cf7d5a 100644 --- a/drivers/firmware/arm_scmi/driver.c +++ b/drivers/firmware/arm_scmi/driver.c @@ -910,6 +910,8 @@ static int do_xfer(const struct scmi_protocol_handle *ph, xfer->hdr.protocol_id, xfer->hdr.seq, xfer->hdr.poll_completion); + /* Clear any stale status */ + xfer->hdr.status = SCMI_SUCCESS; xfer->state = SCMI_XFER_SENT_OK; /* * Even though spinlocking is not needed here since no race is possible diff --git a/drivers/firmware/arm_scmi/shmem.c b/drivers/firmware/arm_scmi/shmem.c index 1dfe534b85184528c31e849668bd38edd1bf4154..87b4f4d35f06230bc161fc4205c7b199e03c0015 100644 --- a/drivers/firmware/arm_scmi/shmem.c +++ b/drivers/firmware/arm_scmi/shmem.c @@ -81,10 +81,11 @@ u32 shmem_read_header(struct scmi_shared_mem __iomem *shmem) void shmem_fetch_response(struct scmi_shared_mem __iomem *shmem, struct scmi_xfer *xfer) { + size_t len = ioread32(&shmem->length); + xfer->hdr.status = ioread32(shmem->msg_payload); /* Skip the length of header and status in shmem area i.e 8 bytes */ - xfer->rx.len = min_t(size_t, xfer->rx.len, - ioread32(&shmem->length) - 8); + xfer->rx.len = min_t(size_t, xfer->rx.len, len > 8 ? len - 8 : 0); /* Take a copy to the rx buffer.. */ memcpy_fromio(xfer->rx.buf, shmem->msg_payload + 4, xfer->rx.len); @@ -93,8 +94,10 @@ void shmem_fetch_response(struct scmi_shared_mem __iomem *shmem, void shmem_fetch_notification(struct scmi_shared_mem __iomem *shmem, size_t max_len, struct scmi_xfer *xfer) { + size_t len = ioread32(&shmem->length); + /* Skip only the length of header in shmem area i.e 4 bytes */ - xfer->rx.len = min_t(size_t, max_len, ioread32(&shmem->length) - 4); + xfer->rx.len = min_t(size_t, max_len, len > 4 ? len - 4 : 0); /* Take a copy to the rx buffer.. */ memcpy_fromio(xfer->rx.buf, shmem->msg_payload, xfer->rx.len); diff --git a/drivers/firmware/arm_scmi/virtio.c b/drivers/firmware/arm_scmi/virtio.c index 33c9b81a55cd11926dad6426cd2e748052150f40..1db975c08896983f994be90fea6a361cc3099389 100644 --- a/drivers/firmware/arm_scmi/virtio.c +++ b/drivers/firmware/arm_scmi/virtio.c @@ -160,7 +160,6 @@ static void scmi_vio_channel_cleanup_sync(struct scmi_vio_channel *vioch) } vioch->shutdown_done = &vioch_shutdown_done; - virtio_break_device(vioch->vqueue->vdev); if (!vioch->is_rx && vioch->deferred_tx_wq) /* Cannot be kicked anymore after this...*/ vioch->deferred_tx_wq = NULL; @@ -482,6 +481,12 @@ static int virtio_chan_free(int id, void *p, void *data) struct scmi_chan_info *cinfo = p; struct scmi_vio_channel *vioch = cinfo->transport_info; + /* + * Break device to inhibit further traffic flowing while shutting down + * the channels: doing it later holding vioch->lock creates unsafe + * locking dependency chains as reported by LOCKDEP. + */ + virtio_break_device(vioch->vqueue->vdev); scmi_vio_channel_cleanup_sync(vioch); scmi_free_channel(cinfo, data, id); diff --git a/drivers/firmware/efi/efi.c b/drivers/firmware/efi/efi.c index 09716eebe8ac3adb2d395fa037284d9971e394bc..a2b0cbc8741c2ff1cf175c4d70a7eba6b7d37050 100644 --- a/drivers/firmware/efi/efi.c +++ b/drivers/firmware/efi/efi.c @@ -394,8 +394,8 @@ static int __init efisubsys_init(void) efi_kobj = kobject_create_and_add("efi", firmware_kobj); if (!efi_kobj) { pr_err("efi: Firmware registration failed.\n"); - destroy_workqueue(efi_rts_wq); - return -ENOMEM; + error = -ENOMEM; + goto err_destroy_wq; } if (efi_rt_services_supported(EFI_RT_SUPPORTED_GET_VARIABLE | @@ -443,7 +443,10 @@ static int __init efisubsys_init(void) err_put: kobject_put(efi_kobj); efi_kobj = NULL; - destroy_workqueue(efi_rts_wq); +err_destroy_wq: + if (efi_rts_wq) + destroy_workqueue(efi_rts_wq); + return error; } diff --git a/drivers/firmware/efi/runtime-wrappers.c b/drivers/firmware/efi/runtime-wrappers.c index 7feee3d9c2bfbeec307612ffdd3373f607cc1b62..1fba4e09cdcff8f02c8daaaff8127da66bbd8942 100644 --- a/drivers/firmware/efi/runtime-wrappers.c +++ b/drivers/firmware/efi/runtime-wrappers.c @@ -62,6 +62,7 @@ struct efi_runtime_work efi_rts_work; \ if (!efi_enabled(EFI_RUNTIME_SERVICES)) { \ pr_warn_once("EFI Runtime Services are disabled!\n"); \ + efi_rts_work.status = EFI_DEVICE_ERROR; \ goto exit; \ } \ \ diff --git a/drivers/firmware/google/coreboot_table.c b/drivers/firmware/google/coreboot_table.c index 2652c396c42368e2342bfca6814a9117ed2b745f..33ae94745aef976f376bab93cbed59024c2a905f 100644 --- a/drivers/firmware/google/coreboot_table.c +++ b/drivers/firmware/google/coreboot_table.c @@ -93,14 +93,19 @@ static int coreboot_table_populate(struct device *dev, void *ptr) for (i = 0; i < header->table_entries; i++) { entry = ptr_entry; - device = kzalloc(sizeof(struct device) + entry->size, GFP_KERNEL); + if (entry->size < sizeof(*entry)) { + dev_warn(dev, "coreboot table entry too small!\n"); + return -EINVAL; + } + + device = kzalloc(sizeof(device->dev) + entry->size, GFP_KERNEL); if (!device) return -ENOMEM; device->dev.parent = dev; device->dev.bus = &coreboot_bus_type; device->dev.release = coreboot_device_release; - memcpy(&device->entry, ptr_entry, entry->size); + memcpy(device->raw, ptr_entry, entry->size); switch (device->entry.tag) { case LB_TAG_CBMEM_ENTRY: diff --git a/drivers/firmware/google/coreboot_table.h b/drivers/firmware/google/coreboot_table.h index 37f4d335a606d715fde2324b12e70e1e18179e2b..d814dca33a084cfd526d54fadf2c4353a292362c 100644 --- a/drivers/firmware/google/coreboot_table.h +++ b/drivers/firmware/google/coreboot_table.h @@ -79,6 +79,7 @@ struct coreboot_device { struct lb_cbmem_ref cbmem_ref; struct lb_cbmem_entry cbmem_entry; struct lb_framebuffer framebuffer; + DECLARE_FLEX_ARRAY(u8, raw); }; }; diff --git a/drivers/firmware/google/gsmi.c b/drivers/firmware/google/gsmi.c index 4e2575dfeb908ff50d37a1c6a5d7be4f6313ba5b..871bedf533a805f65975e1b94ae125d65f431a09 100644 --- a/drivers/firmware/google/gsmi.c +++ b/drivers/firmware/google/gsmi.c @@ -361,9 +361,10 @@ static efi_status_t gsmi_get_variable(efi_char16_t *name, memcpy(data, gsmi_dev.data_buf->start, *data_size); /* All variables are have the following attributes */ - *attr = EFI_VARIABLE_NON_VOLATILE | - EFI_VARIABLE_BOOTSERVICE_ACCESS | - EFI_VARIABLE_RUNTIME_ACCESS; + if (attr) + *attr = EFI_VARIABLE_NON_VOLATILE | + EFI_VARIABLE_BOOTSERVICE_ACCESS | + EFI_VARIABLE_RUNTIME_ACCESS; } spin_unlock_irqrestore(&gsmi_dev.lock, flags); diff --git a/drivers/firmware/psci/psci.c b/drivers/firmware/psci/psci.c index e7bcfca4159f603317660d8525e6eafd4488ec94..447ee4ea5c903421f762e649318bf6ac6ff28662 100644 --- a/drivers/firmware/psci/psci.c +++ b/drivers/firmware/psci/psci.c @@ -440,6 +440,9 @@ static const struct file_operations psci_debugfs_ops = { static int __init psci_debugfs_init(void) { + if (!invoke_psci_fn || !psci_ops.get_version) + return 0; + return PTR_ERR_OR_ZERO(debugfs_create_file("psci", 0444, NULL, NULL, &psci_debugfs_ops)); } diff --git a/drivers/gpio/gpio-eic-sprd.c b/drivers/gpio/gpio-eic-sprd.c index 8d722e026e9c98b2ef935c4f8eff3296cce769ee..84352a6f4973b5be64afc488e00d1a39d732d5d6 100644 --- a/drivers/gpio/gpio-eic-sprd.c +++ b/drivers/gpio/gpio-eic-sprd.c @@ -91,7 +91,6 @@ enum sprd_eic_type { struct sprd_eic { struct gpio_chip chip; - struct irq_chip intc; void __iomem *base[SPRD_EIC_MAX_BANK]; enum sprd_eic_type type; spinlock_t lock; @@ -255,6 +254,8 @@ static void sprd_eic_irq_mask(struct irq_data *data) default: dev_err(chip->parent, "Unsupported EIC type.\n"); } + + gpiochip_disable_irq(chip, offset); } static void sprd_eic_irq_unmask(struct irq_data *data) @@ -263,6 +264,8 @@ static void sprd_eic_irq_unmask(struct irq_data *data) struct sprd_eic *sprd_eic = gpiochip_get_data(chip); u32 offset = irqd_to_hwirq(data); + gpiochip_enable_irq(chip, offset); + switch (sprd_eic->type) { case SPRD_EIC_DEBOUNCE: sprd_eic_update(chip, offset, SPRD_EIC_DBNC_IE, 1); @@ -564,6 +567,15 @@ static void sprd_eic_irq_handler(struct irq_desc *desc) chained_irq_exit(ic, desc); } +static const struct irq_chip sprd_eic_irq = { + .name = "sprd-eic", + .irq_ack = sprd_eic_irq_ack, + .irq_mask = sprd_eic_irq_mask, + .irq_unmask = sprd_eic_irq_unmask, + .irq_set_type = sprd_eic_irq_set_type, + .flags = IRQCHIP_SKIP_SET_WAKE | IRQCHIP_IMMUTABLE, + GPIOCHIP_IRQ_RESOURCE_HELPERS, +}; static int sprd_eic_probe(struct platform_device *pdev) { const struct sprd_eic_variant_data *pdata; @@ -626,15 +638,8 @@ static int sprd_eic_probe(struct platform_device *pdev) break; } - sprd_eic->intc.name = dev_name(&pdev->dev); - sprd_eic->intc.irq_ack = sprd_eic_irq_ack; - sprd_eic->intc.irq_mask = sprd_eic_irq_mask; - sprd_eic->intc.irq_unmask = sprd_eic_irq_unmask; - sprd_eic->intc.irq_set_type = sprd_eic_irq_set_type; - sprd_eic->intc.flags = IRQCHIP_SKIP_SET_WAKE; - irq = &sprd_eic->chip.irq; - irq->chip = &sprd_eic->intc; + gpio_irq_chip_set_chip(irq, &sprd_eic_irq); irq->handler = handle_bad_irq; irq->default_type = IRQ_TYPE_NONE; irq->parent_handler = sprd_eic_irq_handler; diff --git a/drivers/gpio/gpio-ep93xx.c b/drivers/gpio/gpio-ep93xx.c index 2e17797091133119d093d590fbfe8e5e323e1a51..6cedf46efec6cbc2c1c737381d83ce22fae18007 100644 --- a/drivers/gpio/gpio-ep93xx.c +++ b/drivers/gpio/gpio-ep93xx.c @@ -17,6 +17,7 @@ #include #include #include +#include #define EP93XX_GPIO_F_INT_STATUS 0x5c #define EP93XX_GPIO_A_INT_STATUS 0xa0 @@ -40,7 +41,6 @@ #define EP93XX_GPIO_F_IRQ_BASE 80 struct ep93xx_gpio_irq_chip { - struct irq_chip ic; u8 irq_offset; u8 int_unmasked; u8 int_enabled; @@ -148,7 +148,7 @@ static void ep93xx_gpio_f_irq_handler(struct irq_desc *desc) */ struct irq_chip *irqchip = irq_desc_get_chip(desc); unsigned int irq = irq_desc_get_irq(desc); - int port_f_idx = ((irq + 1) & 7) ^ 4; /* {19..22,47..50} -> {0..7} */ + int port_f_idx = (irq & 7) ^ 4; /* {20..23,48..51} -> {0..7} */ int gpio_irq = EP93XX_GPIO_F_IRQ_BASE + port_f_idx; chained_irq_enter(irqchip, desc); @@ -185,6 +185,7 @@ static void ep93xx_gpio_irq_mask_ack(struct irq_data *d) ep93xx_gpio_update_int_params(epg, eic); writeb(port_mask, epg->base + eic->irq_offset + EP93XX_INT_EOI_OFFSET); + gpiochip_disable_irq(gc, irqd_to_hwirq(d)); } static void ep93xx_gpio_irq_mask(struct irq_data *d) @@ -195,6 +196,7 @@ static void ep93xx_gpio_irq_mask(struct irq_data *d) eic->int_unmasked &= ~BIT(d->irq & 7); ep93xx_gpio_update_int_params(epg, eic); + gpiochip_disable_irq(gc, irqd_to_hwirq(d)); } static void ep93xx_gpio_irq_unmask(struct irq_data *d) @@ -203,6 +205,7 @@ static void ep93xx_gpio_irq_unmask(struct irq_data *d) struct ep93xx_gpio_irq_chip *eic = to_ep93xx_gpio_irq_chip(gc); struct ep93xx_gpio *epg = gpiochip_get_data(gc); + gpiochip_enable_irq(gc, irqd_to_hwirq(d)); eic->int_unmasked |= BIT(d->irq & 7); ep93xx_gpio_update_int_params(epg, eic); } @@ -320,15 +323,25 @@ static int ep93xx_gpio_set_config(struct gpio_chip *gc, unsigned offset, return 0; } -static void ep93xx_init_irq_chip(struct device *dev, struct irq_chip *ic) +static void ep93xx_irq_print_chip(struct irq_data *data, struct seq_file *p) { - ic->irq_ack = ep93xx_gpio_irq_ack; - ic->irq_mask_ack = ep93xx_gpio_irq_mask_ack; - ic->irq_mask = ep93xx_gpio_irq_mask; - ic->irq_unmask = ep93xx_gpio_irq_unmask; - ic->irq_set_type = ep93xx_gpio_irq_type; + struct gpio_chip *gc = irq_data_get_irq_chip_data(data); + + seq_printf(p, dev_name(gc->parent)); } +static const struct irq_chip gpio_eic_irq_chip = { + .name = "ep93xx-gpio-eic", + .irq_ack = ep93xx_gpio_irq_ack, + .irq_mask = ep93xx_gpio_irq_mask, + .irq_unmask = ep93xx_gpio_irq_unmask, + .irq_mask_ack = ep93xx_gpio_irq_mask_ack, + .irq_set_type = ep93xx_gpio_irq_type, + .irq_print_chip = ep93xx_irq_print_chip, + .flags = IRQCHIP_IMMUTABLE, + GPIOCHIP_IRQ_RESOURCE_HELPERS, +}; + static int ep93xx_gpio_add_bank(struct ep93xx_gpio_chip *egc, struct platform_device *pdev, struct ep93xx_gpio *epg, @@ -350,8 +363,6 @@ static int ep93xx_gpio_add_bank(struct ep93xx_gpio_chip *egc, girq = &gc->irq; if (bank->has_irq || bank->has_hierarchical_irq) { - struct irq_chip *ic; - gc->set_config = ep93xx_gpio_set_config; egc->eic = devm_kcalloc(dev, 1, sizeof(*egc->eic), @@ -359,12 +370,7 @@ static int ep93xx_gpio_add_bank(struct ep93xx_gpio_chip *egc, if (!egc->eic) return -ENOMEM; egc->eic->irq_offset = bank->irq; - ic = &egc->eic->ic; - ic->name = devm_kasprintf(dev, GFP_KERNEL, "gpio-irq-%s", bank->label); - if (!ic->name) - return -ENOMEM; - ep93xx_init_irq_chip(dev, ic); - girq->chip = ic; + gpio_irq_chip_set_chip(girq, &gpio_eic_irq_chip); } if (bank->has_irq) { diff --git a/drivers/gpio/gpio-mxc.c b/drivers/gpio/gpio-mxc.c index d5626c572d24ecc095eace76ce733b703cddf2b9..9d0cec4b82a334676e0a9d7818f51394838168c7 100644 --- a/drivers/gpio/gpio-mxc.c +++ b/drivers/gpio/gpio-mxc.c @@ -18,6 +18,7 @@ #include #include #include +#include #include #include #include @@ -159,6 +160,7 @@ static int gpio_set_irq_type(struct irq_data *d, u32 type) { struct irq_chip_generic *gc = irq_data_get_irq_chip_data(d); struct mxc_gpio_port *port = gc->private; + unsigned long flags; u32 bit, val; u32 gpio_idx = d->hwirq; int edge; @@ -197,6 +199,8 @@ static int gpio_set_irq_type(struct irq_data *d, u32 type) return -EINVAL; } + raw_spin_lock_irqsave(&port->gc.bgpio_lock, flags); + if (GPIO_EDGE_SEL >= 0) { val = readl(port->base + GPIO_EDGE_SEL); if (edge == GPIO_INT_BOTH_EDGES) @@ -217,15 +221,20 @@ static int gpio_set_irq_type(struct irq_data *d, u32 type) writel(1 << gpio_idx, port->base + GPIO_ISR); port->pad_type[gpio_idx] = type; - return 0; + raw_spin_unlock_irqrestore(&port->gc.bgpio_lock, flags); + + return port->gc.direction_input(&port->gc, gpio_idx); } static void mxc_flip_edge(struct mxc_gpio_port *port, u32 gpio) { void __iomem *reg = port->base; + unsigned long flags; u32 bit, val; int edge; + raw_spin_lock_irqsave(&port->gc.bgpio_lock, flags); + reg += GPIO_ICR1 + ((gpio & 0x10) >> 2); /* lower or upper register */ bit = gpio & 0xf; val = readl(reg); @@ -240,9 +249,12 @@ static void mxc_flip_edge(struct mxc_gpio_port *port, u32 gpio) } else { pr_err("mxc: invalid configuration for GPIO %d: %x\n", gpio, edge); - return; + goto unlock; } writel(val | (edge << (bit << 1)), reg); + +unlock: + raw_spin_unlock_irqrestore(&port->gc.bgpio_lock, flags); } /* handle 32 interrupts in one status register */ diff --git a/drivers/gpio/gpio-pca953x.c b/drivers/gpio/gpio-pca953x.c index a59d61cd44b2e5eceac9bf97c41945ac77bde0f2..5299e5bb76d6e339cf269d4cb560e42fabbf0922 100644 --- a/drivers/gpio/gpio-pca953x.c +++ b/drivers/gpio/gpio-pca953x.c @@ -474,6 +474,9 @@ static u8 pcal6534_recalc_addr(struct pca953x_chip *chip, int reg, int off) case PCAL6524_DEBOUNCE: pinctrl = ((reg & PCAL_PINCTRL_MASK) >> 1) + 0x1c; break; + default: + pinctrl = 0; + break; } return pinctrl + addr + (off / BANK_SZ); diff --git a/drivers/gpio/gpio-pmic-eic-sprd.c b/drivers/gpio/gpio-pmic-eic-sprd.c index e518490c4b681afa6f979f9e8a5ce5a964aa71e1..c3e4d90f6b183318caf7712f7b3f1531114f6bce 100644 --- a/drivers/gpio/gpio-pmic-eic-sprd.c +++ b/drivers/gpio/gpio-pmic-eic-sprd.c @@ -47,7 +47,6 @@ enum { /** * struct sprd_pmic_eic - PMIC EIC controller * @chip: the gpio_chip structure. - * @intc: the irq_chip structure. * @map: the regmap from the parent device. * @offset: the EIC controller's offset address of the PMIC. * @reg: the array to cache the EIC registers. @@ -56,7 +55,6 @@ enum { */ struct sprd_pmic_eic { struct gpio_chip chip; - struct irq_chip intc; struct regmap *map; u32 offset; u8 reg[CACHE_NR_REGS]; @@ -151,15 +149,21 @@ static void sprd_pmic_eic_irq_mask(struct irq_data *data) { struct gpio_chip *chip = irq_data_get_irq_chip_data(data); struct sprd_pmic_eic *pmic_eic = gpiochip_get_data(chip); + u32 offset = irqd_to_hwirq(data); pmic_eic->reg[REG_IE] = 0; pmic_eic->reg[REG_TRIG] = 0; + + gpiochip_disable_irq(chip, offset); } static void sprd_pmic_eic_irq_unmask(struct irq_data *data) { struct gpio_chip *chip = irq_data_get_irq_chip_data(data); struct sprd_pmic_eic *pmic_eic = gpiochip_get_data(chip); + u32 offset = irqd_to_hwirq(data); + + gpiochip_enable_irq(chip, offset); pmic_eic->reg[REG_IE] = 1; pmic_eic->reg[REG_TRIG] = 1; @@ -292,6 +296,17 @@ static irqreturn_t sprd_pmic_eic_irq_handler(int irq, void *data) return IRQ_HANDLED; } +static const struct irq_chip pmic_eic_irq_chip = { + .name = "sprd-pmic-eic", + .irq_mask = sprd_pmic_eic_irq_mask, + .irq_unmask = sprd_pmic_eic_irq_unmask, + .irq_set_type = sprd_pmic_eic_irq_set_type, + .irq_bus_lock = sprd_pmic_eic_bus_lock, + .irq_bus_sync_unlock = sprd_pmic_eic_bus_sync_unlock, + .flags = IRQCHIP_SKIP_SET_WAKE | IRQCHIP_IMMUTABLE, + GPIOCHIP_IRQ_RESOURCE_HELPERS, +}; + static int sprd_pmic_eic_probe(struct platform_device *pdev) { struct gpio_irq_chip *irq; @@ -338,16 +353,8 @@ static int sprd_pmic_eic_probe(struct platform_device *pdev) pmic_eic->chip.set = sprd_pmic_eic_set; pmic_eic->chip.get = sprd_pmic_eic_get; - pmic_eic->intc.name = dev_name(&pdev->dev); - pmic_eic->intc.irq_mask = sprd_pmic_eic_irq_mask; - pmic_eic->intc.irq_unmask = sprd_pmic_eic_irq_unmask; - pmic_eic->intc.irq_set_type = sprd_pmic_eic_irq_set_type; - pmic_eic->intc.irq_bus_lock = sprd_pmic_eic_bus_lock; - pmic_eic->intc.irq_bus_sync_unlock = sprd_pmic_eic_bus_sync_unlock; - pmic_eic->intc.flags = IRQCHIP_SKIP_SET_WAKE; - irq = &pmic_eic->chip.irq; - irq->chip = &pmic_eic->intc; + gpio_irq_chip_set_chip(irq, &pmic_eic_irq_chip); irq->threaded = true; ret = devm_gpiochip_add_data(&pdev->dev, &pmic_eic->chip, pmic_eic); diff --git a/drivers/gpio/gpio-sifive.c b/drivers/gpio/gpio-sifive.c index 238f3210970cfa0ed3d38d43b141afc0e03135e8..bc5660f61c570d770a2d5cb44ee6bd2768aa6189 100644 --- a/drivers/gpio/gpio-sifive.c +++ b/drivers/gpio/gpio-sifive.c @@ -215,6 +215,7 @@ static int sifive_gpio_probe(struct platform_device *pdev) return -ENODEV; } parent = irq_find_host(irq_parent); + of_node_put(irq_parent); if (!parent) { dev_err(dev, "no IRQ parent domain\n"); return -ENODEV; diff --git a/drivers/gpio/gpio-sprd.c b/drivers/gpio/gpio-sprd.c index 9bff63990eee43c0d9f5cc52a50164564cca1f9c..072b4e6532164e5f5e12044fec58cf109513a6de 100644 --- a/drivers/gpio/gpio-sprd.c +++ b/drivers/gpio/gpio-sprd.c @@ -120,6 +120,7 @@ static void sprd_gpio_irq_mask(struct irq_data *data) u32 offset = irqd_to_hwirq(data); sprd_gpio_update(chip, offset, SPRD_GPIO_IE, 0); + gpiochip_disable_irq(chip, offset); } static void sprd_gpio_irq_ack(struct irq_data *data) @@ -136,6 +137,7 @@ static void sprd_gpio_irq_unmask(struct irq_data *data) u32 offset = irqd_to_hwirq(data); sprd_gpio_update(chip, offset, SPRD_GPIO_IE, 1); + gpiochip_enable_irq(chip, offset); } static int sprd_gpio_irq_set_type(struct irq_data *data, @@ -205,13 +207,14 @@ static void sprd_gpio_irq_handler(struct irq_desc *desc) chained_irq_exit(ic, desc); } -static struct irq_chip sprd_gpio_irqchip = { +static const struct irq_chip sprd_gpio_irqchip = { .name = "sprd-gpio", .irq_ack = sprd_gpio_irq_ack, .irq_mask = sprd_gpio_irq_mask, .irq_unmask = sprd_gpio_irq_unmask, .irq_set_type = sprd_gpio_irq_set_type, - .flags = IRQCHIP_SKIP_SET_WAKE, + .flags = IRQCHIP_SKIP_SET_WAKE | IRQCHIP_IMMUTABLE, + GPIOCHIP_IRQ_RESOURCE_HELPERS, }; static int sprd_gpio_probe(struct platform_device *pdev) @@ -245,7 +248,7 @@ static int sprd_gpio_probe(struct platform_device *pdev) sprd_gpio->chip.direction_output = sprd_gpio_direction_output; irq = &sprd_gpio->chip.irq; - irq->chip = &sprd_gpio_irqchip; + gpio_irq_chip_set_chip(irq, &sprd_gpio_irqchip); irq->handler = handle_bad_irq; irq->default_type = IRQ_TYPE_NONE; irq->parent_handler = sprd_gpio_irq_handler; diff --git a/drivers/gpio/gpiolib-acpi.c b/drivers/gpio/gpiolib-acpi.c index bed0380c51360a4e5e8cbd15959f685b4d2922fd..17c53f484280f086d46cb95bf2dd22a978cd96e8 100644 --- a/drivers/gpio/gpiolib-acpi.c +++ b/drivers/gpio/gpiolib-acpi.c @@ -385,7 +385,7 @@ static bool acpi_gpio_in_ignore_list(const char *ignore_list, const char *contro } static bool acpi_gpio_irq_is_wake(struct device *parent, - struct acpi_resource_gpio *agpio) + const struct acpi_resource_gpio *agpio) { unsigned int pin = agpio->pin_table[0]; @@ -778,7 +778,7 @@ static int acpi_populate_gpio_lookup(struct acpi_resource *ares, void *data) lookup->info.pin_config = agpio->pin_config; lookup->info.debounce = agpio->debounce_timeout; lookup->info.gpioint = gpioint; - lookup->info.wake_capable = agpio->wake_capable == ACPI_WAKE_CAPABLE; + lookup->info.wake_capable = acpi_gpio_irq_is_wake(&lookup->info.adev->dev, agpio); /* * Polarity and triggering are only specified for GpioInt @@ -1104,7 +1104,8 @@ int acpi_dev_gpio_irq_wake_get_by(struct acpi_device *adev, const char *name, in dev_dbg(&adev->dev, "IRQ %d already in use\n", irq); } - if (wake_capable) + /* avoid suspend issues with GPIOs when systems are using S3 */ + if (wake_capable && acpi_gbl_FADT.flags & ACPI_FADT_LOW_POWER_S0) *wake_capable = info.wake_capable; return irq; @@ -1623,6 +1624,19 @@ static const struct dmi_system_id gpiolib_acpi_quirks[] __initconst = { .ignore_interrupt = "AMDI0030:00@18", }, }, + { + /* + * Spurious wakeups from TP_ATTN# pin + * Found in BIOS 1.7.8 + * https://gitlab.freedesktop.org/drm/amd/-/issues/1722#note_1720627 + */ + .matches = { + DMI_MATCH(DMI_BOARD_NAME, "NL5xRU"), + }, + .driver_data = &(struct acpi_gpiolib_dmi_quirk) { + .ignore_wake = "ELAN0415:00@9", + }, + }, {} /* Terminating entry */ }; diff --git a/drivers/gpio/gpiolib.c b/drivers/gpio/gpiolib.c index 5a66d9616d7ccedf5e8a43235203760f29f942ee..939c776b948813c1fb5c755643ac15cbe7a4e6d9 100644 --- a/drivers/gpio/gpiolib.c +++ b/drivers/gpio/gpiolib.c @@ -3905,8 +3905,8 @@ static struct gpio_desc *gpiod_find_and_request(struct device *consumer, const char *label, bool platform_lookup_allowed) { + unsigned long lookupflags = GPIO_LOOKUP_FLAGS_DEFAULT; struct gpio_desc *desc = ERR_PTR(-ENOENT); - unsigned long lookupflags; int ret; if (!IS_ERR_OR_NULL(fwnode)) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c index 8b7a09b392ac2e091cbf86cd88e2659cb5fceca9..0f4cb41078c1708df27c90698a8ebb29de94e323 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c @@ -63,6 +63,8 @@ static int amdgpu_cs_parser_init(struct amdgpu_cs_parser *p, amdgpu_ctx_put(p->ctx); return -ECANCELED; } + + amdgpu_sync_create(&p->sync); return 0; } @@ -454,18 +456,6 @@ static int amdgpu_syncobj_lookup_and_add(struct amdgpu_cs_parser *p, } r = amdgpu_sync_fence(&p->sync, fence); - if (r) - goto error; - - /* - * When we have an explicit dependency it might be necessary to insert a - * pipeline sync to make sure that all caches etc are flushed and the - * next job actually sees the results from the previous one. - */ - if (fence->context == p->gang_leader->base.entity->fence_context) - r = amdgpu_sync_fence(&p->gang_leader->explicit_sync, fence); - -error: dma_fence_put(fence); return r; } @@ -1190,10 +1180,19 @@ static int amdgpu_cs_vm_handling(struct amdgpu_cs_parser *p) static int amdgpu_cs_sync_rings(struct amdgpu_cs_parser *p) { struct amdgpu_fpriv *fpriv = p->filp->driver_priv; + struct drm_gpu_scheduler *sched; struct amdgpu_bo_list_entry *e; + struct dma_fence *fence; unsigned int i; int r; + r = amdgpu_ctx_wait_prev_fence(p->ctx, p->entities[p->gang_leader_idx]); + if (r) { + if (r != -ERESTARTSYS) + DRM_ERROR("amdgpu_ctx_wait_prev_fence failed.\n"); + return r; + } + list_for_each_entry(e, &p->validated, tv.head) { struct amdgpu_bo *bo = ttm_to_amdgpu_bo(e->tv.bo); struct dma_resv *resv = bo->tbo.base.resv; @@ -1213,10 +1212,24 @@ static int amdgpu_cs_sync_rings(struct amdgpu_cs_parser *p) return r; } - r = amdgpu_ctx_wait_prev_fence(p->ctx, p->entities[p->gang_leader_idx]); - if (r && r != -ERESTARTSYS) - DRM_ERROR("amdgpu_ctx_wait_prev_fence failed.\n"); - return r; + sched = p->gang_leader->base.entity->rq->sched; + while ((fence = amdgpu_sync_get_fence(&p->sync))) { + struct drm_sched_fence *s_fence = to_drm_sched_fence(fence); + + /* + * When we have an dependency it might be necessary to insert a + * pipeline sync to make sure that all caches etc are flushed and the + * next job actually sees the results from the previous one + * before we start executing on the same scheduler ring. + */ + if (!s_fence || s_fence->sched != sched) + continue; + + r = amdgpu_sync_fence(&p->gang_leader->explicit_sync, fence); + if (r) + return r; + } + return 0; } static void amdgpu_cs_post_dependencies(struct amdgpu_cs_parser *p) @@ -1256,9 +1269,12 @@ static int amdgpu_cs_submit(struct amdgpu_cs_parser *p, continue; fence = &p->jobs[i]->base.s_fence->scheduled; + dma_fence_get(fence); r = drm_sched_job_add_dependency(&leader->base, fence); - if (r) + if (r) { + dma_fence_put(fence); goto error_cleanup; + } } if (p->gang_size > 1) { @@ -1346,6 +1362,7 @@ static void amdgpu_cs_parser_fini(struct amdgpu_cs_parser *parser) { unsigned i; + amdgpu_sync_free(&parser->sync); for (i = 0; i < parser->num_post_deps; i++) { drm_syncobj_put(parser->post_deps[i].syncobj); kfree(parser->post_deps[i].chain); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index 1257745fb202384a2ecb1986fbf824dd7e3b9fe6..5bee3ff623442e925cdf3c775598d1fb2a998791 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -3038,6 +3038,18 @@ static int amdgpu_device_ip_suspend_phase2(struct amdgpu_device *adev) (adev->ip_blocks[i].version->type == AMD_IP_BLOCK_TYPE_SDMA)) continue; + /* Once swPSP provides the IMU, RLC FW binaries to TOS during cold-boot. + * These are in TMR, hence are expected to be reused by PSP-TOS to reload + * from this location and RLC Autoload automatically also gets loaded + * from here based on PMFW -> PSP message during re-init sequence. + * Therefore, the psp suspend & resume should be skipped to avoid destroy + * the TMR and reload FWs again for IMU enabled APU ASICs. + */ + if (amdgpu_in_reset(adev) && + (adev->flags & AMD_IS_APU) && adev->gfx.imu.funcs && + adev->ip_blocks[i].version->type == AMD_IP_BLOCK_TYPE_PSP) + continue; + /* XXX handle errors */ r = adev->ip_blocks[i].version->funcs->suspend(adev); /* XXX handle errors */ diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_mode.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_mode.h index 44c57f4a84c45c81e17eb1514f1afb9b0deeaa84..32fe05c810c6fcd3b89fe89ce7eba88f69c12aaf 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_mode.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_mode.h @@ -549,8 +549,8 @@ struct amdgpu_mst_connector { struct drm_dp_mst_topology_mgr mst_mgr; struct amdgpu_dm_dp_aux dm_dp_aux; - struct drm_dp_mst_port *port; - struct amdgpu_connector *mst_port; + struct drm_dp_mst_port *mst_output_port; + struct amdgpu_connector *mst_root; bool is_mst_connector; struct amdgpu_encoder *mst_encoder; }; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c index bac7976975bd3b3672f87ceb9e48bc0486d96fd7..dcd8c066bc1f5024814853a9f9689b3c8eef4132 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c @@ -391,8 +391,10 @@ int amdgpu_sync_push_to_job(struct amdgpu_sync *sync, struct amdgpu_job *job) dma_fence_get(f); r = drm_sched_job_add_dependency(&job->base, f); - if (r) + if (r) { + dma_fence_put(f); return r; + } } return 0; } diff --git a/drivers/gpu/drm/amd/amdgpu/imu_v11_0.c b/drivers/gpu/drm/amd/amdgpu/imu_v11_0.c index ed0d368149aae6a815df72df42d35c54076459a8..4ab90c7852c3ed3ee033828274997d0777c5cdd7 100644 --- a/drivers/gpu/drm/amd/amdgpu/imu_v11_0.c +++ b/drivers/gpu/drm/amd/amdgpu/imu_v11_0.c @@ -35,6 +35,7 @@ MODULE_FIRMWARE("amdgpu/gc_11_0_0_imu.bin"); MODULE_FIRMWARE("amdgpu/gc_11_0_1_imu.bin"); MODULE_FIRMWARE("amdgpu/gc_11_0_2_imu.bin"); MODULE_FIRMWARE("amdgpu/gc_11_0_3_imu.bin"); +MODULE_FIRMWARE("amdgpu/gc_11_0_4_imu.bin"); static int imu_v11_0_init_microcode(struct amdgpu_device *adev) { diff --git a/drivers/gpu/drm/amd/amdgpu/mes_v11_0.c b/drivers/gpu/drm/amd/amdgpu/mes_v11_0.c index bfa305079bfcf84d38da73a701ae3316fcbb9568..62cdd2113135fe80a5d706799d4a48daf520d00b 100644 --- a/drivers/gpu/drm/amd/amdgpu/mes_v11_0.c +++ b/drivers/gpu/drm/amd/amdgpu/mes_v11_0.c @@ -40,6 +40,8 @@ MODULE_FIRMWARE("amdgpu/gc_11_0_2_mes.bin"); MODULE_FIRMWARE("amdgpu/gc_11_0_2_mes1.bin"); MODULE_FIRMWARE("amdgpu/gc_11_0_3_mes.bin"); MODULE_FIRMWARE("amdgpu/gc_11_0_3_mes1.bin"); +MODULE_FIRMWARE("amdgpu/gc_11_0_4_mes.bin"); +MODULE_FIRMWARE("amdgpu/gc_11_0_4_mes1.bin"); static int mes_v11_0_hw_fini(void *handle); static int mes_v11_0_kiq_hw_init(struct amdgpu_device *adev); @@ -196,7 +198,6 @@ static int mes_v11_0_add_hw_queue(struct amdgpu_mes *mes, mes_add_queue_pkt.trap_handler_addr = input->tba_addr; mes_add_queue_pkt.tma_addr = input->tma_addr; mes_add_queue_pkt.is_kfd_process = input->is_kfd_process; - mes_add_queue_pkt.trap_en = 1; /* For KFD, gds_size is re-used for queue size (needed in MES for AQL queues) */ mes_add_queue_pkt.is_aql_queue = input->is_aql_queue; diff --git a/drivers/gpu/drm/amd/amdgpu/soc21.c b/drivers/gpu/drm/amd/amdgpu/soc21.c index 9c4a29d50f1cd79f68c565a107ad14d2df9a0c1f..e03cf7f766c58485f377653a139d16f91f8ada81 100644 --- a/drivers/gpu/drm/amd/amdgpu/soc21.c +++ b/drivers/gpu/drm/amd/amdgpu/soc21.c @@ -52,6 +52,7 @@ static const struct amdgpu_video_codec_info vcn_4_0_0_video_codecs_encode_array_ { {codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG4_AVC, 4096, 2304, 0)}, {codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_HEVC, 4096, 2304, 0)}, + {codec_info_build(AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_AV1, 8192, 4352, 0)}, }; static const struct amdgpu_video_codec_info vcn_4_0_0_video_codecs_encode_array_vcn1[] = diff --git a/drivers/gpu/drm/amd/amdgpu/vcn_v4_0.c b/drivers/gpu/drm/amd/amdgpu/vcn_v4_0.c index efb22d0975b34c46410627838d399108928afd60..22a41766a8c71cc7cbad317318bca108d82c1ba0 100644 --- a/drivers/gpu/drm/amd/amdgpu/vcn_v4_0.c +++ b/drivers/gpu/drm/amd/amdgpu/vcn_v4_0.c @@ -1710,7 +1710,7 @@ static int vcn_v4_0_dec_msg(struct amdgpu_cs_parser *p, struct amdgpu_job *job, create = ptr + addr + offset - start; - /* H246, HEVC and VP9 can run on any instance */ + /* H264, HEVC and VP9 can run on any instance */ if (create[0] == 0x7 || create[0] == 0x10 || create[0] == 0x11) continue; @@ -1724,7 +1724,29 @@ static int vcn_v4_0_dec_msg(struct amdgpu_cs_parser *p, struct amdgpu_job *job, return r; } -#define RADEON_VCN_ENGINE_TYPE_DECODE (0x00000003) +#define RADEON_VCN_ENGINE_TYPE_ENCODE (0x00000002) +#define RADEON_VCN_ENGINE_TYPE_DECODE (0x00000003) + +#define RADEON_VCN_ENGINE_INFO (0x30000001) +#define RADEON_VCN_ENGINE_INFO_MAX_OFFSET 16 + +#define RENCODE_ENCODE_STANDARD_AV1 2 +#define RENCODE_IB_PARAM_SESSION_INIT 0x00000003 +#define RENCODE_IB_PARAM_SESSION_INIT_MAX_OFFSET 64 + +/* return the offset in ib if id is found, -1 otherwise + * to speed up the searching we only search upto max_offset + */ +static int vcn_v4_0_enc_find_ib_param(struct amdgpu_ib *ib, uint32_t id, int max_offset) +{ + int i; + + for (i = 0; i < ib->length_dw && i < max_offset && ib->ptr[i] >= 8; i += ib->ptr[i]/4) { + if (ib->ptr[i + 1] == id) + return i; + } + return -1; +} static int vcn_v4_0_ring_patch_cs_in_place(struct amdgpu_cs_parser *p, struct amdgpu_job *job, @@ -1734,27 +1756,35 @@ static int vcn_v4_0_ring_patch_cs_in_place(struct amdgpu_cs_parser *p, struct amdgpu_vcn_decode_buffer *decode_buffer; uint64_t addr; uint32_t val; + int idx; /* The first instance can decode anything */ if (!ring->me) return 0; - /* unified queue ib header has 8 double words. */ - if (ib->length_dw < 8) - return 0; - - val = amdgpu_ib_get_value(ib, 6); //RADEON_VCN_ENGINE_TYPE - if (val != RADEON_VCN_ENGINE_TYPE_DECODE) - return 0; - - decode_buffer = (struct amdgpu_vcn_decode_buffer *)&ib->ptr[10]; - - if (!(decode_buffer->valid_buf_flag & 0x1)) + /* RADEON_VCN_ENGINE_INFO is at the top of ib block */ + idx = vcn_v4_0_enc_find_ib_param(ib, RADEON_VCN_ENGINE_INFO, + RADEON_VCN_ENGINE_INFO_MAX_OFFSET); + if (idx < 0) /* engine info is missing */ return 0; - addr = ((u64)decode_buffer->msg_buffer_address_hi) << 32 | - decode_buffer->msg_buffer_address_lo; - return vcn_v4_0_dec_msg(p, job, addr); + val = amdgpu_ib_get_value(ib, idx + 2); /* RADEON_VCN_ENGINE_TYPE */ + if (val == RADEON_VCN_ENGINE_TYPE_DECODE) { + decode_buffer = (struct amdgpu_vcn_decode_buffer *)&ib->ptr[idx + 6]; + + if (!(decode_buffer->valid_buf_flag & 0x1)) + return 0; + + addr = ((u64)decode_buffer->msg_buffer_address_hi) << 32 | + decode_buffer->msg_buffer_address_lo; + return vcn_v4_0_dec_msg(p, job, addr); + } else if (val == RADEON_VCN_ENGINE_TYPE_ENCODE) { + idx = vcn_v4_0_enc_find_ib_param(ib, RENCODE_IB_PARAM_SESSION_INIT, + RENCODE_IB_PARAM_SESSION_INIT_MAX_OFFSET); + if (idx >= 0 && ib->ptr[idx + 2] == RENCODE_ENCODE_STANDARD_AV1) + return vcn_v4_0_limit_sched(p, job); + } + return 0; } static const struct amdgpu_ring_funcs vcn_v4_0_unified_ring_vm_funcs = { diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c index 8e4b668faa359b3e5cf8461fa981afc431d426ca..e1d63826927a85c9cadfb21db3ad970a8e667273 100644 --- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c +++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c @@ -28,7 +28,6 @@ #include "dm_services_types.h" #include "dc.h" -#include "dc_link_dp.h" #include "link_enc_cfg.h" #include "dc/inc/core_types.h" #include "dal_asic_id.h" @@ -39,6 +38,9 @@ #include "dc/dc_edid_parser.h" #include "dc/dc_stat.h" #include "amdgpu_dm_trace.h" +#include "dpcd_defs.h" +#include "link/protocols/link_dpcd.h" +#include "link_service_types.h" #include "vid.h" #include "amdgpu.h" @@ -1225,10 +1227,25 @@ static void mmhub_read_system_context(struct amdgpu_device *adev, struct dc_phy_ pa_config->gart_config.page_table_end_addr = page_table_end.quad_part << 12; pa_config->gart_config.page_table_base_addr = page_table_base.quad_part; - pa_config->is_hvm_enabled = 0; + pa_config->is_hvm_enabled = adev->mode_info.gpu_vm_support; } +static void force_connector_state( + struct amdgpu_dm_connector *aconnector, + enum drm_connector_force force_state) +{ + struct drm_connector *connector = &aconnector->base; + + mutex_lock(&connector->dev->mode_config.mutex); + aconnector->base.force = force_state; + mutex_unlock(&connector->dev->mode_config.mutex); + + mutex_lock(&aconnector->hpd_lock); + drm_kms_helper_connector_hotplug_event(connector); + mutex_unlock(&aconnector->hpd_lock); +} + static void dm_handle_hpd_rx_offload_work(struct work_struct *work) { struct hpd_rx_irq_offload_work *offload_work; @@ -1237,6 +1254,9 @@ static void dm_handle_hpd_rx_offload_work(struct work_struct *work) struct amdgpu_device *adev; enum dc_connection_type new_connection_type = dc_connection_none; unsigned long flags; + union test_response test_response; + + memset(&test_response, 0, sizeof(test_response)); offload_work = container_of(work, struct hpd_rx_irq_offload_work, work); aconnector = offload_work->offload_wq->aconnector; @@ -1261,10 +1281,26 @@ static void dm_handle_hpd_rx_offload_work(struct work_struct *work) goto skip; mutex_lock(&adev->dm.dc_lock); - if (offload_work->data.bytes.device_service_irq.bits.AUTOMATED_TEST) + if (offload_work->data.bytes.device_service_irq.bits.AUTOMATED_TEST) { dc_link_dp_handle_automated_test(dc_link); + + if (aconnector->timing_changed) { + /* force connector disconnect and reconnect */ + force_connector_state(aconnector, DRM_FORCE_OFF); + msleep(100); + force_connector_state(aconnector, DRM_FORCE_UNSPECIFIED); + } + + test_response.bits.ACK = 1; + + core_link_write_dpcd( + dc_link, + DP_TEST_RESPONSE, + &test_response.raw, + sizeof(test_response)); + } else if ((dc_link->connector_signal != SIGNAL_TYPE_EDP) && - hpd_rx_irq_check_link_loss_status(dc_link, &offload_work->data) && + dc_link_check_link_loss_status(dc_link, &offload_work->data) && dc_link_dp_allow_hpd_rx_irq(dc_link)) { dc_link_dp_handle_link_loss(dc_link); spin_lock_irqsave(&offload_work->offload_wq->offload_lock, flags); @@ -2197,7 +2233,7 @@ static void s3_handle_mst(struct drm_device *dev, bool suspend) drm_for_each_connector_iter(connector, &iter) { aconnector = to_amdgpu_dm_connector(connector); if (aconnector->dc_link->type != dc_connection_mst_branch || - aconnector->mst_port) + aconnector->mst_root) continue; mgr = &aconnector->mst_mgr; @@ -2987,6 +3023,10 @@ void amdgpu_dm_update_connector_after_detect( aconnector->edid); } + aconnector->timing_requested = kzalloc(sizeof(struct dc_crtc_timing), GFP_KERNEL); + if (!aconnector->timing_requested) + dm_error("%s: failed to create aconnector->requested_timing\n", __func__); + drm_connector_update_edid_property(connector, aconnector->edid); amdgpu_dm_update_freesync_caps(connector, aconnector->edid); update_connector_ext_caps(aconnector); @@ -2998,6 +3038,8 @@ void amdgpu_dm_update_connector_after_detect( dc_sink_release(aconnector->dc_sink); aconnector->dc_sink = NULL; aconnector->edid = NULL; + kfree(aconnector->timing_requested); + aconnector->timing_requested = NULL; #ifdef CONFIG_DRM_AMD_DC_HDCP /* Set CP to DESIRED if it was ENABLED, so we can re-enable it again on hotplug */ if (connector->state->content_protection == DRM_MODE_CONTENT_PROTECTION_ENABLED) @@ -3042,6 +3084,8 @@ static void handle_hpd_irq_helper(struct amdgpu_dm_connector *aconnector) if (aconnector->fake_enable) aconnector->fake_enable = false; + aconnector->timing_changed = false; + if (!dc_link_detect_sink(aconnector->dc_link, &new_connection_type)) DRM_ERROR("KMS: Failed to detect connector\n"); @@ -5888,6 +5932,14 @@ create_stream_for_sink(struct amdgpu_dm_connector *aconnector, stream, &mode, &aconnector->base, con_state, old_stream, requested_bpc); + if (aconnector->timing_changed) { + DC_LOG_DEBUG("%s: overriding timing for automated test, bpc %d, changing to %d\n", + __func__, + stream->timing.display_color_depth, + aconnector->timing_requested->display_color_depth); + stream->timing = *aconnector->timing_requested; + } + #if defined(CONFIG_DRM_AMD_DC_DCN) /* SST DSC determination policy */ update_dsc_caps(aconnector, sink, stream, &dsc_caps); @@ -6580,11 +6632,11 @@ static int dm_encoder_helper_atomic_check(struct drm_encoder *encoder, int clock, bpp = 0; bool is_y420 = false; - if (!aconnector->port || !aconnector->dc_sink) + if (!aconnector->mst_output_port || !aconnector->dc_sink) return 0; - mst_port = aconnector->port; - mst_mgr = &aconnector->mst_port->mst_mgr; + mst_port = aconnector->mst_output_port; + mst_mgr = &aconnector->mst_root->mst_mgr; if (!crtc_state->connectors_changed && !crtc_state->mode_changed) return 0; @@ -6594,7 +6646,7 @@ static int dm_encoder_helper_atomic_check(struct drm_encoder *encoder, return PTR_ERR(mst_state); if (!mst_state->pbn_div) - mst_state->pbn_div = dm_mst_get_pbn_divider(aconnector->mst_port->dc_link); + mst_state->pbn_div = dm_mst_get_pbn_divider(aconnector->mst_root->dc_link); if (!state->duplicated) { int max_bpc = conn_state->max_requested_bpc; @@ -6640,7 +6692,7 @@ static int dm_update_mst_vcpi_slots_for_dsc(struct drm_atomic_state *state, aconnector = to_amdgpu_dm_connector(connector); - if (!aconnector->port) + if (!aconnector->mst_output_port) continue; if (!new_con_state || !new_con_state->crtc) @@ -6680,7 +6732,7 @@ static int dm_update_mst_vcpi_slots_for_dsc(struct drm_atomic_state *state, dm_conn_state->pbn = pbn; dm_conn_state->vcpi_slots = slot_num; - ret = drm_dp_mst_atomic_enable_dsc(state, aconnector->port, + ret = drm_dp_mst_atomic_enable_dsc(state, aconnector->mst_output_port, dm_conn_state->pbn, false); if (ret < 0) return ret; @@ -6688,7 +6740,7 @@ static int dm_update_mst_vcpi_slots_for_dsc(struct drm_atomic_state *state, continue; } - vcpi = drm_dp_mst_atomic_enable_dsc(state, aconnector->port, pbn, true); + vcpi = drm_dp_mst_atomic_enable_dsc(state, aconnector->mst_output_port, pbn, true); if (vcpi < 0) return vcpi; @@ -7102,7 +7154,7 @@ void amdgpu_dm_connector_init_helper(struct amdgpu_display_manager *dm, adev->mode_info.underscan_vborder_property, 0); - if (!aconnector->mst_port) + if (!aconnector->mst_root) drm_connector_attach_max_bpc_property(&aconnector->base, 8, 16); /* This defaults to the max in the range, but we want 8bpc for non-edp. */ @@ -7120,7 +7172,7 @@ void amdgpu_dm_connector_init_helper(struct amdgpu_display_manager *dm, connector_type == DRM_MODE_CONNECTOR_eDP) { drm_connector_attach_hdr_output_metadata_property(&aconnector->base); - if (!aconnector->mst_port) + if (!aconnector->mst_root) drm_connector_attach_vrr_capable_property(&aconnector->base); #ifdef CONFIG_DRM_AMD_DC_HDCP @@ -7604,6 +7656,7 @@ static void update_freesync_state_on_stream( new_crtc_state->vrr_infopacket = vrr_infopacket; new_stream->vrr_infopacket = vrr_infopacket; + new_stream->allow_freesync = mod_freesync_get_freesync_enabled(&vrr_params); if (new_crtc_state->freesync_vrr_info_changed) DRM_DEBUG_KMS("VRR packet update: crtc=%u enabled=%d state=%d", @@ -8818,22 +8871,15 @@ static void get_freesync_config_for_crtc( struct drm_display_mode *mode = &new_crtc_state->base.mode; int vrefresh = drm_mode_vrefresh(mode); bool fs_vid_mode = false; - bool drr_active = false; new_crtc_state->vrr_supported = new_con_state->freesync_capable && vrefresh >= aconnector->min_vfreq && vrefresh <= aconnector->max_vfreq; - drr_active = new_crtc_state->vrr_supported && - new_crtc_state->freesync_config.state != VRR_STATE_DISABLED && - new_crtc_state->freesync_config.state != VRR_STATE_INACTIVE && - new_crtc_state->freesync_config.state != VRR_STATE_UNSUPPORTED; - - if (drr_active) - new_crtc_state->stream->ignore_msa_timing_param = true; - if (new_crtc_state->vrr_supported) { + new_crtc_state->stream->ignore_msa_timing_param = true; fs_vid_mode = new_crtc_state->freesync_config.state == VRR_STATE_ACTIVE_FIXED; + config.min_refresh_in_uhz = aconnector->min_vfreq * 1000000; config.max_refresh_in_uhz = aconnector->max_vfreq * 1000000; config.vsif_supported = true; @@ -9032,6 +9078,13 @@ static int dm_update_crtc_state(struct amdgpu_display_manager *dm, if (!dm_old_crtc_state->stream) goto skip_modeset; + /* Unset freesync video if it was active before */ + if (dm_old_crtc_state->freesync_config.state == VRR_STATE_ACTIVE_FIXED) { + dm_new_crtc_state->freesync_config.state = VRR_STATE_INACTIVE; + dm_new_crtc_state->freesync_config.fixed_refresh_in_uhz = 0; + } + + /* Now check if we should set freesync video mode */ if (amdgpu_freesync_vid_mode && dm_new_crtc_state->stream && is_timing_unchanged_for_freesync(new_crtc_state, old_crtc_state)) { @@ -9593,7 +9646,7 @@ static int add_affected_mst_dsc_crtcs(struct drm_atomic_state *state, struct drm continue; aconnector = to_amdgpu_dm_connector(connector); - if (!aconnector->port || !aconnector->mst_port) + if (!aconnector->mst_output_port || !aconnector->mst_root) aconnector = NULL; else break; @@ -9602,7 +9655,7 @@ static int add_affected_mst_dsc_crtcs(struct drm_atomic_state *state, struct drm if (!aconnector) return 0; - return drm_dp_mst_add_affected_dsc_crtcs(state, &aconnector->mst_port->mst_mgr); + return drm_dp_mst_add_affected_dsc_crtcs(state, &aconnector->mst_root->mst_mgr); } #endif @@ -9648,6 +9701,8 @@ static int amdgpu_dm_atomic_check(struct drm_device *dev, bool lock_and_validation_needed = false; struct dm_crtc_state *dm_old_crtc_state, *dm_new_crtc_state; #if defined(CONFIG_DRM_AMD_DC_DCN) + struct drm_dp_mst_topology_mgr *mgr; + struct drm_dp_mst_topology_state *mst_state; struct dsc_mst_fairness_vars vars[MAX_PIPES]; #endif @@ -9896,6 +9951,28 @@ static int amdgpu_dm_atomic_check(struct drm_device *dev, lock_and_validation_needed = true; } +#if defined(CONFIG_DRM_AMD_DC_DCN) + /* set the slot info for each mst_state based on the link encoding format */ + for_each_new_mst_mgr_in_state(state, mgr, mst_state, i) { + struct amdgpu_dm_connector *aconnector; + struct drm_connector *connector; + struct drm_connector_list_iter iter; + u8 link_coding_cap; + + drm_connector_list_iter_begin(dev, &iter); + drm_for_each_connector_iter(connector, &iter) { + if (connector->index == mst_state->mgr->conn_base_id) { + aconnector = to_amdgpu_dm_connector(connector); + link_coding_cap = dc_link_dp_mst_decide_link_encoding_format(aconnector->dc_link); + drm_dp_mst_update_slots(mst_state, link_coding_cap); + + break; + } + } + drm_connector_list_iter_end(&iter); + } +#endif + /** * Streams and planes are reset when there are changes that affect * bandwidth. Anything that affects bandwidth needs to go through @@ -10165,11 +10242,15 @@ static bool parse_edid_cea(struct amdgpu_dm_connector *aconnector, struct amdgpu_hdmi_vsdb_info *vsdb_info) { struct amdgpu_device *adev = drm_to_adev(aconnector->base.dev); + bool ret; + mutex_lock(&adev->dm.dc_lock); if (adev->dm.dmub_srv) - return parse_edid_cea_dmub(&adev->dm, edid_ext, len, vsdb_info); + ret = parse_edid_cea_dmub(&adev->dm, edid_ext, len, vsdb_info); else - return parse_edid_cea_dmcu(&adev->dm, edid_ext, len, vsdb_info); + ret = parse_edid_cea_dmcu(&adev->dm, edid_ext, len, vsdb_info); + mutex_unlock(&adev->dm.dc_lock); + return ret; } static int parse_hdmi_amd_vsdb(struct amdgpu_dm_connector *aconnector, @@ -10445,6 +10526,7 @@ int amdgpu_dm_process_dmub_aux_transfer_sync( ret = p_notify->aux_reply.length; *operation_result = p_notify->result; out: + reinit_completion(&adev->dm.dmub_aux_transfer_done); mutex_unlock(&adev->dm.dpia_aux_lock); return ret; } @@ -10472,6 +10554,8 @@ int amdgpu_dm_process_dmub_set_config_sync( *operation_result = SET_CONFIG_UNKNOWN_ERROR; } + if (!is_cmd_complete) + reinit_completion(&adev->dm.dmub_aux_transfer_done); mutex_unlock(&adev->dm.dpia_aux_lock); return ret; } diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.h b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.h index abbbb3813c1e7fa75277f1791011ca8bcf9d7cdd..d89f2eaf9117c0bf06fe13cb859c8c28ba754f5d 100644 --- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.h +++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.h @@ -31,6 +31,7 @@ #include #include #include +#include "link_service_types.h" /* * This file contains the definition for amdgpu_display_manager @@ -604,8 +605,8 @@ struct amdgpu_dm_connector { /* DM only */ struct drm_dp_mst_topology_mgr mst_mgr; struct amdgpu_dm_dp_aux dm_dp_aux; - struct drm_dp_mst_port *port; - struct amdgpu_dm_connector *mst_port; + struct drm_dp_mst_port *mst_output_port; + struct amdgpu_dm_connector *mst_root; struct drm_dp_aux *dsc_aux; /* TODO see if we can merge with ddc_bus or make a dm_connector */ struct amdgpu_i2c_adapter *i2c; @@ -644,6 +645,10 @@ struct amdgpu_dm_connector { /* Record progress status of mst*/ uint8_t mst_status; + + /* Automated testing */ + bool timing_changed; + struct dc_crtc_timing *timing_requested; }; static inline void amdgpu_dm_set_mst_status(uint8_t *status, diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_crc.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_crc.c index 8873ecada27c47a1af2127b942ffe81d6b0f8844..27711743c22ce6d7cfe1576d3a6e2903cf798920 100644 --- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_crc.c +++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_crc.c @@ -344,7 +344,7 @@ int amdgpu_dm_crtc_set_crc_source(struct drm_crtc *crtc, const char *src_name) goto cleanup; } - aux = (aconn->port) ? &aconn->port->aux : &aconn->dm_dp_aux.aux; + aux = (aconn->mst_output_port) ? &aconn->mst_output_port->aux : &aconn->dm_dp_aux.aux; if (!aux) { DRM_DEBUG_DRIVER("No dp aux for amd connector\n"); diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_debugfs.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_debugfs.c index 704860e6ba84b68925b2687c930450fea4849f5b..e783082a4eef521c592f2790be2fabaeab6db70d 100644 --- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_debugfs.c +++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_debugfs.c @@ -34,8 +34,6 @@ #include "dmub/dmub_srv.h" #include "resource.h" #include "dsc.h" -#include "dc_link_dp.h" -#include "dc_link.h" #include "link_hwss.h" #include "dc/dc_dmub_srv.h" @@ -1193,7 +1191,7 @@ static int dp_dsc_fec_support_show(struct seq_file *m, void *data) break; } dpcd_caps = aconnector->dc_link->dpcd_caps; - if (aconnector->port) { + if (aconnector->mst_output_port) { /* aconnector sets dsc_aux during get_modes call * if MST connector has it means it can either * enable DSC on the sink device or on MST branch @@ -1280,7 +1278,7 @@ static ssize_t trigger_hotplug(struct file *f, const char __user *buf, mutex_lock(&aconnector->hpd_lock); /* Don't support for mst end device*/ - if (aconnector->mst_port) { + if (aconnector->mst_root) { mutex_unlock(&aconnector->hpd_lock); return -EINVAL; } @@ -2539,13 +2537,13 @@ static int dp_is_mst_connector_show(struct seq_file *m, void *unused) if (aconnector->mst_mgr.mst_state) { role = "root"; - } else if (aconnector->mst_port && - aconnector->mst_port->mst_mgr.mst_state) { + } else if (aconnector->mst_root && + aconnector->mst_root->mst_mgr.mst_state) { role = "end"; - mgr = &aconnector->mst_port->mst_mgr; - port = aconnector->port; + mgr = &aconnector->mst_root->mst_mgr; + port = aconnector->mst_output_port; drm_modeset_lock(&mgr->base.lock, NULL); if (port->pdt == DP_PEER_DEVICE_MST_BRANCHING && @@ -3392,12 +3390,12 @@ static int trigger_hpd_mst_set(void *data, u64 val) if (!aconnector->dc_link) continue; - if (!aconnector->mst_port) + if (!aconnector->mst_root) continue; link = aconnector->dc_link; dc_link_dp_receiver_power_ctrl(link, false); - drm_dp_mst_topology_mgr_set_mst(&aconnector->mst_port->mst_mgr, false); + drm_dp_mst_topology_mgr_set_mst(&aconnector->mst_root->mst_mgr, false); link->mst_stream_alloc_table.stream_count = 0; memset(link->mst_stream_alloc_table.stream_allocations, 0, sizeof(link->mst_stream_alloc_table.stream_allocations)); diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_helpers.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_helpers.c index 6994c9a1ed85885f6c0d531fedf5be1855bf4301..a50319fc42b11b90bb2ef1566af819384eb93a29 100644 --- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_helpers.c +++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_helpers.c @@ -38,6 +38,8 @@ #include "amdgpu_dm.h" #include "amdgpu_dm_irq.h" #include "amdgpu_dm_mst_types.h" +#include "dpcd_defs.h" +#include "dc/inc/core_types.h" #include "dm_helpers.h" #include "ddc_service_types.h" @@ -120,23 +122,50 @@ enum dc_edid_status dm_helpers_parse_edid_caps( } static void -fill_dc_mst_payload_table_from_drm(struct drm_dp_mst_topology_state *mst_state, - struct amdgpu_dm_connector *aconnector, +fill_dc_mst_payload_table_from_drm(struct dc_link *link, + bool enable, + struct drm_dp_mst_atomic_payload *target_payload, struct dc_dp_mst_stream_allocation_table *table) { struct dc_dp_mst_stream_allocation_table new_table = { 0 }; struct dc_dp_mst_stream_allocation *sa; - struct drm_dp_mst_atomic_payload *payload; + struct link_mst_stream_allocation_table copy_of_link_table = + link->mst_stream_alloc_table; - /* Fill payload info*/ - list_for_each_entry(payload, &mst_state->payloads, next) { - if (payload->delete) - continue; + int i; + int current_hw_table_stream_cnt = copy_of_link_table.stream_count; + struct link_mst_stream_allocation *dc_alloc; - sa = &new_table.stream_allocations[new_table.stream_count]; - sa->slot_count = payload->time_slots; - sa->vcp_id = payload->vcpi; - new_table.stream_count++; + /* TODO: refactor to set link->mst_stream_alloc_table directly if possible.*/ + if (enable) { + dc_alloc = + ©_of_link_table.stream_allocations[current_hw_table_stream_cnt]; + dc_alloc->vcp_id = target_payload->vcpi; + dc_alloc->slot_count = target_payload->time_slots; + } else { + for (i = 0; i < copy_of_link_table.stream_count; i++) { + dc_alloc = + ©_of_link_table.stream_allocations[i]; + + if (dc_alloc->vcp_id == target_payload->vcpi) { + dc_alloc->vcp_id = 0; + dc_alloc->slot_count = 0; + break; + } + } + ASSERT(i != copy_of_link_table.stream_count); + } + + /* Fill payload info*/ + for (i = 0; i < MAX_CONTROLLER_NUM; i++) { + dc_alloc = + ©_of_link_table.stream_allocations[i]; + if (dc_alloc->vcp_id > 0 && dc_alloc->slot_count > 0) { + sa = &new_table.stream_allocations[new_table.stream_count]; + sa->slot_count = dc_alloc->slot_count; + sa->vcp_id = dc_alloc->vcp_id; + new_table.stream_count++; + } } /* Overwrite the old table */ @@ -168,14 +197,14 @@ bool dm_helpers_dp_mst_write_payload_allocation_table( * that blocks before commit guaranteeing that the state * is not gonna be swapped while still in use in commit tail */ - if (!aconnector || !aconnector->mst_port) + if (!aconnector || !aconnector->mst_root) return false; - mst_mgr = &aconnector->mst_port->mst_mgr; + mst_mgr = &aconnector->mst_root->mst_mgr; mst_state = to_drm_dp_mst_topology_state(mst_mgr->base.state); /* It's OK for this to fail */ - payload = drm_atomic_get_mst_payload_state(mst_state, aconnector->port); + payload = drm_atomic_get_mst_payload_state(mst_state, aconnector->mst_output_port); if (enable) drm_dp_add_payload_part1(mst_mgr, mst_state, payload); else @@ -185,7 +214,7 @@ bool dm_helpers_dp_mst_write_payload_allocation_table( * AUX message. The sequence is slot 1-63 allocated sequence for each * stream. AMD ASIC stream slot allocation should follow the same * sequence. copy DRM MST allocation to dc */ - fill_dc_mst_payload_table_from_drm(mst_state, aconnector, proposed_table); + fill_dc_mst_payload_table_from_drm(stream->link, enable, payload, proposed_table); return true; } @@ -220,10 +249,10 @@ enum act_return_status dm_helpers_dp_mst_poll_for_allocation_change_trigger( aconnector = (struct amdgpu_dm_connector *)stream->dm_stream_context; - if (!aconnector || !aconnector->mst_port) + if (!aconnector || !aconnector->mst_root) return ACT_FAILED; - mst_mgr = &aconnector->mst_port->mst_mgr; + mst_mgr = &aconnector->mst_root->mst_mgr; if (!mst_mgr->mst_state) return ACT_FAILED; @@ -247,22 +276,27 @@ bool dm_helpers_dp_mst_send_payload_allocation( struct drm_dp_mst_atomic_payload *payload; enum mst_progress_status set_flag = MST_ALLOCATE_NEW_PAYLOAD; enum mst_progress_status clr_flag = MST_CLEAR_ALLOCATED_PAYLOAD; + int ret = 0; aconnector = (struct amdgpu_dm_connector *)stream->dm_stream_context; - if (!aconnector || !aconnector->mst_port) + if (!aconnector || !aconnector->mst_root) return false; - mst_mgr = &aconnector->mst_port->mst_mgr; + mst_mgr = &aconnector->mst_root->mst_mgr; mst_state = to_drm_dp_mst_topology_state(mst_mgr->base.state); - payload = drm_atomic_get_mst_payload_state(mst_state, aconnector->port); + payload = drm_atomic_get_mst_payload_state(mst_state, aconnector->mst_output_port); + if (!enable) { set_flag = MST_CLEAR_ALLOCATED_PAYLOAD; clr_flag = MST_ALLOCATE_NEW_PAYLOAD; } - if (enable && drm_dp_add_payload_part2(mst_mgr, mst_state->base.state, payload)) { + if (enable) + ret = drm_dp_add_payload_part2(mst_mgr, mst_state->base.state, payload); + + if (ret) { amdgpu_dm_set_mst_status(&aconnector->mst_status, set_flag, false); } else { @@ -683,7 +717,7 @@ bool dm_helpers_dp_write_dsc_enable( aconnector->dsc_aux, stream, enable_dsc); #endif - port = aconnector->port; + port = aconnector->mst_output_port; if (enable) { if (port->passthrough_aux) { @@ -960,6 +994,128 @@ void dm_helpers_mst_enable_stream_features(const struct dc_stream_state *stream) sizeof(new_downspread)); } +bool dm_helpers_dp_handle_test_pattern_request( + struct dc_context *ctx, + const struct dc_link *link, + union link_test_pattern dpcd_test_pattern, + union test_misc dpcd_test_params) +{ + enum dp_test_pattern test_pattern; + enum dp_test_pattern_color_space test_pattern_color_space = + DP_TEST_PATTERN_COLOR_SPACE_UNDEFINED; + enum dc_color_depth requestColorDepth = COLOR_DEPTH_UNDEFINED; + enum dc_pixel_encoding requestPixelEncoding = PIXEL_ENCODING_UNDEFINED; + struct pipe_ctx *pipes = link->dc->current_state->res_ctx.pipe_ctx; + struct pipe_ctx *pipe_ctx = NULL; + struct amdgpu_dm_connector *aconnector = link->priv; + int i; + + for (i = 0; i < MAX_PIPES; i++) { + if (pipes[i].stream == NULL) + continue; + + if (pipes[i].stream->link == link && !pipes[i].top_pipe && + !pipes[i].prev_odm_pipe) { + pipe_ctx = &pipes[i]; + break; + } + } + + if (pipe_ctx == NULL) + return false; + + switch (dpcd_test_pattern.bits.PATTERN) { + case LINK_TEST_PATTERN_COLOR_RAMP: + test_pattern = DP_TEST_PATTERN_COLOR_RAMP; + break; + case LINK_TEST_PATTERN_VERTICAL_BARS: + test_pattern = DP_TEST_PATTERN_VERTICAL_BARS; + break; /* black and white */ + case LINK_TEST_PATTERN_COLOR_SQUARES: + test_pattern = (dpcd_test_params.bits.DYN_RANGE == + TEST_DYN_RANGE_VESA ? + DP_TEST_PATTERN_COLOR_SQUARES : + DP_TEST_PATTERN_COLOR_SQUARES_CEA); + break; + default: + test_pattern = DP_TEST_PATTERN_VIDEO_MODE; + break; + } + + if (dpcd_test_params.bits.CLR_FORMAT == 0) + test_pattern_color_space = DP_TEST_PATTERN_COLOR_SPACE_RGB; + else + test_pattern_color_space = dpcd_test_params.bits.YCBCR_COEFS ? + DP_TEST_PATTERN_COLOR_SPACE_YCBCR709 : + DP_TEST_PATTERN_COLOR_SPACE_YCBCR601; + + switch (dpcd_test_params.bits.BPC) { + case 0: // 6 bits + requestColorDepth = COLOR_DEPTH_666; + break; + case 1: // 8 bits + requestColorDepth = COLOR_DEPTH_888; + break; + case 2: // 10 bits + requestColorDepth = COLOR_DEPTH_101010; + break; + case 3: // 12 bits + requestColorDepth = COLOR_DEPTH_121212; + break; + default: + break; + } + + switch (dpcd_test_params.bits.CLR_FORMAT) { + case 0: + requestPixelEncoding = PIXEL_ENCODING_RGB; + break; + case 1: + requestPixelEncoding = PIXEL_ENCODING_YCBCR422; + break; + case 2: + requestPixelEncoding = PIXEL_ENCODING_YCBCR444; + break; + default: + requestPixelEncoding = PIXEL_ENCODING_RGB; + break; + } + + if ((requestColorDepth != COLOR_DEPTH_UNDEFINED + && pipe_ctx->stream->timing.display_color_depth != requestColorDepth) + || (requestPixelEncoding != PIXEL_ENCODING_UNDEFINED + && pipe_ctx->stream->timing.pixel_encoding != requestPixelEncoding)) { + DC_LOG_DEBUG("%s: original bpc %d pix encoding %d, changing to %d %d\n", + __func__, + pipe_ctx->stream->timing.display_color_depth, + pipe_ctx->stream->timing.pixel_encoding, + requestColorDepth, + requestPixelEncoding); + pipe_ctx->stream->timing.display_color_depth = requestColorDepth; + pipe_ctx->stream->timing.pixel_encoding = requestPixelEncoding; + + dp_update_dsc_config(pipe_ctx); + + aconnector->timing_changed = true; + /* store current timing */ + if (aconnector->timing_requested) + *aconnector->timing_requested = pipe_ctx->stream->timing; + else + DC_LOG_ERROR("%s: timing storage failed\n", __func__); + + } + + dc_link_dp_set_test_pattern( + (struct dc_link *) link, + test_pattern, + test_pattern_color_space, + NULL, + NULL, + 0); + + return false; +} + void dm_set_phyd32clk(struct dc_context *ctx, int freq_khz) { // TODO diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_mst_types.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_mst_types.c index 5fa9bab95038461a36f140b792b7427d1071007f..0bff2cc20b02e19f52639ceb37e2e284b3896145 100644 --- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_mst_types.c +++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_mst_types.c @@ -39,7 +39,6 @@ #include "dc.h" #include "dm_helpers.h" -#include "dc_link_dp.h" #include "ddc_service_types.h" #include "dpcd_defs.h" @@ -134,7 +133,7 @@ dm_dp_mst_connector_destroy(struct drm_connector *connector) kfree(aconnector->edid); drm_connector_cleanup(connector); - drm_dp_mst_put_port_malloc(aconnector->port); + drm_dp_mst_put_port_malloc(aconnector->mst_output_port); kfree(aconnector); } @@ -146,7 +145,7 @@ amdgpu_dm_mst_connector_late_register(struct drm_connector *connector) int r; r = drm_dp_mst_connector_late_register(connector, - amdgpu_dm_connector->port); + amdgpu_dm_connector->mst_output_port); if (r < 0) return r; @@ -162,8 +161,8 @@ amdgpu_dm_mst_connector_early_unregister(struct drm_connector *connector) { struct amdgpu_dm_connector *aconnector = to_amdgpu_dm_connector(connector); - struct drm_dp_mst_port *port = aconnector->port; - struct amdgpu_dm_connector *root = aconnector->mst_port; + struct drm_dp_mst_port *port = aconnector->mst_output_port; + struct amdgpu_dm_connector *root = aconnector->mst_root; struct dc_link *dc_link = aconnector->dc_link; struct dc_sink *dc_sink = aconnector->dc_sink; @@ -213,7 +212,7 @@ bool needs_dsc_aux_workaround(struct dc_link *link) static bool validate_dsc_caps_on_connector(struct amdgpu_dm_connector *aconnector) { struct dc_sink *dc_sink = aconnector->dc_sink; - struct drm_dp_mst_port *port = aconnector->port; + struct drm_dp_mst_port *port = aconnector->mst_output_port; u8 dsc_caps[16] = { 0 }; u8 dsc_branch_dec_caps_raw[3] = { 0 }; // DSC branch decoder caps 0xA0 ~ 0xA2 u8 *dsc_branch_dec_caps = NULL; @@ -231,7 +230,7 @@ static bool validate_dsc_caps_on_connector(struct amdgpu_dm_connector *aconnecto */ if (!aconnector->dsc_aux && !port->parent->port_parent && needs_dsc_aux_workaround(aconnector->dc_link)) - aconnector->dsc_aux = &aconnector->mst_port->dm_dp_aux.aux; + aconnector->dsc_aux = &aconnector->mst_root->dm_dp_aux.aux; if (!aconnector->dsc_aux) return false; @@ -281,7 +280,7 @@ static int dm_dp_mst_get_modes(struct drm_connector *connector) if (!aconnector->edid) { struct edid *edid; - edid = drm_dp_mst_get_edid(connector, &aconnector->mst_port->mst_mgr, aconnector->port); + edid = drm_dp_mst_get_edid(connector, &aconnector->mst_root->mst_mgr, aconnector->mst_output_port); if (!edid) { amdgpu_dm_set_mst_status(&aconnector->mst_status, @@ -410,15 +409,15 @@ dm_dp_mst_detect(struct drm_connector *connector, struct drm_modeset_acquire_ctx *ctx, bool force) { struct amdgpu_dm_connector *aconnector = to_amdgpu_dm_connector(connector); - struct amdgpu_dm_connector *master = aconnector->mst_port; - struct drm_dp_mst_port *port = aconnector->port; + struct amdgpu_dm_connector *master = aconnector->mst_root; + struct drm_dp_mst_port *port = aconnector->mst_output_port; int connection_status; if (drm_connector_is_unregistered(connector)) return connector_status_disconnected; connection_status = drm_dp_mst_detect_port(connector, ctx, &master->mst_mgr, - aconnector->port); + aconnector->mst_output_port); if (port->pdt != DP_PEER_DEVICE_NONE && !port->dpcd_rev) { uint8_t dpcd_rev; @@ -475,8 +474,8 @@ static int dm_dp_mst_atomic_check(struct drm_connector *connector, struct drm_atomic_state *state) { struct amdgpu_dm_connector *aconnector = to_amdgpu_dm_connector(connector); - struct drm_dp_mst_topology_mgr *mst_mgr = &aconnector->mst_port->mst_mgr; - struct drm_dp_mst_port *mst_port = aconnector->port; + struct drm_dp_mst_topology_mgr *mst_mgr = &aconnector->mst_root->mst_mgr; + struct drm_dp_mst_port *mst_port = aconnector->mst_output_port; return drm_dp_atomic_release_time_slots(state, mst_mgr, mst_port); } @@ -538,8 +537,8 @@ dm_dp_add_mst_connector(struct drm_dp_mst_topology_mgr *mgr, return NULL; connector = &aconnector->base; - aconnector->port = port; - aconnector->mst_port = master; + aconnector->mst_output_port = port; + aconnector->mst_root = master; amdgpu_dm_set_mst_status(&aconnector->mst_status, MST_PROBE, true); @@ -927,11 +926,6 @@ static int compute_mst_dsc_configs_for_link(struct drm_atomic_state *state, if (IS_ERR(mst_state)) return PTR_ERR(mst_state); - mst_state->pbn_div = dm_mst_get_pbn_divider(dc_link); -#if defined(CONFIG_DRM_AMD_DC_DCN) - drm_dp_mst_update_slots(mst_state, dc_link_dp_mst_decide_link_encoding_format(dc_link)); -#endif - /* Set up params */ for (i = 0; i < dc_state->stream_count; i++) { struct dc_dsc_policy dsc_policy = {0}; @@ -945,7 +939,7 @@ static int compute_mst_dsc_configs_for_link(struct drm_atomic_state *state, if (!aconnector) continue; - if (!aconnector->port) + if (!aconnector->mst_output_port) continue; stream->timing.flags.DSC = 0; @@ -953,7 +947,7 @@ static int compute_mst_dsc_configs_for_link(struct drm_atomic_state *state, params[count].timing = &stream->timing; params[count].sink = stream->sink; params[count].aconnector = aconnector; - params[count].port = aconnector->port; + params[count].port = aconnector->mst_output_port; params[count].clock_force_enable = aconnector->dsc_settings.dsc_force_enable; if (params[count].clock_force_enable == DSC_CLK_FORCE_ENABLE) debugfs_overwrite = true; @@ -1162,7 +1156,7 @@ int compute_mst_dsc_configs_for_state(struct drm_atomic_state *state, aconnector = (struct amdgpu_dm_connector *)stream->dm_stream_context; - if (!aconnector || !aconnector->dc_sink || !aconnector->port) + if (!aconnector || !aconnector->dc_sink || !aconnector->mst_output_port) continue; if (!aconnector->dc_sink->dsc_caps.dsc_dec_caps.is_dsc_supported) @@ -1177,7 +1171,7 @@ int compute_mst_dsc_configs_for_state(struct drm_atomic_state *state, if (!is_dsc_need_re_compute(state, dc_state, stream->link)) continue; - mst_mgr = aconnector->port->mgr; + mst_mgr = aconnector->mst_output_port->mgr; ret = compute_mst_dsc_configs_for_link(state, dc_state, stream->link, vars, mst_mgr, &link_vars_start_index); if (ret != 0) @@ -1223,7 +1217,7 @@ static int pre_compute_mst_dsc_configs_for_state(struct drm_atomic_state *state, aconnector = (struct amdgpu_dm_connector *)stream->dm_stream_context; - if (!aconnector || !aconnector->dc_sink || !aconnector->port) + if (!aconnector || !aconnector->dc_sink || !aconnector->mst_output_port) continue; if (!aconnector->dc_sink->dsc_caps.dsc_dec_caps.is_dsc_supported) @@ -1235,7 +1229,7 @@ static int pre_compute_mst_dsc_configs_for_state(struct drm_atomic_state *state, if (!is_dsc_need_re_compute(state, dc_state, stream->link)) continue; - mst_mgr = aconnector->port->mgr; + mst_mgr = aconnector->mst_output_port->mgr; ret = compute_mst_dsc_configs_for_link(state, dc_state, stream->link, vars, mst_mgr, &link_vars_start_index); if (ret != 0) @@ -1450,8 +1444,8 @@ enum dc_status dm_dp_mst_is_port_support_mode( * with DSC enabled. */ if (is_dsc_common_config_possible(stream, &bw_range) && - aconnector->port->passthrough_aux) { - mst_mgr = aconnector->port->mgr; + aconnector->mst_output_port->passthrough_aux) { + mst_mgr = aconnector->mst_output_port->mgr; mutex_lock(&mst_mgr->lock); cur_link_settings = stream->link->verified_link_cap; @@ -1459,7 +1453,7 @@ enum dc_status dm_dp_mst_is_port_support_mode( upper_link_bw_in_kbps = dc_link_bandwidth_kbps(aconnector->dc_link, &cur_link_settings ); - down_link_bw_in_kbps = kbps_from_pbn(aconnector->port->full_pbn); + down_link_bw_in_kbps = kbps_from_pbn(aconnector->mst_output_port->full_pbn); /* pick the bottleneck */ end_to_end_bw_in_kbps = min(upper_link_bw_in_kbps, @@ -1483,7 +1477,7 @@ enum dc_status dm_dp_mst_is_port_support_mode( bpp = convert_dc_color_depth_into_bpc(stream->timing.display_color_depth) * 3; pbn = drm_dp_calc_pbn_mode(stream->timing.pix_clk_100hz / 10, bpp, false); - if (pbn > aconnector->port->full_pbn) + if (pbn > aconnector->mst_output_port->full_pbn) return DC_FAIL_BANDWIDTH_VALIDATE; #if defined(CONFIG_DRM_AMD_DC_DCN) } diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_psr.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_psr.c index 872d06fe14364ad1baa9fc706511e980b7ed9874..d647f68fd5630e9609c27dabe2890f6776ef821a 100644 --- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_psr.c +++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_psr.c @@ -122,7 +122,7 @@ bool amdgpu_dm_link_setup_psr(struct dc_stream_state *stream) psr_config.allow_multi_disp_optimizations = (amdgpu_dc_feature_mask & DC_PSR_ALLOW_MULTI_DISP_OPT); - if (!psr_su_set_y_granularity(dc, link, stream, &psr_config)) + if (!psr_su_set_dsc_slice_height(dc, link, stream, &psr_config)) return false; ret = dc_link_setup_psr(link, stream, &psr_config, &psr_context); diff --git a/drivers/gpu/drm/amd/display/dc/Makefile b/drivers/gpu/drm/amd/display/dc/Makefile index 98c508313350ac6ec80e2c843428eee3c88904d2..fec32e20f8548c23ee38ad19705ccacd8abe8da9 100644 --- a/drivers/gpu/drm/amd/display/dc/Makefile +++ b/drivers/gpu/drm/amd/display/dc/Makefile @@ -65,8 +65,7 @@ AMD_DC = $(addsuffix /Makefile, $(addprefix $(FULL_AMD_DISPLAY_PATH)/dc/,$(DC_LI include $(AMD_DC) DISPLAY_CORE = dc.o dc_stat.o dc_link.o dc_resource.o dc_hw_sequencer.o dc_sink.o \ -dc_surface.o dc_link_dp.o dc_debug.o dc_stream.o \ -dc_link_enc_cfg.o +dc_surface.o dc_debug.o dc_stream.o dc_link_enc_cfg.o DISPLAY_CORE += dc_vm_helper.o diff --git a/drivers/gpu/drm/amd/display/dc/clk_mgr/clk_mgr.c b/drivers/gpu/drm/amd/display/dc/clk_mgr/clk_mgr.c index f276abb63bcd7ce4ea2a0e3d7e6beea1581b9d98..69691daf4dbbd12f82b54649c2489335fd55a506 100644 --- a/drivers/gpu/drm/amd/display/dc/clk_mgr/clk_mgr.c +++ b/drivers/gpu/drm/amd/display/dc/clk_mgr/clk_mgr.c @@ -29,6 +29,7 @@ #include "dc_types.h" #include "dccg.h" #include "clk_mgr_internal.h" +#include "link.h" #include "dce100/dce_clk_mgr.h" #include "dce110/dce110_clk_mgr.h" diff --git a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn31/dcn31_clk_mgr.c b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn31/dcn31_clk_mgr.c index 1c0569b1dc8f0c8df9076032b39a89e862f31e4d..f9e2e0c3095e7c652fae5c70823a8531a15218e3 100644 --- a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn31/dcn31_clk_mgr.c +++ b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn31/dcn31_clk_mgr.c @@ -47,6 +47,7 @@ #include "dcn30/dcn30_clk_mgr.h" #include "dc_dmub_srv.h" +#include "link.h" #include "logger_types.h" #undef DC_LOGGER diff --git a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn314/dcn314_clk_mgr.c b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn314/dcn314_clk_mgr.c index 20a06c04e4a1d63f9d710c8f424a67956b154aed..89df7244b272863b97151172f853fce66f7e06c1 100644 --- a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn314/dcn314_clk_mgr.c +++ b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn314/dcn314_clk_mgr.c @@ -48,7 +48,7 @@ #include "dcn31/dcn31_clk_mgr.h" #include "dc_dmub_srv.h" -#include "dc_link_dp.h" +#include "link.h" #include "dcn314_smu.h" diff --git a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn315/dcn315_clk_mgr.c b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn315/dcn315_clk_mgr.c index 07edd9777edfb11eded1c711346ae793ed599b24..8c368bcc8e7e7a543b28507f5bed01a2dc862c26 100644 --- a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn315/dcn315_clk_mgr.c +++ b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn315/dcn315_clk_mgr.c @@ -46,7 +46,7 @@ #define DC_LOGGER \ clk_mgr->base.base.ctx->logger -#include "dc_link_dp.h" +#include "link.h" #define TO_CLK_MGR_DCN315(clk_mgr)\ container_of(clk_mgr, struct clk_mgr_dcn315, base) @@ -87,6 +87,16 @@ static int dcn315_get_active_display_cnt_wa( return display_count; } +bool should_disable_otg(struct pipe_ctx *pipe) +{ + bool ret = true; + + if (pipe->stream->link->link_enc && pipe->stream->link->link_enc->funcs->is_dig_enabled && + pipe->stream->link->link_enc->funcs->is_dig_enabled(pipe->stream->link->link_enc)) + ret = false; + return ret; +} + static void dcn315_disable_otg_wa(struct clk_mgr *clk_mgr_base, struct dc_state *context, bool disable) { struct dc *dc = clk_mgr_base->ctx->dc; @@ -98,12 +108,16 @@ static void dcn315_disable_otg_wa(struct clk_mgr *clk_mgr_base, struct dc_state if (pipe->top_pipe || pipe->prev_odm_pipe) continue; if (pipe->stream && (pipe->stream->dpms_off || pipe->plane_state == NULL || - dc_is_virtual_signal(pipe->stream->signal))) { - if (disable) { - pipe->stream_res.tg->funcs->immediate_disable_crtc(pipe->stream_res.tg); - reset_sync_context_for_pipe(dc, context, i); - } else - pipe->stream_res.tg->funcs->enable_crtc(pipe->stream_res.tg); + dc_is_virtual_signal(pipe->stream->signal))) { + + /* This w/a should not trigger when we have a dig active */ + if (should_disable_otg(pipe)) { + if (disable) { + pipe->stream_res.tg->funcs->immediate_disable_crtc(pipe->stream_res.tg); + reset_sync_context_for_pipe(dc, context, i); + } else + pipe->stream_res.tg->funcs->enable_crtc(pipe->stream_res.tg); + } } } } diff --git a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn316/dcn316_clk_mgr.c b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn316/dcn316_clk_mgr.c index 3edc81e2d417ad99bdd23539d69c415418815458..93db4dbee713e5f1de73571777b37053d1a8f386 100644 --- a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn316/dcn316_clk_mgr.c +++ b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn316/dcn316_clk_mgr.c @@ -39,7 +39,7 @@ #include "dcn316_smu.h" #include "dm_helpers.h" #include "dc_dmub_srv.h" -#include "dc_link_dp.h" +#include "link.h" // DCN316 this is CLK1 instance #define MAX_INSTANCE 7 diff --git a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn32/dcn32_clk_mgr.c b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn32/dcn32_clk_mgr.c index 352c977d1495a39f821725683521f53e103e51de..61768bf726f8cb91e2b26c660899622fe9abe9e1 100644 --- a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn32/dcn32_clk_mgr.c +++ b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn32/dcn32_clk_mgr.c @@ -33,7 +33,6 @@ #include "reg_helper.h" #include "core_types.h" #include "dm_helpers.h" -#include "dc_link_dp.h" #include "link.h" #include "atomfirmware.h" diff --git a/drivers/gpu/drm/amd/display/dc/core/dc.c b/drivers/gpu/drm/amd/display/dc/core/dc.c index 53e586fc150102bb0613e52d83ddebc03e0603fa..f11bc8d9d0c4035e3bec711e36cf1ebae1346cf8 100644 --- a/drivers/gpu/drm/amd/display/dc/core/dc.c +++ b/drivers/gpu/drm/amd/display/dc/core/dc.c @@ -58,7 +58,6 @@ #include "dm_helpers.h" #include "mem_input.h" -#include "dc_link_dp.h" #include "dc_dmub_srv.h" #include "dsc.h" @@ -1659,7 +1658,7 @@ bool dc_validate_boot_timing(const struct dc *dc, return false; } - if (is_edp_ilr_optimization_required(link, crtc_timing)) { + if (link_is_edp_ilr_optimization_required(link, crtc_timing)) { DC_LOG_EVENT_LINK_TRAINING("Seamless boot disabled to optimize eDP link rate\n"); return false; } @@ -2960,6 +2959,9 @@ static void copy_stream_update_to_stream(struct dc *dc, if (update->vsp_infopacket) stream->vsp_infopacket = *update->vsp_infopacket; + if (update->adaptive_sync_infopacket) + stream->adaptive_sync_infopacket = *update->adaptive_sync_infopacket; + if (update->dither_option) stream->dither_option = *update->dither_option; @@ -3165,6 +3167,7 @@ static void commit_planes_do_stream_update(struct dc *dc, stream_update->vsc_infopacket || stream_update->vsp_infopacket || stream_update->hfvsif_infopacket || + stream_update->adaptive_sync_infopacket || stream_update->vtem_infopacket) { resource_build_info_frame(pipe_ctx); dc->hwss.update_info_frame(pipe_ctx); diff --git a/drivers/gpu/drm/amd/display/dc/core/dc_link.c b/drivers/gpu/drm/amd/display/dc/core/dc_link.c index d9e490eca10f41b203c77a41c16d572fd9158486..6475664baa8a2ec17412ae0eeaf63c3126316027 100644 --- a/drivers/gpu/drm/amd/display/dc/core/dc_link.c +++ b/drivers/gpu/drm/amd/display/dc/core/dc_link.c @@ -32,9 +32,8 @@ #include "grph_object_id.h" #include "gpio_service_interface.h" #include "core_status.h" -#include "dc_link_dp.h" -#include "link/link_dp_dpia.h" -#include "link/link_ddc.h" +#include "link/protocols/link_dp_dpia.h" +#include "link/protocols/link_ddc.h" #include "link_hwss.h" #include "link.h" #include "opp.h" @@ -46,17 +45,20 @@ #include "fixed31_32.h" #include "dpcd_defs.h" #include "dmcu.h" +#include "dsc.h" +#include "opp.h" #include "hw/clk_mgr.h" #include "dce/dmub_psr.h" #include "dmub/dmub_srv.h" #include "inc/hw/panel_cntl.h" #include "inc/link_enc_cfg.h" -#include "link/link_dpcd.h" -#include "link/link_dp_trace.h" -#include "link/link_hpd.h" -#include "link/link_dp_training.h" -#include "link/link_dp_phy.h" -#include "link/link_dp_capability.h" +#include "link/protocols/link_dpcd.h" +#include "link/accessories/link_dp_trace.h" +#include "link/protocols/link_hpd.h" +#include "link/protocols/link_dp_training.h" +#include "link/protocols/link_dp_phy.h" +#include "link/protocols/link_dp_capability.h" +#include "link/protocols/link_edp_panel_control.h" #include "dc/dcn30/dcn30_vpg.h" @@ -107,17 +109,6 @@ static void dc_link_destruct(struct dc_link *link) dc_sink_release(link->remote_sinks[i]); } -bool dc_link_wait_for_t12(struct dc_link *link) -{ - if (link->connector_signal == SIGNAL_TYPE_EDP && link->dc->hwss.edp_wait_for_T12) { - link->dc->hwss.edp_wait_for_T12(link); - - return true; - } - - return false; -} - /** * dc_link_detect_sink() - Determine if there is a sink connected * @@ -1094,7 +1085,7 @@ static bool detect_link_and_local_sink(struct dc_link *link, (link->dpcd_sink_ext_caps.bits.oled == 1)) { dpcd_set_source_specific_data(link); msleep(post_oui_delay); - dc_link_set_default_brightness_aux(link); + set_default_brightness_aux(link); //TODO: use cached } @@ -2067,10 +2058,10 @@ static enum dc_status enable_link_dp(struct dc_state *state, if (link->dpcd_sink_ext_caps.bits.oled == 1 || link->dpcd_sink_ext_caps.bits.sdr_aux_backlight_control == 1 || link->dpcd_sink_ext_caps.bits.hdr_aux_backlight_control == 1) { - dc_link_set_default_brightness_aux(link); // TODO: use cached if known + set_default_brightness_aux(link); // TODO: use cached if known if (link->dpcd_sink_ext_caps.bits.oled == 1) msleep(bl_oled_enable_delay); - dc_link_backlight_enable_aux(link, true); + link_backlight_enable_aux(link, true); } return status; @@ -2740,22 +2731,6 @@ static void enable_link_lvds(struct pipe_ctx *pipe_ctx) } -bool dc_power_alpm_dpcd_enable(struct dc_link *link, bool enable) -{ - bool ret = false; - union dpcd_alpm_configuration alpm_config; - - if (link->psr_settings.psr_version == DC_PSR_VERSION_SU_1) { - memset(&alpm_config, 0, sizeof(alpm_config)); - - alpm_config.bits.ENABLE = (enable ? true : false); - ret = dm_helpers_dp_write_dpcd(link->ctx, link, - DP_RECEIVER_ALPM_CONFIG, &alpm_config.raw, - sizeof(alpm_config.raw)); - } - return ret; -} - /****************************enable_link***********************************/ static enum dc_status enable_link( struct dc_state *state, @@ -3035,436 +3010,6 @@ enum dc_status dc_link_validate_mode_timing( return DC_OK; } -static struct abm *get_abm_from_stream_res(const struct dc_link *link) -{ - int i; - struct dc *dc = NULL; - struct abm *abm = NULL; - - if (!link || !link->ctx) - return NULL; - - dc = link->ctx->dc; - - for (i = 0; i < MAX_PIPES; i++) { - struct pipe_ctx pipe_ctx = dc->current_state->res_ctx.pipe_ctx[i]; - struct dc_stream_state *stream = pipe_ctx.stream; - - if (stream && stream->link == link) { - abm = pipe_ctx.stream_res.abm; - break; - } - } - return abm; -} - -int dc_link_get_backlight_level(const struct dc_link *link) -{ - struct abm *abm = get_abm_from_stream_res(link); - struct panel_cntl *panel_cntl = link->panel_cntl; - struct dc *dc = link->ctx->dc; - struct dmcu *dmcu = dc->res_pool->dmcu; - bool fw_set_brightness = true; - - if (dmcu) - fw_set_brightness = dmcu->funcs->is_dmcu_initialized(dmcu); - - if (!fw_set_brightness && panel_cntl->funcs->get_current_backlight) - return panel_cntl->funcs->get_current_backlight(panel_cntl); - else if (abm != NULL && abm->funcs->get_current_backlight != NULL) - return (int) abm->funcs->get_current_backlight(abm); - else - return DC_ERROR_UNEXPECTED; -} - -int dc_link_get_target_backlight_pwm(const struct dc_link *link) -{ - struct abm *abm = get_abm_from_stream_res(link); - - if (abm == NULL || abm->funcs->get_target_backlight == NULL) - return DC_ERROR_UNEXPECTED; - - return (int) abm->funcs->get_target_backlight(abm); -} - -static struct pipe_ctx *get_pipe_from_link(const struct dc_link *link) -{ - int i; - struct dc *dc = link->ctx->dc; - struct pipe_ctx *pipe_ctx = NULL; - - for (i = 0; i < MAX_PIPES; i++) { - if (dc->current_state->res_ctx.pipe_ctx[i].stream) { - if (dc->current_state->res_ctx.pipe_ctx[i].stream->link == link) { - pipe_ctx = &dc->current_state->res_ctx.pipe_ctx[i]; - break; - } - } - } - - return pipe_ctx; -} - -bool dc_link_set_backlight_level(const struct dc_link *link, - uint32_t backlight_pwm_u16_16, - uint32_t frame_ramp) -{ - struct dc *dc = link->ctx->dc; - - DC_LOGGER_INIT(link->ctx->logger); - DC_LOG_BACKLIGHT("New Backlight level: %d (0x%X)\n", - backlight_pwm_u16_16, backlight_pwm_u16_16); - - if (dc_is_embedded_signal(link->connector_signal)) { - struct pipe_ctx *pipe_ctx = get_pipe_from_link(link); - - if (pipe_ctx) { - /* Disable brightness ramping when the display is blanked - * as it can hang the DMCU - */ - if (pipe_ctx->plane_state == NULL) - frame_ramp = 0; - } else { - return false; - } - - dc->hwss.set_backlight_level( - pipe_ctx, - backlight_pwm_u16_16, - frame_ramp); - } - return true; -} - -bool dc_link_set_psr_allow_active(struct dc_link *link, const bool *allow_active, - bool wait, bool force_static, const unsigned int *power_opts) -{ - struct dc *dc = link->ctx->dc; - struct dmcu *dmcu = dc->res_pool->dmcu; - struct dmub_psr *psr = dc->res_pool->psr; - unsigned int panel_inst; - - if (psr == NULL && force_static) - return false; - - if (!dc_get_edp_link_panel_inst(dc, link, &panel_inst)) - return false; - - if ((allow_active != NULL) && (*allow_active == true) && (link->type == dc_connection_none)) { - // Don't enter PSR if panel is not connected - return false; - } - - /* Set power optimization flag */ - if (power_opts && link->psr_settings.psr_power_opt != *power_opts) { - link->psr_settings.psr_power_opt = *power_opts; - - if (psr != NULL && link->psr_settings.psr_feature_enabled && psr->funcs->psr_set_power_opt) - psr->funcs->psr_set_power_opt(psr, link->psr_settings.psr_power_opt, panel_inst); - } - - if (psr != NULL && link->psr_settings.psr_feature_enabled && - force_static && psr->funcs->psr_force_static) - psr->funcs->psr_force_static(psr, panel_inst); - - /* Enable or Disable PSR */ - if (allow_active && link->psr_settings.psr_allow_active != *allow_active) { - link->psr_settings.psr_allow_active = *allow_active; - - if (!link->psr_settings.psr_allow_active) - dc_z10_restore(dc); - - if (psr != NULL && link->psr_settings.psr_feature_enabled) { - psr->funcs->psr_enable(psr, link->psr_settings.psr_allow_active, wait, panel_inst); - } else if ((dmcu != NULL && dmcu->funcs->is_dmcu_initialized(dmcu)) && - link->psr_settings.psr_feature_enabled) - dmcu->funcs->set_psr_enable(dmcu, link->psr_settings.psr_allow_active, wait); - else - return false; - } - - return true; -} - -bool dc_link_get_psr_state(const struct dc_link *link, enum dc_psr_state *state) -{ - struct dc *dc = link->ctx->dc; - struct dmcu *dmcu = dc->res_pool->dmcu; - struct dmub_psr *psr = dc->res_pool->psr; - unsigned int panel_inst; - - if (!dc_get_edp_link_panel_inst(dc, link, &panel_inst)) - return false; - - if (psr != NULL && link->psr_settings.psr_feature_enabled) - psr->funcs->psr_get_state(psr, state, panel_inst); - else if (dmcu != NULL && link->psr_settings.psr_feature_enabled) - dmcu->funcs->get_psr_state(dmcu, state); - - return true; -} - -static inline enum physical_phy_id -transmitter_to_phy_id(enum transmitter transmitter_value) -{ - switch (transmitter_value) { - case TRANSMITTER_UNIPHY_A: - return PHYLD_0; - case TRANSMITTER_UNIPHY_B: - return PHYLD_1; - case TRANSMITTER_UNIPHY_C: - return PHYLD_2; - case TRANSMITTER_UNIPHY_D: - return PHYLD_3; - case TRANSMITTER_UNIPHY_E: - return PHYLD_4; - case TRANSMITTER_UNIPHY_F: - return PHYLD_5; - case TRANSMITTER_NUTMEG_CRT: - return PHYLD_6; - case TRANSMITTER_TRAVIS_CRT: - return PHYLD_7; - case TRANSMITTER_TRAVIS_LCD: - return PHYLD_8; - case TRANSMITTER_UNIPHY_G: - return PHYLD_9; - case TRANSMITTER_COUNT: - return PHYLD_COUNT; - case TRANSMITTER_UNKNOWN: - return PHYLD_UNKNOWN; - default: - WARN_ONCE(1, "Unknown transmitter value %d\n", - transmitter_value); - return PHYLD_UNKNOWN; - } -} - -bool dc_link_setup_psr(struct dc_link *link, - const struct dc_stream_state *stream, struct psr_config *psr_config, - struct psr_context *psr_context) -{ - struct dc *dc; - struct dmcu *dmcu; - struct dmub_psr *psr; - int i; - unsigned int panel_inst; - /* updateSinkPsrDpcdConfig*/ - union dpcd_psr_configuration psr_configuration; - union dpcd_sink_active_vtotal_control_mode vtotal_control = {0}; - - psr_context->controllerId = CONTROLLER_ID_UNDEFINED; - - if (!link) - return false; - - dc = link->ctx->dc; - dmcu = dc->res_pool->dmcu; - psr = dc->res_pool->psr; - - if (!dmcu && !psr) - return false; - - if (!dc_get_edp_link_panel_inst(dc, link, &panel_inst)) - return false; - - - memset(&psr_configuration, 0, sizeof(psr_configuration)); - - psr_configuration.bits.ENABLE = 1; - psr_configuration.bits.CRC_VERIFICATION = 1; - psr_configuration.bits.FRAME_CAPTURE_INDICATION = - psr_config->psr_frame_capture_indication_req; - - /* Check for PSR v2*/ - if (link->psr_settings.psr_version == DC_PSR_VERSION_SU_1) { - /* For PSR v2 selective update. - * Indicates whether sink should start capturing - * immediately following active scan line, - * or starting with the 2nd active scan line. - */ - psr_configuration.bits.LINE_CAPTURE_INDICATION = 0; - /*For PSR v2, determines whether Sink should generate - * IRQ_HPD when CRC mismatch is detected. - */ - psr_configuration.bits.IRQ_HPD_WITH_CRC_ERROR = 1; - /* For PSR v2, set the bit when the Source device will - * be enabling PSR2 operation. - */ - psr_configuration.bits.ENABLE_PSR2 = 1; - /* For PSR v2, the Sink device must be able to receive - * SU region updates early in the frame time. - */ - psr_configuration.bits.EARLY_TRANSPORT_ENABLE = 1; - } - - dm_helpers_dp_write_dpcd( - link->ctx, - link, - 368, - &psr_configuration.raw, - sizeof(psr_configuration.raw)); - - if (link->psr_settings.psr_version == DC_PSR_VERSION_SU_1) { - dc_power_alpm_dpcd_enable(link, true); - psr_context->su_granularity_required = - psr_config->su_granularity_required; - psr_context->su_y_granularity = - psr_config->su_y_granularity; - psr_context->line_time_in_us = - psr_config->line_time_in_us; - - if (link->psr_settings.psr_vtotal_control_support) { - psr_context->rate_control_caps = psr_config->rate_control_caps; - vtotal_control.bits.ENABLE = true; - core_link_write_dpcd(link, DP_SINK_PSR_ACTIVE_VTOTAL_CONTROL_MODE, - &vtotal_control.raw, sizeof(vtotal_control.raw)); - } - } - - psr_context->channel = link->ddc->ddc_pin->hw_info.ddc_channel; - psr_context->transmitterId = link->link_enc->transmitter; - psr_context->engineId = link->link_enc->preferred_engine; - - for (i = 0; i < MAX_PIPES; i++) { - if (dc->current_state->res_ctx.pipe_ctx[i].stream - == stream) { - /* dmcu -1 for all controller id values, - * therefore +1 here - */ - psr_context->controllerId = - dc->current_state->res_ctx. - pipe_ctx[i].stream_res.tg->inst + 1; - break; - } - } - - /* Hardcoded for now. Can be Pcie or Uniphy (or Unknown)*/ - psr_context->phyType = PHY_TYPE_UNIPHY; - /*PhyId is associated with the transmitter id*/ - psr_context->smuPhyId = - transmitter_to_phy_id(link->link_enc->transmitter); - - psr_context->crtcTimingVerticalTotal = stream->timing.v_total; - psr_context->vsync_rate_hz = div64_u64(div64_u64((stream-> - timing.pix_clk_100hz * 100), - stream->timing.v_total), - stream->timing.h_total); - - psr_context->psrSupportedDisplayConfig = true; - psr_context->psrExitLinkTrainingRequired = - psr_config->psr_exit_link_training_required; - psr_context->sdpTransmitLineNumDeadline = - psr_config->psr_sdp_transmit_line_num_deadline; - psr_context->psrFrameCaptureIndicationReq = - psr_config->psr_frame_capture_indication_req; - - psr_context->skipPsrWaitForPllLock = 0; /* only = 1 in KV */ - - psr_context->numberOfControllers = - link->dc->res_pool->timing_generator_count; - - psr_context->rfb_update_auto_en = true; - - /* 2 frames before enter PSR. */ - psr_context->timehyst_frames = 2; - /* half a frame - * (units in 100 lines, i.e. a value of 1 represents 100 lines) - */ - psr_context->hyst_lines = stream->timing.v_total / 2 / 100; - psr_context->aux_repeats = 10; - - psr_context->psr_level.u32all = 0; - - /*skip power down the single pipe since it blocks the cstate*/ -#if defined(CONFIG_DRM_AMD_DC_DCN) - if (link->ctx->asic_id.chip_family >= FAMILY_RV) { - switch(link->ctx->asic_id.chip_family) { - case FAMILY_YELLOW_CARP: - case AMDGPU_FAMILY_GC_10_3_6: - case AMDGPU_FAMILY_GC_11_0_1: - if (dc->debug.disable_z10 || dc->debug.psr_skip_crtc_disable) - psr_context->psr_level.bits.SKIP_CRTC_DISABLE = true; - break; - default: - psr_context->psr_level.bits.SKIP_CRTC_DISABLE = true; - break; - } - } -#else - if (link->ctx->asic_id.chip_family >= FAMILY_RV) - psr_context->psr_level.bits.SKIP_CRTC_DISABLE = true; -#endif - - /* SMU will perform additional powerdown sequence. - * For unsupported ASICs, set psr_level flag to skip PSR - * static screen notification to SMU. - * (Always set for DAL2, did not check ASIC) - */ - psr_context->allow_smu_optimizations = psr_config->allow_smu_optimizations; - psr_context->allow_multi_disp_optimizations = psr_config->allow_multi_disp_optimizations; - - /* Complete PSR entry before aborting to prevent intermittent - * freezes on certain eDPs - */ - psr_context->psr_level.bits.DISABLE_PSR_ENTRY_ABORT = 1; - - /* enable ALPM */ - psr_context->psr_level.bits.DISABLE_ALPM = 0; - psr_context->psr_level.bits.ALPM_DEFAULT_PD_MODE = 1; - - /* Controls additional delay after remote frame capture before - * continuing power down, default = 0 - */ - psr_context->frame_delay = 0; - - if (psr) { - link->psr_settings.psr_feature_enabled = psr->funcs->psr_copy_settings(psr, - link, psr_context, panel_inst); - link->psr_settings.psr_power_opt = 0; - link->psr_settings.psr_allow_active = 0; - } - else - link->psr_settings.psr_feature_enabled = dmcu->funcs->setup_psr(dmcu, link, psr_context); - - /* psr_enabled == 0 indicates setup_psr did not succeed, but this - * should not happen since firmware should be running at this point - */ - if (link->psr_settings.psr_feature_enabled == 0) - ASSERT(0); - - return true; - -} - -void dc_link_get_psr_residency(const struct dc_link *link, uint32_t *residency) -{ - struct dc *dc = link->ctx->dc; - struct dmub_psr *psr = dc->res_pool->psr; - unsigned int panel_inst; - - if (!dc_get_edp_link_panel_inst(dc, link, &panel_inst)) - return; - - /* PSR residency measurements only supported on DMCUB */ - if (psr != NULL && link->psr_settings.psr_feature_enabled) - psr->funcs->psr_get_residency(psr, residency, panel_inst); - else - *residency = 0; -} - -bool dc_link_set_sink_vtotal_in_psr_active(const struct dc_link *link, uint16_t psr_vtotal_idle, uint16_t psr_vtotal_su) -{ - struct dc *dc = link->ctx->dc; - struct dmub_psr *psr = dc->res_pool->psr; - - if (psr == NULL || !link->psr_settings.psr_feature_enabled || !link->psr_settings.psr_vtotal_control_support) - return false; - - psr->funcs->psr_set_sink_vtotal_in_psr_active(psr, psr_vtotal_idle, psr_vtotal_su); - - return true; -} - const struct dc_link_status *dc_link_get_status(const struct dc_link *link) { return &link->link_status; @@ -3999,10 +3544,13 @@ static enum dc_status deallocate_mst_payload(struct pipe_ctx *pipe_ctx) struct fixed31_32 avg_time_slots_per_mtp = dc_fixpt_from_int(0); int i; bool mst_mode = (link->type == dc_connection_mst_branch); + /* adjust for drm changes*/ + bool update_drm_mst_state = true; const struct link_hwss *link_hwss = get_link_hwss(link, &pipe_ctx->link_res); const struct dc_link_settings empty_link_settings = {0}; DC_LOGGER_INIT(link->ctx->logger); + /* deallocate_mst_payload is called before disable link. When mode or * disable/enable monitor, new stream is created which is not in link * stream[] yet. For this, payload is not allocated yet, so de-alloc @@ -4018,7 +3566,7 @@ static enum dc_status deallocate_mst_payload(struct pipe_ctx *pipe_ctx) &empty_link_settings, avg_time_slots_per_mtp); - if (mst_mode) { + if (mst_mode || update_drm_mst_state) { /* when link is in mst mode, reply on mst manager to remove * payload */ @@ -4081,11 +3629,18 @@ static enum dc_status deallocate_mst_payload(struct pipe_ctx *pipe_ctx) stream->ctx, stream); + if (!update_drm_mst_state) + dm_helpers_dp_mst_send_payload_allocation( + stream->ctx, + stream, + false); + } + + if (update_drm_mst_state) dm_helpers_dp_mst_send_payload_allocation( stream->ctx, stream, false); - } return DC_OK; } @@ -4541,110 +4096,6 @@ void core_link_set_avmute(struct pipe_ctx *pipe_ctx, bool enable) dc->hwss.set_avmute(pipe_ctx, enable); } -void dc_link_set_drive_settings(struct dc *dc, - struct link_training_settings *lt_settings, - const struct dc_link *link) -{ - - int i; - struct link_resource link_res; - - for (i = 0; i < dc->link_count; i++) - if (dc->links[i] == link) - break; - - if (i >= dc->link_count) - ASSERT_CRITICAL(false); - - dc_link_get_cur_link_res(link, &link_res); - dc_link_dp_set_drive_settings(dc->links[i], &link_res, lt_settings); -} - -void dc_link_set_preferred_link_settings(struct dc *dc, - struct dc_link_settings *link_setting, - struct dc_link *link) -{ - int i; - struct pipe_ctx *pipe; - struct dc_stream_state *link_stream; - struct dc_link_settings store_settings = *link_setting; - - link->preferred_link_setting = store_settings; - - /* Retrain with preferred link settings only relevant for - * DP signal type - * Check for non-DP signal or if passive dongle present - */ - if (!dc_is_dp_signal(link->connector_signal) || - link->dongle_max_pix_clk > 0) - return; - - for (i = 0; i < MAX_PIPES; i++) { - pipe = &dc->current_state->res_ctx.pipe_ctx[i]; - if (pipe->stream && pipe->stream->link) { - if (pipe->stream->link == link) { - link_stream = pipe->stream; - break; - } - } - } - - /* Stream not found */ - if (i == MAX_PIPES) - return; - - /* Cannot retrain link if backend is off */ - if (link_stream->dpms_off) - return; - - if (link_decide_link_settings(link_stream, &store_settings)) - dp_retrain_link_dp_test(link, &store_settings, false); -} - -void dc_link_set_preferred_training_settings(struct dc *dc, - struct dc_link_settings *link_setting, - struct dc_link_training_overrides *lt_overrides, - struct dc_link *link, - bool skip_immediate_retrain) -{ - if (lt_overrides != NULL) - link->preferred_training_settings = *lt_overrides; - else - memset(&link->preferred_training_settings, 0, sizeof(link->preferred_training_settings)); - - if (link_setting != NULL) { - link->preferred_link_setting = *link_setting; - } else { - link->preferred_link_setting.lane_count = LANE_COUNT_UNKNOWN; - link->preferred_link_setting.link_rate = LINK_RATE_UNKNOWN; - } - - if (link->connector_signal == SIGNAL_TYPE_DISPLAY_PORT && - link->type == dc_connection_mst_branch) - dm_helpers_dp_mst_update_branch_bandwidth(dc->ctx, link); - - /* Retrain now, or wait until next stream update to apply */ - if (skip_immediate_retrain == false) - dc_link_set_preferred_link_settings(dc, &link->preferred_link_setting, link); -} - -void dc_link_set_test_pattern(struct dc_link *link, - enum dp_test_pattern test_pattern, - enum dp_test_pattern_color_space test_pattern_color_space, - const struct link_training_settings *p_link_settings, - const unsigned char *p_custom_pattern, - unsigned int cust_pattern_size) -{ - if (link != NULL) - dc_link_dp_set_test_pattern( - link, - test_pattern, - test_pattern_color_space, - p_link_settings, - p_custom_pattern, - cust_pattern_size); -} - uint32_t dc_link_bandwidth_kbps( const struct dc_link *link, const struct dc_link_settings *link_setting) @@ -4854,3 +4305,591 @@ void dc_restore_link_res_map(const struct dc *dc, uint32_t *map) } } } + +bool dp_validate_mode_timing( + struct dc_link *link, + const struct dc_crtc_timing *timing) +{ + uint32_t req_bw; + uint32_t max_bw; + + const struct dc_link_settings *link_setting; + + /* According to spec, VSC SDP should be used if pixel format is YCbCr420 */ + if (timing->pixel_encoding == PIXEL_ENCODING_YCBCR420 && + !link->dpcd_caps.dprx_feature.bits.VSC_SDP_COLORIMETRY_SUPPORTED && + dal_graphics_object_id_get_connector_id(link->link_id) != CONNECTOR_ID_VIRTUAL) + return false; + + /*always DP fail safe mode*/ + if ((timing->pix_clk_100hz / 10) == (uint32_t) 25175 && + timing->h_addressable == (uint32_t) 640 && + timing->v_addressable == (uint32_t) 480) + return true; + + link_setting = dc_link_get_link_cap(link); + + /* TODO: DYNAMIC_VALIDATION needs to be implemented */ + /*if (flags.DYNAMIC_VALIDATION == 1 && + link->verified_link_cap.lane_count != LANE_COUNT_UNKNOWN) + link_setting = &link->verified_link_cap; + */ + + req_bw = dc_bandwidth_in_kbps_from_timing(timing); + max_bw = dc_link_bandwidth_kbps(link, link_setting); + + if (req_bw <= max_bw) { + /* remember the biggest mode here, during + * initial link training (to get + * verified_link_cap), LS sends event about + * cannot train at reported cap to upper + * layer and upper layer will re-enumerate modes. + * this is not necessary if the lower + * verified_link_cap is enough to drive + * all the modes */ + + /* TODO: DYNAMIC_VALIDATION needs to be implemented */ + /* if (flags.DYNAMIC_VALIDATION == 1) + dpsst->max_req_bw_for_verified_linkcap = dal_max( + dpsst->max_req_bw_for_verified_linkcap, req_bw); */ + return true; + } else + return false; +} + +void dp_enable_mst_on_sink(struct dc_link *link, bool enable) +{ + unsigned char mstmCntl; + + core_link_read_dpcd(link, DP_MSTM_CTRL, &mstmCntl, 1); + if (enable) + mstmCntl |= DP_MST_EN; + else + mstmCntl &= (~DP_MST_EN); + + core_link_write_dpcd(link, DP_MSTM_CTRL, &mstmCntl, 1); +} + +enum dc_status dp_set_fec_ready(struct dc_link *link, const struct link_resource *link_res, bool ready) +{ + /* FEC has to be "set ready" before the link training. + * The policy is to always train with FEC + * if the sink supports it and leave it enabled on link. + * If FEC is not supported, disable it. + */ + struct link_encoder *link_enc = NULL; + enum dc_status status = DC_OK; + uint8_t fec_config = 0; + + link_enc = link_enc_cfg_get_link_enc(link); + ASSERT(link_enc); + + if (!dc_link_should_enable_fec(link)) + return status; + + if (link_enc->funcs->fec_set_ready && + link->dpcd_caps.fec_cap.bits.FEC_CAPABLE) { + if (ready) { + fec_config = 1; + status = core_link_write_dpcd(link, + DP_FEC_CONFIGURATION, + &fec_config, + sizeof(fec_config)); + if (status == DC_OK) { + link_enc->funcs->fec_set_ready(link_enc, true); + link->fec_state = dc_link_fec_ready; + } else { + link_enc->funcs->fec_set_ready(link_enc, false); + link->fec_state = dc_link_fec_not_ready; + dm_error("dpcd write failed to set fec_ready"); + } + } else if (link->fec_state == dc_link_fec_ready) { + fec_config = 0; + status = core_link_write_dpcd(link, + DP_FEC_CONFIGURATION, + &fec_config, + sizeof(fec_config)); + link_enc->funcs->fec_set_ready(link_enc, false); + link->fec_state = dc_link_fec_not_ready; + } + } + + return status; +} + +void dp_set_fec_enable(struct dc_link *link, bool enable) +{ + struct link_encoder *link_enc = NULL; + + link_enc = link_enc_cfg_get_link_enc(link); + ASSERT(link_enc); + + if (!dc_link_should_enable_fec(link)) + return; + + if (link_enc->funcs->fec_set_enable && + link->dpcd_caps.fec_cap.bits.FEC_CAPABLE) { + if (link->fec_state == dc_link_fec_ready && enable) { + /* Accord to DP spec, FEC enable sequence can first + * be transmitted anytime after 1000 LL codes have + * been transmitted on the link after link training + * completion. Using 1 lane RBR should have the maximum + * time for transmitting 1000 LL codes which is 6.173 us. + * So use 7 microseconds delay instead. + */ + udelay(7); + link_enc->funcs->fec_set_enable(link_enc, true); + link->fec_state = dc_link_fec_enabled; + } else if (link->fec_state == dc_link_fec_enabled && !enable) { + link_enc->funcs->fec_set_enable(link_enc, false); + link->fec_state = dc_link_fec_ready; + } + } +} + +// TODO - DP2.0 Link: Fix get_lane_status to handle LTTPR offset (SST and MST) +static void get_lane_status( + struct dc_link *link, + uint32_t lane_count, + union lane_status *status, + union lane_align_status_updated *status_updated) +{ + unsigned int lane; + uint8_t dpcd_buf[3] = {0}; + + if (status == NULL || status_updated == NULL) { + return; + } + + core_link_read_dpcd( + link, + DP_LANE0_1_STATUS, + dpcd_buf, + sizeof(dpcd_buf)); + + for (lane = 0; lane < lane_count; lane++) { + status[lane].raw = dp_get_nibble_at_index(&dpcd_buf[0], lane); + } + + status_updated->raw = dpcd_buf[2]; +} + +bool dpcd_write_128b_132b_sst_payload_allocation_table( + const struct dc_stream_state *stream, + struct dc_link *link, + struct link_mst_stream_allocation_table *proposed_table, + bool allocate) +{ + const uint8_t vc_id = 1; /// VC ID always 1 for SST + const uint8_t start_time_slot = 0; /// Always start at time slot 0 for SST + bool result = false; + uint8_t req_slot_count = 0; + struct fixed31_32 avg_time_slots_per_mtp = { 0 }; + union payload_table_update_status update_status = { 0 }; + const uint32_t max_retries = 30; + uint32_t retries = 0; + DC_LOGGER_INIT(link->ctx->logger); + + if (allocate) { + avg_time_slots_per_mtp = calculate_sst_avg_time_slots_per_mtp(stream, link); + req_slot_count = dc_fixpt_ceil(avg_time_slots_per_mtp); + /// Validation should filter out modes that exceed link BW + ASSERT(req_slot_count <= MAX_MTP_SLOT_COUNT); + if (req_slot_count > MAX_MTP_SLOT_COUNT) + return false; + } else { + /// Leave req_slot_count = 0 if allocate is false. + } + + proposed_table->stream_count = 1; /// Always 1 stream for SST + proposed_table->stream_allocations[0].slot_count = req_slot_count; + proposed_table->stream_allocations[0].vcp_id = vc_id; + + if (link->aux_access_disabled) + return true; + + /// Write DPCD 2C0 = 1 to start updating + update_status.bits.VC_PAYLOAD_TABLE_UPDATED = 1; + core_link_write_dpcd( + link, + DP_PAYLOAD_TABLE_UPDATE_STATUS, + &update_status.raw, + 1); + + /// Program the changes in DPCD 1C0 - 1C2 + ASSERT(vc_id == 1); + core_link_write_dpcd( + link, + DP_PAYLOAD_ALLOCATE_SET, + &vc_id, + 1); + + ASSERT(start_time_slot == 0); + core_link_write_dpcd( + link, + DP_PAYLOAD_ALLOCATE_START_TIME_SLOT, + &start_time_slot, + 1); + + core_link_write_dpcd( + link, + DP_PAYLOAD_ALLOCATE_TIME_SLOT_COUNT, + &req_slot_count, + 1); + + /// Poll till DPCD 2C0 read 1 + /// Try for at least 150ms (30 retries, with 5ms delay after each attempt) + + while (retries < max_retries) { + if (core_link_read_dpcd( + link, + DP_PAYLOAD_TABLE_UPDATE_STATUS, + &update_status.raw, + 1) == DC_OK) { + if (update_status.bits.VC_PAYLOAD_TABLE_UPDATED == 1) { + DC_LOG_DP2("SST Update Payload: downstream payload table updated."); + result = true; + break; + } + } else { + union dpcd_rev dpcdRev; + + if (core_link_read_dpcd( + link, + DP_DPCD_REV, + &dpcdRev.raw, + 1) != DC_OK) { + DC_LOG_ERROR("SST Update Payload: Unable to read DPCD revision " + "of sink while polling payload table " + "updated status bit."); + break; + } + } + retries++; + msleep(5); + } + + if (!result && retries == max_retries) { + DC_LOG_ERROR("SST Update Payload: Payload table not updated after retries, " + "continue on. Something is wrong with the branch."); + // TODO - DP2.0 Payload: Read and log the payload table from downstream branch + } + + return result; +} + +bool dpcd_poll_for_allocation_change_trigger(struct dc_link *link) +{ + /* + * wait for ACT handled + */ + int i; + const int act_retries = 30; + enum act_return_status result = ACT_FAILED; + union payload_table_update_status update_status = {0}; + union lane_status dpcd_lane_status[LANE_COUNT_DP_MAX]; + union lane_align_status_updated lane_status_updated; + DC_LOGGER_INIT(link->ctx->logger); + + if (link->aux_access_disabled) + return true; + for (i = 0; i < act_retries; i++) { + get_lane_status(link, link->cur_link_settings.lane_count, dpcd_lane_status, &lane_status_updated); + + if (!dp_is_cr_done(link->cur_link_settings.lane_count, dpcd_lane_status) || + !dp_is_ch_eq_done(link->cur_link_settings.lane_count, dpcd_lane_status) || + !dp_is_symbol_locked(link->cur_link_settings.lane_count, dpcd_lane_status) || + !dp_is_interlane_aligned(lane_status_updated)) { + DC_LOG_ERROR("SST Update Payload: Link loss occurred while " + "polling for ACT handled."); + result = ACT_LINK_LOST; + break; + } + core_link_read_dpcd( + link, + DP_PAYLOAD_TABLE_UPDATE_STATUS, + &update_status.raw, + 1); + + if (update_status.bits.ACT_HANDLED == 1) { + DC_LOG_DP2("SST Update Payload: ACT handled by downstream."); + result = ACT_SUCCESS; + break; + } + + msleep(5); + } + + if (result == ACT_FAILED) { + DC_LOG_ERROR("SST Update Payload: ACT still not handled after retries, " + "continue on. Something is wrong with the branch."); + } + + return (result == ACT_SUCCESS); +} + +struct fixed31_32 calculate_sst_avg_time_slots_per_mtp( + const struct dc_stream_state *stream, + const struct dc_link *link) +{ + struct fixed31_32 link_bw_effective = + dc_fixpt_from_int( + dc_link_bandwidth_kbps(link, &link->cur_link_settings)); + struct fixed31_32 timeslot_bw_effective = + dc_fixpt_div_int(link_bw_effective, MAX_MTP_SLOT_COUNT); + struct fixed31_32 timing_bw = + dc_fixpt_from_int( + dc_bandwidth_in_kbps_from_timing(&stream->timing)); + struct fixed31_32 avg_time_slots_per_mtp = + dc_fixpt_div(timing_bw, timeslot_bw_effective); + + return avg_time_slots_per_mtp; +} + +void dc_link_clear_dprx_states(struct dc_link *link) +{ + memset(&link->dprx_states, 0, sizeof(link->dprx_states)); +} + +void dp_source_sequence_trace(struct dc_link *link, uint8_t dp_test_mode) +{ + if (link != NULL && link->dc->debug.enable_driver_sequence_debug) + core_link_write_dpcd(link, DP_SOURCE_SEQUENCE, + &dp_test_mode, sizeof(dp_test_mode)); +} + +static void dsc_optc_config_log(struct display_stream_compressor *dsc, + struct dsc_optc_config *config) +{ + uint32_t precision = 1 << 28; + uint32_t bytes_per_pixel_int = config->bytes_per_pixel / precision; + uint32_t bytes_per_pixel_mod = config->bytes_per_pixel % precision; + uint64_t ll_bytes_per_pix_fraq = bytes_per_pixel_mod; + DC_LOGGER_INIT(dsc->ctx->logger); + + /* 7 fractional digits decimal precision for bytes per pixel is enough because DSC + * bits per pixel precision is 1/16th of a pixel, which means bytes per pixel precision is + * 1/16/8 = 1/128 of a byte, or 0.0078125 decimal + */ + ll_bytes_per_pix_fraq *= 10000000; + ll_bytes_per_pix_fraq /= precision; + + DC_LOG_DSC("\tbytes_per_pixel 0x%08x (%d.%07d)", + config->bytes_per_pixel, bytes_per_pixel_int, (uint32_t)ll_bytes_per_pix_fraq); + DC_LOG_DSC("\tis_pixel_format_444 %d", config->is_pixel_format_444); + DC_LOG_DSC("\tslice_width %d", config->slice_width); +} + +bool dp_set_dsc_on_rx(struct pipe_ctx *pipe_ctx, bool enable) +{ + struct dc *dc = pipe_ctx->stream->ctx->dc; + struct dc_stream_state *stream = pipe_ctx->stream; + bool result = false; + + if (dc_is_virtual_signal(stream->signal) || IS_FPGA_MAXIMUS_DC(dc->ctx->dce_environment)) + result = true; + else + result = dm_helpers_dp_write_dsc_enable(dc->ctx, stream, enable); + return result; +} + +/* The stream with these settings can be sent (unblanked) only after DSC was enabled on RX first, + * i.e. after dp_enable_dsc_on_rx() had been called + */ +void dp_set_dsc_on_stream(struct pipe_ctx *pipe_ctx, bool enable) +{ + struct display_stream_compressor *dsc = pipe_ctx->stream_res.dsc; + struct dc *dc = pipe_ctx->stream->ctx->dc; + struct dc_stream_state *stream = pipe_ctx->stream; + struct pipe_ctx *odm_pipe; + int opp_cnt = 1; + DC_LOGGER_INIT(dsc->ctx->logger); + + for (odm_pipe = pipe_ctx->next_odm_pipe; odm_pipe; odm_pipe = odm_pipe->next_odm_pipe) + opp_cnt++; + + if (enable) { + struct dsc_config dsc_cfg; + struct dsc_optc_config dsc_optc_cfg; + enum optc_dsc_mode optc_dsc_mode; + + /* Enable DSC hw block */ + dsc_cfg.pic_width = (stream->timing.h_addressable + stream->timing.h_border_left + stream->timing.h_border_right) / opp_cnt; + dsc_cfg.pic_height = stream->timing.v_addressable + stream->timing.v_border_top + stream->timing.v_border_bottom; + dsc_cfg.pixel_encoding = stream->timing.pixel_encoding; + dsc_cfg.color_depth = stream->timing.display_color_depth; + dsc_cfg.is_odm = pipe_ctx->next_odm_pipe ? true : false; + dsc_cfg.dc_dsc_cfg = stream->timing.dsc_cfg; + ASSERT(dsc_cfg.dc_dsc_cfg.num_slices_h % opp_cnt == 0); + dsc_cfg.dc_dsc_cfg.num_slices_h /= opp_cnt; + + dsc->funcs->dsc_set_config(dsc, &dsc_cfg, &dsc_optc_cfg); + dsc->funcs->dsc_enable(dsc, pipe_ctx->stream_res.opp->inst); + for (odm_pipe = pipe_ctx->next_odm_pipe; odm_pipe; odm_pipe = odm_pipe->next_odm_pipe) { + struct display_stream_compressor *odm_dsc = odm_pipe->stream_res.dsc; + + odm_dsc->funcs->dsc_set_config(odm_dsc, &dsc_cfg, &dsc_optc_cfg); + odm_dsc->funcs->dsc_enable(odm_dsc, odm_pipe->stream_res.opp->inst); + } + dsc_cfg.dc_dsc_cfg.num_slices_h *= opp_cnt; + dsc_cfg.pic_width *= opp_cnt; + + optc_dsc_mode = dsc_optc_cfg.is_pixel_format_444 ? OPTC_DSC_ENABLED_444 : OPTC_DSC_ENABLED_NATIVE_SUBSAMPLED; + + /* Enable DSC in encoder */ + if (dc_is_dp_signal(stream->signal) && !IS_FPGA_MAXIMUS_DC(dc->ctx->dce_environment) + && !link_is_dp_128b_132b_signal(pipe_ctx)) { + DC_LOG_DSC("Setting stream encoder DSC config for engine %d:", (int)pipe_ctx->stream_res.stream_enc->id); + dsc_optc_config_log(dsc, &dsc_optc_cfg); + pipe_ctx->stream_res.stream_enc->funcs->dp_set_dsc_config(pipe_ctx->stream_res.stream_enc, + optc_dsc_mode, + dsc_optc_cfg.bytes_per_pixel, + dsc_optc_cfg.slice_width); + + /* PPS SDP is set elsewhere because it has to be done after DIG FE is connected to DIG BE */ + } + + /* Enable DSC in OPTC */ + DC_LOG_DSC("Setting optc DSC config for tg instance %d:", pipe_ctx->stream_res.tg->inst); + dsc_optc_config_log(dsc, &dsc_optc_cfg); + pipe_ctx->stream_res.tg->funcs->set_dsc_config(pipe_ctx->stream_res.tg, + optc_dsc_mode, + dsc_optc_cfg.bytes_per_pixel, + dsc_optc_cfg.slice_width); + } else { + /* disable DSC in OPTC */ + pipe_ctx->stream_res.tg->funcs->set_dsc_config( + pipe_ctx->stream_res.tg, + OPTC_DSC_DISABLED, 0, 0); + + /* disable DSC in stream encoder */ + if (dc_is_dp_signal(stream->signal)) { + if (link_is_dp_128b_132b_signal(pipe_ctx)) + pipe_ctx->stream_res.hpo_dp_stream_enc->funcs->dp_set_dsc_pps_info_packet( + pipe_ctx->stream_res.hpo_dp_stream_enc, + false, + NULL, + true); + else if (!IS_FPGA_MAXIMUS_DC(dc->ctx->dce_environment)) { + pipe_ctx->stream_res.stream_enc->funcs->dp_set_dsc_config( + pipe_ctx->stream_res.stream_enc, + OPTC_DSC_DISABLED, 0, 0); + pipe_ctx->stream_res.stream_enc->funcs->dp_set_dsc_pps_info_packet( + pipe_ctx->stream_res.stream_enc, false, NULL, true); + } + } + + /* disable DSC block */ + pipe_ctx->stream_res.dsc->funcs->dsc_disable(pipe_ctx->stream_res.dsc); + for (odm_pipe = pipe_ctx->next_odm_pipe; odm_pipe; odm_pipe = odm_pipe->next_odm_pipe) + odm_pipe->stream_res.dsc->funcs->dsc_disable(odm_pipe->stream_res.dsc); + } +} + +bool dp_set_dsc_enable(struct pipe_ctx *pipe_ctx, bool enable) +{ + struct display_stream_compressor *dsc = pipe_ctx->stream_res.dsc; + bool result = false; + + if (!pipe_ctx->stream->timing.flags.DSC) + goto out; + if (!dsc) + goto out; + + if (enable) { + { + dp_set_dsc_on_stream(pipe_ctx, true); + result = true; + } + } else { + dp_set_dsc_on_rx(pipe_ctx, false); + dp_set_dsc_on_stream(pipe_ctx, false); + result = true; + } +out: + return result; +} + +/* + * For dynamic bpp change case, dsc is programmed with MASTER_UPDATE_LOCK enabled; + * hence PPS info packet update need to use frame update instead of immediate update. + * Added parameter immediate_update for this purpose. + * The decision to use frame update is hard-coded in function dp_update_dsc_config(), + * which is the only place where a "false" would be passed in for param immediate_update. + * + * immediate_update is only applicable when DSC is enabled. + */ +bool dp_set_dsc_pps_sdp(struct pipe_ctx *pipe_ctx, bool enable, bool immediate_update) +{ + struct display_stream_compressor *dsc = pipe_ctx->stream_res.dsc; + struct dc_stream_state *stream = pipe_ctx->stream; + DC_LOGGER_INIT(dsc->ctx->logger); + + if (!pipe_ctx->stream->timing.flags.DSC || !dsc) + return false; + + if (enable) { + struct dsc_config dsc_cfg; + uint8_t dsc_packed_pps[128]; + + memset(&dsc_cfg, 0, sizeof(dsc_cfg)); + memset(dsc_packed_pps, 0, 128); + + /* Enable DSC hw block */ + dsc_cfg.pic_width = stream->timing.h_addressable + stream->timing.h_border_left + stream->timing.h_border_right; + dsc_cfg.pic_height = stream->timing.v_addressable + stream->timing.v_border_top + stream->timing.v_border_bottom; + dsc_cfg.pixel_encoding = stream->timing.pixel_encoding; + dsc_cfg.color_depth = stream->timing.display_color_depth; + dsc_cfg.is_odm = pipe_ctx->next_odm_pipe ? true : false; + dsc_cfg.dc_dsc_cfg = stream->timing.dsc_cfg; + + dsc->funcs->dsc_get_packed_pps(dsc, &dsc_cfg, &dsc_packed_pps[0]); + memcpy(&stream->dsc_packed_pps[0], &dsc_packed_pps[0], sizeof(stream->dsc_packed_pps)); + if (dc_is_dp_signal(stream->signal)) { + DC_LOG_DSC("Setting stream encoder DSC PPS SDP for engine %d\n", (int)pipe_ctx->stream_res.stream_enc->id); + if (link_is_dp_128b_132b_signal(pipe_ctx)) + pipe_ctx->stream_res.hpo_dp_stream_enc->funcs->dp_set_dsc_pps_info_packet( + pipe_ctx->stream_res.hpo_dp_stream_enc, + true, + &dsc_packed_pps[0], + immediate_update); + else + pipe_ctx->stream_res.stream_enc->funcs->dp_set_dsc_pps_info_packet( + pipe_ctx->stream_res.stream_enc, + true, + &dsc_packed_pps[0], + immediate_update); + } + } else { + /* disable DSC PPS in stream encoder */ + memset(&stream->dsc_packed_pps[0], 0, sizeof(stream->dsc_packed_pps)); + if (dc_is_dp_signal(stream->signal)) { + if (link_is_dp_128b_132b_signal(pipe_ctx)) + pipe_ctx->stream_res.hpo_dp_stream_enc->funcs->dp_set_dsc_pps_info_packet( + pipe_ctx->stream_res.hpo_dp_stream_enc, + false, + NULL, + true); + else + pipe_ctx->stream_res.stream_enc->funcs->dp_set_dsc_pps_info_packet( + pipe_ctx->stream_res.stream_enc, false, NULL, true); + } + } + + return true; +} + + +bool dp_update_dsc_config(struct pipe_ctx *pipe_ctx) +{ + struct display_stream_compressor *dsc = pipe_ctx->stream_res.dsc; + + if (!pipe_ctx->stream->timing.flags.DSC) + return false; + if (!dsc) + return false; + + dp_set_dsc_on_stream(pipe_ctx, true); + dp_set_dsc_pps_sdp(pipe_ctx, true, false); + return true; +} diff --git a/drivers/gpu/drm/amd/display/dc/core/dc_link_dp.c b/drivers/gpu/drm/amd/display/dc/core/dc_link_dp.c deleted file mode 100644 index 6747e4b199de4b8808f7648a74a99e90eb3d2d78..0000000000000000000000000000000000000000 --- a/drivers/gpu/drm/amd/display/dc/core/dc_link_dp.c +++ /dev/null @@ -1,2375 +0,0 @@ -/* - * Copyright 2015 Advanced Micro Devices, Inc. - * - * Permission is hereby granted, free of charge, to any person obtaining a - * copy of this software and associated documentation files (the "Software"), - * to deal in the Software without restriction, including without limitation - * the rights to use, copy, modify, merge, publish, distribute, sublicense, - * and/or sell copies of the Software, and to permit persons to whom the - * Software is furnished to do so, subject to the following conditions: - * - * The above copyright notice and this permission notice shall be included in - * all copies or substantial portions of the Software. - * - * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR - * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, - * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL - * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR - * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, - * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR - * OTHER DEALINGS IN THE SOFTWARE. - * - * Authors: AMD - */ -#include "dm_services.h" -#include "dc.h" -#include "dc_link_dp.h" -#include "dm_helpers.h" -#include "opp.h" -#include "dsc.h" -#include "resource.h" - -#include "inc/core_types.h" -#include "link_hwss.h" -#include "link/link_ddc.h" -#include "core_status.h" -#include "dpcd_defs.h" - -#include "dc_dmub_srv.h" -#include "dce/dmub_hw_lock_mgr.h" -#include "link/link_dp_dpia.h" -#include "inc/link_enc_cfg.h" -#include "clk_mgr.h" -#include "link/link_dp_trace.h" -#include "link/link_dp_training.h" -#include "link/link_dp_training_fixed_vs_pe_retimer.h" -#include "link/link_dp_training_dpia.h" -#include "link/link_dp_training_auxless.h" -#include "link/link_dp_phy.h" -#include "link/link_dp_capability.h" -#define DC_LOGGER \ - link->ctx->logger - -#define DC_TRACE_LEVEL_MESSAGE(...) /* do nothing */ -#include "link/link_dpcd.h" - -static uint8_t get_nibble_at_index(const uint8_t *buf, - uint32_t index) -{ - uint8_t nibble; - nibble = buf[index / 2]; - - if (index % 2) - nibble >>= 4; - else - nibble &= 0x0F; - - return nibble; -} - -enum dc_status read_hpd_rx_irq_data( - struct dc_link *link, - union hpd_irq_data *irq_data) -{ - static enum dc_status retval; - - /* The HW reads 16 bytes from 200h on HPD, - * but if we get an AUX_DEFER, the HW cannot retry - * and this causes the CTS tests 4.3.2.1 - 3.2.4 to - * fail, so we now explicitly read 6 bytes which is - * the req from the above mentioned test cases. - * - * For DP 1.4 we need to read those from 2002h range. - */ - if (link->dpcd_caps.dpcd_rev.raw < DPCD_REV_14) - retval = core_link_read_dpcd( - link, - DP_SINK_COUNT, - irq_data->raw, - sizeof(union hpd_irq_data)); - else { - /* Read 14 bytes in a single read and then copy only the required fields. - * This is more efficient than doing it in two separate AUX reads. */ - - uint8_t tmp[DP_SINK_STATUS_ESI - DP_SINK_COUNT_ESI + 1]; - - retval = core_link_read_dpcd( - link, - DP_SINK_COUNT_ESI, - tmp, - sizeof(tmp)); - - if (retval != DC_OK) - return retval; - - irq_data->bytes.sink_cnt.raw = tmp[DP_SINK_COUNT_ESI - DP_SINK_COUNT_ESI]; - irq_data->bytes.device_service_irq.raw = tmp[DP_DEVICE_SERVICE_IRQ_VECTOR_ESI0 - DP_SINK_COUNT_ESI]; - irq_data->bytes.lane01_status.raw = tmp[DP_LANE0_1_STATUS_ESI - DP_SINK_COUNT_ESI]; - irq_data->bytes.lane23_status.raw = tmp[DP_LANE2_3_STATUS_ESI - DP_SINK_COUNT_ESI]; - irq_data->bytes.lane_status_updated.raw = tmp[DP_LANE_ALIGN_STATUS_UPDATED_ESI - DP_SINK_COUNT_ESI]; - irq_data->bytes.sink_status.raw = tmp[DP_SINK_STATUS_ESI - DP_SINK_COUNT_ESI]; - } - - return retval; -} - -bool hpd_rx_irq_check_link_loss_status( - struct dc_link *link, - union hpd_irq_data *hpd_irq_dpcd_data) -{ - uint8_t irq_reg_rx_power_state = 0; - enum dc_status dpcd_result = DC_ERROR_UNEXPECTED; - union lane_status lane_status; - uint32_t lane; - bool sink_status_changed; - bool return_code; - - sink_status_changed = false; - return_code = false; - - if (link->cur_link_settings.lane_count == 0) - return return_code; - - /*1. Check that Link Status changed, before re-training.*/ - - /*parse lane status*/ - for (lane = 0; lane < link->cur_link_settings.lane_count; lane++) { - /* check status of lanes 0,1 - * changed DpcdAddress_Lane01Status (0x202) - */ - lane_status.raw = get_nibble_at_index( - &hpd_irq_dpcd_data->bytes.lane01_status.raw, - lane); - - if (!lane_status.bits.CHANNEL_EQ_DONE_0 || - !lane_status.bits.CR_DONE_0 || - !lane_status.bits.SYMBOL_LOCKED_0) { - /* if one of the channel equalization, clock - * recovery or symbol lock is dropped - * consider it as (link has been - * dropped) dp sink status has changed - */ - sink_status_changed = true; - break; - } - } - - /* Check interlane align.*/ - if (sink_status_changed || - !hpd_irq_dpcd_data->bytes.lane_status_updated.bits.INTERLANE_ALIGN_DONE) { - - DC_LOG_HW_HPD_IRQ("%s: Link Status changed.\n", __func__); - - return_code = true; - - /*2. Check that we can handle interrupt: Not in FS DOS, - * Not in "Display Timeout" state, Link is trained. - */ - dpcd_result = core_link_read_dpcd(link, - DP_SET_POWER, - &irq_reg_rx_power_state, - sizeof(irq_reg_rx_power_state)); - - if (dpcd_result != DC_OK) { - DC_LOG_HW_HPD_IRQ("%s: DPCD read failed to obtain power state.\n", - __func__); - } else { - if (irq_reg_rx_power_state != DP_SET_POWER_D0) - return_code = false; - } - } - - return return_code; -} - -bool dp_validate_mode_timing( - struct dc_link *link, - const struct dc_crtc_timing *timing) -{ - uint32_t req_bw; - uint32_t max_bw; - - const struct dc_link_settings *link_setting; - - /* According to spec, VSC SDP should be used if pixel format is YCbCr420 */ - if (timing->pixel_encoding == PIXEL_ENCODING_YCBCR420 && - !link->dpcd_caps.dprx_feature.bits.VSC_SDP_COLORIMETRY_SUPPORTED && - dal_graphics_object_id_get_connector_id(link->link_id) != CONNECTOR_ID_VIRTUAL) - return false; - - /*always DP fail safe mode*/ - if ((timing->pix_clk_100hz / 10) == (uint32_t) 25175 && - timing->h_addressable == (uint32_t) 640 && - timing->v_addressable == (uint32_t) 480) - return true; - - link_setting = dc_link_get_link_cap(link); - - /* TODO: DYNAMIC_VALIDATION needs to be implemented */ - /*if (flags.DYNAMIC_VALIDATION == 1 && - link->verified_link_cap.lane_count != LANE_COUNT_UNKNOWN) - link_setting = &link->verified_link_cap; - */ - - req_bw = dc_bandwidth_in_kbps_from_timing(timing); - max_bw = dc_link_bandwidth_kbps(link, link_setting); - - if (req_bw <= max_bw) { - /* remember the biggest mode here, during - * initial link training (to get - * verified_link_cap), LS sends event about - * cannot train at reported cap to upper - * layer and upper layer will re-enumerate modes. - * this is not necessary if the lower - * verified_link_cap is enough to drive - * all the modes */ - - /* TODO: DYNAMIC_VALIDATION needs to be implemented */ - /* if (flags.DYNAMIC_VALIDATION == 1) - dpsst->max_req_bw_for_verified_linkcap = dal_max( - dpsst->max_req_bw_for_verified_linkcap, req_bw); */ - return true; - } else - return false; -} - -/*************************Short Pulse IRQ***************************/ -bool dc_link_dp_allow_hpd_rx_irq(const struct dc_link *link) -{ - /* - * Don't handle RX IRQ unless one of following is met: - * 1) The link is established (cur_link_settings != unknown) - * 2) We know we're dealing with a branch device, SST or MST - */ - - if ((link->cur_link_settings.lane_count != LANE_COUNT_UNKNOWN) || - is_dp_branch_device(link)) - return true; - - return false; -} - -static bool handle_hpd_irq_psr_sink(struct dc_link *link) -{ - union dpcd_psr_configuration psr_configuration; - - if (!link->psr_settings.psr_feature_enabled) - return false; - - dm_helpers_dp_read_dpcd( - link->ctx, - link, - 368,/*DpcdAddress_PSR_Enable_Cfg*/ - &psr_configuration.raw, - sizeof(psr_configuration.raw)); - - if (psr_configuration.bits.ENABLE) { - unsigned char dpcdbuf[3] = {0}; - union psr_error_status psr_error_status; - union psr_sink_psr_status psr_sink_psr_status; - - dm_helpers_dp_read_dpcd( - link->ctx, - link, - 0x2006, /*DpcdAddress_PSR_Error_Status*/ - (unsigned char *) dpcdbuf, - sizeof(dpcdbuf)); - - /*DPCD 2006h ERROR STATUS*/ - psr_error_status.raw = dpcdbuf[0]; - /*DPCD 2008h SINK PANEL SELF REFRESH STATUS*/ - psr_sink_psr_status.raw = dpcdbuf[2]; - - if (psr_error_status.bits.LINK_CRC_ERROR || - psr_error_status.bits.RFB_STORAGE_ERROR || - psr_error_status.bits.VSC_SDP_ERROR) { - bool allow_active; - - /* Acknowledge and clear error bits */ - dm_helpers_dp_write_dpcd( - link->ctx, - link, - 8198,/*DpcdAddress_PSR_Error_Status*/ - &psr_error_status.raw, - sizeof(psr_error_status.raw)); - - /* PSR error, disable and re-enable PSR */ - if (link->psr_settings.psr_allow_active) { - allow_active = false; - dc_link_set_psr_allow_active(link, &allow_active, true, false, NULL); - allow_active = true; - dc_link_set_psr_allow_active(link, &allow_active, true, false, NULL); - } - - return true; - } else if (psr_sink_psr_status.bits.SINK_SELF_REFRESH_STATUS == - PSR_SINK_STATE_ACTIVE_DISPLAY_FROM_SINK_RFB){ - /* No error is detect, PSR is active. - * We should return with IRQ_HPD handled without - * checking for loss of sync since PSR would have - * powered down main link. - */ - return true; - } - } - return false; -} - -static enum dc_link_rate get_link_rate_from_test_link_rate(uint8_t test_rate) -{ - switch (test_rate) { - case DP_TEST_LINK_RATE_RBR: - return LINK_RATE_LOW; - case DP_TEST_LINK_RATE_HBR: - return LINK_RATE_HIGH; - case DP_TEST_LINK_RATE_HBR2: - return LINK_RATE_HIGH2; - case DP_TEST_LINK_RATE_HBR3: - return LINK_RATE_HIGH3; - case DP_TEST_LINK_RATE_UHBR10: - return LINK_RATE_UHBR10; - case DP_TEST_LINK_RATE_UHBR20: - return LINK_RATE_UHBR20; - case DP_TEST_LINK_RATE_UHBR13_5: - return LINK_RATE_UHBR13_5; - default: - return LINK_RATE_UNKNOWN; - } -} - -static void dp_test_send_link_training(struct dc_link *link) -{ - struct dc_link_settings link_settings = {0}; - uint8_t test_rate = 0; - - core_link_read_dpcd( - link, - DP_TEST_LANE_COUNT, - (unsigned char *)(&link_settings.lane_count), - 1); - core_link_read_dpcd( - link, - DP_TEST_LINK_RATE, - &test_rate, - 1); - link_settings.link_rate = get_link_rate_from_test_link_rate(test_rate); - - /* Set preferred link settings */ - link->verified_link_cap.lane_count = link_settings.lane_count; - link->verified_link_cap.link_rate = link_settings.link_rate; - - dp_retrain_link_dp_test(link, &link_settings, false); -} - -static bool is_dp_phy_sqaure_pattern(enum dp_test_pattern test_pattern) -{ - return (DP_TEST_PATTERN_SQUARE_BEGIN <= test_pattern && - test_pattern <= DP_TEST_PATTERN_SQUARE_END); -} - -/* TODO Raven hbr2 compliance eye output is unstable - * (toggling on and off) with debugger break - * This caueses intermittent PHY automation failure - * Need to look into the root cause */ -static void dp_test_send_phy_test_pattern(struct dc_link *link) -{ - union phy_test_pattern dpcd_test_pattern; - union lane_adjust dpcd_lane_adjustment[2]; - unsigned char dpcd_post_cursor_2_adjustment = 0; - unsigned char test_pattern_buffer[ - (DP_TEST_264BIT_CUSTOM_PATTERN_263_256 - - DP_TEST_264BIT_CUSTOM_PATTERN_7_0)+1] = {0}; - unsigned int test_pattern_size = 0; - enum dp_test_pattern test_pattern; - union lane_adjust dpcd_lane_adjust; - unsigned int lane; - struct link_training_settings link_training_settings; - unsigned char no_preshoot = 0; - unsigned char no_deemphasis = 0; - - dpcd_test_pattern.raw = 0; - memset(dpcd_lane_adjustment, 0, sizeof(dpcd_lane_adjustment)); - memset(&link_training_settings, 0, sizeof(link_training_settings)); - - /* get phy test pattern and pattern parameters from DP receiver */ - core_link_read_dpcd( - link, - DP_PHY_TEST_PATTERN, - &dpcd_test_pattern.raw, - sizeof(dpcd_test_pattern)); - core_link_read_dpcd( - link, - DP_ADJUST_REQUEST_LANE0_1, - &dpcd_lane_adjustment[0].raw, - sizeof(dpcd_lane_adjustment)); - - /* prepare link training settings */ - link_training_settings.link_settings = link->cur_link_settings; - - link_training_settings.lttpr_mode = dc_link_decide_lttpr_mode(link, &link->cur_link_settings); - - if ((link->chip_caps & EXT_DISPLAY_PATH_CAPS__DP_FIXED_VS_EN) && - link_training_settings.lttpr_mode == LTTPR_MODE_TRANSPARENT) - dp_fixed_vs_pe_read_lane_adjust( - link, - link_training_settings.dpcd_lane_settings); - - /*get post cursor 2 parameters - * For DP 1.1a or eariler, this DPCD register's value is 0 - * For DP 1.2 or later: - * Bits 1:0 = POST_CURSOR2_LANE0; Bits 3:2 = POST_CURSOR2_LANE1 - * Bits 5:4 = POST_CURSOR2_LANE2; Bits 7:6 = POST_CURSOR2_LANE3 - */ - core_link_read_dpcd( - link, - DP_ADJUST_REQUEST_POST_CURSOR2, - &dpcd_post_cursor_2_adjustment, - sizeof(dpcd_post_cursor_2_adjustment)); - - /* translate request */ - switch (dpcd_test_pattern.bits.PATTERN) { - case PHY_TEST_PATTERN_D10_2: - test_pattern = DP_TEST_PATTERN_D102; - break; - case PHY_TEST_PATTERN_SYMBOL_ERROR: - test_pattern = DP_TEST_PATTERN_SYMBOL_ERROR; - break; - case PHY_TEST_PATTERN_PRBS7: - test_pattern = DP_TEST_PATTERN_PRBS7; - break; - case PHY_TEST_PATTERN_80BIT_CUSTOM: - test_pattern = DP_TEST_PATTERN_80BIT_CUSTOM; - break; - case PHY_TEST_PATTERN_CP2520_1: - /* CP2520 pattern is unstable, temporarily use TPS4 instead */ - test_pattern = (link->dc->caps.force_dp_tps4_for_cp2520 == 1) ? - DP_TEST_PATTERN_TRAINING_PATTERN4 : - DP_TEST_PATTERN_HBR2_COMPLIANCE_EYE; - break; - case PHY_TEST_PATTERN_CP2520_2: - /* CP2520 pattern is unstable, temporarily use TPS4 instead */ - test_pattern = (link->dc->caps.force_dp_tps4_for_cp2520 == 1) ? - DP_TEST_PATTERN_TRAINING_PATTERN4 : - DP_TEST_PATTERN_HBR2_COMPLIANCE_EYE; - break; - case PHY_TEST_PATTERN_CP2520_3: - test_pattern = DP_TEST_PATTERN_TRAINING_PATTERN4; - break; - case PHY_TEST_PATTERN_128b_132b_TPS1: - test_pattern = DP_TEST_PATTERN_128b_132b_TPS1; - break; - case PHY_TEST_PATTERN_128b_132b_TPS2: - test_pattern = DP_TEST_PATTERN_128b_132b_TPS2; - break; - case PHY_TEST_PATTERN_PRBS9: - test_pattern = DP_TEST_PATTERN_PRBS9; - break; - case PHY_TEST_PATTERN_PRBS11: - test_pattern = DP_TEST_PATTERN_PRBS11; - break; - case PHY_TEST_PATTERN_PRBS15: - test_pattern = DP_TEST_PATTERN_PRBS15; - break; - case PHY_TEST_PATTERN_PRBS23: - test_pattern = DP_TEST_PATTERN_PRBS23; - break; - case PHY_TEST_PATTERN_PRBS31: - test_pattern = DP_TEST_PATTERN_PRBS31; - break; - case PHY_TEST_PATTERN_264BIT_CUSTOM: - test_pattern = DP_TEST_PATTERN_264BIT_CUSTOM; - break; - case PHY_TEST_PATTERN_SQUARE: - test_pattern = DP_TEST_PATTERN_SQUARE; - break; - case PHY_TEST_PATTERN_SQUARE_PRESHOOT_DISABLED: - test_pattern = DP_TEST_PATTERN_SQUARE_PRESHOOT_DISABLED; - no_preshoot = 1; - break; - case PHY_TEST_PATTERN_SQUARE_DEEMPHASIS_DISABLED: - test_pattern = DP_TEST_PATTERN_SQUARE_DEEMPHASIS_DISABLED; - no_deemphasis = 1; - break; - case PHY_TEST_PATTERN_SQUARE_PRESHOOT_DEEMPHASIS_DISABLED: - test_pattern = DP_TEST_PATTERN_SQUARE_PRESHOOT_DEEMPHASIS_DISABLED; - no_preshoot = 1; - no_deemphasis = 1; - break; - default: - test_pattern = DP_TEST_PATTERN_VIDEO_MODE; - break; - } - - if (test_pattern == DP_TEST_PATTERN_80BIT_CUSTOM) { - test_pattern_size = (DP_TEST_80BIT_CUSTOM_PATTERN_79_72 - - DP_TEST_80BIT_CUSTOM_PATTERN_7_0) + 1; - core_link_read_dpcd( - link, - DP_TEST_80BIT_CUSTOM_PATTERN_7_0, - test_pattern_buffer, - test_pattern_size); - } - - if (is_dp_phy_sqaure_pattern(test_pattern)) { - test_pattern_size = 1; // Square pattern data is 1 byte (DP spec) - core_link_read_dpcd( - link, - DP_PHY_SQUARE_PATTERN, - test_pattern_buffer, - test_pattern_size); - } - - if (test_pattern == DP_TEST_PATTERN_264BIT_CUSTOM) { - test_pattern_size = (DP_TEST_264BIT_CUSTOM_PATTERN_263_256- - DP_TEST_264BIT_CUSTOM_PATTERN_7_0) + 1; - core_link_read_dpcd( - link, - DP_TEST_264BIT_CUSTOM_PATTERN_7_0, - test_pattern_buffer, - test_pattern_size); - } - - for (lane = 0; lane < - (unsigned int)(link->cur_link_settings.lane_count); - lane++) { - dpcd_lane_adjust.raw = - get_nibble_at_index(&dpcd_lane_adjustment[0].raw, lane); - if (link_dp_get_encoding_format(&link->cur_link_settings) == - DP_8b_10b_ENCODING) { - link_training_settings.hw_lane_settings[lane].VOLTAGE_SWING = - (enum dc_voltage_swing) - (dpcd_lane_adjust.bits.VOLTAGE_SWING_LANE); - link_training_settings.hw_lane_settings[lane].PRE_EMPHASIS = - (enum dc_pre_emphasis) - (dpcd_lane_adjust.bits.PRE_EMPHASIS_LANE); - link_training_settings.hw_lane_settings[lane].POST_CURSOR2 = - (enum dc_post_cursor2) - ((dpcd_post_cursor_2_adjustment >> (lane * 2)) & 0x03); - } else if (link_dp_get_encoding_format(&link->cur_link_settings) == - DP_128b_132b_ENCODING) { - link_training_settings.hw_lane_settings[lane].FFE_PRESET.settings.level = - dpcd_lane_adjust.tx_ffe.PRESET_VALUE; - link_training_settings.hw_lane_settings[lane].FFE_PRESET.settings.no_preshoot = no_preshoot; - link_training_settings.hw_lane_settings[lane].FFE_PRESET.settings.no_deemphasis = no_deemphasis; - } - } - - dp_hw_to_dpcd_lane_settings(&link_training_settings, - link_training_settings.hw_lane_settings, - link_training_settings.dpcd_lane_settings); - /*Usage: Measure DP physical lane signal - * by DP SI test equipment automatically. - * PHY test pattern request is generated by equipment via HPD interrupt. - * HPD needs to be active all the time. HPD should be active - * all the time. Do not touch it. - * forward request to DS - */ - dc_link_dp_set_test_pattern( - link, - test_pattern, - DP_TEST_PATTERN_COLOR_SPACE_UNDEFINED, - &link_training_settings, - test_pattern_buffer, - test_pattern_size); -} - -static void dp_test_send_link_test_pattern(struct dc_link *link) -{ - union link_test_pattern dpcd_test_pattern; - union test_misc dpcd_test_params; - enum dp_test_pattern test_pattern; - enum dp_test_pattern_color_space test_pattern_color_space = - DP_TEST_PATTERN_COLOR_SPACE_UNDEFINED; - enum dc_color_depth requestColorDepth = COLOR_DEPTH_UNDEFINED; - struct pipe_ctx *pipes = link->dc->current_state->res_ctx.pipe_ctx; - struct pipe_ctx *pipe_ctx = NULL; - int i; - - memset(&dpcd_test_pattern, 0, sizeof(dpcd_test_pattern)); - memset(&dpcd_test_params, 0, sizeof(dpcd_test_params)); - - for (i = 0; i < MAX_PIPES; i++) { - if (pipes[i].stream == NULL) - continue; - - if (pipes[i].stream->link == link && !pipes[i].top_pipe && !pipes[i].prev_odm_pipe) { - pipe_ctx = &pipes[i]; - break; - } - } - - if (pipe_ctx == NULL) - return; - - /* get link test pattern and pattern parameters */ - core_link_read_dpcd( - link, - DP_TEST_PATTERN, - &dpcd_test_pattern.raw, - sizeof(dpcd_test_pattern)); - core_link_read_dpcd( - link, - DP_TEST_MISC0, - &dpcd_test_params.raw, - sizeof(dpcd_test_params)); - - switch (dpcd_test_pattern.bits.PATTERN) { - case LINK_TEST_PATTERN_COLOR_RAMP: - test_pattern = DP_TEST_PATTERN_COLOR_RAMP; - break; - case LINK_TEST_PATTERN_VERTICAL_BARS: - test_pattern = DP_TEST_PATTERN_VERTICAL_BARS; - break; /* black and white */ - case LINK_TEST_PATTERN_COLOR_SQUARES: - test_pattern = (dpcd_test_params.bits.DYN_RANGE == - TEST_DYN_RANGE_VESA ? - DP_TEST_PATTERN_COLOR_SQUARES : - DP_TEST_PATTERN_COLOR_SQUARES_CEA); - break; - default: - test_pattern = DP_TEST_PATTERN_VIDEO_MODE; - break; - } - - if (dpcd_test_params.bits.CLR_FORMAT == 0) - test_pattern_color_space = DP_TEST_PATTERN_COLOR_SPACE_RGB; - else - test_pattern_color_space = dpcd_test_params.bits.YCBCR_COEFS ? - DP_TEST_PATTERN_COLOR_SPACE_YCBCR709 : - DP_TEST_PATTERN_COLOR_SPACE_YCBCR601; - - switch (dpcd_test_params.bits.BPC) { - case 0: // 6 bits - requestColorDepth = COLOR_DEPTH_666; - break; - case 1: // 8 bits - requestColorDepth = COLOR_DEPTH_888; - break; - case 2: // 10 bits - requestColorDepth = COLOR_DEPTH_101010; - break; - case 3: // 12 bits - requestColorDepth = COLOR_DEPTH_121212; - break; - default: - break; - } - - switch (dpcd_test_params.bits.CLR_FORMAT) { - case 0: - pipe_ctx->stream->timing.pixel_encoding = PIXEL_ENCODING_RGB; - break; - case 1: - pipe_ctx->stream->timing.pixel_encoding = PIXEL_ENCODING_YCBCR422; - break; - case 2: - pipe_ctx->stream->timing.pixel_encoding = PIXEL_ENCODING_YCBCR444; - break; - default: - pipe_ctx->stream->timing.pixel_encoding = PIXEL_ENCODING_RGB; - break; - } - - - if (requestColorDepth != COLOR_DEPTH_UNDEFINED - && pipe_ctx->stream->timing.display_color_depth != requestColorDepth) { - DC_LOG_DEBUG("%s: original bpc %d, changing to %d\n", - __func__, - pipe_ctx->stream->timing.display_color_depth, - requestColorDepth); - pipe_ctx->stream->timing.display_color_depth = requestColorDepth; - } - - dp_update_dsc_config(pipe_ctx); - - dc_link_dp_set_test_pattern( - link, - test_pattern, - test_pattern_color_space, - NULL, - NULL, - 0); -} - -static void dp_test_get_audio_test_data(struct dc_link *link, bool disable_video) -{ - union audio_test_mode dpcd_test_mode = {0}; - struct audio_test_pattern_type dpcd_pattern_type = {0}; - union audio_test_pattern_period dpcd_pattern_period[AUDIO_CHANNELS_COUNT] = {0}; - enum dp_test_pattern test_pattern = DP_TEST_PATTERN_AUDIO_OPERATOR_DEFINED; - - struct pipe_ctx *pipes = link->dc->current_state->res_ctx.pipe_ctx; - struct pipe_ctx *pipe_ctx = &pipes[0]; - unsigned int channel_count; - unsigned int channel = 0; - unsigned int modes = 0; - unsigned int sampling_rate_in_hz = 0; - - // get audio test mode and test pattern parameters - core_link_read_dpcd( - link, - DP_TEST_AUDIO_MODE, - &dpcd_test_mode.raw, - sizeof(dpcd_test_mode)); - - core_link_read_dpcd( - link, - DP_TEST_AUDIO_PATTERN_TYPE, - &dpcd_pattern_type.value, - sizeof(dpcd_pattern_type)); - - channel_count = min(dpcd_test_mode.bits.channel_count + 1, AUDIO_CHANNELS_COUNT); - - // read pattern periods for requested channels when sawTooth pattern is requested - if (dpcd_pattern_type.value == AUDIO_TEST_PATTERN_SAWTOOTH || - dpcd_pattern_type.value == AUDIO_TEST_PATTERN_OPERATOR_DEFINED) { - - test_pattern = (dpcd_pattern_type.value == AUDIO_TEST_PATTERN_SAWTOOTH) ? - DP_TEST_PATTERN_AUDIO_SAWTOOTH : DP_TEST_PATTERN_AUDIO_OPERATOR_DEFINED; - // read period for each channel - for (channel = 0; channel < channel_count; channel++) { - core_link_read_dpcd( - link, - DP_TEST_AUDIO_PERIOD_CH1 + channel, - &dpcd_pattern_period[channel].raw, - sizeof(dpcd_pattern_period[channel])); - } - } - - // translate sampling rate - switch (dpcd_test_mode.bits.sampling_rate) { - case AUDIO_SAMPLING_RATE_32KHZ: - sampling_rate_in_hz = 32000; - break; - case AUDIO_SAMPLING_RATE_44_1KHZ: - sampling_rate_in_hz = 44100; - break; - case AUDIO_SAMPLING_RATE_48KHZ: - sampling_rate_in_hz = 48000; - break; - case AUDIO_SAMPLING_RATE_88_2KHZ: - sampling_rate_in_hz = 88200; - break; - case AUDIO_SAMPLING_RATE_96KHZ: - sampling_rate_in_hz = 96000; - break; - case AUDIO_SAMPLING_RATE_176_4KHZ: - sampling_rate_in_hz = 176400; - break; - case AUDIO_SAMPLING_RATE_192KHZ: - sampling_rate_in_hz = 192000; - break; - default: - sampling_rate_in_hz = 0; - break; - } - - link->audio_test_data.flags.test_requested = 1; - link->audio_test_data.flags.disable_video = disable_video; - link->audio_test_data.sampling_rate = sampling_rate_in_hz; - link->audio_test_data.channel_count = channel_count; - link->audio_test_data.pattern_type = test_pattern; - - if (test_pattern == DP_TEST_PATTERN_AUDIO_SAWTOOTH) { - for (modes = 0; modes < pipe_ctx->stream->audio_info.mode_count; modes++) { - link->audio_test_data.pattern_period[modes] = dpcd_pattern_period[modes].bits.pattern_period; - } - } -} - -void dc_link_dp_handle_automated_test(struct dc_link *link) -{ - union test_request test_request; - union test_response test_response; - - memset(&test_request, 0, sizeof(test_request)); - memset(&test_response, 0, sizeof(test_response)); - - core_link_read_dpcd( - link, - DP_TEST_REQUEST, - &test_request.raw, - sizeof(union test_request)); - if (test_request.bits.LINK_TRAINING) { - /* ACK first to let DP RX test box monitor LT sequence */ - test_response.bits.ACK = 1; - core_link_write_dpcd( - link, - DP_TEST_RESPONSE, - &test_response.raw, - sizeof(test_response)); - dp_test_send_link_training(link); - /* no acknowledge request is needed again */ - test_response.bits.ACK = 0; - } - if (test_request.bits.LINK_TEST_PATTRN) { - dp_test_send_link_test_pattern(link); - test_response.bits.ACK = 1; - } - - if (test_request.bits.AUDIO_TEST_PATTERN) { - dp_test_get_audio_test_data(link, test_request.bits.TEST_AUDIO_DISABLED_VIDEO); - test_response.bits.ACK = 1; - } - - if (test_request.bits.PHY_TEST_PATTERN) { - dp_test_send_phy_test_pattern(link); - test_response.bits.ACK = 1; - } - - /* send request acknowledgment */ - if (test_response.bits.ACK) - core_link_write_dpcd( - link, - DP_TEST_RESPONSE, - &test_response.raw, - sizeof(test_response)); -} - -void dc_link_dp_handle_link_loss(struct dc_link *link) -{ - int i; - struct pipe_ctx *pipe_ctx; - - for (i = 0; i < MAX_PIPES; i++) { - pipe_ctx = &link->dc->current_state->res_ctx.pipe_ctx[i]; - if (pipe_ctx && pipe_ctx->stream && pipe_ctx->stream->link == link) - break; - } - - if (pipe_ctx == NULL || pipe_ctx->stream == NULL) - return; - - for (i = 0; i < MAX_PIPES; i++) { - pipe_ctx = &link->dc->current_state->res_ctx.pipe_ctx[i]; - if (pipe_ctx && pipe_ctx->stream && !pipe_ctx->stream->dpms_off && - pipe_ctx->stream->link == link && !pipe_ctx->prev_odm_pipe) - core_link_disable_stream(pipe_ctx); - } - - for (i = 0; i < MAX_PIPES; i++) { - pipe_ctx = &link->dc->current_state->res_ctx.pipe_ctx[i]; - if (pipe_ctx && pipe_ctx->stream && !pipe_ctx->stream->dpms_off - && pipe_ctx->stream->link == link && !pipe_ctx->prev_odm_pipe) { - // Always use max settings here for DP 1.4a LL Compliance CTS - if (link->is_automated) { - pipe_ctx->link_config.dp_link_settings.lane_count = - link->verified_link_cap.lane_count; - pipe_ctx->link_config.dp_link_settings.link_rate = - link->verified_link_cap.link_rate; - pipe_ctx->link_config.dp_link_settings.link_spread = - link->verified_link_cap.link_spread; - } - core_link_enable_stream(link->dc->current_state, pipe_ctx); - } - } -} - -bool dc_link_handle_hpd_rx_irq(struct dc_link *link, union hpd_irq_data *out_hpd_irq_dpcd_data, bool *out_link_loss, - bool defer_handling, bool *has_left_work) -{ - union hpd_irq_data hpd_irq_dpcd_data = {0}; - union device_service_irq device_service_clear = {0}; - enum dc_status result; - bool status = false; - - if (out_link_loss) - *out_link_loss = false; - - if (has_left_work) - *has_left_work = false; - /* For use cases related to down stream connection status change, - * PSR and device auto test, refer to function handle_sst_hpd_irq - * in DAL2.1*/ - - DC_LOG_HW_HPD_IRQ("%s: Got short pulse HPD on link %d\n", - __func__, link->link_index); - - - /* All the "handle_hpd_irq_xxx()" methods - * should be called only after - * dal_dpsst_ls_read_hpd_irq_data - * Order of calls is important too - */ - result = read_hpd_rx_irq_data(link, &hpd_irq_dpcd_data); - if (out_hpd_irq_dpcd_data) - *out_hpd_irq_dpcd_data = hpd_irq_dpcd_data; - - if (result != DC_OK) { - DC_LOG_HW_HPD_IRQ("%s: DPCD read failed to obtain irq data\n", - __func__); - return false; - } - - if (hpd_irq_dpcd_data.bytes.device_service_irq.bits.AUTOMATED_TEST) { - // Workaround for DP 1.4a LL Compliance CTS as USB4 has to share encoders unlike DP and USBC - link->is_automated = true; - device_service_clear.bits.AUTOMATED_TEST = 1; - core_link_write_dpcd( - link, - DP_DEVICE_SERVICE_IRQ_VECTOR, - &device_service_clear.raw, - sizeof(device_service_clear.raw)); - device_service_clear.raw = 0; - if (defer_handling && has_left_work) - *has_left_work = true; - else - dc_link_dp_handle_automated_test(link); - return false; - } - - if (!dc_link_dp_allow_hpd_rx_irq(link)) { - DC_LOG_HW_HPD_IRQ("%s: skipping HPD handling on %d\n", - __func__, link->link_index); - return false; - } - - if (handle_hpd_irq_psr_sink(link)) - /* PSR-related error was detected and handled */ - return true; - - /* If PSR-related error handled, Main link may be off, - * so do not handle as a normal sink status change interrupt. - */ - - if (hpd_irq_dpcd_data.bytes.device_service_irq.bits.UP_REQ_MSG_RDY) { - if (defer_handling && has_left_work) - *has_left_work = true; - return true; - } - - /* check if we have MST msg and return since we poll for it */ - if (hpd_irq_dpcd_data.bytes.device_service_irq.bits.DOWN_REP_MSG_RDY) { - if (defer_handling && has_left_work) - *has_left_work = true; - return false; - } - - /* For now we only handle 'Downstream port status' case. - * If we got sink count changed it means - * Downstream port status changed, - * then DM should call DC to do the detection. - * NOTE: Do not handle link loss on eDP since it is internal link*/ - if ((link->connector_signal != SIGNAL_TYPE_EDP) && - hpd_rx_irq_check_link_loss_status( - link, - &hpd_irq_dpcd_data)) { - /* Connectivity log: link loss */ - CONN_DATA_LINK_LOSS(link, - hpd_irq_dpcd_data.raw, - sizeof(hpd_irq_dpcd_data), - "Status: "); - - if (defer_handling && has_left_work) - *has_left_work = true; - else - dc_link_dp_handle_link_loss(link); - - status = false; - if (out_link_loss) - *out_link_loss = true; - - dp_trace_link_loss_increment(link); - } - - if (link->type == dc_connection_sst_branch && - hpd_irq_dpcd_data.bytes.sink_cnt.bits.SINK_COUNT - != link->dpcd_sink_count) - status = true; - - /* reasons for HPD RX: - * 1. Link Loss - ie Re-train the Link - * 2. MST sideband message - * 3. Automated Test - ie. Internal Commit - * 4. CP (copy protection) - (not interesting for DM???) - * 5. DRR - * 6. Downstream Port status changed - * -ie. Detect - this the only one - * which is interesting for DM because - * it must call dc_link_detect. - */ - return status; -} - -static bool is_dp_phy_pattern(enum dp_test_pattern test_pattern) -{ - if ((DP_TEST_PATTERN_PHY_PATTERN_BEGIN <= test_pattern && - test_pattern <= DP_TEST_PATTERN_PHY_PATTERN_END) || - test_pattern == DP_TEST_PATTERN_VIDEO_MODE) - return true; - else - return false; -} - -static void set_crtc_test_pattern(struct dc_link *link, - struct pipe_ctx *pipe_ctx, - enum dp_test_pattern test_pattern, - enum dp_test_pattern_color_space test_pattern_color_space) -{ - enum controller_dp_test_pattern controller_test_pattern; - enum dc_color_depth color_depth = pipe_ctx-> - stream->timing.display_color_depth; - struct bit_depth_reduction_params params; - struct output_pixel_processor *opp = pipe_ctx->stream_res.opp; - int width = pipe_ctx->stream->timing.h_addressable + - pipe_ctx->stream->timing.h_border_left + - pipe_ctx->stream->timing.h_border_right; - int height = pipe_ctx->stream->timing.v_addressable + - pipe_ctx->stream->timing.v_border_bottom + - pipe_ctx->stream->timing.v_border_top; - - memset(¶ms, 0, sizeof(params)); - - switch (test_pattern) { - case DP_TEST_PATTERN_COLOR_SQUARES: - controller_test_pattern = - CONTROLLER_DP_TEST_PATTERN_COLORSQUARES; - break; - case DP_TEST_PATTERN_COLOR_SQUARES_CEA: - controller_test_pattern = - CONTROLLER_DP_TEST_PATTERN_COLORSQUARES_CEA; - break; - case DP_TEST_PATTERN_VERTICAL_BARS: - controller_test_pattern = - CONTROLLER_DP_TEST_PATTERN_VERTICALBARS; - break; - case DP_TEST_PATTERN_HORIZONTAL_BARS: - controller_test_pattern = - CONTROLLER_DP_TEST_PATTERN_HORIZONTALBARS; - break; - case DP_TEST_PATTERN_COLOR_RAMP: - controller_test_pattern = - CONTROLLER_DP_TEST_PATTERN_COLORRAMP; - break; - default: - controller_test_pattern = - CONTROLLER_DP_TEST_PATTERN_VIDEOMODE; - break; - } - - switch (test_pattern) { - case DP_TEST_PATTERN_COLOR_SQUARES: - case DP_TEST_PATTERN_COLOR_SQUARES_CEA: - case DP_TEST_PATTERN_VERTICAL_BARS: - case DP_TEST_PATTERN_HORIZONTAL_BARS: - case DP_TEST_PATTERN_COLOR_RAMP: - { - /* disable bit depth reduction */ - pipe_ctx->stream->bit_depth_params = params; - opp->funcs->opp_program_bit_depth_reduction(opp, ¶ms); - if (pipe_ctx->stream_res.tg->funcs->set_test_pattern) - pipe_ctx->stream_res.tg->funcs->set_test_pattern(pipe_ctx->stream_res.tg, - controller_test_pattern, color_depth); - else if (link->dc->hwss.set_disp_pattern_generator) { - struct pipe_ctx *odm_pipe; - enum controller_dp_color_space controller_color_space; - int opp_cnt = 1; - int offset = 0; - int dpg_width = width; - - switch (test_pattern_color_space) { - case DP_TEST_PATTERN_COLOR_SPACE_RGB: - controller_color_space = CONTROLLER_DP_COLOR_SPACE_RGB; - break; - case DP_TEST_PATTERN_COLOR_SPACE_YCBCR601: - controller_color_space = CONTROLLER_DP_COLOR_SPACE_YCBCR601; - break; - case DP_TEST_PATTERN_COLOR_SPACE_YCBCR709: - controller_color_space = CONTROLLER_DP_COLOR_SPACE_YCBCR709; - break; - case DP_TEST_PATTERN_COLOR_SPACE_UNDEFINED: - default: - controller_color_space = CONTROLLER_DP_COLOR_SPACE_UDEFINED; - DC_LOG_ERROR("%s: Color space must be defined for test pattern", __func__); - ASSERT(0); - break; - } - - for (odm_pipe = pipe_ctx->next_odm_pipe; odm_pipe; odm_pipe = odm_pipe->next_odm_pipe) - opp_cnt++; - dpg_width = width / opp_cnt; - offset = dpg_width; - - link->dc->hwss.set_disp_pattern_generator(link->dc, - pipe_ctx, - controller_test_pattern, - controller_color_space, - color_depth, - NULL, - dpg_width, - height, - 0); - - for (odm_pipe = pipe_ctx->next_odm_pipe; odm_pipe; odm_pipe = odm_pipe->next_odm_pipe) { - struct output_pixel_processor *odm_opp = odm_pipe->stream_res.opp; - - odm_opp->funcs->opp_program_bit_depth_reduction(odm_opp, ¶ms); - link->dc->hwss.set_disp_pattern_generator(link->dc, - odm_pipe, - controller_test_pattern, - controller_color_space, - color_depth, - NULL, - dpg_width, - height, - offset); - offset += offset; - } - } - } - break; - case DP_TEST_PATTERN_VIDEO_MODE: - { - /* restore bitdepth reduction */ - resource_build_bit_depth_reduction_params(pipe_ctx->stream, ¶ms); - pipe_ctx->stream->bit_depth_params = params; - opp->funcs->opp_program_bit_depth_reduction(opp, ¶ms); - if (pipe_ctx->stream_res.tg->funcs->set_test_pattern) - pipe_ctx->stream_res.tg->funcs->set_test_pattern(pipe_ctx->stream_res.tg, - CONTROLLER_DP_TEST_PATTERN_VIDEOMODE, - color_depth); - else if (link->dc->hwss.set_disp_pattern_generator) { - struct pipe_ctx *odm_pipe; - int opp_cnt = 1; - int dpg_width; - - for (odm_pipe = pipe_ctx->next_odm_pipe; odm_pipe; odm_pipe = odm_pipe->next_odm_pipe) - opp_cnt++; - - dpg_width = width / opp_cnt; - for (odm_pipe = pipe_ctx->next_odm_pipe; odm_pipe; odm_pipe = odm_pipe->next_odm_pipe) { - struct output_pixel_processor *odm_opp = odm_pipe->stream_res.opp; - - odm_opp->funcs->opp_program_bit_depth_reduction(odm_opp, ¶ms); - link->dc->hwss.set_disp_pattern_generator(link->dc, - odm_pipe, - CONTROLLER_DP_TEST_PATTERN_VIDEOMODE, - CONTROLLER_DP_COLOR_SPACE_UDEFINED, - color_depth, - NULL, - dpg_width, - height, - 0); - } - link->dc->hwss.set_disp_pattern_generator(link->dc, - pipe_ctx, - CONTROLLER_DP_TEST_PATTERN_VIDEOMODE, - CONTROLLER_DP_COLOR_SPACE_UDEFINED, - color_depth, - NULL, - dpg_width, - height, - 0); - } - } - break; - - default: - break; - } -} - -bool dc_link_dp_set_test_pattern( - struct dc_link *link, - enum dp_test_pattern test_pattern, - enum dp_test_pattern_color_space test_pattern_color_space, - const struct link_training_settings *p_link_settings, - const unsigned char *p_custom_pattern, - unsigned int cust_pattern_size) -{ - struct pipe_ctx *pipes = link->dc->current_state->res_ctx.pipe_ctx; - struct pipe_ctx *pipe_ctx = NULL; - unsigned int lane; - unsigned int i; - unsigned char link_qual_pattern[LANE_COUNT_DP_MAX] = {0}; - union dpcd_training_pattern training_pattern; - enum dpcd_phy_test_patterns pattern; - - memset(&training_pattern, 0, sizeof(training_pattern)); - - for (i = 0; i < MAX_PIPES; i++) { - if (pipes[i].stream == NULL) - continue; - - if (pipes[i].stream->link == link && !pipes[i].top_pipe && !pipes[i].prev_odm_pipe) { - pipe_ctx = &pipes[i]; - break; - } - } - - if (pipe_ctx == NULL) - return false; - - /* Reset CRTC Test Pattern if it is currently running and request is VideoMode */ - if (link->test_pattern_enabled && test_pattern == - DP_TEST_PATTERN_VIDEO_MODE) { - /* Set CRTC Test Pattern */ - set_crtc_test_pattern(link, pipe_ctx, test_pattern, test_pattern_color_space); - dp_set_hw_test_pattern(link, &pipe_ctx->link_res, test_pattern, - (uint8_t *)p_custom_pattern, - (uint32_t)cust_pattern_size); - - /* Unblank Stream */ - link->dc->hwss.unblank_stream( - pipe_ctx, - &link->verified_link_cap); - /* TODO:m_pHwss->MuteAudioEndpoint - * (pPathMode->pDisplayPath, false); - */ - - /* Reset Test Pattern state */ - link->test_pattern_enabled = false; - - return true; - } - - /* Check for PHY Test Patterns */ - if (is_dp_phy_pattern(test_pattern)) { - /* Set DPCD Lane Settings before running test pattern */ - if (p_link_settings != NULL) { - if ((link->chip_caps & EXT_DISPLAY_PATH_CAPS__DP_FIXED_VS_EN) && - p_link_settings->lttpr_mode == LTTPR_MODE_TRANSPARENT) { - dp_fixed_vs_pe_set_retimer_lane_settings( - link, - p_link_settings->dpcd_lane_settings, - p_link_settings->link_settings.lane_count); - } else { - dp_set_hw_lane_settings(link, &pipe_ctx->link_res, p_link_settings, DPRX); - } - dpcd_set_lane_settings(link, p_link_settings, DPRX); - } - - /* Blank stream if running test pattern */ - if (test_pattern != DP_TEST_PATTERN_VIDEO_MODE) { - /*TODO: - * m_pHwss-> - * MuteAudioEndpoint(pPathMode->pDisplayPath, true); - */ - /* Blank stream */ - link->dc->hwss.blank_stream(pipe_ctx); - } - - dp_set_hw_test_pattern(link, &pipe_ctx->link_res, test_pattern, - (uint8_t *)p_custom_pattern, - (uint32_t)cust_pattern_size); - - if (test_pattern != DP_TEST_PATTERN_VIDEO_MODE) { - /* Set Test Pattern state */ - link->test_pattern_enabled = true; - if (p_link_settings != NULL) - dpcd_set_link_settings(link, - p_link_settings); - } - - switch (test_pattern) { - case DP_TEST_PATTERN_VIDEO_MODE: - pattern = PHY_TEST_PATTERN_NONE; - break; - case DP_TEST_PATTERN_D102: - pattern = PHY_TEST_PATTERN_D10_2; - break; - case DP_TEST_PATTERN_SYMBOL_ERROR: - pattern = PHY_TEST_PATTERN_SYMBOL_ERROR; - break; - case DP_TEST_PATTERN_PRBS7: - pattern = PHY_TEST_PATTERN_PRBS7; - break; - case DP_TEST_PATTERN_80BIT_CUSTOM: - pattern = PHY_TEST_PATTERN_80BIT_CUSTOM; - break; - case DP_TEST_PATTERN_CP2520_1: - pattern = PHY_TEST_PATTERN_CP2520_1; - break; - case DP_TEST_PATTERN_CP2520_2: - pattern = PHY_TEST_PATTERN_CP2520_2; - break; - case DP_TEST_PATTERN_CP2520_3: - pattern = PHY_TEST_PATTERN_CP2520_3; - break; - case DP_TEST_PATTERN_128b_132b_TPS1: - pattern = PHY_TEST_PATTERN_128b_132b_TPS1; - break; - case DP_TEST_PATTERN_128b_132b_TPS2: - pattern = PHY_TEST_PATTERN_128b_132b_TPS2; - break; - case DP_TEST_PATTERN_PRBS9: - pattern = PHY_TEST_PATTERN_PRBS9; - break; - case DP_TEST_PATTERN_PRBS11: - pattern = PHY_TEST_PATTERN_PRBS11; - break; - case DP_TEST_PATTERN_PRBS15: - pattern = PHY_TEST_PATTERN_PRBS15; - break; - case DP_TEST_PATTERN_PRBS23: - pattern = PHY_TEST_PATTERN_PRBS23; - break; - case DP_TEST_PATTERN_PRBS31: - pattern = PHY_TEST_PATTERN_PRBS31; - break; - case DP_TEST_PATTERN_264BIT_CUSTOM: - pattern = PHY_TEST_PATTERN_264BIT_CUSTOM; - break; - case DP_TEST_PATTERN_SQUARE: - pattern = PHY_TEST_PATTERN_SQUARE; - break; - case DP_TEST_PATTERN_SQUARE_PRESHOOT_DISABLED: - pattern = PHY_TEST_PATTERN_SQUARE_PRESHOOT_DISABLED; - break; - case DP_TEST_PATTERN_SQUARE_DEEMPHASIS_DISABLED: - pattern = PHY_TEST_PATTERN_SQUARE_DEEMPHASIS_DISABLED; - break; - case DP_TEST_PATTERN_SQUARE_PRESHOOT_DEEMPHASIS_DISABLED: - pattern = PHY_TEST_PATTERN_SQUARE_PRESHOOT_DEEMPHASIS_DISABLED; - break; - default: - return false; - } - - if (test_pattern == DP_TEST_PATTERN_VIDEO_MODE - /*TODO:&& !pPathMode->pDisplayPath->IsTargetPoweredOn()*/) - return false; - - if (link->dpcd_caps.dpcd_rev.raw >= DPCD_REV_12) { - if (is_dp_phy_sqaure_pattern(test_pattern)) - core_link_write_dpcd(link, - DP_LINK_SQUARE_PATTERN, - p_custom_pattern, - 1); - - /* tell receiver that we are sending qualification - * pattern DP 1.2 or later - DP receiver's link quality - * pattern is set using DPCD LINK_QUAL_LANEx_SET - * register (0x10B~0x10E)\ - */ - for (lane = 0; lane < LANE_COUNT_DP_MAX; lane++) - link_qual_pattern[lane] = - (unsigned char)(pattern); - - core_link_write_dpcd(link, - DP_LINK_QUAL_LANE0_SET, - link_qual_pattern, - sizeof(link_qual_pattern)); - } else if (link->dpcd_caps.dpcd_rev.raw >= DPCD_REV_10 || - link->dpcd_caps.dpcd_rev.raw == 0) { - /* tell receiver that we are sending qualification - * pattern DP 1.1a or earlier - DP receiver's link - * quality pattern is set using - * DPCD TRAINING_PATTERN_SET -> LINK_QUAL_PATTERN_SET - * register (0x102). We will use v_1.3 when we are - * setting test pattern for DP 1.1. - */ - core_link_read_dpcd(link, DP_TRAINING_PATTERN_SET, - &training_pattern.raw, - sizeof(training_pattern)); - training_pattern.v1_3.LINK_QUAL_PATTERN_SET = pattern; - core_link_write_dpcd(link, DP_TRAINING_PATTERN_SET, - &training_pattern.raw, - sizeof(training_pattern)); - } - } else { - enum dc_color_space color_space = COLOR_SPACE_UNKNOWN; - - switch (test_pattern_color_space) { - case DP_TEST_PATTERN_COLOR_SPACE_RGB: - color_space = COLOR_SPACE_SRGB; - if (test_pattern == DP_TEST_PATTERN_COLOR_SQUARES_CEA) - color_space = COLOR_SPACE_SRGB_LIMITED; - break; - - case DP_TEST_PATTERN_COLOR_SPACE_YCBCR601: - color_space = COLOR_SPACE_YCBCR601; - if (test_pattern == DP_TEST_PATTERN_COLOR_SQUARES_CEA) - color_space = COLOR_SPACE_YCBCR601_LIMITED; - break; - case DP_TEST_PATTERN_COLOR_SPACE_YCBCR709: - color_space = COLOR_SPACE_YCBCR709; - if (test_pattern == DP_TEST_PATTERN_COLOR_SQUARES_CEA) - color_space = COLOR_SPACE_YCBCR709_LIMITED; - break; - default: - break; - } - - if (pipe_ctx->stream_res.tg->funcs->lock_doublebuffer_enable) { - if (pipe_ctx->stream && should_use_dmub_lock(pipe_ctx->stream->link)) { - union dmub_hw_lock_flags hw_locks = { 0 }; - struct dmub_hw_lock_inst_flags inst_flags = { 0 }; - - hw_locks.bits.lock_dig = 1; - inst_flags.dig_inst = pipe_ctx->stream_res.tg->inst; - - dmub_hw_lock_mgr_cmd(link->ctx->dmub_srv, - true, - &hw_locks, - &inst_flags); - } else - pipe_ctx->stream_res.tg->funcs->lock_doublebuffer_enable( - pipe_ctx->stream_res.tg); - } - - pipe_ctx->stream_res.tg->funcs->lock(pipe_ctx->stream_res.tg); - /* update MSA to requested color space */ - pipe_ctx->stream_res.stream_enc->funcs->dp_set_stream_attribute(pipe_ctx->stream_res.stream_enc, - &pipe_ctx->stream->timing, - color_space, - pipe_ctx->stream->use_vsc_sdp_for_colorimetry, - link->dpcd_caps.dprx_feature.bits.SST_SPLIT_SDP_CAP); - - if (pipe_ctx->stream->use_vsc_sdp_for_colorimetry) { - if (test_pattern == DP_TEST_PATTERN_COLOR_SQUARES_CEA) - pipe_ctx->stream->vsc_infopacket.sb[17] |= (1 << 7); // sb17 bit 7 Dynamic Range: 0 = VESA range, 1 = CTA range - else - pipe_ctx->stream->vsc_infopacket.sb[17] &= ~(1 << 7); - resource_build_info_frame(pipe_ctx); - link->dc->hwss.update_info_frame(pipe_ctx); - } - - /* CRTC Patterns */ - set_crtc_test_pattern(link, pipe_ctx, test_pattern, test_pattern_color_space); - pipe_ctx->stream_res.tg->funcs->unlock(pipe_ctx->stream_res.tg); - pipe_ctx->stream_res.tg->funcs->wait_for_state(pipe_ctx->stream_res.tg, - CRTC_STATE_VACTIVE); - pipe_ctx->stream_res.tg->funcs->wait_for_state(pipe_ctx->stream_res.tg, - CRTC_STATE_VBLANK); - pipe_ctx->stream_res.tg->funcs->wait_for_state(pipe_ctx->stream_res.tg, - CRTC_STATE_VACTIVE); - - if (pipe_ctx->stream_res.tg->funcs->lock_doublebuffer_disable) { - if (pipe_ctx->stream && should_use_dmub_lock(pipe_ctx->stream->link)) { - union dmub_hw_lock_flags hw_locks = { 0 }; - struct dmub_hw_lock_inst_flags inst_flags = { 0 }; - - hw_locks.bits.lock_dig = 1; - inst_flags.dig_inst = pipe_ctx->stream_res.tg->inst; - - dmub_hw_lock_mgr_cmd(link->ctx->dmub_srv, - false, - &hw_locks, - &inst_flags); - } else - pipe_ctx->stream_res.tg->funcs->lock_doublebuffer_disable( - pipe_ctx->stream_res.tg); - } - - /* Set Test Pattern state */ - link->test_pattern_enabled = true; - } - - return true; -} - -void dp_enable_mst_on_sink(struct dc_link *link, bool enable) -{ - unsigned char mstmCntl; - - core_link_read_dpcd(link, DP_MSTM_CTRL, &mstmCntl, 1); - if (enable) - mstmCntl |= DP_MST_EN; - else - mstmCntl &= (~DP_MST_EN); - - core_link_write_dpcd(link, DP_MSTM_CTRL, &mstmCntl, 1); -} - -void dp_set_panel_mode(struct dc_link *link, enum dp_panel_mode panel_mode) -{ - union dpcd_edp_config edp_config_set; - bool panel_mode_edp = false; - - memset(&edp_config_set, '\0', sizeof(union dpcd_edp_config)); - - if (panel_mode != DP_PANEL_MODE_DEFAULT) { - - switch (panel_mode) { - case DP_PANEL_MODE_EDP: - case DP_PANEL_MODE_SPECIAL: - panel_mode_edp = true; - break; - - default: - break; - } - - /*set edp panel mode in receiver*/ - core_link_read_dpcd( - link, - DP_EDP_CONFIGURATION_SET, - &edp_config_set.raw, - sizeof(edp_config_set.raw)); - - if (edp_config_set.bits.PANEL_MODE_EDP - != panel_mode_edp) { - enum dc_status result; - - edp_config_set.bits.PANEL_MODE_EDP = - panel_mode_edp; - result = core_link_write_dpcd( - link, - DP_EDP_CONFIGURATION_SET, - &edp_config_set.raw, - sizeof(edp_config_set.raw)); - - ASSERT(result == DC_OK); - } - } - DC_LOG_DETECTION_DP_CAPS("Link: %d eDP panel mode supported: %d " - "eDP panel mode enabled: %d \n", - link->link_index, - link->dpcd_caps.panel_mode_edp, - panel_mode_edp); -} - -enum dp_panel_mode dp_get_panel_mode(struct dc_link *link) -{ - /* We need to explicitly check that connector - * is not DP. Some Travis_VGA get reported - * by video bios as DP. - */ - if (link->connector_signal != SIGNAL_TYPE_DISPLAY_PORT) { - - switch (link->dpcd_caps.branch_dev_id) { - case DP_BRANCH_DEVICE_ID_0022B9: - /* alternate scrambler reset is required for Travis - * for the case when external chip does not - * provide sink device id, alternate scrambler - * scheme will be overriden later by querying - * Encoder features - */ - if (strncmp( - link->dpcd_caps.branch_dev_name, - DP_VGA_LVDS_CONVERTER_ID_2, - sizeof( - link->dpcd_caps. - branch_dev_name)) == 0) { - return DP_PANEL_MODE_SPECIAL; - } - break; - case DP_BRANCH_DEVICE_ID_00001A: - /* alternate scrambler reset is required for Travis - * for the case when external chip does not provide - * sink device id, alternate scrambler scheme will - * be overriden later by querying Encoder feature - */ - if (strncmp(link->dpcd_caps.branch_dev_name, - DP_VGA_LVDS_CONVERTER_ID_3, - sizeof( - link->dpcd_caps. - branch_dev_name)) == 0) { - return DP_PANEL_MODE_SPECIAL; - } - break; - default: - break; - } - } - - if (link->dpcd_caps.panel_mode_edp && - (link->connector_signal == SIGNAL_TYPE_EDP || - (link->connector_signal == SIGNAL_TYPE_DISPLAY_PORT && - link->is_internal_display))) { - return DP_PANEL_MODE_EDP; - } - - return DP_PANEL_MODE_DEFAULT; -} - -enum dc_status dp_set_fec_ready(struct dc_link *link, const struct link_resource *link_res, bool ready) -{ - /* FEC has to be "set ready" before the link training. - * The policy is to always train with FEC - * if the sink supports it and leave it enabled on link. - * If FEC is not supported, disable it. - */ - struct link_encoder *link_enc = NULL; - enum dc_status status = DC_OK; - uint8_t fec_config = 0; - - link_enc = link_enc_cfg_get_link_enc(link); - ASSERT(link_enc); - - if (!dc_link_should_enable_fec(link)) - return status; - - if (link_enc->funcs->fec_set_ready && - link->dpcd_caps.fec_cap.bits.FEC_CAPABLE) { - if (ready) { - fec_config = 1; - status = core_link_write_dpcd(link, - DP_FEC_CONFIGURATION, - &fec_config, - sizeof(fec_config)); - if (status == DC_OK) { - link_enc->funcs->fec_set_ready(link_enc, true); - link->fec_state = dc_link_fec_ready; - } else { - link_enc->funcs->fec_set_ready(link_enc, false); - link->fec_state = dc_link_fec_not_ready; - dm_error("dpcd write failed to set fec_ready"); - } - } else if (link->fec_state == dc_link_fec_ready) { - fec_config = 0; - status = core_link_write_dpcd(link, - DP_FEC_CONFIGURATION, - &fec_config, - sizeof(fec_config)); - link_enc->funcs->fec_set_ready(link_enc, false); - link->fec_state = dc_link_fec_not_ready; - } - } - - return status; -} - -void dp_set_fec_enable(struct dc_link *link, bool enable) -{ - struct link_encoder *link_enc = NULL; - - link_enc = link_enc_cfg_get_link_enc(link); - ASSERT(link_enc); - - if (!dc_link_should_enable_fec(link)) - return; - - if (link_enc->funcs->fec_set_enable && - link->dpcd_caps.fec_cap.bits.FEC_CAPABLE) { - if (link->fec_state == dc_link_fec_ready && enable) { - /* Accord to DP spec, FEC enable sequence can first - * be transmitted anytime after 1000 LL codes have - * been transmitted on the link after link training - * completion. Using 1 lane RBR should have the maximum - * time for transmitting 1000 LL codes which is 6.173 us. - * So use 7 microseconds delay instead. - */ - udelay(7); - link_enc->funcs->fec_set_enable(link_enc, true); - link->fec_state = dc_link_fec_enabled; - } else if (link->fec_state == dc_link_fec_enabled && !enable) { - link_enc->funcs->fec_set_enable(link_enc, false); - link->fec_state = dc_link_fec_ready; - } - } -} - -bool dc_link_set_backlight_level_nits(struct dc_link *link, - bool isHDR, - uint32_t backlight_millinits, - uint32_t transition_time_in_ms) -{ - struct dpcd_source_backlight_set dpcd_backlight_set; - uint8_t backlight_control = isHDR ? 1 : 0; - - if (!link || (link->connector_signal != SIGNAL_TYPE_EDP && - link->connector_signal != SIGNAL_TYPE_DISPLAY_PORT)) - return false; - - // OLEDs have no PWM, they can only use AUX - if (link->dpcd_sink_ext_caps.bits.oled == 1) - backlight_control = 1; - - *(uint32_t *)&dpcd_backlight_set.backlight_level_millinits = backlight_millinits; - *(uint16_t *)&dpcd_backlight_set.backlight_transition_time_ms = (uint16_t)transition_time_in_ms; - - - if (core_link_write_dpcd(link, DP_SOURCE_BACKLIGHT_LEVEL, - (uint8_t *)(&dpcd_backlight_set), - sizeof(dpcd_backlight_set)) != DC_OK) - return false; - - if (core_link_write_dpcd(link, DP_SOURCE_BACKLIGHT_CONTROL, - &backlight_control, 1) != DC_OK) - return false; - - return true; -} - -bool dc_link_get_backlight_level_nits(struct dc_link *link, - uint32_t *backlight_millinits_avg, - uint32_t *backlight_millinits_peak) -{ - union dpcd_source_backlight_get dpcd_backlight_get; - - memset(&dpcd_backlight_get, 0, sizeof(union dpcd_source_backlight_get)); - - if (!link || (link->connector_signal != SIGNAL_TYPE_EDP && - link->connector_signal != SIGNAL_TYPE_DISPLAY_PORT)) - return false; - - if (!core_link_read_dpcd(link, DP_SOURCE_BACKLIGHT_CURRENT_PEAK, - dpcd_backlight_get.raw, - sizeof(union dpcd_source_backlight_get))) - return false; - - *backlight_millinits_avg = - dpcd_backlight_get.bytes.backlight_millinits_avg; - *backlight_millinits_peak = - dpcd_backlight_get.bytes.backlight_millinits_peak; - - /* On non-supported panels dpcd_read usually succeeds with 0 returned */ - if (*backlight_millinits_avg == 0 || - *backlight_millinits_avg > *backlight_millinits_peak) - return false; - - return true; -} - -bool dc_link_backlight_enable_aux(struct dc_link *link, bool enable) -{ - uint8_t backlight_enable = enable ? 1 : 0; - - if (!link || (link->connector_signal != SIGNAL_TYPE_EDP && - link->connector_signal != SIGNAL_TYPE_DISPLAY_PORT)) - return false; - - if (core_link_write_dpcd(link, DP_SOURCE_BACKLIGHT_ENABLE, - &backlight_enable, 1) != DC_OK) - return false; - - return true; -} - -// we read default from 0x320 because we expect BIOS wrote it there -// regular get_backlight_nit reads from panel set at 0x326 -bool dc_link_read_default_bl_aux(struct dc_link *link, uint32_t *backlight_millinits) -{ - if (!link || (link->connector_signal != SIGNAL_TYPE_EDP && - link->connector_signal != SIGNAL_TYPE_DISPLAY_PORT)) - return false; - - if (!core_link_read_dpcd(link, DP_SOURCE_BACKLIGHT_LEVEL, - (uint8_t *) backlight_millinits, - sizeof(uint32_t))) - return false; - - return true; -} - -bool dc_link_set_default_brightness_aux(struct dc_link *link) -{ - uint32_t default_backlight; - - if (link && link->dpcd_sink_ext_caps.bits.oled == 1) { - if (!dc_link_read_default_bl_aux(link, &default_backlight)) - default_backlight = 150000; - // if < 5 nits or > 5000, it might be wrong readback - if (default_backlight < 5000 || default_backlight > 5000000) - default_backlight = 150000; // - - return dc_link_set_backlight_level_nits(link, true, - default_backlight, 0); - } - return false; -} - -bool is_edp_ilr_optimization_required(struct dc_link *link, struct dc_crtc_timing *crtc_timing) -{ - struct dc_link_settings link_setting; - uint8_t link_bw_set; - uint8_t link_rate_set; - uint32_t req_bw; - union lane_count_set lane_count_set = {0}; - - ASSERT(link || crtc_timing); // invalid input - - if (link->dpcd_caps.edp_supported_link_rates_count == 0 || - !link->panel_config.ilr.optimize_edp_link_rate) - return false; - - - // Read DPCD 00100h to find if standard link rates are set - core_link_read_dpcd(link, DP_LINK_BW_SET, - &link_bw_set, sizeof(link_bw_set)); - - if (link_bw_set) { - DC_LOG_EVENT_LINK_TRAINING("eDP ILR: Optimization required, VBIOS used link_bw_set\n"); - return true; - } - - // Read DPCD 00115h to find the edp link rate set used - core_link_read_dpcd(link, DP_LINK_RATE_SET, - &link_rate_set, sizeof(link_rate_set)); - - // Read DPCD 00101h to find out the number of lanes currently set - core_link_read_dpcd(link, DP_LANE_COUNT_SET, - &lane_count_set.raw, sizeof(lane_count_set)); - - req_bw = dc_bandwidth_in_kbps_from_timing(crtc_timing); - - if (!crtc_timing->flags.DSC) - dc_link_decide_edp_link_settings(link, &link_setting, req_bw); - else - decide_edp_link_settings_with_dsc(link, &link_setting, req_bw, LINK_RATE_UNKNOWN); - - if (link->dpcd_caps.edp_supported_link_rates[link_rate_set] != link_setting.link_rate || - lane_count_set.bits.LANE_COUNT_SET != link_setting.lane_count) { - DC_LOG_EVENT_LINK_TRAINING("eDP ILR: Optimization required, VBIOS link_rate_set not optimal\n"); - return true; - } - - DC_LOG_EVENT_LINK_TRAINING("eDP ILR: No optimization required, VBIOS set optimal link_rate_set\n"); - return false; -} - - -// TODO - DP2.0 Link: Fix get_lane_status to handle LTTPR offset (SST and MST) -static void get_lane_status( - struct dc_link *link, - uint32_t lane_count, - union lane_status *status, - union lane_align_status_updated *status_updated) -{ - unsigned int lane; - uint8_t dpcd_buf[3] = {0}; - - if (status == NULL || status_updated == NULL) { - return; - } - - core_link_read_dpcd( - link, - DP_LANE0_1_STATUS, - dpcd_buf, - sizeof(dpcd_buf)); - - for (lane = 0; lane < lane_count; lane++) { - status[lane].raw = get_nibble_at_index(&dpcd_buf[0], lane); - } - - status_updated->raw = dpcd_buf[2]; -} - -bool dpcd_write_128b_132b_sst_payload_allocation_table( - const struct dc_stream_state *stream, - struct dc_link *link, - struct link_mst_stream_allocation_table *proposed_table, - bool allocate) -{ - const uint8_t vc_id = 1; /// VC ID always 1 for SST - const uint8_t start_time_slot = 0; /// Always start at time slot 0 for SST - bool result = false; - uint8_t req_slot_count = 0; - struct fixed31_32 avg_time_slots_per_mtp = { 0 }; - union payload_table_update_status update_status = { 0 }; - const uint32_t max_retries = 30; - uint32_t retries = 0; - - if (allocate) { - avg_time_slots_per_mtp = calculate_sst_avg_time_slots_per_mtp(stream, link); - req_slot_count = dc_fixpt_ceil(avg_time_slots_per_mtp); - /// Validation should filter out modes that exceed link BW - ASSERT(req_slot_count <= MAX_MTP_SLOT_COUNT); - if (req_slot_count > MAX_MTP_SLOT_COUNT) - return false; - } else { - /// Leave req_slot_count = 0 if allocate is false. - } - - proposed_table->stream_count = 1; /// Always 1 stream for SST - proposed_table->stream_allocations[0].slot_count = req_slot_count; - proposed_table->stream_allocations[0].vcp_id = vc_id; - - if (link->aux_access_disabled) - return true; - - /// Write DPCD 2C0 = 1 to start updating - update_status.bits.VC_PAYLOAD_TABLE_UPDATED = 1; - core_link_write_dpcd( - link, - DP_PAYLOAD_TABLE_UPDATE_STATUS, - &update_status.raw, - 1); - - /// Program the changes in DPCD 1C0 - 1C2 - ASSERT(vc_id == 1); - core_link_write_dpcd( - link, - DP_PAYLOAD_ALLOCATE_SET, - &vc_id, - 1); - - ASSERT(start_time_slot == 0); - core_link_write_dpcd( - link, - DP_PAYLOAD_ALLOCATE_START_TIME_SLOT, - &start_time_slot, - 1); - - core_link_write_dpcd( - link, - DP_PAYLOAD_ALLOCATE_TIME_SLOT_COUNT, - &req_slot_count, - 1); - - /// Poll till DPCD 2C0 read 1 - /// Try for at least 150ms (30 retries, with 5ms delay after each attempt) - - while (retries < max_retries) { - if (core_link_read_dpcd( - link, - DP_PAYLOAD_TABLE_UPDATE_STATUS, - &update_status.raw, - 1) == DC_OK) { - if (update_status.bits.VC_PAYLOAD_TABLE_UPDATED == 1) { - DC_LOG_DP2("SST Update Payload: downstream payload table updated."); - result = true; - break; - } - } else { - union dpcd_rev dpcdRev; - - if (core_link_read_dpcd( - link, - DP_DPCD_REV, - &dpcdRev.raw, - 1) != DC_OK) { - DC_LOG_ERROR("SST Update Payload: Unable to read DPCD revision " - "of sink while polling payload table " - "updated status bit."); - break; - } - } - retries++; - msleep(5); - } - - if (!result && retries == max_retries) { - DC_LOG_ERROR("SST Update Payload: Payload table not updated after retries, " - "continue on. Something is wrong with the branch."); - // TODO - DP2.0 Payload: Read and log the payload table from downstream branch - } - - return result; -} - -bool dpcd_poll_for_allocation_change_trigger(struct dc_link *link) -{ - /* - * wait for ACT handled - */ - int i; - const int act_retries = 30; - enum act_return_status result = ACT_FAILED; - union payload_table_update_status update_status = {0}; - union lane_status dpcd_lane_status[LANE_COUNT_DP_MAX]; - union lane_align_status_updated lane_status_updated; - - if (link->aux_access_disabled) - return true; - for (i = 0; i < act_retries; i++) { - get_lane_status(link, link->cur_link_settings.lane_count, dpcd_lane_status, &lane_status_updated); - - if (!dp_is_cr_done(link->cur_link_settings.lane_count, dpcd_lane_status) || - !dp_is_ch_eq_done(link->cur_link_settings.lane_count, dpcd_lane_status) || - !dp_is_symbol_locked(link->cur_link_settings.lane_count, dpcd_lane_status) || - !dp_is_interlane_aligned(lane_status_updated)) { - DC_LOG_ERROR("SST Update Payload: Link loss occurred while " - "polling for ACT handled."); - result = ACT_LINK_LOST; - break; - } - core_link_read_dpcd( - link, - DP_PAYLOAD_TABLE_UPDATE_STATUS, - &update_status.raw, - 1); - - if (update_status.bits.ACT_HANDLED == 1) { - DC_LOG_DP2("SST Update Payload: ACT handled by downstream."); - result = ACT_SUCCESS; - break; - } - - msleep(5); - } - - if (result == ACT_FAILED) { - DC_LOG_ERROR("SST Update Payload: ACT still not handled after retries, " - "continue on. Something is wrong with the branch."); - } - - return (result == ACT_SUCCESS); -} - -struct fixed31_32 calculate_sst_avg_time_slots_per_mtp( - const struct dc_stream_state *stream, - const struct dc_link *link) -{ - struct fixed31_32 link_bw_effective = - dc_fixpt_from_int( - dc_link_bandwidth_kbps(link, &link->cur_link_settings)); - struct fixed31_32 timeslot_bw_effective = - dc_fixpt_div_int(link_bw_effective, MAX_MTP_SLOT_COUNT); - struct fixed31_32 timing_bw = - dc_fixpt_from_int( - dc_bandwidth_in_kbps_from_timing(&stream->timing)); - struct fixed31_32 avg_time_slots_per_mtp = - dc_fixpt_div(timing_bw, timeslot_bw_effective); - - return avg_time_slots_per_mtp; -} - -void edp_panel_backlight_power_on(struct dc_link *link, bool wait_for_hpd) -{ - if (link->connector_signal != SIGNAL_TYPE_EDP) - return; - - link->dc->hwss.edp_power_control(link, true); - if (wait_for_hpd) - link->dc->hwss.edp_wait_for_hpd_ready(link, true); - if (link->dc->hwss.edp_backlight_control) - link->dc->hwss.edp_backlight_control(link, true); -} - -void dc_link_clear_dprx_states(struct dc_link *link) -{ - memset(&link->dprx_states, 0, sizeof(link->dprx_states)); -} - -void dp_source_sequence_trace(struct dc_link *link, uint8_t dp_test_mode) -{ - if (link != NULL && link->dc->debug.enable_driver_sequence_debug) - core_link_write_dpcd(link, DP_SOURCE_SEQUENCE, - &dp_test_mode, sizeof(dp_test_mode)); -} - -void edp_add_delay_for_T9(struct dc_link *link) -{ - if (link && link->panel_config.pps.extra_delay_backlight_off > 0) - udelay(link->panel_config.pps.extra_delay_backlight_off * 1000); -} - -bool edp_receiver_ready_T9(struct dc_link *link) -{ - unsigned int tries = 0; - unsigned char sinkstatus = 0; - unsigned char edpRev = 0; - enum dc_status result = DC_OK; - - result = core_link_read_dpcd(link, DP_EDP_DPCD_REV, &edpRev, sizeof(edpRev)); - - /* start from eDP version 1.2, SINK_STAUS indicate the sink is ready.*/ - if (result == DC_OK && edpRev >= DP_EDP_12) { - do { - sinkstatus = 1; - result = core_link_read_dpcd(link, DP_SINK_STATUS, &sinkstatus, sizeof(sinkstatus)); - if (sinkstatus == 0) - break; - if (result != DC_OK) - break; - udelay(100); //MAx T9 - } while (++tries < 50); - } - - return result; -} -bool edp_receiver_ready_T7(struct dc_link *link) -{ - unsigned char sinkstatus = 0; - unsigned char edpRev = 0; - enum dc_status result = DC_OK; - - /* use absolute time stamp to constrain max T7*/ - unsigned long long enter_timestamp = 0; - unsigned long long finish_timestamp = 0; - unsigned long long time_taken_in_ns = 0; - - result = core_link_read_dpcd(link, DP_EDP_DPCD_REV, &edpRev, sizeof(edpRev)); - - if (result == DC_OK && edpRev >= DP_EDP_12) { - /* start from eDP version 1.2, SINK_STAUS indicate the sink is ready.*/ - enter_timestamp = dm_get_timestamp(link->ctx); - do { - sinkstatus = 0; - result = core_link_read_dpcd(link, DP_SINK_STATUS, &sinkstatus, sizeof(sinkstatus)); - if (sinkstatus == 1) - break; - if (result != DC_OK) - break; - udelay(25); - finish_timestamp = dm_get_timestamp(link->ctx); - time_taken_in_ns = dm_get_elapse_time_in_ns(link->ctx, finish_timestamp, enter_timestamp); - } while (time_taken_in_ns < 50 * 1000000); //MAx T7 is 50ms - } - - if (link && link->panel_config.pps.extra_t7_ms > 0) - udelay(link->panel_config.pps.extra_t7_ms * 1000); - - return result; -} - -void dp_retrain_link_dp_test(struct dc_link *link, - struct dc_link_settings *link_setting, - bool skip_video_pattern) -{ - struct pipe_ctx *pipe; - unsigned int i; - - udelay(100); - - for (i = 0; i < MAX_PIPES; i++) { - pipe = &link->dc->current_state->res_ctx.pipe_ctx[i]; - if (pipe->stream != NULL && - pipe->stream->link == link && - !pipe->stream->dpms_off && - !pipe->top_pipe && !pipe->prev_odm_pipe) { - core_link_disable_stream(pipe); - pipe->link_config.dp_link_settings = *link_setting; - update_dp_encoder_resources_for_test_harness( - link->dc, - pipe->stream->ctx->dc->current_state, - pipe); - } - } - - for (i = 0; i < MAX_PIPES; i++) { - pipe = &link->dc->current_state->res_ctx.pipe_ctx[i]; - if (pipe->stream != NULL && - pipe->stream->link == link && - !pipe->stream->dpms_off && - !pipe->top_pipe && !pipe->prev_odm_pipe) { - core_link_enable_stream( - pipe->stream->ctx->dc->current_state, - pipe); - } - } -} - -#undef DC_LOGGER -#define DC_LOGGER \ - dsc->ctx->logger -static void dsc_optc_config_log(struct display_stream_compressor *dsc, - struct dsc_optc_config *config) -{ - uint32_t precision = 1 << 28; - uint32_t bytes_per_pixel_int = config->bytes_per_pixel / precision; - uint32_t bytes_per_pixel_mod = config->bytes_per_pixel % precision; - uint64_t ll_bytes_per_pix_fraq = bytes_per_pixel_mod; - - /* 7 fractional digits decimal precision for bytes per pixel is enough because DSC - * bits per pixel precision is 1/16th of a pixel, which means bytes per pixel precision is - * 1/16/8 = 1/128 of a byte, or 0.0078125 decimal - */ - ll_bytes_per_pix_fraq *= 10000000; - ll_bytes_per_pix_fraq /= precision; - - DC_LOG_DSC("\tbytes_per_pixel 0x%08x (%d.%07d)", - config->bytes_per_pixel, bytes_per_pixel_int, (uint32_t)ll_bytes_per_pix_fraq); - DC_LOG_DSC("\tis_pixel_format_444 %d", config->is_pixel_format_444); - DC_LOG_DSC("\tslice_width %d", config->slice_width); -} - -bool dp_set_dsc_on_rx(struct pipe_ctx *pipe_ctx, bool enable) -{ - struct dc *dc = pipe_ctx->stream->ctx->dc; - struct dc_stream_state *stream = pipe_ctx->stream; - bool result = false; - - if (dc_is_virtual_signal(stream->signal) || IS_FPGA_MAXIMUS_DC(dc->ctx->dce_environment)) - result = true; - else - result = dm_helpers_dp_write_dsc_enable(dc->ctx, stream, enable); - return result; -} - -/* The stream with these settings can be sent (unblanked) only after DSC was enabled on RX first, - * i.e. after dp_enable_dsc_on_rx() had been called - */ -void dp_set_dsc_on_stream(struct pipe_ctx *pipe_ctx, bool enable) -{ - struct display_stream_compressor *dsc = pipe_ctx->stream_res.dsc; - struct dc *dc = pipe_ctx->stream->ctx->dc; - struct dc_stream_state *stream = pipe_ctx->stream; - struct pipe_ctx *odm_pipe; - int opp_cnt = 1; - - for (odm_pipe = pipe_ctx->next_odm_pipe; odm_pipe; odm_pipe = odm_pipe->next_odm_pipe) - opp_cnt++; - - if (enable) { - struct dsc_config dsc_cfg; - struct dsc_optc_config dsc_optc_cfg; - enum optc_dsc_mode optc_dsc_mode; - - /* Enable DSC hw block */ - dsc_cfg.pic_width = (stream->timing.h_addressable + stream->timing.h_border_left + stream->timing.h_border_right) / opp_cnt; - dsc_cfg.pic_height = stream->timing.v_addressable + stream->timing.v_border_top + stream->timing.v_border_bottom; - dsc_cfg.pixel_encoding = stream->timing.pixel_encoding; - dsc_cfg.color_depth = stream->timing.display_color_depth; - dsc_cfg.is_odm = pipe_ctx->next_odm_pipe ? true : false; - dsc_cfg.dc_dsc_cfg = stream->timing.dsc_cfg; - ASSERT(dsc_cfg.dc_dsc_cfg.num_slices_h % opp_cnt == 0); - dsc_cfg.dc_dsc_cfg.num_slices_h /= opp_cnt; - - dsc->funcs->dsc_set_config(dsc, &dsc_cfg, &dsc_optc_cfg); - dsc->funcs->dsc_enable(dsc, pipe_ctx->stream_res.opp->inst); - for (odm_pipe = pipe_ctx->next_odm_pipe; odm_pipe; odm_pipe = odm_pipe->next_odm_pipe) { - struct display_stream_compressor *odm_dsc = odm_pipe->stream_res.dsc; - - odm_dsc->funcs->dsc_set_config(odm_dsc, &dsc_cfg, &dsc_optc_cfg); - odm_dsc->funcs->dsc_enable(odm_dsc, odm_pipe->stream_res.opp->inst); - } - dsc_cfg.dc_dsc_cfg.num_slices_h *= opp_cnt; - dsc_cfg.pic_width *= opp_cnt; - - optc_dsc_mode = dsc_optc_cfg.is_pixel_format_444 ? OPTC_DSC_ENABLED_444 : OPTC_DSC_ENABLED_NATIVE_SUBSAMPLED; - - /* Enable DSC in encoder */ - if (dc_is_dp_signal(stream->signal) && !IS_FPGA_MAXIMUS_DC(dc->ctx->dce_environment) - && !link_is_dp_128b_132b_signal(pipe_ctx)) { - DC_LOG_DSC("Setting stream encoder DSC config for engine %d:", (int)pipe_ctx->stream_res.stream_enc->id); - dsc_optc_config_log(dsc, &dsc_optc_cfg); - pipe_ctx->stream_res.stream_enc->funcs->dp_set_dsc_config(pipe_ctx->stream_res.stream_enc, - optc_dsc_mode, - dsc_optc_cfg.bytes_per_pixel, - dsc_optc_cfg.slice_width); - - /* PPS SDP is set elsewhere because it has to be done after DIG FE is connected to DIG BE */ - } - - /* Enable DSC in OPTC */ - DC_LOG_DSC("Setting optc DSC config for tg instance %d:", pipe_ctx->stream_res.tg->inst); - dsc_optc_config_log(dsc, &dsc_optc_cfg); - pipe_ctx->stream_res.tg->funcs->set_dsc_config(pipe_ctx->stream_res.tg, - optc_dsc_mode, - dsc_optc_cfg.bytes_per_pixel, - dsc_optc_cfg.slice_width); - } else { - /* disable DSC in OPTC */ - pipe_ctx->stream_res.tg->funcs->set_dsc_config( - pipe_ctx->stream_res.tg, - OPTC_DSC_DISABLED, 0, 0); - - /* disable DSC in stream encoder */ - if (dc_is_dp_signal(stream->signal)) { - if (link_is_dp_128b_132b_signal(pipe_ctx)) - pipe_ctx->stream_res.hpo_dp_stream_enc->funcs->dp_set_dsc_pps_info_packet( - pipe_ctx->stream_res.hpo_dp_stream_enc, - false, - NULL, - true); - else if (!IS_FPGA_MAXIMUS_DC(dc->ctx->dce_environment)) { - pipe_ctx->stream_res.stream_enc->funcs->dp_set_dsc_config( - pipe_ctx->stream_res.stream_enc, - OPTC_DSC_DISABLED, 0, 0); - pipe_ctx->stream_res.stream_enc->funcs->dp_set_dsc_pps_info_packet( - pipe_ctx->stream_res.stream_enc, false, NULL, true); - } - } - - /* disable DSC block */ - pipe_ctx->stream_res.dsc->funcs->dsc_disable(pipe_ctx->stream_res.dsc); - for (odm_pipe = pipe_ctx->next_odm_pipe; odm_pipe; odm_pipe = odm_pipe->next_odm_pipe) - odm_pipe->stream_res.dsc->funcs->dsc_disable(odm_pipe->stream_res.dsc); - } -} - -bool dp_set_dsc_enable(struct pipe_ctx *pipe_ctx, bool enable) -{ - struct display_stream_compressor *dsc = pipe_ctx->stream_res.dsc; - bool result = false; - - if (!pipe_ctx->stream->timing.flags.DSC) - goto out; - if (!dsc) - goto out; - - if (enable) { - { - dp_set_dsc_on_stream(pipe_ctx, true); - result = true; - } - } else { - dp_set_dsc_on_rx(pipe_ctx, false); - dp_set_dsc_on_stream(pipe_ctx, false); - result = true; - } -out: - return result; -} - -/* - * For dynamic bpp change case, dsc is programmed with MASTER_UPDATE_LOCK enabled; - * hence PPS info packet update need to use frame update instead of immediate update. - * Added parameter immediate_update for this purpose. - * The decision to use frame update is hard-coded in function dp_update_dsc_config(), - * which is the only place where a "false" would be passed in for param immediate_update. - * - * immediate_update is only applicable when DSC is enabled. - */ -bool dp_set_dsc_pps_sdp(struct pipe_ctx *pipe_ctx, bool enable, bool immediate_update) -{ - struct display_stream_compressor *dsc = pipe_ctx->stream_res.dsc; - struct dc_stream_state *stream = pipe_ctx->stream; - - if (!pipe_ctx->stream->timing.flags.DSC || !dsc) - return false; - - if (enable) { - struct dsc_config dsc_cfg; - uint8_t dsc_packed_pps[128]; - - memset(&dsc_cfg, 0, sizeof(dsc_cfg)); - memset(dsc_packed_pps, 0, 128); - - /* Enable DSC hw block */ - dsc_cfg.pic_width = stream->timing.h_addressable + stream->timing.h_border_left + stream->timing.h_border_right; - dsc_cfg.pic_height = stream->timing.v_addressable + stream->timing.v_border_top + stream->timing.v_border_bottom; - dsc_cfg.pixel_encoding = stream->timing.pixel_encoding; - dsc_cfg.color_depth = stream->timing.display_color_depth; - dsc_cfg.is_odm = pipe_ctx->next_odm_pipe ? true : false; - dsc_cfg.dc_dsc_cfg = stream->timing.dsc_cfg; - - dsc->funcs->dsc_get_packed_pps(dsc, &dsc_cfg, &dsc_packed_pps[0]); - memcpy(&stream->dsc_packed_pps[0], &dsc_packed_pps[0], sizeof(stream->dsc_packed_pps)); - if (dc_is_dp_signal(stream->signal)) { - DC_LOG_DSC("Setting stream encoder DSC PPS SDP for engine %d\n", (int)pipe_ctx->stream_res.stream_enc->id); - if (link_is_dp_128b_132b_signal(pipe_ctx)) - pipe_ctx->stream_res.hpo_dp_stream_enc->funcs->dp_set_dsc_pps_info_packet( - pipe_ctx->stream_res.hpo_dp_stream_enc, - true, - &dsc_packed_pps[0], - immediate_update); - else - pipe_ctx->stream_res.stream_enc->funcs->dp_set_dsc_pps_info_packet( - pipe_ctx->stream_res.stream_enc, - true, - &dsc_packed_pps[0], - immediate_update); - } - } else { - /* disable DSC PPS in stream encoder */ - memset(&stream->dsc_packed_pps[0], 0, sizeof(stream->dsc_packed_pps)); - if (dc_is_dp_signal(stream->signal)) { - if (link_is_dp_128b_132b_signal(pipe_ctx)) - pipe_ctx->stream_res.hpo_dp_stream_enc->funcs->dp_set_dsc_pps_info_packet( - pipe_ctx->stream_res.hpo_dp_stream_enc, - false, - NULL, - true); - else - pipe_ctx->stream_res.stream_enc->funcs->dp_set_dsc_pps_info_packet( - pipe_ctx->stream_res.stream_enc, false, NULL, true); - } - } - - return true; -} - - -bool dp_update_dsc_config(struct pipe_ctx *pipe_ctx) -{ - struct display_stream_compressor *dsc = pipe_ctx->stream_res.dsc; - - if (!pipe_ctx->stream->timing.flags.DSC) - return false; - if (!dsc) - return false; - - dp_set_dsc_on_stream(pipe_ctx, true); - dp_set_dsc_pps_sdp(pipe_ctx, true, false); - return true; -} - -#undef DC_LOGGER -#define DC_LOGGER \ - link->ctx->logger diff --git a/drivers/gpu/drm/amd/display/dc/core/dc_link_enc_cfg.c b/drivers/gpu/drm/amd/display/dc/core/dc_link_enc_cfg.c index fa2ba3fc683b4125aadaec58394134f71efae32f..74e465ba158d2c03504d3e1c6d697f29a06867ed 100644 --- a/drivers/gpu/drm/amd/display/dc/core/dc_link_enc_cfg.c +++ b/drivers/gpu/drm/amd/display/dc/core/dc_link_enc_cfg.c @@ -24,7 +24,6 @@ #include "link_enc_cfg.h" #include "resource.h" -#include "dc_link_dp.h" #include "link.h" #define DC_LOGGER dc->ctx->logger diff --git a/drivers/gpu/drm/amd/display/dc/core/dc_resource.c b/drivers/gpu/drm/amd/display/dc/core/dc_resource.c index a5b5f8592c1b80b8d0ce1ef44b016954d0e1a892..e41bbcbc71460f02fd512a7e91a34864e1c36bca 100644 --- a/drivers/gpu/drm/amd/display/dc/core/dc_resource.c +++ b/drivers/gpu/drm/amd/display/dc/core/dc_resource.c @@ -40,12 +40,11 @@ #include "virtual/virtual_stream_encoder.h" #include "dpcd_defs.h" #include "link_enc_cfg.h" -#include "dc_link_dp.h" #include "link.h" #include "virtual/virtual_link_hwss.h" -#include "link/link_hwss_dio.h" -#include "link/link_hwss_dpia.h" -#include "link/link_hwss_hpo_dp.h" +#include "link/hwss/link_hwss_dio.h" +#include "link/hwss/link_hwss_dpia.h" +#include "link/hwss/link_hwss_hpo_dp.h" #if defined(CONFIG_DRM_AMD_DC_SI) #include "dce60/dce60_resource.h" @@ -3270,6 +3269,50 @@ static void set_hfvs_info_packet( *info_packet = stream->hfvsif_infopacket; } +static void adaptive_sync_override_dp_info_packets_sdp_line_num( + const struct dc_crtc_timing *timing, + struct enc_sdp_line_num *sdp_line_num, + struct _vcs_dpi_display_pipe_dest_params_st *pipe_dlg_param) +{ + uint32_t asic_blank_start = 0; + uint32_t asic_blank_end = 0; + uint32_t v_update = 0; + + const struct dc_crtc_timing *tg = timing; + + /* blank_start = frame end - front porch */ + asic_blank_start = tg->v_total - tg->v_front_porch; + + /* blank_end = blank_start - active */ + asic_blank_end = (asic_blank_start - tg->v_border_bottom - + tg->v_addressable - tg->v_border_top); + + if (pipe_dlg_param->vstartup_start > asic_blank_end) { + v_update = (tg->v_total - (pipe_dlg_param->vstartup_start - asic_blank_end)); + sdp_line_num->adaptive_sync_line_num_valid = true; + sdp_line_num->adaptive_sync_line_num = (tg->v_total - v_update - 1); + } else { + sdp_line_num->adaptive_sync_line_num_valid = false; + sdp_line_num->adaptive_sync_line_num = 0; + } +} + +static void set_adaptive_sync_info_packet( + struct dc_info_packet *info_packet, + const struct dc_stream_state *stream, + struct encoder_info_frame *info_frame, + struct _vcs_dpi_display_pipe_dest_params_st *pipe_dlg_param) +{ + if (!stream->adaptive_sync_infopacket.valid) + return; + + adaptive_sync_override_dp_info_packets_sdp_line_num( + &stream->timing, + &info_frame->sdp_line_num, + pipe_dlg_param); + + *info_packet = stream->adaptive_sync_infopacket; +} static void set_vtem_info_packet( struct dc_info_packet *info_packet, @@ -3362,6 +3405,7 @@ void resource_build_info_frame(struct pipe_ctx *pipe_ctx) info->vsc.valid = false; info->hfvsif.valid = false; info->vtem.valid = false; + info->adaptive_sync.valid = false; signal = pipe_ctx->stream->signal; /* HDMi and DP have different info packets*/ @@ -3382,6 +3426,10 @@ void resource_build_info_frame(struct pipe_ctx *pipe_ctx) set_spd_info_packet(&info->spd, pipe_ctx->stream); set_hdr_static_info_packet(&info->hdrsmd, pipe_ctx->stream); + set_adaptive_sync_info_packet(&info->adaptive_sync, + pipe_ctx->stream, + info, + &pipe_ctx->pipe_dlg_param); } patch_gamut_packet_checksum(&info->gamut); diff --git a/drivers/gpu/drm/amd/display/dc/dc.h b/drivers/gpu/drm/amd/display/dc/dc.h index 22e754ad22c834bdc634fb326af0ac1428dfe27a..42ce453064838be1f7f90f99dc1da51c54131cb9 100644 --- a/drivers/gpu/drm/amd/display/dc/dc.h +++ b/drivers/gpu/drm/amd/display/dc/dc.h @@ -47,7 +47,7 @@ struct aux_payload; struct set_config_cmd_payload; struct dmub_notification; -#define DC_VER "3.2.218" +#define DC_VER "3.2.220" #define MAX_SURFACES 3 #define MAX_PLANES 6 @@ -1414,6 +1414,7 @@ struct dpcd_caps { union dpcd_fec_capability fec_cap; struct dpcd_dsc_capabilities dsc_caps; struct dc_lttpr_caps lttpr_caps; + struct adaptive_sync_caps adaptive_sync_caps; struct dpcd_usb4_dp_tunneling_info usb4_dp_tun_info; union dp_128b_132b_supported_link_rates dp_128b_132b_supported_link_rates; diff --git a/drivers/gpu/drm/amd/display/dc/dc_dp_types.h b/drivers/gpu/drm/amd/display/dc/dc_dp_types.h index 84da54358922996043a3831082162a900aa2cb79..184583807d45db806f64bb02ab6270055112062b 100644 --- a/drivers/gpu/drm/amd/display/dc/dc_dp_types.h +++ b/drivers/gpu/drm/amd/display/dc/dc_dp_types.h @@ -502,7 +502,11 @@ union down_spread_ctrl { 1 = Main link signal is downspread <= 0.5% with frequency in the range of 30kHz ~ 33kHz*/ uint8_t SPREAD_AMP:1; - uint8_t RESERVED2:2;/*Bit 6:5 = RESERVED. Read all 0s*/ + uint8_t RESERVED2:1;/*Bit 5 = RESERVED. Read all 0s*/ + /* Bit 6 = FIXED_VTOTAL_AS_SDP_EN_IN_PR_ACTIVE. + 0 = FIXED_VTOTAL_AS_SDP_EN_IN_PR_ACTIVE is not enabled by the Source device (default) + 1 = FIXED_VTOTAL_AS_SDP_EN_IN_PR_ACTIVE is enabled by Source device */ + uint8_t FIXED_VTOTAL_AS_SDP_EN_IN_PR_ACTIVE:1; /*Bit 7 = MSA_TIMING_PAR_IGNORE_EN 0 = Source device will send valid data for the MSA Timing Params 1 = Source device may send invalid data for these MSA Timing Params*/ @@ -858,6 +862,21 @@ struct psr_caps { unsigned int psr_power_opt_flag; }; +union dpcd_dprx_feature_enumeration_list_cont_1 { + struct { + uint8_t ADAPTIVE_SYNC_SDP_SUPPORT:1; + uint8_t AS_SDP_FIRST_HALF_LINE_OR_3840_PIXEL_CYCLE_WINDOW_NOT_SUPPORTED: 1; + uint8_t RESERVED0: 2; + uint8_t VSC_EXT_SDP_VER1_SUPPORT: 1; + uint8_t RESERVED1: 3; + } bits; + uint8_t raw; +}; + +struct adaptive_sync_caps { + union dpcd_dprx_feature_enumeration_list_cont_1 dp_adap_sync_caps; +}; + /* Length of router topology ID read from DPCD in bytes. */ #define DPCD_USB4_TOPOLOGY_ID_LEN 5 diff --git a/drivers/gpu/drm/amd/display/dc/dc_link.h b/drivers/gpu/drm/amd/display/dc/dc_link.h index 48f6a5b0933611ab3fabd607fd429ee3be065644..1927eacbfa71a5008436f64891d51744e992d4d1 100644 --- a/drivers/gpu/drm/amd/display/dc/dc_link.h +++ b/drivers/gpu/drm/amd/display/dc/dc_link.h @@ -280,6 +280,7 @@ struct dc_link { bool dp_keep_receiver_powered; bool dp_skip_DID2; bool dp_skip_reset_segment; + bool dp_skip_fs_144hz; bool dp_mot_reset_segment; /* Some USB4 docks do not handle turning off MST DSC once it has been enabled. */ bool dpia_mst_dsc_always_on; @@ -370,11 +371,6 @@ bool dc_link_get_backlight_level_nits(struct dc_link *link, uint32_t *backlight_millinits, uint32_t *backlight_millinits_peak); -bool dc_link_backlight_enable_aux(struct dc_link *link, bool enable); - -bool dc_link_read_default_bl_aux(struct dc_link *link, uint32_t *backlight_millinits); -bool dc_link_set_default_brightness_aux(struct dc_link *link); - int dc_link_get_backlight_level(const struct dc_link *dc_link); int dc_link_get_target_backlight_pwm(const struct dc_link *link); @@ -388,16 +384,10 @@ bool dc_link_setup_psr(struct dc_link *dc_link, const struct dc_stream_state *stream, struct psr_config *psr_config, struct psr_context *psr_context); -bool dc_power_alpm_dpcd_enable(struct dc_link *link, bool enable); - -void dc_link_get_psr_residency(const struct dc_link *link, uint32_t *residency); - void dc_link_blank_all_dp_displays(struct dc *dc); void dc_link_blank_all_edp_displays(struct dc *dc); void dc_link_blank_dp_stream(struct dc_link *link, bool hw_init); -bool dc_link_set_sink_vtotal_in_psr_active(const struct dc_link *link, - uint16_t psr_vtotal_idle, uint16_t psr_vtotal_su); /* Request DC to detect if there is a Panel connected. * boot - If this call is during initial boot. @@ -441,7 +431,8 @@ bool dc_link_wait_for_t12(struct dc_link *link); void dc_link_dp_handle_automated_test(struct dc_link *link); void dc_link_dp_handle_link_loss(struct dc_link *link); bool dc_link_dp_allow_hpd_rx_irq(const struct dc_link *link); - +bool dc_link_check_link_loss_status(struct dc_link *link, + union hpd_irq_data *hpd_irq_dpcd_data); struct dc_sink_init_data; struct dc_sink *dc_link_add_remote_sink( @@ -456,11 +447,6 @@ void dc_link_remove_remote_sink( /* Used by diagnostics for virtual link at the moment */ -void dc_link_dp_set_drive_settings( - struct dc_link *link, - const struct link_resource *link_res, - struct link_training_settings *lt_settings); - bool dc_link_dp_set_test_pattern( struct dc_link *link, enum dp_test_pattern test_pattern, @@ -589,4 +575,56 @@ void dc_link_dp_receiver_power_ctrl(struct dc_link *link, bool on); bool dc_link_decide_edp_link_settings(struct dc_link *link, struct dc_link_settings *link_setting, uint32_t req_bw); +void dc_link_edp_panel_backlight_power_on(struct dc_link *link, + bool wait_for_hpd); + +#define LINK_TRAINING_ATTEMPTS 4 +#define LINK_TRAINING_RETRY_DELAY 50 /* ms */ +#define MAX_MTP_SLOT_COUNT 64 +#define TRAINING_AUX_RD_INTERVAL 100 //us +#define LINK_AUX_WAKE_TIMEOUT_MS 1500 // Timeout when trying to wake unresponsive DPRX. + +struct dc_link; +struct dc_stream_state; +struct dc_link_settings; + +enum { + /* + * Some receivers fail to train on first try and are good + * on subsequent tries. 2 retries should be plenty. If we + * don't have a successful training then we don't expect to + * ever get one. + */ + LINK_TRAINING_MAX_VERIFY_RETRY = 2, + PEAK_FACTOR_X1000 = 1006, +}; + +bool dp_validate_mode_timing( + struct dc_link *link, + const struct dc_crtc_timing *timing); + +void dp_enable_mst_on_sink(struct dc_link *link, bool enable); + +enum dc_status dp_set_fec_ready(struct dc_link *link, const struct link_resource *link_res, bool ready); +void dp_set_fec_enable(struct dc_link *link, bool enable); +bool dp_set_dsc_enable(struct pipe_ctx *pipe_ctx, bool enable); +bool dp_set_dsc_pps_sdp(struct pipe_ctx *pipe_ctx, bool enable, bool immediate_update); +void dp_set_dsc_on_stream(struct pipe_ctx *pipe_ctx, bool enable); +bool dp_update_dsc_config(struct pipe_ctx *pipe_ctx); +bool dp_set_dsc_on_rx(struct pipe_ctx *pipe_ctx, bool enable); + +bool dpcd_write_128b_132b_sst_payload_allocation_table( + const struct dc_stream_state *stream, + struct dc_link *link, + struct link_mst_stream_allocation_table *proposed_table, + bool allocate); + +bool dpcd_poll_for_allocation_change_trigger(struct dc_link *link); + +struct fixed31_32 calculate_sst_avg_time_slots_per_mtp( + const struct dc_stream_state *stream, + const struct dc_link *link); +void setup_dp_hpo_stream(struct pipe_ctx *pipe_ctx, bool enable); +void dp_source_sequence_trace(struct dc_link *link, uint8_t dp_test_mode); + #endif /* DC_LINK_H_ */ diff --git a/drivers/gpu/drm/amd/display/dc/dc_stream.h b/drivers/gpu/drm/amd/display/dc/dc_stream.h index ef33d7d8a2bf79f8524bb6b7baca914d450b6b9a..567452599659c303eab99545bbb7fac8761ab25d 100644 --- a/drivers/gpu/drm/amd/display/dc/dc_stream.h +++ b/drivers/gpu/drm/amd/display/dc/dc_stream.h @@ -190,6 +190,7 @@ struct dc_stream_state { struct dc_info_packet vsp_infopacket; struct dc_info_packet hfvsif_infopacket; struct dc_info_packet vtem_infopacket; + struct dc_info_packet adaptive_sync_infopacket; uint8_t dsc_packed_pps[128]; struct rect src; /* composition area */ struct rect dst; /* stream addressable area */ @@ -313,6 +314,7 @@ struct dc_stream_update { struct dc_info_packet *vsp_infopacket; struct dc_info_packet *hfvsif_infopacket; struct dc_info_packet *vtem_infopacket; + struct dc_info_packet *adaptive_sync_infopacket; bool *dpms_off; bool integer_scaling_update; bool *allow_freesync; diff --git a/drivers/gpu/drm/amd/display/dc/dc_types.h b/drivers/gpu/drm/amd/display/dc/dc_types.h index c73a655bd68777f04994f204ac5c45b22f10817c..f653eca09ba79adfb1503998acad987744551d4a 100644 --- a/drivers/gpu/drm/amd/display/dc/dc_types.h +++ b/drivers/gpu/drm/amd/display/dc/dc_types.h @@ -691,6 +691,7 @@ struct psr_config { uint8_t su_y_granularity; unsigned int line_time_in_us; uint8_t rate_control_caps; + uint16_t dsc_slice_height; }; union dmcu_psr_level { @@ -802,6 +803,7 @@ struct psr_context { uint8_t su_y_granularity; unsigned int line_time_in_us; uint8_t rate_control_caps; + uint16_t dsc_slice_height; }; struct colorspace_transform { diff --git a/drivers/gpu/drm/amd/display/dc/dce/dmub_psr.c b/drivers/gpu/drm/amd/display/dc/dce/dmub_psr.c index 2d3201b77d6a0d66eb0f88221fbe19c2e429c8f0..1e2d2cbe2c373788eaf2d681394e64e24bd336a1 100644 --- a/drivers/gpu/drm/amd/display/dc/dce/dmub_psr.c +++ b/drivers/gpu/drm/amd/display/dc/dce/dmub_psr.c @@ -417,6 +417,7 @@ static bool dmub_psr_copy_settings(struct dmub_psr *dmub, copy_settings_data->relock_delay_frame_cnt = 0; if (link->dpcd_caps.sink_dev_id == DP_BRANCH_DEVICE_ID_001CF8) copy_settings_data->relock_delay_frame_cnt = 2; + copy_settings_data->dsc_slice_height = psr_context->dsc_slice_height; dc_dmub_srv_cmd_queue(dc->dmub_srv, &cmd); dc_dmub_srv_cmd_execute(dc->dmub_srv); diff --git a/drivers/gpu/drm/amd/display/dc/dce110/dce110_hw_sequencer.c b/drivers/gpu/drm/amd/display/dc/dce110/dce110_hw_sequencer.c index a51bd21a796f974499255e9a3802af5903c18f43..adb36be8190e52d4a7db5e5c99dd33375c0cf68f 100644 --- a/drivers/gpu/drm/amd/display/dc/dce110/dce110_hw_sequencer.c +++ b/drivers/gpu/drm/amd/display/dc/dce110/dce110_hw_sequencer.c @@ -47,7 +47,6 @@ #include "link_enc_cfg.h" #include "link_hwss.h" #include "link.h" -#include "dc_link_dp.h" #include "dccg.h" #include "clock_source.h" #include "clk_mgr.h" @@ -65,7 +64,6 @@ #include "dcn10/dcn10_hw_sequencer.h" -#include "link/link_dp_trace.h" #include "dce110_hw_sequencer.h" #define GAMMA_HW_POINTS_NUM 256 @@ -653,10 +651,16 @@ void dce110_update_info_frame(struct pipe_ctx *pipe_ctx) pipe_ctx->stream_res.stream_enc->funcs->update_hdmi_info_packets( pipe_ctx->stream_res.stream_enc, &pipe_ctx->stream_res.encoder_info_frame); - else + else { + if (pipe_ctx->stream_res.stream_enc->funcs->update_dp_info_packets_sdp_line_num) + pipe_ctx->stream_res.stream_enc->funcs->update_dp_info_packets_sdp_line_num( + pipe_ctx->stream_res.stream_enc, + &pipe_ctx->stream_res.encoder_info_frame); + pipe_ctx->stream_res.stream_enc->funcs->update_dp_info_packets( pipe_ctx->stream_res.stream_enc, &pipe_ctx->stream_res.encoder_info_frame); + } } void dce110_enable_stream(struct pipe_ctx *pipe_ctx) @@ -807,19 +811,19 @@ void dce110_edp_power_control( div64_u64(dm_get_elapse_time_in_ns( ctx, current_ts, - dp_trace_get_edp_poweroff_timestamp(link)), 1000000); + link_dp_trace_get_edp_poweroff_timestamp(link)), 1000000); unsigned long long time_since_edp_poweron_ms = div64_u64(dm_get_elapse_time_in_ns( ctx, current_ts, - dp_trace_get_edp_poweron_timestamp(link)), 1000000); + link_dp_trace_get_edp_poweron_timestamp(link)), 1000000); DC_LOG_HW_RESUME_S3( "%s: transition: power_up=%d current_ts=%llu edp_poweroff=%llu edp_poweron=%llu time_since_edp_poweroff_ms=%llu time_since_edp_poweron_ms=%llu", __func__, power_up, current_ts, - dp_trace_get_edp_poweroff_timestamp(link), - dp_trace_get_edp_poweron_timestamp(link), + link_dp_trace_get_edp_poweroff_timestamp(link), + link_dp_trace_get_edp_poweron_timestamp(link), time_since_edp_poweroff_ms, time_since_edp_poweron_ms); @@ -834,7 +838,7 @@ void dce110_edp_power_control( link->panel_config.pps.extra_t12_ms; /* Adjust remaining_min_edp_poweroff_time_ms if this is not the first time. */ - if (dp_trace_get_edp_poweroff_timestamp(link) != 0) { + if (link_dp_trace_get_edp_poweroff_timestamp(link) != 0) { if (time_since_edp_poweroff_ms < remaining_min_edp_poweroff_time_ms) remaining_min_edp_poweroff_time_ms = remaining_min_edp_poweroff_time_ms - time_since_edp_poweroff_ms; @@ -894,13 +898,13 @@ void dce110_edp_power_control( __func__, (power_up ? "On":"Off"), bp_result); - dp_trace_set_edp_power_timestamp(link, power_up); + link_dp_trace_set_edp_power_timestamp(link, power_up); DC_LOG_HW_RESUME_S3( "%s: updated values: edp_poweroff=%llu edp_poweron=%llu\n", __func__, - dp_trace_get_edp_poweroff_timestamp(link), - dp_trace_get_edp_poweron_timestamp(link)); + link_dp_trace_get_edp_poweroff_timestamp(link), + link_dp_trace_get_edp_poweron_timestamp(link)); if (bp_result != BP_RESULT_OK) DC_LOG_ERROR( @@ -928,14 +932,14 @@ void dce110_edp_wait_for_T12( return; if (!link->panel_cntl->funcs->is_panel_powered_on(link->panel_cntl) && - dp_trace_get_edp_poweroff_timestamp(link) != 0) { + link_dp_trace_get_edp_poweroff_timestamp(link) != 0) { unsigned int t12_duration = 500; // Default T12 as per spec unsigned long long current_ts = dm_get_timestamp(ctx); unsigned long long time_since_edp_poweroff_ms = div64_u64(dm_get_elapse_time_in_ns( ctx, current_ts, - dp_trace_get_edp_poweroff_timestamp(link)), 1000000); + link_dp_trace_get_edp_poweroff_timestamp(link)), 1000000); t12_duration += link->panel_config.pps.extra_t12_ms; // Add extra T12 @@ -1016,7 +1020,7 @@ void dce110_edp_backlight_control( * we shouldn't be doing power-sequencing, hence we can skip * waiting for T7-ready. */ - edp_receiver_ready_T7(link); + link_edp_receiver_ready_T7(link); else DC_LOG_DC("edp_receiver_ready_T7 skipped\n"); } @@ -1047,7 +1051,7 @@ void dce110_edp_backlight_control( if (link->dpcd_sink_ext_caps.bits.oled || link->dpcd_sink_ext_caps.bits.hdr_aux_backlight_control == 1 || link->dpcd_sink_ext_caps.bits.sdr_aux_backlight_control == 1) - dc_link_backlight_enable_aux(link, enable); + link_backlight_enable_aux(link, enable); /*edp 1.2*/ if (cntl.action == TRANSMITTER_CONTROL_BACKLIGHT_OFF) { @@ -1059,7 +1063,7 @@ void dce110_edp_backlight_control( * we shouldn't be doing power-sequencing, hence we can skip * waiting for T9-ready. */ - edp_add_delay_for_T9(link); + link_edp_add_delay_for_T9(link); else DC_LOG_DC("edp_receiver_ready_T9 skipped\n"); } @@ -1243,7 +1247,7 @@ void dce110_blank_stream(struct pipe_ctx *pipe_ctx) * we shouldn't be doing power-sequencing, hence we can skip * waiting for T9-ready. */ - edp_receiver_ready_T9(link); + link_edp_receiver_ready_T9(link); } } } diff --git a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_hw_sequencer.c b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_hw_sequencer.c index 0a0c930c1626db6b09b3bbf837bb6d264244402c..bb155734ac93da07a590c4d2521de7e0b7db9bea 100644 --- a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_hw_sequencer.c +++ b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_hw_sequencer.c @@ -45,7 +45,6 @@ #include "dcn10_hubp.h" #include "dcn10_hubbub.h" #include "dcn10_cm_common.h" -#include "dc_link_dp.h" #include "dccg.h" #include "clk_mgr.h" #include "link_hwss.h" @@ -56,7 +55,6 @@ #include "dce/dmub_hw_lock_mgr.h" #include "dc_trace.h" #include "dce/dmub_outbox.h" -#include "inc/dc_link_dp.h" #include "link.h" #define DC_LOGGER_INIT(logger) diff --git a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_stream_encoder.c b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_stream_encoder.c index 1527c3b4fb19777a31379f43f93dec8bc27107a8..a1e32b9380506c932736f46d1b36fcc6111671e3 100644 --- a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_stream_encoder.c +++ b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_stream_encoder.c @@ -28,7 +28,7 @@ #include "dcn10_stream_encoder.h" #include "reg_helper.h" #include "hw_shared.h" -#include "dc_link_dp.h" +#include "link.h" #include "dpcd_defs.h" #include "dcn30/dcn30_afmt.h" @@ -753,12 +753,19 @@ void enc1_stream_encoder_update_dp_info_packets( * use other packetIndex (such as 5,6) for other info packet */ + if (info_frame->adaptive_sync.valid) + enc1_update_generic_info_packet( + enc1, + 5, /* packetIndex */ + &info_frame->adaptive_sync); + /* enable/disable transmission of packet(s). * If enabled, packet transmission begins on the next frame */ REG_UPDATE(DP_SEC_CNTL, DP_SEC_GSP0_ENABLE, info_frame->vsc.valid); REG_UPDATE(DP_SEC_CNTL, DP_SEC_GSP2_ENABLE, info_frame->spd.valid); REG_UPDATE(DP_SEC_CNTL, DP_SEC_GSP3_ENABLE, info_frame->hdrsmd.valid); + REG_UPDATE(DP_SEC_CNTL, DP_SEC_GSP5_ENABLE, info_frame->adaptive_sync.valid); /* This bit is the master enable bit. * When enabling secondary stream engine, diff --git a/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_hwseq.c b/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_hwseq.c index 6bfa16d9135fe4c91d7f9990ded1e86e92920af8..916dceecd3de2726ae644ee3f7ff656442e60332 100644 --- a/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_hwseq.c +++ b/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_hwseq.c @@ -46,7 +46,6 @@ #include "dchubbub.h" #include "reg_helper.h" #include "dcn10/dcn10_cm_common.h" -#include "dc_link_dp.h" #include "vm_helper.h" #include "dccg.h" #include "dc_dmub_srv.h" @@ -2023,8 +2022,11 @@ void dcn20_prepare_bandwidth( } } - /* program dchubbub watermarks */ - dc->wm_optimized_required = hubbub->funcs->program_watermarks(hubbub, + /* program dchubbub watermarks: + * For assigning wm_optimized_required, use |= operator since we don't want + * to clear the value if the optimize has not happened yet + */ + dc->wm_optimized_required |= hubbub->funcs->program_watermarks(hubbub, &context->bw_ctx.bw.dcn.watermarks, dc->res_pool->ref_clocks.dchub_ref_clock_inKhz / 1000, false); diff --git a/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_stream_encoder.c b/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_stream_encoder.c index cacf3f5298b099448ecb76b78db957ad790a8352..48f662b031ebca4f33cf22fa3a9840fd9212412f 100644 --- a/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_stream_encoder.c +++ b/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_stream_encoder.c @@ -29,7 +29,7 @@ #include "dcn20_stream_encoder.h" #include "reg_helper.h" #include "hw_shared.h" -#include "dc_link_dp.h" +#include "link.h" #include "dpcd_defs.h" #define DC_LOGGER \ @@ -423,6 +423,22 @@ void enc2_set_dynamic_metadata(struct stream_encoder *enc, } } +static void enc2_stream_encoder_update_dp_info_packets_sdp_line_num( + struct stream_encoder *enc, + struct encoder_info_frame *info_frame) +{ + struct dcn10_stream_encoder *enc1 = DCN10STRENC_FROM_STRENC(enc); + + if (info_frame->adaptive_sync.valid == true && + info_frame->sdp_line_num.adaptive_sync_line_num_valid == true) { + //00: REFER_TO_DP_SOF, 01: REFER_TO_OTG_SOF + REG_UPDATE(DP_SEC_CNTL1, DP_SEC_GSP5_LINE_REFERENCE, 1); + + REG_UPDATE(DP_SEC_CNTL5, DP_SEC_GSP5_LINE_NUM, + info_frame->sdp_line_num.adaptive_sync_line_num); + } +} + static void enc2_stream_encoder_update_dp_info_packets( struct stream_encoder *enc, const struct encoder_info_frame *info_frame) @@ -587,6 +603,8 @@ static const struct stream_encoder_funcs dcn20_str_enc_funcs = { enc2_stream_encoder_update_hdmi_info_packets, .stop_hdmi_info_packets = enc2_stream_encoder_stop_hdmi_info_packets, + .update_dp_info_packets_sdp_line_num = + enc2_stream_encoder_update_dp_info_packets_sdp_line_num, .update_dp_info_packets = enc2_stream_encoder_update_dp_info_packets, .send_immediate_sdp_message = diff --git a/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_dio_stream_encoder.c b/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_dio_stream_encoder.c index 17df53793c9221de444e571891f09d77e8d70a7f..5f9079d3943a67e1fb2250c108996d60852bc9ff 100644 --- a/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_dio_stream_encoder.c +++ b/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_dio_stream_encoder.c @@ -404,6 +404,22 @@ static void enc3_read_state(struct stream_encoder *enc, struct enc_state *s) } } +void enc3_stream_encoder_update_dp_info_packets_sdp_line_num( + struct stream_encoder *enc, + struct encoder_info_frame *info_frame) +{ + struct dcn10_stream_encoder *enc1 = DCN10STRENC_FROM_STRENC(enc); + + if (info_frame->adaptive_sync.valid == true && + info_frame->sdp_line_num.adaptive_sync_line_num_valid == true) { + //00: REFER_TO_DP_SOF, 01: REFER_TO_OTG_SOF + REG_UPDATE(DP_SEC_CNTL1, DP_SEC_GSP5_LINE_REFERENCE, 1); + + REG_UPDATE(DP_SEC_CNTL5, DP_SEC_GSP5_LINE_NUM, + info_frame->sdp_line_num.adaptive_sync_line_num); + } +} + void enc3_stream_encoder_update_dp_info_packets( struct stream_encoder *enc, const struct encoder_info_frame *info_frame) @@ -452,12 +468,20 @@ void enc3_stream_encoder_update_dp_info_packets( * use other packetIndex (such as 5,6) for other info packet */ + if (info_frame->adaptive_sync.valid) + enc->vpg->funcs->update_generic_info_packet( + enc->vpg, + 5, /* packetIndex */ + &info_frame->adaptive_sync, + true); + /* enable/disable transmission of packet(s). * If enabled, packet transmission begins on the next frame */ REG_UPDATE(DP_SEC_CNTL, DP_SEC_GSP0_ENABLE, info_frame->vsc.valid); REG_UPDATE(DP_SEC_CNTL, DP_SEC_GSP2_ENABLE, info_frame->spd.valid); REG_UPDATE(DP_SEC_CNTL, DP_SEC_GSP3_ENABLE, info_frame->hdrsmd.valid); + REG_UPDATE(DP_SEC_CNTL, DP_SEC_GSP5_ENABLE, info_frame->adaptive_sync.valid); /* This bit is the master enable bit. * When enabling secondary stream engine, @@ -803,6 +827,8 @@ static const struct stream_encoder_funcs dcn30_str_enc_funcs = { enc3_stream_encoder_update_hdmi_info_packets, .stop_hdmi_info_packets = enc3_stream_encoder_stop_hdmi_info_packets, + .update_dp_info_packets_sdp_line_num = + enc3_stream_encoder_update_dp_info_packets_sdp_line_num, .update_dp_info_packets = enc3_stream_encoder_update_dp_info_packets, .stop_dp_info_packets = diff --git a/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_dio_stream_encoder.h b/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_dio_stream_encoder.h index 54ee230e7f98d8dfdd94cb8f8347f38058703521..06310973ded2d95ed5963b8f7f7bbc3f47a121ef 100644 --- a/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_dio_stream_encoder.h +++ b/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_dio_stream_encoder.h @@ -292,6 +292,10 @@ void enc3_stream_encoder_update_hdmi_info_packets( void enc3_stream_encoder_stop_hdmi_info_packets( struct stream_encoder *enc); +void enc3_stream_encoder_update_dp_info_packets_sdp_line_num( + struct stream_encoder *enc, + struct encoder_info_frame *info_frame); + void enc3_stream_encoder_update_dp_info_packets( struct stream_encoder *enc, const struct encoder_info_frame *info_frame); diff --git a/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_hwseq.c b/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_hwseq.c index 7360b3ce4283c64f23f131e43b78661b70165d7d..444f9fad3de6aa75d51836a396186c6730227b58 100644 --- a/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_hwseq.c +++ b/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_hwseq.c @@ -50,7 +50,7 @@ #include "dpcd_defs.h" #include "../dcn20/dcn20_hwseq.h" #include "dcn30_resource.h" -#include "inc/dc_link_dp.h" +#include "link.h" @@ -674,10 +674,16 @@ void dcn30_update_info_frame(struct pipe_ctx *pipe_ctx) pipe_ctx->stream_res.stream_enc->funcs->update_hdmi_info_packets( pipe_ctx->stream_res.stream_enc, &pipe_ctx->stream_res.encoder_info_frame); - else + else { + if (pipe_ctx->stream_res.stream_enc->funcs->update_dp_info_packets_sdp_line_num) + pipe_ctx->stream_res.stream_enc->funcs->update_dp_info_packets_sdp_line_num( + pipe_ctx->stream_res.stream_enc, + &pipe_ctx->stream_res.encoder_info_frame); + pipe_ctx->stream_res.stream_enc->funcs->update_dp_info_packets( pipe_ctx->stream_res.stream_enc, &pipe_ctx->stream_res.encoder_info_frame); + } } void dcn30_program_dmdata_engine(struct pipe_ctx *pipe_ctx) diff --git a/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_hpo_dp_stream_encoder.c b/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_hpo_dp_stream_encoder.c index 16639bd03adf7c40ea2ed54be5a7a7d8dbc3c496..d76f55a12eb41ce0cd6f9a6a9fdfd6c5a3eb0fae 100644 --- a/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_hpo_dp_stream_encoder.c +++ b/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_hpo_dp_stream_encoder.c @@ -430,6 +430,22 @@ static void dcn31_hpo_dp_stream_enc_set_stream_attribute( MSA_DATA_LANE_3, 0); } +static void dcn31_hpo_dp_stream_enc_update_dp_info_packets_sdp_line_num( + struct hpo_dp_stream_encoder *enc, + struct encoder_info_frame *info_frame) +{ + struct dcn31_hpo_dp_stream_encoder *enc3 = DCN3_1_HPO_DP_STREAM_ENC_FROM_HPO_STREAM_ENC(enc); + + if (info_frame->adaptive_sync.valid == true && + info_frame->sdp_line_num.adaptive_sync_line_num_valid == true) { + //00: REFER_TO_DP_SOF, 01: REFER_TO_OTG_SOF + REG_UPDATE(DP_SYM32_ENC_SDP_GSP_CONTROL5, GSP_SOF_REFERENCE, 1); + + REG_UPDATE(DP_SYM32_ENC_SDP_GSP_CONTROL5, GSP_TRANSMISSION_LINE_NUMBER, + info_frame->sdp_line_num.adaptive_sync_line_num); + } +} + static void dcn31_hpo_dp_stream_enc_update_dp_info_packets( struct hpo_dp_stream_encoder *enc, const struct encoder_info_frame *info_frame) @@ -458,12 +474,20 @@ static void dcn31_hpo_dp_stream_enc_update_dp_info_packets( &info_frame->hdrsmd, true); + if (info_frame->adaptive_sync.valid) + enc->vpg->funcs->update_generic_info_packet( + enc->vpg, + 5, /* packetIndex */ + &info_frame->adaptive_sync, + true); + /* enable/disable transmission of packet(s). * If enabled, packet transmission begins on the next frame */ REG_UPDATE(DP_SYM32_ENC_SDP_GSP_CONTROL0, GSP_VIDEO_CONTINUOUS_TRANSMISSION_ENABLE, info_frame->vsc.valid); REG_UPDATE(DP_SYM32_ENC_SDP_GSP_CONTROL2, GSP_VIDEO_CONTINUOUS_TRANSMISSION_ENABLE, info_frame->spd.valid); REG_UPDATE(DP_SYM32_ENC_SDP_GSP_CONTROL3, GSP_VIDEO_CONTINUOUS_TRANSMISSION_ENABLE, info_frame->hdrsmd.valid); + REG_UPDATE(DP_SYM32_ENC_SDP_GSP_CONTROL5, GSP_VIDEO_CONTINUOUS_TRANSMISSION_ENABLE, info_frame->adaptive_sync.valid); /* check if dynamic metadata packet transmission is enabled */ REG_GET(DP_SYM32_ENC_SDP_METADATA_PACKET_CONTROL, @@ -714,6 +738,7 @@ static const struct hpo_dp_stream_encoder_funcs dcn30_str_enc_funcs = { .dp_blank = dcn31_hpo_dp_stream_enc_dp_blank, .disable = dcn31_hpo_dp_stream_enc_disable, .set_stream_attribute = dcn31_hpo_dp_stream_enc_set_stream_attribute, + .update_dp_info_packets_sdp_line_num = dcn31_hpo_dp_stream_enc_update_dp_info_packets_sdp_line_num, .update_dp_info_packets = dcn31_hpo_dp_stream_enc_update_dp_info_packets, .stop_dp_info_packets = dcn31_hpo_dp_stream_enc_stop_dp_info_packets, .dp_set_dsc_pps_info_packet = dcn31_hpo_dp_stream_enc_set_dsc_pps_info_packet, diff --git a/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_hwseq.c b/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_hwseq.c index 0e1949d9ea582c5702f515197aec30179d39bba7..9871f9e189d3dfac62cc785b71a86bc46b510f25 100644 --- a/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_hwseq.c +++ b/drivers/gpu/drm/amd/display/dc/dcn31/dcn31_hwseq.c @@ -45,7 +45,6 @@ #include "link_hwss.h" #include "dpcd_defs.h" #include "dce/dmub_outbox.h" -#include "dc_link_dp.h" #include "link.h" #include "dcn10/dcn10_hw_sequencer.h" #include "inc/link_enc_cfg.h" @@ -421,6 +420,11 @@ void dcn31_update_info_frame(struct pipe_ctx *pipe_ctx) &pipe_ctx->stream_res.encoder_info_frame); return; } else { + if (pipe_ctx->stream_res.stream_enc->funcs->update_dp_info_packets_sdp_line_num) + pipe_ctx->stream_res.stream_enc->funcs->update_dp_info_packets_sdp_line_num( + pipe_ctx->stream_res.stream_enc, + &pipe_ctx->stream_res.encoder_info_frame); + pipe_ctx->stream_res.stream_enc->funcs->update_dp_info_packets( pipe_ctx->stream_res.stream_enc, &pipe_ctx->stream_res.encoder_info_frame); diff --git a/drivers/gpu/drm/amd/display/dc/dcn314/dcn314_dio_stream_encoder.c b/drivers/gpu/drm/amd/display/dc/dcn314/dcn314_dio_stream_encoder.c index 67f4589f3e2392d6fb54f037fdf1dcda836d8245..b818ebe7d06de769ed94cecfd20063b5d1454148 100644 --- a/drivers/gpu/drm/amd/display/dc/dcn314/dcn314_dio_stream_encoder.c +++ b/drivers/gpu/drm/amd/display/dc/dcn314/dcn314_dio_stream_encoder.c @@ -30,7 +30,7 @@ #include "dcn314_dio_stream_encoder.h" #include "reg_helper.h" #include "hw_shared.h" -#include "dc_link_dp.h" +#include "link.h" #include "dpcd_defs.h" #define DC_LOGGER \ @@ -429,6 +429,8 @@ static const struct stream_encoder_funcs dcn314_str_enc_funcs = { enc3_stream_encoder_update_hdmi_info_packets, .stop_hdmi_info_packets = enc3_stream_encoder_stop_hdmi_info_packets, + .update_dp_info_packets_sdp_line_num = + enc3_stream_encoder_update_dp_info_packets_sdp_line_num, .update_dp_info_packets = enc3_stream_encoder_update_dp_info_packets, .stop_dp_info_packets = diff --git a/drivers/gpu/drm/amd/display/dc/dcn314/dcn314_dio_stream_encoder.h b/drivers/gpu/drm/amd/display/dc/dcn314/dcn314_dio_stream_encoder.h index 33dfdf8b4100fefaaaa0d4ce7f866fb124c499f6..ed07723879036a9bfba404e8dae0ecf21d0b1ed9 100644 --- a/drivers/gpu/drm/amd/display/dc/dcn314/dcn314_dio_stream_encoder.h +++ b/drivers/gpu/drm/amd/display/dc/dcn314/dcn314_dio_stream_encoder.h @@ -280,6 +280,10 @@ void enc3_stream_encoder_update_hdmi_info_packets( void enc3_stream_encoder_stop_hdmi_info_packets( struct stream_encoder *enc); +void enc3_stream_encoder_update_dp_info_packets_sdp_line_num( + struct stream_encoder *enc, + struct encoder_info_frame *info_frame); + void enc3_stream_encoder_update_dp_info_packets( struct stream_encoder *enc, const struct encoder_info_frame *info_frame); diff --git a/drivers/gpu/drm/amd/display/dc/dcn314/dcn314_hwseq.c b/drivers/gpu/drm/amd/display/dc/dcn314/dcn314_hwseq.c index 7980462e3abefcf26bd6d039f4a245199a27282d..d725260336fe62b846b35f7f841c3df8e7a4e7ab 100644 --- a/drivers/gpu/drm/amd/display/dc/dcn314/dcn314_hwseq.c +++ b/drivers/gpu/drm/amd/display/dc/dcn314/dcn314_hwseq.c @@ -46,9 +46,7 @@ #include "link_hwss.h" #include "dpcd_defs.h" #include "dce/dmub_outbox.h" -#include "dc_link_dp.h" #include "link.h" -#include "inc/dc_link_dp.h" #include "dcn10/dcn10_hw_sequencer.h" #include "inc/link_enc_cfg.h" #include "dcn30/dcn30_vpg.h" diff --git a/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_dio_stream_encoder.c b/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_dio_stream_encoder.c index f01968f6d18263bb9c024538f6b21f3e3551d252..3c0ce9404699ab47d606b5268fa7f9db31959a5c 100644 --- a/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_dio_stream_encoder.c +++ b/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_dio_stream_encoder.c @@ -29,7 +29,7 @@ #include "dcn32_dio_stream_encoder.h" #include "reg_helper.h" #include "hw_shared.h" -#include "dc_link_dp.h" +#include "link.h" #include "dpcd_defs.h" #define DC_LOGGER \ @@ -463,6 +463,8 @@ static const struct stream_encoder_funcs dcn32_str_enc_funcs = { enc3_stream_encoder_update_hdmi_info_packets, .stop_hdmi_info_packets = enc3_stream_encoder_stop_hdmi_info_packets, + .update_dp_info_packets_sdp_line_num = + enc3_stream_encoder_update_dp_info_packets_sdp_line_num, .update_dp_info_packets = enc3_stream_encoder_update_dp_info_packets, .stop_dp_info_packets = diff --git a/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_hwseq.c b/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_hwseq.c index 3b44006e1a80d7199f8ee3326050cdf84eb197a3..addfde594622232590393f558e66a013b29e3af1 100644 --- a/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_hwseq.c +++ b/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_hwseq.c @@ -51,7 +51,6 @@ #include "dce/dmub_hw_lock_mgr.h" #include "dcn32_resource.h" #include "link.h" -#include "dc_link_dp.h" #include "dmub/inc/dmub_subvp_state.h" #define DC_LOGGER_INIT(logger) @@ -247,6 +246,13 @@ bool dcn32_apply_idle_power_optimizations(struct dc *dc, bool enable) if (!dc->ctx->dmub_srv) return false; + for (i = 0; i < dc->current_state->stream_count; i++) { + /* MALL SS messaging is not supported with PSR at this time */ + if (dc->current_state->streams[i] != NULL && + dc->current_state->streams[i]->link->psr_settings.psr_version != DC_PSR_VERSION_UNSUPPORTED) + return false; + } + if (enable) { if (dc->current_state) { @@ -689,6 +695,7 @@ static void dcn32_initialize_min_clocks(struct dc *dc) { struct dc_clocks *clocks = &dc->current_state->bw_ctx.bw.dcn.clk; + clocks->dcfclk_deep_sleep_khz = DCN3_2_DCFCLK_DS_INIT_KHZ; clocks->dcfclk_khz = dc->clk_mgr->bw_params->clk_table.entries[0].dcfclk_mhz * 1000; clocks->socclk_khz = dc->clk_mgr->bw_params->clk_table.entries[0].socclk_mhz * 1000; clocks->dramclk_khz = dc->clk_mgr->bw_params->clk_table.entries[0].memclk_mhz * 1000; @@ -800,6 +807,16 @@ void dcn32_init_hw(struct dc *dc) !dc->res_pool->hubbub->ctx->dc->debug.disable_stutter); dcn32_initialize_min_clocks(dc); + + /* On HW init, allow idle optimizations after pipes have been turned off. + * + * In certain D3 cases (i.e. BOCO / BOMACO) it's possible that hardware state + * is reset (i.e. not in idle at the time hw init is called), but software state + * still has idle_optimizations = true, so we must disable idle optimizations first + * (i.e. set false), then re-enable (set true). + */ + dc_allow_idle_optimizations(dc, false); + dc_allow_idle_optimizations(dc, true); } /* In headless boot cases, DIG may be turned diff --git a/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_resource.c b/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_resource.c index 47dc96acdacb2a3d7b258c14c41cecb986de7115..74e50c09bb62f9156a8436d4b86025e5b9b5b8a9 100644 --- a/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_resource.c +++ b/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_resource.c @@ -57,7 +57,6 @@ #include "dcn31/dcn31_hpo_dp_stream_encoder.h" #include "dcn31/dcn31_hpo_dp_link_encoder.h" #include "dcn32/dcn32_hpo_dp_link_encoder.h" -#include "dc_link_dp.h" #include "dcn31/dcn31_apg.h" #include "dcn31/dcn31_dio_link_encoder.h" #include "dcn32/dcn32_dio_link_encoder.h" @@ -2150,13 +2149,19 @@ static bool dcn32_resource_construct( dc->caps.max_cursor_size = 64; dc->caps.min_horizontal_blanking_period = 80; dc->caps.dmdata_alloc_size = 2048; - dc->caps.mall_size_per_mem_channel = 0; + dc->caps.mall_size_per_mem_channel = 4; dc->caps.mall_size_total = 0; dc->caps.cursor_cache_size = dc->caps.max_cursor_size * dc->caps.max_cursor_size * 8; dc->caps.cache_line_size = 64; dc->caps.cache_num_ways = 16; - dc->caps.max_cab_allocation_bytes = 67108864; // 64MB = 1024 * 1024 * 64 + + /* Calculate the available MALL space */ + dc->caps.max_cab_allocation_bytes = dcn32_calc_num_avail_chans_for_mall( + dc, dc->ctx->dc_bios->vram_info.num_chans) * + dc->caps.mall_size_per_mem_channel * 1024 * 1024; + dc->caps.mall_size_total = dc->caps.max_cab_allocation_bytes; + dc->caps.subvp_fw_processing_delay_us = 15; dc->caps.subvp_drr_max_vblank_margin_us = 40; dc->caps.subvp_prefetch_end_to_mall_start_us = 15; @@ -2593,3 +2598,55 @@ struct pipe_ctx *dcn32_acquire_idle_pipe_for_head_pipe_in_layer( return idle_pipe; } + +unsigned int dcn32_calc_num_avail_chans_for_mall(struct dc *dc, int num_chans) +{ + /* + * DCN32 and DCN321 SKUs may have different sizes for MALL + * but we may not be able to access all the MALL space. + * If the num_chans is power of 2, then we can access all + * of the available MALL space. Otherwise, we can only + * access: + * + * max_cab_size_in_bytes = total_cache_size_in_bytes * + * ((2^floor(log2(num_chans)))/num_chans) + * + * Calculating the MALL sizes for all available SKUs, we + * have come up with the follow simplified check. + * - we have max_chans which provides the max MALL size. + * Each chans supports 4MB of MALL so: + * + * total_cache_size_in_bytes = max_chans * 4 MB + * + * - we have avail_chans which shows the number of channels + * we can use if we can't access the entire MALL space. + * It is generally half of max_chans + * - so we use the following checks: + * + * if (num_chans == max_chans), return max_chans + * if (num_chans < max_chans), return avail_chans + * + * - exception is GC_11_0_0 where we can't access max_chans, + * so we define max_avail_chans as the maximum available + * MALL space + * + */ + int gc_11_0_0_max_chans = 48; + int gc_11_0_0_max_avail_chans = 32; + int gc_11_0_0_avail_chans = 16; + int gc_11_0_3_max_chans = 16; + int gc_11_0_3_avail_chans = 8; + int gc_11_0_2_max_chans = 8; + int gc_11_0_2_avail_chans = 4; + + if (ASICREV_IS_GC_11_0_0(dc->ctx->asic_id.hw_internal_rev)) { + return (num_chans == gc_11_0_0_max_chans) ? + gc_11_0_0_max_avail_chans : gc_11_0_0_avail_chans; + } else if (ASICREV_IS_GC_11_0_2(dc->ctx->asic_id.hw_internal_rev)) { + return (num_chans == gc_11_0_2_max_chans) ? + gc_11_0_2_max_chans : gc_11_0_2_avail_chans; + } else { // if (ASICREV_IS_GC_11_0_3(dc->ctx->asic_id.hw_internal_rev)) { + return (num_chans == gc_11_0_3_max_chans) ? + gc_11_0_3_max_chans : gc_11_0_3_avail_chans; + } +} diff --git a/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_resource.h b/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_resource.h index b07d3b0e6a5c8695f81efcfb564e181f0e23b5e2..a09db7c63a04fcb32623811e65587462df6582ff 100644 --- a/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_resource.h +++ b/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_resource.h @@ -38,6 +38,7 @@ #define DCN3_2_MBLK_HEIGHT_4BPE 128 #define DCN3_2_MBLK_HEIGHT_8BPE 64 #define DCN3_2_VMIN_DISPCLK_HZ 717000000 +#define DCN3_2_DCFCLK_DS_INIT_KHZ 10000 // Choose 10Mhz for init DCFCLK DS freq #define TO_DCN32_RES_POOL(pool)\ container_of(pool, struct dcn32_resource_pool, base) @@ -122,6 +123,7 @@ bool dcn32_mpo_in_use(struct dc_state *context); bool dcn32_any_surfaces_rotated(struct dc *dc, struct dc_state *context); bool dcn32_is_center_timing(struct pipe_ctx *pipe); +bool dcn32_is_psr_capable(struct pipe_ctx *pipe); struct pipe_ctx *dcn32_acquire_idle_pipe_for_head_pipe_in_layer( struct dc_state *state, @@ -146,6 +148,8 @@ void dcn32_restore_mall_state(struct dc *dc, bool dcn32_allow_subvp_with_active_margin(struct pipe_ctx *pipe); +unsigned int dcn32_calc_num_avail_chans_for_mall(struct dc *dc, int num_chans); + /* definitions for run time init of reg offsets */ /* CLK SRC */ diff --git a/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_resource_helpers.c b/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_resource_helpers.c index 0fc79d75ce766eb65f99c2e7d02817a2e9e7faee..3a2d7bcc4b6d6cb62a7238a7438ee7ed9cc7d009 100644 --- a/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_resource_helpers.c +++ b/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_resource_helpers.c @@ -251,6 +251,16 @@ bool dcn32_is_center_timing(struct pipe_ctx *pipe) return is_center_timing; } +bool dcn32_is_psr_capable(struct pipe_ctx *pipe) +{ + bool psr_capable = false; + + if (pipe->stream && pipe->stream->link->psr_settings.psr_version != DC_PSR_VERSION_UNSUPPORTED) { + psr_capable = true; + } + return psr_capable; +} + /** * ******************************************************************************************* * dcn32_determine_det_override: Determine DET allocation for each pipe diff --git a/drivers/gpu/drm/amd/display/dc/dcn321/dcn321_resource.c b/drivers/gpu/drm/amd/display/dc/dcn321/dcn321_resource.c index 260d71ca020568db11f28735fa0608c160daf18a..fd57e0167737108d4c082ce09b6c18d02dbd95f0 100644 --- a/drivers/gpu/drm/amd/display/dc/dcn321/dcn321_resource.c +++ b/drivers/gpu/drm/amd/display/dc/dcn321/dcn321_resource.c @@ -60,7 +60,6 @@ #include "dcn31/dcn31_hpo_dp_stream_encoder.h" #include "dcn31/dcn31_hpo_dp_link_encoder.h" #include "dcn32/dcn32_hpo_dp_link_encoder.h" -#include "dc_link_dp.h" #include "dcn31/dcn31_apg.h" #include "dcn31/dcn31_dio_link_encoder.h" #include "dcn32/dcn32_dio_link_encoder.h" @@ -1703,11 +1702,18 @@ static bool dcn321_resource_construct( dc->caps.max_cursor_size = 64; dc->caps.min_horizontal_blanking_period = 80; dc->caps.dmdata_alloc_size = 2048; - dc->caps.mall_size_per_mem_channel = 0; + dc->caps.mall_size_per_mem_channel = 4; dc->caps.mall_size_total = 0; dc->caps.cursor_cache_size = dc->caps.max_cursor_size * dc->caps.max_cursor_size * 8; dc->caps.cache_line_size = 64; dc->caps.cache_num_ways = 16; + + /* Calculate the available MALL space */ + dc->caps.max_cab_allocation_bytes = dcn32_calc_num_avail_chans_for_mall( + dc, dc->ctx->dc_bios->vram_info.num_chans) * + dc->caps.mall_size_per_mem_channel * 1024 * 1024; + dc->caps.mall_size_total = dc->caps.max_cab_allocation_bytes; + dc->caps.max_cab_allocation_bytes = 33554432; // 32MB = 1024 * 1024 * 32 dc->caps.subvp_fw_processing_delay_us = 15; dc->caps.subvp_drr_max_vblank_margin_us = 40; diff --git a/drivers/gpu/drm/amd/display/dc/dm_helpers.h b/drivers/gpu/drm/amd/display/dc/dm_helpers.h index af1c50ed905abdfa3b26ca5a272bc889ca13f857..c624c768cc3d1df0973e30b2e86f7ac96ea2da43 100644 --- a/drivers/gpu/drm/amd/display/dc/dm_helpers.h +++ b/drivers/gpu/drm/amd/display/dc/dm_helpers.h @@ -161,6 +161,12 @@ enum dc_edid_status dm_helpers_read_local_edid( struct dc_link *link, struct dc_sink *sink); +bool dm_helpers_dp_handle_test_pattern_request( + struct dc_context *ctx, + const struct dc_link *link, + union link_test_pattern dpcd_test_pattern, + union test_misc dpcd_test_params); + void dm_set_dcn_clocks( struct dc_context *ctx, struct dc_clocks *clks); diff --git a/drivers/gpu/drm/amd/display/dc/dml/dcn20/dcn20_fpu.c b/drivers/gpu/drm/amd/display/dc/dml/dcn20/dcn20_fpu.c index d2b89c50be2a10b398124bacf5ed2c32538848ca..197df404761ac219db24ebbad1f9f17be2080a25 100644 --- a/drivers/gpu/drm/amd/display/dc/dml/dcn20/dcn20_fpu.c +++ b/drivers/gpu/drm/amd/display/dc/dml/dcn20/dcn20_fpu.c @@ -26,7 +26,6 @@ #include "resource.h" #include "clk_mgr.h" -#include "dc_link_dp.h" #include "dchubbub.h" #include "dcn20/dcn20_resource.h" #include "dcn21/dcn21_resource.h" @@ -1004,6 +1003,39 @@ static enum dcn_zstate_support_state decide_zstate_support(struct dc *dc, struc } } +static void dcn20_adjust_freesync_v_startup( + const struct dc_crtc_timing *dc_crtc_timing, int *vstartup_start) +{ + struct dc_crtc_timing patched_crtc_timing; + uint32_t asic_blank_end = 0; + uint32_t asic_blank_start = 0; + uint32_t newVstartup = 0; + + patched_crtc_timing = *dc_crtc_timing; + + if (patched_crtc_timing.flags.INTERLACE == 1) { + if (patched_crtc_timing.v_front_porch < 2) + patched_crtc_timing.v_front_porch = 2; + } else { + if (patched_crtc_timing.v_front_porch < 1) + patched_crtc_timing.v_front_porch = 1; + } + + /* blank_start = frame end - front porch */ + asic_blank_start = patched_crtc_timing.v_total - + patched_crtc_timing.v_front_porch; + + /* blank_end = blank_start - active */ + asic_blank_end = asic_blank_start - + patched_crtc_timing.v_border_bottom - + patched_crtc_timing.v_addressable - + patched_crtc_timing.v_border_top; + + newVstartup = asic_blank_end + (patched_crtc_timing.v_total - asic_blank_start); + + *vstartup_start = ((newVstartup > *vstartup_start) ? newVstartup : *vstartup_start); +} + void dcn20_calculate_dlg_params( struct dc *dc, struct dc_state *context, display_e2e_pipe_params_st *pipes, @@ -1063,6 +1095,11 @@ void dcn20_calculate_dlg_params( context->res_ctx.pipe_ctx[i].plane_res.bw.dppclk_khz = pipes[pipe_idx].clks_cfg.dppclk_mhz * 1000; context->res_ctx.pipe_ctx[i].pipe_dlg_param = pipes[pipe_idx].pipe.dest; + if (context->res_ctx.pipe_ctx[i].stream->adaptive_sync_infopacket.valid) + dcn20_adjust_freesync_v_startup( + &context->res_ctx.pipe_ctx[i].stream->timing, + &context->res_ctx.pipe_ctx[i].pipe_dlg_param.vstartup_start); + pipe_idx++; } /*save a original dppclock copy*/ diff --git a/drivers/gpu/drm/amd/display/dc/dml/dcn32/dcn32_fpu.c b/drivers/gpu/drm/amd/display/dc/dml/dcn32/dcn32_fpu.c index 7feb8759e4755303c9e2449e91d791e55c7ffc81..0dc1a03999b6f52246d952bf2f6e8601488fdc52 100644 --- a/drivers/gpu/drm/amd/display/dc/dml/dcn32/dcn32_fpu.c +++ b/drivers/gpu/drm/amd/display/dc/dml/dcn32/dcn32_fpu.c @@ -24,7 +24,6 @@ * */ #include "dcn32_fpu.h" -#include "dc_link_dp.h" #include "dcn32/dcn32_resource.h" #include "dcn20/dcn20_resource.h" #include "display_mode_vba_util_32.h" @@ -692,7 +691,7 @@ static bool dcn32_assign_subvp_pipe(struct dc *dc, * to combine this with SubVP can cause issues with the scheduling). * - Not TMZ surface */ - if (pipe->plane_state && !pipe->top_pipe && !dcn32_is_center_timing(pipe) && + if (pipe->plane_state && !pipe->top_pipe && !dcn32_is_center_timing(pipe) && !dcn32_is_psr_capable(pipe) && pipe->stream->mall_stream_config.type == SUBVP_NONE && refresh_rate < 120 && !pipe->plane_state->address.tmz_surface && (vba->ActiveDRAMClockChangeLatencyMarginPerState[vba->VoltageLevel][vba->maxMpcComb][vba->pipe_plane[pipe_idx]] <= 0 || (vba->ActiveDRAMClockChangeLatencyMarginPerState[vba->VoltageLevel][vba->maxMpcComb][vba->pipe_plane[pipe_idx]] > 0 && @@ -1272,6 +1271,38 @@ static bool is_dtbclk_required(struct dc *dc, struct dc_state *context) return false; } +static void dcn20_adjust_freesync_v_startup(const struct dc_crtc_timing *dc_crtc_timing, int *vstartup_start) +{ + struct dc_crtc_timing patched_crtc_timing; + uint32_t asic_blank_end = 0; + uint32_t asic_blank_start = 0; + uint32_t newVstartup = 0; + + patched_crtc_timing = *dc_crtc_timing; + + if (patched_crtc_timing.flags.INTERLACE == 1) { + if (patched_crtc_timing.v_front_porch < 2) + patched_crtc_timing.v_front_porch = 2; + } else { + if (patched_crtc_timing.v_front_porch < 1) + patched_crtc_timing.v_front_porch = 1; + } + + /* blank_start = frame end - front porch */ + asic_blank_start = patched_crtc_timing.v_total - + patched_crtc_timing.v_front_porch; + + /* blank_end = blank_start - active */ + asic_blank_end = asic_blank_start - + patched_crtc_timing.v_border_bottom - + patched_crtc_timing.v_addressable - + patched_crtc_timing.v_border_top; + + newVstartup = asic_blank_end + (patched_crtc_timing.v_total - asic_blank_start); + + *vstartup_start = ((newVstartup > *vstartup_start) ? newVstartup : *vstartup_start); +} + static void dcn32_calculate_dlg_params(struct dc *dc, struct dc_state *context, display_e2e_pipe_params_st *pipes, int pipe_cnt, int vlevel) @@ -1375,6 +1406,11 @@ static void dcn32_calculate_dlg_params(struct dc *dc, struct dc_state *context, } } + if (context->res_ctx.pipe_ctx[i].stream->adaptive_sync_infopacket.valid) + dcn20_adjust_freesync_v_startup( + &context->res_ctx.pipe_ctx[i].stream->timing, + &context->res_ctx.pipe_ctx[i].pipe_dlg_param.vstartup_start); + pipe_idx++; } /* If DCN isn't making memory requests we can allow pstate change and lower clocks */ @@ -1601,16 +1637,12 @@ bool dcn32_internal_validate_bw(struct dc *dc, * to support with Prefetch mode 1 (dm_prefetch_support_fclk_and_stutter == 2) */ context->bw_ctx.dml.soc.allow_for_pstate_or_stutter_in_vblank_final = - dm_prefetch_support_fclk_and_stutter; + dm_prefetch_support_none; + context->bw_ctx.dml.validate_max_state = fast_validate; vlevel = dml_get_voltage_level(&context->bw_ctx.dml, pipes, pipe_cnt); - /* Last attempt with Prefetch mode 2 (dm_prefetch_support_stutter == 3) */ - if (vlevel == context->bw_ctx.dml.soc.num_states) { - context->bw_ctx.dml.soc.allow_for_pstate_or_stutter_in_vblank_final = - dm_prefetch_support_stutter; - vlevel = dml_get_voltage_level(&context->bw_ctx.dml, pipes, pipe_cnt); - } + context->bw_ctx.dml.validate_max_state = false; if (vlevel < context->bw_ctx.dml.soc.num_states) { memset(split, 0, sizeof(split)); @@ -2499,8 +2531,11 @@ void dcn32_update_bw_bounding_box_fpu(struct dc *dc, struct clk_bw_params *bw_pa } /* Override from VBIOS for num_chan */ - if (dc->ctx->dc_bios->vram_info.num_chans) + if (dc->ctx->dc_bios->vram_info.num_chans) { dcn3_2_soc.num_chans = dc->ctx->dc_bios->vram_info.num_chans; + dcn3_2_soc.mall_allocated_for_dcn_mbytes = (double)(dcn32_calc_num_avail_chans_for_mall(dc, + dc->ctx->dc_bios->vram_info.num_chans) * dc->caps.mall_size_per_mem_channel); + } if (dc->ctx->dc_bios->vram_info.dram_channel_width_bytes) dcn3_2_soc.dram_channel_width_bytes = dc->ctx->dc_bios->vram_info.dram_channel_width_bytes; diff --git a/drivers/gpu/drm/amd/display/dc/dml/dcn32/display_mode_vba_32.c b/drivers/gpu/drm/amd/display/dc/dml/dcn32/display_mode_vba_32.c index 6c5ab5c26b38240bafacc33f1fed2c1f7d1c9bca..23e50d15e9aefd2fa4995aa3b1fa751ae20b66f1 100644 --- a/drivers/gpu/drm/amd/display/dc/dml/dcn32/display_mode_vba_32.c +++ b/drivers/gpu/drm/amd/display/dc/dml/dcn32/display_mode_vba_32.c @@ -1639,9 +1639,14 @@ static void DISPCLKDPPCLKDCFCLKDeepSleepPrefetchParametersWatermarksAndPerforman static void mode_support_configuration(struct vba_vars_st *v, struct display_mode_lib *mode_lib) { - int i, j; + int i, j, start_state; - for (i = v->soc.num_states - 1; i >= 0; i--) { + if (mode_lib->validate_max_state) + start_state = v->soc.num_states - 1; + else + start_state = 0; + + for (i = v->soc.num_states - 1; i >= start_state; i--) { for (j = 0; j < 2; j++) { if (mode_lib->vba.ScaleRatioAndTapsSupport == true && mode_lib->vba.SourceFormatPixelAndScanSupport == true @@ -1710,7 +1715,7 @@ static void mode_support_configuration(struct vba_vars_st *v, void dml32_ModeSupportAndSystemConfigurationFull(struct display_mode_lib *mode_lib) { struct vba_vars_st *v = &mode_lib->vba; - int i, j; + int i, j, start_state; unsigned int k, m; unsigned int MaximumMPCCombine; unsigned int NumberOfNonCombinedSurfaceOfMaximumBandwidth; @@ -1723,6 +1728,10 @@ void dml32_ModeSupportAndSystemConfigurationFull(struct display_mode_lib *mode_l #endif /*MODE SUPPORT, VOLTAGE STATE AND SOC CONFIGURATION*/ + if (mode_lib->validate_max_state) + start_state = v->soc.num_states - 1; + else + start_state = 0; /*Scale Ratio, taps Support Check*/ @@ -2012,7 +2021,7 @@ void dml32_ModeSupportAndSystemConfigurationFull(struct display_mode_lib *mode_l mode_lib->vba.MPCCombineMethodIncompatible = v->dummy_vars.dml32_ModeSupportAndSystemConfigurationFull.MPCCombineMethodAsNeededForPStateChangeAndVoltage && v->dummy_vars.dml32_ModeSupportAndSystemConfigurationFull.MPCCombineMethodAsPossible; - for (i = 0; i < v->soc.num_states; i++) { + for (i = start_state; i < v->soc.num_states; i++) { for (j = 0; j < 2; j++) { mode_lib->vba.TotalNumberOfActiveDPP[i][j] = 0; mode_lib->vba.TotalAvailablePipesSupport[i][j] = true; @@ -2289,7 +2298,7 @@ void dml32_ModeSupportAndSystemConfigurationFull(struct display_mode_lib *mode_l } } - for (i = 0; i < v->soc.num_states; ++i) { + for (i = start_state; i < v->soc.num_states; ++i) { mode_lib->vba.ExceededMultistreamSlots[i] = false; for (k = 0; k < mode_lib->vba.NumberOfActiveSurfaces; ++k) { if (mode_lib->vba.OutputMultistreamEn[k] == true && mode_lib->vba.OutputMultistreamId[k] == k) { @@ -2389,7 +2398,7 @@ void dml32_ModeSupportAndSystemConfigurationFull(struct display_mode_lib *mode_l } } - for (i = 0; i < v->soc.num_states; ++i) { + for (i = start_state; i < v->soc.num_states; ++i) { mode_lib->vba.DTBCLKRequiredMoreThanSupported[i] = false; for (k = 0; k < mode_lib->vba.NumberOfActiveSurfaces; ++k) { if (mode_lib->vba.BlendingAndTiming[k] == k @@ -2406,7 +2415,7 @@ void dml32_ModeSupportAndSystemConfigurationFull(struct display_mode_lib *mode_l } } - for (i = 0; i < v->soc.num_states; ++i) { + for (i = start_state; i < v->soc.num_states; ++i) { mode_lib->vba.ODMCombine2To1SupportCheckOK[i] = true; mode_lib->vba.ODMCombine4To1SupportCheckOK[i] = true; for (k = 0; k < mode_lib->vba.NumberOfActiveSurfaces; ++k) { @@ -2424,7 +2433,7 @@ void dml32_ModeSupportAndSystemConfigurationFull(struct display_mode_lib *mode_l } } - for (i = 0; i < v->soc.num_states; i++) { + for (i = start_state; i < v->soc.num_states; i++) { mode_lib->vba.DSCCLKRequiredMoreThanSupported[i] = false; for (k = 0; k <= mode_lib->vba.NumberOfActiveSurfaces - 1; k++) { if (mode_lib->vba.BlendingAndTiming[k] == k) { @@ -2461,7 +2470,7 @@ void dml32_ModeSupportAndSystemConfigurationFull(struct display_mode_lib *mode_l /* Check DSC Unit and Slices Support */ v->dummy_vars.dml32_ModeSupportAndSystemConfigurationFull.TotalDSCUnitsRequired = 0; - for (i = 0; i < v->soc.num_states; ++i) { + for (i = start_state; i < v->soc.num_states; ++i) { mode_lib->vba.NotEnoughDSCUnits[i] = false; mode_lib->vba.NotEnoughDSCSlices[i] = false; v->dummy_vars.dml32_ModeSupportAndSystemConfigurationFull.TotalDSCUnitsRequired = 0; @@ -2496,7 +2505,7 @@ void dml32_ModeSupportAndSystemConfigurationFull(struct display_mode_lib *mode_l } /*DSC Delay per state*/ - for (i = 0; i < v->soc.num_states; ++i) { + for (i = start_state; i < v->soc.num_states; ++i) { for (k = 0; k < mode_lib->vba.NumberOfActiveSurfaces; ++k) { mode_lib->vba.DSCDelayPerState[i][k] = dml32_DSCDelayRequirement( mode_lib->vba.RequiresDSC[i][k], mode_lib->vba.ODMCombineEnablePerState[i][k], @@ -2523,7 +2532,7 @@ void dml32_ModeSupportAndSystemConfigurationFull(struct display_mode_lib *mode_l //Calculate Swath, DET Configuration, DCFCLKDeepSleep // - for (i = 0; i < (int) v->soc.num_states; ++i) { + for (i = start_state; i < (int) v->soc.num_states; ++i) { for (j = 0; j <= 1; ++j) { for (k = 0; k < mode_lib->vba.NumberOfActiveSurfaces; ++k) { mode_lib->vba.RequiredDPPCLKThisState[k] = mode_lib->vba.RequiredDPPCLK[i][j][k]; @@ -2661,7 +2670,7 @@ void dml32_ModeSupportAndSystemConfigurationFull(struct display_mode_lib *mode_l mode_lib->vba.SurfaceSizeInMALL, &mode_lib->vba.ExceededMALLSize); - for (i = 0; i < v->soc.num_states; i++) { + for (i = start_state; i < v->soc.num_states; i++) { for (j = 0; j < 2; j++) { for (k = 0; k <= mode_lib->vba.NumberOfActiveSurfaces - 1; k++) { mode_lib->vba.swath_width_luma_ub_this_state[k] = @@ -2888,7 +2897,7 @@ void dml32_ModeSupportAndSystemConfigurationFull(struct display_mode_lib *mode_l } //Calculate Return BW - for (i = 0; i < (int) v->soc.num_states; ++i) { + for (i = start_state; i < (int) v->soc.num_states; ++i) { for (j = 0; j <= 1; ++j) { for (k = 0; k <= mode_lib->vba.NumberOfActiveSurfaces - 1; k++) { if (mode_lib->vba.BlendingAndTiming[k] == k) { @@ -2967,7 +2976,7 @@ void dml32_ModeSupportAndSystemConfigurationFull(struct display_mode_lib *mode_l &mode_lib->vba.MinPrefetchMode, &mode_lib->vba.MaxPrefetchMode); - for (i = 0; i < (int) v->soc.num_states; ++i) { + for (i = start_state; i < (int) v->soc.num_states; ++i) { for (j = 0; j <= 1; ++j) mode_lib->vba.DCFCLKState[i][j] = mode_lib->vba.DCFCLKPerState[i]; } @@ -3089,7 +3098,7 @@ void dml32_ModeSupportAndSystemConfigurationFull(struct display_mode_lib *mode_l mode_lib->vba.DCFCLKState); } // UseMinimumRequiredDCFCLK == true - for (i = 0; i < (int) v->soc.num_states; ++i) { + for (i = start_state; i < (int) v->soc.num_states; ++i) { for (j = 0; j <= 1; ++j) { mode_lib->vba.ReturnBWPerState[i][j] = dml32_get_return_bw_mbps(&mode_lib->vba.soc, i, mode_lib->vba.HostVMEnable, mode_lib->vba.DCFCLKState[i][j], @@ -3098,7 +3107,7 @@ void dml32_ModeSupportAndSystemConfigurationFull(struct display_mode_lib *mode_l } //Re-ordering Buffer Support Check - for (i = 0; i < (int) v->soc.num_states; ++i) { + for (i = start_state; i < (int) v->soc.num_states; ++i) { for (j = 0; j <= 1; ++j) { if ((mode_lib->vba.ROBBufferSizeInKByte - mode_lib->vba.PixelChunkSizeInKByte) * 1024 / mode_lib->vba.ReturnBWPerState[i][j] @@ -3120,7 +3129,7 @@ void dml32_ModeSupportAndSystemConfigurationFull(struct display_mode_lib *mode_l + mode_lib->vba.ReadBandwidthChroma[k]; } - for (i = 0; i < (int) v->soc.num_states; ++i) { + for (i = start_state; i < (int) v->soc.num_states; ++i) { for (j = 0; j <= 1; ++j) { mode_lib->vba.MaxTotalVerticalActiveAvailableBandwidth[i][j] = dml_min3(mode_lib->vba.ReturnBusWidth * mode_lib->vba.DCFCLKState[i][j] @@ -3144,7 +3153,7 @@ void dml32_ModeSupportAndSystemConfigurationFull(struct display_mode_lib *mode_l /* Prefetch Check */ - for (i = 0; i < (int) v->soc.num_states; ++i) { + for (i = start_state; i < (int) v->soc.num_states; ++i) { for (j = 0; j <= 1; ++j) { mode_lib->vba.TimeCalc = 24 / mode_lib->vba.ProjectedDCFCLKDeepSleep[i][j]; @@ -3662,7 +3671,7 @@ void dml32_ModeSupportAndSystemConfigurationFull(struct display_mode_lib *mode_l MaximumMPCCombine = 0; - for (i = v->soc.num_states; i >= 0; i--) { + for (i = v->soc.num_states; i >= start_state; i--) { if (i == v->soc.num_states || mode_lib->vba.ModeSupport[i][0] == true || mode_lib->vba.ModeSupport[i][1] == true) { mode_lib->vba.VoltageLevel = i; diff --git a/drivers/gpu/drm/amd/display/dc/dml/dcn321/dcn321_fpu.c b/drivers/gpu/drm/amd/display/dc/dml/dcn321/dcn321_fpu.c index 0ea406145c1d79397dbfe7d453951aa63ebf33b9..b80cef70fa60f113f7223e51fcfdfebb9c607dd1 100644 --- a/drivers/gpu/drm/amd/display/dc/dml/dcn321/dcn321_fpu.c +++ b/drivers/gpu/drm/amd/display/dc/dml/dcn321/dcn321_fpu.c @@ -534,8 +534,11 @@ void dcn321_update_bw_bounding_box_fpu(struct dc *dc, struct clk_bw_params *bw_p } /* Override from VBIOS for num_chan */ - if (dc->ctx->dc_bios->vram_info.num_chans) + if (dc->ctx->dc_bios->vram_info.num_chans) { dcn3_21_soc.num_chans = dc->ctx->dc_bios->vram_info.num_chans; + dcn3_21_soc.mall_allocated_for_dcn_mbytes = (double)(dcn32_calc_num_avail_chans_for_mall(dc, + dc->ctx->dc_bios->vram_info.num_chans) * dc->caps.mall_size_per_mem_channel); + } if (dc->ctx->dc_bios->vram_info.dram_channel_width_bytes) dcn3_21_soc.dram_channel_width_bytes = dc->ctx->dc_bios->vram_info.dram_channel_width_bytes; diff --git a/drivers/gpu/drm/amd/display/dc/dml/display_mode_lib.h b/drivers/gpu/drm/amd/display/dc/dml/display_mode_lib.h index 3d643d50c3eb59e69e99c4865dfb06803c1404fe..a9d49ef58fb59c07daac02ae46ae8274199a8157 100644 --- a/drivers/gpu/drm/amd/display/dc/dml/display_mode_lib.h +++ b/drivers/gpu/drm/amd/display/dc/dml/display_mode_lib.h @@ -91,6 +91,7 @@ struct display_mode_lib { struct dal_logger *logger; struct dml_funcs funcs; struct _vcs_dpi_display_e2e_pipe_params_st dml_pipe_state[6]; + bool validate_max_state; }; void dml_init_instance(struct display_mode_lib *lib, diff --git a/drivers/gpu/drm/amd/display/dc/hdcp/hdcp_msg.c b/drivers/gpu/drm/amd/display/dc/hdcp/hdcp_msg.c index 906a43e85f6dbadfef123f0df1c0f61f36ad6e9a..e1422e5e86c92e57573ddc43de405b90caa3c492 100644 --- a/drivers/gpu/drm/amd/display/dc/hdcp/hdcp_msg.c +++ b/drivers/gpu/drm/amd/display/dc/hdcp/hdcp_msg.c @@ -32,7 +32,7 @@ #include "core_types.h" #include "link.h" #include "link_hwss.h" -#include "link/link_dpcd.h" +#include "link/protocols/link_dpcd.h" #define DC_LOGGER \ link->ctx->logger diff --git a/drivers/gpu/drm/amd/display/dc/inc/dc_link_dp.h b/drivers/gpu/drm/amd/display/dc/inc/dc_link_dp.h deleted file mode 100644 index 52e1aad1fce8917abe4f1e821f42d88e2df4823e..0000000000000000000000000000000000000000 --- a/drivers/gpu/drm/amd/display/dc/inc/dc_link_dp.h +++ /dev/null @@ -1,117 +0,0 @@ -/* - * Copyright 2015 Advanced Micro Devices, Inc. - * - * Permission is hereby granted, free of charge, to any person obtaining a - * copy of this software and associated documentation files (the "Software"), - * to deal in the Software without restriction, including without limitation - * the rights to use, copy, modify, merge, publish, distribute, sublicense, - * and/or sell copies of the Software, and to permit persons to whom the - * Software is furnished to do so, subject to the following conditions: - * - * The above copyright notice and this permission notice shall be included in - * all copies or substantial portions of the Software. - * - * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR - * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, - * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL - * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR - * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, - * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR - * OTHER DEALINGS IN THE SOFTWARE. - * - * Authors: AMD - * - */ - -#ifndef __DC_LINK_DP_H__ -#define __DC_LINK_DP_H__ - -#define LINK_TRAINING_ATTEMPTS 4 -#define LINK_TRAINING_RETRY_DELAY 50 /* ms */ -#define MAX_MTP_SLOT_COUNT 64 -#define TRAINING_AUX_RD_INTERVAL 100 //us -#define LINK_AUX_WAKE_TIMEOUT_MS 1500 // Timeout when trying to wake unresponsive DPRX. - -struct dc_link; -struct dc_stream_state; -struct dc_link_settings; - -enum { - /* - * Some receivers fail to train on first try and are good - * on subsequent tries. 2 retries should be plenty. If we - * don't have a successful training then we don't expect to - * ever get one. - */ - LINK_TRAINING_MAX_VERIFY_RETRY = 2, - PEAK_FACTOR_X1000 = 1006, -}; - - -bool dp_verify_link_cap_with_retries( - struct dc_link *link, - struct dc_link_settings *known_limit_link_setting, - int attempts); - -bool dp_validate_mode_timing( - struct dc_link *link, - const struct dc_crtc_timing *timing); - -bool hpd_rx_irq_check_link_loss_status(struct dc_link *link, - union hpd_irq_data *hpd_irq_dpcd_data); -enum dc_status read_hpd_rx_irq_data( - struct dc_link *link, - union hpd_irq_data *irq_data); - -bool is_edp_ilr_optimization_required(struct dc_link *link, struct dc_crtc_timing *crtc_timing); - -void dp_enable_mst_on_sink(struct dc_link *link, bool enable); - -enum dp_panel_mode dp_get_panel_mode(struct dc_link *link); -void dp_set_panel_mode(struct dc_link *link, enum dp_panel_mode panel_mode); - -void dpcd_write_cable_id_to_dprx(struct dc_link *link); - -enum dc_status dp_set_fec_ready(struct dc_link *link, const struct link_resource *link_res, bool ready); -void dp_set_fec_enable(struct dc_link *link, bool enable); -bool dp_set_dsc_enable(struct pipe_ctx *pipe_ctx, bool enable); -bool dp_set_dsc_pps_sdp(struct pipe_ctx *pipe_ctx, bool enable, bool immediate_update); -void dp_set_dsc_on_stream(struct pipe_ctx *pipe_ctx, bool enable); -bool dp_update_dsc_config(struct pipe_ctx *pipe_ctx); -bool dp_set_dsc_on_rx(struct pipe_ctx *pipe_ctx, bool enable); - -/* Initialize output parameter lt_settings. */ -void dp_decide_training_settings( - struct dc_link *link, - const struct dc_link_settings *link_setting, - struct link_training_settings *lt_settings); - -bool dpcd_write_128b_132b_sst_payload_allocation_table( - const struct dc_stream_state *stream, - struct dc_link *link, - struct link_mst_stream_allocation_table *proposed_table, - bool allocate); - -bool dpcd_poll_for_allocation_change_trigger(struct dc_link *link); - -struct fixed31_32 calculate_sst_avg_time_slots_per_mtp( - const struct dc_stream_state *stream, - const struct dc_link *link); -void enable_dp_hpo_output(struct dc_link *link, - const struct link_resource *link_res, - const struct dc_link_settings *link_settings); -void disable_dp_hpo_output(struct dc_link *link, - const struct link_resource *link_res, - enum signal_type signal); -void setup_dp_hpo_stream(struct pipe_ctx *pipe_ctx, bool enable); -void edp_panel_backlight_power_on(struct dc_link *link, bool wait_for_hpd); -void dp_source_sequence_trace(struct dc_link *link, uint8_t dp_test_mode); -void edp_add_delay_for_T9(struct dc_link *link); -bool edp_receiver_ready_T9(struct dc_link *link); -bool edp_receiver_ready_T7(struct dc_link *link); - -void dp_retrain_link_dp_test(struct dc_link *link, - struct dc_link_settings *link_setting, - bool skip_video_pattern); - -#endif /* __DC_LINK_DP_H__ */ diff --git a/drivers/gpu/drm/amd/display/dc/inc/hw/stream_encoder.h b/drivers/gpu/drm/amd/display/dc/inc/hw/stream_encoder.h index 42db4b7b79fdc4ffaaabefc31abed719d2d31a75..bb5ad70d42662fbe4a87456f1fd57822260ed24b 100644 --- a/drivers/gpu/drm/amd/display/dc/inc/hw/stream_encoder.h +++ b/drivers/gpu/drm/amd/display/dc/inc/hw/stream_encoder.h @@ -72,6 +72,12 @@ enum dynamic_metadata_mode { dmdata_dolby_vision }; +struct enc_sdp_line_num { + /* Adaptive Sync SDP */ + bool adaptive_sync_line_num_valid; + uint32_t adaptive_sync_line_num; +}; + struct encoder_info_frame { /* auxiliary video information */ struct dc_info_packet avi; @@ -85,6 +91,9 @@ struct encoder_info_frame { struct dc_info_packet vsc; /* HDR Static MetaData */ struct dc_info_packet hdrsmd; + /* Adaptive Sync SDP*/ + struct dc_info_packet adaptive_sync; + struct enc_sdp_line_num sdp_line_num; }; struct encoder_unblank_param { @@ -154,6 +163,10 @@ struct stream_encoder_funcs { void (*stop_hdmi_info_packets)( struct stream_encoder *enc); + void (*update_dp_info_packets_sdp_line_num)( + struct stream_encoder *enc, + struct encoder_info_frame *info_frame); + void (*update_dp_info_packets)( struct stream_encoder *enc, const struct encoder_info_frame *info_frame); @@ -302,6 +315,10 @@ struct hpo_dp_stream_encoder_funcs { bool compressed_format, bool double_buffer_en); + void (*update_dp_info_packets_sdp_line_num)( + struct hpo_dp_stream_encoder *enc, + struct encoder_info_frame *info_frame); + void (*update_dp_info_packets)( struct hpo_dp_stream_encoder *enc, const struct encoder_info_frame *info_frame); diff --git a/drivers/gpu/drm/amd/display/dc/inc/link.h b/drivers/gpu/drm/amd/display/dc/inc/link.h index 3945522fb798213b68b95b1663349424d8d320e6..d4cebf49e5d440cfe77fce9c4019668dd68ea44c 100644 --- a/drivers/gpu/drm/amd/display/dc/inc/link.h +++ b/drivers/gpu/drm/amd/display/dc/inc/link.h @@ -89,4 +89,21 @@ bool link_decide_link_settings( struct dc_stream_state *stream, struct dc_link_settings *link_setting); +void link_dp_trace_set_edp_power_timestamp(struct dc_link *link, + bool power_up); +uint64_t link_dp_trace_get_edp_poweron_timestamp(struct dc_link *link); +uint64_t link_dp_trace_get_edp_poweroff_timestamp(struct dc_link *link); + +bool link_is_edp_ilr_optimization_required(struct dc_link *link, + struct dc_crtc_timing *crtc_timing); + +bool link_backlight_enable_aux(struct dc_link *link, bool enable); +void link_edp_add_delay_for_T9(struct dc_link *link); +bool link_edp_receiver_ready_T9(struct dc_link *link); +bool link_edp_receiver_ready_T7(struct dc_link *link); +bool link_power_alpm_dpcd_enable(struct dc_link *link, bool enable); +bool link_set_sink_vtotal_in_psr_active(const struct dc_link *link, + uint16_t psr_vtotal_idle, uint16_t psr_vtotal_su); +void link_get_psr_residency(const struct dc_link *link, uint32_t *residency); + #endif /* __DC_LINK_HPD_H__ */ diff --git a/drivers/gpu/drm/amd/display/dc/link/Makefile b/drivers/gpu/drm/amd/display/dc/link/Makefile index 4dee0e6248b1df9058be35116a7235f9a88889e9..0f642cbac3d57d44c81945c363fdfea464774387 100644 --- a/drivers/gpu/drm/amd/display/dc/link/Makefile +++ b/drivers/gpu/drm/amd/display/dc/link/Makefile @@ -23,12 +23,34 @@ # It abstracts the control and status of back end pipe such as DIO, HPO, DPIA, # PHY, HPD, DDC and etc). -LINK = link_hwss_dio.o link_hwss_dpia.o link_hwss_hpo_dp.o link_dp_trace.o \ -link_hpd.o link_ddc.o link_dpcd.o link_dp_dpia.o link_dp_training.o \ -link_dp_training_8b_10b.o link_dp_training_128b_132b.o link_dp_training_dpia.o \ -link_dp_training_auxless.o link_dp_training_fixed_vs_pe_retimer.o link_dp_phy.o \ -link_dp_capability.o +############################################################################### +# accessories +############################################################################### +LINK_ACCESSORIES = link_dp_trace.o link_dp_cts.o -AMD_DAL_LINK = $(addprefix $(AMDDALPATH)/dc/link/,$(LINK)) +AMD_DAL_LINK_ACCESSORIES = $(addprefix $(AMDDALPATH)/dc/link/accessories/, \ +$(LINK_ACCESSORIES)) -AMD_DISPLAY_FILES += $(AMD_DAL_LINK) +AMD_DISPLAY_FILES += $(AMD_DAL_LINK_ACCESSORIES) +############################################################################### +# hwss +############################################################################### +LINK_HWSS = link_hwss_dio.o link_hwss_dpia.o link_hwss_hpo_dp.o + +AMD_DAL_LINK_HWSS = $(addprefix $(AMDDALPATH)/dc/link/hwss/, \ +$(LINK_HWSS)) + +AMD_DISPLAY_FILES += $(AMD_DAL_LINK_HWSS) +############################################################################### +# protocols +############################################################################### +LINK_PROTOCOLS = link_hpd.o link_ddc.o link_dpcd.o link_dp_dpia.o \ +link_dp_training.o link_dp_training_8b_10b.o link_dp_training_128b_132b.o \ +link_dp_training_dpia.o link_dp_training_auxless.o \ +link_dp_training_fixed_vs_pe_retimer.o link_dp_phy.o link_dp_capability.o \ +link_edp_panel_control.o link_dp_irq_handler.o + +AMD_DAL_LINK_PROTOCOLS = $(addprefix $(AMDDALPATH)/dc/link/protocols/, \ +$(LINK_PROTOCOLS)) + +AMD_DISPLAY_FILES += $(AMD_DAL_LINK_PROTOCOLS) \ No newline at end of file diff --git a/drivers/gpu/drm/amd/display/dc/link/accessories/link_dp_cts.c b/drivers/gpu/drm/amd/display/dc/link/accessories/link_dp_cts.c new file mode 100644 index 0000000000000000000000000000000000000000..7fb2c0b4c4bb406e532ff47120b2a567e060a417 --- /dev/null +++ b/drivers/gpu/drm/amd/display/dc/link/accessories/link_dp_cts.c @@ -0,0 +1,1055 @@ +/* + * Copyright 2022 Advanced Micro Devices, Inc. + * + * Permission is hereby granted, free of charge, to any person obtaining a + * copy of this software and associated documentation files (the "Software"), + * to deal in the Software without restriction, including without limitation + * the rights to use, copy, modify, merge, publish, distribute, sublicense, + * and/or sell copies of the Software, and to permit persons to whom the + * Software is furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL + * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR + * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, + * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR + * OTHER DEALINGS IN THE SOFTWARE. + * + * Authors: AMD + * + */ +#include "link_dp_cts.h" +#include "link/protocols/link_dpcd.h" +#include "link/protocols/link_dp_training.h" +#include "link/protocols/link_dp_phy.h" +#include "link/protocols/link_dp_training_fixed_vs_pe_retimer.h" +#include "resource.h" +#include "dm_helpers.h" +#include "dc_dmub_srv.h" +#include "dce/dmub_hw_lock_mgr.h" + +#define DC_LOGGER \ + link->ctx->logger + +static enum dc_link_rate get_link_rate_from_test_link_rate(uint8_t test_rate) +{ + switch (test_rate) { + case DP_TEST_LINK_RATE_RBR: + return LINK_RATE_LOW; + case DP_TEST_LINK_RATE_HBR: + return LINK_RATE_HIGH; + case DP_TEST_LINK_RATE_HBR2: + return LINK_RATE_HIGH2; + case DP_TEST_LINK_RATE_HBR3: + return LINK_RATE_HIGH3; + case DP_TEST_LINK_RATE_UHBR10: + return LINK_RATE_UHBR10; + case DP_TEST_LINK_RATE_UHBR20: + return LINK_RATE_UHBR20; + case DP_TEST_LINK_RATE_UHBR13_5: + return LINK_RATE_UHBR13_5; + default: + return LINK_RATE_UNKNOWN; + } +} + +static bool is_dp_phy_sqaure_pattern(enum dp_test_pattern test_pattern) +{ + return (DP_TEST_PATTERN_SQUARE_BEGIN <= test_pattern && + test_pattern <= DP_TEST_PATTERN_SQUARE_END); +} + +static bool is_dp_phy_pattern(enum dp_test_pattern test_pattern) +{ + if ((DP_TEST_PATTERN_PHY_PATTERN_BEGIN <= test_pattern && + test_pattern <= DP_TEST_PATTERN_PHY_PATTERN_END) || + test_pattern == DP_TEST_PATTERN_VIDEO_MODE) + return true; + else + return false; +} + +void dp_retrain_link_dp_test(struct dc_link *link, + struct dc_link_settings *link_setting, + bool skip_video_pattern) +{ + struct pipe_ctx *pipe; + unsigned int i; + + udelay(100); + + for (i = 0; i < MAX_PIPES; i++) { + pipe = &link->dc->current_state->res_ctx.pipe_ctx[i]; + if (pipe->stream != NULL && + pipe->stream->link == link && + !pipe->stream->dpms_off && + !pipe->top_pipe && !pipe->prev_odm_pipe) { + core_link_disable_stream(pipe); + pipe->link_config.dp_link_settings = *link_setting; + update_dp_encoder_resources_for_test_harness( + link->dc, + pipe->stream->ctx->dc->current_state, + pipe); + } + } + + for (i = 0; i < MAX_PIPES; i++) { + pipe = &link->dc->current_state->res_ctx.pipe_ctx[i]; + if (pipe->stream != NULL && + pipe->stream->link == link && + !pipe->stream->dpms_off && + !pipe->top_pipe && !pipe->prev_odm_pipe) { + core_link_enable_stream( + pipe->stream->ctx->dc->current_state, + pipe); + } + } +} + +static void dp_test_send_link_training(struct dc_link *link) +{ + struct dc_link_settings link_settings = {0}; + uint8_t test_rate = 0; + + core_link_read_dpcd( + link, + DP_TEST_LANE_COUNT, + (unsigned char *)(&link_settings.lane_count), + 1); + core_link_read_dpcd( + link, + DP_TEST_LINK_RATE, + &test_rate, + 1); + link_settings.link_rate = get_link_rate_from_test_link_rate(test_rate); + + /* Set preferred link settings */ + link->verified_link_cap.lane_count = link_settings.lane_count; + link->verified_link_cap.link_rate = link_settings.link_rate; + + dp_retrain_link_dp_test(link, &link_settings, false); +} + +static void dp_test_get_audio_test_data(struct dc_link *link, bool disable_video) +{ + union audio_test_mode dpcd_test_mode = {0}; + struct audio_test_pattern_type dpcd_pattern_type = {0}; + union audio_test_pattern_period dpcd_pattern_period[AUDIO_CHANNELS_COUNT] = {0}; + enum dp_test_pattern test_pattern = DP_TEST_PATTERN_AUDIO_OPERATOR_DEFINED; + + struct pipe_ctx *pipes = link->dc->current_state->res_ctx.pipe_ctx; + struct pipe_ctx *pipe_ctx = &pipes[0]; + unsigned int channel_count; + unsigned int channel = 0; + unsigned int modes = 0; + unsigned int sampling_rate_in_hz = 0; + + // get audio test mode and test pattern parameters + core_link_read_dpcd( + link, + DP_TEST_AUDIO_MODE, + &dpcd_test_mode.raw, + sizeof(dpcd_test_mode)); + + core_link_read_dpcd( + link, + DP_TEST_AUDIO_PATTERN_TYPE, + &dpcd_pattern_type.value, + sizeof(dpcd_pattern_type)); + + channel_count = min(dpcd_test_mode.bits.channel_count + 1, AUDIO_CHANNELS_COUNT); + + // read pattern periods for requested channels when sawTooth pattern is requested + if (dpcd_pattern_type.value == AUDIO_TEST_PATTERN_SAWTOOTH || + dpcd_pattern_type.value == AUDIO_TEST_PATTERN_OPERATOR_DEFINED) { + + test_pattern = (dpcd_pattern_type.value == AUDIO_TEST_PATTERN_SAWTOOTH) ? + DP_TEST_PATTERN_AUDIO_SAWTOOTH : DP_TEST_PATTERN_AUDIO_OPERATOR_DEFINED; + // read period for each channel + for (channel = 0; channel < channel_count; channel++) { + core_link_read_dpcd( + link, + DP_TEST_AUDIO_PERIOD_CH1 + channel, + &dpcd_pattern_period[channel].raw, + sizeof(dpcd_pattern_period[channel])); + } + } + + // translate sampling rate + switch (dpcd_test_mode.bits.sampling_rate) { + case AUDIO_SAMPLING_RATE_32KHZ: + sampling_rate_in_hz = 32000; + break; + case AUDIO_SAMPLING_RATE_44_1KHZ: + sampling_rate_in_hz = 44100; + break; + case AUDIO_SAMPLING_RATE_48KHZ: + sampling_rate_in_hz = 48000; + break; + case AUDIO_SAMPLING_RATE_88_2KHZ: + sampling_rate_in_hz = 88200; + break; + case AUDIO_SAMPLING_RATE_96KHZ: + sampling_rate_in_hz = 96000; + break; + case AUDIO_SAMPLING_RATE_176_4KHZ: + sampling_rate_in_hz = 176400; + break; + case AUDIO_SAMPLING_RATE_192KHZ: + sampling_rate_in_hz = 192000; + break; + default: + sampling_rate_in_hz = 0; + break; + } + + link->audio_test_data.flags.test_requested = 1; + link->audio_test_data.flags.disable_video = disable_video; + link->audio_test_data.sampling_rate = sampling_rate_in_hz; + link->audio_test_data.channel_count = channel_count; + link->audio_test_data.pattern_type = test_pattern; + + if (test_pattern == DP_TEST_PATTERN_AUDIO_SAWTOOTH) { + for (modes = 0; modes < pipe_ctx->stream->audio_info.mode_count; modes++) { + link->audio_test_data.pattern_period[modes] = dpcd_pattern_period[modes].bits.pattern_period; + } + } +} + +/* TODO Raven hbr2 compliance eye output is unstable + * (toggling on and off) with debugger break + * This caueses intermittent PHY automation failure + * Need to look into the root cause */ +static void dp_test_send_phy_test_pattern(struct dc_link *link) +{ + union phy_test_pattern dpcd_test_pattern; + union lane_adjust dpcd_lane_adjustment[2]; + unsigned char dpcd_post_cursor_2_adjustment = 0; + unsigned char test_pattern_buffer[ + (DP_TEST_264BIT_CUSTOM_PATTERN_263_256 - + DP_TEST_264BIT_CUSTOM_PATTERN_7_0)+1] = {0}; + unsigned int test_pattern_size = 0; + enum dp_test_pattern test_pattern; + union lane_adjust dpcd_lane_adjust; + unsigned int lane; + struct link_training_settings link_training_settings; + unsigned char no_preshoot = 0; + unsigned char no_deemphasis = 0; + + dpcd_test_pattern.raw = 0; + memset(dpcd_lane_adjustment, 0, sizeof(dpcd_lane_adjustment)); + memset(&link_training_settings, 0, sizeof(link_training_settings)); + + /* get phy test pattern and pattern parameters from DP receiver */ + core_link_read_dpcd( + link, + DP_PHY_TEST_PATTERN, + &dpcd_test_pattern.raw, + sizeof(dpcd_test_pattern)); + core_link_read_dpcd( + link, + DP_ADJUST_REQUEST_LANE0_1, + &dpcd_lane_adjustment[0].raw, + sizeof(dpcd_lane_adjustment)); + + /* prepare link training settings */ + link_training_settings.link_settings = link->cur_link_settings; + + link_training_settings.lttpr_mode = dc_link_decide_lttpr_mode(link, &link->cur_link_settings); + + if ((link->chip_caps & EXT_DISPLAY_PATH_CAPS__DP_FIXED_VS_EN) && + link_training_settings.lttpr_mode == LTTPR_MODE_TRANSPARENT) + dp_fixed_vs_pe_read_lane_adjust( + link, + link_training_settings.dpcd_lane_settings); + + /*get post cursor 2 parameters + * For DP 1.1a or eariler, this DPCD register's value is 0 + * For DP 1.2 or later: + * Bits 1:0 = POST_CURSOR2_LANE0; Bits 3:2 = POST_CURSOR2_LANE1 + * Bits 5:4 = POST_CURSOR2_LANE2; Bits 7:6 = POST_CURSOR2_LANE3 + */ + core_link_read_dpcd( + link, + DP_ADJUST_REQUEST_POST_CURSOR2, + &dpcd_post_cursor_2_adjustment, + sizeof(dpcd_post_cursor_2_adjustment)); + + /* translate request */ + switch (dpcd_test_pattern.bits.PATTERN) { + case PHY_TEST_PATTERN_D10_2: + test_pattern = DP_TEST_PATTERN_D102; + break; + case PHY_TEST_PATTERN_SYMBOL_ERROR: + test_pattern = DP_TEST_PATTERN_SYMBOL_ERROR; + break; + case PHY_TEST_PATTERN_PRBS7: + test_pattern = DP_TEST_PATTERN_PRBS7; + break; + case PHY_TEST_PATTERN_80BIT_CUSTOM: + test_pattern = DP_TEST_PATTERN_80BIT_CUSTOM; + break; + case PHY_TEST_PATTERN_CP2520_1: + /* CP2520 pattern is unstable, temporarily use TPS4 instead */ + test_pattern = (link->dc->caps.force_dp_tps4_for_cp2520 == 1) ? + DP_TEST_PATTERN_TRAINING_PATTERN4 : + DP_TEST_PATTERN_HBR2_COMPLIANCE_EYE; + break; + case PHY_TEST_PATTERN_CP2520_2: + /* CP2520 pattern is unstable, temporarily use TPS4 instead */ + test_pattern = (link->dc->caps.force_dp_tps4_for_cp2520 == 1) ? + DP_TEST_PATTERN_TRAINING_PATTERN4 : + DP_TEST_PATTERN_HBR2_COMPLIANCE_EYE; + break; + case PHY_TEST_PATTERN_CP2520_3: + test_pattern = DP_TEST_PATTERN_TRAINING_PATTERN4; + break; + case PHY_TEST_PATTERN_128b_132b_TPS1: + test_pattern = DP_TEST_PATTERN_128b_132b_TPS1; + break; + case PHY_TEST_PATTERN_128b_132b_TPS2: + test_pattern = DP_TEST_PATTERN_128b_132b_TPS2; + break; + case PHY_TEST_PATTERN_PRBS9: + test_pattern = DP_TEST_PATTERN_PRBS9; + break; + case PHY_TEST_PATTERN_PRBS11: + test_pattern = DP_TEST_PATTERN_PRBS11; + break; + case PHY_TEST_PATTERN_PRBS15: + test_pattern = DP_TEST_PATTERN_PRBS15; + break; + case PHY_TEST_PATTERN_PRBS23: + test_pattern = DP_TEST_PATTERN_PRBS23; + break; + case PHY_TEST_PATTERN_PRBS31: + test_pattern = DP_TEST_PATTERN_PRBS31; + break; + case PHY_TEST_PATTERN_264BIT_CUSTOM: + test_pattern = DP_TEST_PATTERN_264BIT_CUSTOM; + break; + case PHY_TEST_PATTERN_SQUARE: + test_pattern = DP_TEST_PATTERN_SQUARE; + break; + case PHY_TEST_PATTERN_SQUARE_PRESHOOT_DISABLED: + test_pattern = DP_TEST_PATTERN_SQUARE_PRESHOOT_DISABLED; + no_preshoot = 1; + break; + case PHY_TEST_PATTERN_SQUARE_DEEMPHASIS_DISABLED: + test_pattern = DP_TEST_PATTERN_SQUARE_DEEMPHASIS_DISABLED; + no_deemphasis = 1; + break; + case PHY_TEST_PATTERN_SQUARE_PRESHOOT_DEEMPHASIS_DISABLED: + test_pattern = DP_TEST_PATTERN_SQUARE_PRESHOOT_DEEMPHASIS_DISABLED; + no_preshoot = 1; + no_deemphasis = 1; + break; + default: + test_pattern = DP_TEST_PATTERN_VIDEO_MODE; + break; + } + + if (test_pattern == DP_TEST_PATTERN_80BIT_CUSTOM) { + test_pattern_size = (DP_TEST_80BIT_CUSTOM_PATTERN_79_72 - + DP_TEST_80BIT_CUSTOM_PATTERN_7_0) + 1; + core_link_read_dpcd( + link, + DP_TEST_80BIT_CUSTOM_PATTERN_7_0, + test_pattern_buffer, + test_pattern_size); + } + + if (is_dp_phy_sqaure_pattern(test_pattern)) { + test_pattern_size = 1; // Square pattern data is 1 byte (DP spec) + core_link_read_dpcd( + link, + DP_PHY_SQUARE_PATTERN, + test_pattern_buffer, + test_pattern_size); + } + + if (test_pattern == DP_TEST_PATTERN_264BIT_CUSTOM) { + test_pattern_size = (DP_TEST_264BIT_CUSTOM_PATTERN_263_256- + DP_TEST_264BIT_CUSTOM_PATTERN_7_0) + 1; + core_link_read_dpcd( + link, + DP_TEST_264BIT_CUSTOM_PATTERN_7_0, + test_pattern_buffer, + test_pattern_size); + } + + for (lane = 0; lane < + (unsigned int)(link->cur_link_settings.lane_count); + lane++) { + dpcd_lane_adjust.raw = + dp_get_nibble_at_index(&dpcd_lane_adjustment[0].raw, lane); + if (link_dp_get_encoding_format(&link->cur_link_settings) == + DP_8b_10b_ENCODING) { + link_training_settings.hw_lane_settings[lane].VOLTAGE_SWING = + (enum dc_voltage_swing) + (dpcd_lane_adjust.bits.VOLTAGE_SWING_LANE); + link_training_settings.hw_lane_settings[lane].PRE_EMPHASIS = + (enum dc_pre_emphasis) + (dpcd_lane_adjust.bits.PRE_EMPHASIS_LANE); + link_training_settings.hw_lane_settings[lane].POST_CURSOR2 = + (enum dc_post_cursor2) + ((dpcd_post_cursor_2_adjustment >> (lane * 2)) & 0x03); + } else if (link_dp_get_encoding_format(&link->cur_link_settings) == + DP_128b_132b_ENCODING) { + link_training_settings.hw_lane_settings[lane].FFE_PRESET.settings.level = + dpcd_lane_adjust.tx_ffe.PRESET_VALUE; + link_training_settings.hw_lane_settings[lane].FFE_PRESET.settings.no_preshoot = no_preshoot; + link_training_settings.hw_lane_settings[lane].FFE_PRESET.settings.no_deemphasis = no_deemphasis; + } + } + + dp_hw_to_dpcd_lane_settings(&link_training_settings, + link_training_settings.hw_lane_settings, + link_training_settings.dpcd_lane_settings); + /*Usage: Measure DP physical lane signal + * by DP SI test equipment automatically. + * PHY test pattern request is generated by equipment via HPD interrupt. + * HPD needs to be active all the time. HPD should be active + * all the time. Do not touch it. + * forward request to DS + */ + dc_link_dp_set_test_pattern( + link, + test_pattern, + DP_TEST_PATTERN_COLOR_SPACE_UNDEFINED, + &link_training_settings, + test_pattern_buffer, + test_pattern_size); +} + +static void set_crtc_test_pattern(struct dc_link *link, + struct pipe_ctx *pipe_ctx, + enum dp_test_pattern test_pattern, + enum dp_test_pattern_color_space test_pattern_color_space) +{ + enum controller_dp_test_pattern controller_test_pattern; + enum dc_color_depth color_depth = pipe_ctx-> + stream->timing.display_color_depth; + struct bit_depth_reduction_params params; + struct output_pixel_processor *opp = pipe_ctx->stream_res.opp; + int width = pipe_ctx->stream->timing.h_addressable + + pipe_ctx->stream->timing.h_border_left + + pipe_ctx->stream->timing.h_border_right; + int height = pipe_ctx->stream->timing.v_addressable + + pipe_ctx->stream->timing.v_border_bottom + + pipe_ctx->stream->timing.v_border_top; + + memset(¶ms, 0, sizeof(params)); + + switch (test_pattern) { + case DP_TEST_PATTERN_COLOR_SQUARES: + controller_test_pattern = + CONTROLLER_DP_TEST_PATTERN_COLORSQUARES; + break; + case DP_TEST_PATTERN_COLOR_SQUARES_CEA: + controller_test_pattern = + CONTROLLER_DP_TEST_PATTERN_COLORSQUARES_CEA; + break; + case DP_TEST_PATTERN_VERTICAL_BARS: + controller_test_pattern = + CONTROLLER_DP_TEST_PATTERN_VERTICALBARS; + break; + case DP_TEST_PATTERN_HORIZONTAL_BARS: + controller_test_pattern = + CONTROLLER_DP_TEST_PATTERN_HORIZONTALBARS; + break; + case DP_TEST_PATTERN_COLOR_RAMP: + controller_test_pattern = + CONTROLLER_DP_TEST_PATTERN_COLORRAMP; + break; + default: + controller_test_pattern = + CONTROLLER_DP_TEST_PATTERN_VIDEOMODE; + break; + } + + switch (test_pattern) { + case DP_TEST_PATTERN_COLOR_SQUARES: + case DP_TEST_PATTERN_COLOR_SQUARES_CEA: + case DP_TEST_PATTERN_VERTICAL_BARS: + case DP_TEST_PATTERN_HORIZONTAL_BARS: + case DP_TEST_PATTERN_COLOR_RAMP: + { + /* disable bit depth reduction */ + pipe_ctx->stream->bit_depth_params = params; + opp->funcs->opp_program_bit_depth_reduction(opp, ¶ms); + if (pipe_ctx->stream_res.tg->funcs->set_test_pattern) + pipe_ctx->stream_res.tg->funcs->set_test_pattern(pipe_ctx->stream_res.tg, + controller_test_pattern, color_depth); + else if (link->dc->hwss.set_disp_pattern_generator) { + struct pipe_ctx *odm_pipe; + enum controller_dp_color_space controller_color_space; + int opp_cnt = 1; + int offset = 0; + int dpg_width = width; + + switch (test_pattern_color_space) { + case DP_TEST_PATTERN_COLOR_SPACE_RGB: + controller_color_space = CONTROLLER_DP_COLOR_SPACE_RGB; + break; + case DP_TEST_PATTERN_COLOR_SPACE_YCBCR601: + controller_color_space = CONTROLLER_DP_COLOR_SPACE_YCBCR601; + break; + case DP_TEST_PATTERN_COLOR_SPACE_YCBCR709: + controller_color_space = CONTROLLER_DP_COLOR_SPACE_YCBCR709; + break; + case DP_TEST_PATTERN_COLOR_SPACE_UNDEFINED: + default: + controller_color_space = CONTROLLER_DP_COLOR_SPACE_UDEFINED; + DC_LOG_ERROR("%s: Color space must be defined for test pattern", __func__); + ASSERT(0); + break; + } + + for (odm_pipe = pipe_ctx->next_odm_pipe; odm_pipe; odm_pipe = odm_pipe->next_odm_pipe) + opp_cnt++; + dpg_width = width / opp_cnt; + offset = dpg_width; + + link->dc->hwss.set_disp_pattern_generator(link->dc, + pipe_ctx, + controller_test_pattern, + controller_color_space, + color_depth, + NULL, + dpg_width, + height, + 0); + + for (odm_pipe = pipe_ctx->next_odm_pipe; odm_pipe; odm_pipe = odm_pipe->next_odm_pipe) { + struct output_pixel_processor *odm_opp = odm_pipe->stream_res.opp; + + odm_opp->funcs->opp_program_bit_depth_reduction(odm_opp, ¶ms); + link->dc->hwss.set_disp_pattern_generator(link->dc, + odm_pipe, + controller_test_pattern, + controller_color_space, + color_depth, + NULL, + dpg_width, + height, + offset); + offset += offset; + } + } + } + break; + case DP_TEST_PATTERN_VIDEO_MODE: + { + /* restore bitdepth reduction */ + resource_build_bit_depth_reduction_params(pipe_ctx->stream, ¶ms); + pipe_ctx->stream->bit_depth_params = params; + opp->funcs->opp_program_bit_depth_reduction(opp, ¶ms); + if (pipe_ctx->stream_res.tg->funcs->set_test_pattern) + pipe_ctx->stream_res.tg->funcs->set_test_pattern(pipe_ctx->stream_res.tg, + CONTROLLER_DP_TEST_PATTERN_VIDEOMODE, + color_depth); + else if (link->dc->hwss.set_disp_pattern_generator) { + struct pipe_ctx *odm_pipe; + int opp_cnt = 1; + int dpg_width; + + for (odm_pipe = pipe_ctx->next_odm_pipe; odm_pipe; odm_pipe = odm_pipe->next_odm_pipe) + opp_cnt++; + + dpg_width = width / opp_cnt; + for (odm_pipe = pipe_ctx->next_odm_pipe; odm_pipe; odm_pipe = odm_pipe->next_odm_pipe) { + struct output_pixel_processor *odm_opp = odm_pipe->stream_res.opp; + + odm_opp->funcs->opp_program_bit_depth_reduction(odm_opp, ¶ms); + link->dc->hwss.set_disp_pattern_generator(link->dc, + odm_pipe, + CONTROLLER_DP_TEST_PATTERN_VIDEOMODE, + CONTROLLER_DP_COLOR_SPACE_UDEFINED, + color_depth, + NULL, + dpg_width, + height, + 0); + } + link->dc->hwss.set_disp_pattern_generator(link->dc, + pipe_ctx, + CONTROLLER_DP_TEST_PATTERN_VIDEOMODE, + CONTROLLER_DP_COLOR_SPACE_UDEFINED, + color_depth, + NULL, + dpg_width, + height, + 0); + } + } + break; + + default: + break; + } +} + +void dc_link_dp_handle_automated_test(struct dc_link *link) +{ + union test_request test_request; + union test_response test_response; + + memset(&test_request, 0, sizeof(test_request)); + memset(&test_response, 0, sizeof(test_response)); + + core_link_read_dpcd( + link, + DP_TEST_REQUEST, + &test_request.raw, + sizeof(union test_request)); + if (test_request.bits.LINK_TRAINING) { + /* ACK first to let DP RX test box monitor LT sequence */ + test_response.bits.ACK = 1; + core_link_write_dpcd( + link, + DP_TEST_RESPONSE, + &test_response.raw, + sizeof(test_response)); + dp_test_send_link_training(link); + /* no acknowledge request is needed again */ + test_response.bits.ACK = 0; + } + if (test_request.bits.LINK_TEST_PATTRN) { + union test_misc dpcd_test_params; + union link_test_pattern dpcd_test_pattern; + + memset(&dpcd_test_pattern, 0, sizeof(dpcd_test_pattern)); + memset(&dpcd_test_params, 0, sizeof(dpcd_test_params)); + + /* get link test pattern and pattern parameters */ + core_link_read_dpcd( + link, + DP_TEST_PATTERN, + &dpcd_test_pattern.raw, + sizeof(dpcd_test_pattern)); + core_link_read_dpcd( + link, + DP_TEST_MISC0, + &dpcd_test_params.raw, + sizeof(dpcd_test_params)); + test_response.bits.ACK = dm_helpers_dp_handle_test_pattern_request(link->ctx, link, + dpcd_test_pattern, dpcd_test_params) ? 1 : 0; + } + + if (test_request.bits.AUDIO_TEST_PATTERN) { + dp_test_get_audio_test_data(link, test_request.bits.TEST_AUDIO_DISABLED_VIDEO); + test_response.bits.ACK = 1; + } + + if (test_request.bits.PHY_TEST_PATTERN) { + dp_test_send_phy_test_pattern(link); + test_response.bits.ACK = 1; + } + + /* send request acknowledgment */ + if (test_response.bits.ACK) + core_link_write_dpcd( + link, + DP_TEST_RESPONSE, + &test_response.raw, + sizeof(test_response)); +} + +bool dc_link_dp_set_test_pattern( + struct dc_link *link, + enum dp_test_pattern test_pattern, + enum dp_test_pattern_color_space test_pattern_color_space, + const struct link_training_settings *p_link_settings, + const unsigned char *p_custom_pattern, + unsigned int cust_pattern_size) +{ + struct pipe_ctx *pipes = link->dc->current_state->res_ctx.pipe_ctx; + struct pipe_ctx *pipe_ctx = NULL; + unsigned int lane; + unsigned int i; + unsigned char link_qual_pattern[LANE_COUNT_DP_MAX] = {0}; + union dpcd_training_pattern training_pattern; + enum dpcd_phy_test_patterns pattern; + + memset(&training_pattern, 0, sizeof(training_pattern)); + + for (i = 0; i < MAX_PIPES; i++) { + if (pipes[i].stream == NULL) + continue; + + if (pipes[i].stream->link == link && !pipes[i].top_pipe && !pipes[i].prev_odm_pipe) { + pipe_ctx = &pipes[i]; + break; + } + } + + if (pipe_ctx == NULL) + return false; + + /* Reset CRTC Test Pattern if it is currently running and request is VideoMode */ + if (link->test_pattern_enabled && test_pattern == + DP_TEST_PATTERN_VIDEO_MODE) { + /* Set CRTC Test Pattern */ + set_crtc_test_pattern(link, pipe_ctx, test_pattern, test_pattern_color_space); + dp_set_hw_test_pattern(link, &pipe_ctx->link_res, test_pattern, + (uint8_t *)p_custom_pattern, + (uint32_t)cust_pattern_size); + + /* Unblank Stream */ + link->dc->hwss.unblank_stream( + pipe_ctx, + &link->verified_link_cap); + /* TODO:m_pHwss->MuteAudioEndpoint + * (pPathMode->pDisplayPath, false); + */ + + /* Reset Test Pattern state */ + link->test_pattern_enabled = false; + + return true; + } + + /* Check for PHY Test Patterns */ + if (is_dp_phy_pattern(test_pattern)) { + /* Set DPCD Lane Settings before running test pattern */ + if (p_link_settings != NULL) { + if ((link->chip_caps & EXT_DISPLAY_PATH_CAPS__DP_FIXED_VS_EN) && + p_link_settings->lttpr_mode == LTTPR_MODE_TRANSPARENT) { + dp_fixed_vs_pe_set_retimer_lane_settings( + link, + p_link_settings->dpcd_lane_settings, + p_link_settings->link_settings.lane_count); + } else { + dp_set_hw_lane_settings(link, &pipe_ctx->link_res, p_link_settings, DPRX); + } + dpcd_set_lane_settings(link, p_link_settings, DPRX); + } + + /* Blank stream if running test pattern */ + if (test_pattern != DP_TEST_PATTERN_VIDEO_MODE) { + /*TODO: + * m_pHwss-> + * MuteAudioEndpoint(pPathMode->pDisplayPath, true); + */ + /* Blank stream */ + link->dc->hwss.blank_stream(pipe_ctx); + } + + dp_set_hw_test_pattern(link, &pipe_ctx->link_res, test_pattern, + (uint8_t *)p_custom_pattern, + (uint32_t)cust_pattern_size); + + if (test_pattern != DP_TEST_PATTERN_VIDEO_MODE) { + /* Set Test Pattern state */ + link->test_pattern_enabled = true; + if (p_link_settings != NULL) + dpcd_set_link_settings(link, + p_link_settings); + } + + switch (test_pattern) { + case DP_TEST_PATTERN_VIDEO_MODE: + pattern = PHY_TEST_PATTERN_NONE; + break; + case DP_TEST_PATTERN_D102: + pattern = PHY_TEST_PATTERN_D10_2; + break; + case DP_TEST_PATTERN_SYMBOL_ERROR: + pattern = PHY_TEST_PATTERN_SYMBOL_ERROR; + break; + case DP_TEST_PATTERN_PRBS7: + pattern = PHY_TEST_PATTERN_PRBS7; + break; + case DP_TEST_PATTERN_80BIT_CUSTOM: + pattern = PHY_TEST_PATTERN_80BIT_CUSTOM; + break; + case DP_TEST_PATTERN_CP2520_1: + pattern = PHY_TEST_PATTERN_CP2520_1; + break; + case DP_TEST_PATTERN_CP2520_2: + pattern = PHY_TEST_PATTERN_CP2520_2; + break; + case DP_TEST_PATTERN_CP2520_3: + pattern = PHY_TEST_PATTERN_CP2520_3; + break; + case DP_TEST_PATTERN_128b_132b_TPS1: + pattern = PHY_TEST_PATTERN_128b_132b_TPS1; + break; + case DP_TEST_PATTERN_128b_132b_TPS2: + pattern = PHY_TEST_PATTERN_128b_132b_TPS2; + break; + case DP_TEST_PATTERN_PRBS9: + pattern = PHY_TEST_PATTERN_PRBS9; + break; + case DP_TEST_PATTERN_PRBS11: + pattern = PHY_TEST_PATTERN_PRBS11; + break; + case DP_TEST_PATTERN_PRBS15: + pattern = PHY_TEST_PATTERN_PRBS15; + break; + case DP_TEST_PATTERN_PRBS23: + pattern = PHY_TEST_PATTERN_PRBS23; + break; + case DP_TEST_PATTERN_PRBS31: + pattern = PHY_TEST_PATTERN_PRBS31; + break; + case DP_TEST_PATTERN_264BIT_CUSTOM: + pattern = PHY_TEST_PATTERN_264BIT_CUSTOM; + break; + case DP_TEST_PATTERN_SQUARE: + pattern = PHY_TEST_PATTERN_SQUARE; + break; + case DP_TEST_PATTERN_SQUARE_PRESHOOT_DISABLED: + pattern = PHY_TEST_PATTERN_SQUARE_PRESHOOT_DISABLED; + break; + case DP_TEST_PATTERN_SQUARE_DEEMPHASIS_DISABLED: + pattern = PHY_TEST_PATTERN_SQUARE_DEEMPHASIS_DISABLED; + break; + case DP_TEST_PATTERN_SQUARE_PRESHOOT_DEEMPHASIS_DISABLED: + pattern = PHY_TEST_PATTERN_SQUARE_PRESHOOT_DEEMPHASIS_DISABLED; + break; + default: + return false; + } + + if (test_pattern == DP_TEST_PATTERN_VIDEO_MODE + /*TODO:&& !pPathMode->pDisplayPath->IsTargetPoweredOn()*/) + return false; + + if (link->dpcd_caps.dpcd_rev.raw >= DPCD_REV_12) { + if (is_dp_phy_sqaure_pattern(test_pattern)) + core_link_write_dpcd(link, + DP_LINK_SQUARE_PATTERN, + p_custom_pattern, + 1); + + /* tell receiver that we are sending qualification + * pattern DP 1.2 or later - DP receiver's link quality + * pattern is set using DPCD LINK_QUAL_LANEx_SET + * register (0x10B~0x10E)\ + */ + for (lane = 0; lane < LANE_COUNT_DP_MAX; lane++) + link_qual_pattern[lane] = + (unsigned char)(pattern); + + core_link_write_dpcd(link, + DP_LINK_QUAL_LANE0_SET, + link_qual_pattern, + sizeof(link_qual_pattern)); + } else if (link->dpcd_caps.dpcd_rev.raw >= DPCD_REV_10 || + link->dpcd_caps.dpcd_rev.raw == 0) { + /* tell receiver that we are sending qualification + * pattern DP 1.1a or earlier - DP receiver's link + * quality pattern is set using + * DPCD TRAINING_PATTERN_SET -> LINK_QUAL_PATTERN_SET + * register (0x102). We will use v_1.3 when we are + * setting test pattern for DP 1.1. + */ + core_link_read_dpcd(link, DP_TRAINING_PATTERN_SET, + &training_pattern.raw, + sizeof(training_pattern)); + training_pattern.v1_3.LINK_QUAL_PATTERN_SET = pattern; + core_link_write_dpcd(link, DP_TRAINING_PATTERN_SET, + &training_pattern.raw, + sizeof(training_pattern)); + } + } else { + enum dc_color_space color_space = COLOR_SPACE_UNKNOWN; + + switch (test_pattern_color_space) { + case DP_TEST_PATTERN_COLOR_SPACE_RGB: + color_space = COLOR_SPACE_SRGB; + if (test_pattern == DP_TEST_PATTERN_COLOR_SQUARES_CEA) + color_space = COLOR_SPACE_SRGB_LIMITED; + break; + + case DP_TEST_PATTERN_COLOR_SPACE_YCBCR601: + color_space = COLOR_SPACE_YCBCR601; + if (test_pattern == DP_TEST_PATTERN_COLOR_SQUARES_CEA) + color_space = COLOR_SPACE_YCBCR601_LIMITED; + break; + case DP_TEST_PATTERN_COLOR_SPACE_YCBCR709: + color_space = COLOR_SPACE_YCBCR709; + if (test_pattern == DP_TEST_PATTERN_COLOR_SQUARES_CEA) + color_space = COLOR_SPACE_YCBCR709_LIMITED; + break; + default: + break; + } + + if (pipe_ctx->stream_res.tg->funcs->lock_doublebuffer_enable) { + if (pipe_ctx->stream && should_use_dmub_lock(pipe_ctx->stream->link)) { + union dmub_hw_lock_flags hw_locks = { 0 }; + struct dmub_hw_lock_inst_flags inst_flags = { 0 }; + + hw_locks.bits.lock_dig = 1; + inst_flags.dig_inst = pipe_ctx->stream_res.tg->inst; + + dmub_hw_lock_mgr_cmd(link->ctx->dmub_srv, + true, + &hw_locks, + &inst_flags); + } else + pipe_ctx->stream_res.tg->funcs->lock_doublebuffer_enable( + pipe_ctx->stream_res.tg); + } + + pipe_ctx->stream_res.tg->funcs->lock(pipe_ctx->stream_res.tg); + /* update MSA to requested color space */ + pipe_ctx->stream_res.stream_enc->funcs->dp_set_stream_attribute(pipe_ctx->stream_res.stream_enc, + &pipe_ctx->stream->timing, + color_space, + pipe_ctx->stream->use_vsc_sdp_for_colorimetry, + link->dpcd_caps.dprx_feature.bits.SST_SPLIT_SDP_CAP); + + if (pipe_ctx->stream->use_vsc_sdp_for_colorimetry) { + if (test_pattern == DP_TEST_PATTERN_COLOR_SQUARES_CEA) + pipe_ctx->stream->vsc_infopacket.sb[17] |= (1 << 7); // sb17 bit 7 Dynamic Range: 0 = VESA range, 1 = CTA range + else + pipe_ctx->stream->vsc_infopacket.sb[17] &= ~(1 << 7); + resource_build_info_frame(pipe_ctx); + link->dc->hwss.update_info_frame(pipe_ctx); + } + + /* CRTC Patterns */ + set_crtc_test_pattern(link, pipe_ctx, test_pattern, test_pattern_color_space); + pipe_ctx->stream_res.tg->funcs->unlock(pipe_ctx->stream_res.tg); + pipe_ctx->stream_res.tg->funcs->wait_for_state(pipe_ctx->stream_res.tg, + CRTC_STATE_VACTIVE); + pipe_ctx->stream_res.tg->funcs->wait_for_state(pipe_ctx->stream_res.tg, + CRTC_STATE_VBLANK); + pipe_ctx->stream_res.tg->funcs->wait_for_state(pipe_ctx->stream_res.tg, + CRTC_STATE_VACTIVE); + + if (pipe_ctx->stream_res.tg->funcs->lock_doublebuffer_disable) { + if (pipe_ctx->stream && should_use_dmub_lock(pipe_ctx->stream->link)) { + union dmub_hw_lock_flags hw_locks = { 0 }; + struct dmub_hw_lock_inst_flags inst_flags = { 0 }; + + hw_locks.bits.lock_dig = 1; + inst_flags.dig_inst = pipe_ctx->stream_res.tg->inst; + + dmub_hw_lock_mgr_cmd(link->ctx->dmub_srv, + false, + &hw_locks, + &inst_flags); + } else + pipe_ctx->stream_res.tg->funcs->lock_doublebuffer_disable( + pipe_ctx->stream_res.tg); + } + + /* Set Test Pattern state */ + link->test_pattern_enabled = true; + } + + return true; +} + +void dc_link_set_drive_settings(struct dc *dc, + struct link_training_settings *lt_settings, + const struct dc_link *link) +{ + + int i; + struct link_resource link_res; + + for (i = 0; i < dc->link_count; i++) + if (dc->links[i] == link) + break; + + if (i >= dc->link_count) + ASSERT_CRITICAL(false); + + dc_link_get_cur_link_res(link, &link_res); + dp_set_drive_settings(dc->links[i], &link_res, lt_settings); +} + +void dc_link_set_preferred_link_settings(struct dc *dc, + struct dc_link_settings *link_setting, + struct dc_link *link) +{ + int i; + struct pipe_ctx *pipe; + struct dc_stream_state *link_stream; + struct dc_link_settings store_settings = *link_setting; + + link->preferred_link_setting = store_settings; + + /* Retrain with preferred link settings only relevant for + * DP signal type + * Check for non-DP signal or if passive dongle present + */ + if (!dc_is_dp_signal(link->connector_signal) || + link->dongle_max_pix_clk > 0) + return; + + for (i = 0; i < MAX_PIPES; i++) { + pipe = &dc->current_state->res_ctx.pipe_ctx[i]; + if (pipe->stream && pipe->stream->link) { + if (pipe->stream->link == link) { + link_stream = pipe->stream; + break; + } + } + } + + /* Stream not found */ + if (i == MAX_PIPES) + return; + + /* Cannot retrain link if backend is off */ + if (link_stream->dpms_off) + return; + + if (link_decide_link_settings(link_stream, &store_settings)) + dp_retrain_link_dp_test(link, &store_settings, false); +} + +void dc_link_set_preferred_training_settings(struct dc *dc, + struct dc_link_settings *link_setting, + struct dc_link_training_overrides *lt_overrides, + struct dc_link *link, + bool skip_immediate_retrain) +{ + if (lt_overrides != NULL) + link->preferred_training_settings = *lt_overrides; + else + memset(&link->preferred_training_settings, 0, sizeof(link->preferred_training_settings)); + + if (link_setting != NULL) { + link->preferred_link_setting = *link_setting; + } else { + link->preferred_link_setting.lane_count = LANE_COUNT_UNKNOWN; + link->preferred_link_setting.link_rate = LINK_RATE_UNKNOWN; + } + + if (link->connector_signal == SIGNAL_TYPE_DISPLAY_PORT && + link->type == dc_connection_mst_branch) + dm_helpers_dp_mst_update_branch_bandwidth(dc->ctx, link); + + /* Retrain now, or wait until next stream update to apply */ + if (skip_immediate_retrain == false) + dc_link_set_preferred_link_settings(dc, &link->preferred_link_setting, link); +} + +void dc_link_set_test_pattern(struct dc_link *link, + enum dp_test_pattern test_pattern, + enum dp_test_pattern_color_space test_pattern_color_space, + const struct link_training_settings *p_link_settings, + const unsigned char *p_custom_pattern, + unsigned int cust_pattern_size) +{ + if (link != NULL) + dc_link_dp_set_test_pattern( + link, + test_pattern, + test_pattern_color_space, + p_link_settings, + p_custom_pattern, + cust_pattern_size); +} diff --git a/drivers/gpu/drm/amd/display/dc/link/accessories/link_dp_cts.h b/drivers/gpu/drm/amd/display/dc/link/accessories/link_dp_cts.h new file mode 100644 index 0000000000000000000000000000000000000000..7f17838b653b77bf3666c493739939a0783614b2 --- /dev/null +++ b/drivers/gpu/drm/amd/display/dc/link/accessories/link_dp_cts.h @@ -0,0 +1,33 @@ +/* + * Copyright 2023 Advanced Micro Devices, Inc. + * + * Permission is hereby granted, free of charge, to any person obtaining a + * copy of this software and associated documentation files (the "Software"), + * to deal in the Software without restriction, including without limitation + * the rights to use, copy, modify, merge, publish, distribute, sublicense, + * and/or sell copies of the Software, and to permit persons to whom the + * Software is furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL + * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR + * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, + * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR + * OTHER DEALINGS IN THE SOFTWARE. + * + * Authors: AMD + * + */ +#ifndef __LINK_DP_CTS_H__ +#define __LINK_DP_CTS_H__ +#include "link.h" + +void dp_retrain_link_dp_test(struct dc_link *link, + struct dc_link_settings *link_setting, + bool skip_video_pattern); + +#endif /* __LINK_DP_CTS_H__ */ diff --git a/drivers/gpu/drm/amd/display/dc/link/link_dp_trace.c b/drivers/gpu/drm/amd/display/dc/link/accessories/link_dp_trace.c similarity index 95% rename from drivers/gpu/drm/amd/display/dc/link/link_dp_trace.c rename to drivers/gpu/drm/amd/display/dc/link/accessories/link_dp_trace.c index 2c1a3bfcdb5067630b4a117cf92c8acde05a6de3..04838a31e5139212917b663f787494c8ddb9c669 100644 --- a/drivers/gpu/drm/amd/display/dc/link/link_dp_trace.c +++ b/drivers/gpu/drm/amd/display/dc/link/accessories/link_dp_trace.c @@ -145,7 +145,7 @@ unsigned int dc_dp_trace_get_link_loss_count(struct dc_link *link) return link->dp_trace.link_loss_count; } -void dp_trace_set_edp_power_timestamp(struct dc_link *link, +void link_dp_trace_set_edp_power_timestamp(struct dc_link *link, bool power_up) { if (!power_up) @@ -155,12 +155,12 @@ void dp_trace_set_edp_power_timestamp(struct dc_link *link, link->dp_trace.edp_trace_power_timestamps.poweron = dm_get_timestamp(link->dc->ctx); } -uint64_t dp_trace_get_edp_poweron_timestamp(struct dc_link *link) +uint64_t link_dp_trace_get_edp_poweron_timestamp(struct dc_link *link) { return link->dp_trace.edp_trace_power_timestamps.poweron; } -uint64_t dp_trace_get_edp_poweroff_timestamp(struct dc_link *link) +uint64_t link_dp_trace_get_edp_poweroff_timestamp(struct dc_link *link) { return link->dp_trace.edp_trace_power_timestamps.poweroff; -} \ No newline at end of file +} diff --git a/drivers/gpu/drm/amd/display/dc/link/link_dp_trace.h b/drivers/gpu/drm/amd/display/dc/link/accessories/link_dp_trace.h similarity index 91% rename from drivers/gpu/drm/amd/display/dc/link/link_dp_trace.h rename to drivers/gpu/drm/amd/display/dc/link/accessories/link_dp_trace.h index 26700e3cd65e9c5136bfcd095e1ee2d1371e2e20..702f97c6ead02f026c13e3902988147d1b451e92 100644 --- a/drivers/gpu/drm/amd/display/dc/link/link_dp_trace.h +++ b/drivers/gpu/drm/amd/display/dc/link/accessories/link_dp_trace.h @@ -54,9 +54,4 @@ struct dp_trace_lt_counts *dc_dp_trace_get_lt_counts(struct dc_link *link, bool in_detection); unsigned int dc_dp_trace_get_link_loss_count(struct dc_link *link); -void dp_trace_set_edp_power_timestamp(struct dc_link *link, - bool power_up); -uint64_t dp_trace_get_edp_poweron_timestamp(struct dc_link *link); -uint64_t dp_trace_get_edp_poweroff_timestamp(struct dc_link *link); - #endif /* __LINK_DP_TRACE_H__ */ diff --git a/drivers/gpu/drm/amd/display/dc/link/link_hwss_dio.c b/drivers/gpu/drm/amd/display/dc/link/hwss/link_hwss_dio.c similarity index 99% rename from drivers/gpu/drm/amd/display/dc/link/link_hwss_dio.c rename to drivers/gpu/drm/amd/display/dc/link/hwss/link_hwss_dio.c index 33148b753c03b485f887213da6583f088d097b7e..841a23da5dd32ef5aef07f0697d217dccfc661c8 100644 --- a/drivers/gpu/drm/amd/display/dc/link/link_hwss_dio.c +++ b/drivers/gpu/drm/amd/display/dc/link/hwss/link_hwss_dio.c @@ -24,7 +24,6 @@ */ #include "link_hwss_dio.h" #include "core_types.h" -#include "dc_link_dp.h" #include "link_enc_cfg.h" void set_dio_throttled_vcp_size(struct pipe_ctx *pipe_ctx, diff --git a/drivers/gpu/drm/amd/display/dc/link/link_hwss_dio.h b/drivers/gpu/drm/amd/display/dc/link/hwss/link_hwss_dio.h similarity index 99% rename from drivers/gpu/drm/amd/display/dc/link/link_hwss_dio.h rename to drivers/gpu/drm/amd/display/dc/link/hwss/link_hwss_dio.h index 9a108c3d7831563a83d72fb9af96441f3aa60bd5..8b8a099feeb09d81b713b783a4c26c1475e1034e 100644 --- a/drivers/gpu/drm/amd/display/dc/link/link_hwss_dio.h +++ b/drivers/gpu/drm/amd/display/dc/link/hwss/link_hwss_dio.h @@ -26,6 +26,7 @@ #define __LINK_HWSS_DIO_H__ #include "link_hwss.h" +#include "link.h" const struct link_hwss *get_dio_link_hwss(void); bool can_use_dio_link_hwss(const struct dc_link *link, diff --git a/drivers/gpu/drm/amd/display/dc/link/link_hwss_dpia.c b/drivers/gpu/drm/amd/display/dc/link/hwss/link_hwss_dpia.c similarity index 100% rename from drivers/gpu/drm/amd/display/dc/link/link_hwss_dpia.c rename to drivers/gpu/drm/amd/display/dc/link/hwss/link_hwss_dpia.c diff --git a/drivers/gpu/drm/amd/display/dc/link/link_hwss_dpia.h b/drivers/gpu/drm/amd/display/dc/link/hwss/link_hwss_dpia.h similarity index 100% rename from drivers/gpu/drm/amd/display/dc/link/link_hwss_dpia.h rename to drivers/gpu/drm/amd/display/dc/link/hwss/link_hwss_dpia.h diff --git a/drivers/gpu/drm/amd/display/dc/link/link_hwss_hpo_dp.c b/drivers/gpu/drm/amd/display/dc/link/hwss/link_hwss_hpo_dp.c similarity index 99% rename from drivers/gpu/drm/amd/display/dc/link/link_hwss_hpo_dp.c rename to drivers/gpu/drm/amd/display/dc/link/hwss/link_hwss_hpo_dp.c index 164d631e88099abd931b747a04731878247b4530..d8947f3233f31db78d305a6322e898fa7072c680 100644 --- a/drivers/gpu/drm/amd/display/dc/link/link_hwss_hpo_dp.c +++ b/drivers/gpu/drm/amd/display/dc/link/hwss/link_hwss_hpo_dp.c @@ -26,7 +26,6 @@ #include "dm_helpers.h" #include "core_types.h" #include "dccg.h" -#include "dc_link_dp.h" #include "clk_mgr.h" static enum phyd32clk_clock_source get_phyd32clk_src(struct dc_link *link) diff --git a/drivers/gpu/drm/amd/display/dc/link/link_hwss_hpo_dp.h b/drivers/gpu/drm/amd/display/dc/link/hwss/link_hwss_hpo_dp.h similarity index 98% rename from drivers/gpu/drm/amd/display/dc/link/link_hwss_hpo_dp.h rename to drivers/gpu/drm/amd/display/dc/link/hwss/link_hwss_hpo_dp.h index 57d447ec27b87d434be675b4b010a801817ac5a3..3cbb94b41a23163b948ecf9a10654447b468b3da 100644 --- a/drivers/gpu/drm/amd/display/dc/link/link_hwss_hpo_dp.h +++ b/drivers/gpu/drm/amd/display/dc/link/hwss/link_hwss_hpo_dp.h @@ -26,6 +26,7 @@ #define __LINK_HWSS_HPO_DP_H__ #include "link_hwss.h" +#include "link.h" bool can_use_hpo_dp_link_hwss(const struct dc_link *link, const struct link_resource *link_res); diff --git a/drivers/gpu/drm/amd/display/dc/link/link_ddc.c b/drivers/gpu/drm/amd/display/dc/link/protocols/link_ddc.c similarity index 100% rename from drivers/gpu/drm/amd/display/dc/link/link_ddc.c rename to drivers/gpu/drm/amd/display/dc/link/protocols/link_ddc.c diff --git a/drivers/gpu/drm/amd/display/dc/link/link_ddc.h b/drivers/gpu/drm/amd/display/dc/link/protocols/link_ddc.h similarity index 100% rename from drivers/gpu/drm/amd/display/dc/link/link_ddc.h rename to drivers/gpu/drm/amd/display/dc/link/protocols/link_ddc.h diff --git a/drivers/gpu/drm/amd/display/dc/link/link_dp_capability.c b/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_capability.c similarity index 98% rename from drivers/gpu/drm/amd/display/dc/link/link_dp_capability.c rename to drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_capability.c index e72ad1b8330f86b6146fcb47aafd6bef0609afd1..138e5684c7fd343126b3f1fd2fdd58d27b76e32b 100644 --- a/drivers/gpu/drm/amd/display/dc/link/link_dp_capability.c +++ b/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_capability.c @@ -39,12 +39,13 @@ #include "link_dpcd.h" #include "link_dp_dpia.h" #include "link_dp_phy.h" -#include "link_dp_trace.h" +#include "link_edp_panel_control.h" +#include "link_dp_irq_handler.h" +#include "link/accessories/link_dp_trace.h" #include "link_dp_training.h" #include "atomfirmware.h" #include "resource.h" #include "link_enc_cfg.h" -#include "dc_link_dp.h" #include "dc_dmub_srv.h" #define DC_LOGGER \ @@ -1462,7 +1463,7 @@ enum dc_status dp_retrieve_lttpr_cap(struct dc_link *link) bool vbios_lttpr_interop = link->dc->caps.vbios_lttpr_aware; if (!vbios_lttpr_interop || !link->dc->caps.extended_aux_timeout_support) - return DC_ERROR_UNEXPECTED; + return DC_NOT_SUPPORTED; /* By reading LTTPR capability, RX assumes that we will enable * LTTPR extended aux timeout if LTTPR is present. @@ -1645,6 +1646,22 @@ static bool retrieve_link_cap(struct dc_link *link) if (status != DC_OK) dm_error("%s: Read DPRX caps data failed.\n", __func__); + + /* AdaptiveSyncCapability */ + dpcd_dprx_data = 0; + for (i = 0; i < read_dpcd_retry_cnt; i++) { + status = core_link_read_dpcd( + link, DP_DPRX_FEATURE_ENUMERATION_LIST_CONT_1, + &dpcd_dprx_data, sizeof(dpcd_dprx_data)); + if (status == DC_OK) + break; + } + + link->dpcd_caps.adaptive_sync_caps.dp_adap_sync_caps.raw = dpcd_dprx_data; + + if (status != DC_OK) + dm_error("%s: Read DPRX caps data failed. Addr:%#x\n", + __func__, DP_DPRX_FEATURE_ENUMERATION_LIST_CONT_1); } else { @@ -1925,7 +1942,7 @@ void detect_edp_sink_caps(struct dc_link *link) link->dpcd_caps.set_power_state_capable_edp = (general_edp_cap & DP_EDP_SET_POWER_CAP) ? true:false; - dc_link_set_default_brightness_aux(link); + set_default_brightness_aux(link); core_link_read_dpcd(link, DP_EDP_DPCD_REV, &link->dpcd_caps.edp_rev, @@ -2110,8 +2127,8 @@ static bool dp_verify_link_cap( if (status == LINK_TRAINING_SUCCESS) { success = true; udelay(1000); - if (read_hpd_rx_irq_data(link, &irq_data) == DC_OK && - hpd_rx_irq_check_link_loss_status( + if (dp_read_hpd_rx_irq_data(link, &irq_data) == DC_OK && + dc_link_check_link_loss_status( link, &irq_data)) (*fail_count)++; diff --git a/drivers/gpu/drm/amd/display/dc/link/link_dp_capability.h b/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_capability.h similarity index 85% rename from drivers/gpu/drm/amd/display/dc/link/link_dp_capability.h rename to drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_capability.h index 5500744d2e47bbdf1daf3f0cd744ecc4515397bf..f79e4a4a9db628ccd0454287ee01cd845ae0a5ba 100644 --- a/drivers/gpu/drm/amd/display/dc/link/link_dp_capability.h +++ b/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_capability.h @@ -46,6 +46,15 @@ bool is_dp_active_dongle(const struct dc_link *link); bool is_dp_branch_device(const struct dc_link *link); +void dpcd_write_cable_id_to_dprx(struct dc_link *link); + +/* Initialize output parameter lt_settings. */ +void dp_decide_training_settings( + struct dc_link *link, + const struct dc_link_settings *link_setting, + struct link_training_settings *lt_settings); + + bool decide_edp_link_settings_with_dsc(struct dc_link *link, struct dc_link_settings *link_setting, uint32_t req_bw, @@ -62,5 +71,9 @@ bool decide_fallback_link_setting( struct dc_link_settings *cur, enum link_training_result training_result); +bool dp_verify_link_cap_with_retries( + struct dc_link *link, + struct dc_link_settings *known_limit_link_setting, + int attempts); #endif /* __DC_LINK_DP_CAPABILITY_H__ */ diff --git a/drivers/gpu/drm/amd/display/dc/link/link_dp_dpia.c b/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_dpia.c similarity index 99% rename from drivers/gpu/drm/amd/display/dc/link/link_dp_dpia.c rename to drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_dpia.c index 6136db392548103f4a389887a53f1c19d8af1fd6..a57a2be2eba0789817d25d5e022cb02e4d334b76 100644 --- a/drivers/gpu/drm/amd/display/dc/link/link_dp_dpia.c +++ b/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_dpia.c @@ -27,7 +27,6 @@ #include "dc.h" #include "inc/core_status.h" #include "dc_link.h" -#include "dc_link_dp.h" #include "dpcd_defs.h" #include "link_dp_dpia.h" @@ -104,4 +103,3 @@ bool dc_link_dpia_query_hpd_status(struct dc_link *link) return is_hpd_high; } - diff --git a/drivers/gpu/drm/amd/display/dc/link/link_dp_dpia.h b/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_dpia.h similarity index 100% rename from drivers/gpu/drm/amd/display/dc/link/link_dp_dpia.h rename to drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_dpia.h diff --git a/drivers/gpu/drm/amd/display/dc/link/link_dp_dpia_bw.c b/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_dpia_bw.c similarity index 100% rename from drivers/gpu/drm/amd/display/dc/link/link_dp_dpia_bw.c rename to drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_dpia_bw.c diff --git a/drivers/gpu/drm/amd/display/dc/link/link_dp_dpia_bw.h b/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_dpia_bw.h similarity index 100% rename from drivers/gpu/drm/amd/display/dc/link/link_dp_dpia_bw.h rename to drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_dpia_bw.h diff --git a/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_irq_handler.c b/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_irq_handler.c new file mode 100644 index 0000000000000000000000000000000000000000..9a832a9ea42d9efc2e3a28f061e49835e5a45986 --- /dev/null +++ b/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_irq_handler.c @@ -0,0 +1,401 @@ +/* + * Copyright 2022 Advanced Micro Devices, Inc. + * + * Permission is hereby granted, free of charge, to any person obtaining a + * copy of this software and associated documentation files (the "Software"), + * to deal in the Software without restriction, including without limitation + * the rights to use, copy, modify, merge, publish, distribute, sublicense, + * and/or sell copies of the Software, and to permit persons to whom the + * Software is furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL + * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR + * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, + * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR + * OTHER DEALINGS IN THE SOFTWARE. + * + * Authors: AMD + * + */ + +/* FILE POLICY AND INTENDED USAGE: + * + * + */ + +#include "link_dp_irq_handler.h" +#include "link_dpcd.h" +#include "link_dp_training.h" +#include "link_dp_capability.h" +#include "link/accessories/link_dp_trace.h" +#include "dm_helpers.h" + +#define DC_LOGGER_INIT(logger) + +bool dc_link_check_link_loss_status( + struct dc_link *link, + union hpd_irq_data *hpd_irq_dpcd_data) +{ + uint8_t irq_reg_rx_power_state = 0; + enum dc_status dpcd_result = DC_ERROR_UNEXPECTED; + union lane_status lane_status; + uint32_t lane; + bool sink_status_changed; + bool return_code; + + sink_status_changed = false; + return_code = false; + + if (link->cur_link_settings.lane_count == 0) + return return_code; + + /*1. Check that Link Status changed, before re-training.*/ + + /*parse lane status*/ + for (lane = 0; lane < link->cur_link_settings.lane_count; lane++) { + /* check status of lanes 0,1 + * changed DpcdAddress_Lane01Status (0x202) + */ + lane_status.raw = dp_get_nibble_at_index( + &hpd_irq_dpcd_data->bytes.lane01_status.raw, + lane); + + if (!lane_status.bits.CHANNEL_EQ_DONE_0 || + !lane_status.bits.CR_DONE_0 || + !lane_status.bits.SYMBOL_LOCKED_0) { + /* if one of the channel equalization, clock + * recovery or symbol lock is dropped + * consider it as (link has been + * dropped) dp sink status has changed + */ + sink_status_changed = true; + break; + } + } + + /* Check interlane align.*/ + if (sink_status_changed || + !hpd_irq_dpcd_data->bytes.lane_status_updated.bits.INTERLANE_ALIGN_DONE) { + + DC_LOG_HW_HPD_IRQ("%s: Link Status changed.\n", __func__); + + return_code = true; + + /*2. Check that we can handle interrupt: Not in FS DOS, + * Not in "Display Timeout" state, Link is trained. + */ + dpcd_result = core_link_read_dpcd(link, + DP_SET_POWER, + &irq_reg_rx_power_state, + sizeof(irq_reg_rx_power_state)); + + if (dpcd_result != DC_OK) { + DC_LOG_HW_HPD_IRQ("%s: DPCD read failed to obtain power state.\n", + __func__); + } else { + if (irq_reg_rx_power_state != DP_SET_POWER_D0) + return_code = false; + } + } + + return return_code; +} + +static bool handle_hpd_irq_psr_sink(struct dc_link *link) +{ + union dpcd_psr_configuration psr_configuration; + + if (!link->psr_settings.psr_feature_enabled) + return false; + + dm_helpers_dp_read_dpcd( + link->ctx, + link, + 368,/*DpcdAddress_PSR_Enable_Cfg*/ + &psr_configuration.raw, + sizeof(psr_configuration.raw)); + + if (psr_configuration.bits.ENABLE) { + unsigned char dpcdbuf[3] = {0}; + union psr_error_status psr_error_status; + union psr_sink_psr_status psr_sink_psr_status; + + dm_helpers_dp_read_dpcd( + link->ctx, + link, + 0x2006, /*DpcdAddress_PSR_Error_Status*/ + (unsigned char *) dpcdbuf, + sizeof(dpcdbuf)); + + /*DPCD 2006h ERROR STATUS*/ + psr_error_status.raw = dpcdbuf[0]; + /*DPCD 2008h SINK PANEL SELF REFRESH STATUS*/ + psr_sink_psr_status.raw = dpcdbuf[2]; + + if (psr_error_status.bits.LINK_CRC_ERROR || + psr_error_status.bits.RFB_STORAGE_ERROR || + psr_error_status.bits.VSC_SDP_ERROR) { + bool allow_active; + + /* Acknowledge and clear error bits */ + dm_helpers_dp_write_dpcd( + link->ctx, + link, + 8198,/*DpcdAddress_PSR_Error_Status*/ + &psr_error_status.raw, + sizeof(psr_error_status.raw)); + + /* PSR error, disable and re-enable PSR */ + if (link->psr_settings.psr_allow_active) { + allow_active = false; + dc_link_set_psr_allow_active(link, &allow_active, true, false, NULL); + allow_active = true; + dc_link_set_psr_allow_active(link, &allow_active, true, false, NULL); + } + + return true; + } else if (psr_sink_psr_status.bits.SINK_SELF_REFRESH_STATUS == + PSR_SINK_STATE_ACTIVE_DISPLAY_FROM_SINK_RFB){ + /* No error is detect, PSR is active. + * We should return with IRQ_HPD handled without + * checking for loss of sync since PSR would have + * powered down main link. + */ + return true; + } + } + return false; +} + +void dc_link_dp_handle_link_loss(struct dc_link *link) +{ + int i; + struct pipe_ctx *pipe_ctx; + + for (i = 0; i < MAX_PIPES; i++) { + pipe_ctx = &link->dc->current_state->res_ctx.pipe_ctx[i]; + if (pipe_ctx && pipe_ctx->stream && pipe_ctx->stream->link == link) + break; + } + + if (pipe_ctx == NULL || pipe_ctx->stream == NULL) + return; + + for (i = 0; i < MAX_PIPES; i++) { + pipe_ctx = &link->dc->current_state->res_ctx.pipe_ctx[i]; + if (pipe_ctx && pipe_ctx->stream && !pipe_ctx->stream->dpms_off && + pipe_ctx->stream->link == link && !pipe_ctx->prev_odm_pipe) + core_link_disable_stream(pipe_ctx); + } + + for (i = 0; i < MAX_PIPES; i++) { + pipe_ctx = &link->dc->current_state->res_ctx.pipe_ctx[i]; + if (pipe_ctx && pipe_ctx->stream && !pipe_ctx->stream->dpms_off + && pipe_ctx->stream->link == link && !pipe_ctx->prev_odm_pipe) { + // Always use max settings here for DP 1.4a LL Compliance CTS + if (link->is_automated) { + pipe_ctx->link_config.dp_link_settings.lane_count = + link->verified_link_cap.lane_count; + pipe_ctx->link_config.dp_link_settings.link_rate = + link->verified_link_cap.link_rate; + pipe_ctx->link_config.dp_link_settings.link_spread = + link->verified_link_cap.link_spread; + } + core_link_enable_stream(link->dc->current_state, pipe_ctx); + } + } +} + +enum dc_status dp_read_hpd_rx_irq_data( + struct dc_link *link, + union hpd_irq_data *irq_data) +{ + static enum dc_status retval; + + /* The HW reads 16 bytes from 200h on HPD, + * but if we get an AUX_DEFER, the HW cannot retry + * and this causes the CTS tests 4.3.2.1 - 3.2.4 to + * fail, so we now explicitly read 6 bytes which is + * the req from the above mentioned test cases. + * + * For DP 1.4 we need to read those from 2002h range. + */ + if (link->dpcd_caps.dpcd_rev.raw < DPCD_REV_14) + retval = core_link_read_dpcd( + link, + DP_SINK_COUNT, + irq_data->raw, + sizeof(union hpd_irq_data)); + else { + /* Read 14 bytes in a single read and then copy only the required fields. + * This is more efficient than doing it in two separate AUX reads. */ + + uint8_t tmp[DP_SINK_STATUS_ESI - DP_SINK_COUNT_ESI + 1]; + + retval = core_link_read_dpcd( + link, + DP_SINK_COUNT_ESI, + tmp, + sizeof(tmp)); + + if (retval != DC_OK) + return retval; + + irq_data->bytes.sink_cnt.raw = tmp[DP_SINK_COUNT_ESI - DP_SINK_COUNT_ESI]; + irq_data->bytes.device_service_irq.raw = tmp[DP_DEVICE_SERVICE_IRQ_VECTOR_ESI0 - DP_SINK_COUNT_ESI]; + irq_data->bytes.lane01_status.raw = tmp[DP_LANE0_1_STATUS_ESI - DP_SINK_COUNT_ESI]; + irq_data->bytes.lane23_status.raw = tmp[DP_LANE2_3_STATUS_ESI - DP_SINK_COUNT_ESI]; + irq_data->bytes.lane_status_updated.raw = tmp[DP_LANE_ALIGN_STATUS_UPDATED_ESI - DP_SINK_COUNT_ESI]; + irq_data->bytes.sink_status.raw = tmp[DP_SINK_STATUS_ESI - DP_SINK_COUNT_ESI]; + } + + return retval; +} + +/*************************Short Pulse IRQ***************************/ +bool dc_link_dp_allow_hpd_rx_irq(const struct dc_link *link) +{ + /* + * Don't handle RX IRQ unless one of following is met: + * 1) The link is established (cur_link_settings != unknown) + * 2) We know we're dealing with a branch device, SST or MST + */ + + if ((link->cur_link_settings.lane_count != LANE_COUNT_UNKNOWN) || + is_dp_branch_device(link)) + return true; + + return false; +} + +bool dc_link_handle_hpd_rx_irq(struct dc_link *link, union hpd_irq_data *out_hpd_irq_dpcd_data, bool *out_link_loss, + bool defer_handling, bool *has_left_work) +{ + union hpd_irq_data hpd_irq_dpcd_data = {0}; + union device_service_irq device_service_clear = {0}; + enum dc_status result; + bool status = false; + + if (out_link_loss) + *out_link_loss = false; + + if (has_left_work) + *has_left_work = false; + /* For use cases related to down stream connection status change, + * PSR and device auto test, refer to function handle_sst_hpd_irq + * in DAL2.1*/ + + DC_LOG_HW_HPD_IRQ("%s: Got short pulse HPD on link %d\n", + __func__, link->link_index); + + + /* All the "handle_hpd_irq_xxx()" methods + * should be called only after + * dal_dpsst_ls_read_hpd_irq_data + * Order of calls is important too + */ + result = dp_read_hpd_rx_irq_data(link, &hpd_irq_dpcd_data); + if (out_hpd_irq_dpcd_data) + *out_hpd_irq_dpcd_data = hpd_irq_dpcd_data; + + if (result != DC_OK) { + DC_LOG_HW_HPD_IRQ("%s: DPCD read failed to obtain irq data\n", + __func__); + return false; + } + + if (hpd_irq_dpcd_data.bytes.device_service_irq.bits.AUTOMATED_TEST) { + // Workaround for DP 1.4a LL Compliance CTS as USB4 has to share encoders unlike DP and USBC + link->is_automated = true; + device_service_clear.bits.AUTOMATED_TEST = 1; + core_link_write_dpcd( + link, + DP_DEVICE_SERVICE_IRQ_VECTOR, + &device_service_clear.raw, + sizeof(device_service_clear.raw)); + device_service_clear.raw = 0; + if (defer_handling && has_left_work) + *has_left_work = true; + else + dc_link_dp_handle_automated_test(link); + return false; + } + + if (!dc_link_dp_allow_hpd_rx_irq(link)) { + DC_LOG_HW_HPD_IRQ("%s: skipping HPD handling on %d\n", + __func__, link->link_index); + return false; + } + + if (handle_hpd_irq_psr_sink(link)) + /* PSR-related error was detected and handled */ + return true; + + /* If PSR-related error handled, Main link may be off, + * so do not handle as a normal sink status change interrupt. + */ + + if (hpd_irq_dpcd_data.bytes.device_service_irq.bits.UP_REQ_MSG_RDY) { + if (defer_handling && has_left_work) + *has_left_work = true; + return true; + } + + /* check if we have MST msg and return since we poll for it */ + if (hpd_irq_dpcd_data.bytes.device_service_irq.bits.DOWN_REP_MSG_RDY) { + if (defer_handling && has_left_work) + *has_left_work = true; + return false; + } + + /* For now we only handle 'Downstream port status' case. + * If we got sink count changed it means + * Downstream port status changed, + * then DM should call DC to do the detection. + * NOTE: Do not handle link loss on eDP since it is internal link*/ + if ((link->connector_signal != SIGNAL_TYPE_EDP) && + dc_link_check_link_loss_status( + link, + &hpd_irq_dpcd_data)) { + /* Connectivity log: link loss */ + CONN_DATA_LINK_LOSS(link, + hpd_irq_dpcd_data.raw, + sizeof(hpd_irq_dpcd_data), + "Status: "); + + if (defer_handling && has_left_work) + *has_left_work = true; + else + dc_link_dp_handle_link_loss(link); + + status = false; + if (out_link_loss) + *out_link_loss = true; + + dp_trace_link_loss_increment(link); + } + + if (link->type == dc_connection_sst_branch && + hpd_irq_dpcd_data.bytes.sink_cnt.bits.SINK_COUNT + != link->dpcd_sink_count) + status = true; + + /* reasons for HPD RX: + * 1. Link Loss - ie Re-train the Link + * 2. MST sideband message + * 3. Automated Test - ie. Internal Commit + * 4. CP (copy protection) - (not interesting for DM???) + * 5. DRR + * 6. Downstream Port status changed + * -ie. Detect - this the only one + * which is interesting for DM because + * it must call dc_link_detect. + */ + return status; +} + diff --git a/drivers/gpu/drm/amd/display/dc/link/link_hwss_hpo_frl.h b/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_irq_handler.h similarity index 80% rename from drivers/gpu/drm/amd/display/dc/link/link_hwss_hpo_frl.h rename to drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_irq_handler.h index ea8d9760132fd790eee26b6bdaa2399e75dcff59..cac7acd33511d849be022ccd3e840ebc0e3b736e 100644 --- a/drivers/gpu/drm/amd/display/dc/link/link_hwss_hpo_frl.h +++ b/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_irq_handler.h @@ -22,13 +22,14 @@ * Authors: AMD * */ -#ifndef __LINK_HWSS_HPO_FRL_H__ -#define __LINK_HWSS_HPO_FRL_H__ -#include "link_hwss.h" +#ifndef __DC_LINK_DP_IRQ_HANDLER_H__ +#define __DC_LINK_DP_IRQ_HANDLER_H__ -bool can_use_hpo_frl_link_hwss(const struct dc_link *link, - const struct link_resource *link_res); -const struct link_hwss *get_hpo_frl_link_hwss(void); +#include "link.h" -#endif /* __LINK_HWSS_HPO_FRL_H__ */ +enum dc_status dp_read_hpd_rx_irq_data( + struct dc_link *link, + union hpd_irq_data *irq_data); + +#endif /* __DC_LINK_DP_IRQ_HANDLER_H__ */ diff --git a/drivers/gpu/drm/amd/display/dc/link/link_dp_phy.c b/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_phy.c similarity index 98% rename from drivers/gpu/drm/amd/display/dc/link/link_dp_phy.c rename to drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_phy.c index afe3b21335c27232c8955a2a0570d44f2836c5fb..5e531620926fef130b2e818cc78d4823d7d6d667 100644 --- a/drivers/gpu/drm/amd/display/dc/link/link_dp_phy.c +++ b/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_phy.c @@ -36,27 +36,10 @@ #include "link_dp_capability.h" #include "clk_mgr.h" #include "resource.h" -#include "dc_link_dp.h" #define DC_LOGGER \ link->ctx->logger -void dc_link_dp_set_drive_settings( - struct dc_link *link, - const struct link_resource *link_res, - struct link_training_settings *lt_settings) -{ - /* program ASIC PHY settings*/ - dp_set_hw_lane_settings(link, link_res, lt_settings, DPRX); - - dp_hw_to_dpcd_lane_settings(lt_settings, - lt_settings->hw_lane_settings, - lt_settings->dpcd_lane_settings); - - /* Notify DP sink the PHY settings from source */ - dpcd_set_lane_settings(link, lt_settings, DPRX); -} - void dc_link_dp_receiver_power_ctrl(struct dc_link *link, bool on) { uint8_t state; @@ -143,3 +126,19 @@ void dp_set_hw_lane_settings( link_settings->hw_lane_settings, sizeof(link->cur_lane_setting)); } + +void dp_set_drive_settings( + struct dc_link *link, + const struct link_resource *link_res, + struct link_training_settings *lt_settings) +{ + /* program ASIC PHY settings*/ + dp_set_hw_lane_settings(link, link_res, lt_settings, DPRX); + + dp_hw_to_dpcd_lane_settings(lt_settings, + lt_settings->hw_lane_settings, + lt_settings->dpcd_lane_settings); + + /* Notify DP sink the PHY settings from source */ + dpcd_set_lane_settings(link, lt_settings, DPRX); +} diff --git a/drivers/gpu/drm/amd/display/dc/link/link_dp_phy.h b/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_phy.h similarity index 93% rename from drivers/gpu/drm/amd/display/dc/link/link_dp_phy.h rename to drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_phy.h index 717e078fd564b598b4c1bfe5eb8664bdf1bea942..850da64c90058b8979685937e95deffa95c153ac 100644 --- a/drivers/gpu/drm/amd/display/dc/link/link_dp_phy.h +++ b/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_phy.h @@ -48,4 +48,9 @@ void dp_set_hw_lane_settings( const struct link_training_settings *link_settings, uint32_t offset); +void dp_set_drive_settings( + struct dc_link *link, + const struct link_resource *link_res, + struct link_training_settings *lt_settings); + #endif /* __DC_LINK_DP_PHY_H__ */ diff --git a/drivers/gpu/drm/amd/display/dc/link/link_dp_training.c b/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_training.c similarity index 99% rename from drivers/gpu/drm/amd/display/dc/link/link_dp_training.c rename to drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_training.c index e49e0258a1bdc98a5da08b3a1b5f2dbdf2b195ea..18ec09b5a8ba47c6f9ab475941cd3f8c7ad6983f 100644 --- a/drivers/gpu/drm/amd/display/dc/link/link_dp_training.c +++ b/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_training.c @@ -37,10 +37,10 @@ #include "link_dp_training_dpia.h" #include "link_dp_training_fixed_vs_pe_retimer.h" #include "link_dpcd.h" -#include "link_dp_trace.h" +#include "link/accessories/link_dp_trace.h" #include "link_dp_phy.h" #include "link_dp_capability.h" -#include "dc_link_dp.h" +#include "link_edp_panel_control.h" #include "atomfirmware.h" #include "link_enc_cfg.h" #include "resource.h" @@ -239,7 +239,7 @@ enum dpcd_training_patterns return dpcd_tr_pattern; } -static uint8_t get_nibble_at_index(const uint8_t *buf, +uint8_t dp_get_nibble_at_index(const uint8_t *buf, uint32_t index) { uint8_t nibble; @@ -519,7 +519,7 @@ enum link_training_result dp_check_link_loss_status( /* * check lanes status */ - lane_status.raw = get_nibble_at_index(&dpcd_buf[2], lane); + lane_status.raw = dp_get_nibble_at_index(&dpcd_buf[2], lane); if (!lane_status.bits.CHANNEL_EQ_DONE_0 || !lane_status.bits.CR_DONE_0 || @@ -578,9 +578,9 @@ enum dc_status dp_get_lane_status_and_lane_adjust( lane++) { ln_status[lane].raw = - get_nibble_at_index(&dpcd_buf[0], lane); + dp_get_nibble_at_index(&dpcd_buf[0], lane); ln_adjust[lane].raw = - get_nibble_at_index(&dpcd_buf[lane_adjust_offset], lane); + dp_get_nibble_at_index(&dpcd_buf[lane_adjust_offset], lane); } ln_align->raw = dpcd_buf[2]; @@ -1389,7 +1389,7 @@ static bool perform_post_lt_adj_req_sequence( dp_decide_lane_settings(lt_settings, dpcd_lane_adjust, lt_settings->hw_lane_settings, lt_settings->dpcd_lane_settings); - dc_link_dp_set_drive_settings(link, + dp_set_drive_settings(link, link_res, lt_settings); break; diff --git a/drivers/gpu/drm/amd/display/dc/link/link_dp_training.h b/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_training.h similarity index 98% rename from drivers/gpu/drm/amd/display/dc/link/link_dp_training.h rename to drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_training.h index 376d370e3bbbe755b25b228db7f60f8d440e35f6..a04948635369f639cad6190a6e3f9ff9a451ec3f 100644 --- a/drivers/gpu/drm/amd/display/dc/link/link_dp_training.h +++ b/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_training.h @@ -176,4 +176,7 @@ void dp_log_training_result( uint32_t dp_translate_training_aux_read_interval( uint32_t dpcd_aux_read_interval); + +uint8_t dp_get_nibble_at_index(const uint8_t *buf, + uint32_t index); #endif /* __DC_LINK_DP_TRAINING_H__ */ diff --git a/drivers/gpu/drm/amd/display/dc/link/link_dp_training_128b_132b.c b/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_training_128b_132b.c similarity index 99% rename from drivers/gpu/drm/amd/display/dc/link/link_dp_training_128b_132b.c rename to drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_training_128b_132b.c index bfabebed58689ea3af227cab92b31c2bf06e6f78..23d380f09a21c8a0371e31207d475e59035e2de0 100644 --- a/drivers/gpu/drm/amd/display/dc/link/link_dp_training_128b_132b.c +++ b/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_training_128b_132b.c @@ -32,7 +32,6 @@ #include "link_dpcd.h" #include "link_dp_phy.h" #include "link_dp_capability.h" -#include "dc_link_dp.h" #define DC_LOGGER \ link->ctx->logger diff --git a/drivers/gpu/drm/amd/display/dc/link/link_dp_training_128b_132b.h b/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_training_128b_132b.h similarity index 100% rename from drivers/gpu/drm/amd/display/dc/link/link_dp_training_128b_132b.h rename to drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_training_128b_132b.h diff --git a/drivers/gpu/drm/amd/display/dc/link/link_dp_training_8b_10b.c b/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_training_8b_10b.c similarity index 99% rename from drivers/gpu/drm/amd/display/dc/link/link_dp_training_8b_10b.c rename to drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_training_8b_10b.c index ec8b619d51c5e8ae520c3acef4416b784a54218f..14b98e096d392691c51885461928d9cfe713dcf3 100644 --- a/drivers/gpu/drm/amd/display/dc/link/link_dp_training_8b_10b.c +++ b/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_training_8b_10b.c @@ -31,7 +31,6 @@ #include "link_dpcd.h" #include "link_dp_phy.h" #include "link_dp_capability.h" -#include "dc_link_dp.h" #define DC_LOGGER \ link->ctx->logger diff --git a/drivers/gpu/drm/amd/display/dc/link/link_dp_training_8b_10b.h b/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_training_8b_10b.h similarity index 100% rename from drivers/gpu/drm/amd/display/dc/link/link_dp_training_8b_10b.h rename to drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_training_8b_10b.h diff --git a/drivers/gpu/drm/amd/display/dc/link/link_dp_training_auxless.c b/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_training_auxless.c similarity index 99% rename from drivers/gpu/drm/amd/display/dc/link/link_dp_training_auxless.c rename to drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_training_auxless.c index f84b6ea53e8b44e5c7c7ba02cc1aeef2e0831507..e50ec5012559bd4d07ec3dc204e182d7e52aa9d6 100644 --- a/drivers/gpu/drm/amd/display/dc/link/link_dp_training_auxless.c +++ b/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_training_auxless.c @@ -28,7 +28,6 @@ */ #include "link_dp_training_auxless.h" #include "link_dp_phy.h" -#include "dc_link_dp.h" #define DC_LOGGER \ link->ctx->logger bool dc_link_dp_perform_link_training_skip_aux( diff --git a/drivers/gpu/drm/amd/display/dc/link/link_dp_training_auxless.h b/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_training_auxless.h similarity index 100% rename from drivers/gpu/drm/amd/display/dc/link/link_dp_training_auxless.h rename to drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_training_auxless.h diff --git a/drivers/gpu/drm/amd/display/dc/link/link_dp_training_dpia.c b/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_training_dpia.c similarity index 99% rename from drivers/gpu/drm/amd/display/dc/link/link_dp_training_dpia.c rename to drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_training_dpia.c index cf47db1c21410c9044d81b91ab8c7bca7428de82..05657627ae45ea9b2851e6d93809ae569e4e7bbb 100644 --- a/drivers/gpu/drm/amd/display/dc/link/link_dp_training_dpia.c +++ b/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_training_dpia.c @@ -30,7 +30,6 @@ #include "dc.h" #include "inc/core_status.h" #include "dc_link.h" -#include "dc_link_dp.h" #include "dpcd_defs.h" #include "link_dp_dpia.h" diff --git a/drivers/gpu/drm/amd/display/dc/link/link_dp_training_dpia.h b/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_training_dpia.h similarity index 100% rename from drivers/gpu/drm/amd/display/dc/link/link_dp_training_dpia.h rename to drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_training_dpia.h diff --git a/drivers/gpu/drm/amd/display/dc/link/link_dp_training_fixed_vs_pe_retimer.c b/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_training_fixed_vs_pe_retimer.c similarity index 99% rename from drivers/gpu/drm/amd/display/dc/link/link_dp_training_fixed_vs_pe_retimer.c rename to drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_training_fixed_vs_pe_retimer.c index 860b5eea89aa965609825374217ef7d9456297b2..a4071d2959a03303d67b0f4b700d4817e76bf75b 100644 --- a/drivers/gpu/drm/amd/display/dc/link/link_dp_training_fixed_vs_pe_retimer.c +++ b/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_training_fixed_vs_pe_retimer.c @@ -36,7 +36,6 @@ #include "link_dpcd.h" #include "link_dp_phy.h" #include "link_dp_capability.h" -#include "dc_link_dp.h" #define DC_LOGGER \ link->ctx->logger diff --git a/drivers/gpu/drm/amd/display/dc/link/link_dp_training_fixed_vs_pe_retimer.h b/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_training_fixed_vs_pe_retimer.h similarity index 100% rename from drivers/gpu/drm/amd/display/dc/link/link_dp_training_fixed_vs_pe_retimer.h rename to drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_training_fixed_vs_pe_retimer.h diff --git a/drivers/gpu/drm/amd/display/dc/link/link_dpcd.c b/drivers/gpu/drm/amd/display/dc/link/protocols/link_dpcd.c similarity index 100% rename from drivers/gpu/drm/amd/display/dc/link/link_dpcd.c rename to drivers/gpu/drm/amd/display/dc/link/protocols/link_dpcd.c diff --git a/drivers/gpu/drm/amd/display/dc/link/link_dpcd.h b/drivers/gpu/drm/amd/display/dc/link/protocols/link_dpcd.h similarity index 100% rename from drivers/gpu/drm/amd/display/dc/link/link_dpcd.h rename to drivers/gpu/drm/amd/display/dc/link/protocols/link_dpcd.h diff --git a/drivers/gpu/drm/amd/display/dc/link/protocols/link_edp_panel_control.c b/drivers/gpu/drm/amd/display/dc/link/protocols/link_edp_panel_control.c new file mode 100644 index 0000000000000000000000000000000000000000..97e02b5b21ae3d00cb7f3e7a4659f10cb351a5dd --- /dev/null +++ b/drivers/gpu/drm/amd/display/dc/link/protocols/link_edp_panel_control.c @@ -0,0 +1,833 @@ +/* + * Copyright 2022 Advanced Micro Devices, Inc. + * + * Permission is hereby granted, free of charge, to any person obtaining a + * copy of this software and associated documentation files (the "Software"), + * to deal in the Software without restriction, including without limitation + * the rights to use, copy, modify, merge, publish, distribute, sublicense, + * and/or sell copies of the Software, and to permit persons to whom the + * Software is furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL + * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR + * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, + * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR + * OTHER DEALINGS IN THE SOFTWARE. + * + * Authors: AMD + * + */ + +/* FILE POLICY AND INTENDED USAGE: + * This file implements retrieval and configuration of eDP panel features such + * as PSR and ABM and it also manages specs defined eDP panel power sequences. + */ + +#include "link_edp_panel_control.h" +#include "link_dpcd.h" +#include "link_dp_capability.h" +#include "dm_helpers.h" +#include "dal_asic_id.h" +#include "dce/dmub_psr.h" +#include "abm.h" +#define DC_LOGGER_INIT(logger) + +void dp_set_panel_mode(struct dc_link *link, enum dp_panel_mode panel_mode) +{ + union dpcd_edp_config edp_config_set; + bool panel_mode_edp = false; + + memset(&edp_config_set, '\0', sizeof(union dpcd_edp_config)); + + if (panel_mode != DP_PANEL_MODE_DEFAULT) { + + switch (panel_mode) { + case DP_PANEL_MODE_EDP: + case DP_PANEL_MODE_SPECIAL: + panel_mode_edp = true; + break; + + default: + break; + } + + /*set edp panel mode in receiver*/ + core_link_read_dpcd( + link, + DP_EDP_CONFIGURATION_SET, + &edp_config_set.raw, + sizeof(edp_config_set.raw)); + + if (edp_config_set.bits.PANEL_MODE_EDP + != panel_mode_edp) { + enum dc_status result; + + edp_config_set.bits.PANEL_MODE_EDP = + panel_mode_edp; + result = core_link_write_dpcd( + link, + DP_EDP_CONFIGURATION_SET, + &edp_config_set.raw, + sizeof(edp_config_set.raw)); + + ASSERT(result == DC_OK); + } + } + DC_LOG_DETECTION_DP_CAPS("Link: %d eDP panel mode supported: %d " + "eDP panel mode enabled: %d \n", + link->link_index, + link->dpcd_caps.panel_mode_edp, + panel_mode_edp); +} + +enum dp_panel_mode dp_get_panel_mode(struct dc_link *link) +{ + /* We need to explicitly check that connector + * is not DP. Some Travis_VGA get reported + * by video bios as DP. + */ + if (link->connector_signal != SIGNAL_TYPE_DISPLAY_PORT) { + + switch (link->dpcd_caps.branch_dev_id) { + case DP_BRANCH_DEVICE_ID_0022B9: + /* alternate scrambler reset is required for Travis + * for the case when external chip does not + * provide sink device id, alternate scrambler + * scheme will be overriden later by querying + * Encoder features + */ + if (strncmp( + link->dpcd_caps.branch_dev_name, + DP_VGA_LVDS_CONVERTER_ID_2, + sizeof( + link->dpcd_caps. + branch_dev_name)) == 0) { + return DP_PANEL_MODE_SPECIAL; + } + break; + case DP_BRANCH_DEVICE_ID_00001A: + /* alternate scrambler reset is required for Travis + * for the case when external chip does not provide + * sink device id, alternate scrambler scheme will + * be overriden later by querying Encoder feature + */ + if (strncmp(link->dpcd_caps.branch_dev_name, + DP_VGA_LVDS_CONVERTER_ID_3, + sizeof( + link->dpcd_caps. + branch_dev_name)) == 0) { + return DP_PANEL_MODE_SPECIAL; + } + break; + default: + break; + } + } + + if (link->dpcd_caps.panel_mode_edp && + (link->connector_signal == SIGNAL_TYPE_EDP || + (link->connector_signal == SIGNAL_TYPE_DISPLAY_PORT && + link->is_internal_display))) { + return DP_PANEL_MODE_EDP; + } + + return DP_PANEL_MODE_DEFAULT; +} + +bool dc_link_set_backlight_level_nits(struct dc_link *link, + bool isHDR, + uint32_t backlight_millinits, + uint32_t transition_time_in_ms) +{ + struct dpcd_source_backlight_set dpcd_backlight_set; + uint8_t backlight_control = isHDR ? 1 : 0; + + if (!link || (link->connector_signal != SIGNAL_TYPE_EDP && + link->connector_signal != SIGNAL_TYPE_DISPLAY_PORT)) + return false; + + // OLEDs have no PWM, they can only use AUX + if (link->dpcd_sink_ext_caps.bits.oled == 1) + backlight_control = 1; + + *(uint32_t *)&dpcd_backlight_set.backlight_level_millinits = backlight_millinits; + *(uint16_t *)&dpcd_backlight_set.backlight_transition_time_ms = (uint16_t)transition_time_in_ms; + + + if (core_link_write_dpcd(link, DP_SOURCE_BACKLIGHT_LEVEL, + (uint8_t *)(&dpcd_backlight_set), + sizeof(dpcd_backlight_set)) != DC_OK) + return false; + + if (core_link_write_dpcd(link, DP_SOURCE_BACKLIGHT_CONTROL, + &backlight_control, 1) != DC_OK) + return false; + + return true; +} + +bool dc_link_get_backlight_level_nits(struct dc_link *link, + uint32_t *backlight_millinits_avg, + uint32_t *backlight_millinits_peak) +{ + union dpcd_source_backlight_get dpcd_backlight_get; + + memset(&dpcd_backlight_get, 0, sizeof(union dpcd_source_backlight_get)); + + if (!link || (link->connector_signal != SIGNAL_TYPE_EDP && + link->connector_signal != SIGNAL_TYPE_DISPLAY_PORT)) + return false; + + if (!core_link_read_dpcd(link, DP_SOURCE_BACKLIGHT_CURRENT_PEAK, + dpcd_backlight_get.raw, + sizeof(union dpcd_source_backlight_get))) + return false; + + *backlight_millinits_avg = + dpcd_backlight_get.bytes.backlight_millinits_avg; + *backlight_millinits_peak = + dpcd_backlight_get.bytes.backlight_millinits_peak; + + /* On non-supported panels dpcd_read usually succeeds with 0 returned */ + if (*backlight_millinits_avg == 0 || + *backlight_millinits_avg > *backlight_millinits_peak) + return false; + + return true; +} + +bool link_backlight_enable_aux(struct dc_link *link, bool enable) +{ + uint8_t backlight_enable = enable ? 1 : 0; + + if (!link || (link->connector_signal != SIGNAL_TYPE_EDP && + link->connector_signal != SIGNAL_TYPE_DISPLAY_PORT)) + return false; + + if (core_link_write_dpcd(link, DP_SOURCE_BACKLIGHT_ENABLE, + &backlight_enable, 1) != DC_OK) + return false; + + return true; +} + +// we read default from 0x320 because we expect BIOS wrote it there +// regular get_backlight_nit reads from panel set at 0x326 +static bool read_default_bl_aux(struct dc_link *link, uint32_t *backlight_millinits) +{ + if (!link || (link->connector_signal != SIGNAL_TYPE_EDP && + link->connector_signal != SIGNAL_TYPE_DISPLAY_PORT)) + return false; + + if (!core_link_read_dpcd(link, DP_SOURCE_BACKLIGHT_LEVEL, + (uint8_t *) backlight_millinits, + sizeof(uint32_t))) + return false; + + return true; +} + +bool set_default_brightness_aux(struct dc_link *link) +{ + uint32_t default_backlight; + + if (link && link->dpcd_sink_ext_caps.bits.oled == 1) { + if (!read_default_bl_aux(link, &default_backlight)) + default_backlight = 150000; + // if < 5 nits or > 5000, it might be wrong readback + if (default_backlight < 5000 || default_backlight > 5000000) + default_backlight = 150000; // + + return dc_link_set_backlight_level_nits(link, true, + default_backlight, 0); + } + return false; +} + +bool link_is_edp_ilr_optimization_required(struct dc_link *link, + struct dc_crtc_timing *crtc_timing) +{ + struct dc_link_settings link_setting; + uint8_t link_bw_set; + uint8_t link_rate_set; + uint32_t req_bw; + union lane_count_set lane_count_set = {0}; + + ASSERT(link || crtc_timing); // invalid input + + if (link->dpcd_caps.edp_supported_link_rates_count == 0 || + !link->panel_config.ilr.optimize_edp_link_rate) + return false; + + + // Read DPCD 00100h to find if standard link rates are set + core_link_read_dpcd(link, DP_LINK_BW_SET, + &link_bw_set, sizeof(link_bw_set)); + + if (link_bw_set) { + DC_LOG_EVENT_LINK_TRAINING("eDP ILR: Optimization required, VBIOS used link_bw_set\n"); + return true; + } + + // Read DPCD 00115h to find the edp link rate set used + core_link_read_dpcd(link, DP_LINK_RATE_SET, + &link_rate_set, sizeof(link_rate_set)); + + // Read DPCD 00101h to find out the number of lanes currently set + core_link_read_dpcd(link, DP_LANE_COUNT_SET, + &lane_count_set.raw, sizeof(lane_count_set)); + + req_bw = dc_bandwidth_in_kbps_from_timing(crtc_timing); + + if (!crtc_timing->flags.DSC) + dc_link_decide_edp_link_settings(link, &link_setting, req_bw); + else + decide_edp_link_settings_with_dsc(link, &link_setting, req_bw, LINK_RATE_UNKNOWN); + + if (link->dpcd_caps.edp_supported_link_rates[link_rate_set] != link_setting.link_rate || + lane_count_set.bits.LANE_COUNT_SET != link_setting.lane_count) { + DC_LOG_EVENT_LINK_TRAINING("eDP ILR: Optimization required, VBIOS link_rate_set not optimal\n"); + return true; + } + + DC_LOG_EVENT_LINK_TRAINING("eDP ILR: No optimization required, VBIOS set optimal link_rate_set\n"); + return false; +} + +void dc_link_edp_panel_backlight_power_on(struct dc_link *link, bool wait_for_hpd) +{ + if (link->connector_signal != SIGNAL_TYPE_EDP) + return; + + link->dc->hwss.edp_power_control(link, true); + if (wait_for_hpd) + link->dc->hwss.edp_wait_for_hpd_ready(link, true); + if (link->dc->hwss.edp_backlight_control) + link->dc->hwss.edp_backlight_control(link, true); +} + +bool dc_link_wait_for_t12(struct dc_link *link) +{ + if (link->connector_signal == SIGNAL_TYPE_EDP && link->dc->hwss.edp_wait_for_T12) { + link->dc->hwss.edp_wait_for_T12(link); + + return true; + } + + return false; +} + +void link_edp_add_delay_for_T9(struct dc_link *link) +{ + if (link && link->panel_config.pps.extra_delay_backlight_off > 0) + udelay(link->panel_config.pps.extra_delay_backlight_off * 1000); +} + +bool link_edp_receiver_ready_T9(struct dc_link *link) +{ + unsigned int tries = 0; + unsigned char sinkstatus = 0; + unsigned char edpRev = 0; + enum dc_status result = DC_OK; + + result = core_link_read_dpcd(link, DP_EDP_DPCD_REV, &edpRev, sizeof(edpRev)); + + /* start from eDP version 1.2, SINK_STAUS indicate the sink is ready.*/ + if (result == DC_OK && edpRev >= DP_EDP_12) { + do { + sinkstatus = 1; + result = core_link_read_dpcd(link, DP_SINK_STATUS, &sinkstatus, sizeof(sinkstatus)); + if (sinkstatus == 0) + break; + if (result != DC_OK) + break; + udelay(100); //MAx T9 + } while (++tries < 50); + } + + return result; +} + +bool link_edp_receiver_ready_T7(struct dc_link *link) +{ + unsigned char sinkstatus = 0; + unsigned char edpRev = 0; + enum dc_status result = DC_OK; + + /* use absolute time stamp to constrain max T7*/ + unsigned long long enter_timestamp = 0; + unsigned long long finish_timestamp = 0; + unsigned long long time_taken_in_ns = 0; + + result = core_link_read_dpcd(link, DP_EDP_DPCD_REV, &edpRev, sizeof(edpRev)); + + if (result == DC_OK && edpRev >= DP_EDP_12) { + /* start from eDP version 1.2, SINK_STAUS indicate the sink is ready.*/ + enter_timestamp = dm_get_timestamp(link->ctx); + do { + sinkstatus = 0; + result = core_link_read_dpcd(link, DP_SINK_STATUS, &sinkstatus, sizeof(sinkstatus)); + if (sinkstatus == 1) + break; + if (result != DC_OK) + break; + udelay(25); + finish_timestamp = dm_get_timestamp(link->ctx); + time_taken_in_ns = dm_get_elapse_time_in_ns(link->ctx, finish_timestamp, enter_timestamp); + } while (time_taken_in_ns < 50 * 1000000); //MAx T7 is 50ms + } + + if (link && link->panel_config.pps.extra_t7_ms > 0) + udelay(link->panel_config.pps.extra_t7_ms * 1000); + + return result; +} + +bool link_power_alpm_dpcd_enable(struct dc_link *link, bool enable) +{ + bool ret = false; + union dpcd_alpm_configuration alpm_config; + + if (link->psr_settings.psr_version == DC_PSR_VERSION_SU_1) { + memset(&alpm_config, 0, sizeof(alpm_config)); + + alpm_config.bits.ENABLE = (enable ? true : false); + ret = dm_helpers_dp_write_dpcd(link->ctx, link, + DP_RECEIVER_ALPM_CONFIG, &alpm_config.raw, + sizeof(alpm_config.raw)); + } + return ret; +} + +static struct pipe_ctx *get_pipe_from_link(const struct dc_link *link) +{ + int i; + struct dc *dc = link->ctx->dc; + struct pipe_ctx *pipe_ctx = NULL; + + for (i = 0; i < MAX_PIPES; i++) { + if (dc->current_state->res_ctx.pipe_ctx[i].stream) { + if (dc->current_state->res_ctx.pipe_ctx[i].stream->link == link) { + pipe_ctx = &dc->current_state->res_ctx.pipe_ctx[i]; + break; + } + } + } + + return pipe_ctx; +} + +bool dc_link_set_backlight_level(const struct dc_link *link, + uint32_t backlight_pwm_u16_16, + uint32_t frame_ramp) +{ + struct dc *dc = link->ctx->dc; + + DC_LOGGER_INIT(link->ctx->logger); + DC_LOG_BACKLIGHT("New Backlight level: %d (0x%X)\n", + backlight_pwm_u16_16, backlight_pwm_u16_16); + + if (dc_is_embedded_signal(link->connector_signal)) { + struct pipe_ctx *pipe_ctx = get_pipe_from_link(link); + + if (pipe_ctx) { + /* Disable brightness ramping when the display is blanked + * as it can hang the DMCU + */ + if (pipe_ctx->plane_state == NULL) + frame_ramp = 0; + } else { + return false; + } + + dc->hwss.set_backlight_level( + pipe_ctx, + backlight_pwm_u16_16, + frame_ramp); + } + return true; +} + +bool dc_link_set_psr_allow_active(struct dc_link *link, const bool *allow_active, + bool wait, bool force_static, const unsigned int *power_opts) +{ + struct dc *dc = link->ctx->dc; + struct dmcu *dmcu = dc->res_pool->dmcu; + struct dmub_psr *psr = dc->res_pool->psr; + unsigned int panel_inst; + + if (psr == NULL && force_static) + return false; + + if (!dc_get_edp_link_panel_inst(dc, link, &panel_inst)) + return false; + + if ((allow_active != NULL) && (*allow_active == true) && (link->type == dc_connection_none)) { + // Don't enter PSR if panel is not connected + return false; + } + + /* Set power optimization flag */ + if (power_opts && link->psr_settings.psr_power_opt != *power_opts) { + link->psr_settings.psr_power_opt = *power_opts; + + if (psr != NULL && link->psr_settings.psr_feature_enabled && psr->funcs->psr_set_power_opt) + psr->funcs->psr_set_power_opt(psr, link->psr_settings.psr_power_opt, panel_inst); + } + + if (psr != NULL && link->psr_settings.psr_feature_enabled && + force_static && psr->funcs->psr_force_static) + psr->funcs->psr_force_static(psr, panel_inst); + + /* Enable or Disable PSR */ + if (allow_active && link->psr_settings.psr_allow_active != *allow_active) { + link->psr_settings.psr_allow_active = *allow_active; + + if (!link->psr_settings.psr_allow_active) + dc_z10_restore(dc); + + if (psr != NULL && link->psr_settings.psr_feature_enabled) { + psr->funcs->psr_enable(psr, link->psr_settings.psr_allow_active, wait, panel_inst); + } else if ((dmcu != NULL && dmcu->funcs->is_dmcu_initialized(dmcu)) && + link->psr_settings.psr_feature_enabled) + dmcu->funcs->set_psr_enable(dmcu, link->psr_settings.psr_allow_active, wait); + else + return false; + } + return true; +} + +bool dc_link_get_psr_state(const struct dc_link *link, enum dc_psr_state *state) +{ + struct dc *dc = link->ctx->dc; + struct dmcu *dmcu = dc->res_pool->dmcu; + struct dmub_psr *psr = dc->res_pool->psr; + unsigned int panel_inst; + + if (!dc_get_edp_link_panel_inst(dc, link, &panel_inst)) + return false; + + if (psr != NULL && link->psr_settings.psr_feature_enabled) + psr->funcs->psr_get_state(psr, state, panel_inst); + else if (dmcu != NULL && link->psr_settings.psr_feature_enabled) + dmcu->funcs->get_psr_state(dmcu, state); + + return true; +} + +static inline enum physical_phy_id +transmitter_to_phy_id(struct dc_link *link) +{ + struct dc_context *dc_ctx = link->ctx; + enum transmitter transmitter_value = link->link_enc->transmitter; + + switch (transmitter_value) { + case TRANSMITTER_UNIPHY_A: + return PHYLD_0; + case TRANSMITTER_UNIPHY_B: + return PHYLD_1; + case TRANSMITTER_UNIPHY_C: + return PHYLD_2; + case TRANSMITTER_UNIPHY_D: + return PHYLD_3; + case TRANSMITTER_UNIPHY_E: + return PHYLD_4; + case TRANSMITTER_UNIPHY_F: + return PHYLD_5; + case TRANSMITTER_NUTMEG_CRT: + return PHYLD_6; + case TRANSMITTER_TRAVIS_CRT: + return PHYLD_7; + case TRANSMITTER_TRAVIS_LCD: + return PHYLD_8; + case TRANSMITTER_UNIPHY_G: + return PHYLD_9; + case TRANSMITTER_COUNT: + return PHYLD_COUNT; + case TRANSMITTER_UNKNOWN: + return PHYLD_UNKNOWN; + default: + DC_ERROR("Unknown transmitter value %d\n", transmitter_value); + return PHYLD_UNKNOWN; + } +} + +bool dc_link_setup_psr(struct dc_link *link, + const struct dc_stream_state *stream, struct psr_config *psr_config, + struct psr_context *psr_context) +{ + struct dc *dc; + struct dmcu *dmcu; + struct dmub_psr *psr; + int i; + unsigned int panel_inst; + /* updateSinkPsrDpcdConfig*/ + union dpcd_psr_configuration psr_configuration; + union dpcd_sink_active_vtotal_control_mode vtotal_control = {0}; + + psr_context->controllerId = CONTROLLER_ID_UNDEFINED; + + if (!link) + return false; + + dc = link->ctx->dc; + dmcu = dc->res_pool->dmcu; + psr = dc->res_pool->psr; + + if (!dmcu && !psr) + return false; + + if (!dc_get_edp_link_panel_inst(dc, link, &panel_inst)) + return false; + + + memset(&psr_configuration, 0, sizeof(psr_configuration)); + + psr_configuration.bits.ENABLE = 1; + psr_configuration.bits.CRC_VERIFICATION = 1; + psr_configuration.bits.FRAME_CAPTURE_INDICATION = + psr_config->psr_frame_capture_indication_req; + + /* Check for PSR v2*/ + if (link->psr_settings.psr_version == DC_PSR_VERSION_SU_1) { + /* For PSR v2 selective update. + * Indicates whether sink should start capturing + * immediately following active scan line, + * or starting with the 2nd active scan line. + */ + psr_configuration.bits.LINE_CAPTURE_INDICATION = 0; + /*For PSR v2, determines whether Sink should generate + * IRQ_HPD when CRC mismatch is detected. + */ + psr_configuration.bits.IRQ_HPD_WITH_CRC_ERROR = 1; + /* For PSR v2, set the bit when the Source device will + * be enabling PSR2 operation. + */ + psr_configuration.bits.ENABLE_PSR2 = 1; + /* For PSR v2, the Sink device must be able to receive + * SU region updates early in the frame time. + */ + psr_configuration.bits.EARLY_TRANSPORT_ENABLE = 1; + } + + dm_helpers_dp_write_dpcd( + link->ctx, + link, + 368, + &psr_configuration.raw, + sizeof(psr_configuration.raw)); + + if (link->psr_settings.psr_version == DC_PSR_VERSION_SU_1) { + link_power_alpm_dpcd_enable(link, true); + psr_context->su_granularity_required = + psr_config->su_granularity_required; + psr_context->su_y_granularity = + psr_config->su_y_granularity; + psr_context->line_time_in_us = psr_config->line_time_in_us; + + /* linux must be able to expose AMD Source DPCD definition + * in order to support FreeSync PSR + */ + if (link->psr_settings.psr_vtotal_control_support) { + psr_context->rate_control_caps = psr_config->rate_control_caps; + vtotal_control.bits.ENABLE = true; + core_link_write_dpcd(link, DP_SINK_PSR_ACTIVE_VTOTAL_CONTROL_MODE, + &vtotal_control.raw, sizeof(vtotal_control.raw)); + } + } + + psr_context->channel = link->ddc->ddc_pin->hw_info.ddc_channel; + psr_context->transmitterId = link->link_enc->transmitter; + psr_context->engineId = link->link_enc->preferred_engine; + + for (i = 0; i < MAX_PIPES; i++) { + if (dc->current_state->res_ctx.pipe_ctx[i].stream + == stream) { + /* dmcu -1 for all controller id values, + * therefore +1 here + */ + psr_context->controllerId = + dc->current_state->res_ctx. + pipe_ctx[i].stream_res.tg->inst + 1; + break; + } + } + + /* Hardcoded for now. Can be Pcie or Uniphy (or Unknown)*/ + psr_context->phyType = PHY_TYPE_UNIPHY; + /*PhyId is associated with the transmitter id*/ + psr_context->smuPhyId = transmitter_to_phy_id(link); + + psr_context->crtcTimingVerticalTotal = stream->timing.v_total; + psr_context->vsync_rate_hz = div64_u64(div64_u64((stream-> + timing.pix_clk_100hz * 100), + stream->timing.v_total), + stream->timing.h_total); + + psr_context->psrSupportedDisplayConfig = true; + psr_context->psrExitLinkTrainingRequired = + psr_config->psr_exit_link_training_required; + psr_context->sdpTransmitLineNumDeadline = + psr_config->psr_sdp_transmit_line_num_deadline; + psr_context->psrFrameCaptureIndicationReq = + psr_config->psr_frame_capture_indication_req; + + psr_context->skipPsrWaitForPllLock = 0; /* only = 1 in KV */ + + psr_context->numberOfControllers = + link->dc->res_pool->timing_generator_count; + + psr_context->rfb_update_auto_en = true; + + /* 2 frames before enter PSR. */ + psr_context->timehyst_frames = 2; + /* half a frame + * (units in 100 lines, i.e. a value of 1 represents 100 lines) + */ + psr_context->hyst_lines = stream->timing.v_total / 2 / 100; + psr_context->aux_repeats = 10; + + psr_context->psr_level.u32all = 0; + + /*skip power down the single pipe since it blocks the cstate*/ +#if defined(CONFIG_DRM_AMD_DC_DCN) + if (link->ctx->asic_id.chip_family >= FAMILY_RV) { + switch (link->ctx->asic_id.chip_family) { + case FAMILY_YELLOW_CARP: + case AMDGPU_FAMILY_GC_10_3_6: + case AMDGPU_FAMILY_GC_11_0_1: + if (dc->debug.disable_z10 || dc->debug.psr_skip_crtc_disable) + psr_context->psr_level.bits.SKIP_CRTC_DISABLE = true; + break; + default: + psr_context->psr_level.bits.SKIP_CRTC_DISABLE = true; + break; + } + } +#else + if (link->ctx->asic_id.chip_family >= FAMILY_RV) + psr_context->psr_level.bits.SKIP_CRTC_DISABLE = true; +#endif + + /* SMU will perform additional powerdown sequence. + * For unsupported ASICs, set psr_level flag to skip PSR + * static screen notification to SMU. + * (Always set for DAL2, did not check ASIC) + */ + psr_context->allow_smu_optimizations = psr_config->allow_smu_optimizations; + psr_context->allow_multi_disp_optimizations = psr_config->allow_multi_disp_optimizations; + + /* Complete PSR entry before aborting to prevent intermittent + * freezes on certain eDPs + */ + psr_context->psr_level.bits.DISABLE_PSR_ENTRY_ABORT = 1; + + /* Disable ALPM first for compatible non-ALPM panel now */ + psr_context->psr_level.bits.DISABLE_ALPM = 0; + psr_context->psr_level.bits.ALPM_DEFAULT_PD_MODE = 1; + + /* Controls additional delay after remote frame capture before + * continuing power down, default = 0 + */ + psr_context->frame_delay = 0; + + psr_context->dsc_slice_height = psr_config->dsc_slice_height; + + if (psr) { + link->psr_settings.psr_feature_enabled = psr->funcs->psr_copy_settings(psr, + link, psr_context, panel_inst); + link->psr_settings.psr_power_opt = 0; + link->psr_settings.psr_allow_active = 0; + } else { + link->psr_settings.psr_feature_enabled = dmcu->funcs->setup_psr(dmcu, link, psr_context); + } + + /* psr_enabled == 0 indicates setup_psr did not succeed, but this + * should not happen since firmware should be running at this point + */ + if (link->psr_settings.psr_feature_enabled == 0) + ASSERT(0); + + return true; + +} + +void link_get_psr_residency(const struct dc_link *link, uint32_t *residency) +{ + struct dc *dc = link->ctx->dc; + struct dmub_psr *psr = dc->res_pool->psr; + unsigned int panel_inst; + + if (!dc_get_edp_link_panel_inst(dc, link, &panel_inst)) + return; + + // PSR residency measurements only supported on DMCUB + if (psr != NULL && link->psr_settings.psr_feature_enabled) + psr->funcs->psr_get_residency(psr, residency, panel_inst); + else + *residency = 0; +} +bool link_set_sink_vtotal_in_psr_active(const struct dc_link *link, uint16_t psr_vtotal_idle, uint16_t psr_vtotal_su) +{ + struct dc *dc = link->ctx->dc; + struct dmub_psr *psr = dc->res_pool->psr; + + if (psr == NULL || !link->psr_settings.psr_feature_enabled || !link->psr_settings.psr_vtotal_control_support) + return false; + + psr->funcs->psr_set_sink_vtotal_in_psr_active(psr, psr_vtotal_idle, psr_vtotal_su); + + return true; +} + +static struct abm *get_abm_from_stream_res(const struct dc_link *link) +{ + int i; + struct dc *dc = link->ctx->dc; + struct abm *abm = NULL; + + for (i = 0; i < MAX_PIPES; i++) { + struct pipe_ctx pipe_ctx = dc->current_state->res_ctx.pipe_ctx[i]; + struct dc_stream_state *stream = pipe_ctx.stream; + + if (stream && stream->link == link) { + abm = pipe_ctx.stream_res.abm; + break; + } + } + return abm; +} + +int dc_link_get_backlight_level(const struct dc_link *link) +{ + struct abm *abm = get_abm_from_stream_res(link); + struct panel_cntl *panel_cntl = link->panel_cntl; + struct dc *dc = link->ctx->dc; + struct dmcu *dmcu = dc->res_pool->dmcu; + bool fw_set_brightness = true; + + if (dmcu) + fw_set_brightness = dmcu->funcs->is_dmcu_initialized(dmcu); + + if (!fw_set_brightness && panel_cntl->funcs->get_current_backlight) + return panel_cntl->funcs->get_current_backlight(panel_cntl); + else if (abm != NULL && abm->funcs->get_current_backlight != NULL) + return (int) abm->funcs->get_current_backlight(abm); + else + return DC_ERROR_UNEXPECTED; +} + +int dc_link_get_target_backlight_pwm(const struct dc_link *link) +{ + struct abm *abm = get_abm_from_stream_res(link); + + if (abm == NULL || abm->funcs->get_target_backlight == NULL) + return DC_ERROR_UNEXPECTED; + + return (int) abm->funcs->get_target_backlight(abm); +} diff --git a/drivers/gpu/drm/amd/display/dc/link/protocols/link_edp_panel_control.h b/drivers/gpu/drm/amd/display/dc/link/protocols/link_edp_panel_control.h new file mode 100644 index 0000000000000000000000000000000000000000..7f91a564b0893e6d4d6f096765154d735c94f7bf --- /dev/null +++ b/drivers/gpu/drm/amd/display/dc/link/protocols/link_edp_panel_control.h @@ -0,0 +1,33 @@ +/* + * Copyright 2022 Advanced Micro Devices, Inc. + * + * Permission is hereby granted, free of charge, to any person obtaining a + * copy of this software and associated documentation files (the "Software"), + * to deal in the Software without restriction, including without limitation + * the rights to use, copy, modify, merge, publish, distribute, sublicense, + * and/or sell copies of the Software, and to permit persons to whom the + * Software is furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL + * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR + * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, + * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR + * OTHER DEALINGS IN THE SOFTWARE. + * + * Authors: AMD + * + */ + +#ifndef __DC_LINK_EDP_PANEL_CONTROL_H__ +#define __DC_LINK_EDP_PANEL_CONTROL_H__ +#include "link.h" + +enum dp_panel_mode dp_get_panel_mode(struct dc_link *link); +void dp_set_panel_mode(struct dc_link *link, enum dp_panel_mode panel_mode); +bool set_default_brightness_aux(struct dc_link *link); +#endif /* __DC_LINK_EDP_POWER_CONTROL_H__ */ diff --git a/drivers/gpu/drm/amd/display/dc/link/link_hpd.c b/drivers/gpu/drm/amd/display/dc/link/protocols/link_hpd.c similarity index 100% rename from drivers/gpu/drm/amd/display/dc/link/link_hpd.c rename to drivers/gpu/drm/amd/display/dc/link/protocols/link_hpd.c diff --git a/drivers/gpu/drm/amd/display/dc/link/link_hpd.h b/drivers/gpu/drm/amd/display/dc/link/protocols/link_hpd.h similarity index 100% rename from drivers/gpu/drm/amd/display/dc/link/link_hpd.h rename to drivers/gpu/drm/amd/display/dc/link/protocols/link_hpd.h diff --git a/drivers/gpu/drm/amd/display/dmub/inc/dmub_cmd.h b/drivers/gpu/drm/amd/display/dmub/inc/dmub_cmd.h index 06c553b6132212816fd32e800bef6aeed9812a1e..04df407092b1ed42bb5e199c9f9c23f92ced7608 100644 --- a/drivers/gpu/drm/amd/display/dmub/inc/dmub_cmd.h +++ b/drivers/gpu/drm/amd/display/dmub/inc/dmub_cmd.h @@ -1968,6 +1968,14 @@ struct dmub_cmd_psr_copy_settings_data { * Explicit padding to 2 byte boundary. */ uint8_t pad3; + /** + * DSC Slice height. + */ + uint16_t dsc_slice_height; + /** + * Explicit padding to 4 byte boundary. + */ + uint16_t pad; }; /** @@ -3112,7 +3120,7 @@ struct dmub_rb_cmd_panel_cntl { struct dmub_cmd_lvtma_control_data { uint8_t uc_pwr_action; /**< LVTMA_ACTION */ uint8_t bypass_panel_control_wait; - uint8_t reserved_0[2]; + uint8_t reserved_0[2]; /**< For future use */ uint8_t panel_inst; /**< LVTMA control instance */ uint8_t reserved_1[3]; /**< For future use */ }; diff --git a/drivers/gpu/drm/amd/display/modules/inc/mod_info_packet.h b/drivers/gpu/drm/amd/display/modules/inc/mod_info_packet.h index edf5845f6a1f77082ac3dc62b00265d2f49f89bb..91fe039c0c95981109a0651909f3221f7da0ea90 100644 --- a/drivers/gpu/drm/amd/display/modules/inc/mod_info_packet.h +++ b/drivers/gpu/drm/amd/display/modules/inc/mod_info_packet.h @@ -41,4 +41,40 @@ void mod_build_vsc_infopacket(const struct dc_stream_state *stream, void mod_build_hf_vsif_infopacket(const struct dc_stream_state *stream, struct dc_info_packet *info_packet); +enum adaptive_sync_type { + ADAPTIVE_SYNC_TYPE_NONE = 0, + ADAPTIVE_SYNC_TYPE_DP = 1, + ADAPTIVE_SYNC_TYPE_PCON_IN_WHITELIST = 2, + ADAPTIVE_SYNC_TYPE_PCON_NOT_IN_WHITELIST = 3, + ADAPTIVE_SYNC_TYPE_EDP = 4, +}; + +enum adaptive_sync_sdp_version { + AS_SDP_VER_0 = 0x0, + AS_SDP_VER_1 = 0x1, + AS_SDP_VER_2 = 0x2, +}; + +#define AS_DP_SDP_LENGTH (9) + +struct frame_duration_op { + bool support; + unsigned char frame_duration_hex; +}; + +struct AS_Df_params { + bool supportMode; + struct frame_duration_op increase; + struct frame_duration_op decrease; +}; + +void mod_build_adaptive_sync_infopacket(const struct dc_stream_state *stream, + enum adaptive_sync_type asType, const struct AS_Df_params *param, + struct dc_info_packet *info_packet); + +void mod_build_adaptive_sync_infopacket_v2(const struct dc_stream_state *stream, + const struct AS_Df_params *param, struct dc_info_packet *info_packet); + +void mod_build_adaptive_sync_infopacket_v1(struct dc_info_packet *info_packet); + #endif diff --git a/drivers/gpu/drm/amd/display/modules/info_packet/info_packet.c b/drivers/gpu/drm/amd/display/modules/info_packet/info_packet.c index 69691058ab89817c6d7edf50fc47c49cd9fcbcbd..a8a31d0a7a7fec66a29fcac9b5e45625af357655 100644 --- a/drivers/gpu/drm/amd/display/modules/info_packet/info_packet.c +++ b/drivers/gpu/drm/amd/display/modules/info_packet/info_packet.c @@ -519,3 +519,58 @@ void mod_build_hf_vsif_infopacket(const struct dc_stream_state *stream, info_packet->valid = true; } +void mod_build_adaptive_sync_infopacket(const struct dc_stream_state *stream, + enum adaptive_sync_type asType, + const struct AS_Df_params *param, + struct dc_info_packet *info_packet) +{ + info_packet->valid = false; + + memset(info_packet, 0, sizeof(struct dc_info_packet)); + + switch (asType) { + case ADAPTIVE_SYNC_TYPE_DP: + if (stream != NULL) + mod_build_adaptive_sync_infopacket_v2(stream, param, info_packet); + break; + case ADAPTIVE_SYNC_TYPE_PCON_IN_WHITELIST: + mod_build_adaptive_sync_infopacket_v1(info_packet); + break; + case ADAPTIVE_SYNC_TYPE_NONE: + case ADAPTIVE_SYNC_TYPE_PCON_NOT_IN_WHITELIST: + default: + break; + } +} + +void mod_build_adaptive_sync_infopacket_v1(struct dc_info_packet *info_packet) +{ + info_packet->valid = true; + // HEADER {HB0, HB1, HB2, HB3} = {00, Type, Version, Length} + info_packet->hb0 = 0x00; + info_packet->hb1 = 0x22; + info_packet->hb2 = AS_SDP_VER_1; + info_packet->hb3 = 0x00; +} + +void mod_build_adaptive_sync_infopacket_v2(const struct dc_stream_state *stream, + const struct AS_Df_params *param, + struct dc_info_packet *info_packet) +{ + info_packet->valid = true; + // HEADER {HB0, HB1, HB2, HB3} = {00, Type, Version, Length} + info_packet->hb0 = 0x00; + info_packet->hb1 = 0x22; + info_packet->hb2 = AS_SDP_VER_2; + info_packet->hb3 = AS_DP_SDP_LENGTH; + + //Payload + info_packet->sb[0] = param->supportMode; //1: AVT; 0: FAVT + info_packet->sb[1] = (stream->timing.v_total & 0x00FF); + info_packet->sb[2] = (stream->timing.v_total & 0xFF00) >> 8; + //info_packet->sb[3] = 0x00; Target RR, not use fot AVT + info_packet->sb[4] = (param->increase.support << 6 | param->decrease.support << 7); + info_packet->sb[5] = param->increase.frame_duration_hex; + info_packet->sb[6] = param->decrease.frame_duration_hex; +} + diff --git a/drivers/gpu/drm/amd/display/modules/power/power_helpers.c b/drivers/gpu/drm/amd/display/modules/power/power_helpers.c index cf4fa87c7db60a1b1c4e61eea8a144b9035239ba..e39b133d05af4e47f17c8993080160726c8ab3f6 100644 --- a/drivers/gpu/drm/amd/display/modules/power/power_helpers.c +++ b/drivers/gpu/drm/amd/display/modules/power/power_helpers.c @@ -917,13 +917,14 @@ bool mod_power_only_edp(const struct dc_state *context, const struct dc_stream_s return context && context->stream_count == 1 && dc_is_embedded_signal(stream->signal); } -bool psr_su_set_y_granularity(struct dc *dc, struct dc_link *link, +bool psr_su_set_dsc_slice_height(struct dc *dc, struct dc_link *link, struct dc_stream_state *stream, struct psr_config *config) { uint16_t pic_height; - uint8_t slice_height; + uint16_t slice_height; + config->dsc_slice_height = 0; if ((link->connector_signal & SIGNAL_TYPE_EDP) && (!dc->caps.edp_dsc_support || link->panel_config.dsc.disable_dsc_edp || @@ -934,6 +935,7 @@ bool psr_su_set_y_granularity(struct dc *dc, struct dc_link *link, pic_height = stream->timing.v_addressable + stream->timing.v_border_top + stream->timing.v_border_bottom; slice_height = pic_height / stream->timing.dsc_cfg.num_slices_v; + config->dsc_slice_height = slice_height; if (slice_height) { if (config->su_y_granularity && @@ -941,8 +943,6 @@ bool psr_su_set_y_granularity(struct dc *dc, struct dc_link *link, ASSERT(0); return false; } - - config->su_y_granularity = slice_height; } return true; diff --git a/drivers/gpu/drm/amd/display/modules/power/power_helpers.h b/drivers/gpu/drm/amd/display/modules/power/power_helpers.h index bb16b37b83da75fced224d041436b2a618dfc028..1d3079e56799f3daf4c9d119873a036d52f93e13 100644 --- a/drivers/gpu/drm/amd/display/modules/power/power_helpers.h +++ b/drivers/gpu/drm/amd/display/modules/power/power_helpers.h @@ -59,7 +59,7 @@ void mod_power_calc_psr_configs(struct psr_config *psr_config, const struct dc_stream_state *stream); bool mod_power_only_edp(const struct dc_state *context, const struct dc_stream_state *stream); -bool psr_su_set_y_granularity(struct dc *dc, struct dc_link *link, +bool psr_su_set_dsc_slice_height(struct dc *dc, struct dc_link *link, struct dc_stream_state *stream, struct psr_config *config); #endif /* MODULES_POWER_POWER_HELPERS_H_ */ diff --git a/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_0_ppt.c b/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_0_ppt.c index d0cdc578344d8d2af25c0bc1b10de275a8d2d9ed..f89ce6575b15c45564f1ba4a535856f733bdd564 100644 --- a/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_0_ppt.c +++ b/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_0_ppt.c @@ -145,6 +145,7 @@ static struct cmn2asic_msg_mapping smu_v13_0_0_message_map[SMU_MSG_MAX_COUNT] = MSG_MAP(SetBadMemoryPagesRetiredFlagsPerChannel, PPSMC_MSG_SetBadMemoryPagesRetiredFlagsPerChannel, 0), MSG_MAP(AllowGpo, PPSMC_MSG_SetGpoAllow, 0), + MSG_MAP(AllowIHHostInterrupt, PPSMC_MSG_AllowIHHostInterrupt, 0), }; static struct cmn2asic_mapping smu_v13_0_0_clk_map[SMU_CLK_COUNT] = { diff --git a/drivers/gpu/drm/display/drm_dp_mst_topology.c b/drivers/gpu/drm/display/drm_dp_mst_topology.c index 5861b0a6247bc1bb10d5ae4dda45f27f537750d8..847c10aa2098ceb279dfae8b61e87281a102d14a 100644 --- a/drivers/gpu/drm/display/drm_dp_mst_topology.c +++ b/drivers/gpu/drm/display/drm_dp_mst_topology.c @@ -3377,6 +3377,9 @@ void drm_dp_remove_payload(struct drm_dp_mst_topology_mgr *mgr, mgr->payload_count--; mgr->next_start_slot -= payload->time_slots; + + if (payload->delete) + drm_dp_mst_put_port_malloc(payload->port); } EXPORT_SYMBOL(drm_dp_remove_payload); @@ -4335,7 +4338,6 @@ int drm_dp_atomic_release_time_slots(struct drm_atomic_state *state, drm_dbg_atomic(mgr->dev, "[MST PORT:%p] TU %d -> 0\n", port, payload->time_slots); if (!payload->delete) { - drm_dp_mst_put_port_malloc(port); payload->pbn = 0; payload->delete = true; topology_state->payload_mask &= ~BIT(payload->vcpi - 1); diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c index 11bb5939947180ef922049cd16470ccdd7bacef5..3d1f50f481cfda6d85845c47a95d2218028f6c51 100644 --- a/drivers/gpu/drm/drm_buddy.c +++ b/drivers/gpu/drm/drm_buddy.c @@ -38,6 +38,25 @@ static void drm_block_free(struct drm_buddy *mm, kmem_cache_free(slab_blocks, block); } +static void list_insert_sorted(struct drm_buddy *mm, + struct drm_buddy_block *block) +{ + struct drm_buddy_block *node; + struct list_head *head; + + head = &mm->free_list[drm_buddy_block_order(block)]; + if (list_empty(head)) { + list_add(&block->link, head); + return; + } + + list_for_each_entry(node, head, link) + if (drm_buddy_block_offset(block) < drm_buddy_block_offset(node)) + break; + + __list_add(&block->link, node->link.prev, &node->link); +} + static void mark_allocated(struct drm_buddy_block *block) { block->header &= ~DRM_BUDDY_HEADER_STATE; @@ -52,8 +71,7 @@ static void mark_free(struct drm_buddy *mm, block->header &= ~DRM_BUDDY_HEADER_STATE; block->header |= DRM_BUDDY_FREE; - list_add(&block->link, - &mm->free_list[drm_buddy_block_order(block)]); + list_insert_sorted(mm, block); } static void mark_split(struct drm_buddy_block *block) @@ -387,20 +405,26 @@ alloc_range_bias(struct drm_buddy *mm, } static struct drm_buddy_block * -get_maxblock(struct list_head *head) +get_maxblock(struct drm_buddy *mm, unsigned int order) { struct drm_buddy_block *max_block = NULL, *node; + unsigned int i; - max_block = list_first_entry_or_null(head, - struct drm_buddy_block, - link); - if (!max_block) - return NULL; + for (i = order; i <= mm->max_order; ++i) { + if (!list_empty(&mm->free_list[i])) { + node = list_last_entry(&mm->free_list[i], + struct drm_buddy_block, + link); + if (!max_block) { + max_block = node; + continue; + } - list_for_each_entry(node, head, link) { - if (drm_buddy_block_offset(node) > - drm_buddy_block_offset(max_block)) - max_block = node; + if (drm_buddy_block_offset(node) > + drm_buddy_block_offset(max_block)) { + max_block = node; + } + } } return max_block; @@ -412,20 +436,23 @@ alloc_from_freelist(struct drm_buddy *mm, unsigned long flags) { struct drm_buddy_block *block = NULL; - unsigned int i; + unsigned int tmp; int err; - for (i = order; i <= mm->max_order; ++i) { - if (flags & DRM_BUDDY_TOPDOWN_ALLOCATION) { - block = get_maxblock(&mm->free_list[i]); - if (block) - break; - } else { - block = list_first_entry_or_null(&mm->free_list[i], - struct drm_buddy_block, - link); - if (block) - break; + if (flags & DRM_BUDDY_TOPDOWN_ALLOCATION) { + block = get_maxblock(mm, order); + if (block) + /* Store the obtained block order */ + tmp = drm_buddy_block_order(block); + } else { + for (tmp = order; tmp <= mm->max_order; ++tmp) { + if (!list_empty(&mm->free_list[tmp])) { + block = list_last_entry(&mm->free_list[tmp], + struct drm_buddy_block, + link); + if (block) + break; + } } } @@ -434,18 +461,18 @@ alloc_from_freelist(struct drm_buddy *mm, BUG_ON(!drm_buddy_block_is_free(block)); - while (i != order) { + while (tmp != order) { err = split_block(mm, block); if (unlikely(err)) goto err_undo; block = block->right; - i--; + tmp--; } return block; err_undo: - if (i != order) + if (tmp != order) __drm_buddy_free(mm, block); return ERR_PTR(err); } diff --git a/drivers/gpu/drm/drm_fb_helper.c b/drivers/gpu/drm/drm_fb_helper.c index 97b34f377fe8ae6c272117346304fe24374f81a9..28c428e9c530896c19a37c41bc864c94f3cb4d84 100644 --- a/drivers/gpu/drm/drm_fb_helper.c +++ b/drivers/gpu/drm/drm_fb_helper.c @@ -30,7 +30,9 @@ #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt #include +#include #include +#include #include #include @@ -1952,6 +1954,7 @@ static int drm_fb_helper_find_sizes(struct drm_fb_helper *fb_helper, static int drm_fb_helper_single_fb_probe(struct drm_fb_helper *fb_helper) { struct drm_client_dev *client = &fb_helper->client; + struct drm_device *dev = fb_helper->dev; struct drm_fb_helper_surface_size sizes; int ret; @@ -1973,6 +1976,11 @@ static int drm_fb_helper_single_fb_probe(struct drm_fb_helper *fb_helper) return ret; strcpy(fb_helper->fb->comm, "[fbcon]"); + + /* Set the fb info for vgaswitcheroo clients. Does nothing otherwise. */ + if (dev_is_pci(dev->dev)) + vga_switcheroo_client_fb_set(to_pci_dev(dev->dev), fb_helper->info); + return 0; } diff --git a/drivers/gpu/drm/drm_fbdev_generic.c b/drivers/gpu/drm/drm_fbdev_generic.c index 43f94aa9e01525a70f45f54b0577e52e2c4f4e92..365f80717fa130412d6ecf67c06b05d5fdfc87ef 100644 --- a/drivers/gpu/drm/drm_fbdev_generic.c +++ b/drivers/gpu/drm/drm_fbdev_generic.c @@ -155,11 +155,6 @@ static const struct fb_ops drm_fbdev_fb_ops = { .fb_imageblit = drm_fbdev_fb_imageblit, }; -static struct fb_deferred_io drm_fbdev_defio = { - .delay = HZ / 20, - .deferred_io = drm_fb_helper_deferred_io, -}; - /* * This function uses the client API to create a framebuffer backed by a dumb buffer. */ @@ -206,8 +201,14 @@ static int drm_fbdev_fb_probe(struct drm_fb_helper *fb_helper, return -ENOMEM; info->flags |= FBINFO_VIRTFB | FBINFO_READS_FAST; - info->fbdefio = &drm_fbdev_defio; - fb_deferred_io_init(info); + /* Set a default deferred I/O handler */ + fb_helper->fbdefio.delay = HZ / 20; + fb_helper->fbdefio.deferred_io = drm_fb_helper_deferred_io; + + info->fbdefio = &fb_helper->fbdefio; + ret = fb_deferred_io_init(info); + if (ret) + return ret; } else { /* buffer is mapped for HW framebuffer */ ret = drm_client_buffer_vmap(fb_helper->buffer, &map); diff --git a/drivers/gpu/drm/drm_panel_orientation_quirks.c b/drivers/gpu/drm/drm_panel_orientation_quirks.c index ca531dbb749d143945fa4eae34563919bef8e438..b409fe256fd0a991236d1c15668ba21eed5d6f43 100644 --- a/drivers/gpu/drm/drm_panel_orientation_quirks.c +++ b/drivers/gpu/drm/drm_panel_orientation_quirks.c @@ -316,6 +316,12 @@ static const struct dmi_system_id orientation_data[] = { DMI_EXACT_MATCH(DMI_PRODUCT_VERSION, "Lenovo ideapad D330-10IGM"), }, .driver_data = (void *)&lcd1200x1920_rightside_up, + }, { /* Lenovo Ideapad D330-10IGL (HD) */ + .matches = { + DMI_EXACT_MATCH(DMI_SYS_VENDOR, "LENOVO"), + DMI_EXACT_MATCH(DMI_PRODUCT_VERSION, "Lenovo ideapad D330-10IGL"), + }, + .driver_data = (void *)&lcd800x1280_rightside_up, }, { /* Lenovo Yoga Book X90F / X91F / X91L */ .matches = { /* Non exact match to match all versions */ diff --git a/drivers/gpu/drm/drm_vma_manager.c b/drivers/gpu/drm/drm_vma_manager.c index 7de37f8c68fd0c76cb936c67f93152209cf4cb33..83229a031af0f829691934c8dae72d3ba6686853 100644 --- a/drivers/gpu/drm/drm_vma_manager.c +++ b/drivers/gpu/drm/drm_vma_manager.c @@ -240,27 +240,8 @@ void drm_vma_offset_remove(struct drm_vma_offset_manager *mgr, } EXPORT_SYMBOL(drm_vma_offset_remove); -/** - * drm_vma_node_allow - Add open-file to list of allowed users - * @node: Node to modify - * @tag: Tag of file to remove - * - * Add @tag to the list of allowed open-files for this node. If @tag is - * already on this list, the ref-count is incremented. - * - * The list of allowed-users is preserved across drm_vma_offset_add() and - * drm_vma_offset_remove() calls. You may even call it if the node is currently - * not added to any offset-manager. - * - * You must remove all open-files the same number of times as you added them - * before destroying the node. Otherwise, you will leak memory. - * - * This is locked against concurrent access internally. - * - * RETURNS: - * 0 on success, negative error code on internal failure (out-of-mem) - */ -int drm_vma_node_allow(struct drm_vma_offset_node *node, struct drm_file *tag) +static int vma_node_allow(struct drm_vma_offset_node *node, + struct drm_file *tag, bool ref_counted) { struct rb_node **iter; struct rb_node *parent = NULL; @@ -282,7 +263,8 @@ int drm_vma_node_allow(struct drm_vma_offset_node *node, struct drm_file *tag) entry = rb_entry(*iter, struct drm_vma_offset_file, vm_rb); if (tag == entry->vm_tag) { - entry->vm_count++; + if (ref_counted) + entry->vm_count++; goto unlock; } else if (tag > entry->vm_tag) { iter = &(*iter)->rb_right; @@ -307,8 +289,58 @@ int drm_vma_node_allow(struct drm_vma_offset_node *node, struct drm_file *tag) kfree(new); return ret; } + +/** + * drm_vma_node_allow - Add open-file to list of allowed users + * @node: Node to modify + * @tag: Tag of file to remove + * + * Add @tag to the list of allowed open-files for this node. If @tag is + * already on this list, the ref-count is incremented. + * + * The list of allowed-users is preserved across drm_vma_offset_add() and + * drm_vma_offset_remove() calls. You may even call it if the node is currently + * not added to any offset-manager. + * + * You must remove all open-files the same number of times as you added them + * before destroying the node. Otherwise, you will leak memory. + * + * This is locked against concurrent access internally. + * + * RETURNS: + * 0 on success, negative error code on internal failure (out-of-mem) + */ +int drm_vma_node_allow(struct drm_vma_offset_node *node, struct drm_file *tag) +{ + return vma_node_allow(node, tag, true); +} EXPORT_SYMBOL(drm_vma_node_allow); +/** + * drm_vma_node_allow_once - Add open-file to list of allowed users + * @node: Node to modify + * @tag: Tag of file to remove + * + * Add @tag to the list of allowed open-files for this node. + * + * The list of allowed-users is preserved across drm_vma_offset_add() and + * drm_vma_offset_remove() calls. You may even call it if the node is currently + * not added to any offset-manager. + * + * This is not ref-counted unlike drm_vma_node_allow() hence drm_vma_node_revoke() + * should only be called once after this. + * + * This is locked against concurrent access internally. + * + * RETURNS: + * 0 on success, negative error code on internal failure (out-of-mem) + */ +int drm_vma_node_allow_once(struct drm_vma_offset_node *node, struct drm_file *tag) +{ + return vma_node_allow(node, tag, false); +} +EXPORT_SYMBOL(drm_vma_node_allow_once); + /** * drm_vma_node_revoke - Remove open-file from list of allowed users * @node: Node to modify diff --git a/drivers/gpu/drm/i915/Makefile b/drivers/gpu/drm/i915/Makefile index f47f00b162a484cc1deebd44812b2a7fc288015e..918470a045915dee109fad640d3fff0839f81307 100644 --- a/drivers/gpu/drm/i915/Makefile +++ b/drivers/gpu/drm/i915/Makefile @@ -63,6 +63,7 @@ i915-y += i915_driver.o \ # core peripheral code i915-y += \ soc/intel_dram.o \ + soc/intel_gmch.o \ soc/intel_pch.o # core library code @@ -266,6 +267,7 @@ i915-y += \ display/intel_quirks.o \ display/intel_sprite.o \ display/intel_tc.o \ + display/intel_vblank.o \ display/intel_vga.o \ display/i9xx_plane.o \ display/skl_scaler.o \ diff --git a/drivers/gpu/drm/i915/display/g4x_dp.c b/drivers/gpu/drm/i915/display/g4x_dp.c index 24ef36ec2d3d3ca69b7f48c8f6506ef4c12c9511..fa754038d669200757c625632e67d3949f7b4a83 100644 --- a/drivers/gpu/drm/i915/display/g4x_dp.c +++ b/drivers/gpu/drm/i915/display/g4x_dp.c @@ -398,6 +398,8 @@ static void intel_dp_get_config(struct intel_encoder *encoder, if (intel_dp_is_edp(intel_dp)) intel_edp_fixup_vbt_bpp(encoder, pipe_config->pipe_bpp); + + intel_audio_codec_get_config(encoder, pipe_config); } static void diff --git a/drivers/gpu/drm/i915/display/g4x_hdmi.c b/drivers/gpu/drm/i915/display/g4x_hdmi.c index c3580d96765c6c27c7d50dad316adcac822e541e..64c3b39907028499994fc5c15c039c390686f137 100644 --- a/drivers/gpu/drm/i915/display/g4x_hdmi.c +++ b/drivers/gpu/drm/i915/display/g4x_hdmi.c @@ -155,6 +155,8 @@ static void intel_hdmi_get_config(struct intel_encoder *encoder, intel_read_infoframe(encoder, pipe_config, HDMI_INFOFRAME_TYPE_VENDOR, &pipe_config->infoframes.hdmi); + + intel_audio_codec_get_config(encoder, pipe_config); } static void g4x_hdmi_enable_port(struct intel_encoder *encoder, diff --git a/drivers/gpu/drm/i915/display/icl_dsi.c b/drivers/gpu/drm/i915/display/icl_dsi.c index ae14c794c4bc09f8eeb63edb3c478f0bf9781271..d56d01f07bb716468b0420b493d92372b4ce58e4 100644 --- a/drivers/gpu/drm/i915/display/icl_dsi.c +++ b/drivers/gpu/drm/i915/display/icl_dsi.c @@ -2054,7 +2054,7 @@ void icl_dsi_init(struct drm_i915_private *dev_priv) goto err; } - intel_panel_init(intel_connector); + intel_panel_init(intel_connector, NULL); intel_backlight_setup(intel_connector, INVALID_PIPE); diff --git a/drivers/gpu/drm/i915/display/intel_atomic_plane.c b/drivers/gpu/drm/i915/display/intel_atomic_plane.c index 10e1fc9d069827c7a34fad44bea0e4c3deaf2972..1409bcfb6fd3d99d8636d190b73dce9918a58c83 100644 --- a/drivers/gpu/drm/i915/display/intel_atomic_plane.c +++ b/drivers/gpu/drm/i915/display/intel_atomic_plane.c @@ -36,6 +36,7 @@ #include "gt/intel_rps.h" +#include "i915_config.h" #include "intel_atomic_plane.h" #include "intel_cdclk.h" #include "intel_display_trace.h" diff --git a/drivers/gpu/drm/i915/display/intel_audio.c b/drivers/gpu/drm/i915/display/intel_audio.c index 626c47e96a6d227df6c55cc736071afb9e412115..a9335c856644f05ae3d07739d0ec91aed75a67c0 100644 --- a/drivers/gpu/drm/i915/display/intel_audio.c +++ b/drivers/gpu/drm/i915/display/intel_audio.c @@ -71,6 +71,8 @@ struct intel_audio_funcs { void (*audio_codec_disable)(struct intel_encoder *encoder, const struct intel_crtc_state *old_crtc_state, const struct drm_connector_state *old_conn_state); + void (*audio_codec_get_config)(struct intel_encoder *encoder, + struct intel_crtc_state *crtc_state); }; /* DP N/M table */ @@ -314,6 +316,27 @@ static int g4x_eld_buffer_size(struct drm_i915_private *i915) return REG_FIELD_GET(G4X_ELD_BUFFER_SIZE_MASK, tmp); } +static void g4x_audio_codec_get_config(struct intel_encoder *encoder, + struct intel_crtc_state *crtc_state) +{ + struct drm_i915_private *i915 = to_i915(encoder->base.dev); + u32 *eld = (u32 *)crtc_state->eld; + int eld_buffer_size, len, i; + u32 tmp; + + tmp = intel_de_read(i915, G4X_AUD_CNTL_ST); + if ((tmp & G4X_ELD_VALID) == 0) + return; + + intel_de_rmw(i915, G4X_AUD_CNTL_ST, G4X_ELD_ADDRESS_MASK, 0); + + eld_buffer_size = g4x_eld_buffer_size(i915); + len = min_t(int, sizeof(crtc_state->eld) / 4, eld_buffer_size); + + for (i = 0; i < len; i++) + eld[i] = intel_de_read(i915, G4X_HDMIW_HDMIEDID); +} + static void g4x_audio_codec_disable(struct intel_encoder *encoder, const struct intel_crtc_state *old_crtc_state, const struct drm_connector_state *old_conn_state) @@ -335,8 +358,7 @@ static void g4x_audio_codec_enable(struct intel_encoder *encoder, { struct drm_i915_private *i915 = to_i915(encoder->base.dev); struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc); - struct drm_connector *connector = conn_state->connector; - const u32 *eld = (const u32 *)connector->eld; + const u32 *eld = (const u32 *)crtc_state->eld; int eld_buffer_size, len, i; intel_crtc_wait_for_next_vblank(crtc); @@ -345,7 +367,7 @@ static void g4x_audio_codec_enable(struct intel_encoder *encoder, G4X_ELD_VALID | G4X_ELD_ADDRESS_MASK, 0); eld_buffer_size = g4x_eld_buffer_size(i915); - len = min(drm_eld_size(connector->eld) / 4, eld_buffer_size); + len = min(drm_eld_size(crtc_state->eld) / 4, eld_buffer_size); for (i = 0; i < len; i++) intel_de_write(i915, G4X_HDMIW_HDMIEDID, eld[i]); @@ -459,17 +481,6 @@ hsw_audio_config_update(struct intel_encoder *encoder, hsw_hdmi_audio_config_update(encoder, crtc_state); } -/* ELD buffer size in dwords */ -static int hsw_eld_buffer_size(struct drm_i915_private *i915, - enum transcoder cpu_transcoder) -{ - u32 tmp; - - tmp = intel_de_read(i915, HSW_AUD_DIP_ELD_CTRL(cpu_transcoder)); - - return REG_FIELD_GET(IBX_ELD_BUFFER_SIZE_MASK, tmp); -} - static void hsw_audio_codec_disable(struct intel_encoder *encoder, const struct intel_crtc_state *old_crtc_state, const struct drm_connector_state *old_conn_state) @@ -618,10 +629,7 @@ static void hsw_audio_codec_enable(struct intel_encoder *encoder, { struct drm_i915_private *i915 = to_i915(encoder->base.dev); struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc); - struct drm_connector *connector = conn_state->connector; enum transcoder cpu_transcoder = crtc_state->cpu_transcoder; - const u32 *eld = (const u32 *)connector->eld; - int eld_buffer_size, len, i; mutex_lock(&i915->display.audio.mutex); @@ -639,25 +647,10 @@ static void hsw_audio_codec_enable(struct intel_encoder *encoder, intel_de_rmw(i915, HSW_AUD_PIN_ELD_CP_VLD, AUDIO_ELD_VALID(cpu_transcoder), 0); - /* Reset ELD address */ - intel_de_rmw(i915, HSW_AUD_DIP_ELD_CTRL(cpu_transcoder), - IBX_ELD_ADDRESS_MASK, 0); - - eld_buffer_size = hsw_eld_buffer_size(i915, cpu_transcoder); - len = min(drm_eld_size(connector->eld) / 4, eld_buffer_size); - - for (i = 0; i < len; i++) - intel_de_write(i915, HSW_AUD_EDID_DATA(cpu_transcoder), eld[i]); - for (; i < eld_buffer_size; i++) - intel_de_write(i915, HSW_AUD_EDID_DATA(cpu_transcoder), 0); - - drm_WARN_ON(&i915->drm, - (intel_de_read(i915, HSW_AUD_DIP_ELD_CTRL(cpu_transcoder)) & - IBX_ELD_ADDRESS_MASK) != 0); - - /* ELD valid */ - intel_de_rmw(i915, HSW_AUD_PIN_ELD_CP_VLD, - 0, AUDIO_ELD_VALID(cpu_transcoder)); + /* + * The audio componenent is used to convey the ELD + * instead using of the hardware ELD buffer. + */ /* Enable timestamps */ hsw_audio_config_update(encoder, crtc_state); @@ -665,47 +658,33 @@ static void hsw_audio_codec_enable(struct intel_encoder *encoder, mutex_unlock(&i915->display.audio.mutex); } -struct ilk_audio_regs { +struct ibx_audio_regs { i915_reg_t hdmiw_hdmiedid, aud_config, aud_cntl_st, aud_cntrl_st2; }; -static void ilk_audio_regs_init(struct drm_i915_private *i915, +static void ibx_audio_regs_init(struct drm_i915_private *i915, enum pipe pipe, - struct ilk_audio_regs *regs) + struct ibx_audio_regs *regs) { - if (HAS_PCH_IBX(i915)) { - regs->hdmiw_hdmiedid = IBX_HDMIW_HDMIEDID(pipe); - regs->aud_config = IBX_AUD_CFG(pipe); - regs->aud_cntl_st = IBX_AUD_CNTL_ST(pipe); - regs->aud_cntrl_st2 = IBX_AUD_CNTL_ST2; - } else if (IS_VALLEYVIEW(i915) || IS_CHERRYVIEW(i915)) { + if (IS_VALLEYVIEW(i915) || IS_CHERRYVIEW(i915)) { regs->hdmiw_hdmiedid = VLV_HDMIW_HDMIEDID(pipe); regs->aud_config = VLV_AUD_CFG(pipe); regs->aud_cntl_st = VLV_AUD_CNTL_ST(pipe); regs->aud_cntrl_st2 = VLV_AUD_CNTL_ST2; - } else { + } else if (HAS_PCH_CPT(i915)) { regs->hdmiw_hdmiedid = CPT_HDMIW_HDMIEDID(pipe); regs->aud_config = CPT_AUD_CFG(pipe); regs->aud_cntl_st = CPT_AUD_CNTL_ST(pipe); regs->aud_cntrl_st2 = CPT_AUD_CNTRL_ST2; + } else if (HAS_PCH_IBX(i915)) { + regs->hdmiw_hdmiedid = IBX_HDMIW_HDMIEDID(pipe); + regs->aud_config = IBX_AUD_CFG(pipe); + regs->aud_cntl_st = IBX_AUD_CNTL_ST(pipe); + regs->aud_cntrl_st2 = IBX_AUD_CNTL_ST2; } } -/* ELD buffer size in dwords */ -static int ilk_eld_buffer_size(struct drm_i915_private *i915, - enum pipe pipe) -{ - struct ilk_audio_regs regs; - u32 tmp; - - ilk_audio_regs_init(i915, pipe, ®s); - - tmp = intel_de_read(i915, regs.aud_cntl_st); - - return REG_FIELD_GET(IBX_ELD_BUFFER_SIZE_MASK, tmp); -} - -static void ilk_audio_codec_disable(struct intel_encoder *encoder, +static void ibx_audio_codec_disable(struct intel_encoder *encoder, const struct intel_crtc_state *old_crtc_state, const struct drm_connector_state *old_conn_state) { @@ -713,12 +692,12 @@ static void ilk_audio_codec_disable(struct intel_encoder *encoder, struct intel_crtc *crtc = to_intel_crtc(old_crtc_state->uapi.crtc); enum port port = encoder->port; enum pipe pipe = crtc->pipe; - struct ilk_audio_regs regs; + struct ibx_audio_regs regs; if (drm_WARN_ON(&i915->drm, port == PORT_A)) return; - ilk_audio_regs_init(i915, pipe, ®s); + ibx_audio_regs_init(i915, pipe, ®s); mutex_lock(&i915->display.audio.mutex); @@ -741,25 +720,22 @@ static void ilk_audio_codec_disable(struct intel_encoder *encoder, intel_crtc_wait_for_next_vblank(crtc); } -static void ilk_audio_codec_enable(struct intel_encoder *encoder, +static void ibx_audio_codec_enable(struct intel_encoder *encoder, const struct intel_crtc_state *crtc_state, const struct drm_connector_state *conn_state) { struct drm_i915_private *i915 = to_i915(encoder->base.dev); struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc); - struct drm_connector *connector = conn_state->connector; - const u32 *eld = (const u32 *)connector->eld; enum port port = encoder->port; enum pipe pipe = crtc->pipe; - int eld_buffer_size, len, i; - struct ilk_audio_regs regs; + struct ibx_audio_regs regs; if (drm_WARN_ON(&i915->drm, port == PORT_A)) return; intel_crtc_wait_for_next_vblank(crtc); - ilk_audio_regs_init(i915, pipe, ®s); + ibx_audio_regs_init(i915, pipe, ®s); mutex_lock(&i915->display.audio.mutex); @@ -767,24 +743,10 @@ static void ilk_audio_codec_enable(struct intel_encoder *encoder, intel_de_rmw(i915, regs.aud_cntrl_st2, IBX_ELD_VALID(port), 0); - /* Reset ELD address */ - intel_de_rmw(i915, regs.aud_cntl_st, - IBX_ELD_ADDRESS_MASK, 0); - - eld_buffer_size = ilk_eld_buffer_size(i915, pipe); - len = min(drm_eld_size(connector->eld) / 4, eld_buffer_size); - - for (i = 0; i < len; i++) - intel_de_write(i915, regs.hdmiw_hdmiedid, eld[i]); - for (; i < eld_buffer_size; i++) - intel_de_write(i915, regs.hdmiw_hdmiedid, 0); - - drm_WARN_ON(&i915->drm, - (intel_de_read(i915, regs.aud_cntl_st) & IBX_ELD_ADDRESS_MASK) != 0); - - /* ELD valid */ - intel_de_rmw(i915, regs.aud_cntrl_st2, - 0, IBX_ELD_VALID(port)); + /* + * The audio componenent is used to convey the ELD + * instead using of the hardware ELD buffer. + */ /* Enable timestamps */ intel_de_rmw(i915, regs.aud_config, @@ -809,6 +771,30 @@ void intel_audio_sdp_split_update(struct intel_encoder *encoder, crtc_state->sdp_split_enable ? AUD_ENABLE_SDP_SPLIT : 0); } +bool intel_audio_compute_config(struct intel_encoder *encoder, + struct intel_crtc_state *crtc_state, + struct drm_connector_state *conn_state) +{ + struct drm_i915_private *i915 = to_i915(encoder->base.dev); + struct drm_connector *connector = conn_state->connector; + const struct drm_display_mode *adjusted_mode = + &crtc_state->hw.adjusted_mode; + + if (!connector->eld[0]) { + drm_dbg_kms(&i915->drm, + "Bogus ELD on [CONNECTOR:%d:%s]\n", + connector->base.id, connector->name); + return false; + } + + BUILD_BUG_ON(sizeof(crtc_state->eld) != sizeof(connector->eld)); + memcpy(crtc_state->eld, connector->eld, sizeof(crtc_state->eld)); + + crtc_state->eld[6] = drm_av_sync_delay(connector, adjusted_mode) / 2; + + return true; +} + /** * intel_audio_codec_enable - Enable the audio codec for HD audio * @encoder: encoder on which to enable audio @@ -825,27 +811,19 @@ void intel_audio_codec_enable(struct intel_encoder *encoder, struct drm_i915_private *i915 = to_i915(encoder->base.dev); struct i915_audio_component *acomp = i915->display.audio.component; struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc); - struct drm_connector *connector = conn_state->connector; - const struct drm_display_mode *adjusted_mode = - &crtc_state->hw.adjusted_mode; + struct intel_connector *connector = to_intel_connector(conn_state->connector); + struct intel_audio_state *audio_state; enum port port = encoder->port; enum pipe pipe = crtc->pipe; if (!crtc_state->has_audio) return; - drm_dbg_kms(&i915->drm, "[CONNECTOR:%d:%s][ENCODER:%d:%s] Enable audio codec on pipe %c, %u bytes ELD\n", - connector->base.id, connector->name, + drm_dbg_kms(&i915->drm, "[CONNECTOR:%d:%s][ENCODER:%d:%s] Enable audio codec on [CRTC:%d:%s], %u bytes ELD\n", + connector->base.base.id, connector->base.name, encoder->base.base.id, encoder->base.name, - pipe_name(pipe), drm_eld_size(connector->eld)); - - /* FIXME precompute the ELD in .compute_config() */ - if (!connector->eld[0]) - drm_dbg_kms(&i915->drm, - "Bogus ELD on [CONNECTOR:%d:%s]\n", - connector->base.id, connector->name); - - connector->eld[6] = drm_av_sync_delay(connector, adjusted_mode) / 2; + crtc->base.base.id, crtc->base.name, + drm_eld_size(crtc_state->eld)); if (i915->display.funcs.audio) i915->display.funcs.audio->audio_codec_enable(encoder, @@ -853,10 +831,13 @@ void intel_audio_codec_enable(struct intel_encoder *encoder, conn_state); mutex_lock(&i915->display.audio.mutex); - encoder->audio_connector = connector; - /* referred in audio callbacks */ - i915->display.audio.encoder_map[pipe] = encoder; + audio_state = &i915->display.audio.state[pipe]; + + audio_state->encoder = encoder; + BUILD_BUG_ON(sizeof(audio_state->eld) != sizeof(crtc_state->eld)); + memcpy(audio_state->eld, crtc_state->eld, sizeof(audio_state->eld)); + mutex_unlock(&i915->display.audio.mutex); if (acomp && acomp->base.audio_ops && @@ -868,7 +849,7 @@ void intel_audio_codec_enable(struct intel_encoder *encoder, (int)port, (int)pipe); } - intel_lpe_audio_notify(i915, pipe, port, connector->eld, + intel_lpe_audio_notify(i915, pipe, port, crtc_state->eld, crtc_state->port_clock, intel_crtc_has_dp_encoder(crtc_state)); } @@ -889,16 +870,18 @@ void intel_audio_codec_disable(struct intel_encoder *encoder, struct drm_i915_private *i915 = to_i915(encoder->base.dev); struct i915_audio_component *acomp = i915->display.audio.component; struct intel_crtc *crtc = to_intel_crtc(old_crtc_state->uapi.crtc); - struct drm_connector *connector = old_conn_state->connector; + struct intel_connector *connector = to_intel_connector(old_conn_state->connector); + struct intel_audio_state *audio_state; enum port port = encoder->port; enum pipe pipe = crtc->pipe; if (!old_crtc_state->has_audio) return; - drm_dbg_kms(&i915->drm, "[CONNECTOR:%d:%s][ENCODER:%d:%s] Disable audio codec on pipe %c\n", - connector->base.id, connector->name, - encoder->base.base.id, encoder->base.name, pipe_name(pipe)); + drm_dbg_kms(&i915->drm, "[CONNECTOR:%d:%s][ENCODER:%d:%s] Disable audio codec on [CRTC:%d:%s]\n", + connector->base.base.id, connector->base.name, + encoder->base.base.id, encoder->base.name, + crtc->base.base.id, crtc->base.name); if (i915->display.funcs.audio) i915->display.funcs.audio->audio_codec_disable(encoder, @@ -906,8 +889,12 @@ void intel_audio_codec_disable(struct intel_encoder *encoder, old_conn_state); mutex_lock(&i915->display.audio.mutex); - encoder->audio_connector = NULL; - i915->display.audio.encoder_map[pipe] = NULL; + + audio_state = &i915->display.audio.state[pipe]; + + audio_state->encoder = NULL; + memset(audio_state->eld, 0, sizeof(audio_state->eld)); + mutex_unlock(&i915->display.audio.mutex); if (acomp && acomp->base.audio_ops && @@ -922,19 +909,52 @@ void intel_audio_codec_disable(struct intel_encoder *encoder, intel_lpe_audio_notify(i915, pipe, port, NULL, 0, false); } +static void intel_acomp_get_config(struct intel_encoder *encoder, + struct intel_crtc_state *crtc_state) +{ + struct drm_i915_private *i915 = to_i915(encoder->base.dev); + struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc); + struct intel_audio_state *audio_state; + enum pipe pipe = crtc->pipe; + + mutex_lock(&i915->display.audio.mutex); + + audio_state = &i915->display.audio.state[pipe]; + + if (audio_state->encoder) + memcpy(crtc_state->eld, audio_state->eld, sizeof(audio_state->eld)); + + mutex_unlock(&i915->display.audio.mutex); +} + +void intel_audio_codec_get_config(struct intel_encoder *encoder, + struct intel_crtc_state *crtc_state) +{ + struct drm_i915_private *i915 = to_i915(encoder->base.dev); + + if (!crtc_state->has_audio) + return; + + if (i915->display.funcs.audio) + i915->display.funcs.audio->audio_codec_get_config(encoder, crtc_state); +} + static const struct intel_audio_funcs g4x_audio_funcs = { .audio_codec_enable = g4x_audio_codec_enable, .audio_codec_disable = g4x_audio_codec_disable, + .audio_codec_get_config = g4x_audio_codec_get_config, }; -static const struct intel_audio_funcs ilk_audio_funcs = { - .audio_codec_enable = ilk_audio_codec_enable, - .audio_codec_disable = ilk_audio_codec_disable, +static const struct intel_audio_funcs ibx_audio_funcs = { + .audio_codec_enable = ibx_audio_codec_enable, + .audio_codec_disable = ibx_audio_codec_disable, + .audio_codec_get_config = intel_acomp_get_config, }; static const struct intel_audio_funcs hsw_audio_funcs = { .audio_codec_enable = hsw_audio_codec_enable, .audio_codec_disable = hsw_audio_codec_disable, + .audio_codec_get_config = intel_acomp_get_config, }; /** @@ -945,12 +965,11 @@ void intel_audio_hooks_init(struct drm_i915_private *i915) { if (IS_G4X(i915)) i915->display.funcs.audio = &g4x_audio_funcs; - else if (IS_VALLEYVIEW(i915) || IS_CHERRYVIEW(i915)) - i915->display.funcs.audio = &ilk_audio_funcs; + else if (IS_VALLEYVIEW(i915) || IS_CHERRYVIEW(i915) || + HAS_PCH_CPT(i915) || HAS_PCH_IBX(i915)) + i915->display.funcs.audio = &ibx_audio_funcs; else if (IS_HASWELL(i915) || DISPLAY_VER(i915) >= 8) i915->display.funcs.audio = &hsw_audio_funcs; - else if (HAS_PCH_SPLIT(i915)) - i915->display.funcs.audio = &ilk_audio_funcs; } struct aud_ts_cdclk_m_n { @@ -1128,35 +1147,32 @@ static int i915_audio_component_get_cdclk_freq(struct device *kdev) } /* - * get the intel_encoder according to the parameter port and pipe - * intel_encoder is saved by the index of pipe - * MST & (pipe >= 0): return the audio.encoder_map[pipe], + * get the intel audio state according to the parameter port and pipe + * MST & (pipe >= 0): return the audio.state[pipe].encoder], * when port is matched * MST & (pipe < 0): this is invalid * Non-MST & (pipe >= 0): only pipe = 0 (the first device entry) * will get the right intel_encoder with port matched * Non-MST & (pipe < 0): get the right intel_encoder with port matched */ -static struct intel_encoder *get_saved_enc(struct drm_i915_private *i915, - int port, int pipe) +static struct intel_audio_state *find_audio_state(struct drm_i915_private *i915, + int port, int pipe) { /* MST */ if (pipe >= 0) { + struct intel_audio_state *audio_state; struct intel_encoder *encoder; if (drm_WARN_ON(&i915->drm, - pipe >= ARRAY_SIZE(i915->display.audio.encoder_map))) + pipe >= ARRAY_SIZE(i915->display.audio.state))) return NULL; - encoder = i915->display.audio.encoder_map[pipe]; - /* - * when bootup, audio driver may not know it is - * MST or not. So it will poll all the port & pipe - * combinations - */ + audio_state = &i915->display.audio.state[pipe]; + encoder = audio_state->encoder; + if (encoder && encoder->port == port && encoder->type == INTEL_OUTPUT_DP_MST) - return encoder; + return audio_state; } /* Non-MST */ @@ -1164,13 +1180,15 @@ static struct intel_encoder *get_saved_enc(struct drm_i915_private *i915, return NULL; for_each_pipe(i915, pipe) { + struct intel_audio_state *audio_state; struct intel_encoder *encoder; - encoder = i915->display.audio.encoder_map[pipe]; + audio_state = &i915->display.audio.state[pipe]; + encoder = audio_state->encoder; if (encoder && encoder->port == port && encoder->type != INTEL_OUTPUT_DP_MST) - return encoder; + return audio_state; } return NULL; @@ -1181,6 +1199,7 @@ static int i915_audio_component_sync_audio_rate(struct device *kdev, int port, { struct drm_i915_private *i915 = kdev_to_i915(kdev); struct i915_audio_component *acomp = i915->display.audio.component; + const struct intel_audio_state *audio_state; struct intel_encoder *encoder; struct intel_crtc *crtc; unsigned long cookie; @@ -1192,20 +1211,22 @@ static int i915_audio_component_sync_audio_rate(struct device *kdev, int port, cookie = i915_audio_component_get_power(kdev); mutex_lock(&i915->display.audio.mutex); - /* 1. get the pipe */ - encoder = get_saved_enc(i915, port, pipe); - if (!encoder || !encoder->base.crtc) { - drm_dbg_kms(&i915->drm, "Not valid for port %c\n", - port_name(port)); + audio_state = find_audio_state(i915, port, pipe); + if (!audio_state) { + drm_dbg_kms(&i915->drm, "Not valid for port %c\n", port_name(port)); err = -ENODEV; goto unlock; } + encoder = audio_state->encoder; + + /* FIXME stop using the legacy crtc pointer */ crtc = to_intel_crtc(encoder->base.crtc); /* port must be valid now, otherwise the pipe will be invalid */ acomp->aud_sample_rate[port] = rate; + /* FIXME get rid of the crtc->config stuff */ hsw_audio_config_update(encoder, crtc->config); unlock: @@ -1219,24 +1240,22 @@ static int i915_audio_component_get_eld(struct device *kdev, int port, unsigned char *buf, int max_bytes) { struct drm_i915_private *i915 = kdev_to_i915(kdev); - struct intel_encoder *intel_encoder; - const u8 *eld; - int ret = -EINVAL; + const struct intel_audio_state *audio_state; + int ret = 0; mutex_lock(&i915->display.audio.mutex); - intel_encoder = get_saved_enc(i915, port, pipe); - if (!intel_encoder) { - drm_dbg_kms(&i915->drm, "Not valid for port %c\n", - port_name(port)); + audio_state = find_audio_state(i915, port, pipe); + if (!audio_state) { + drm_dbg_kms(&i915->drm, "Not valid for port %c\n", port_name(port)); mutex_unlock(&i915->display.audio.mutex); - return ret; + return -EINVAL; } - ret = 0; - *enabled = intel_encoder->audio_connector != NULL; + *enabled = audio_state->encoder != NULL; if (*enabled) { - eld = intel_encoder->audio_connector->eld; + const u8 *eld = audio_state->eld; + ret = drm_eld_size(eld); memcpy(buf, eld, min(max_bytes, ret)); } diff --git a/drivers/gpu/drm/i915/display/intel_audio.h b/drivers/gpu/drm/i915/display/intel_audio.h index 1b87257c6a175d739a2563941cbaff2e95137a0b..07d034a981e90e914e4ff7945834f4f68ee0a277 100644 --- a/drivers/gpu/drm/i915/display/intel_audio.h +++ b/drivers/gpu/drm/i915/display/intel_audio.h @@ -6,18 +6,25 @@ #ifndef __INTEL_AUDIO_H__ #define __INTEL_AUDIO_H__ +#include + struct drm_connector_state; struct drm_i915_private; struct intel_crtc_state; struct intel_encoder; void intel_audio_hooks_init(struct drm_i915_private *dev_priv); +bool intel_audio_compute_config(struct intel_encoder *encoder, + struct intel_crtc_state *crtc_state, + struct drm_connector_state *conn_state); void intel_audio_codec_enable(struct intel_encoder *encoder, const struct intel_crtc_state *crtc_state, const struct drm_connector_state *conn_state); void intel_audio_codec_disable(struct intel_encoder *encoder, const struct intel_crtc_state *old_crtc_state, const struct drm_connector_state *old_conn_state); +void intel_audio_codec_get_config(struct intel_encoder *encoder, + struct intel_crtc_state *crtc_state); void intel_audio_cdclk_change_pre(struct drm_i915_private *dev_priv); void intel_audio_cdclk_change_post(struct drm_i915_private *dev_priv); void intel_audio_init(struct drm_i915_private *dev_priv); diff --git a/drivers/gpu/drm/i915/display/intel_bios.c b/drivers/gpu/drm/i915/display/intel_bios.c index 55544d48431888c9b055346dfcab7cd5c36cd2d8..e6ca51232dcf8be78ac715f97e5bc290bf9cf96b 100644 --- a/drivers/gpu/drm/i915/display/intel_bios.c +++ b/drivers/gpu/drm/i915/display/intel_bios.c @@ -25,16 +25,15 @@ * */ -#include #include #include - -#include "display/intel_display.h" -#include "display/intel_display_types.h" -#include "display/intel_gmbus.h" +#include #include "i915_drv.h" #include "i915_reg.h" +#include "intel_display.h" +#include "intel_display_types.h" +#include "intel_gmbus.h" #define _INTEL_BIOS_PRIVATE #include "intel_vbt_defs.h" @@ -620,14 +619,14 @@ static void dump_pnp_id(struct drm_i915_private *i915, static int opregion_get_panel_type(struct drm_i915_private *i915, const struct intel_bios_encoder_data *devdata, - const struct edid *edid, bool use_fallback) + const struct drm_edid *drm_edid, bool use_fallback) { return intel_opregion_get_panel_type(i915); } static int vbt_get_panel_type(struct drm_i915_private *i915, const struct intel_bios_encoder_data *devdata, - const struct edid *edid, bool use_fallback) + const struct drm_edid *drm_edid, bool use_fallback) { const struct bdb_lvds_options *lvds_options; @@ -652,12 +651,13 @@ static int vbt_get_panel_type(struct drm_i915_private *i915, static int pnpid_get_panel_type(struct drm_i915_private *i915, const struct intel_bios_encoder_data *devdata, - const struct edid *edid, bool use_fallback) + const struct drm_edid *drm_edid, bool use_fallback) { const struct bdb_lvds_lfp_data *data; const struct bdb_lvds_lfp_data_ptrs *ptrs; const struct lvds_pnp_id *edid_id; struct lvds_pnp_id edid_id_nodate; + const struct edid *edid = drm_edid_raw(drm_edid); /* FIXME */ int i, best = -1; if (!edid) @@ -701,7 +701,7 @@ static int pnpid_get_panel_type(struct drm_i915_private *i915, static int fallback_get_panel_type(struct drm_i915_private *i915, const struct intel_bios_encoder_data *devdata, - const struct edid *edid, bool use_fallback) + const struct drm_edid *drm_edid, bool use_fallback) { return use_fallback ? 0 : -1; } @@ -715,13 +715,13 @@ enum panel_type { static int get_panel_type(struct drm_i915_private *i915, const struct intel_bios_encoder_data *devdata, - const struct edid *edid, bool use_fallback) + const struct drm_edid *drm_edid, bool use_fallback) { struct { const char *name; int (*get_panel_type)(struct drm_i915_private *i915, const struct intel_bios_encoder_data *devdata, - const struct edid *edid, bool use_fallback); + const struct drm_edid *drm_edid, bool use_fallback); int panel_type; } panel_types[] = { [PANEL_TYPE_OPREGION] = { @@ -745,7 +745,7 @@ static int get_panel_type(struct drm_i915_private *i915, for (i = 0; i < ARRAY_SIZE(panel_types); i++) { panel_types[i].panel_type = panel_types[i].get_panel_type(i915, devdata, - edid, use_fallback); + drm_edid, use_fallback); drm_WARN_ON(&i915->drm, panel_types[i].panel_type > 0xf && panel_types[i].panel_type != 0xff); @@ -3187,7 +3187,7 @@ void intel_bios_init(struct drm_i915_private *i915) static void intel_bios_init_panel(struct drm_i915_private *i915, struct intel_panel *panel, const struct intel_bios_encoder_data *devdata, - const struct edid *edid, + const struct drm_edid *drm_edid, bool use_fallback) { /* already have it? */ @@ -3197,7 +3197,7 @@ static void intel_bios_init_panel(struct drm_i915_private *i915, } panel->vbt.panel_type = get_panel_type(i915, devdata, - edid, use_fallback); + drm_edid, use_fallback); if (panel->vbt.panel_type < 0) { drm_WARN_ON(&i915->drm, use_fallback); return; @@ -3228,9 +3228,9 @@ void intel_bios_init_panel_early(struct drm_i915_private *i915, void intel_bios_init_panel_late(struct drm_i915_private *i915, struct intel_panel *panel, const struct intel_bios_encoder_data *devdata, - const struct edid *edid) + const struct drm_edid *drm_edid) { - intel_bios_init_panel(i915, panel, devdata, edid, true); + intel_bios_init_panel(i915, panel, devdata, drm_edid, true); } /** diff --git a/drivers/gpu/drm/i915/display/intel_bios.h b/drivers/gpu/drm/i915/display/intel_bios.h index ff1fdd2e0c1c5cf367336ce036f6653dd5c2c77f..d221f784aa88d207d1820f4dac285bf7bcd6e757 100644 --- a/drivers/gpu/drm/i915/display/intel_bios.h +++ b/drivers/gpu/drm/i915/display/intel_bios.h @@ -32,8 +32,8 @@ #include +struct drm_edid; struct drm_i915_private; -struct edid; struct intel_bios_encoder_data; struct intel_crtc_state; struct intel_encoder; @@ -238,7 +238,7 @@ void intel_bios_init_panel_early(struct drm_i915_private *dev_priv, void intel_bios_init_panel_late(struct drm_i915_private *dev_priv, struct intel_panel *panel, const struct intel_bios_encoder_data *devdata, - const struct edid *edid); + const struct drm_edid *drm_edid); void intel_bios_fini_panel(struct intel_panel *panel); void intel_bios_driver_remove(struct drm_i915_private *dev_priv); bool intel_bios_is_valid_vbt(const void *buf, size_t size); diff --git a/drivers/gpu/drm/i915/display/intel_bw.h b/drivers/gpu/drm/i915/display/intel_bw.h index cb7ee3a24a58c4afe35d248102cbd11e5fb5f00b..f20292143745502c48e05078052644531dc2c864 100644 --- a/drivers/gpu/drm/i915/display/intel_bw.h +++ b/drivers/gpu/drm/i915/display/intel_bw.h @@ -8,7 +8,7 @@ #include -#include "intel_display.h" +#include "intel_display_limits.h" #include "intel_display_power.h" #include "intel_global_state.h" diff --git a/drivers/gpu/drm/i915/display/intel_cdclk.h b/drivers/gpu/drm/i915/display/intel_cdclk.h index c674879a84a58c765d1cfccc44b17e0f3eb2e240..51e2f6a11ce4135a9b5b359736c67f3e67436874 100644 --- a/drivers/gpu/drm/i915/display/intel_cdclk.h +++ b/drivers/gpu/drm/i915/display/intel_cdclk.h @@ -8,7 +8,7 @@ #include -#include "intel_display.h" +#include "intel_display_limits.h" #include "intel_global_state.h" struct drm_i915_private; diff --git a/drivers/gpu/drm/i915/display/intel_color.c b/drivers/gpu/drm/i915/display/intel_color.c index d57631b0bb9a8ed3fa934f6394584a97b975940f..8d97c299e6577b786dc39aa881e184b386b1ccd9 100644 --- a/drivers/gpu/drm/i915/display/intel_color.c +++ b/drivers/gpu/drm/i915/display/intel_color.c @@ -847,17 +847,6 @@ static void ilk_lut_write(const struct intel_crtc_state *crtc_state, intel_de_write_fw(i915, reg, val); } -static void ilk_lut_write_indexed(const struct intel_crtc_state *crtc_state, - i915_reg_t reg, u32 val) -{ - struct drm_i915_private *i915 = to_i915(crtc_state->uapi.crtc->dev); - - if (crtc_state->dsb) - intel_dsb_indexed_reg_write(crtc_state->dsb, reg, val); - else - intel_de_write_fw(i915, reg, val); -} - static void ilk_load_lut_8(const struct intel_crtc_state *crtc_state, const struct drm_property_blob *blob) { @@ -962,8 +951,8 @@ static void bdw_load_lut_10(const struct intel_crtc_state *crtc_state, prec_index); for (i = 0; i < lut_size; i++) - ilk_lut_write_indexed(crtc_state, PREC_PAL_DATA(pipe), - ilk_lut_10(&lut[i])); + ilk_lut_write(crtc_state, PREC_PAL_DATA(pipe), + ilk_lut_10(&lut[i])); /* * Reset the index, otherwise it prevents the legacy palette to be @@ -1093,13 +1082,13 @@ static void glk_load_degamma_lut(const struct intel_crtc_state *crtc_state, * ToDo: Extend to max 7.0. Enable 32 bit input value * as compared to just 16 to achieve this. */ - ilk_lut_write_indexed(crtc_state, PRE_CSC_GAMC_DATA(pipe), - lut[i].green); + ilk_lut_write(crtc_state, PRE_CSC_GAMC_DATA(pipe), + lut[i].green); } /* Clamp values > 1.0. */ while (i++ < glk_degamma_lut_size(i915)) - ilk_lut_write_indexed(crtc_state, PRE_CSC_GAMC_DATA(pipe), 1 << 16); + ilk_lut_write(crtc_state, PRE_CSC_GAMC_DATA(pipe), 1 << 16); ilk_lut_write(crtc_state, PRE_CSC_GAMC_INDEX(pipe), 0); } @@ -1165,10 +1154,10 @@ icl_program_gamma_superfine_segment(const struct intel_crtc_state *crtc_state) for (i = 0; i < 9; i++) { const struct drm_color_lut *entry = &lut[i]; - ilk_lut_write_indexed(crtc_state, PREC_PAL_MULTI_SEG_DATA(pipe), - ilk_lut_12p4_ldw(entry)); - ilk_lut_write_indexed(crtc_state, PREC_PAL_MULTI_SEG_DATA(pipe), - ilk_lut_12p4_udw(entry)); + ilk_lut_write(crtc_state, PREC_PAL_MULTI_SEG_DATA(pipe), + ilk_lut_12p4_ldw(entry)); + ilk_lut_write(crtc_state, PREC_PAL_MULTI_SEG_DATA(pipe), + ilk_lut_12p4_udw(entry)); } ilk_lut_write(crtc_state, PREC_PAL_MULTI_SEG_INDEX(pipe), @@ -1204,10 +1193,10 @@ icl_program_gamma_multi_segment(const struct intel_crtc_state *crtc_state) for (i = 1; i < 257; i++) { entry = &lut[i * 8]; - ilk_lut_write_indexed(crtc_state, PREC_PAL_DATA(pipe), - ilk_lut_12p4_ldw(entry)); - ilk_lut_write_indexed(crtc_state, PREC_PAL_DATA(pipe), - ilk_lut_12p4_udw(entry)); + ilk_lut_write(crtc_state, PREC_PAL_DATA(pipe), + ilk_lut_12p4_ldw(entry)); + ilk_lut_write(crtc_state, PREC_PAL_DATA(pipe), + ilk_lut_12p4_udw(entry)); } /* @@ -1225,10 +1214,10 @@ icl_program_gamma_multi_segment(const struct intel_crtc_state *crtc_state) for (i = 0; i < 256; i++) { entry = &lut[i * 8 * 128]; - ilk_lut_write_indexed(crtc_state, PREC_PAL_DATA(pipe), - ilk_lut_12p4_ldw(entry)); - ilk_lut_write_indexed(crtc_state, PREC_PAL_DATA(pipe), - ilk_lut_12p4_udw(entry)); + ilk_lut_write(crtc_state, PREC_PAL_DATA(pipe), + ilk_lut_12p4_ldw(entry)); + ilk_lut_write(crtc_state, PREC_PAL_DATA(pipe), + ilk_lut_12p4_udw(entry)); } ilk_lut_write(crtc_state, PREC_PAL_INDEX(pipe), @@ -1391,7 +1380,7 @@ void intel_color_prepare_commit(struct intel_crtc_state *crtc_state) /* FIXME DSB has issues loading LUTs, disable it for now */ return; - crtc_state->dsb = intel_dsb_prepare(crtc); + crtc_state->dsb = intel_dsb_prepare(crtc, 1024); } void intel_color_cleanup_commit(struct intel_crtc_state *crtc_state) diff --git a/drivers/gpu/drm/i915/display/intel_connector.c b/drivers/gpu/drm/i915/display/intel_connector.c index 562da3b741e2d6c9f0d918ebabb17791d90be680..257afac348397dcaf794480aa0f149cbf4e46f25 100644 --- a/drivers/gpu/drm/i915/display/intel_connector.c +++ b/drivers/gpu/drm/i915/display/intel_connector.c @@ -95,13 +95,10 @@ void intel_connector_destroy(struct drm_connector *connector) { struct intel_connector *intel_connector = to_intel_connector(connector); - kfree(intel_connector->detect_edid); + drm_edid_free(intel_connector->detect_edid); intel_hdcp_cleanup(intel_connector); - if (!IS_ERR_OR_NULL(intel_connector->edid)) - kfree(intel_connector->edid); - intel_panel_fini(intel_connector); drm_connector_cleanup(connector); diff --git a/drivers/gpu/drm/i915/display/intel_crtc.c b/drivers/gpu/drm/i915/display/intel_crtc.c index 037fc140b585caec6de732d9ea18c1b5dd18a151..82be0fbe99342bf901b9473f5f95afe4fedd8b2c 100644 --- a/drivers/gpu/drm/i915/display/intel_crtc.c +++ b/drivers/gpu/drm/i915/display/intel_crtc.c @@ -28,6 +28,7 @@ #include "intel_pipe_crc.h" #include "intel_psr.h" #include "intel_sprite.h" +#include "intel_vblank.h" #include "intel_vrr.h" #include "skl_universal_plane.h" diff --git a/drivers/gpu/drm/i915/display/intel_crtc_state_dump.c b/drivers/gpu/drm/i915/display/intel_crtc_state_dump.c index e3273fe8ddacc2817311704b2009482ceaa1382b..2422d6ef57776487e6fc34d535f8493d919007e8 100644 --- a/drivers/gpu/drm/i915/display/intel_crtc_state_dump.c +++ b/drivers/gpu/drm/i915/display/intel_crtc_state_dump.c @@ -3,6 +3,8 @@ * Copyright © 2022 Intel Corporation */ +#include + #include "i915_drv.h" #include "intel_crtc_state_dump.h" #include "intel_display_types.h" @@ -56,6 +58,17 @@ intel_dump_dp_vsc_sdp(struct drm_i915_private *i915, drm_dp_vsc_sdp_log(KERN_DEBUG, i915->drm.dev, vsc); } +static void +intel_dump_buffer(struct drm_i915_private *i915, + const char *prefix, const u8 *buf, size_t len) +{ + if (!drm_debug_enabled(DRM_UT_KMS)) + return; + + print_hex_dump(KERN_DEBUG, prefix, DUMP_PREFIX_NONE, + 16, 0, buf, len, false); +} + #define OUTPUT_TYPE(x) [INTEL_OUTPUT_ ## x] = #x static const char * const output_type_str[] = { @@ -236,6 +249,10 @@ void intel_crtc_state_dump(const struct intel_crtc_state *pipe_config, intel_hdmi_infoframe_enable(DP_SDP_VSC)) intel_dump_dp_vsc_sdp(i915, &pipe_config->infoframes.vsc); + if (pipe_config->has_audio) + intel_dump_buffer(i915, "ELD: ", pipe_config->eld, + drm_eld_size(pipe_config->eld)); + drm_dbg_kms(&i915->drm, "vrr: %s, vmin: %d, vmax: %d, pipeline full: %d, guardband: %d flipline: %d, vmin vblank: %d, vmax vblank: %d\n", str_yes_no(pipe_config->vrr.enable), pipe_config->vrr.vmin, pipe_config->vrr.vmax, diff --git a/drivers/gpu/drm/i915/display/intel_ddi.c b/drivers/gpu/drm/i915/display/intel_ddi.c index 1f5a471a0adf277034f4bcf96f3254707ac09548..254559abedfba554e834860ae6303f95c5162444 100644 --- a/drivers/gpu/drm/i915/display/intel_ddi.c +++ b/drivers/gpu/drm/i915/display/intel_ddi.c @@ -3496,6 +3496,8 @@ static void intel_ddi_get_config(struct intel_encoder *encoder, intel_read_dp_sdp(encoder, pipe_config, DP_SDP_VSC); intel_psr_get_config(encoder, pipe_config); + + intel_audio_codec_get_config(encoder, pipe_config); } void intel_ddi_get_clock(struct intel_encoder *encoder, @@ -4323,7 +4325,7 @@ void intel_ddi_init(struct drm_i915_private *dev_priv, enum port port) } if (intel_phy_is_snps(dev_priv, phy) && - dev_priv->snps_phy_failed_calibration & BIT(phy)) { + dev_priv->display.snps.phy_failed_calibration & BIT(phy)) { drm_dbg_kms(&dev_priv->drm, "SNPS PHY %c failed to calibrate, proceeding anyway\n", phy_name(phy)); diff --git a/drivers/gpu/drm/i915/display/intel_de.h b/drivers/gpu/drm/i915/display/intel_de.h index 3dbd76fdabd651d7193881b3c5a82a9e5c4725b0..42552d8c151e868e30b0bd58cce80d85ef359e42 100644 --- a/drivers/gpu/drm/i915/display/intel_de.h +++ b/drivers/gpu/drm/i915/display/intel_de.h @@ -22,6 +22,13 @@ intel_de_read8(struct drm_i915_private *i915, i915_reg_t reg) return intel_uncore_read8(&i915->uncore, reg); } +static inline u64 +intel_de_read64_2x32(struct drm_i915_private *i915, + i915_reg_t lower_reg, i915_reg_t upper_reg) +{ + return intel_uncore_read64_2x32(&i915->uncore, lower_reg, upper_reg); +} + static inline void intel_de_posting_read(struct drm_i915_private *i915, i915_reg_t reg) { diff --git a/drivers/gpu/drm/i915/display/intel_display.c b/drivers/gpu/drm/i915/display/intel_display.c index e75b9b2a0e015a5a99a2a085b303d81ecb961657..717ca3d7890d34e5d3ee1bf8cb37727279bad13a 100644 --- a/drivers/gpu/drm/i915/display/intel_display.c +++ b/drivers/gpu/drm/i915/display/intel_display.c @@ -24,15 +24,15 @@ * Eric Anholt */ -#include +#include #include #include #include #include -#include #include #include #include +#include #include #include @@ -45,28 +45,6 @@ #include #include -#include "display/intel_audio.h" -#include "display/intel_crt.h" -#include "display/intel_ddi.h" -#include "display/intel_display_debugfs.h" -#include "display/intel_display_power.h" -#include "display/intel_dp.h" -#include "display/intel_dp_mst.h" -#include "display/intel_dpll.h" -#include "display/intel_dpll_mgr.h" -#include "display/intel_drrs.h" -#include "display/intel_dsi.h" -#include "display/intel_dvo.h" -#include "display/intel_fb.h" -#include "display/intel_gmbus.h" -#include "display/intel_hdmi.h" -#include "display/intel_lvds.h" -#include "display/intel_sdvo.h" -#include "display/intel_snps_phy.h" -#include "display/intel_tv.h" -#include "display/intel_vdsc.h" -#include "display/intel_vrr.h" - #include "gem/i915_gem_lmem.h" #include "gem/i915_gem_object.h" @@ -76,31 +54,48 @@ #include "i915_drv.h" #include "i915_reg.h" #include "i915_utils.h" +#include "i9xx_plane.h" #include "icl_dsi.h" #include "intel_acpi.h" #include "intel_atomic.h" #include "intel_atomic_plane.h" +#include "intel_audio.h" #include "intel_bw.h" #include "intel_cdclk.h" #include "intel_color.h" +#include "intel_crt.h" #include "intel_crtc.h" #include "intel_crtc_state_dump.h" +#include "intel_ddi.h" #include "intel_de.h" +#include "intel_display_debugfs.h" +#include "intel_display_power.h" #include "intel_display_types.h" #include "intel_dmc.h" +#include "intel_dp.h" #include "intel_dp_link_training.h" +#include "intel_dp_mst.h" #include "intel_dpio_phy.h" +#include "intel_dpll.h" +#include "intel_dpll_mgr.h" #include "intel_dpt.h" +#include "intel_drrs.h" +#include "intel_dsi.h" +#include "intel_dvo.h" +#include "intel_fb.h" #include "intel_fbc.h" #include "intel_fbdev.h" #include "intel_fdi.h" #include "intel_fifo_underrun.h" #include "intel_frontbuffer.h" +#include "intel_gmbus.h" #include "intel_hdcp.h" +#include "intel_hdmi.h" #include "intel_hotplug.h" #include "intel_hti.h" -#include "intel_modeset_verify.h" +#include "intel_lvds.h" #include "intel_modeset_setup.h" +#include "intel_modeset_verify.h" #include "intel_overlay.h" #include "intel_panel.h" #include "intel_pch_display.h" @@ -112,10 +107,15 @@ #include "intel_pps.h" #include "intel_psr.h" #include "intel_quirks.h" +#include "intel_sdvo.h" +#include "intel_snps_phy.h" #include "intel_sprite.h" #include "intel_tc.h" +#include "intel_tv.h" +#include "intel_vblank.h" +#include "intel_vdsc.h" #include "intel_vga.h" -#include "i9xx_plane.h" +#include "intel_vrr.h" #include "skl_scaler.h" #include "skl_universal_plane.h" #include "skl_watermark.h" @@ -385,41 +385,6 @@ struct intel_crtc *intel_master_crtc(const struct intel_crtc_state *crtc_state) return to_intel_crtc(crtc_state->uapi.crtc); } -static bool pipe_scanline_is_moving(struct drm_i915_private *dev_priv, - enum pipe pipe) -{ - i915_reg_t reg = PIPEDSL(pipe); - u32 line1, line2; - - line1 = intel_de_read(dev_priv, reg) & PIPEDSL_LINE_MASK; - msleep(5); - line2 = intel_de_read(dev_priv, reg) & PIPEDSL_LINE_MASK; - - return line1 != line2; -} - -static void wait_for_pipe_scanline_moving(struct intel_crtc *crtc, bool state) -{ - struct drm_i915_private *dev_priv = to_i915(crtc->base.dev); - enum pipe pipe = crtc->pipe; - - /* Wait for the display line to settle/start moving */ - if (wait_for(pipe_scanline_is_moving(dev_priv, pipe) == state, 100)) - drm_err(&dev_priv->drm, - "pipe %c scanline %s wait timed out\n", - pipe_name(pipe), str_on_off(state)); -} - -static void intel_wait_for_pipe_scanline_stopped(struct intel_crtc *crtc) -{ - wait_for_pipe_scanline_moving(crtc, false); -} - -static void intel_wait_for_pipe_scanline_moving(struct intel_crtc *crtc) -{ - wait_for_pipe_scanline_moving(crtc, true); -} - static void intel_wait_for_pipe_off(const struct intel_crtc_state *old_crtc_state) { @@ -1095,22 +1060,6 @@ intel_get_crtc_new_encoder(const struct intel_atomic_state *state, return encoder; } -static void cpt_verify_modeset(struct drm_i915_private *dev_priv, - enum pipe pipe) -{ - i915_reg_t dslreg = PIPEDSL(pipe); - u32 temp; - - temp = intel_de_read(dev_priv, dslreg); - udelay(500); - if (wait_for(intel_de_read(dev_priv, dslreg) != temp, 5)) { - if (wait_for(intel_de_read(dev_priv, dslreg) != temp, 5)) - drm_err(&dev_priv->drm, - "mode set failed: pipe %c stuck\n", - pipe_name(pipe)); - } -} - static void ilk_pfit_enable(const struct intel_crtc_state *crtc_state) { struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc); @@ -1805,7 +1754,7 @@ static void ilk_crtc_enable(struct intel_atomic_state *state, intel_encoders_enable(state, crtc); if (HAS_PCH_CPT(dev_priv)) - cpt_verify_modeset(dev_priv, pipe); + intel_wait_for_pipe_scanline_moving(crtc); /* * Must wait for vblank to avoid spurious PCH FIFO underruns. @@ -1918,6 +1867,8 @@ static void hsw_crtc_enable(struct intel_atomic_state *state, if (drm_WARN_ON(&dev_priv->drm, crtc->active)) return; + intel_dmc_enable_pipe(dev_priv, crtc->pipe); + if (!new_crtc_state->bigjoiner_pipes) { intel_encoders_pre_pll_enable(state, crtc); @@ -2053,6 +2004,7 @@ static void hsw_crtc_disable(struct intel_atomic_state *state, { const struct intel_crtc_state *old_crtc_state = intel_atomic_get_old_crtc_state(state, crtc); + struct drm_i915_private *i915 = to_i915(crtc->base.dev); /* * FIXME collapse everything to one hook. @@ -2062,6 +2014,8 @@ static void hsw_crtc_disable(struct intel_atomic_state *state, intel_encoders_disable(state, crtc); intel_encoders_post_disable(state, crtc); } + + intel_dmc_disable_pipe(i915, crtc->pipe); } static void i9xx_pfit_enable(const struct intel_crtc_state *crtc_state) @@ -3292,7 +3246,7 @@ static bool i9xx_get_pipe_config(struct intel_crtc *crtc, if (DISPLAY_VER(dev_priv) >= 4) { /* No way to read it out on pipes B and C */ if (IS_CHERRYVIEW(dev_priv) && crtc->pipe != PIPE_A) - tmp = dev_priv->chv_dpll_md[crtc->pipe]; + tmp = dev_priv->display.state.chv_dpll_md[crtc->pipe]; else tmp = intel_de_read(dev_priv, DPLL_MD(crtc->pipe)); pipe_config->pixel_multiplier = @@ -5429,6 +5383,12 @@ intel_compare_dp_vsc_sdp(const struct drm_dp_vsc_sdp *a, return memcmp(a, b, sizeof(*a)) == 0; } +static bool +intel_compare_buffer(const u8 *a, const u8 *b, size_t len) +{ + return memcmp(a, b, len) == 0; +} + static void pipe_config_infoframe_mismatch(struct drm_i915_private *dev_priv, bool fastset, const char *name, @@ -5479,6 +5439,30 @@ pipe_config_dp_vsc_sdp_mismatch(struct drm_i915_private *dev_priv, } } +static void +pipe_config_buffer_mismatch(struct drm_i915_private *dev_priv, + bool fastset, const char *name, + const u8 *a, const u8 *b, size_t len) +{ + if (fastset) { + if (!drm_debug_enabled(DRM_UT_KMS)) + return; + + drm_dbg_kms(&dev_priv->drm, + "fastset mismatch in %s buffer\n", name); + print_hex_dump(KERN_DEBUG, "expected: ", DUMP_PREFIX_NONE, + 16, 0, a, len, false); + print_hex_dump(KERN_DEBUG, "found: ", DUMP_PREFIX_NONE, + 16, 0, b, len, false); + } else { + drm_err(&dev_priv->drm, "mismatch in %s buffer\n", name); + print_hex_dump(KERN_ERR, "expected: ", DUMP_PREFIX_NONE, + 16, 0, a, len, false); + print_hex_dump(KERN_ERR, "found: ", DUMP_PREFIX_NONE, + 16, 0, b, len, false); + } +} + static void __printf(4, 5) pipe_config_mismatch(bool fastset, const struct intel_crtc *crtc, const char *name, const char *format, ...) @@ -5677,6 +5661,18 @@ intel_pipe_config_compare(const struct intel_crtc_state *current_config, } \ } while (0) +#define PIPE_CONF_CHECK_BUFFER(name, len) do { \ + BUILD_BUG_ON(sizeof(current_config->name) != (len)); \ + BUILD_BUG_ON(sizeof(pipe_config->name) != (len)); \ + if (!intel_compare_buffer(current_config->name, pipe_config->name, (len))) { \ + pipe_config_buffer_mismatch(dev_priv, fastset, __stringify(name), \ + current_config->name, \ + pipe_config->name, \ + (len)); \ + ret = false; \ + } \ +} while (0) + #define PIPE_CONF_CHECK_COLOR_LUT(lut, is_pre_csc_lut) do { \ if (current_config->gamma_mode == pipe_config->gamma_mode && \ !intel_color_lut_equal(current_config, \ @@ -5748,6 +5744,7 @@ intel_pipe_config_compare(const struct intel_crtc_state *current_config, PIPE_CONF_CHECK_BOOL(fec_enable); PIPE_CONF_CHECK_BOOL_INCOMPLETE(has_audio); + PIPE_CONF_CHECK_BUFFER(eld, MAX_ELD_BYTES); PIPE_CONF_CHECK_X(gmch_pfit.control); /* pfit ratios are autocomputed by the hw on gen4+ */ diff --git a/drivers/gpu/drm/i915/display/intel_display.h b/drivers/gpu/drm/i915/display/intel_display.h index ef73730f32b095cb3428ff36e45eb26d1a27c26e..cb6f520cc575f10ab979a48f68ea7f38d1a28086 100644 --- a/drivers/gpu/drm/i915/display/intel_display.h +++ b/drivers/gpu/drm/i915/display/intel_display.h @@ -28,6 +28,7 @@ #include #include "i915_reg_defs.h" +#include "intel_display_limits.h" enum drm_scaling_filter; struct dpll; @@ -62,51 +63,9 @@ struct intel_remapped_info; struct intel_rotation_info; struct pci_dev; -/* - * Keep the pipe enum values fixed: the code assumes that PIPE_A=0, the - * rest have consecutive values and match the enum values of transcoders - * with a 1:1 transcoder -> pipe mapping. - */ -enum pipe { - INVALID_PIPE = -1, - - PIPE_A = 0, - PIPE_B, - PIPE_C, - PIPE_D, - _PIPE_EDP, - - I915_MAX_PIPES = _PIPE_EDP -}; #define pipe_name(p) ((p) + 'A') -enum transcoder { - INVALID_TRANSCODER = -1, - /* - * The following transcoders have a 1:1 transcoder -> pipe mapping, - * keep their values fixed: the code assumes that TRANSCODER_A=0, the - * rest have consecutive values and match the enum values of the pipes - * they map to. - */ - TRANSCODER_A = PIPE_A, - TRANSCODER_B = PIPE_B, - TRANSCODER_C = PIPE_C, - TRANSCODER_D = PIPE_D, - - /* - * The following transcoders can map to any pipe, their enum value - * doesn't need to stay fixed. - */ - TRANSCODER_EDP, - TRANSCODER_DSI_0, - TRANSCODER_DSI_1, - TRANSCODER_DSI_A = TRANSCODER_DSI_0, /* legacy DSI */ - TRANSCODER_DSI_C = TRANSCODER_DSI_1, /* legacy DSI */ - - I915_MAX_TRANSCODERS -}; - static inline const char *transcoder_name(enum transcoder transcoder) { switch (transcoder) { @@ -147,29 +106,6 @@ enum i9xx_plane_id { #define plane_name(p) ((p) + 'A') #define sprite_name(p, s) ((p) * RUNTIME_INFO(dev_priv)->num_sprites[(p)] + (s) + 'A') -/* - * Per-pipe plane identifier. - * I915_MAX_PLANES in the enum below is the maximum (across all platforms) - * number of planes per CRTC. Not all platforms really have this many planes, - * which means some arrays of size I915_MAX_PLANES may have unused entries - * between the topmost sprite plane and the cursor plane. - * - * This is expected to be passed to various register macros - * (eg. PLANE_CTL(), PS_PLANE_SEL(), etc.) so adjust with care. - */ -enum plane_id { - PLANE_PRIMARY, - PLANE_SPRITE0, - PLANE_SPRITE1, - PLANE_SPRITE2, - PLANE_SPRITE3, - PLANE_SPRITE4, - PLANE_SPRITE5, - PLANE_CURSOR, - - I915_MAX_PLANES, -}; - #define for_each_plane_id_on_crtc(__crtc, __p) \ for ((__p) = PLANE_PRIMARY; (__p) < I915_MAX_PLANES; (__p)++) \ for_each_if((__crtc)->plane_ids_mask & BIT(__p)) @@ -182,34 +118,6 @@ enum plane_id { for_each_dbuf_slice((__dev_priv), (__slice)) \ for_each_if((__mask) & BIT(__slice)) -enum port { - PORT_NONE = -1, - - PORT_A = 0, - PORT_B, - PORT_C, - PORT_D, - PORT_E, - PORT_F, - PORT_G, - PORT_H, - PORT_I, - - /* tgl+ */ - PORT_TC1 = PORT_D, - PORT_TC2, - PORT_TC3, - PORT_TC4, - PORT_TC5, - PORT_TC6, - - /* XE_LPD repositions D/E offsets and bitfields */ - PORT_D_XELPD = PORT_TC5, - PORT_E_XELPD, - - I915_MAX_PORTS -}; - #define port_name(p) ((p) + 'A') /* @@ -312,27 +220,6 @@ enum phy_fia { FIA3, }; -enum hpd_pin { - HPD_NONE = 0, - HPD_TV = HPD_NONE, /* TV is known to be unreliable */ - HPD_CRT, - HPD_SDVO_B, - HPD_SDVO_C, - HPD_PORT_A, - HPD_PORT_B, - HPD_PORT_C, - HPD_PORT_D, - HPD_PORT_E, - HPD_PORT_TC1, - HPD_PORT_TC2, - HPD_PORT_TC3, - HPD_PORT_TC4, - HPD_PORT_TC5, - HPD_PORT_TC6, - - HPD_NUM_PINS -}; - #define for_each_hpd_pin(__pin) \ for ((__pin) = (HPD_NONE + 1); (__pin) < HPD_NUM_PINS; (__pin)++) diff --git a/drivers/gpu/drm/i915/display/intel_display_core.h b/drivers/gpu/drm/i915/display/intel_display_core.h index 57ddce3ba02b397eeb21bb59bef53b139b85c81b..fb8670aa2932103946ffb54cbaba9a16f455fdd0 100644 --- a/drivers/gpu/drm/i915/display/intel_display_core.h +++ b/drivers/gpu/drm/i915/display/intel_display_core.h @@ -17,7 +17,7 @@ #include #include "intel_cdclk.h" -#include "intel_display.h" +#include "intel_display_limits.h" #include "intel_display_power.h" #include "intel_dmc.h" #include "intel_dpll_mgr.h" @@ -87,6 +87,11 @@ struct intel_wm_funcs { int (*compute_global_watermarks)(struct intel_atomic_state *state); }; +struct intel_audio_state { + struct intel_encoder *encoder; + u8 eld[MAX_ELD_BYTES]; +}; + struct intel_audio { /* hda/i915 audio component */ struct i915_audio_component *component; @@ -96,8 +101,8 @@ struct intel_audio { int power_refcount; u32 freq_cntrl; - /* Used to save the pipe-to-encoder mapping for audio */ - struct intel_encoder *encoder_map[I915_MAX_PIPES]; + /* current audio state for the audio component hooks */ + struct intel_audio_state state[I915_MAX_PIPES]; /* necessary resource sharing with HDMI LPE audio driver. */ struct { @@ -122,6 +127,11 @@ struct intel_dpll { int nssc; int ssc; } ref_clks; + + /* + * Bitmask of PLLs using the PCH SSC, indexed using enum intel_dpll_id. + */ + u8 pch_ssc_use; }; struct intel_frontbuffer_tracking { @@ -428,6 +438,24 @@ struct intel_display { u32 block_time_us; } sagv; + struct { + /* + * DG2: Mask of PHYs that were not calibrated by the firmware + * and should not be used. + */ + u8 phy_failed_calibration; + } snps; + + struct { + /* + * Shadows for CHV DPLL_MD regs to keep the state + * checker somewhat working in the presence hardware + * crappiness (can't read out DPLL_MD for pipes B & C). + */ + u32 chv_dpll_md[I915_MAX_PIPES]; + u32 bxt_phy_grc; + } state; + struct { /* ordered wq for modesets */ struct workqueue_struct *modeset; diff --git a/drivers/gpu/drm/i915/display/intel_display_limits.h b/drivers/gpu/drm/i915/display/intel_display_limits.h new file mode 100644 index 0000000000000000000000000000000000000000..5126d0b5ae5dc3045e9a7aa04f7f56f2414eb241 --- /dev/null +++ b/drivers/gpu/drm/i915/display/intel_display_limits.h @@ -0,0 +1,124 @@ +/* SPDX-License-Identifier: MIT */ +/* + * Copyright © 2022 Intel Corporation + */ + +#ifndef __INTEL_DISPLAY_LIMITS_H__ +#define __INTEL_DISPLAY_LIMITS_H__ + +/* + * Keep the pipe enum values fixed: the code assumes that PIPE_A=0, the + * rest have consecutive values and match the enum values of transcoders + * with a 1:1 transcoder -> pipe mapping. + */ +enum pipe { + INVALID_PIPE = -1, + + PIPE_A = 0, + PIPE_B, + PIPE_C, + PIPE_D, + _PIPE_EDP, + + I915_MAX_PIPES = _PIPE_EDP +}; + +enum transcoder { + INVALID_TRANSCODER = -1, + /* + * The following transcoders have a 1:1 transcoder -> pipe mapping, + * keep their values fixed: the code assumes that TRANSCODER_A=0, the + * rest have consecutive values and match the enum values of the pipes + * they map to. + */ + TRANSCODER_A = PIPE_A, + TRANSCODER_B = PIPE_B, + TRANSCODER_C = PIPE_C, + TRANSCODER_D = PIPE_D, + + /* + * The following transcoders can map to any pipe, their enum value + * doesn't need to stay fixed. + */ + TRANSCODER_EDP, + TRANSCODER_DSI_0, + TRANSCODER_DSI_1, + TRANSCODER_DSI_A = TRANSCODER_DSI_0, /* legacy DSI */ + TRANSCODER_DSI_C = TRANSCODER_DSI_1, /* legacy DSI */ + + I915_MAX_TRANSCODERS +}; + +/* + * Per-pipe plane identifier. + * I915_MAX_PLANES in the enum below is the maximum (across all platforms) + * number of planes per CRTC. Not all platforms really have this many planes, + * which means some arrays of size I915_MAX_PLANES may have unused entries + * between the topmost sprite plane and the cursor plane. + * + * This is expected to be passed to various register macros + * (eg. PLANE_CTL(), PS_PLANE_SEL(), etc.) so adjust with care. + */ +enum plane_id { + PLANE_PRIMARY, + PLANE_SPRITE0, + PLANE_SPRITE1, + PLANE_SPRITE2, + PLANE_SPRITE3, + PLANE_SPRITE4, + PLANE_SPRITE5, + PLANE_CURSOR, + + I915_MAX_PLANES, +}; + +enum port { + PORT_NONE = -1, + + PORT_A = 0, + PORT_B, + PORT_C, + PORT_D, + PORT_E, + PORT_F, + PORT_G, + PORT_H, + PORT_I, + + /* tgl+ */ + PORT_TC1 = PORT_D, + PORT_TC2, + PORT_TC3, + PORT_TC4, + PORT_TC5, + PORT_TC6, + + /* XE_LPD repositions D/E offsets and bitfields */ + PORT_D_XELPD = PORT_TC5, + PORT_E_XELPD, + + I915_MAX_PORTS +}; + +enum hpd_pin { + HPD_NONE = 0, + HPD_TV = HPD_NONE, /* TV is known to be unreliable */ + HPD_CRT, + HPD_SDVO_B, + HPD_SDVO_C, + HPD_PORT_A, + HPD_PORT_B, + HPD_PORT_C, + HPD_PORT_D, + HPD_PORT_E, + HPD_PORT_TC1, + HPD_PORT_TC2, + HPD_PORT_TC3, + HPD_PORT_TC4, + HPD_PORT_TC5, + HPD_PORT_TC6, + + HPD_NUM_PINS +}; + +#endif /* __INTEL_DISPLAY_LIMITS_H__ */ diff --git a/drivers/gpu/drm/i915/display/intel_display_power_map.c b/drivers/gpu/drm/i915/display/intel_display_power_map.c index f5d66ca85b19b75850f8cf0c337539202be41e6a..6645eb1911d8507ea161c93d3a6629180e9e7bc1 100644 --- a/drivers/gpu/drm/i915/display/intel_display_power_map.c +++ b/drivers/gpu/drm/i915/display/intel_display_power_map.c @@ -10,6 +10,7 @@ #include "intel_display_power_map.h" #include "intel_display_power_well.h" +#include "intel_display_types.h" #define __LIST_INLINE_ELEMS(__elem_type, ...) \ ((__elem_type[]) { __VA_ARGS__ }) diff --git a/drivers/gpu/drm/i915/display/intel_display_trace.h b/drivers/gpu/drm/i915/display/intel_display_trace.h index 725aba3fa531378ff774c8dc316f24e25d6d7289..651ea8564e1b7d379abef7cf4028f93f4a14b471 100644 --- a/drivers/gpu/drm/i915/display/intel_display_trace.h +++ b/drivers/gpu/drm/i915/display/intel_display_trace.h @@ -17,6 +17,7 @@ #include "i915_irq.h" #include "intel_crtc.h" #include "intel_display_types.h" +#include "intel_vblank.h" #define __dev_name_i915(i915) dev_name((i915)->drm.dev) #define __dev_name_kms(obj) dev_name((obj)->base.dev->dev) diff --git a/drivers/gpu/drm/i915/display/intel_display_types.h b/drivers/gpu/drm/i915/display/intel_display_types.h index 32e8b2fc3cc642f1e02b9308956b6f02e32bf824..54c517ca9632fb1be3150c40a3935cbbbf63956c 100644 --- a/drivers/gpu/drm/i915/display/intel_display_types.h +++ b/drivers/gpu/drm/i915/display/intel_display_types.h @@ -50,6 +50,7 @@ #include "i915_vma_types.h" #include "intel_bios.h" #include "intel_display.h" +#include "intel_display_limits.h" #include "intel_display_power.h" #include "intel_dpll_mgr.h" #include "intel_pm_types.h" @@ -262,8 +263,6 @@ struct intel_encoder { enum hpd_pin hpd_pin; enum intel_display_power_domain power_domain; - /* for communication with audio component; protected by av_mutex */ - const struct drm_connector *audio_connector; /* VBT information for this encoder (may be NULL for older platforms) */ const struct intel_bios_encoder_data *devdata; @@ -351,6 +350,9 @@ struct intel_vbt_panel_data { }; struct intel_panel { + /* Fixed EDID for eDP and LVDS. May hold ERR_PTR for invalid EDID. */ + const struct drm_edid *fixed_edid; + struct list_head fixed_modes; /* backlight */ @@ -591,9 +593,8 @@ struct intel_connector { /* Panel info for eDP and LVDS */ struct intel_panel panel; - /* Cached EDID for eDP and LVDS. May hold ERR_PTR for invalid EDID. */ - struct edid *edid; - struct edid *detect_edid; + /* Cached EDID for detect. */ + const struct drm_edid *detect_edid; /* Number of times hotplug detection was tried after an HPD interrupt */ int hotplug_retries; @@ -1261,6 +1262,8 @@ struct intel_crtc_state { struct drm_dp_vsc_sdp vsc; } infoframes; + u8 eld[MAX_ELD_BYTES]; + /* HDMI scrambling status */ bool hdmi_scrambling; diff --git a/drivers/gpu/drm/i915/display/intel_dmc.c b/drivers/gpu/drm/i915/display/intel_dmc.c index 4124b3d371109ae47685a0490592edb39dfa925a..257aa2b7cf2045d116a98088f8fa6db5a36935dd 100644 --- a/drivers/gpu/drm/i915/display/intel_dmc.c +++ b/drivers/gpu/drm/i915/display/intel_dmc.c @@ -42,51 +42,61 @@ #define DMC_VERSION_MAJOR(version) ((version) >> 16) #define DMC_VERSION_MINOR(version) ((version) & 0xffff) -#define DMC_PATH(platform, major, minor) \ - "i915/" \ - __stringify(platform) "_dmc_ver" \ - __stringify(major) "_" \ +#define DMC_PATH(platform) \ + "i915/" __stringify(platform) "_dmc.bin" + +/* + * New DMC additions should not use this. This is used solely to remain + * compatible with systems that have not yet updated DMC blobs to use + * unversioned file names. + */ +#define DMC_LEGACY_PATH(platform, major, minor) \ + "i915/" \ + __stringify(platform) "_dmc_ver" \ + __stringify(major) "_" \ __stringify(minor) ".bin" #define DISPLAY_VER13_DMC_MAX_FW_SIZE 0x20000 #define DISPLAY_VER12_DMC_MAX_FW_SIZE ICL_DMC_MAX_FW_SIZE -#define DG2_DMC_PATH DMC_PATH(dg2, 2, 08) +#define DG2_DMC_PATH DMC_LEGACY_PATH(dg2, 2, 08) MODULE_FIRMWARE(DG2_DMC_PATH); -#define ADLP_DMC_PATH DMC_PATH(adlp, 2, 16) +#define ADLP_DMC_PATH DMC_PATH(adlp) +#define ADLP_DMC_FALLBACK_PATH DMC_LEGACY_PATH(adlp, 2, 16) MODULE_FIRMWARE(ADLP_DMC_PATH); +MODULE_FIRMWARE(ADLP_DMC_FALLBACK_PATH); -#define ADLS_DMC_PATH DMC_PATH(adls, 2, 01) +#define ADLS_DMC_PATH DMC_LEGACY_PATH(adls, 2, 01) MODULE_FIRMWARE(ADLS_DMC_PATH); -#define DG1_DMC_PATH DMC_PATH(dg1, 2, 02) +#define DG1_DMC_PATH DMC_LEGACY_PATH(dg1, 2, 02) MODULE_FIRMWARE(DG1_DMC_PATH); -#define RKL_DMC_PATH DMC_PATH(rkl, 2, 03) +#define RKL_DMC_PATH DMC_LEGACY_PATH(rkl, 2, 03) MODULE_FIRMWARE(RKL_DMC_PATH); -#define TGL_DMC_PATH DMC_PATH(tgl, 2, 12) +#define TGL_DMC_PATH DMC_LEGACY_PATH(tgl, 2, 12) MODULE_FIRMWARE(TGL_DMC_PATH); -#define ICL_DMC_PATH DMC_PATH(icl, 1, 09) +#define ICL_DMC_PATH DMC_LEGACY_PATH(icl, 1, 09) #define ICL_DMC_MAX_FW_SIZE 0x6000 MODULE_FIRMWARE(ICL_DMC_PATH); -#define GLK_DMC_PATH DMC_PATH(glk, 1, 04) +#define GLK_DMC_PATH DMC_LEGACY_PATH(glk, 1, 04) #define GLK_DMC_MAX_FW_SIZE 0x4000 MODULE_FIRMWARE(GLK_DMC_PATH); -#define KBL_DMC_PATH DMC_PATH(kbl, 1, 04) +#define KBL_DMC_PATH DMC_LEGACY_PATH(kbl, 1, 04) #define KBL_DMC_MAX_FW_SIZE BXT_DMC_MAX_FW_SIZE MODULE_FIRMWARE(KBL_DMC_PATH); -#define SKL_DMC_PATH DMC_PATH(skl, 1, 27) +#define SKL_DMC_PATH DMC_LEGACY_PATH(skl, 1, 27) #define SKL_DMC_MAX_FW_SIZE BXT_DMC_MAX_FW_SIZE MODULE_FIRMWARE(SKL_DMC_PATH); -#define BXT_DMC_PATH DMC_PATH(bxt, 1, 07) +#define BXT_DMC_PATH DMC_LEGACY_PATH(bxt, 1, 07) #define BXT_DMC_MAX_FW_SIZE 0x3000 MODULE_FIRMWARE(BXT_DMC_PATH); @@ -97,6 +107,8 @@ MODULE_FIRMWARE(BXT_DMC_PATH); #define DMC_V3_MAX_MMIO_COUNT 20 #define DMC_V1_MMIO_START_RANGE 0x80000 +#define PIPE_TO_DMC_ID(pipe) (DMC_FW_PIPEA + ((pipe) - PIPE_A)) + struct intel_css_header { /* 0x09 for DMC */ u32 module_type; @@ -396,6 +408,28 @@ static void pipedmc_clock_gating_wa(struct drm_i915_private *i915, bool enable) PIPEDMC_GATING_DIS, 0); } +void intel_dmc_enable_pipe(struct drm_i915_private *i915, enum pipe pipe) +{ + if (!has_dmc_id_fw(i915, PIPE_TO_DMC_ID(pipe))) + return; + + if (DISPLAY_VER(i915) >= 14) + intel_de_rmw(i915, MTL_PIPEDMC_CONTROL, 0, PIPEDMC_ENABLE_MTL(pipe)); + else + intel_de_rmw(i915, PIPEDMC_CONTROL(pipe), 0, PIPEDMC_ENABLE); +} + +void intel_dmc_disable_pipe(struct drm_i915_private *i915, enum pipe pipe) +{ + if (!has_dmc_id_fw(i915, PIPE_TO_DMC_ID(pipe))) + return; + + if (DISPLAY_VER(i915) >= 14) + intel_de_rmw(i915, MTL_PIPEDMC_CONTROL, PIPEDMC_ENABLE_MTL(pipe), 0); + else + intel_de_rmw(i915, PIPEDMC_CONTROL(pipe), PIPEDMC_ENABLE, 0); +} + /** * intel_dmc_load_program() - write the firmware from memory to register. * @dev_priv: i915 drm device. @@ -821,16 +855,40 @@ static void intel_dmc_runtime_pm_put(struct drm_i915_private *dev_priv) intel_display_power_put(dev_priv, POWER_DOMAIN_INIT, wakeref); } +static const char *dmc_fallback_path(struct drm_i915_private *i915) +{ + if (IS_ALDERLAKE_P(i915)) + return ADLP_DMC_FALLBACK_PATH; + + return NULL; +} + static void dmc_load_work_fn(struct work_struct *work) { struct drm_i915_private *dev_priv; struct intel_dmc *dmc; const struct firmware *fw = NULL; + const char *fallback_path; + int err; dev_priv = container_of(work, typeof(*dev_priv), display.dmc.work); dmc = &dev_priv->display.dmc; - request_firmware(&fw, dev_priv->display.dmc.fw_path, dev_priv->drm.dev); + err = request_firmware(&fw, dev_priv->display.dmc.fw_path, dev_priv->drm.dev); + + if (err == -ENOENT && !dev_priv->params.dmc_firmware_path) { + fallback_path = dmc_fallback_path(dev_priv); + if (fallback_path) { + drm_dbg_kms(&dev_priv->drm, + "%s not found, falling back to %s\n", + dmc->fw_path, + fallback_path); + err = request_firmware(&fw, fallback_path, dev_priv->drm.dev); + if (err == 0) + dev_priv->display.dmc.fw_path = fallback_path; + } + } + parse_dmc_fw(dev_priv, fw); if (intel_dmc_has_payload(dev_priv)) { diff --git a/drivers/gpu/drm/i915/display/intel_dmc.h b/drivers/gpu/drm/i915/display/intel_dmc.h index 435eab9b016b427c70307eb52a8ca15b6079d1de..fd1725de42891f89e611627d17201743eecae1cf 100644 --- a/drivers/gpu/drm/i915/display/intel_dmc.h +++ b/drivers/gpu/drm/i915/display/intel_dmc.h @@ -13,6 +13,8 @@ struct drm_i915_error_state_buf; struct drm_i915_private; +enum pipe; + enum { DMC_FW_MAIN = 0, DMC_FW_PIPEA, @@ -47,6 +49,8 @@ struct intel_dmc { void intel_dmc_ucode_init(struct drm_i915_private *i915); void intel_dmc_load_program(struct drm_i915_private *i915); void intel_dmc_disable_program(struct drm_i915_private *i915); +void intel_dmc_enable_pipe(struct drm_i915_private *i915, enum pipe pipe); +void intel_dmc_disable_pipe(struct drm_i915_private *i915, enum pipe pipe); void intel_dmc_ucode_fini(struct drm_i915_private *i915); void intel_dmc_ucode_suspend(struct drm_i915_private *i915); void intel_dmc_ucode_resume(struct drm_i915_private *i915); diff --git a/drivers/gpu/drm/i915/display/intel_dmc_regs.h b/drivers/gpu/drm/i915/display/intel_dmc_regs.h index 5e5e41644ddfdb0e787ec9393672572d794da2c9..cf10094acae3d42f15988304a63e999fc31d69e6 100644 --- a/drivers/gpu/drm/i915/display/intel_dmc_regs.h +++ b/drivers/gpu/drm/i915/display/intel_dmc_regs.h @@ -11,6 +11,16 @@ #define DMC_PROGRAM(addr, i) _MMIO((addr) + (i) * 4) #define DMC_SSP_BASE_ADDR_GEN9 0x00002FC0 +#define _PIPEDMC_CONTROL_A 0x45250 +#define _PIPEDMC_CONTROL_B 0x45254 +#define PIPEDMC_CONTROL(pipe) _MMIO_PIPE(pipe, \ + _PIPEDMC_CONTROL_A, \ + _PIPEDMC_CONTROL_B) +#define PIPEDMC_ENABLE REG_BIT(0) + +#define MTL_PIPEDMC_CONTROL _MMIO(0x45250) +#define PIPEDMC_ENABLE_MTL(pipe) REG_BIT(((pipe) - PIPE_A) * 4) + #define _ADLP_PIPEDMC_REG_MMIO_BASE_A 0x5f000 #define _TGL_PIPEDMC_REG_MMIO_BASE_A 0x92000 diff --git a/drivers/gpu/drm/i915/display/intel_dp.c b/drivers/gpu/drm/i915/display/intel_dp.c index 80d95cec8f9dd38047a90d19231ca987f8f24596..62cbab7402e938d1d7d3a25b8aabb8723ef47fd7 100644 --- a/drivers/gpu/drm/i915/display/intel_dp.c +++ b/drivers/gpu/drm/i915/display/intel_dp.c @@ -2080,7 +2080,9 @@ intel_dp_compute_config(struct intel_encoder *encoder, if (HAS_PCH_SPLIT(dev_priv) && !HAS_DDI(dev_priv) && encoder->port != PORT_A) pipe_config->has_pch_encoder = true; - pipe_config->has_audio = intel_dp_has_audio(encoder, pipe_config, conn_state); + pipe_config->has_audio = + intel_dp_has_audio(encoder, pipe_config, conn_state) && + intel_audio_compute_config(encoder, pipe_config, conn_state); fixed_mode = intel_panel_fixed_mode(connector, adjusted_mode); if (intel_dp_is_edp(intel_dp) && fixed_mode) { @@ -3648,12 +3650,11 @@ static u8 intel_dp_autotest_edid(struct intel_dp *intel_dp) intel_dp->aux.i2c_defer_count); intel_dp->compliance.test_data.edid = INTEL_DP_RESOLUTION_FAILSAFE; } else { - struct edid *block = intel_connector->detect_edid; + /* FIXME: Get rid of drm_edid_raw() */ + const struct edid *block = drm_edid_raw(intel_connector->detect_edid); - /* We have to write the checksum - * of the last block read - */ - block += intel_connector->detect_edid->extensions; + /* We have to write the checksum of the last block read */ + block += block->extensions; if (drm_dp_dpcd_writeb(&intel_dp->aux, DP_TEST_EDID_CHECKSUM, block->checksum) <= 0) @@ -4475,29 +4476,34 @@ bool intel_digital_port_connected(struct intel_encoder *encoder) return is_connected; } -static struct edid * +static const struct drm_edid * intel_dp_get_edid(struct intel_dp *intel_dp) { - struct intel_connector *intel_connector = intel_dp->attached_connector; + struct intel_connector *connector = intel_dp->attached_connector; + const struct drm_edid *fixed_edid = connector->panel.fixed_edid; - /* use cached edid if we have one */ - if (intel_connector->edid) { + /* Use panel fixed edid if we have one */ + if (fixed_edid) { /* invalid edid */ - if (IS_ERR(intel_connector->edid)) + if (IS_ERR(fixed_edid)) return NULL; - return drm_edid_duplicate(intel_connector->edid); - } else - return drm_get_edid(&intel_connector->base, - &intel_dp->aux.ddc); + return drm_edid_dup(fixed_edid); + } + + return drm_edid_read_ddc(&connector->base, &intel_dp->aux.ddc); } static void intel_dp_update_dfp(struct intel_dp *intel_dp, - const struct edid *edid) + const struct drm_edid *drm_edid) { struct drm_i915_private *i915 = dp_to_i915(intel_dp); struct intel_connector *connector = intel_dp->attached_connector; + const struct edid *edid; + + /* FIXME: Get rid of drm_edid_raw() */ + edid = drm_edid_raw(drm_edid); intel_dp->dfp.max_bpc = drm_dp_downstream_max_bpc(intel_dp->dpcd, @@ -4597,21 +4603,27 @@ intel_dp_set_edid(struct intel_dp *intel_dp) { struct drm_i915_private *i915 = dp_to_i915(intel_dp); struct intel_connector *connector = intel_dp->attached_connector; - struct edid *edid; + const struct drm_edid *drm_edid; + const struct edid *edid; bool vrr_capable; intel_dp_unset_edid(intel_dp); - edid = intel_dp_get_edid(intel_dp); - connector->detect_edid = edid; + drm_edid = intel_dp_get_edid(intel_dp); + connector->detect_edid = drm_edid; + + /* Below we depend on display info having been updated */ + drm_edid_connector_update(&connector->base, drm_edid); vrr_capable = intel_vrr_is_capable(connector); drm_dbg_kms(&i915->drm, "[CONNECTOR:%d:%s] VRR capable: %s\n", connector->base.base.id, connector->base.name, str_yes_no(vrr_capable)); drm_connector_set_vrr_capable_property(&connector->base, vrr_capable); - intel_dp_update_dfp(intel_dp, edid); + intel_dp_update_dfp(intel_dp, drm_edid); intel_dp_update_420(intel_dp); + /* FIXME: Get rid of drm_edid_raw() */ + edid = drm_edid_raw(drm_edid); if (edid && edid->input & DRM_EDID_INPUT_DIGITAL) { intel_dp->has_hdmi_sink = drm_detect_hdmi_monitor(edid); intel_dp->has_audio = drm_detect_monitor_audio(edid); @@ -4626,7 +4638,7 @@ intel_dp_unset_edid(struct intel_dp *intel_dp) struct intel_connector *connector = intel_dp->attached_connector; drm_dp_cec_unset_edid(&intel_dp->aux); - kfree(connector->detect_edid); + drm_edid_free(connector->detect_edid); connector->detect_edid = NULL; intel_dp->has_hdmi_sink = false; @@ -4790,12 +4802,10 @@ intel_dp_force(struct drm_connector *connector) static int intel_dp_get_modes(struct drm_connector *connector) { struct intel_connector *intel_connector = to_intel_connector(connector); - struct edid *edid; - int num_modes = 0; + int num_modes; - edid = intel_connector->detect_edid; - if (edid) - num_modes = intel_connector_update_modes(connector, edid); + /* drm_edid_connector_update() done in ->detect() or ->force() */ + num_modes = drm_edid_connector_add_modes(connector); /* Also add fixed mode, which may or may not be present in EDID */ if (intel_dp_is_edp(intel_attached_dp(intel_connector))) @@ -4804,7 +4814,7 @@ static int intel_dp_get_modes(struct drm_connector *connector) if (num_modes) return num_modes; - if (!edid) { + if (!intel_connector->detect_edid) { struct intel_dp *intel_dp = intel_attached_dp(intel_connector); struct drm_display_mode *mode; @@ -5240,7 +5250,7 @@ static bool intel_edp_init_connector(struct intel_dp *intel_dp, struct drm_display_mode *fixed_mode; struct intel_encoder *encoder = &dp_to_dig_port(intel_dp)->base; bool has_dpcd; - struct edid *edid; + const struct drm_edid *drm_edid; if (!intel_dp_is_edp(intel_dp)) return true; @@ -5287,29 +5297,28 @@ static bool intel_edp_init_connector(struct intel_dp *intel_dp, } mutex_lock(&dev_priv->drm.mode_config.mutex); - edid = drm_get_edid(connector, &intel_dp->aux.ddc); - if (!edid) { + drm_edid = drm_edid_read_ddc(connector, &intel_dp->aux.ddc); + if (!drm_edid) { /* Fallback to EDID from ACPI OpRegion, if any */ - edid = intel_opregion_get_edid(intel_connector); - if (edid) + drm_edid = intel_opregion_get_edid(intel_connector); + if (drm_edid) drm_dbg_kms(&dev_priv->drm, "[CONNECTOR:%d:%s] Using OpRegion EDID\n", connector->base.id, connector->name); } - if (edid) { - if (drm_add_edid_modes(connector, edid)) { - drm_connector_update_edid_property(connector, edid); - } else { - kfree(edid); - edid = ERR_PTR(-EINVAL); + if (drm_edid) { + if (drm_edid_connector_update(connector, drm_edid) || + !drm_edid_connector_add_modes(connector)) { + drm_edid_connector_update(connector, NULL); + drm_edid_free(drm_edid); + drm_edid = ERR_PTR(-EINVAL); } } else { - edid = ERR_PTR(-ENOENT); + drm_edid = ERR_PTR(-ENOENT); } - intel_connector->edid = edid; - intel_bios_init_panel_late(dev_priv, &intel_connector->panel, - encoder->devdata, IS_ERR(edid) ? NULL : edid); + intel_bios_init_panel_late(dev_priv, &intel_connector->panel, encoder->devdata, + IS_ERR(drm_edid) ? NULL : drm_edid); intel_panel_add_edid_fixed_modes(intel_connector, true); @@ -5333,7 +5342,7 @@ static bool intel_edp_init_connector(struct intel_dp *intel_dp, goto out_vdd_off; } - intel_panel_init(intel_connector); + intel_panel_init(intel_connector, drm_edid); intel_edp_backlight_setup(intel_dp, intel_connector); diff --git a/drivers/gpu/drm/i915/display/intel_dpio_phy.c b/drivers/gpu/drm/i915/display/intel_dpio_phy.c index 7eb7440b31803f7b46f6c82f102bd152f42683b8..565c06de2432dbc2b53d4147077d3372e02fd319 100644 --- a/drivers/gpu/drm/i915/display/intel_dpio_phy.c +++ b/drivers/gpu/drm/i915/display/intel_dpio_phy.c @@ -376,7 +376,7 @@ static void _bxt_ddi_phy_init(struct drm_i915_private *dev_priv, if (bxt_ddi_phy_is_enabled(dev_priv, phy)) { /* Still read out the GRC value for state verification */ if (phy_info->rcomp_phy != -1) - dev_priv->bxt_phy_grc = bxt_get_grc(dev_priv, phy); + dev_priv->display.state.bxt_phy_grc = bxt_get_grc(dev_priv, phy); if (bxt_ddi_phy_verify_state(dev_priv, phy)) { drm_dbg(&dev_priv->drm, "DDI PHY %d already enabled, " @@ -442,8 +442,9 @@ static void _bxt_ddi_phy_init(struct drm_i915_private *dev_priv, * the corresponding calibrated value from PHY1, and disable * the automatic calibration on PHY0. */ - val = dev_priv->bxt_phy_grc = bxt_get_grc(dev_priv, - phy_info->rcomp_phy); + val = bxt_get_grc(dev_priv, phy_info->rcomp_phy); + dev_priv->display.state.bxt_phy_grc = val; + grc_code = val << GRC_CODE_FAST_SHIFT | val << GRC_CODE_SLOW_SHIFT | val; @@ -568,7 +569,7 @@ bool bxt_ddi_phy_verify_state(struct drm_i915_private *dev_priv, "BXT_PORT_CL2CM_DW6(%d)", phy); if (phy_info->rcomp_phy != -1) { - u32 grc_code = dev_priv->bxt_phy_grc; + u32 grc_code = dev_priv->display.state.bxt_phy_grc; grc_code = grc_code << GRC_CODE_FAST_SHIFT | grc_code << GRC_CODE_SLOW_SHIFT | diff --git a/drivers/gpu/drm/i915/display/intel_dpll.c b/drivers/gpu/drm/i915/display/intel_dpll.c index c236aafe9be0151c1f455a03ae93e3ae960fcf16..4e9c18be7e1fd1f0b8d20d95424b8e200020c25a 100644 --- a/drivers/gpu/drm/i915/display/intel_dpll.c +++ b/drivers/gpu/drm/i915/display/intel_dpll.c @@ -1910,7 +1910,7 @@ void chv_enable_pll(const struct intel_crtc_state *crtc_state) intel_de_write(dev_priv, DPLL_MD(PIPE_B), crtc_state->dpll_hw_state.dpll_md); intel_de_write(dev_priv, CBR4_VLV, 0); - dev_priv->chv_dpll_md[pipe] = crtc_state->dpll_hw_state.dpll_md; + dev_priv->display.state.chv_dpll_md[pipe] = crtc_state->dpll_hw_state.dpll_md; /* * DPLLB VGA mode also seems to cause problems. diff --git a/drivers/gpu/drm/i915/display/intel_dpll_mgr.c b/drivers/gpu/drm/i915/display/intel_dpll_mgr.c index 1974eb580ed19812b1fafadb1f5597202e5bf67b..380368eff31a20f6d0a364098c0dc681900205bc 100644 --- a/drivers/gpu/drm/i915/display/intel_dpll_mgr.c +++ b/drivers/gpu/drm/i915/display/intel_dpll_mgr.c @@ -618,7 +618,7 @@ static void hsw_ddi_wrpll_disable(struct drm_i915_private *dev_priv, * Try to set up the PCH reference clock once all DPLLs * that depend on it have been shut down. */ - if (dev_priv->pch_ssc_use & BIT(id)) + if (dev_priv->display.dpll.pch_ssc_use & BIT(id)) intel_init_pch_refclk(dev_priv); } @@ -636,7 +636,7 @@ static void hsw_ddi_spll_disable(struct drm_i915_private *dev_priv, * Try to set up the PCH reference clock once all DPLLs * that depend on it have been shut down. */ - if (dev_priv->pch_ssc_use & BIT(id)) + if (dev_priv->display.dpll.pch_ssc_use & BIT(id)) intel_init_pch_refclk(dev_priv); } diff --git a/drivers/gpu/drm/i915/display/intel_drrs.c b/drivers/gpu/drm/i915/display/intel_drrs.c index 5b9e44443814e97462310746df4698a1b332b9b0..29c6421cd666002104bfc114053eff163bb44388 100644 --- a/drivers/gpu/drm/i915/display/intel_drrs.c +++ b/drivers/gpu/drm/i915/display/intel_drrs.c @@ -374,16 +374,16 @@ static int intel_drrs_debugfs_ctl_set(void *data, u64 val) return ret; } -DEFINE_SIMPLE_ATTRIBUTE(intel_drrs_debugfs_ctl_fops, - NULL, intel_drrs_debugfs_ctl_set, "%llu\n"); +DEFINE_DEBUGFS_ATTRIBUTE(intel_drrs_debugfs_ctl_fops, + NULL, intel_drrs_debugfs_ctl_set, "%llu\n"); void intel_drrs_crtc_debugfs_add(struct intel_crtc *crtc) { debugfs_create_file("i915_drrs_status", 0444, crtc->base.debugfs_entry, crtc, &intel_drrs_debugfs_status_fops); - debugfs_create_file("i915_drrs_ctl", 0644, crtc->base.debugfs_entry, - crtc, &intel_drrs_debugfs_ctl_fops); + debugfs_create_file_unsafe("i915_drrs_ctl", 0644, crtc->base.debugfs_entry, + crtc, &intel_drrs_debugfs_ctl_fops); } static int intel_drrs_debugfs_type_show(struct seq_file *m, void *unused) diff --git a/drivers/gpu/drm/i915/display/intel_dsb.c b/drivers/gpu/drm/i915/display/intel_dsb.c index 3d63c1bf1e4f9c2fd5d2bd9910491234b4b75ba0..96bc117fd6a0ba9e5415ca675588e2dc23dd0ca4 100644 --- a/drivers/gpu/drm/i915/display/intel_dsb.c +++ b/drivers/gpu/drm/i915/display/intel_dsb.c @@ -30,21 +30,24 @@ struct intel_dsb { struct intel_crtc *crtc; /* - * free_pos will point the first free entry position - * and help in calculating tail of command buffer. + * maximum number of dwords the buffer will hold. */ - int free_pos; + unsigned int size; /* - * ins_start_offset will help to store start address of the dsb + * free_pos will point the first free dword and + * help in calculating tail of command buffer. + */ + unsigned int free_pos; + + /* + * ins_start_offset will help to store start dword of the dsb * instuction and help in identifying the batch of auto-increment * register. */ - u32 ins_start_offset; + unsigned int ins_start_offset; }; -#define DSB_BUF_SIZE (2 * PAGE_SIZE) - /** * DOC: DSB * @@ -64,80 +67,86 @@ struct intel_dsb { /* DSB opcodes. */ #define DSB_OPCODE_SHIFT 24 +#define DSB_OPCODE_NOOP 0x0 #define DSB_OPCODE_MMIO_WRITE 0x1 +#define DSB_OPCODE_WAIT_USEC 0x2 +#define DSB_OPCODE_WAIT_LINES 0x3 +#define DSB_OPCODE_WAIT_VBLANKS 0x4 +#define DSB_OPCODE_WAIT_DSL_IN 0x5 +#define DSB_OPCODE_WAIT_DSL_OUT 0x6 +#define DSB_OPCODE_INTERRUPT 0x7 #define DSB_OPCODE_INDEXED_WRITE 0x9 +#define DSB_OPCODE_POLL 0xA #define DSB_BYTE_EN 0xF #define DSB_BYTE_EN_SHIFT 20 #define DSB_REG_VALUE_MASK 0xfffff +static bool assert_dsb_has_room(struct intel_dsb *dsb) +{ + struct intel_crtc *crtc = dsb->crtc; + struct drm_i915_private *i915 = to_i915(crtc->base.dev); + + /* each instruction is 2 dwords */ + return !drm_WARN(&i915->drm, dsb->free_pos > dsb->size - 2, + "DSB buffer overflow\n"); +} + static bool is_dsb_busy(struct drm_i915_private *i915, enum pipe pipe, enum dsb_id id) { - return DSB_STATUS & intel_de_read(i915, DSB_CTRL(pipe, id)); + return intel_de_read(i915, DSB_CTRL(pipe, id)) & DSB_STATUS_BUSY; } -static bool intel_dsb_enable_engine(struct drm_i915_private *i915, - enum pipe pipe, enum dsb_id id) +static void intel_dsb_emit(struct intel_dsb *dsb, u32 ldw, u32 udw) { - u32 dsb_ctrl; + u32 *buf = dsb->cmd_buf; - dsb_ctrl = intel_de_read(i915, DSB_CTRL(pipe, id)); - if (DSB_STATUS & dsb_ctrl) { - drm_dbg_kms(&i915->drm, "DSB engine is busy.\n"); - return false; - } + if (!assert_dsb_has_room(dsb)) + return; - dsb_ctrl |= DSB_ENABLE; - intel_de_write(i915, DSB_CTRL(pipe, id), dsb_ctrl); + /* Every instruction should be 8 byte aligned. */ + dsb->free_pos = ALIGN(dsb->free_pos, 2); + + dsb->ins_start_offset = dsb->free_pos; - intel_de_posting_read(i915, DSB_CTRL(pipe, id)); - return true; + buf[dsb->free_pos++] = ldw; + buf[dsb->free_pos++] = udw; } -static bool intel_dsb_disable_engine(struct drm_i915_private *i915, - enum pipe pipe, enum dsb_id id) +static bool intel_dsb_prev_ins_is_write(struct intel_dsb *dsb, + u32 opcode, i915_reg_t reg) { - u32 dsb_ctrl; + const u32 *buf = dsb->cmd_buf; + u32 prev_opcode, prev_reg; - dsb_ctrl = intel_de_read(i915, DSB_CTRL(pipe, id)); - if (DSB_STATUS & dsb_ctrl) { - drm_dbg_kms(&i915->drm, "DSB engine is busy.\n"); - return false; - } + prev_opcode = buf[dsb->ins_start_offset + 1] >> DSB_OPCODE_SHIFT; + prev_reg = buf[dsb->ins_start_offset + 1] & DSB_REG_VALUE_MASK; + + return prev_opcode == opcode && prev_reg == i915_mmio_reg_offset(reg); +} - dsb_ctrl &= ~DSB_ENABLE; - intel_de_write(i915, DSB_CTRL(pipe, id), dsb_ctrl); +static bool intel_dsb_prev_ins_is_mmio_write(struct intel_dsb *dsb, i915_reg_t reg) +{ + return intel_dsb_prev_ins_is_write(dsb, DSB_OPCODE_MMIO_WRITE, reg); +} - intel_de_posting_read(i915, DSB_CTRL(pipe, id)); - return true; +static bool intel_dsb_prev_ins_is_indexed_write(struct intel_dsb *dsb, i915_reg_t reg) +{ + return intel_dsb_prev_ins_is_write(dsb, DSB_OPCODE_INDEXED_WRITE, reg); } /** - * intel_dsb_indexed_reg_write() -Write to the DSB context for auto - * increment register. + * intel_dsb_reg_write() - Emit register wriite to the DSB context * @dsb: DSB context * @reg: register address. * @val: value. * * This function is used for writing register-value pair in command - * buffer of DSB for auto-increment register. During command buffer overflow, - * a warning is thrown and rest all erroneous condition register programming - * is done through mmio write. + * buffer of DSB. */ - -void intel_dsb_indexed_reg_write(struct intel_dsb *dsb, - i915_reg_t reg, u32 val) +void intel_dsb_reg_write(struct intel_dsb *dsb, + i915_reg_t reg, u32 val) { - struct intel_crtc *crtc = dsb->crtc; - struct drm_i915_private *dev_priv = to_i915(crtc->base.dev); - u32 *buf = dsb->cmd_buf; - u32 reg_val; - - if (drm_WARN_ON(&dev_priv->drm, dsb->free_pos >= DSB_BUF_SIZE)) { - drm_dbg_kms(&dev_priv->drm, "DSB buffer overflow\n"); - return; - } - /* * For example the buffer will look like below for 3 dwords for auto * increment register: @@ -154,65 +163,55 @@ void intel_dsb_indexed_reg_write(struct intel_dsb *dsb, * we are writing odd no of dwords, Zeros will be added in the end for * padding. */ - reg_val = buf[dsb->ins_start_offset + 1] & DSB_REG_VALUE_MASK; - if (reg_val != i915_mmio_reg_offset(reg)) { - /* Every instruction should be 8 byte aligned. */ - dsb->free_pos = ALIGN(dsb->free_pos, 2); + if (!intel_dsb_prev_ins_is_mmio_write(dsb, reg) && + !intel_dsb_prev_ins_is_indexed_write(dsb, reg)) { + intel_dsb_emit(dsb, val, + (DSB_OPCODE_MMIO_WRITE << DSB_OPCODE_SHIFT) | + (DSB_BYTE_EN << DSB_BYTE_EN_SHIFT) | + i915_mmio_reg_offset(reg)); + } else { + u32 *buf = dsb->cmd_buf; - dsb->ins_start_offset = dsb->free_pos; + if (!assert_dsb_has_room(dsb)) + return; - /* Update the size. */ - buf[dsb->free_pos++] = 1; + /* convert to indexed write? */ + if (intel_dsb_prev_ins_is_mmio_write(dsb, reg)) { + u32 prev_val = buf[dsb->ins_start_offset + 0]; - /* Update the opcode and reg. */ - buf[dsb->free_pos++] = (DSB_OPCODE_INDEXED_WRITE << - DSB_OPCODE_SHIFT) | - i915_mmio_reg_offset(reg); + buf[dsb->ins_start_offset + 0] = 1; /* count */ + buf[dsb->ins_start_offset + 1] = + (DSB_OPCODE_INDEXED_WRITE << DSB_OPCODE_SHIFT) | + i915_mmio_reg_offset(reg); + buf[dsb->ins_start_offset + 2] = prev_val; - /* Update the value. */ - buf[dsb->free_pos++] = val; - } else { - /* Update the new value. */ - buf[dsb->free_pos++] = val; + dsb->free_pos++; + } - /* Update the size. */ + buf[dsb->free_pos++] = val; + /* Update the count */ buf[dsb->ins_start_offset]++; - } - /* if number of data words is odd, then the last dword should be 0.*/ - if (dsb->free_pos & 0x1) - buf[dsb->free_pos] = 0; + /* if number of data words is odd, then the last dword should be 0.*/ + if (dsb->free_pos & 0x1) + buf[dsb->free_pos] = 0; + } } -/** - * intel_dsb_reg_write() -Write to the DSB context for normal - * register. - * @crtc_state: intel_crtc_state structure - * @reg: register address. - * @val: value. - * - * This function is used for writing register-value pair in command - * buffer of DSB. During command buffer overflow, a warning is thrown - * and rest all erroneous condition register programming is done - * through mmio write. - */ -void intel_dsb_reg_write(struct intel_dsb *dsb, - i915_reg_t reg, u32 val) +static u32 intel_dsb_align_tail(struct intel_dsb *dsb) { - struct intel_crtc *crtc = dsb->crtc; - struct drm_i915_private *dev_priv = to_i915(crtc->base.dev); - u32 *buf = dsb->cmd_buf; + u32 aligned_tail, tail; - if (drm_WARN_ON(&dev_priv->drm, dsb->free_pos >= DSB_BUF_SIZE)) { - drm_dbg_kms(&dev_priv->drm, "DSB buffer overflow\n"); - return; - } + tail = dsb->free_pos * 4; + aligned_tail = ALIGN(tail, CACHELINE_BYTES); - dsb->ins_start_offset = dsb->free_pos; - buf[dsb->free_pos++] = val; - buf[dsb->free_pos++] = (DSB_OPCODE_MMIO_WRITE << DSB_OPCODE_SHIFT) | - (DSB_BYTE_EN << DSB_BYTE_EN_SHIFT) | - i915_mmio_reg_offset(reg); + if (aligned_tail > tail) + memset(&dsb->cmd_buf[dsb->free_pos], 0, + aligned_tail - tail); + + dsb->free_pos = aligned_tail / 4; + + return aligned_tail; } /** @@ -228,50 +227,41 @@ void intel_dsb_commit(struct intel_dsb *dsb) enum pipe pipe = crtc->pipe; u32 tail; - if (!(dsb && dsb->free_pos)) + tail = intel_dsb_align_tail(dsb); + if (tail == 0) return; - if (!intel_dsb_enable_engine(dev_priv, pipe, dsb->id)) - goto reset; - if (is_dsb_busy(dev_priv, pipe, dsb->id)) { - drm_err(&dev_priv->drm, - "HEAD_PTR write failed - dsb engine is busy.\n"); + drm_err(&dev_priv->drm, "DSB engine is busy.\n"); goto reset; } + + intel_de_write(dev_priv, DSB_CTRL(pipe, dsb->id), + DSB_ENABLE); intel_de_write(dev_priv, DSB_HEAD(pipe, dsb->id), i915_ggtt_offset(dsb->vma)); + intel_de_write(dev_priv, DSB_TAIL(pipe, dsb->id), + i915_ggtt_offset(dsb->vma) + tail); - tail = ALIGN(dsb->free_pos * 4, CACHELINE_BYTES); - if (tail > dsb->free_pos * 4) - memset(&dsb->cmd_buf[dsb->free_pos], 0, - (tail - dsb->free_pos * 4)); - - if (is_dsb_busy(dev_priv, pipe, dsb->id)) { - drm_err(&dev_priv->drm, - "TAIL_PTR write failed - dsb engine is busy.\n"); - goto reset; - } drm_dbg_kms(&dev_priv->drm, "DSB execution started - head 0x%x, tail 0x%x\n", - i915_ggtt_offset(dsb->vma), tail); - intel_de_write(dev_priv, DSB_TAIL(pipe, dsb->id), - i915_ggtt_offset(dsb->vma) + tail); - if (wait_for(!is_dsb_busy(dev_priv, pipe, dsb->id), 1)) { + i915_ggtt_offset(dsb->vma), + i915_ggtt_offset(dsb->vma) + tail); + + if (wait_for(!is_dsb_busy(dev_priv, pipe, dsb->id), 1)) drm_err(&dev_priv->drm, "Timed out waiting for DSB workload completion.\n"); - goto reset; - } reset: dsb->free_pos = 0; dsb->ins_start_offset = 0; - intel_dsb_disable_engine(dev_priv, pipe, dsb->id); + intel_de_write(dev_priv, DSB_CTRL(pipe, dsb->id), 0); } /** * intel_dsb_prepare() - Allocate, pin and map the DSB command buffer. * @crtc: the CRTC + * @max_cmds: number of commands we need to fit into command buffer * * This function prepare the command buffer which is used to store dsb * instructions with data. @@ -279,25 +269,30 @@ void intel_dsb_commit(struct intel_dsb *dsb) * Returns: * DSB context, NULL on failure */ -struct intel_dsb *intel_dsb_prepare(struct intel_crtc *crtc) +struct intel_dsb *intel_dsb_prepare(struct intel_crtc *crtc, + unsigned int max_cmds) { struct drm_i915_private *i915 = to_i915(crtc->base.dev); - struct intel_dsb *dsb; struct drm_i915_gem_object *obj; + intel_wakeref_t wakeref; + struct intel_dsb *dsb; struct i915_vma *vma; + unsigned int size; u32 *buf; - intel_wakeref_t wakeref; if (!HAS_DSB(i915)) return NULL; - dsb = kmalloc(sizeof(*dsb), GFP_KERNEL); + dsb = kzalloc(sizeof(*dsb), GFP_KERNEL); if (!dsb) goto out; wakeref = intel_runtime_pm_get(&i915->runtime_pm); - obj = i915_gem_object_create_internal(i915, DSB_BUF_SIZE); + /* ~1 qword per instruction, full cachelines */ + size = ALIGN(max_cmds * 8, CACHELINE_BYTES); + + obj = i915_gem_object_create_internal(i915, PAGE_ALIGN(size)); if (IS_ERR(obj)) goto out_put_rpm; @@ -319,6 +314,7 @@ struct intel_dsb *intel_dsb_prepare(struct intel_crtc *crtc) dsb->vma = vma; dsb->crtc = crtc; dsb->cmd_buf = buf; + dsb->size = size / 4; /* in dwords */ dsb->free_pos = 0; dsb->ins_start_offset = 0; diff --git a/drivers/gpu/drm/i915/display/intel_dsb.h b/drivers/gpu/drm/i915/display/intel_dsb.h index 25f13c4d5389ff45b6c76c42fd3113412d2e650d..05c221b6d0a4d82fdfd7f41637a28077a496f235 100644 --- a/drivers/gpu/drm/i915/display/intel_dsb.h +++ b/drivers/gpu/drm/i915/display/intel_dsb.h @@ -13,12 +13,11 @@ struct intel_crtc; struct intel_dsb; -struct intel_dsb *intel_dsb_prepare(struct intel_crtc *crtc); +struct intel_dsb *intel_dsb_prepare(struct intel_crtc *crtc, + unsigned int max_cmds); void intel_dsb_cleanup(struct intel_dsb *dsb); void intel_dsb_reg_write(struct intel_dsb *dsb, i915_reg_t reg, u32 val); -void intel_dsb_indexed_reg_write(struct intel_dsb *dsb, - i915_reg_t reg, u32 val); void intel_dsb_commit(struct intel_dsb *dsb); #endif diff --git a/drivers/gpu/drm/i915/display/intel_dvo.c b/drivers/gpu/drm/i915/display/intel_dvo.c index 4aeae0f3ac917259b954eb0e383758433442d07e..0be8105cb18aa3fad7c96c45820140e31aed178d 100644 --- a/drivers/gpu/drm/i915/display/intel_dvo.c +++ b/drivers/gpu/drm/i915/display/intel_dvo.c @@ -554,6 +554,6 @@ void intel_dvo_init(struct drm_i915_private *i915) */ intel_panel_add_encoder_fixed_mode(connector, encoder); - intel_panel_init(connector); + intel_panel_init(connector, NULL); } } diff --git a/drivers/gpu/drm/i915/display/intel_dvo_dev.h b/drivers/gpu/drm/i915/display/intel_dvo_dev.h index ea8eb7dcee38aadf8c7c9b2ea03b7ac792cf880d..f7e98e1c64707a40213692647a02386ad5744c2f 100644 --- a/drivers/gpu/drm/i915/display/intel_dvo_dev.h +++ b/drivers/gpu/drm/i915/display/intel_dvo_dev.h @@ -25,7 +25,7 @@ #include "i915_reg_defs.h" -#include "intel_display.h" +#include "intel_display_limits.h" enum drm_connector_status; struct drm_display_mode; diff --git a/drivers/gpu/drm/i915/display/intel_fbc.c b/drivers/gpu/drm/i915/display/intel_fbc.c index 5e69d3c11d21cb5ed9d2b0fe72f3cdf3767569ba..b507ff944864ebd7a5e62818650c10338001f68d 100644 --- a/drivers/gpu/drm/i915/display/intel_fbc.c +++ b/drivers/gpu/drm/i915/display/intel_fbc.c @@ -331,15 +331,15 @@ static void i8xx_fbc_program_cfb(struct intel_fbc *fbc) { struct drm_i915_private *i915 = fbc->i915; - GEM_BUG_ON(range_overflows_end_t(u64, i915->dsm.start, + GEM_BUG_ON(range_overflows_end_t(u64, i915->dsm.stolen.start, fbc->compressed_fb.start, U32_MAX)); - GEM_BUG_ON(range_overflows_end_t(u64, i915->dsm.start, + GEM_BUG_ON(range_overflows_end_t(u64, i915->dsm.stolen.start, fbc->compressed_llb.start, U32_MAX)); intel_de_write(i915, FBC_CFB_BASE, - i915->dsm.start + fbc->compressed_fb.start); + i915->dsm.stolen.start + fbc->compressed_fb.start); intel_de_write(i915, FBC_LL_BASE, - i915->dsm.start + fbc->compressed_llb.start); + i915->dsm.stolen.start + fbc->compressed_llb.start); } static const struct intel_fbc_funcs i8xx_fbc_funcs = { @@ -712,7 +712,7 @@ static u64 intel_fbc_stolen_end(struct drm_i915_private *i915) * underruns, even if that range is not reserved by the BIOS. */ if (IS_BROADWELL(i915) || (DISPLAY_VER(i915) == 9 && !IS_BROXTON(i915))) - end = resource_size(&i915->dsm) - 8 * 1024 * 1024; + end = resource_size(&i915->dsm.stolen) - 8 * 1024 * 1024; else end = U64_MAX; @@ -1807,10 +1807,10 @@ static int intel_fbc_debugfs_false_color_set(void *data, u64 val) return 0; } -DEFINE_SIMPLE_ATTRIBUTE(intel_fbc_debugfs_false_color_fops, - intel_fbc_debugfs_false_color_get, - intel_fbc_debugfs_false_color_set, - "%llu\n"); +DEFINE_DEBUGFS_ATTRIBUTE(intel_fbc_debugfs_false_color_fops, + intel_fbc_debugfs_false_color_get, + intel_fbc_debugfs_false_color_set, + "%llu\n"); static void intel_fbc_debugfs_add(struct intel_fbc *fbc, struct dentry *parent) @@ -1819,8 +1819,8 @@ static void intel_fbc_debugfs_add(struct intel_fbc *fbc, fbc, &intel_fbc_debugfs_status_fops); if (fbc->funcs->set_false_color) - debugfs_create_file("i915_fbc_false_color", 0644, parent, - fbc, &intel_fbc_debugfs_false_color_fops); + debugfs_create_file_unsafe("i915_fbc_false_color", 0644, parent, + fbc, &intel_fbc_debugfs_false_color_fops); } void intel_fbc_crtc_debugfs_add(struct intel_crtc *crtc) diff --git a/drivers/gpu/drm/i915/display/intel_fbdev.c b/drivers/gpu/drm/i915/display/intel_fbdev.c index b6f5c554b50fecebfac865fcdf9bb6a3cb360bdc..ed197db5861d699efee61eed3f904179e5aaf18a 100644 --- a/drivers/gpu/drm/i915/display/intel_fbdev.c +++ b/drivers/gpu/drm/i915/display/intel_fbdev.c @@ -170,7 +170,7 @@ static int intelfb_alloc(struct drm_fb_helper *helper, * important and we should probably use that space with FBC or other * features. */ - if (size * 2 < dev_priv->stolen_usable_size) + if (size * 2 < dev_priv->dsm.usable_size) obj = i915_gem_object_create_stolen(dev_priv, size); if (IS_ERR(obj)) obj = i915_gem_object_create_shmem(dev_priv, size); diff --git a/drivers/gpu/drm/i915/display/intel_hdmi.c b/drivers/gpu/drm/i915/display/intel_hdmi.c index 6a2ee342eab59506106deb20a408e37928391d4e..c0ce6d3dc505663d4a2e7efb446c2c44aa92da9a 100644 --- a/drivers/gpu/drm/i915/display/intel_hdmi.c +++ b/drivers/gpu/drm/i915/display/intel_hdmi.c @@ -44,6 +44,7 @@ #include "i915_drv.h" #include "i915_reg.h" #include "intel_atomic.h" +#include "intel_audio.h" #include "intel_connector.h" #include "intel_ddi.h" #include "intel_de.h" @@ -537,9 +538,7 @@ void hsw_write_infoframe(struct intel_encoder *encoder, 0); /* Wa_14013475917 */ - if ((DISPLAY_VER(dev_priv) == 13 || - IS_MTL_DISPLAY_STEP(dev_priv, STEP_A0, STEP_B0)) && crtc_state->has_psr && - type == DP_SDP_VSC) + if (IS_DISPLAY_VER(dev_priv, 13, 14) && crtc_state->has_psr && type == DP_SDP_VSC) return; val |= hsw_infoframe_enable(type); @@ -2271,7 +2270,8 @@ int intel_hdmi_compute_config(struct intel_encoder *encoder, pipe_config->pixel_multiplier = 2; pipe_config->has_audio = - intel_hdmi_has_audio(encoder, pipe_config, conn_state); + intel_hdmi_has_audio(encoder, pipe_config, conn_state) && + intel_audio_compute_config(encoder, pipe_config, conn_state); /* * Try to respect downstream TMDS clock limits first, if @@ -2360,7 +2360,7 @@ intel_hdmi_unset_edid(struct drm_connector *connector) intel_hdmi->dp_dual_mode.type = DRM_DP_DUAL_MODE_NONE; intel_hdmi->dp_dual_mode.max_tmds_clock = 0; - kfree(to_intel_connector(connector)->detect_edid); + drm_edid_free(to_intel_connector(connector)->detect_edid); to_intel_connector(connector)->detect_edid = NULL; } @@ -2421,7 +2421,8 @@ intel_hdmi_set_edid(struct drm_connector *connector) struct drm_i915_private *dev_priv = to_i915(connector->dev); struct intel_hdmi *intel_hdmi = intel_attached_hdmi(to_intel_connector(connector)); intel_wakeref_t wakeref; - struct edid *edid; + const struct drm_edid *drm_edid; + const struct edid *edid; bool connected = false; struct i2c_adapter *i2c; @@ -2429,17 +2430,23 @@ intel_hdmi_set_edid(struct drm_connector *connector) i2c = intel_gmbus_get_adapter(dev_priv, intel_hdmi->ddc_bus); - edid = drm_get_edid(connector, i2c); + drm_edid = drm_edid_read_ddc(connector, i2c); - if (!edid && !intel_gmbus_is_forced_bit(i2c)) { + if (!drm_edid && !intel_gmbus_is_forced_bit(i2c)) { drm_dbg_kms(&dev_priv->drm, "HDMI GMBUS EDID read failed, retry using GPIO bit-banging\n"); intel_gmbus_force_bit(i2c, true); - edid = drm_get_edid(connector, i2c); + drm_edid = drm_edid_read_ddc(connector, i2c); intel_gmbus_force_bit(i2c, false); } - to_intel_connector(connector)->detect_edid = edid; + /* Below we depend on display info having been updated */ + drm_edid_connector_update(connector, drm_edid); + + to_intel_connector(connector)->detect_edid = drm_edid; + + /* FIXME: Get rid of drm_edid_raw() */ + edid = drm_edid_raw(drm_edid); if (edid && edid->input & DRM_EDID_INPUT_DIGITAL) { intel_hdmi->has_audio = drm_detect_monitor_audio(edid); intel_hdmi->has_hdmi_sink = drm_detect_hdmi_monitor(edid); @@ -2515,13 +2522,8 @@ intel_hdmi_force(struct drm_connector *connector) static int intel_hdmi_get_modes(struct drm_connector *connector) { - struct edid *edid; - - edid = to_intel_connector(connector)->detect_edid; - if (edid == NULL) - return 0; - - return intel_connector_update_modes(connector, edid); + /* drm_edid_connector_update() done in ->detect() or ->force() */ + return drm_edid_connector_add_modes(connector); } static struct i2c_adapter * diff --git a/drivers/gpu/drm/i915/display/intel_lvds.c b/drivers/gpu/drm/i915/display/intel_lvds.c index aecec992cd0d2d332bfed1df489b1c831f5fab08..a1557d84ce0a2d549b3cb7beb52026496c9e299b 100644 --- a/drivers/gpu/drm/i915/display/intel_lvds.c +++ b/drivers/gpu/drm/i915/display/intel_lvds.c @@ -477,10 +477,14 @@ static int intel_lvds_compute_config(struct intel_encoder *intel_encoder, static int intel_lvds_get_modes(struct drm_connector *connector) { struct intel_connector *intel_connector = to_intel_connector(connector); + const struct drm_edid *fixed_edid = intel_connector->panel.fixed_edid; - /* use cached edid if we have one */ - if (!IS_ERR_OR_NULL(intel_connector->edid)) - return drm_add_edid_modes(connector, intel_connector->edid); + /* Use panel fixed edid if we have one */ + if (!IS_ERR_OR_NULL(fixed_edid)) { + drm_edid_connector_update(connector, fixed_edid); + + return drm_edid_connector_add_modes(connector); + } return intel_panel_get_modes(intel_connector); } @@ -834,7 +838,7 @@ void intel_lvds_init(struct drm_i915_private *dev_priv) struct intel_connector *intel_connector; struct drm_connector *connector; struct drm_encoder *encoder; - struct edid *edid; + const struct drm_edid *drm_edid; i915_reg_t lvds_reg; u32 lvds; u8 pin; @@ -945,27 +949,34 @@ void intel_lvds_init(struct drm_i915_private *dev_priv) * preferred mode is the right one. */ mutex_lock(&dev_priv->drm.mode_config.mutex); - if (vga_switcheroo_handler_flags() & VGA_SWITCHEROO_CAN_SWITCH_DDC) + if (vga_switcheroo_handler_flags() & VGA_SWITCHEROO_CAN_SWITCH_DDC) { + const struct edid *edid; + + /* FIXME: Make drm_get_edid_switcheroo() return drm_edid */ edid = drm_get_edid_switcheroo(connector, - intel_gmbus_get_adapter(dev_priv, pin)); - else - edid = drm_get_edid(connector, - intel_gmbus_get_adapter(dev_priv, pin)); - if (edid) { - if (drm_add_edid_modes(connector, edid)) { - drm_connector_update_edid_property(connector, - edid); - } else { + intel_gmbus_get_adapter(dev_priv, pin)); + if (edid) { + drm_edid = drm_edid_alloc(edid, (edid->extensions + 1) * EDID_LENGTH); kfree(edid); - edid = ERR_PTR(-EINVAL); + } else { + drm_edid = NULL; } } else { - edid = ERR_PTR(-ENOENT); + drm_edid = drm_edid_read_ddc(connector, + intel_gmbus_get_adapter(dev_priv, pin)); + } + if (drm_edid) { + if (drm_edid_connector_update(connector, drm_edid) || + !drm_edid_connector_add_modes(connector)) { + drm_edid_connector_update(connector, NULL); + drm_edid_free(drm_edid); + drm_edid = ERR_PTR(-EINVAL); + } + } else { + drm_edid = ERR_PTR(-ENOENT); } - intel_connector->edid = edid; - intel_bios_init_panel_late(dev_priv, &intel_connector->panel, NULL, - IS_ERR(edid) ? NULL : edid); + IS_ERR(drm_edid) ? NULL : drm_edid); /* Try EDID first */ intel_panel_add_edid_fixed_modes(intel_connector, true); @@ -988,7 +999,7 @@ void intel_lvds_init(struct drm_i915_private *dev_priv) if (!intel_panel_preferred_fixed_mode(intel_connector)) goto failed; - intel_panel_init(intel_connector); + intel_panel_init(intel_connector, drm_edid); intel_backlight_setup(intel_connector, INVALID_PIPE); diff --git a/drivers/gpu/drm/i915/display/intel_modeset_setup.c b/drivers/gpu/drm/i915/display/intel_modeset_setup.c index 96395bfbd41dfb56999cac9bbff892349ac5259e..52cdbd4fc2fa0d56b73e7ee08f9e2ed2fe1ea3f5 100644 --- a/drivers/gpu/drm/i915/display/intel_modeset_setup.c +++ b/drivers/gpu/drm/i915/display/intel_modeset_setup.c @@ -698,8 +698,10 @@ void intel_modeset_setup_hw_state(struct drm_i915_private *i915, drm_crtc_vblank_reset(&crtc->base); - if (crtc_state->hw.active) + if (crtc_state->hw.active) { + intel_dmc_enable_pipe(i915, crtc->pipe); intel_crtc_vblank_on(crtc_state); + } } intel_fbc_sanitize(i915); diff --git a/drivers/gpu/drm/i915/display/intel_opregion.c b/drivers/gpu/drm/i915/display/intel_opregion.c index e0184745632c079bd90b8eba74ea05d584aeaab8..b8dce0576512aab714d340e1a0e851f810102ca7 100644 --- a/drivers/gpu/drm/i915/display/intel_opregion.c +++ b/drivers/gpu/drm/i915/display/intel_opregion.c @@ -1101,41 +1101,34 @@ intel_opregion_get_panel_type(struct drm_i915_private *dev_priv) * The EDID in the OpRegion, or NULL if there is none or it's invalid. * */ -struct edid *intel_opregion_get_edid(struct intel_connector *intel_connector) +const struct drm_edid *intel_opregion_get_edid(struct intel_connector *intel_connector) { struct drm_connector *connector = &intel_connector->base; struct drm_i915_private *i915 = to_i915(connector->dev); struct intel_opregion *opregion = &i915->display.opregion; - const void *in_edid; - const struct edid *edid; - struct edid *new_edid; + const struct drm_edid *drm_edid; + const void *edid; int len; if (!opregion->asle_ext) return NULL; - in_edid = opregion->asle_ext->bddc; + edid = opregion->asle_ext->bddc; /* Validity corresponds to number of 128-byte blocks */ len = (opregion->asle_ext->phed & ASLE_PHED_EDID_VALID_MASK) * 128; - if (!len || !memchr_inv(in_edid, 0, len)) + if (!len || !memchr_inv(edid, 0, len)) return NULL; - edid = in_edid; + drm_edid = drm_edid_alloc(edid, len); - if (len < EDID_LENGTH * (1 + edid->extensions)) { - drm_dbg_kms(&i915->drm, "Invalid EDID in ACPI OpRegion (Mailbox #5): too short\n"); - return NULL; - } - new_edid = drm_edid_duplicate(edid); - if (!new_edid) - return NULL; - if (!drm_edid_is_valid(new_edid)) { - kfree(new_edid); + if (!drm_edid_valid(drm_edid)) { drm_dbg_kms(&i915->drm, "Invalid EDID in ACPI OpRegion (Mailbox #5)\n"); - return NULL; + drm_edid_free(drm_edid); + drm_edid = NULL; } - return new_edid; + + return drm_edid; } bool intel_opregion_headless_sku(struct drm_i915_private *i915) diff --git a/drivers/gpu/drm/i915/display/intel_opregion.h b/drivers/gpu/drm/i915/display/intel_opregion.h index 2f261f98540081e4593f4da32cdd29d3aeabec12..d02e6696a050e12dec491bd2674bdbda88f3cf28 100644 --- a/drivers/gpu/drm/i915/display/intel_opregion.h +++ b/drivers/gpu/drm/i915/display/intel_opregion.h @@ -74,7 +74,7 @@ int intel_opregion_notify_encoder(struct intel_encoder *intel_encoder, int intel_opregion_notify_adapter(struct drm_i915_private *dev_priv, pci_power_t state); int intel_opregion_get_panel_type(struct drm_i915_private *dev_priv); -struct edid *intel_opregion_get_edid(struct intel_connector *connector); +const struct drm_edid *intel_opregion_get_edid(struct intel_connector *connector); bool intel_opregion_headless_sku(struct drm_i915_private *i915); @@ -123,7 +123,7 @@ static inline int intel_opregion_get_panel_type(struct drm_i915_private *dev) return -ENODEV; } -static inline struct edid * +static inline const struct drm_edid * intel_opregion_get_edid(struct intel_connector *connector) { return NULL; diff --git a/drivers/gpu/drm/i915/display/intel_panel.c b/drivers/gpu/drm/i915/display/intel_panel.c index 3b1004b019a80e77e9bd271a35f4f7fcab2910d7..42aa04bac261d261d4549a0a6b08ea743c4ab613 100644 --- a/drivers/gpu/drm/i915/display/intel_panel.c +++ b/drivers/gpu/drm/i915/display/intel_panel.c @@ -31,6 +31,8 @@ #include #include +#include + #include "i915_reg.h" #include "intel_backlight.h" #include "intel_connector.h" @@ -670,10 +672,13 @@ void intel_panel_init_alloc(struct intel_connector *connector) INIT_LIST_HEAD(&panel->fixed_modes); } -int intel_panel_init(struct intel_connector *connector) +int intel_panel_init(struct intel_connector *connector, + const struct drm_edid *fixed_edid) { struct intel_panel *panel = &connector->panel; + panel->fixed_edid = fixed_edid; + intel_backlight_init_funcs(panel); if (!has_drrs_modes(connector)) @@ -692,6 +697,9 @@ void intel_panel_fini(struct intel_connector *connector) struct intel_panel *panel = &connector->panel; struct drm_display_mode *fixed_mode, *next; + if (!IS_ERR_OR_NULL(panel->fixed_edid)) + drm_edid_free(panel->fixed_edid); + intel_backlight_destroy(panel); intel_bios_fini_panel(panel); diff --git a/drivers/gpu/drm/i915/display/intel_panel.h b/drivers/gpu/drm/i915/display/intel_panel.h index 4b51e1c51da62518f0c3c14164a0f154a182ed82..15a8c897b33fea3825b3a0facb709f65ccc1339b 100644 --- a/drivers/gpu/drm/i915/display/intel_panel.h +++ b/drivers/gpu/drm/i915/display/intel_panel.h @@ -13,13 +13,15 @@ enum drrs_type; struct drm_connector; struct drm_connector_state; struct drm_display_mode; +struct drm_edid; struct drm_i915_private; struct intel_connector; struct intel_crtc_state; struct intel_encoder; void intel_panel_init_alloc(struct intel_connector *connector); -int intel_panel_init(struct intel_connector *connector); +int intel_panel_init(struct intel_connector *connector, + const struct drm_edid *fixed_edid); void intel_panel_fini(struct intel_connector *connector); enum drm_connector_status intel_panel_detect(struct drm_connector *connector, bool force); diff --git a/drivers/gpu/drm/i915/display/intel_pch_refclk.c b/drivers/gpu/drm/i915/display/intel_pch_refclk.c index 08a94365b7d13b6a2be7e49588c76ffcb786c342..3657b2940702976088808ba8cfbab6df715f74fd 100644 --- a/drivers/gpu/drm/i915/display/intel_pch_refclk.c +++ b/drivers/gpu/drm/i915/display/intel_pch_refclk.c @@ -467,24 +467,24 @@ static void lpt_init_pch_refclk(struct drm_i915_private *dev_priv) * clock hierarchy. That would also allow us to do * clock bending finally. */ - dev_priv->pch_ssc_use = 0; + dev_priv->display.dpll.pch_ssc_use = 0; if (spll_uses_pch_ssc(dev_priv)) { drm_dbg_kms(&dev_priv->drm, "SPLL using PCH SSC\n"); - dev_priv->pch_ssc_use |= BIT(DPLL_ID_SPLL); + dev_priv->display.dpll.pch_ssc_use |= BIT(DPLL_ID_SPLL); } if (wrpll_uses_pch_ssc(dev_priv, DPLL_ID_WRPLL1)) { drm_dbg_kms(&dev_priv->drm, "WRPLL1 using PCH SSC\n"); - dev_priv->pch_ssc_use |= BIT(DPLL_ID_WRPLL1); + dev_priv->display.dpll.pch_ssc_use |= BIT(DPLL_ID_WRPLL1); } if (wrpll_uses_pch_ssc(dev_priv, DPLL_ID_WRPLL2)) { drm_dbg_kms(&dev_priv->drm, "WRPLL2 using PCH SSC\n"); - dev_priv->pch_ssc_use |= BIT(DPLL_ID_WRPLL2); + dev_priv->display.dpll.pch_ssc_use |= BIT(DPLL_ID_WRPLL2); } - if (dev_priv->pch_ssc_use) + if (dev_priv->display.dpll.pch_ssc_use) return; if (has_fdi) { diff --git a/drivers/gpu/drm/i915/display/intel_pipe_crc.c b/drivers/gpu/drm/i915/display/intel_pipe_crc.c index e9774670e3f6442c144a84e6d3e20653b380a99e..8d3ea8d7b737202af307143672d6050409374500 100644 --- a/drivers/gpu/drm/i915/display/intel_pipe_crc.c +++ b/drivers/gpu/drm/i915/display/intel_pipe_crc.c @@ -72,14 +72,13 @@ static int i8xx_pipe_crc_ctl_reg(enum intel_pipe_crc_source *source, return 0; } -static int i9xx_pipe_crc_auto_source(struct drm_i915_private *dev_priv, - enum pipe pipe, - enum intel_pipe_crc_source *source) +static void i9xx_pipe_crc_auto_source(struct drm_i915_private *dev_priv, + enum pipe pipe, + enum intel_pipe_crc_source *source) { struct intel_encoder *encoder; struct intel_crtc *crtc; struct intel_digital_port *dig_port; - int ret = 0; *source = INTEL_PIPE_CRC_SOURCE_PIPE; @@ -121,8 +120,6 @@ static int i9xx_pipe_crc_auto_source(struct drm_i915_private *dev_priv, } } drm_modeset_unlock_all(&dev_priv->drm); - - return ret; } static int vlv_pipe_crc_ctl_reg(struct drm_i915_private *dev_priv, @@ -132,11 +129,8 @@ static int vlv_pipe_crc_ctl_reg(struct drm_i915_private *dev_priv, { bool need_stable_symbols = false; - if (*source == INTEL_PIPE_CRC_SOURCE_AUTO) { - int ret = i9xx_pipe_crc_auto_source(dev_priv, pipe, source); - if (ret) - return ret; - } + if (*source == INTEL_PIPE_CRC_SOURCE_AUTO) + i9xx_pipe_crc_auto_source(dev_priv, pipe, source); switch (*source) { case INTEL_PIPE_CRC_SOURCE_PIPE: @@ -200,11 +194,8 @@ static int i9xx_pipe_crc_ctl_reg(struct drm_i915_private *dev_priv, enum intel_pipe_crc_source *source, u32 *val) { - if (*source == INTEL_PIPE_CRC_SOURCE_AUTO) { - int ret = i9xx_pipe_crc_auto_source(dev_priv, pipe, source); - if (ret) - return ret; - } + if (*source == INTEL_PIPE_CRC_SOURCE_AUTO) + i9xx_pipe_crc_auto_source(dev_priv, pipe, source); switch (*source) { case INTEL_PIPE_CRC_SOURCE_PIPE: diff --git a/drivers/gpu/drm/i915/display/intel_plane_initial.c b/drivers/gpu/drm/i915/display/intel_plane_initial.c index 76be796df255d94f67d9b8dd3070d08d095ff701..bb6ea7de5c61ea2bbac4b8d2c6af7fe9a3ff9708 100644 --- a/drivers/gpu/drm/i915/display/intel_plane_initial.c +++ b/drivers/gpu/drm/i915/display/intel_plane_initial.c @@ -107,7 +107,7 @@ initial_plane_vma(struct drm_i915_private *i915, */ if (IS_ENABLED(CONFIG_FRAMEBUFFER_CONSOLE) && mem == i915->mm.stolen_region && - size * 2 > i915->stolen_usable_size) + size * 2 > i915->dsm.usable_size) return NULL; obj = i915_gem_object_create_region_at(mem, phys_base, size, 0); diff --git a/drivers/gpu/drm/i915/display/intel_psr.c b/drivers/gpu/drm/i915/display/intel_psr.c index d0d774219cc5ea73e9f4eb29a300ac0ff3b9d07f..7a72e15e68369f2e2bdbf58afabd48ab06c5de68 100644 --- a/drivers/gpu/drm/i915/display/intel_psr.c +++ b/drivers/gpu/drm/i915/display/intel_psr.c @@ -24,14 +24,13 @@ #include #include -#include "display/intel_dp.h" - #include "i915_drv.h" #include "i915_reg.h" #include "intel_atomic.h" #include "intel_crtc.h" #include "intel_de.h" #include "intel_display_types.h" +#include "intel_dp.h" #include "intel_dp_aux.h" #include "intel_hdmi.h" #include "intel_psr.h" @@ -1112,6 +1111,8 @@ static u32 wa_16013835468_bit_get(struct intel_dp *intel_dp) return LATENCY_REPORTING_REMOVED_PIPE_B; case PIPE_C: return LATENCY_REPORTING_REMOVED_PIPE_C; + case PIPE_D: + return LATENCY_REPORTING_REMOVED_PIPE_D; default: MISSING_CASE(intel_dp->psr.pipe); return 0; @@ -1163,6 +1164,23 @@ static void intel_psr_enable_source(struct intel_dp *intel_dp, intel_dp->psr.psr2_sel_fetch_enabled ? IGNORE_PSR2_HW_TRACKING : 0); + /* + * Wa_16013835468 + * Wa_14015648006 + */ + if (IS_MTL_DISPLAY_STEP(dev_priv, STEP_A0, STEP_B0) || + IS_DISPLAY_VER(dev_priv, 12, 13)) { + u16 vtotal, vblank; + + vtotal = crtc_state->uapi.adjusted_mode.crtc_vtotal - + crtc_state->uapi.adjusted_mode.crtc_vdisplay; + vblank = crtc_state->uapi.adjusted_mode.crtc_vblank_end - + crtc_state->uapi.adjusted_mode.crtc_vblank_start; + if (vblank > vtotal) + intel_de_rmw(dev_priv, GEN8_CHICKEN_DCPR_1, 0, + wa_16013835468_bit_get(intel_dp)); + } + if (intel_dp->psr.psr2_enabled) { if (DISPLAY_VER(dev_priv) == 9) intel_de_rmw(dev_priv, CHICKEN_TRANS(cpu_transcoder), 0, @@ -1196,20 +1214,6 @@ static void intel_psr_enable_source(struct intel_dp *intel_dp, else if (IS_ALDERLAKE_P(dev_priv)) intel_de_rmw(dev_priv, CLKGATE_DIS_MISC, 0, CLKGATE_DIS_MISC_DMASC_GATING_DIS); - - /* Wa_16013835468:tgl[b0+], dg1 */ - if (IS_TGL_DISPLAY_STEP(dev_priv, STEP_B0, STEP_FOREVER) || - IS_DG1(dev_priv)) { - u16 vtotal, vblank; - - vtotal = crtc_state->uapi.adjusted_mode.crtc_vtotal - - crtc_state->uapi.adjusted_mode.crtc_vdisplay; - vblank = crtc_state->uapi.adjusted_mode.crtc_vblank_end - - crtc_state->uapi.adjusted_mode.crtc_vblank_start; - if (vblank > vtotal) - intel_de_rmw(dev_priv, GEN8_CHICKEN_DCPR_1, 0, - wa_16013835468_bit_get(intel_dp)); - } } } @@ -1362,6 +1366,15 @@ static void intel_psr_disable_locked(struct intel_dp *intel_dp) intel_de_rmw(dev_priv, CHICKEN_PAR1_1, DIS_RAM_BYPASS_PSR2_MAN_TRACK, 0); + /* + * Wa_16013835468 + * Wa_14015648006 + */ + if (IS_MTL_DISPLAY_STEP(dev_priv, STEP_A0, STEP_B0) || + IS_DISPLAY_VER(dev_priv, 12, 13)) + intel_de_rmw(dev_priv, GEN8_CHICKEN_DCPR_1, + wa_16013835468_bit_get(intel_dp), 0); + if (intel_dp->psr.psr2_enabled) { /* Wa_16011168373:adl-p */ if (IS_ADLP_DISPLAY_STEP(dev_priv, STEP_A0, STEP_B0)) @@ -1377,12 +1390,6 @@ static void intel_psr_disable_locked(struct intel_dp *intel_dp) else if (IS_ALDERLAKE_P(dev_priv)) intel_de_rmw(dev_priv, CLKGATE_DIS_MISC, CLKGATE_DIS_MISC_DMASC_GATING_DIS, 0); - - /* Wa_16013835468:tgl[b0+], dg1 */ - if (IS_TGL_DISPLAY_STEP(dev_priv, STEP_B0, STEP_FOREVER) || - IS_DG1(dev_priv)) - intel_de_rmw(dev_priv, GEN8_CHICKEN_DCPR_1, - wa_16013835468_bit_get(intel_dp), 0); } intel_snps_phy_update_psr_power_state(dev_priv, phy, false); @@ -1835,6 +1842,12 @@ int intel_psr2_sel_fetch_update(struct intel_atomic_state *state, if (full_update) goto skip_sel_fetch_set_loop; + /* Wa_14014971492 */ + if ((IS_MTL_DISPLAY_STEP(dev_priv, STEP_A0, STEP_B0) || + IS_ALDERLAKE_P(dev_priv) || IS_TIGERLAKE(dev_priv)) && + crtc_state->splitter.enable) + pipe_clip.y1 = 0; + ret = drm_atomic_add_affected_planes(&state->base, &crtc->base); if (ret) return ret; diff --git a/drivers/gpu/drm/i915/display/intel_sdvo.c b/drivers/gpu/drm/i915/display/intel_sdvo.c index 21805c15d5eb8ab93435d5e935e0db50c1879c0a..e12ba458636c1e3cf4267e44cb44c6e21450dfe9 100644 --- a/drivers/gpu/drm/i915/display/intel_sdvo.c +++ b/drivers/gpu/drm/i915/display/intel_sdvo.c @@ -39,6 +39,7 @@ #include "i915_drv.h" #include "i915_reg.h" #include "intel_atomic.h" +#include "intel_audio.h" #include "intel_connector.h" #include "intel_crtc.h" #include "intel_de.h" @@ -1068,7 +1069,8 @@ static ssize_t intel_sdvo_read_infoframe(struct intel_sdvo *intel_sdvo, &tx_rate, 1)) return -ENXIO; - if (tx_rate == SDVO_HBUF_TX_DISABLED) + /* TX_DISABLED doesn't mean disabled for ELD */ + if (if_index != SDVO_HBUF_INDEX_ELD && tx_rate == SDVO_HBUF_TX_DISABLED) return 0; if (!intel_sdvo_get_hbuf_size(intel_sdvo, &hbuf_size)) @@ -1185,6 +1187,28 @@ static void intel_sdvo_get_avi_infoframe(struct intel_sdvo *intel_sdvo, frame->any.type, HDMI_INFOFRAME_TYPE_AVI); } +static void intel_sdvo_get_eld(struct intel_sdvo *intel_sdvo, + struct intel_crtc_state *crtc_state) +{ + struct drm_i915_private *i915 = to_i915(intel_sdvo->base.base.dev); + ssize_t len; + u8 val; + + if (!crtc_state->has_audio) + return; + + if (!intel_sdvo_get_value(intel_sdvo, SDVO_CMD_GET_AUDIO_STAT, &val, 1)) + return; + + if ((val & SDVO_AUDIO_ELD_VALID) == 0) + return; + + len = intel_sdvo_read_infoframe(intel_sdvo, SDVO_HBUF_INDEX_ELD, + crtc_state->eld, sizeof(crtc_state->eld)); + if (len < 0) + drm_dbg_kms(&i915->drm, "failed to read ELD\n"); +} + static bool intel_sdvo_set_tv_format(struct intel_sdvo *intel_sdvo, const struct drm_connector_state *conn_state) { @@ -1378,7 +1402,9 @@ static int intel_sdvo_compute_config(struct intel_encoder *encoder, pipe_config->has_hdmi_sink = intel_has_hdmi_sink(intel_sdvo, conn_state); - pipe_config->has_audio = intel_sdvo_has_audio(encoder, pipe_config, conn_state); + pipe_config->has_audio = + intel_sdvo_has_audio(encoder, pipe_config, conn_state) && + intel_audio_compute_config(encoder, pipe_config, conn_state); pipe_config->limited_color_range = intel_sdvo_limited_color_range(encoder, pipe_config, @@ -1729,9 +1755,7 @@ static void intel_sdvo_get_config(struct intel_encoder *encoder, if (intel_sdvo_get_value(intel_sdvo, SDVO_CMD_GET_AUDIO_STAT, &val, 1)) { - u8 mask = SDVO_AUDIO_ELD_VALID | SDVO_AUDIO_PRESENCE_DETECT; - - if ((val & mask) == mask) + if (val & SDVO_AUDIO_PRESENCE_DETECT) pipe_config->has_audio = true; } @@ -1742,6 +1766,8 @@ static void intel_sdvo_get_config(struct intel_encoder *encoder, } intel_sdvo_get_avi_infoframe(intel_sdvo, pipe_config); + + intel_sdvo_get_eld(intel_sdvo, pipe_config); } static void intel_sdvo_disable_audio(struct intel_sdvo *intel_sdvo) @@ -1753,12 +1779,7 @@ static void intel_sdvo_enable_audio(struct intel_sdvo *intel_sdvo, const struct intel_crtc_state *crtc_state, const struct drm_connector_state *conn_state) { - const struct drm_display_mode *adjusted_mode = - &crtc_state->hw.adjusted_mode; - struct drm_connector *connector = conn_state->connector; - u8 *eld = connector->eld; - - eld[6] = drm_av_sync_delay(connector, adjusted_mode) / 2; + const u8 *eld = crtc_state->eld; intel_sdvo_set_audio_state(intel_sdvo, 0); @@ -2903,7 +2924,7 @@ intel_sdvo_lvds_init(struct intel_sdvo *intel_sdvo, u16 type) mutex_unlock(&i915->drm.mode_config.mutex); } - intel_panel_init(intel_connector); + intel_panel_init(intel_connector, NULL); if (!intel_panel_preferred_fixed_mode(intel_connector)) goto err; diff --git a/drivers/gpu/drm/i915/display/intel_snps_phy.c b/drivers/gpu/drm/i915/display/intel_snps_phy.c index 9494cfd45519016e2174bedbc97c46f3442b2c02..c65c771f5c461f018bcc4620809782571109eea0 100644 --- a/drivers/gpu/drm/i915/display/intel_snps_phy.c +++ b/drivers/gpu/drm/i915/display/intel_snps_phy.c @@ -40,7 +40,7 @@ void intel_snps_phy_wait_for_calibration(struct drm_i915_private *i915) */ if (intel_de_wait_for_clear(i915, DG2_PHY_MISC(phy), DG2_PHY_DP_TX_ACK_MASK, 25)) - i915->snps_phy_failed_calibration |= BIT(phy); + i915->display.snps.phy_failed_calibration |= BIT(phy); } } diff --git a/drivers/gpu/drm/i915/display/intel_vblank.c b/drivers/gpu/drm/i915/display/intel_vblank.c new file mode 100644 index 0000000000000000000000000000000000000000..4c83e2320bcacc4f4238cebeac0119f600d67d5f --- /dev/null +++ b/drivers/gpu/drm/i915/display/intel_vblank.c @@ -0,0 +1,441 @@ +// SPDX-License-Identifier: MIT +/* + * Copyright © 2022-2023 Intel Corporation + */ + +#include "i915_drv.h" +#include "i915_reg.h" +#include "intel_de.h" +#include "intel_display_types.h" +#include "intel_vblank.h" + +/* + * This timing diagram depicts the video signal in and + * around the vertical blanking period. + * + * Assumptions about the fictitious mode used in this example: + * vblank_start >= 3 + * vsync_start = vblank_start + 1 + * vsync_end = vblank_start + 2 + * vtotal = vblank_start + 3 + * + * start of vblank: + * latch double buffered registers + * increment frame counter (ctg+) + * generate start of vblank interrupt (gen4+) + * | + * | frame start: + * | generate frame start interrupt (aka. vblank interrupt) (gmch) + * | may be shifted forward 1-3 extra lines via PIPECONF + * | | + * | | start of vsync: + * | | generate vsync interrupt + * | | | + * ___xxxx___ ___xxxx___ ___xxxx___ ___xxxx___ ___xxxx___ ___xxxx + * . \hs/ . \hs/ \hs/ \hs/ . \hs/ + * ----va---> <-----------------vb--------------------> <--------va------------- + * | | <----vs-----> | + * -vbs-----> <---vbs+1---> <---vbs+2---> <-----0-----> <-----1-----> <-----2--- (scanline counter gen2) + * -vbs-2---> <---vbs-1---> <---vbs-----> <---vbs+1---> <---vbs+2---> <-----0--- (scanline counter gen3+) + * -vbs-2---> <---vbs-2---> <---vbs-1---> <---vbs-----> <---vbs+1---> <---vbs+2- (scanline counter hsw+ hdmi) + * | | | + * last visible pixel first visible pixel + * | increment frame counter (gen3/4) + * pixel counter = vblank_start * htotal pixel counter = 0 (gen3/4) + * + * x = horizontal active + * _ = horizontal blanking + * hs = horizontal sync + * va = vertical active + * vb = vertical blanking + * vs = vertical sync + * vbs = vblank_start (number) + * + * Summary: + * - most events happen at the start of horizontal sync + * - frame start happens at the start of horizontal blank, 1-4 lines + * (depending on PIPECONF settings) after the start of vblank + * - gen3/4 pixel and frame counter are synchronized with the start + * of horizontal active on the first line of vertical active + */ + +/* + * Called from drm generic code, passed a 'crtc', which we use as a pipe index. + */ +u32 i915_get_vblank_counter(struct drm_crtc *crtc) +{ + struct drm_i915_private *dev_priv = to_i915(crtc->dev); + struct drm_vblank_crtc *vblank = &dev_priv->drm.vblank[drm_crtc_index(crtc)]; + const struct drm_display_mode *mode = &vblank->hwmode; + enum pipe pipe = to_intel_crtc(crtc)->pipe; + u32 pixel, vbl_start, hsync_start, htotal; + u64 frame; + + /* + * On i965gm TV output the frame counter only works up to + * the point when we enable the TV encoder. After that the + * frame counter ceases to work and reads zero. We need a + * vblank wait before enabling the TV encoder and so we + * have to enable vblank interrupts while the frame counter + * is still in a working state. However the core vblank code + * does not like us returning non-zero frame counter values + * when we've told it that we don't have a working frame + * counter. Thus we must stop non-zero values leaking out. + */ + if (!vblank->max_vblank_count) + return 0; + + htotal = mode->crtc_htotal; + hsync_start = mode->crtc_hsync_start; + vbl_start = mode->crtc_vblank_start; + if (mode->flags & DRM_MODE_FLAG_INTERLACE) + vbl_start = DIV_ROUND_UP(vbl_start, 2); + + /* Convert to pixel count */ + vbl_start *= htotal; + + /* Start of vblank event occurs at start of hsync */ + vbl_start -= htotal - hsync_start; + + /* + * High & low register fields aren't synchronized, so make sure + * we get a low value that's stable across two reads of the high + * register. + */ + frame = intel_de_read64_2x32(dev_priv, PIPEFRAMEPIXEL(pipe), PIPEFRAME(pipe)); + + pixel = frame & PIPE_PIXEL_MASK; + frame = (frame >> PIPE_FRAME_LOW_SHIFT) & 0xffffff; + + /* + * The frame counter increments at beginning of active. + * Cook up a vblank counter by also checking the pixel + * counter against vblank start. + */ + return (frame + (pixel >= vbl_start)) & 0xffffff; +} + +u32 g4x_get_vblank_counter(struct drm_crtc *crtc) +{ + struct drm_i915_private *dev_priv = to_i915(crtc->dev); + struct drm_vblank_crtc *vblank = &dev_priv->drm.vblank[drm_crtc_index(crtc)]; + enum pipe pipe = to_intel_crtc(crtc)->pipe; + + if (!vblank->max_vblank_count) + return 0; + + return intel_de_read(dev_priv, PIPE_FRMCOUNT_G4X(pipe)); +} + +static u32 intel_crtc_scanlines_since_frame_timestamp(struct intel_crtc *crtc) +{ + struct drm_i915_private *dev_priv = to_i915(crtc->base.dev); + struct drm_vblank_crtc *vblank = + &crtc->base.dev->vblank[drm_crtc_index(&crtc->base)]; + const struct drm_display_mode *mode = &vblank->hwmode; + u32 htotal = mode->crtc_htotal; + u32 clock = mode->crtc_clock; + u32 scan_prev_time, scan_curr_time, scan_post_time; + + /* + * To avoid the race condition where we might cross into the + * next vblank just between the PIPE_FRMTMSTMP and TIMESTAMP_CTR + * reads. We make sure we read PIPE_FRMTMSTMP and TIMESTAMP_CTR + * during the same frame. + */ + do { + /* + * This field provides read back of the display + * pipe frame time stamp. The time stamp value + * is sampled at every start of vertical blank. + */ + scan_prev_time = intel_de_read_fw(dev_priv, + PIPE_FRMTMSTMP(crtc->pipe)); + + /* + * The TIMESTAMP_CTR register has the current + * time stamp value. + */ + scan_curr_time = intel_de_read_fw(dev_priv, IVB_TIMESTAMP_CTR); + + scan_post_time = intel_de_read_fw(dev_priv, + PIPE_FRMTMSTMP(crtc->pipe)); + } while (scan_post_time != scan_prev_time); + + return div_u64(mul_u32_u32(scan_curr_time - scan_prev_time, + clock), 1000 * htotal); +} + +/* + * On certain encoders on certain platforms, pipe + * scanline register will not work to get the scanline, + * since the timings are driven from the PORT or issues + * with scanline register updates. + * This function will use Framestamp and current + * timestamp registers to calculate the scanline. + */ +static u32 __intel_get_crtc_scanline_from_timestamp(struct intel_crtc *crtc) +{ + struct drm_vblank_crtc *vblank = + &crtc->base.dev->vblank[drm_crtc_index(&crtc->base)]; + const struct drm_display_mode *mode = &vblank->hwmode; + u32 vblank_start = mode->crtc_vblank_start; + u32 vtotal = mode->crtc_vtotal; + u32 scanline; + + scanline = intel_crtc_scanlines_since_frame_timestamp(crtc); + scanline = min(scanline, vtotal - 1); + scanline = (scanline + vblank_start) % vtotal; + + return scanline; +} + +/* + * intel_de_read_fw(), only for fast reads of display block, no need for + * forcewake etc. + */ +static int __intel_get_crtc_scanline(struct intel_crtc *crtc) +{ + struct drm_device *dev = crtc->base.dev; + struct drm_i915_private *dev_priv = to_i915(dev); + const struct drm_display_mode *mode; + struct drm_vblank_crtc *vblank; + enum pipe pipe = crtc->pipe; + int position, vtotal; + + if (!crtc->active) + return 0; + + vblank = &crtc->base.dev->vblank[drm_crtc_index(&crtc->base)]; + mode = &vblank->hwmode; + + if (crtc->mode_flags & I915_MODE_FLAG_GET_SCANLINE_FROM_TIMESTAMP) + return __intel_get_crtc_scanline_from_timestamp(crtc); + + vtotal = mode->crtc_vtotal; + if (mode->flags & DRM_MODE_FLAG_INTERLACE) + vtotal /= 2; + + position = intel_de_read_fw(dev_priv, PIPEDSL(pipe)) & PIPEDSL_LINE_MASK; + + /* + * On HSW, the DSL reg (0x70000) appears to return 0 if we + * read it just before the start of vblank. So try it again + * so we don't accidentally end up spanning a vblank frame + * increment, causing the pipe_update_end() code to squak at us. + * + * The nature of this problem means we can't simply check the ISR + * bit and return the vblank start value; nor can we use the scanline + * debug register in the transcoder as it appears to have the same + * problem. We may need to extend this to include other platforms, + * but so far testing only shows the problem on HSW. + */ + if (HAS_DDI(dev_priv) && !position) { + int i, temp; + + for (i = 0; i < 100; i++) { + udelay(1); + temp = intel_de_read_fw(dev_priv, PIPEDSL(pipe)) & PIPEDSL_LINE_MASK; + if (temp != position) { + position = temp; + break; + } + } + } + + /* + * See update_scanline_offset() for the details on the + * scanline_offset adjustment. + */ + return (position + crtc->scanline_offset) % vtotal; +} + +static bool i915_get_crtc_scanoutpos(struct drm_crtc *_crtc, + bool in_vblank_irq, + int *vpos, int *hpos, + ktime_t *stime, ktime_t *etime, + const struct drm_display_mode *mode) +{ + struct drm_device *dev = _crtc->dev; + struct drm_i915_private *dev_priv = to_i915(dev); + struct intel_crtc *crtc = to_intel_crtc(_crtc); + enum pipe pipe = crtc->pipe; + int position; + int vbl_start, vbl_end, hsync_start, htotal, vtotal; + unsigned long irqflags; + bool use_scanline_counter = DISPLAY_VER(dev_priv) >= 5 || + IS_G4X(dev_priv) || DISPLAY_VER(dev_priv) == 2 || + crtc->mode_flags & I915_MODE_FLAG_USE_SCANLINE_COUNTER; + + if (drm_WARN_ON(&dev_priv->drm, !mode->crtc_clock)) { + drm_dbg(&dev_priv->drm, + "trying to get scanoutpos for disabled pipe %c\n", + pipe_name(pipe)); + return false; + } + + htotal = mode->crtc_htotal; + hsync_start = mode->crtc_hsync_start; + vtotal = mode->crtc_vtotal; + vbl_start = mode->crtc_vblank_start; + vbl_end = mode->crtc_vblank_end; + + if (mode->flags & DRM_MODE_FLAG_INTERLACE) { + vbl_start = DIV_ROUND_UP(vbl_start, 2); + vbl_end /= 2; + vtotal /= 2; + } + + /* + * Lock uncore.lock, as we will do multiple timing critical raw + * register reads, potentially with preemption disabled, so the + * following code must not block on uncore.lock. + */ + spin_lock_irqsave(&dev_priv->uncore.lock, irqflags); + + /* preempt_disable_rt() should go right here in PREEMPT_RT patchset. */ + + /* Get optional system timestamp before query. */ + if (stime) + *stime = ktime_get(); + + if (crtc->mode_flags & I915_MODE_FLAG_VRR) { + int scanlines = intel_crtc_scanlines_since_frame_timestamp(crtc); + + position = __intel_get_crtc_scanline(crtc); + + /* + * Already exiting vblank? If so, shift our position + * so it looks like we're already apporaching the full + * vblank end. This should make the generated timestamp + * more or less match when the active portion will start. + */ + if (position >= vbl_start && scanlines < position) + position = min(crtc->vmax_vblank_start + scanlines, vtotal - 1); + } else if (use_scanline_counter) { + /* No obvious pixelcount register. Only query vertical + * scanout position from Display scan line register. + */ + position = __intel_get_crtc_scanline(crtc); + } else { + /* + * Have access to pixelcount since start of frame. + * We can split this into vertical and horizontal + * scanout position. + */ + position = (intel_de_read_fw(dev_priv, PIPEFRAMEPIXEL(pipe)) & PIPE_PIXEL_MASK) >> PIPE_PIXEL_SHIFT; + + /* convert to pixel counts */ + vbl_start *= htotal; + vbl_end *= htotal; + vtotal *= htotal; + + /* + * In interlaced modes, the pixel counter counts all pixels, + * so one field will have htotal more pixels. In order to avoid + * the reported position from jumping backwards when the pixel + * counter is beyond the length of the shorter field, just + * clamp the position the length of the shorter field. This + * matches how the scanline counter based position works since + * the scanline counter doesn't count the two half lines. + */ + if (position >= vtotal) + position = vtotal - 1; + + /* + * Start of vblank interrupt is triggered at start of hsync, + * just prior to the first active line of vblank. However we + * consider lines to start at the leading edge of horizontal + * active. So, should we get here before we've crossed into + * the horizontal active of the first line in vblank, we would + * not set the DRM_SCANOUTPOS_INVBL flag. In order to fix that, + * always add htotal-hsync_start to the current pixel position. + */ + position = (position + htotal - hsync_start) % vtotal; + } + + /* Get optional system timestamp after query. */ + if (etime) + *etime = ktime_get(); + + /* preempt_enable_rt() should go right here in PREEMPT_RT patchset. */ + + spin_unlock_irqrestore(&dev_priv->uncore.lock, irqflags); + + /* + * While in vblank, position will be negative + * counting up towards 0 at vbl_end. And outside + * vblank, position will be positive counting + * up since vbl_end. + */ + if (position >= vbl_start) + position -= vbl_end; + else + position += vtotal - vbl_end; + + if (use_scanline_counter) { + *vpos = position; + *hpos = 0; + } else { + *vpos = position / htotal; + *hpos = position - (*vpos * htotal); + } + + return true; +} + +bool intel_crtc_get_vblank_timestamp(struct drm_crtc *crtc, int *max_error, + ktime_t *vblank_time, bool in_vblank_irq) +{ + return drm_crtc_vblank_helper_get_vblank_timestamp_internal( + crtc, max_error, vblank_time, in_vblank_irq, + i915_get_crtc_scanoutpos); +} + +int intel_get_crtc_scanline(struct intel_crtc *crtc) +{ + struct drm_i915_private *dev_priv = to_i915(crtc->base.dev); + unsigned long irqflags; + int position; + + spin_lock_irqsave(&dev_priv->uncore.lock, irqflags); + position = __intel_get_crtc_scanline(crtc); + spin_unlock_irqrestore(&dev_priv->uncore.lock, irqflags); + + return position; +} + +static bool pipe_scanline_is_moving(struct drm_i915_private *dev_priv, + enum pipe pipe) +{ + i915_reg_t reg = PIPEDSL(pipe); + u32 line1, line2; + + line1 = intel_de_read(dev_priv, reg) & PIPEDSL_LINE_MASK; + msleep(5); + line2 = intel_de_read(dev_priv, reg) & PIPEDSL_LINE_MASK; + + return line1 != line2; +} + +static void wait_for_pipe_scanline_moving(struct intel_crtc *crtc, bool state) +{ + struct drm_i915_private *dev_priv = to_i915(crtc->base.dev); + enum pipe pipe = crtc->pipe; + + /* Wait for the display line to settle/start moving */ + if (wait_for(pipe_scanline_is_moving(dev_priv, pipe) == state, 100)) + drm_err(&dev_priv->drm, + "pipe %c scanline %s wait timed out\n", + pipe_name(pipe), str_on_off(state)); +} + +void intel_wait_for_pipe_scanline_stopped(struct intel_crtc *crtc) +{ + wait_for_pipe_scanline_moving(crtc, false); +} + +void intel_wait_for_pipe_scanline_moving(struct intel_crtc *crtc) +{ + wait_for_pipe_scanline_moving(crtc, true); +} diff --git a/drivers/gpu/drm/i915/display/intel_vblank.h b/drivers/gpu/drm/i915/display/intel_vblank.h new file mode 100644 index 0000000000000000000000000000000000000000..c9fea2c2a99073a8bceef0f871c273dc9baba4bf --- /dev/null +++ b/drivers/gpu/drm/i915/display/intel_vblank.h @@ -0,0 +1,23 @@ +/* SPDX-License-Identifier: MIT */ +/* + * Copyright © 2022-2023 Intel Corporation + */ + +#ifndef __INTEL_VBLANK_H__ +#define __INTEL_VBLANK_H__ + +#include +#include + +struct drm_crtc; +struct intel_crtc; + +u32 i915_get_vblank_counter(struct drm_crtc *crtc); +u32 g4x_get_vblank_counter(struct drm_crtc *crtc); +bool intel_crtc_get_vblank_timestamp(struct drm_crtc *crtc, int *max_error, + ktime_t *vblank_time, bool in_vblank_irq); +int intel_get_crtc_scanline(struct intel_crtc *crtc); +void intel_wait_for_pipe_scanline_stopped(struct intel_crtc *crtc); +void intel_wait_for_pipe_scanline_moving(struct intel_crtc *crtc); + +#endif /* __INTEL_VBLANK_H__ */ diff --git a/drivers/gpu/drm/i915/display/intel_vga.c b/drivers/gpu/drm/i915/display/intel_vga.c index a69bfcac9a94542ae948030ed1dfb18a87c40b20..286a0bdd28c66d8654d8b5de82676d6e500e04d8 100644 --- a/drivers/gpu/drm/i915/display/intel_vga.c +++ b/drivers/gpu/drm/i915/display/intel_vga.c @@ -6,9 +6,10 @@ #include #include -#include #include