Commit 7060377c authored by Will Deacon's avatar Will Deacon
Browse files

Merge branch 'for-joerg/mtk' into for-joerg/arm-smmu/updates 

Merge in Mediatek support from Yong Wu which introduces significant
changes to the TLB invalidation and Arm short-descriptor code in the
io-pgtable layer.

* for-joerg/mtk: (40 commits)
  MAINTAINERS: Add entry for MediaTek IOMMU
  iommu/mediatek: Add mt8192 support
  iommu/mediatek: Remove unnecessary check in attach_device
  iommu/mediatek: Support master use iova over 32bit
  iommu/mediatek: Add iova reserved function
  iommu/mediatek: Support for multi domains
  iommu/mediatek: Add get_domain_id from dev->dma_range_map
  iommu/mediatek: Add iova_region structure
  iommu/mediatek: Move geometry.aperture updating into domain_finalise
  iommu/mediatek: Move domain_finalise into attach_device
  iommu/mediatek: Adjust the structure
  iommu/mediatek: Support report iova 34bit translation fault in ISR
  iommu/mediatek: Support up to 34bit iova in tlb flush
  iommu/mediatek: Add power-domain operation
  iommu/mediatek: Add pm runtime callback
  iommu/mediatek: Add device link for smi-common and m4u
  iommu/mediatek: Add error handle for mtk_iommu_probe
  iommu/mediatek: Move hw_init into attach_device
  iommu/mediatek: Update oas for v7s
  iommu/mediatek: Add a flag for iova 34bits case
  ...
parents 34eb9359 6af48738

Documentation/devicetree/bindings/iommu/mediatek,iommu.txt

deleted100644 → 0
+0 −105
Original line number Diff line number Diff line 
* Mediatek IOMMU Architecture Implementation



  Some Mediatek SOCs contain a Multimedia Memory Management Unit (M4U), and

this M4U have two generations of HW architecture. Generation one uses flat

pagetable, and only supports 4K size page mapping. Generation two uses the

ARM Short-Descriptor translation table format for address translation.



  About the M4U Hardware Block Diagram, please check below:



              EMI (External Memory Interface)

               |

              m4u (Multimedia Memory Management Unit)

               |

          +--------+

          |        |

      gals0-rx   gals1-rx    (Global Async Local Sync rx)

          |        |

          |        |

      gals0-tx   gals1-tx    (Global Async Local Sync tx)

          |        |          Some SoCs may have GALS.

          +--------+

               |

           SMI Common(Smart Multimedia Interface Common)

               |

       +----------------+-------

       |                |

       |             gals-rx        There may be GALS in some larbs.

       |                |

       |                |

       |             gals-tx

       |                |

   SMI larb0        SMI larb1   ... SoCs have several SMI local arbiter(larb).

   (display)         (vdec)

       |                |

       |                |

 +-----+-----+     +----+----+

 |     |     |     |    |    |

 |     |     |...  |    |    |  ... There are different ports in each larb.

 |     |     |     |    |    |

OVL0 RDMA0 WDMA0  MC   PP   VLD



  As above, The Multimedia HW will go through SMI and M4U while it

access EMI. SMI is a bridge between m4u and the Multimedia HW. It contain

smi local arbiter and smi common. It will control whether the Multimedia

HW should go though the m4u for translation or bypass it and talk

directly with EMI. And also SMI help control the power domain and clocks for

each local arbiter.

  Normally we specify a local arbiter(larb) for each multimedia HW

like display, video decode, and camera. And there are different ports

in each larb. Take a example, There are many ports like MC, PP, VLD in the

video decode local arbiter, all these ports are according to the video HW.

  In some SoCs, there may be a GALS(Global Async Local Sync) module between

smi-common and m4u, and additional GALS module between smi-larb and

smi-common. GALS can been seen as a "asynchronous fifo" which could help

synchronize for the modules in different clock frequency.



Required properties:

- compatible : must be one of the following string:

	"mediatek,mt2701-m4u" for mt2701 which uses generation one m4u HW.

	"mediatek,mt2712-m4u" for mt2712 which uses generation two m4u HW.

	"mediatek,mt6779-m4u" for mt6779 which uses generation two m4u HW.

	"mediatek,mt7623-m4u", "mediatek,mt2701-m4u" for mt7623 which uses

						     generation one m4u HW.

	"mediatek,mt8167-m4u" for mt8167 which uses generation two m4u HW.

	"mediatek,mt8173-m4u" for mt8173 which uses generation two m4u HW.

	"mediatek,mt8183-m4u" for mt8183 which uses generation two m4u HW.

- reg : m4u register base and size.

- interrupts : the interrupt of m4u.

- clocks : must contain one entry for each clock-names.

- clock-names : Only 1 optional clock:

  - "bclk": the block clock of m4u.

  Here is the list which require this "bclk":

  - mt2701, mt2712, mt7623 and mt8173.

  Note that m4u use the EMI clock which always has been enabled before kernel

  if there is no this "bclk".

- mediatek,larbs : List of phandle to the local arbiters in the current Socs.

	Refer to bindings/memory-controllers/mediatek,smi-larb.txt. It must sort

	according to the local arbiter index, like larb0, larb1, larb2...

- iommu-cells : must be 1. This is the mtk_m4u_id according to the HW.

	Specifies the mtk_m4u_id as defined in

	dt-binding/memory/mt2701-larb-port.h for mt2701, mt7623

	dt-binding/memory/mt2712-larb-port.h for mt2712,

	dt-binding/memory/mt6779-larb-port.h for mt6779,

	dt-binding/memory/mt8167-larb-port.h for mt8167,

	dt-binding/memory/mt8173-larb-port.h for mt8173, and

	dt-binding/memory/mt8183-larb-port.h for mt8183.



Example:

	iommu: iommu@10205000 {

		compatible = "mediatek,mt8173-m4u";

		reg = <0 0x10205000 0 0x1000>;

		interrupts = <GIC_SPI 139 IRQ_TYPE_LEVEL_LOW>;

		clocks = <&infracfg CLK_INFRA_M4U>;

		clock-names = "bclk";

		mediatek,larbs = <&larb0 &larb1 &larb2 &larb3 &larb4 &larb5>;

		#iommu-cells = <1>;

	};



Example for a client device:

	display {

		compatible = "mediatek,mt8173-disp";

		iommus = <&iommu M4U_PORT_DISP_OVL0>,

			 <&iommu M4U_PORT_DISP_RDMA0>;

		...

	};

Documentation/devicetree/bindings/iommu/mediatek,iommu.yaml

0 → 100644
+183 −0
Original line number Diff line number Diff line 
# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)

%YAML 1.2

---

$id: http://devicetree.org/schemas/iommu/mediatek,iommu.yaml#

$schema: http://devicetree.org/meta-schemas/core.yaml#



title: MediaTek IOMMU Architecture Implementation



maintainers:

  - Yong Wu <yong.wu@mediatek.com>



description: |+

  Some MediaTek SOCs contain a Multimedia Memory Management Unit (M4U), and

  this M4U have two generations of HW architecture. Generation one uses flat

  pagetable, and only supports 4K size page mapping. Generation two uses the

  ARM Short-Descriptor translation table format for address translation.



  About the M4U Hardware Block Diagram, please check below:



                EMI (External Memory Interface)

                 |

                m4u (Multimedia Memory Management Unit)

                 |

            +--------+

            |        |

        gals0-rx   gals1-rx    (Global Async Local Sync rx)

            |        |

            |        |

        gals0-tx   gals1-tx    (Global Async Local Sync tx)

            |        |          Some SoCs may have GALS.

            +--------+

                 |

             SMI Common(Smart Multimedia Interface Common)

                 |

         +----------------+-------

         |                |

         |             gals-rx        There may be GALS in some larbs.

         |                |

         |                |

         |             gals-tx

         |                |

     SMI larb0        SMI larb1   ... SoCs have several SMI local arbiter(larb).

     (display)         (vdec)

         |                |

         |                |

   +-----+-----+     +----+----+

   |     |     |     |    |    |

   |     |     |...  |    |    |  ... There are different ports in each larb.

   |     |     |     |    |    |

  OVL0 RDMA0 WDMA0  MC   PP   VLD



  As above, The Multimedia HW will go through SMI and M4U while it

  access EMI. SMI is a bridge between m4u and the Multimedia HW. It contain

  smi local arbiter and smi common. It will control whether the Multimedia

  HW should go though the m4u for translation or bypass it and talk

  directly with EMI. And also SMI help control the power domain and clocks for

  each local arbiter.



  Normally we specify a local arbiter(larb) for each multimedia HW

  like display, video decode, and camera. And there are different ports

  in each larb. Take a example, There are many ports like MC, PP, VLD in the

  video decode local arbiter, all these ports are according to the video HW.



  In some SoCs, there may be a GALS(Global Async Local Sync) module between

  smi-common and m4u, and additional GALS module between smi-larb and

  smi-common. GALS can been seen as a "asynchronous fifo" which could help

  synchronize for the modules in different clock frequency.



properties:

  compatible:

    oneOf:

      - enum:

          - mediatek,mt2701-m4u  # generation one

          - mediatek,mt2712-m4u  # generation two

          - mediatek,mt6779-m4u  # generation two

          - mediatek,mt8167-m4u  # generation two

          - mediatek,mt8173-m4u  # generation two

          - mediatek,mt8183-m4u  # generation two

          - mediatek,mt8192-m4u  # generation two



      - description: mt7623 generation one

        items:

          - const: mediatek,mt7623-m4u

          - const: mediatek,mt2701-m4u



  reg:

    maxItems: 1



  interrupts:

    maxItems: 1



  clocks:

    items:

      - description: bclk is the block clock.



  clock-names:

    items:

      - const: bclk



  mediatek,larbs:

    $ref: /schemas/types.yaml#/definitions/phandle-array

    minItems: 1

    maxItems: 32

    description: |

      List of phandle to the local arbiters in the current Socs.

      Refer to bindings/memory-controllers/mediatek,smi-larb.yaml. It must sort

      according to the local arbiter index, like larb0, larb1, larb2...



  '#iommu-cells':

    const: 1

    description: |

      This is the mtk_m4u_id according to the HW. Specifies the mtk_m4u_id as

      defined in

      dt-binding/memory/mt2701-larb-port.h for mt2701 and mt7623,

      dt-binding/memory/mt2712-larb-port.h for mt2712,

      dt-binding/memory/mt6779-larb-port.h for mt6779,

      dt-binding/memory/mt8167-larb-port.h for mt8167,

      dt-binding/memory/mt8173-larb-port.h for mt8173,

      dt-binding/memory/mt8183-larb-port.h for mt8183,

      dt-binding/memory/mt8192-larb-port.h for mt8192.



  power-domains:

    maxItems: 1



required:

  - compatible

  - reg

  - interrupts

  - mediatek,larbs

  - '#iommu-cells'



allOf:

  - if:

      properties:

        compatible:

          contains:

            enum:

              - mediatek,mt2701-m4u

              - mediatek,mt2712-m4u

              - mediatek,mt8173-m4u

              - mediatek,mt8192-m4u



    then:

      required:

        - clocks



  - if:

      properties:

        compatible:

          enum:

            - mediatek,mt8192-m4u



    then:

      required:

        - power-domains



additionalProperties: false



examples:

  - |

    #include <dt-bindings/clock/mt8173-clk.h>

    #include <dt-bindings/interrupt-controller/arm-gic.h>



    iommu: iommu@10205000 {

            compatible = "mediatek,mt8173-m4u";

            reg = <0x10205000 0x1000>;

            interrupts = <GIC_SPI 139 IRQ_TYPE_LEVEL_LOW>;

            clocks = <&infracfg CLK_INFRA_M4U>;

            clock-names = "bclk";

            mediatek,larbs = <&larb0 &larb1 &larb2

                              &larb3 &larb4 &larb5>;

            #iommu-cells = <1>;

    };



  - |

    #include <dt-bindings/memory/mt8173-larb-port.h>



    /* Example for a client device */

    display {

           compatible = "mediatek,mt8173-disp";

           iommus = <&iommu M4U_PORT_DISP_OVL0>,

                    <&iommu M4U_PORT_DISP_RDMA0>;

     };

MAINTAINERS

+9 −0
Original line number Diff line number Diff line
@@ -11182,6 +11182,15 @@ S: Maintained 
F:	Documentation/devicetree/bindings/i2c/i2c-mt65xx.txt

F:	drivers/i2c/busses/i2c-mt65xx.c









MEDIATEK IOMMU DRIVER









M:	Yong Wu <yong.wu@mediatek.com>









L:	iommu@lists.linux-foundation.org









L:	linux-mediatek@lists.infradead.org (moderated for non-subscribers)









S:	Supported









F:	Documentation/devicetree/bindings/iommu/mediatek*









F:	drivers/iommu/mtk-iommu*









F:	include/dt-bindings/memory/mt*-port.h

















MEDIATEK JPEG DRIVER









M:	Rick Chang <rick.chang@mediatek.com>









M:	Bin Liu <bin.liu@mediatek.com>
































drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c



+1
−1






























Original line number
Diff line number
Diff line







@@ -2306,7 +2306,7 @@ static void arm_smmu_iotlb_sync(struct iommu_domain *domain,







	struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain);



















	arm_smmu_tlb_inv_range_domain(gather->start,









				      gather->end - gather->start,









				      gather->end - gather->start + 1,









				      gather->pgsize, true, smmu_domain);









}










































drivers/iommu/io-pgtable-arm-v7s.c



+31
−34






























Original line number
Diff line number
Diff line







@@ -44,26 +44,25 @@

















/*









 * We have 32 bits total; 12 bits resolved at level 1, 8 bits at level 2,









 * and 12 bits in a page. With some carefully-chosen coefficients we can









 * hide the ugly inconsistencies behind these macros and at least let the









 * rest of the code pretend to be somewhat sane.









 * and 12 bits in a page.









 * MediaTek extend 2 bits to reach 34bits, 14 bits at lvl1 and 8 bits at lvl2.









 */









#define ARM_V7S_ADDR_BITS		32









#define _ARM_V7S_LVL_BITS(lvl)		(16 - (lvl) * 4)









#define ARM_V7S_LVL_SHIFT(lvl)		(ARM_V7S_ADDR_BITS - (4 + 8 * (lvl)))









#define _ARM_V7S_LVL_BITS(lvl, cfg)	((lvl) == 1 ? ((cfg)->ias - 20) : 8)









#define ARM_V7S_LVL_SHIFT(lvl)		((lvl) == 1 ? 20 : 12)









#define ARM_V7S_TABLE_SHIFT		10



















#define ARM_V7S_PTES_PER_LVL(lvl)	(1 << _ARM_V7S_LVL_BITS(lvl))









#define ARM_V7S_TABLE_SIZE(lvl)						\









	(ARM_V7S_PTES_PER_LVL(lvl) * sizeof(arm_v7s_iopte))









#define ARM_V7S_PTES_PER_LVL(lvl, cfg)	(1 << _ARM_V7S_LVL_BITS(lvl, cfg))









#define ARM_V7S_TABLE_SIZE(lvl, cfg)						\









	(ARM_V7S_PTES_PER_LVL(lvl, cfg) * sizeof(arm_v7s_iopte))



















#define ARM_V7S_BLOCK_SIZE(lvl)		(1UL << ARM_V7S_LVL_SHIFT(lvl))









#define ARM_V7S_LVL_MASK(lvl)		((u32)(~0U << ARM_V7S_LVL_SHIFT(lvl)))









#define ARM_V7S_TABLE_MASK		((u32)(~0U << ARM_V7S_TABLE_SHIFT))









#define _ARM_V7S_IDX_MASK(lvl)		(ARM_V7S_PTES_PER_LVL(lvl) - 1)









#define ARM_V7S_LVL_IDX(addr, lvl)	({				\









#define _ARM_V7S_IDX_MASK(lvl, cfg)	(ARM_V7S_PTES_PER_LVL(lvl, cfg) - 1)









#define ARM_V7S_LVL_IDX(addr, lvl, cfg)	({				\









	int _l = lvl;							\









	((u32)(addr) >> ARM_V7S_LVL_SHIFT(_l)) & _ARM_V7S_IDX_MASK(_l); \









	((addr) >> ARM_V7S_LVL_SHIFT(_l)) & _ARM_V7S_IDX_MASK(_l, cfg); \









})



















/*








@@ -112,9 +111,10 @@







#define ARM_V7S_TEX_MASK		0x7









#define ARM_V7S_ATTR_TEX(val)		(((val) & ARM_V7S_TEX_MASK) << ARM_V7S_TEX_SHIFT)



















/* MediaTek extend the two bits for PA 32bit/33bit */









/* MediaTek extend the bits below for PA 32bit/33bit/34bit */









#define ARM_V7S_ATTR_MTK_PA_BIT32	BIT(9)









#define ARM_V7S_ATTR_MTK_PA_BIT33	BIT(4)









#define ARM_V7S_ATTR_MTK_PA_BIT34	BIT(5)



















/* *well, except for TEX on level 2 large pages, of course :( */









#define ARM_V7S_CONT_PAGE_TEX_SHIFT	6








@@ -194,6 +194,8 @@ static arm_v7s_iopte paddr_to_iopte(phys_addr_t paddr, int lvl,







		pte |= ARM_V7S_ATTR_MTK_PA_BIT32;









	if (paddr & BIT_ULL(33))









		pte |= ARM_V7S_ATTR_MTK_PA_BIT33;









	if (paddr & BIT_ULL(34))









		pte |= ARM_V7S_ATTR_MTK_PA_BIT34;









	return pte;









}


















@@ -218,6 +220,8 @@ static phys_addr_t iopte_to_paddr(arm_v7s_iopte pte, int lvl,







		paddr |= BIT_ULL(32);









	if (pte & ARM_V7S_ATTR_MTK_PA_BIT33)









		paddr |= BIT_ULL(33);









	if (pte & ARM_V7S_ATTR_MTK_PA_BIT34)









		paddr |= BIT_ULL(34);









	return paddr;









}


















@@ -234,7 +238,7 @@ static void *__arm_v7s_alloc_table(int lvl, gfp_t gfp,







	struct device *dev = cfg->iommu_dev;









	phys_addr_t phys;









	dma_addr_t dma;









	size_t size = ARM_V7S_TABLE_SIZE(lvl);









	size_t size = ARM_V7S_TABLE_SIZE(lvl, cfg);









	void *table = NULL;



















	if (lvl == 1)








@@ -280,7 +284,7 @@ static void __arm_v7s_free_table(void *table, int lvl,







{









	struct io_pgtable_cfg *cfg = &data->iop.cfg;









	struct device *dev = cfg->iommu_dev;









	size_t size = ARM_V7S_TABLE_SIZE(lvl);









	size_t size = ARM_V7S_TABLE_SIZE(lvl, cfg);



















	if (!cfg->coherent_walk)









		dma_unmap_single(dev, __arm_v7s_dma_addr(table), size,








@@ -424,7 +428,7 @@ static int arm_v7s_init_pte(struct arm_v7s_io_pgtable *data,







			arm_v7s_iopte *tblp;









			size_t sz = ARM_V7S_BLOCK_SIZE(lvl);



















			tblp = ptep - ARM_V7S_LVL_IDX(iova, lvl);









			tblp = ptep - ARM_V7S_LVL_IDX(iova, lvl, cfg);









			if (WARN_ON(__arm_v7s_unmap(data, NULL, iova + i * sz,









						    sz, lvl, tblp) != sz))









				return -EINVAL;








@@ -477,7 +481,7 @@ static int __arm_v7s_map(struct arm_v7s_io_pgtable *data, unsigned long iova,







	int num_entries = size >> ARM_V7S_LVL_SHIFT(lvl);



















	/* Find our entry at the current level */









	ptep += ARM_V7S_LVL_IDX(iova, lvl);









	ptep += ARM_V7S_LVL_IDX(iova, lvl, cfg);



















	/* If we can install a leaf entry at this level, then do so */









	if (num_entries)








@@ -519,7 +523,6 @@ static int arm_v7s_map(struct io_pgtable_ops *ops, unsigned long iova,







			phys_addr_t paddr, size_t size, int prot, gfp_t gfp)









{









	struct arm_v7s_io_pgtable *data = io_pgtable_ops_to_data(ops);









	struct io_pgtable *iop = &data->iop;









	int ret;



















	if (WARN_ON(iova >= (1ULL << data->iop.cfg.ias) ||








@@ -535,12 +538,7 @@ static int arm_v7s_map(struct io_pgtable_ops *ops, unsigned long iova,







	 * Synchronise all PTE updates for the new mapping before there's









	 * a chance for anything to kick off a table walk for the new iova.









	 */









	if (iop->cfg.quirks & IO_PGTABLE_QUIRK_TLBI_ON_MAP) {









		io_pgtable_tlb_flush_walk(iop, iova, size,









					  ARM_V7S_BLOCK_SIZE(2));









	} else {









	wmb();









	}



















	return ret;









}








@@ -550,7 +548,7 @@ static void arm_v7s_free_pgtable(struct io_pgtable *iop)







	struct arm_v7s_io_pgtable *data = io_pgtable_to_data(iop);









	int i;



















	for (i = 0; i < ARM_V7S_PTES_PER_LVL(1); i++) {









	for (i = 0; i < ARM_V7S_PTES_PER_LVL(1, &data->iop.cfg); i++) {









		arm_v7s_iopte pte = data->pgd[i];



















		if (ARM_V7S_PTE_IS_TABLE(pte, 1))








@@ -602,9 +600,9 @@ static size_t arm_v7s_split_blk_unmap(struct arm_v7s_io_pgtable *data,







	if (!tablep)









		return 0; /* Bytes unmapped */



















	num_ptes = ARM_V7S_PTES_PER_LVL(2);









	num_ptes = ARM_V7S_PTES_PER_LVL(2, cfg);









	num_entries = size >> ARM_V7S_LVL_SHIFT(2);









	unmap_idx = ARM_V7S_LVL_IDX(iova, 2);









	unmap_idx = ARM_V7S_LVL_IDX(iova, 2, cfg);



















	pte = arm_v7s_prot_to_pte(arm_v7s_pte_to_prot(blk_pte, 1), 2, cfg);









	if (num_entries > 1)








@@ -646,7 +644,7 @@ static size_t __arm_v7s_unmap(struct arm_v7s_io_pgtable *data,







	if (WARN_ON(lvl > 2))









		return 0;



















	idx = ARM_V7S_LVL_IDX(iova, lvl);









	idx = ARM_V7S_LVL_IDX(iova, lvl, &iop->cfg);









	ptep += idx;









	do {









		pte[i] = READ_ONCE(ptep[i]);








@@ -717,7 +715,7 @@ static size_t arm_v7s_unmap(struct io_pgtable_ops *ops, unsigned long iova,







{









	struct arm_v7s_io_pgtable *data = io_pgtable_ops_to_data(ops);



















	if (WARN_ON(upper_32_bits(iova)))









	if (WARN_ON(iova >= (1ULL << data->iop.cfg.ias)))









		return 0;



















	return __arm_v7s_unmap(data, gather, iova, size, 1, data->pgd);








@@ -732,7 +730,7 @@ static phys_addr_t arm_v7s_iova_to_phys(struct io_pgtable_ops *ops,







	u32 mask;



















	do {









		ptep += ARM_V7S_LVL_IDX(iova, ++lvl);









		ptep += ARM_V7S_LVL_IDX(iova, ++lvl, &data->iop.cfg);









		pte = READ_ONCE(*ptep);









		ptep = iopte_deref(pte, lvl, data);









	} while (ARM_V7S_PTE_IS_TABLE(pte, lvl));








@@ -751,15 +749,14 @@ static struct io_pgtable *arm_v7s_alloc_pgtable(struct io_pgtable_cfg *cfg,







{









	struct arm_v7s_io_pgtable *data;



















	if (cfg->ias > ARM_V7S_ADDR_BITS)









	if (cfg->ias > (arm_v7s_is_mtk_enabled(cfg) ? 34 : ARM_V7S_ADDR_BITS))









		return NULL;



















	if (cfg->oas > (arm_v7s_is_mtk_enabled(cfg) ? 34 : ARM_V7S_ADDR_BITS))









	if (cfg->oas > (arm_v7s_is_mtk_enabled(cfg) ? 35 : ARM_V7S_ADDR_BITS))









		return NULL;



















	if (cfg->quirks & ~(IO_PGTABLE_QUIRK_ARM_NS |









			    IO_PGTABLE_QUIRK_NO_PERMS |









			    IO_PGTABLE_QUIRK_TLBI_ON_MAP |









			    IO_PGTABLE_QUIRK_ARM_MTK_EXT |









			    IO_PGTABLE_QUIRK_NON_STRICT))









		return NULL;








@@ -775,8 +772,8 @@ static struct io_pgtable *arm_v7s_alloc_pgtable(struct io_pgtable_cfg *cfg,

















	spin_lock_init(&data->split_lock);









	data->l2_tables = kmem_cache_create("io-pgtable_armv7s_l2",









					    ARM_V7S_TABLE_SIZE(2),









					    ARM_V7S_TABLE_SIZE(2),









					    ARM_V7S_TABLE_SIZE(2, cfg),









					    ARM_V7S_TABLE_SIZE(2, cfg),









					    ARM_V7S_TABLE_SLAB_FLAGS, NULL);









	if (!data->l2_tables)









		goto out_free_data;