TLB invalidation is used by more than USM (page faults) so break this
code out into its own file.

Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

We can't currently do this due to TLB invalidation done handler
expecting the seqno being received in-order, with the fast-path a TLB
invalidation done could pass one being processed in the slow-path in an
extreme corner case. Remove TLB invalidation done from the fast-path for
now and in a follow up reenable this once the TLB invalidation done
handler can deal with out of order seqno.

Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

emit_pte assumes the size argument is 4k aligned, this may not be true
for the PTEs emitted for CSS as seen by below call stack:

[   56.734228] xe_migrate_copy:585: size=327680, ccs_start=327680, css_size=1280,4096
[   56.734250] xe_migrate_copy:643: size=262144
[   56.734252] emit_pte:404: ptes=64
[   56.734255] emit_pte:418: chunk=64
[   56.734257] xe_migrate_copy:650: size=1024	@ CCS emit PTE
[   56.734259] emit_pte:404: ptes=1
[   56.734261] emit_pte:418: chunk=1
[   56.734339] xe_migrate_copy:643: size=65536
[   56.734342] emit_pte:404: ptes=16
[   56.734344] emit_pte:418: chunk=16
[   56.734346] xe_migrate_copy:650: size=256	# CCS emit PTE
[   56.734348] emit_pte:404: ptes=1
[   56.734350] emit_pte:418: chunk=1
[   56.734352] xe_res_next:174: size=4096, remaining=0

Update emit_pte to handle sizes less than 4k.

Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

Scratch page is in VRAM, and therefore requires 64K GTT layout. In GGTT
world this just means having 16 consecutive entries, with 64K GTT
alignment for the GTT address of the first entry (also matching physical
alignment). However to keep things simple just dump it into system
memory, like we already do for ppGTT.  While we are here, also give it
known default value.

Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

Spec says we need to use 64K VRAM pages for GGTT on platforms like DG2.
In GGTT this just means aligning the GTT address to 64K and ensuring
that we have 16 consecutive entries each pointing to the respective 4K
entry. We already ensure we have 64K pages underneath, so it's just a
case of forcing the GTT alignment.

Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

On DG2 when running the xe_vm IGT, the kernel generates loads of CAT
errors and GT resets (sometimes at least).  On small-bar systems seems
to trigger a lot more easily (maybe due to difference in allocation
strategy). Appears to be related to scratch, since we seem to use the
64K TLB hint on scratch entries, even though the scratch page is a 4K
vram page. Bumping the scratch page size and physical alignment seems
to fix it. Or at least we no longer hit:

[  148.872683] xe 0000:03:00.0: [drm] Engine memory cat error: guc_id=0
[  148.872701] xe 0000:03:00.0: [drm] Engine memory cat error: guc_id=0
[  148.875108] WARNING: CPU: 0 PID: 953 at drivers/gpu/drm/xe/xe_guc_submit.c:797

However to keep things simple, so we don't have to deal with 64K TLB
hints, just move the scratch page into system memory on platforms that
require 64K VRAM pages.

Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

We need to ensure we don't leak the contents to userspace.

Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

On DGFX this blows up if can call this with a system memory object:

XE_BUG_ON(!mem_type_is_vram(place->mem_type) && place->mem_type != XE_PL_STOLEN);

If we consider dpt it looks like we can already in theory hit this, if
we run out of vram and stolen vram. It at least seems reasonable to
allow calling this on any object which supports CPU access.

Note this also changes the behaviour with stolen VRAM and suspend, such
that we no longer attempt to migrate stolen objects into system memory.
However nothing in stolen should ever need to be restored (same on
integrated), so should be fine. Also on small-bar systems the stolen
portion is pretty much always non-CPU accessible, and currently pinned
objects use plain memcpy when being moved, which doesn't play nicely.

Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

I saw a flicker when booting xe, and it's very likely that the original
FB was not mapped at the same place when inheriting, fix it.

Signed-off-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

The comparison with < 0 suggests that the memory device access
should be signed to handle underflow. This makes it work more reliably.

As a result, the max refcount is now S32_MAX instead.

Signed-off-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

This is intended to get some properties that are of interest of UMDs
like the ban state.

Cc: Matthew Brost <matthew.brost@intel.com>
Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

Engine property get uAPI will be added, so to avoid ambiguity here
renaming XE_ENGINE_PROPERTY_X to XE_ENGINE_SET_PROPERTY_X.

No changes in behavior.

Cc: Matthew Brost <matthew.brost@intel.com>
Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

This aligns with other drivers and fixes build failure when
CONFIG_PM_SLEEP is not set, such as on RISC-V.

Signed-off-by: Francois Dugast <francois.dugast@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

Some tests are meant to run only on real hardware. Skip those,
if no device was found.

Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
Reviewed-by: Lucas De Marchi <lucas.demarchi@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

When adding the frequency management, Meteor Lake platform
was left behind. Handling it properly now.

Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Cc: Francois Dugast <francois.dugast@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

[  117.901473] xe 0000:00:02.0: [drm] GuC load failed: status = 0x400000A0
[  117.901506] xe 0000:00:02.0: [drm] GuC load failed: status: Reset = 0, BootROM = 0x50, UKernel = 0x00, MIA = 0x00, Auth = 0x01

Signed-off-by: Philippe Lecluse <philippe.lecluse@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

At present the interrupts are enabled while initializing the last GT.
But this is incorrect for a Multi-GT platform, as root GT initialization
will fail with interrupt disabled. Interrupts are required for
the GuC submission triggered during initialization.
Enable the interrupt during the root GT initialization.

Signed-off-by: Balasubramani Vivekanandan <balasubramani.vivekanandan@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

ERROR:root:../drivers/gpu/drm/xe/display/intel_fbdev.c:585:5: error: redefinition of ‘intel_fbdev_init’
  585 | int intel_fbdev_init(struct drm_device *dev)
      |     ^~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/display/intel_fbdev.c:55:
../drivers/gpu/drm/xe/display/intel_fbdev.h:26:19: note: previous definition of ‘intel_fbdev_init’ with type ‘int(struct drm_device *)’
   26 | static inline int intel_fbdev_init(struct drm_device *dev)
      |                   ^~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/display/intel_fbdev.c:626:6: error: redefinition of ‘intel_fbdev_initial_config_async’
  626 | void intel_fbdev_initial_config_async(struct drm_device *dev)
      |      ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/display/intel_fbdev.h:31:20: note: previous definition of ‘intel_fbdev_initial_config_async’ with type ‘void(struct drm_device *)’
   31 | static inline void intel_fbdev_initial_config_async(struct drm_device *dev)
      |                    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/display/intel_fbdev.c:646:6: error: redefinition of ‘intel_fbdev_unregister’
  646 | void intel_fbdev_unregister(struct drm_i915_private *dev_priv)
      |      ^~~~~~~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/display/intel_fbdev.h:35:20: note: previous definition of ‘intel_fbdev_unregister’ with type ‘void(struct xe_device *)’
   35 | static inline void intel_fbdev_unregister(struct drm_i915_private *dev_priv)
      |                    ^~~~~~~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/display/intel_fbdev.c:661:6: error: redefinition of ‘intel_fbdev_fini’
  661 | void intel_fbdev_fini(struct drm_i915_private *dev_priv)
      |      ^~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/display/intel_fbdev.h:39:20: note: previous definition of ‘intel_fbdev_fini’ with type ‘void(struct xe_device *)’
   39 | static inline void intel_fbdev_fini(struct drm_i915_private *dev_priv)
      |                    ^~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/display/intel_fbdev.c:692:6: error: redefinition of ‘intel_fbdev_set_suspend’
  692 | void intel_fbdev_set_suspend(struct drm_device *dev, int state, bool synchronous)
      |      ^~~~~~~~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/display/intel_fbdev.h:43:20: note: previous definition of ‘intel_fbdev_set_suspend’ with type ‘void(struct drm_device *, int,  bool)’ {aka ‘void(struct drm_device *, int,  _Bool)’}
   43 | static inline void intel_fbdev_set_suspend(struct drm_device *dev, int state, bool synchronous)
      |                    ^~~~~~~~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/display/intel_fbdev.c:751:6: error: redefinition of ‘intel_fbdev_output_poll_changed’
  751 | void intel_fbdev_output_poll_changed(struct drm_device *dev)
      |      ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/display/intel_fbdev.h:47:20: note: previous definition of ‘intel_fbdev_output_poll_changed’ with type ‘void(struct drm_device *)’
   47 | static inline void intel_fbdev_output_poll_changed(struct drm_device *dev)
      |                    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/display/intel_fbdev.c:770:6: error: redefinition of ‘intel_fbdev_restore_mode’
  770 | void intel_fbdev_restore_mode(struct drm_device *dev)
      |      ^~~~~~~~~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/display/intel_fbdev.h:51:20: note: previous definition of ‘intel_fbdev_restore_mode’ with type ‘void(struct drm_device *)’
   51 | static inline void intel_fbdev_restore_mode(struct drm_device *dev)
      |                    ^~~~~~~~~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/display/intel_fbdev.c:785:27: error: redefinition of ‘intel_fbdev_framebuffer’
  785 | struct intel_framebuffer *intel_fbdev_framebuffer(struct intel_fbdev *fbdev)
      |                           ^~~~~~~~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/display/intel_fbdev.h:54:41: note: previous definition of ‘intel_fbdev_framebuffer’ with type ‘struct intel_framebuffer *(struct intel_fbdev *)’
   54 | static inline struct intel_framebuffer *intel_fbdev_framebuffer(struct intel_fbdev *fbdev)
      |                                         ^~~~~~~~~~~~~~~~~~~~~~~

Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

KUnit reuquires debugfs, as otherwise, it won't build:

$ make ARCH=x86_64 O=.kunit --jobs=8
ERROR:root:../drivers/gpu/drm/xe/display/intel_display_debugfs.c:1612:6: error: redefinition of ‘intel_display_debugfs_register’
 1612 | void intel_display_debugfs_register(struct drm_i915_private *i915)
      |      ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/display/intel_display_debugfs.c:18:
../drivers/gpu/drm/xe/display/intel_display_debugfs.h:18:20: note: previous definition of ‘intel_display_debugfs_register’ with type ‘void(struct xe_device *)’
   18 | static inline void intel_display_debugfs_register(struct drm_i915_private *i915) {}
      |                    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/display/intel_display_debugfs.c:1935:6: error: redefinition of ‘intel_connector_debugfs_add’
 1935 | void intel_connector_debugfs_add(struct intel_connector *intel_connector)
      |      ^~~~~~~~~~~~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/display/intel_display_debugfs.h:19:20: note: previous definition of ‘intel_connector_debugfs_add’ with type ‘void(struct intel_connector *)’
   19 | static inline void intel_connector_debugfs_add(struct intel_connector *connector) {}
      |                    ^~~~~~~~~~~~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/display/intel_display_debugfs.c:1993:6: error: redefinition of ‘intel_crtc_debugfs_add’
 1993 | void intel_crtc_debugfs_add(struct drm_crtc *crtc)
      |      ^~~~~~~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/display/intel_display_debugfs.h:20:20: note: previous definition of ‘intel_crtc_debugfs_add’ with type ‘void(struct drm_crtc *)’
   20 | static inline void intel_crtc_debugfs_add(struct drm_crtc *crtc) {}
      |                    ^~~~~~~~~~~~~~~~~~~~~~

Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

Some of the tests may benefit from running with ARCH=um, forgoing any
additional setup on the CI build side. Add min config for that.

Tested with:

	./tools/testing/kunit/kunit.py build \
		--kunitconfig drivers/gpu/drm/xe/.kunitconfig \
		--jobs $(nproc) \
		--build_dir build_kunit

Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Cc: Mauro Carvalho Chehab <mchehab@kernel.org>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: Mauro Carvalho Chehab <mchehab@kernel.org>

mem_type field was added in commit d8b52a02

 ("drm/xe: Implement
stolen memory.") to designate the TTM memory type for that mgr. Add
kernel-doc with its description.

Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

Fix the following error while building for 32b:

	In file included from ../drivers/gpu/drm/xe/xe_gt.c:6:
	../drivers/gpu/drm/xe/xe_gt.c: In function ‘gt_ttm_mgr_init’:
	../include/linux/minmax.h:20:35: error: comparison of distinct pointer types lacks a cast [-Werror]
	   20 |         (!!(sizeof((typeof(x) *)1 == (typeof(y) *)1)))
	      |                                   ^~

Cast it to u64 so size of the second operand matches the first one when
building it for 32 bits.

Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

Leave the types as u64, but cast the pointers to unsigned long before
assigning so the compiler doesn't throw warning about casting a pointer
to integer of different size.

Also, size_t should use %zu, not %ld.

Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

writeq() and readq() and other functions working on 64 bit variables
are not provided by 32b arch. For that it's needed to choose between
linux/io-64-nonatomic-hi-lo.h and linux/io-64-nonatomic-lo-hi.h,
spliting the read/write in 2 accesses. For xe driver, it doesn't matter
much, so just choose one and include in xe_mmio.h.

This also removes some ifdef CONFIG_64BIT we had around because of the
missing 64bit functions.

Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

Either use the proper format or cast up to 64b depending on the case.

Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

Make sure it's 64bit value in both 32b and 64b arch.

Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

As also cause issue on PVC, moving back
to what we did before stolen was
introduced

Signed-off-by: Philippe Lecluse <philippe.lecluse@intel.com>
Reviewed-by: Matthew Auld <matthew.auld@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

On DG2 we are now getting:

[  104.456607] xe 0000:03:00.0: [drm] *ERROR* PCODE timeout, retrying with preemption disabled

Looks like we just need to invert the error check for
xe_pcode_try_request(), which returns zero on success.

Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

On DG1, BAR2 is not reliable for reporting Vram size, need to use GSMBASE.
Simplify xe_mmio_total_vram_size to report vram size and usable size.

Signed-off-by: Philippe Lecluse <philippe.lecluse@intel.com>
Reviewed-by: Matthew Auld <matthew.auld@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

The bo_create ioctl relied on the internal ordering of memory regions to
be the same, make sure we don't allocate stolen instead of VRAM0.

Also remove a debug warning left in.

Signed-off-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Reviewed-by: Philippe Lecluse <philippe.lecluse1@gmail.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

SRIOV has a use case of GuC MMIO send / recv, add a function for it.

Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Philippe Lecluse <philippe.lecluse1@gmail.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

Now aligns with the xe_guc_ct_send naming.

Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Philippe Lecluse <philippe.lecluse1@gmail.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

Starting in 70.6.* GuC firmware the CSS header includes the submission
version, pull this from the CSS header. Prior 70.* versions accidentally
omitted this informatio so hard code to the correct values. This
information will be used by VFs when communicating with the PF.

Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Philippe Lecluse <philippe.lecluse1@gmail.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

The blob doesn't fully support this yet, so fake for now to ensure our
driver load order is correct.

Once the blob supports pulling gt->info.engine_mask from the blob, this
patch can be removed.

Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

Fix the below splat:

[  142.510525] [IGT] xe_exec_basic: starting subtest once-userptr
[  142.511339] BUG: kernel NULL pointer dereference, address: 0000000000000228
[  142.518311] #PF: supervisor read access in kernel mode
[  142.523458] #PF: error_code(0x0000) - not-present page
[  142.528604] PGD 0 P4D 0
[  142.531153] Oops: 0000 [#1] PREEMPT SMP NOPTI
[  142.535518] CPU: 4 PID: 1199 Comm: kworker/u16:8 Not tainted 6.1.0-rc1-xe+ #1
[  142.542656] Hardware name: Intel Corporation Tiger Lake Client Platform/TigerLake U DDR4 SODIMM RVP, BIOS TGLSFWI1.R00.3243.A01.2006102133 06/10/2020
[  142.556033] Workqueue: events_unbound async_op_work_func [xe]
[  142.561810] RIP: 0010:xe_bo_is_stolen+0x0/0x20 [xe]
[  142.566709] Code: 20 c8 75 05 83 fa 07 74 05 c3 cc cc cc cc 48 8b 87 08 02 00 00 0f b6 80 2c ff ff ff c3 cc cc cc cc 66 0f 1f 84 00 00 00 00 00 <48> 8b 87 28 02 00 00 83 78 10 07 0f 94 c0 c3 cc cc cc cc 66 66 2e
[  142.585447] RSP: 0018:ffffc900019eb888 EFLAGS: 00010246
[  142.590678] RAX: 0000000000000002 RBX: 0000000000000000 RCX: ffff88813f6a2108
[  142.597821] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000
[  142.604962] RBP: ffffc900019ebbc0 R08: 0000000000000001 R09: 0000000000000000
[  142.612101] R10: 0000000000000001 R11: 0000000000000001 R12: ffff88814107d600
[  142.619242] R13: ffffc900019eba20 R14: ffff888140442000 R15: 0000000000000000
[  142.626378] FS:  0000000000000000(0000) GS:ffff88849fa00000(0000) knlGS:0000000000000000
[  142.634468] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  142.640219] CR2: 0000000000000228 CR3: 000000010a4c0006 CR4: 0000000000770ee0
[  142.647361] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[  142.654505] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[  142.661639] PKRU: 55555554
[  142.664367] Call Trace:
[  142.666830]  <TASK>
[  142.668947]  __xe_pt_bind_vma+0x1a1/0xa50 [xe]
[  142.673417]  ? unwind_next_frame+0x187/0x770
[  142.677699]  ? __thaw_task+0xc0/0xc0
[  142.681293]  ? __lock_acquire+0x5e4/0x26e0
[  142.685409]  ? lockdep_hardirqs_on+0xbf/0x140
[  142.689779]  ? lock_acquire+0xd2/0x310
[  142.693548]  ? mark_held_locks+0x49/0x80
[  142.697485]  ? xe_vm_bind_vma+0xf1/0x3d0 [xe]
[  142.701866]  xe_vm_bind_vma+0xf1/0x3d0 [xe]
[  142.706082]  xe_vm_bind+0x76/0x140 [xe]
[  142.709944]  vm_bind_ioctl+0x26f/0xb40 [xe]
[  142.714161]  ? async_op_work_func+0x20c/0x450 [xe]
[  142.718974]  async_op_work_func+0x20c/0x450 [xe]
[  142.723620]  process_one_work+0x263/0x580
[  142.727645]  ? process_one_work+0x580/0x580
[  142.731839]  worker_thread+0x4d/0x3b0
[  142.735518]  ? process_one_work+0x580/0x580
[  142.739714]  kthread+0xeb/0x120
[  142.742872]  ? kthread_complete_and_exit+0x20/0x20
[  142.747671]  ret_from_fork+0x1f/0x30
[  142.751264]  </TASK>

Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

This adds support for stolen memory, with the same allocator as
vram_mgr. This allows us to skip a whole lot of copy-paste,
by re-using parts of xe_ttm_vram_mgr.

The stolen memory may be bound using VM_BIND, so it performs like any
other memory region.

We should be able to map a stolen BO directly using the physical memory
location instead of through GGTT even on old platforms, but I don't know
what the effects are on coherency.

Signed-off-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

When a job is inflight we may access memory to read the hardware seqno.
All user jobs have VM open which has a ref but kernel jobs do not
require VM so it is possible to not have memory ref. To avoid this, take
a memory ref on kernel job creation.

Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

Add kerneldoc for structs and external functions.

Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Cc: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: Mauro Carvalho Chehab <mchehab@kernel.org>

Expand xe_mmio_wait32 to accept atomic and then use
that directly when possible, and create own routine to
wait for the pcode status.

Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>

Another clean-up towards killing the usage of i915_utils.h

Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>