Commit 0119c894 authored by Tomeu Vizoso's avatar Tomeu Vizoso Committed by Daniel Vetter
Browse files

drm: Add initial ci/ subdirectory 



Developers can easily execute several tests on different devices
by just pushing their branch to their fork in a repository hosted
on gitlab.freedesktop.org which has an infrastructure to run jobs
in several runners and farms with different devices.

There are also other automated tools that uprev dependencies,
monitor the infra, and so on that are already used by the Mesa
project, and we can reuse them too.

Also, store expectations about what the DRM drivers are supposed
to pass in the IGT test suite. By storing the test expectations
along with the code, we can make sure both stay in sync with each
other so we can know when a code change breaks those expectations.

Also, include a configuration file that points to the out-of-tree
CI scripts.

This will allow all contributors to drm to reuse the infrastructure
already in gitlab.freedesktop.org to test the driver on several
generations of the hardware.

Signed-off-by: default avatarTomeu Vizoso <tomeu.vizoso@collabora.com>
Signed-off-by: default avatarHelen Koike <helen.koike@collabora.com>
Acked-by: default avatarDaniel Stone <daniels@collabora.com>
Acked-by: default avatarRob Clark <robdclark@gmail.com>
Tested-by: default avatarRob Clark <robdclark@gmail.com>
[sima: Remove top-level empty file test, spotted by sfr]
Signed-off-by: default avatarDaniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/msgid/20230811171953.176431-1-helen.koike@collabora.com
parent 3698a75f

Documentation/gpu/automated_testing.rst

0 → 100644
+144 −0
Original line number Diff line number Diff line 
.. SPDX-License-Identifier: GPL-2.0+



=========================================

Automated testing of the DRM subsystem

=========================================



Introduction

============



Making sure that changes to the core or drivers don't introduce regressions can

be very time-consuming when lots of different hardware configurations need to

be tested. Moreover, it isn't practical for each person interested in this

testing to have to acquire and maintain what can be a considerable amount of

hardware.



Also, it is desirable for developers to check for regressions in their code by

themselves, instead of relying on the maintainers to find them and then

reporting back.



There are facilities in gitlab.freedesktop.org to automatically test Mesa that

can be used as well for testing the DRM subsystem. This document explains how

people interested in testing it can use this shared infrastructure to save

quite some time and effort.





Relevant files

==============



drivers/gpu/drm/ci/gitlab-ci.yml

--------------------------------



This is the root configuration file for GitLab CI. Among other less interesting

bits, it specifies the specific version of the scripts to be used. There are

some variables that can be modified to change the behavior of the pipeline:



DRM_CI_PROJECT_PATH

    Repository that contains the Mesa software infrastructure for CI



DRM_CI_COMMIT_SHA

    A particular revision to use from that repository



UPSTREAM_REPO

    URL to git repository containing the target branch



TARGET_BRANCH

    Branch to which this branch is to be merged into



IGT_VERSION

    Revision of igt-gpu-tools being used, from

    https://gitlab.freedesktop.org/drm/igt-gpu-tools



drivers/gpu/drm/ci/testlist.txt

-------------------------------



IGT tests to be run on all drivers (unless mentioned in a driver's *-skips.txt

file, see below).



drivers/gpu/drm/ci/${DRIVER_NAME}-${HW_REVISION}-fails.txt

----------------------------------------------------------



Lists the known failures for a given driver on a specific hardware revision.



drivers/gpu/drm/ci/${DRIVER_NAME}-${HW_REVISION}-flakes.txt

-----------------------------------------------------------



Lists the tests that for a given driver on a specific hardware revision are

known to behave unreliably. These tests won't cause a job to fail regardless of

the result. They will still be run.



drivers/gpu/drm/ci/${DRIVER_NAME}-${HW_REVISION}-skips.txt

-----------------------------------------------------------



Lists the tests that won't be run for a given driver on a specific hardware

revision. These are usually tests that interfere with the running of the test

list due to hanging the machine, causing OOM, taking too long, etc.





How to enable automated testing on your tree

============================================



1. Create a Linux tree in https://gitlab.freedesktop.org/ if you don't have one

yet



2. In your kernel repo's configuration (eg.

https://gitlab.freedesktop.org/janedoe/linux/-/settings/ci_cd), change the

CI/CD configuration file from .gitlab-ci.yml to

drivers/gpu/drm/ci/gitlab-ci.yml.



3. Next time you push to this repository, you will see a CI pipeline being

created (eg. https://gitlab.freedesktop.org/janedoe/linux/-/pipelines)



4. The various jobs will be run and when the pipeline is finished, all jobs

should be green unless a regression has been found.





How to update test expectations

===============================



If your changes to the code fix any tests, you will have to remove one or more

lines from one or more of the files in

drivers/gpu/drm/ci/${DRIVER_NAME}_*_fails.txt, for each of the test platforms

affected by the change.





How to expand coverage

======================



If your code changes make it possible to run more tests (by solving reliability

issues, for example), you can remove tests from the flakes and/or skips lists,

and then the expected results if there are any known failures.



If there is a need for updating the version of IGT being used (maybe you have

added more tests to it), update the IGT_VERSION variable at the top of the

gitlab-ci.yml file.





How to test your changes to the scripts

=======================================



For testing changes to the scripts in the drm-ci repo, change the

DRM_CI_PROJECT_PATH and DRM_CI_COMMIT_SHA variables in

drivers/gpu/drm/ci/gitlab-ci.yml to match your fork of the project (eg.

janedoe/drm-ci). This fork needs to be in https://gitlab.freedesktop.org/.





How to incorporate external fixes in your testing

=================================================



Often, regressions in other trees will prevent testing changes local to the

tree under test. These fixes will be automatically merged in during the build

jobs from a branch in the target tree that is named as

${TARGET_BRANCH}-external-fixes.



If the pipeline is not in a merge request and a branch with the same name

exists in the local tree, commits from that branch will be merged in as well.





How to deal with automated testing labs that may be down

========================================================



If a hardware farm is down and thus causing pipelines to fail that would

otherwise pass, one can disable all jobs that would be submitted to that farm

by editing the file at

https://gitlab.freedesktop.org/gfx-ci/lab-status/-/blob/main/lab-status.yml.

Documentation/gpu/index.rst

+1 −0
Original line number Diff line number Diff line
@@ -17,6 +17,7 @@ GPU Driver Developer's Guide 
   backlight

   vga-switcheroo

   vgaarbiter

   automated_testing

   todo

   rfc/index

MAINTAINERS

+8 −0
Original line number Diff line number Diff line
@@ -7132,6 +7132,14 @@ T: git git://anongit.freedesktop.org/drm/drm-misc 
F:	drivers/gpu/drm/ttm/

F:	include/drm/ttm/









DRM AUTOMATED TESTING









M:	Helen Koike <helen.koike@collabora.com>









L:	dri-devel@lists.freedesktop.org









S:	Maintained









T:	git git://anongit.freedesktop.org/drm/drm-misc









F:	Documentation/gpu/automated_testing.rst









F:	drivers/gpu/drm/ci/

















DSBR100 USB FM RADIO DRIVER









M:	Alexey Klimov <klimov.linux@gmail.com>









L:	linux-media@vger.kernel.org
































drivers/gpu/drm/ci/arm.config
0 → 100644



+69
−0






























Original line number
Diff line number
Diff line









CONFIG_LOCALVERSION_AUTO=y









CONFIG_DEBUG_KERNEL=y



















CONFIG_CRYPTO_ZSTD=y









CONFIG_ZRAM_MEMORY_TRACKING=y









CONFIG_ZRAM_WRITEBACK=y









CONFIG_ZRAM=y









CONFIG_ZSMALLOC_STAT=y



















# abootimg with a 'dummy' rootfs fails with root=/dev/nfs









CONFIG_BLK_DEV_INITRD=n



















CONFIG_DEVFREQ_GOV_PERFORMANCE=y









CONFIG_DEVFREQ_GOV_POWERSAVE=y









CONFIG_DEVFREQ_GOV_USERSPACE=y









CONFIG_DEVFREQ_GOV_PASSIVE=y









CONFIG_DEVFREQ_GOV_SIMPLE_ONDEMAND=y



















CONFIG_DRM=y









CONFIG_DRM_ETNAVIV=y









CONFIG_DRM_ROCKCHIP=y









CONFIG_DRM_PANFROST=y









CONFIG_DRM_LIMA=y









CONFIG_DRM_PANEL_SIMPLE=y









CONFIG_PWM_CROS_EC=y









CONFIG_BACKLIGHT_PWM=y



















CONFIG_ROCKCHIP_CDN_DP=n



















CONFIG_SPI_ROCKCHIP=y









CONFIG_PWM_ROCKCHIP=y









CONFIG_PHY_ROCKCHIP_DP=y









CONFIG_DWMAC_ROCKCHIP=y



















CONFIG_MFD_RK808=y









CONFIG_REGULATOR_RK808=y









CONFIG_RTC_DRV_RK808=y









CONFIG_COMMON_CLK_RK808=y



















CONFIG_REGULATOR_FAN53555=y









CONFIG_REGULATOR=y



















CONFIG_REGULATOR_VCTRL=y



















CONFIG_KASAN=n









CONFIG_KASAN_INLINE=n









CONFIG_STACKTRACE=n



















CONFIG_TMPFS=y



















CONFIG_PROVE_LOCKING=n









CONFIG_DEBUG_LOCKDEP=n









CONFIG_SOFTLOCKUP_DETECTOR=n









CONFIG_BOOTPARAM_SOFTLOCKUP_PANIC=n



















CONFIG_FW_LOADER_COMPRESS=y



















CONFIG_USB_USBNET=y









CONFIG_NETDEVICES=y









CONFIG_USB_NET_DRIVERS=y









CONFIG_USB_RTL8152=y









CONFIG_USB_NET_AX8817X=y









CONFIG_USB_NET_SMSC95XX=y



















# TK1









CONFIG_ARM_TEGRA_DEVFREQ=y



















# 32-bit build failure









CONFIG_DRM_MSM=n
























drivers/gpu/drm/ci/arm64.config
0 → 100644



+199
−0






























Original line number
Diff line number
Diff line









CONFIG_LOCALVERSION_AUTO=y









CONFIG_DEBUG_KERNEL=y



















CONFIG_CRYPTO_ZSTD=y









CONFIG_ZRAM_MEMORY_TRACKING=y









CONFIG_ZRAM_WRITEBACK=y









CONFIG_ZRAM=y









CONFIG_ZSMALLOC_STAT=y



















# abootimg with a 'dummy' rootfs fails with root=/dev/nfs









CONFIG_BLK_DEV_INITRD=n



















CONFIG_DEVFREQ_GOV_PERFORMANCE=y









CONFIG_DEVFREQ_GOV_POWERSAVE=y









CONFIG_DEVFREQ_GOV_USERSPACE=y









CONFIG_DEVFREQ_GOV_PASSIVE=y



















CONFIG_DRM=y









CONFIG_DRM_ROCKCHIP=y









CONFIG_DRM_PANFROST=y









CONFIG_DRM_LIMA=y









CONFIG_DRM_PANEL_SIMPLE=y









CONFIG_DRM_PANEL_EDP=y









CONFIG_DRM_MSM=y









CONFIG_DRM_ETNAVIV=y









CONFIG_DRM_I2C_ADV7511=y









CONFIG_PWM_CROS_EC=y









CONFIG_BACKLIGHT_PWM=y



















CONFIG_ROCKCHIP_CDN_DP=n



















CONFIG_SPI_ROCKCHIP=y









CONFIG_PWM_ROCKCHIP=y









CONFIG_PHY_ROCKCHIP_DP=y









CONFIG_DWMAC_ROCKCHIP=y









CONFIG_STMMAC_ETH=y









CONFIG_TYPEC_FUSB302=y









CONFIG_TYPEC=y









CONFIG_TYPEC_TCPM=y



















# MSM platform bits



















# For CONFIG_QCOM_LMH









CONFIG_OF=y



















CONFIG_ARM_SMMU_QCOM=y









CONFIG_QCOM_COMMAND_DB=y









CONFIG_QCOM_RPMHPD=y









CONFIG_QCOM_RPMPD=y









CONFIG_QCOM_OCMEM=y









CONFIG_SDM_GPUCC_845=y









CONFIG_SDM_VIDEOCC_845=y









CONFIG_SDM_DISPCC_845=y









CONFIG_SDM_LPASSCC_845=y









CONFIG_SDM_CAMCC_845=y









CONFIG_RESET_QCOM_PDC=y









CONFIG_DRM_TI_SN65DSI86=y









CONFIG_I2C_QCOM_GENI=y









CONFIG_SPI_QCOM_GENI=y









CONFIG_PHY_QCOM_QUSB2=y









CONFIG_PHY_QCOM_QMP=y









CONFIG_MSM_GCC_8996=y









CONFIG_QCOM_CLK_APCC_MSM8996=y









CONFIG_QCOM_LLCC=y









CONFIG_QCOM_LMH=y









CONFIG_QCOM_SPMI_TEMP_ALARM=y









CONFIG_QCOM_WDT=y









CONFIG_POWER_RESET_QCOM_PON=y









CONFIG_RTC_DRV_PM8XXX=y









CONFIG_INTERCONNECT=y









CONFIG_INTERCONNECT_QCOM=y









CONFIG_INTERCONNECT_QCOM_MSM8996=y









CONFIG_INTERCONNECT_QCOM_SDM845=y









CONFIG_INTERCONNECT_QCOM_MSM8916=y









CONFIG_INTERCONNECT_QCOM_MSM8996=y









CONFIG_INTERCONNECT_QCOM_OSM_L3=y









CONFIG_INTERCONNECT_QCOM_SC7180=y









CONFIG_INTERCONNECT_QCOM_SM8350=y









CONFIG_CRYPTO_DEV_QCOM_RNG=y









CONFIG_SC_DISPCC_7180=y









CONFIG_SC_GPUCC_7180=y









CONFIG_SM_GPUCC_8350=y









CONFIG_QCOM_SPMI_ADC5=y









CONFIG_DRM_PARADE_PS8640=y









CONFIG_DRM_LONTIUM_LT9611UXC=y









CONFIG_PHY_QCOM_USB_HS=y









CONFIG_QCOM_GPI_DMA=y









CONFIG_USB_ONBOARD_HUB=y









CONFIG_NVMEM_QCOM_QFPROM=y









CONFIG_PHY_QCOM_USB_SNPS_FEMTO_V2=y





























# db410c ethernet









CONFIG_USB_RTL8152=y









# db820c ethernet









CONFIG_ATL1C=y









# Chromebooks ethernet









CONFIG_USB_ONBOARD_HUB=y









# 888 HDK ethernet









CONFIG_USB_LAN78XX=y



















CONFIG_ARCH_ALPINE=n









CONFIG_ARCH_BCM2835=y









CONFIG_ARCH_BCM_IPROC=n









CONFIG_ARCH_BERLIN=n









CONFIG_ARCH_BRCMSTB=n









CONFIG_ARCH_EXYNOS=n









CONFIG_ARCH_K3=n









CONFIG_ARCH_LAYERSCAPE=n









CONFIG_ARCH_LG1K=n









CONFIG_ARCH_HISI=n









CONFIG_ARCH_MVEBU=n









CONFIG_ARCH_SEATTLE=n









CONFIG_ARCH_SYNQUACER=n









CONFIG_ARCH_RENESAS=n









CONFIG_ARCH_R8A774A1=n









CONFIG_ARCH_R8A774C0=n









CONFIG_ARCH_R8A7795=n









CONFIG_ARCH_R8A7796=n









CONFIG_ARCH_R8A77965=n









CONFIG_ARCH_R8A77970=n









CONFIG_ARCH_R8A77980=n









CONFIG_ARCH_R8A77990=n









CONFIG_ARCH_R8A77995=n









CONFIG_ARCH_STRATIX10=n









CONFIG_ARCH_TEGRA=n









CONFIG_ARCH_SPRD=n









CONFIG_ARCH_THUNDER=n









CONFIG_ARCH_THUNDER2=n









CONFIG_ARCH_UNIPHIER=n









CONFIG_ARCH_VEXPRESS=n









CONFIG_ARCH_XGENE=n









CONFIG_ARCH_ZX=n









CONFIG_ARCH_ZYNQMP=n



















# Strip out some stuff we don't need for graphics testing, to reduce









# the build.









CONFIG_CAN=n









CONFIG_WIRELESS=n









CONFIG_RFKILL=n









CONFIG_WLAN=n



















CONFIG_REGULATOR_FAN53555=y









CONFIG_REGULATOR=y



















CONFIG_REGULATOR_VCTRL=y



















CONFIG_KASAN=n









CONFIG_KASAN_INLINE=n









CONFIG_STACKTRACE=n



















CONFIG_TMPFS=y



















CONFIG_PROVE_LOCKING=n









CONFIG_DEBUG_LOCKDEP=n









CONFIG_SOFTLOCKUP_DETECTOR=y









CONFIG_BOOTPARAM_SOFTLOCKUP_PANIC=y



















CONFIG_DETECT_HUNG_TASK=y



















CONFIG_FW_LOADER_COMPRESS=y









CONFIG_FW_LOADER_USER_HELPER=n



















CONFIG_USB_USBNET=y









CONFIG_NETDEVICES=y









CONFIG_USB_NET_DRIVERS=y









CONFIG_USB_RTL8152=y









CONFIG_USB_NET_AX8817X=y









CONFIG_USB_NET_SMSC95XX=y



















# For amlogic









CONFIG_MESON_GXL_PHY=y









CONFIG_MDIO_BUS_MUX_MESON_G12A=y









CONFIG_DRM_MESON=y



















# For Mediatek









CONFIG_DRM_MEDIATEK=y









CONFIG_PWM_MEDIATEK=y









CONFIG_DRM_MEDIATEK_HDMI=y









CONFIG_GNSS=y









CONFIG_GNSS_MTK_SERIAL=y









CONFIG_HW_RANDOM=y









CONFIG_HW_RANDOM_MTK=y









CONFIG_MTK_DEVAPC=y









CONFIG_PWM_MTK_DISP=y









CONFIG_MTK_CMDQ=y



















# For nouveau.  Note that DRM must be a module so that it's loaded after NFS is up to provide the firmware.









CONFIG_ARCH_TEGRA=y









CONFIG_DRM_NOUVEAU=m









CONFIG_DRM_TEGRA=m









CONFIG_R8169=y









CONFIG_STAGING=y









CONFIG_DRM_TEGRA_STAGING=y









CONFIG_TEGRA_HOST1X=y









CONFIG_ARM_TEGRA_DEVFREQ=y









CONFIG_TEGRA_SOCTHERM=y









CONFIG_DRM_TEGRA_DEBUG=y









CONFIG_PWM_TEGRA=y