Merge tag 'linux-kselftest-kunit-6.6-rc1' of...

- ``--json``: If set, stores the test results in a JSON format and prints to `stdout` or

  saves to a file if a filename is specified.

- ``--filter``: Specifies filters on test attributes, for example, ``speed!=slow``.

  Multiple filters can be used by wrapping input in quotes and separating filters

  by commas. Example: ``--filter "speed>slow, module=example"``.

- ``--filter_action``: If set to ``skip``, filtered tests will be shown as skipped

  in the output rather than showing no output.

- ``--list_tests``: If set, lists all tests that will be run.

- ``--list_tests_attr``: If set, lists all tests that will be run and all of their

  attributes.

	# Reset coverage counters before running the test.

	$ echo 0 > /sys/kernel/debug/gcov/reset

	$ modprobe kunit-example-test

Test Attributes and Filtering

=============================

Test suites and cases can be marked with test attributes, such as speed of

test. These attributes will later be printed in test output and can be used to

filter test execution.

Marking Test Attributes

-----------------------

Tests are marked with an attribute by including a ``kunit_attributes`` object

in the test definition.

Test cases can be marked using the ``KUNIT_CASE_ATTR(test_name, attributes)``

macro to define the test case instead of ``KUNIT_CASE(test_name)``.

.. code-block:: c

	static const struct kunit_attributes example_attr = {

		.speed = KUNIT_VERY_SLOW,

	};

	static struct kunit_case example_test_cases[] = {

		KUNIT_CASE_ATTR(example_test, example_attr),

	};

.. note::

	To mark a test case as slow, you can also use ``KUNIT_CASE_SLOW(test_name)``.

	This is a helpful macro as the slow attribute is the most commonly used.

Test suites can be marked with an attribute by setting the "attr" field in the

suite definition.

.. code-block:: c

	static const struct kunit_attributes example_attr = {

		.speed = KUNIT_VERY_SLOW,

	};

	static struct kunit_suite example_test_suite = {

		...,

		.attr = example_attr,

	};

.. note::

	Not all attributes need to be set in a ``kunit_attributes`` object. Unset

	attributes will remain uninitialized and act as though the attribute is set

	to 0 or NULL. Thus, if an attribute is set to 0, it is treated as unset.

	These unset attributes will not be reported and may act as a default value

	for filtering purposes.

Reporting Attributes

--------------------

When a user runs tests, attributes will be present in the raw kernel output (in

KTAP format). Note that attributes will be hidden by default in kunit.py output

for all passing tests but the raw kernel output can be accessed using the

``--raw_output`` flag. This is an example of how test attributes for test cases

will be formatted in kernel output:

.. code-block:: none

	# example_test.speed: slow

	ok 1 example_test

This is an example of how test attributes for test suites will be formatted in

kernel output:

.. code-block:: none

	  KTAP version 2

	  # Subtest: example_suite

	  # module: kunit_example_test

	  1..3

	  ...

	ok 1 example_suite

Additionally, users can output a full attribute report of tests with their

attributes, using the command line flag ``--list_tests_attr``:

.. code-block:: bash

	kunit.py run "example" --list_tests_attr

.. note::

	This report can be accessed when running KUnit manually by passing in the

	module_param ``kunit.action=list_attr``.

Filtering

---------

Users can filter tests using the ``--filter`` command line flag when running

tests. As an example:

.. code-block:: bash

	kunit.py run --filter speed=slow

You can also use the following operations on filters: "<", ">", "<=", ">=",

"!=", and "=". Example:

.. code-block:: bash

	kunit.py run --filter "speed>slow"

This example will run all tests with speeds faster than slow. Note that the

characters < and > are often interpreted by the shell, so they may need to be

quoted or escaped, as above.

Additionally, you can use multiple filters at once. Simply separate filters

using commas. Example:

.. code-block:: bash

	kunit.py run --filter "speed>slow, module=kunit_example_test"

.. note::

	You can use this filtering feature when running KUnit manually by passing

	the filter as a module param: ``kunit.filter="speed>slow, speed<=normal"``.

Filtered tests will not run or show up in the test output. You can use the

``--filter_action=skip`` flag to skip filtered tests instead. These tests will be

shown in the test output in the test but will not run. To use this feature when

running KUnit manually, use the module param ``kunit.filter_action=skip``.

Rules of Filtering Procedure

----------------------------

Since both suites and test cases can have attributes, there may be conflicts

between attributes during filtering. The process of filtering follows these

rules:

- Filtering always operates at a per-test level.

- If a test has an attribute set, then the test's value is filtered on.

- Otherwise, the value falls back to the suite's value.

- If neither are set, the attribute has a global "default" value, which is used.

List of Current Attributes

--------------------------

``speed``

This attribute indicates the speed of a test's execution (how slow or fast the

test is).

This attribute is saved as an enum with the following categories: "normal",

"slow", or "very_slow". The assumed default speed for tests is "normal". This

indicates that the test takes a relatively trivial amount of time (less than

1 second), regardless of the machine it is running on. Any test slower than

this could be marked as "slow" or "very_slow".

The macro ``KUNIT_CASE_SLOW(test_name)`` can be easily used to set the speed

of a test case to "slow".

``module``

This attribute indicates the name of the module associated with the test.

This attribute is automatically saved as a string and is printed for each suite.

Tests can also be filtered using this attribute.

F:	Documentation/dev-tools/kunit/

F:	include/kunit/

F:	lib/kunit/

F:	rust/kernel/kunit.rs

F:	scripts/rustdoc_test_*

F:	tools/testing/kunit/

KERNEL USERMODE HELPER

/* SPDX-License-Identifier: GPL-2.0 */

/*

 * KUnit API to save and access test attributes

 *

 * Copyright (C) 2023, Google LLC.

 * Author: Rae Moar <rmoar@google.com>

 */

#ifndef _KUNIT_ATTRIBUTES_H

#define _KUNIT_ATTRIBUTES_H

/*

 * struct kunit_attr_filter - representation of attributes filter with the

 * attribute object and string input

 */

struct kunit_attr_filter {

	struct kunit_attr *attr;

	char *input;

};

/*

 * Returns the name of the filter's attribute.

 */

const char *kunit_attr_filter_name(struct kunit_attr_filter filter);

/*

 * Print all test attributes for a test case or suite.

 * Output format for test cases: "# <test_name>.<attribute>: <value>"

 * Output format for test suites: "# <attribute>: <value>"

 */

void kunit_print_attr(void *test_or_suite, bool is_test, unsigned int test_level);

/*

 * Returns the number of fitlers in input.

 */

int kunit_get_filter_count(char *input);

/*

 * Parse attributes filter input and return an objects containing the

 * attribute object and the string input of the next filter.

 */

struct kunit_attr_filter kunit_next_attr_filter(char **filters, int *err);

/*

 * Returns a copy of the suite containing only tests that pass the filter.

 */

struct kunit_suite *kunit_filter_attr_tests(const struct kunit_suite *const suite,

		struct kunit_attr_filter filter, char *action, int *err);

#endif /* _KUNIT_ATTRIBUTES_H */

#if !IS_ENABLED(CONFIG_KUNIT)

/* If CONFIG_KUNIT is not enabled, these stubs quietly disappear. */

#define KUNIT_TRIGGER_STATIC_STUB(real_fn_name, args...) do {} while (0)

#define KUNIT_STATIC_STUB_REDIRECT(real_fn_name, args...) do {} while (0)

#else

 * This is a function prologue which is used to allow calls to the current

 * function to be redirected by a KUnit test. KUnit tests can call

 * kunit_activate_static_stub() to pass a replacement function in. The

 * replacement function will be called by KUNIT_TRIGGER_STATIC_STUB(), which

 * replacement function will be called by KUNIT_STATIC_STUB_REDIRECT(), which

 * will then return from the function. If the caller is not in a KUnit context,

 * the function will continue execution as normal.

 *

 * When activated, calls to real_fn_addr from within this test (even if called

 * indirectly) will instead call replacement_addr. The function pointed to by

 * real_fn_addr must begin with the static stub prologue in

 * KUNIT_TRIGGER_STATIC_STUB() for this to work. real_fn_addr and

 * KUNIT_STATIC_STUB_REDIRECT() for this to work. real_fn_addr and

 * replacement_addr must have the same type.

 *

 * The redirection can be disabled again with kunit_deactivate_static_stub().