Merge tag 'mm-nonmm-stable-2023-04-27-16-01' of...

Nicolas Saenz Julienne <nsaenz@kernel.org> <nsaenzjulienne@suse.de>

Nicolas Saenz Julienne <nsaenz@kernel.org> <nsaenzjulienne@suse.com>

Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>

Oleksandr Natalenko <oleksandr@natalenko.name> <oleksandr@redhat.com>

Oleksij Rempel <linux@rempel-privat.de> <bug-track@fisher-privat.net>

Oleksij Rempel <linux@rempel-privat.de> <external.Oleksij.Rempel@de.bosch.com>

Oleksij Rempel <linux@rempel-privat.de> <fixed-term.Oleksij.Rempel@de.bosch.com>

Paul E. McKenney <paulmck@kernel.org> <paulmck@linux.ibm.com>

Paul E. McKenney <paulmck@kernel.org> <paulmck@linux.vnet.ibm.com>

Paul E. McKenney <paulmck@kernel.org> <paulmck@us.ibm.com>

Paul Mackerras <paulus@ozlabs.org> <paulus@samba.org>

Paul Mackerras <paulus@ozlabs.org> <paulus@au1.ibm.com>

Peter A Jonsson <pj@ludd.ltu.se>

Peter Oruba <peter.oruba@amd.com>

Peter Oruba <peter@oruba.de>

e) thrashing

f) direct compact

g) write-protect copy

h) IRQ/SOFTIRQ

and makes these statistics available to userspace through

the taskstats interface.

for a description of the fields pertaining to delay accounting.

It will generally be in the form of counters returning the cumulative

delay seen for cpu, sync block I/O, swapin, memory reclaim, thrash page

cache, direct compact, write-protect copy etc.

cache, direct compact, write-protect copy, IRQ/SOFTIRQ etc.

Taking the difference of two successive readings of a given

counter (say cpu_delay_total) for a task will give the delay

	CPU             count     real total  virtual total    delay total  delay average

	                    8        7000000        6872122        3382277          0.423ms

	IO              count    delay total  delay average

		            0              0              0ms

                   0              0          0.000ms

	SWAP            count    delay total  delay average

	                    0              0              0ms

                       0              0          0.000ms

	RECLAIM         count    delay total  delay average

		            0              0              0ms

                   0              0          0.000ms

	THRASHING       count    delay total  delay average

	                    0              0              0ms

                       0              0          0.000ms

	COMPACT         count    delay total  delay average

	                    0              0              0ms

                       0              0          0.000ms

	WPCOPY          count    delay total  delay average

                            0              0              0ms

                       0              0          0.000ms

	IRQ             count    delay total  delay average

                       0              0          0.000ms

Get IO accounting for pid 1, it works only with -p::

kcov: code coverage for fuzzing

KCOV: code coverage for fuzzing

===============================

kcov exposes kernel code coverage information in a form suitable for coverage-

guided fuzzing (randomized testing). Coverage data of a running kernel is

exported via the "kcov" debugfs file. Coverage collection is enabled on a task

basis, and thus it can capture precise coverage of a single system call.

KCOV collects and exposes kernel code coverage information in a form suitable

for coverage-guided fuzzing. Coverage data of a running kernel is exported via

the ``kcov`` debugfs file. Coverage collection is enabled on a task basis, and

thus KCOV can capture precise coverage of a single system call.

Note that kcov does not aim to collect as much coverage as possible. It aims

to collect more or less stable coverage that is function of syscall inputs.

To achieve this goal it does not collect coverage in soft/hard interrupts

and instrumentation of some inherently non-deterministic parts of kernel is

disabled (e.g. scheduler, locking).

Note that KCOV does not aim to collect as much coverage as possible. It aims

to collect more or less stable coverage that is a function of syscall inputs.

To achieve this goal, it does not collect coverage in soft/hard interrupts

(unless remove coverage collection is enabled, see below) and from some

inherently non-deterministic parts of the kernel (e.g. scheduler, locking).

kcov is also able to collect comparison operands from the instrumented code

(this feature currently requires that the kernel is compiled with clang).

Besides collecting code coverage, KCOV can also collect comparison operands.

See the "Comparison operands collection" section for details.

Besides collecting coverage data from syscall handlers, KCOV can also collect

coverage for annotated parts of the kernel executing in background kernel

tasks or soft interrupts. See the "Remote coverage collection" section for

details.

Prerequisites

-------------

Configure the kernel with::

KCOV relies on compiler instrumentation and requires GCC 6.1.0 or later

or any Clang version supported by the kernel.

        CONFIG_KCOV=y

Collecting comparison operands is supported with GCC 8+ or with Clang.

CONFIG_KCOV requires gcc 6.1.0 or later.

To enable KCOV, configure the kernel with::

If the comparison operands need to be collected, set::

        CONFIG_KCOV=y

To enable comparison operands collection, set::

	CONFIG_KCOV_ENABLE_COMPARISONS=y

Profiling data will only become accessible once debugfs has been mounted::

Coverage data only becomes accessible once debugfs has been mounted::

        mount -t debugfs none /sys/kernel/debug

Coverage collection

-------------------

The following program demonstrates coverage collection from within a test

program using kcov:

The following program demonstrates how to use KCOV to collect coverage for a

single syscall from within a test program:

.. code-block:: c

		perror("ioctl"), exit(1);

	/* Reset coverage from the tail of the ioctl() call. */

	__atomic_store_n(&cover[0], 0, __ATOMIC_RELAXED);

	/* That's the target syscal call. */

	/* Call the target syscall call. */

	read(-1, NULL, 0);

	/* Read number of PCs collected. */

	n = __atomic_load_n(&cover[0], __ATOMIC_RELAXED);

	return 0;

    }

After piping through addr2line output of the program looks as follows::

After piping through ``addr2line`` the output of the program looks as follows::

    SyS_read

    fs/read_write.c:562

    fs/read_write.c:562

If a program needs to collect coverage from several threads (independently),

it needs to open /sys/kernel/debug/kcov in each thread separately.

it needs to open ``/sys/kernel/debug/kcov`` in each thread separately.

The interface is fine-grained to allow efficient forking of test processes.

That is, a parent process opens /sys/kernel/debug/kcov, enables trace mode,

mmaps coverage buffer and then forks child processes in a loop. Child processes

only need to enable coverage (disable happens automatically on thread end).

That is, a parent process opens ``/sys/kernel/debug/kcov``, enables trace mode,

mmaps coverage buffer, and then forks child processes in a loop. The child

processes only need to enable coverage (it gets disabled automatically when

a thread exits).

Comparison operands collection

------------------------------

	return 0;

    }

Note that the kcov modes (coverage collection or comparison operands) are

mutually exclusive.

Note that the KCOV modes (collection of code coverage or comparison operands)

are mutually exclusive.

Remote coverage collection

--------------------------

With KCOV_ENABLE coverage is collected only for syscalls that are issued

from the current process. With KCOV_REMOTE_ENABLE it's possible to collect

coverage for arbitrary parts of the kernel code, provided that those parts

are annotated with kcov_remote_start()/kcov_remote_stop().

This allows to collect coverage from two types of kernel background

threads: the global ones, that are spawned during kernel boot in a limited

number of instances (e.g. one USB hub_event() worker thread is spawned per

USB HCD); and the local ones, that are spawned when a user interacts with

some kernel interface (e.g. vhost workers); as well as from soft

interrupts.

To enable collecting coverage from a global background thread or from a

softirq, a unique global handle must be assigned and passed to the

corresponding kcov_remote_start() call. Then a userspace process can pass

a list of such handles to the KCOV_REMOTE_ENABLE ioctl in the handles

array field of the kcov_remote_arg struct. This will attach the used kcov

device to the code sections, that are referenced by those handles.

Since there might be many local background threads spawned from different

userspace processes, we can't use a single global handle per annotation.

Instead, the userspace process passes a non-zero handle through the

common_handle field of the kcov_remote_arg struct. This common handle gets

saved to the kcov_handle field in the current task_struct and needs to be

passed to the newly spawned threads via custom annotations. Those threads

should in turn be annotated with kcov_remote_start()/kcov_remote_stop().

Internally kcov stores handles as u64 integers. The top byte of a handle

is used to denote the id of a subsystem that this handle belongs to, and

the lower 4 bytes are used to denote the id of a thread instance within

that subsystem. A reserved value 0 is used as a subsystem id for common

handles as they don't belong to a particular subsystem. The bytes 4-7 are

currently reserved and must be zero. In the future the number of bytes

used for the subsystem or handle ids might be increased.

When a particular userspace process collects coverage via a common

handle, kcov will collect coverage for each code section that is annotated

to use the common handle obtained as kcov_handle from the current

task_struct. However non common handles allow to collect coverage

selectively from different subsystems.

Besides collecting coverage data from handlers of syscalls issued from a

userspace process, KCOV can also collect coverage for parts of the kernel

executing in other contexts - so-called "remote" coverage.

Using KCOV to collect remote coverage requires:

1. Modifying kernel code to annotate the code section from where coverage

   should be collected with ``kcov_remote_start`` and ``kcov_remote_stop``.

2. Using ``KCOV_REMOTE_ENABLE`` instead of ``KCOV_ENABLE`` in the userspace

   process that collects coverage.

Both ``kcov_remote_start`` and ``kcov_remote_stop`` annotations and the

``KCOV_REMOTE_ENABLE`` ioctl accept handles that identify particular coverage

collection sections. The way a handle is used depends on the context where the

matching code section executes.

KCOV supports collecting remote coverage from the following contexts:

1. Global kernel background tasks. These are the tasks that are spawned during

   kernel boot in a limited number of instances (e.g. one USB ``hub_event``

   worker is spawned per one USB HCD).

2. Local kernel background tasks. These are spawned when a userspace process

   interacts with some kernel interface and are usually killed when the process

   exits (e.g. vhost workers).

3. Soft interrupts.

For #1 and #3, a unique global handle must be chosen and passed to the

corresponding ``kcov_remote_start`` call. Then a userspace process must pass

this handle to ``KCOV_REMOTE_ENABLE`` in the ``handles`` array field of the

``kcov_remote_arg`` struct. This will attach the used KCOV device to the code

section referenced by this handle. Multiple global handles identifying

different code sections can be passed at once.

For #2, the userspace process instead must pass a non-zero handle through the

``common_handle`` field of the ``kcov_remote_arg`` struct. This common handle

gets saved to the ``kcov_handle`` field in the current ``task_struct`` and

needs to be passed to the newly spawned local tasks via custom kernel code

modifications. Those tasks should in turn use the passed handle in their

``kcov_remote_start`` and ``kcov_remote_stop`` annotations.

KCOV follows a predefined format for both global and common handles. Each

handle is a ``u64`` integer. Currently, only the one top and the lower 4 bytes

are used. Bytes 4-7 are reserved and must be zero.

For global handles, the top byte of the handle denotes the id of a subsystem

this handle belongs to. For example, KCOV uses ``1`` as the USB subsystem id.

The lower 4 bytes of a global handle denote the id of a task instance within

that subsystem. For example, each ``hub_event`` worker uses the USB bus number

as the task instance id.

For common handles, a reserved value ``0`` is used as a subsystem id, as such

handles don't belong to a particular subsystem. The lower 4 bytes of a common

handle identify a collective instance of all local tasks spawned by the

userspace process that passed a common handle to ``KCOV_REMOTE_ENABLE``.

In practice, any value can be used for common handle instance id if coverage

is only collected from a single userspace process on the system. However, if

common handles are used by multiple processes, unique instance ids must be

used for each process. One option is to use the process id as the common

handle instance id.

The following program demonstrates using KCOV to collect coverage from both

local tasks spawned by the process and the global task that handles USB bus #1:

.. code-block:: c

  Gid:    100     100     100     100

  FDSize: 256

  Groups: 100 14 16

  Kthread:    0

  VmPeak:     5004 kB

  VmSize:     5004 kB

  VmLck:         0 kB

 NSpid                       descendant namespace process ID hierarchy

 NSpgid                      descendant namespace process group ID hierarchy

 NSsid                       descendant namespace session ID hierarchy

 Kthread                     kernel thread flag, 1 is yes, 0 is no

 VmPeak                      peak virtual memory size

 VmSize                      total program size

 VmLck                       locked memory size

	Fixes: 1f2e3d4c5b6a ("The first line of the commit specified by the first 12 characters of its SHA-1 ID")

Another tag is used for linking web pages with additional backgrounds or

details, for example a report about a bug fixed by the patch or a document

with a specification implemented by the patch::

details, for example an earlier discussion which leads to the patch or a

document with a specification implemented by the patch::

	Link: https://example.com/somewhere.html  optional-other-stuff

by tools like b4 or a git hook like the one described in

'Documentation/maintainer/configure-git.rst'.

A third kind of tag is used to document who was involved in the development of

If the URL points to a public bug report being fixed by the patch, use the

"Closes:" tag instead::

	Closes: https://example.com/issues/1234  optional-other-stuff

Some bug trackers have the ability to close issues automatically when a

commit with such a tag is applied. Some bots monitoring mailing lists can

also track such tags and take certain actions. Private bug trackers and

invalid URLs are forbidden.

Another kind of tag is used to document who was involved in the development of

the patch. Each of these uses this format::

	tag: Full Name <email address>  optional-other-stuff

 - Reported-by: names a user who reported a problem which is fixed by this

   patch; this tag is used to give credit to the (often underappreciated)

   people who test our code and let us know when things do not work

   correctly. Note, this tag should be followed by a Link: tag pointing to the

   report, unless the report is not available on the web.

   correctly. Note, this tag should be followed by a Closes: tag pointing to

   the report, unless the report is not available on the web. The Link: tag

   can be used instead of Closes: if the patch fixes a part of the issue(s)

   being reported.

 - Cc: the named person received a copy of the patch and had the

   opportunity to comment on it.