Commit 594d3234 authored by Christoph Hellwig's avatar Christoph Hellwig Committed by Alexei Starovoitov
Browse files

bpf, docs: Better document the atomic instructions 



Use proper tables and RST markup to document the atomic instructions
in a structured way.

Signed-off-by: default avatarChristoph Hellwig <hch@lst.de>
Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
Link: https://lore.kernel.org/bpf/20220131183638.3934982-6-hch@lst.de
parent 5ca15b8a

Documentation/bpf/instruction-set.rst

+49 −27
Original line number Diff line number Diff line
@@ -249,39 +249,65 @@ Where size is one of: ``BPF_B``, ``BPF_H``, ``BPF_W``, or ``BPF_DW``. 
Atomic operations

-----------------



eBPF includes atomic operations, which use the immediate field for extra

encoding::

Atomic operations are operations that operate on memory and can not be

interrupted or corrupted by other access to the same memory region

by other eBPF programs or means outside of this specification.



   .imm = BPF_ADD, .code = BPF_ATOMIC | BPF_W  | BPF_STX: lock xadd *(u32 *)(dst_reg + off16) += src_reg

   .imm = BPF_ADD, .code = BPF_ATOMIC | BPF_DW | BPF_STX: lock xadd *(u64 *)(dst_reg + off16) += src_reg

All atomic operations supported by eBPF are encoded as store operations

that use the ``BPF_ATOMIC`` mode modifier as follows:



The basic atomic operations supported are::

  * ``BPF_ATOMIC | BPF_W | BPF_STX`` for 32-bit operations

  * ``BPF_ATOMIC | BPF_DW | BPF_STX`` for 64-bit operations

  * 8-bit and 16-bit wide atomic operations are not supported.



    BPF_ADD

    BPF_AND

    BPF_OR

    BPF_XOR

The imm field is used to encode the actual atomic operation.

Simple atomic operation use a subset of the values defined to encode

arithmetic operations in the imm field to encode the atomic operation:



Each having equivalent semantics with the ``BPF_ADD`` example, that is: the

memory location addresed by ``dst_reg + off`` is atomically modified, with

``src_reg`` as the other operand. If the ``BPF_FETCH`` flag is set in the

immediate, then these operations also overwrite ``src_reg`` with the

value that was in memory before it was modified.

  ========  =====  ===========

  imm       value  description

  ========  =====  ===========

  BPF_ADD   0x00   atomic add

  BPF_OR    0x40   atomic or

  BPF_AND   0x50   atomic and

  BPF_XOR   0xa0   atomic xor

  ========  =====  ===========



The more special operations are::



    BPF_XCHG

``BPF_ATOMIC | BPF_W  | BPF_STX`` with imm = BPF_ADD means::



This atomically exchanges ``src_reg`` with the value addressed by ``dst_reg +

off``. ::

  *(u32 *)(dst_reg + off16) += src_reg



    BPF_CMPXCHG

``BPF_ATOMIC | BPF_DW | BPF_STX`` with imm = BPF ADD means::



This atomically compares the value addressed by ``dst_reg + off`` with

``R0``. If they match it is replaced with ``src_reg``. In either case, the

value that was there before is zero-extended and loaded back to ``R0``.

  *(u64 *)(dst_reg + off16) += src_reg



Note that 1 and 2 byte atomic operations are not supported.

``BPF_XADD`` is a deprecated name for ``BPF_ATOMIC | BPF_ADD``.



In addition to the simple atomic operations, there also is a modifier and

two complex atomic operations:



  ===========  ================  ===========================

  imm          value             description

  ===========  ================  ===========================

  BPF_FETCH    0x01              modifier: return old value

  BPF_XCHG     0xe0 | BPF_FETCH  atomic exchange

  BPF_CMPXCHG  0xf0 | BPF_FETCH  atomic compare and exchange

  ===========  ================  ===========================



The ``BPF_FETCH`` modifier is optional for simple atomic operations, and

always set for the complex atomic operations.  If the ``BPF_FETCH`` flag

is set, then the operation also overwrites ``src_reg`` with the value that

was in memory before it was modified.



The ``BPF_XCHG`` operation atomically exchanges ``src_reg`` with the value

addressed by ``dst_reg + off``.



The ``BPF_CMPXCHG`` operation atomically compares the value addressed by

``dst_reg + off`` with ``R0``. If they match, the value addressed by

``dst_reg + off`` is replaced with ``src_reg``. In either case, the

value that was at ``dst_reg + off`` before the operation is zero-extended

and loaded back to ``R0``.



Clang can generate atomic instructions by default when ``-mcpu=v3`` is

enabled. If a lower version for ``-mcpu`` is set, the only atomic instruction
@@ -289,10 +315,6 @@ Clang can generate is ``BPF_ADD`` *without* ``BPF_FETCH``. If you need to enable 
the atomics features, while keeping a lower ``-mcpu`` version, you can use

``-Xclang -target-feature -Xclang +alu32``.



You may encounter ``BPF_XADD`` - this is a legacy name for ``BPF_ATOMIC``,

referring to the exclusive-add operation encoded when the immediate field is

zero.



64-bit immediate instructions

-----------------------------