Commit be315829 authored by Jakub Kicinski's avatar Jakub Kicinski
Browse files
Andrii Nakryiko says:

====================
bpf-next 2021-12-10 v2

We've added 115 non-merge commits during the last 26 day(s) which contain
a total of 182 files changed, 5747 insertions(+), 2564 deletions(-).

The main changes are:

1) Various samples fixes, from Alexander Lobakin.

2) BPF CO-RE support in kernel and light skeleton, from Alexei Starovoitov.

3) A batch of new unified APIs for libbpf, logging improvements, version
   querying, etc. Also a batch of old deprecations for old APIs and various
   bug fixes, in preparation for libbpf 1.0, from Andrii Nakryiko.

4) BPF documentation reorganization and improvements, from Christoph Hellwig
   and Dave Tucker.

5) Support for declarative initialization of BPF_MAP_TYPE_PROG_ARRAY in
   libbpf, from Hengqi Chen.

6) Verifier log fixes, from Hou Tao.

7) Runtime-bounded loops support with bpf_loop() helper, from Joanne Koong.

8) Extend branch record capturing to all platforms that support it,
   from Kajol Jain.

9) Light skeleton codegen improvements, from Kumar Kartikeya Dwivedi.

10) bpftool doc-generating script improvements, from Quentin Monnet.

11) Two libbpf v0.6 bug fixes, from Shuyi Cheng and Vincent Minet.

12) Deprecation warning fix for perf/bpf_counter, from Song Liu.

13) MAX_TAIL_CALL_CNT unification and MIPS build fix for libbpf,
    from Tiezhu Yang.

14) BTF_KING_TYPE_TAG follow-up fixes, from Yonghong Song.

15) Selftests fixes and improvements, from Ilya Leoshkevich, Jean-Philippe
    Brucker, Jiri Olsa, Maxim Mikityanskiy, Tirthendu Sarkar, Yucong Sun,
    and others.

* https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next: (115 commits)
  libbpf: Add "bool skipped" to struct bpf_map
  libbpf: Fix typo in btf__dedup@LIBBPF_0.0.2 definition
  bpftool: Switch bpf_object__load_xattr() to bpf_object__load()
  selftests/bpf: Remove the only use of deprecated bpf_object__load_xattr()
  selftests/bpf: Add test for libbpf's custom log_buf behavior
  selftests/bpf: Replace all uses of bpf_load_btf() with bpf_btf_load()
  libbpf: Deprecate bpf_object__load_xattr()
  libbpf: Add per-program log buffer setter and getter
  libbpf: Preserve kernel error code and remove kprobe prog type guessing
  libbpf: Improve logging around BPF program loading
  libbpf: Allow passing user log setting through bpf_object_open_opts
  libbpf: Allow passing preallocated log_buf when loading BTF into kernel
  libbpf: Add OPTS-based bpf_btf_load() API
  libbpf: Fix bpf_prog_load() log_buf logic for log_level 0
  samples/bpf: Remove unneeded variable
  bpf: Remove redundant assignment to pointer t
  selftests/bpf: Fix a compilation warning
  perf/bpf_counter: Use bpf_map_create instead of bpf_create_map
  samples: bpf: Fix 'unknown warning group' build warning on Clang
  samples: bpf: Fix xdp_sample_user.o linking with Clang
  ...
====================

Link: https://lore.kernel.org/r/20211210234746.2100561-1-andrii@kernel.org


Signed-off-by: default avatarJakub Kicinski <kuba@kernel.org>
parents db104154 229fae38
Loading
Loading
Loading
Loading
+22 −22
Original line number Diff line number Diff line
@@ -3,7 +3,7 @@ BPF Type Format (BTF)
=====================

1. Introduction
***************
===============

BTF (BPF Type Format) is the metadata format which encodes the debug info
related to BPF program/map. The name BTF was used initially to describe data
@@ -30,7 +30,7 @@ sections are discussed in details in :ref:`BTF_Type_String`.
.. _BTF_Type_String:

2. BTF Type and String Encoding
*******************************
===============================

The file ``include/uapi/linux/btf.h`` provides high-level definition of how
types/strings are encoded.
@@ -57,13 +57,13 @@ little-endian target. The ``btf_header`` is designed to be extensible with
generated.

2.1 String Encoding
===================
-------------------

The first string in the string section must be a null string. The rest of
string table is a concatenation of other null-terminated strings.

2.2 Type Encoding
=================
-----------------

The type id ``0`` is reserved for ``void`` type. The type section is parsed
sequentially and type id is assigned to each recognized type starting from id
@@ -504,7 +504,7 @@ valid index (starting from 0) pointing to a member or an argument.
 * ``type``: the type with ``btf_type_tag`` attribute

3. BTF Kernel API
*****************
=================

The following bpf syscall command involves BTF:
   * BPF_BTF_LOAD: load a blob of BTF data into kernel
@@ -547,14 +547,14 @@ The workflow typically looks like:


3.1 BPF_BTF_LOAD
================
----------------

Load a blob of BTF data into kernel. A blob of data, described in
:ref:`BTF_Type_String`, can be directly loaded into the kernel. A ``btf_fd``
is returned to a userspace.

3.2 BPF_MAP_CREATE
==================
------------------

A map can be created with ``btf_fd`` and specified key/value type id.::

@@ -581,7 +581,7 @@ automatically.
.. _BPF_Prog_Load:

3.3 BPF_PROG_LOAD
=================
-----------------

During prog_load, func_info and line_info can be passed to kernel with proper
values for the following attributes:
@@ -631,7 +631,7 @@ For line_info, the line number and column number are defined as below:
    #define BPF_LINE_INFO_LINE_COL(line_col)        ((line_col) & 0x3ff)

3.4 BPF_{PROG,MAP}_GET_NEXT_ID
==============================
------------------------------

In kernel, every loaded program, map or btf has a unique id. The id won't
change during the lifetime of a program, map, or btf.
@@ -641,13 +641,13 @@ each command, to user space, for bpf program or maps, respectively, so an
inspection tool can inspect all programs and maps.

3.5 BPF_{PROG,MAP}_GET_FD_BY_ID
===============================
-------------------------------

An introspection tool cannot use id to get details about program or maps.
A file descriptor needs to be obtained first for reference-counting purpose.

3.6 BPF_OBJ_GET_INFO_BY_FD
==========================
--------------------------

Once a program/map fd is acquired, an introspection tool can get the detailed
information from kernel about this fd, some of which are BTF-related. For
@@ -656,7 +656,7 @@ example, ``bpf_map_info`` returns ``btf_id`` and key/value type ids.
bpf byte codes, and jited_line_info.

3.7 BPF_BTF_GET_FD_BY_ID
========================
------------------------

With ``btf_id`` obtained in ``bpf_map_info`` and ``bpf_prog_info``, bpf
syscall command BPF_BTF_GET_FD_BY_ID can retrieve a btf fd. Then, with
@@ -668,10 +668,10 @@ tool has full btf knowledge and is able to pretty print map key/values, dump
func signatures and line info, along with byte/jit codes.

4. ELF File Format Interface
****************************
============================

4.1 .BTF section
================
----------------

The .BTF section contains type and string data. The format of this section is
same as the one describe in :ref:`BTF_Type_String`.
@@ -679,7 +679,7 @@ same as the one describe in :ref:`BTF_Type_String`.
.. _BTF_Ext_Section:

4.2 .BTF.ext section
====================
--------------------

The .BTF.ext section encodes func_info and line_info which needs loader
manipulation before loading into the kernel.
@@ -743,7 +743,7 @@ bpf_insn``. For ELF API, the ``insn_off`` is the byte offset from the
beginning of section (``btf_ext_info_sec->sec_name_off``).

4.2 .BTF_ids section
====================
--------------------

The .BTF_ids section encodes BTF ID values that are used within the kernel.

@@ -804,10 +804,10 @@ All the BTF ID lists and sets are compiled in the .BTF_ids section and
resolved during the linking phase of kernel build by ``resolve_btfids`` tool.

5. Using BTF
************
============

5.1 bpftool map pretty print
============================
----------------------------

With BTF, the map key/value can be printed based on fields rather than simply
raw bytes. This is especially valuable for large structure or if your data
@@ -849,7 +849,7 @@ bpftool is able to pretty print like below:
      ]

5.2 bpftool prog dump
=====================
---------------------

The following is an example showing how func_info and line_info can help prog
dump with better kernel symbol names, function prototypes and line
@@ -883,7 +883,7 @@ information.::
    [...]

5.3 Verifier Log
================
----------------

The following is an example of how line_info can help debugging verification
failure.::
@@ -909,7 +909,7 @@ failure.::
        R2 offset is outside of the packet

6. BTF Generation
*****************
=================

You need latest pahole

@@ -1016,6 +1016,6 @@ format.::
            .long   8206                    # Line 8 Col 14

7. Testing
**********
==========

Kernel bpf selftest `test_btf.c` provides extensive set of BTF-related tests.
+11 −0
Original line number Diff line number Diff line
================================
Frequently asked questions (FAQ)
================================

Two sets of Questions and Answers (Q&A) are maintained.

.. toctree::
   :maxdepth: 1

   bpf_design_QA
   bpf_devel_QA
+7 −0
Original line number Diff line number Diff line
Helper functions
================

* `bpf-helpers(7)`_ maintains a list of helpers available to eBPF programs.

.. Links
.. _bpf-helpers(7): https://man7.org/linux/man-pages/man7/bpf-helpers.7.html
 No newline at end of file
+15 −87
Original line number Diff line number Diff line
@@ -5,104 +5,32 @@ BPF Documentation
This directory contains documentation for the BPF (Berkeley Packet
Filter) facility, with a focus on the extended BPF version (eBPF).

This kernel side documentation is still work in progress. The main
textual documentation is (for historical reasons) described in
:ref:`networking-filter`, which describe both classical and extended
BPF instruction-set.
This kernel side documentation is still work in progress.
The Cilium project also maintains a `BPF and XDP Reference Guide`_
that goes into great technical depth about the BPF Architecture.

libbpf
======

Documentation/bpf/libbpf/index.rst is a userspace library for loading and interacting with bpf programs.

BPF Type Format (BTF)
=====================

.. toctree::
   :maxdepth: 1

   instruction-set
   verifier
   libbpf/index
   btf


Frequently asked questions (FAQ)
================================

Two sets of Questions and Answers (Q&A) are maintained.

.. toctree::
   :maxdepth: 1

   bpf_design_QA
   bpf_devel_QA

Syscall API
===========

The primary info for the bpf syscall is available in the `man-pages`_
for `bpf(2)`_. For more information about the userspace API, see
Documentation/userspace-api/ebpf/index.rst.

Helper functions
================

* `bpf-helpers(7)`_ maintains a list of helpers available to eBPF programs.


Program types
=============

.. toctree::
   :maxdepth: 1

   prog_cgroup_sockopt
   prog_cgroup_sysctl
   prog_flow_dissector
   bpf_lsm
   prog_sk_lookup


Map types
=========

.. toctree::
   :maxdepth: 1

   map_cgroup_storage


Testing and debugging BPF
=========================

.. toctree::
   :maxdepth: 1

   drgn
   s390


Licensing
=========

.. toctree::
   :maxdepth: 1

   faq
   syscall_api
   helpers
   programs
   maps
   bpf_licensing
   test_debug
   other

.. only::  subproject and html

Other
=====

.. toctree::
   :maxdepth: 1
   Indices
   =======

   ringbuf
   llvm_reloc
   * :ref:`genindex`

.. Links:
.. _networking-filter: ../networking/filter.rst
.. _man-pages: https://www.kernel.org/doc/man-pages/
.. _bpf(2): https://man7.org/linux/man-pages/man2/bpf.2.html
.. _bpf-helpers(7): https://man7.org/linux/man-pages/man7/bpf-helpers.7.html
.. _BPF and XDP Reference Guide: https://docs.cilium.io/en/latest/bpf/
+467 −0

File added.

Preview size limit exceeded, changes collapsed.

Loading