Commit 3bbc8ee7 authored by Alexei Starovoitov's avatar Alexei Starovoitov
Browse files

Merge branch 'Improve XDP samples usability and output' 

Kumar Kartikeya says:

====================

This set revamps XDP samples related to redirection to show better output and
implement missing features consolidating all their differences and giving them a
consistent look and feel, by implementing common features and command line
options.  Some of the TODO items like reporting redirect error numbers
(ENETDOWN, EINVAL, ENOSPC, etc.) have also been implemented.

Some of the features are:
* Received packet statistics
* xdp_redirect/xdp_redirect_map tracepoint statistics
* xdp_redirect_err/xdp_redirect_map_err tracepoint statistics (with support for
  showing exact errno)
* xdp_cpumap_enqueue/xdp_cpumap_kthread tracepoint statistics
* xdp_devmap_xmit tracepoint statistics
* xdp_exception tracepoint statistics
* Per ifindex pair devmap_xmit stats shown dynamically (for xdp_monitor) to
  decompose the total.
* Use of BPF skeleton and BPF static linking to share BPF programs.
* Use of vmlinux.h and tp_btf for raw_tracepoint support.
* Removal of redundant -N/--native-mode option (enforced by default now)
* ... and massive cleanups all over the place.

All tracepoints also use raw_tp now, and tracepoints like xdp_redirect
are only enabled when requested explicitly to capture successful redirection
statistics.

The set of programs converted as part of this series are:
 * xdp_redirect_cpu
 * xdp_redirect_map_multi
 * xdp_redirect_map
 * xdp_redirect
 * xdp_monitor

 Explanation of the output:

There is now a concise output mode by default that shows primarily four fields:
  rx/s        Number of packets received per second
  redir/s     Number of packets successfully redirected per second
  err,drop/s  Aggregated count of errors per second (including dropped packets)
  xmit/s      Number of packets transmitted on the output device per second

Some examples:
 ; sudo ./xdp_redirect_map veth0 veth1 -s
Redirecting from veth0 (ifindex 15; driver veth) to veth1 (ifindex 14; driver veth)
veth0->veth1                    0 rx/s                  0 redir/s		0 err,drop/s               0 xmit/s
veth0->veth1            9,998,660 rx/s          9,998,658 redir/s		0 err,drop/s       9,998,654 xmit/s
...

There is also a verbose mode, that can also be enabled by default using -v (--verbose).
The output mode can be switched dynamically at runtime using Ctrl + \ (SIGQUIT).

To make the concise output more useful, the errors that occur are expanded inline
(as if verbose mode was enabled) to let the user pin down the source of the
problem without having to clutter output (or possibly miss it) or always use verbose mode.

For instance, let's consider a case where the output device link state is set to
down while redirection is happening:

[...]
veth0->veth1           24,503,376 rx/s                  0 err,drop/s      24,503,372 xmit/s
veth0->veth1           25,044,775 rx/s                  0 err,drop/s      25,044,783 xmit/s
veth0->veth1           25,263,046 rx/s                  4 err,drop/s      25,263,028 xmit/s
  redirect_err                  4 error/s
    ENETDOWN                    4 error/s
[...]

The same holds for xdp_exception actions.

An example of how a complete xdp_redirect_map session would look:

 ; sudo ./xdp_redirect_map veth0 veth1
Redirecting from veth0 (ifindex 5; driver veth) to veth1 (ifindex 4; driver veth)
veth0->veth1            7,411,506 rx/s                  0 err,drop/s    7,411,470 xmit/s
veth0->veth1            8,931,770 rx/s                  0 err,drop/s    8,931,771 xmit/s
^\
veth0->veth1            8,787,295 rx/s                  0 err,drop/s    8,787,325 xmit/s
  receive total         8,787,295 pkt/s                 0 drop/s                0 error/s
    cpu:7               8,787,295 pkt/s                 0 drop/s                0 error/s
  redirect_err                  0 error/s
  xdp_exception                 0 hit/s
  xmit veth0->veth1     8,787,325 xmit/s                0 drop/s                0 drv_err/s          2.00 bulk-avg
     cpu:7              8,787,325 xmit/s                0 drop/s                0 drv_err/s          2.00 bulk-avg

veth0->veth1            8,842,610 rx/s                  0 err,drop/s    8,842,606 xmit/s
  receive total         8,842,610 pkt/s                 0 drop/s                0 error/s
    cpu:7               8,842,610 pkt/s                 0 drop/s                0 error/s
  redirect_err                  0 error/s
  xdp_exception                 0 hit/s
  xmit veth0->veth1     8,842,606 xmit/s                0 drop/s                0 drv_err/s          2.00 bulk-avg
     cpu:7              8,842,606 xmit/s                0 drop/s                0 drv_err/s          2.00 bulk-avg

^C
  Packets received    : 33,973,181
  Average packets/s   : 4,246,648
  Packets transmitted : 33,973,172
  Average transmit/s  : 4,246,647

The xdp_redirect tracepoint (for success stats) needs to be enabled explicitly
using --stats/-s. Documentation for entire output and options is provided when
user specifies --help/-h with a sample.

Changelog:
----------
v3 -> v4:
v3: https://lore.kernel.org/bpf/20210728165552.435050-1-memxor@gmail.com

 * Address all feedback from Daniel
  * Use READ_ONCE/WRITE_ONCE from linux/compiler.h (cannot directly include
    due to conflicts with vmlinux.h)
  * Fix MAX_CPUS hardcoding by switching to mmapable array maps, that are
    resized based on the value of libbpf_num_possible_cpus
  * s/ELEMENTS_OF/ARRAY_SIZE/g
  * Use tools/include/linux/hashtable.h
  * Coding style fixes
  * Remove hyperlinks for tracepoints
  * Split into smaller reviewable changes
 * Restore support for specifying custom xdp_redirect_cpu cpumap prog with some
   enhancements, including built-in programs for common actions (pass, drop,
   redirect). By default, cpumap prog is now disabled.
 * Misc bug fixes all over the place

  The printing stuff is a lot more basic without hyperlink support, hence it
  has not been exported into a more general facility.

v2 -> v3
v2: https://lore.kernel.org/bpf/20210721212833.701342-1-memxor@gmail.com

 * Address all feedback from Andrii
  * Replace usage of libbpf hashmap (internal API) with custom one
  * Rename ATOMIC_* macros to NO_TEAR_* to better reflect their use
  * Use size_t as a portable word sized data type
  * Set libbpf_set_strict_mode
  * Invert conditions in BPF programs to exit early and reduce nesting
  * Use canonical SEC("xdp") naming for all XDP BPF progams
 * Add missing help description for cpumap enqueue and kthread tracepoints
 * Move private struct declarations from xdp_sample_user.h to .c file
 * Improve help output for cpumap enqueue and cpumap kthread tracepoints
 * Fix a bug where keys array for BPF_MAP_LOOKUP_BATCH is overallocated
 * Fix some conditions for printing stats (earlier only checked pps, now pps,
   drop, err and print if any is greater than zero)
 * Fix alloc_stats_record to properly return and cleanup allocated memory on
   allocation failure instead of calling exit(3)
 * Bump bpf_map_lookup_batch count to 32 to reduce lookup time with multiple
   devices in map
 * Fix a bug where devmap_xmit_multi stats are not printed when previous record
   is missing (i.e. when the first time stats are printed), by simply using a
   dummy record that is zeroed out
 * Also print per-CPU counts for devmap_xmit_multi which we collect already
 * Change mac_map to be BPF_MAP_TYPE_HASH instead of array to prevent resizing
   to a large size when max_ifindex is high, in xdp_redirect_map_multi
 * Fix instance of strerror(errno) in sample_install_xdp to use saved errno
 * Provide a usage function from samples helper
 * Provide a fix where incorrect stats are shown for parallel sessions of
   xdp_redirect_* samples by introducing matching support for input device(s),
   output device(s) and cpumap map id for enqueue and kthread stats.
   Only xdp_monitor doesn't filter stats, all others do.

RFC (v1) -> v2
RFC (v1): https://lore.kernel.org/bpf/20210528235250.2635167-1-memxor@gmail.com



 * Address all feedback from Andrii
   * Use BPF static linking
   * Use vmlinux.h
   * Use BPF_PROG macro
   * Use global variables instead of maps
 * Use of tp_btf for raw_tracepoint progs
 * Switch to timerfd for polling
 * Use libbpf hashmap for maintaing device sets for per ifindex pair
   devmap_xmit stats
 * Fix Makefile to specify object dependencies properly
 * Use in-tree bpftool
 * ... misc fixes and cleanups all over the place
====================

Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
parents d7af7e49 594a116b

samples/bpf/Makefile

+94 −15
Original line number Diff line number Diff line
@@ -39,11 +39,6 @@ tprogs-y += lwt_len_hist 
tprogs-y += xdp_tx_iptunnel

tprogs-y += test_map_in_map

tprogs-y += per_socket_stats_example

tprogs-y += xdp_redirect

tprogs-y += xdp_redirect_map

tprogs-y += xdp_redirect_map_multi

tprogs-y += xdp_redirect_cpu

tprogs-y += xdp_monitor

tprogs-y += xdp_rxq_info

tprogs-y += syscall_tp

tprogs-y += cpustat
@@ -57,11 +52,18 @@ tprogs-y += xdp_sample_pkts 
tprogs-y += ibumad

tprogs-y += hbm



tprogs-y += xdp_redirect_cpu

tprogs-y += xdp_redirect_map_multi

tprogs-y += xdp_redirect_map

tprogs-y += xdp_redirect

tprogs-y += xdp_monitor



# Libbpf dependencies

LIBBPF = $(TOOLS_PATH)/lib/bpf/libbpf.a



CGROUP_HELPERS := ../../tools/testing/selftests/bpf/cgroup_helpers.o

TRACE_HELPERS := ../../tools/testing/selftests/bpf/trace_helpers.o

XDP_SAMPLE := xdp_sample_user.o



fds_example-objs := fds_example.o

sockex1-objs := sockex1_user.o
@@ -98,11 +100,6 @@ lwt_len_hist-objs := lwt_len_hist_user.o 
xdp_tx_iptunnel-objs := xdp_tx_iptunnel_user.o

test_map_in_map-objs := test_map_in_map_user.o

per_socket_stats_example-objs := cookie_uid_helper_example.o

xdp_redirect-objs := xdp_redirect_user.o

xdp_redirect_map-objs := xdp_redirect_map_user.o

xdp_redirect_map_multi-objs := xdp_redirect_map_multi_user.o

xdp_redirect_cpu-objs := xdp_redirect_cpu_user.o

xdp_monitor-objs := xdp_monitor_user.o

xdp_rxq_info-objs := xdp_rxq_info_user.o

syscall_tp-objs := syscall_tp_user.o

cpustat-objs := cpustat_user.o
@@ -116,6 +113,12 @@ xdp_sample_pkts-objs := xdp_sample_pkts_user.o 
ibumad-objs := ibumad_user.o

hbm-objs := hbm.o $(CGROUP_HELPERS)



xdp_redirect_map_multi-objs := xdp_redirect_map_multi_user.o $(XDP_SAMPLE)

xdp_redirect_cpu-objs := xdp_redirect_cpu_user.o $(XDP_SAMPLE)

xdp_redirect_map-objs := xdp_redirect_map_user.o $(XDP_SAMPLE)

xdp_redirect-objs := xdp_redirect_user.o $(XDP_SAMPLE)

xdp_monitor-objs := xdp_monitor_user.o $(XDP_SAMPLE)



# Tell kbuild to always build the programs

always-y := $(tprogs-y)

always-y += sockex1_kern.o
@@ -160,11 +163,6 @@ always-y += tcp_clamp_kern.o 
always-y += tcp_basertt_kern.o

always-y += tcp_tos_reflect_kern.o

always-y += tcp_dumpstats_kern.o

always-y += xdp_redirect_kern.o

always-y += xdp_redirect_map_kern.o

always-y += xdp_redirect_map_multi_kern.o

always-y += xdp_redirect_cpu_kern.o

always-y += xdp_monitor_kern.o

always-y += xdp_rxq_info_kern.o

always-y += xdp2skb_meta_kern.o

always-y += syscall_tp_kern.o
@@ -276,6 +274,11 @@ $(LIBBPF): FORCE 
	$(MAKE) -C $(dir $@) RM='rm -rf' EXTRA_CFLAGS="$(TPROGS_CFLAGS)" \

		LDFLAGS=$(TPROGS_LDFLAGS) srctree=$(BPF_SAMPLES_PATH)/../../ O=



BPFTOOLDIR := $(TOOLS_PATH)/bpf/bpftool

BPFTOOL := $(BPFTOOLDIR)/bpftool

$(BPFTOOL): $(wildcard $(BPFTOOLDIR)/*.[ch] $(BPFTOOLDIR)/Makefile)

	    $(MAKE) -C $(BPFTOOLDIR) srctree=$(BPF_SAMPLES_PATH)/../../



$(obj)/syscall_nrs.h:	$(obj)/syscall_nrs.s FORCE

	$(call filechk,offsets,__SYSCALL_NRS_H__)
@@ -306,6 +309,12 @@ verify_target_bpf: verify_cmds 
$(BPF_SAMPLES_PATH)/*.c: verify_target_bpf $(LIBBPF)

$(src)/*.c: verify_target_bpf $(LIBBPF)



$(obj)/xdp_redirect_cpu_user.o: $(obj)/xdp_redirect_cpu.skel.h

$(obj)/xdp_redirect_map_multi_user.o: $(obj)/xdp_redirect_map_multi.skel.h

$(obj)/xdp_redirect_map_user.o: $(obj)/xdp_redirect_map.skel.h

$(obj)/xdp_redirect_user.o: $(obj)/xdp_redirect.skel.h

$(obj)/xdp_monitor_user.o: $(obj)/xdp_monitor.skel.h



$(obj)/tracex5_kern.o: $(obj)/syscall_nrs.h

$(obj)/hbm_out_kern.o: $(src)/hbm.h $(src)/hbm_kern.h

$(obj)/hbm.o: $(src)/hbm.h
@@ -313,6 +322,76 @@ $(obj)/hbm_edt_kern.o: $(src)/hbm.h $(src)/hbm_kern.h 


-include $(BPF_SAMPLES_PATH)/Makefile.target



VMLINUX_BTF_PATHS ?= $(if $(O),$(O)/vmlinux)				\

		     $(if $(KBUILD_OUTPUT),$(KBUILD_OUTPUT)/vmlinux)	\

		     ../../../../vmlinux				\

		     /sys/kernel/btf/vmlinux				\

		     /boot/vmlinux-$(shell uname -r)

VMLINUX_BTF ?= $(abspath $(firstword $(wildcard $(VMLINUX_BTF_PATHS))))



ifeq ($(VMLINUX_BTF),)

$(error Cannot find a vmlinux for VMLINUX_BTF at any of "$(VMLINUX_BTF_PATHS)")

endif



$(obj)/vmlinux.h: $(VMLINUX_BTF) $(BPFTOOL)

ifeq ($(VMLINUX_H),)

	$(Q)$(BPFTOOL) btf dump file $(VMLINUX_BTF) format c > $@

else

	$(Q)cp "$(VMLINUX_H)" $@

endif



clean-files += vmlinux.h



# Get Clang's default includes on this system, as opposed to those seen by

# '-target bpf'. This fixes "missing" files on some architectures/distros,

# such as asm/byteorder.h, asm/socket.h, asm/sockios.h, sys/cdefs.h etc.

#

# Use '-idirafter': Don't interfere with include mechanics except where the

# build would have failed anyways.

define get_sys_includes

$(shell $(1) -v -E - </dev/null 2>&1 \

        | sed -n '/<...> search starts here:/,/End of search list./{ s| \(/.*\)|-idirafter \1|p }') \

$(shell $(1) -dM -E - </dev/null | grep '#define __riscv_xlen ' | sed 's/#define /-D/' | sed 's/ /=/')

endef



CLANG_SYS_INCLUDES = $(call get_sys_includes,$(CLANG))



$(obj)/xdp_redirect_cpu.bpf.o: $(obj)/xdp_sample.bpf.o

$(obj)/xdp_redirect_map_multi.bpf.o: $(obj)/xdp_sample.bpf.o

$(obj)/xdp_redirect_map.bpf.o: $(obj)/xdp_sample.bpf.o

$(obj)/xdp_redirect.bpf.o: $(obj)/xdp_sample.bpf.o

$(obj)/xdp_monitor.bpf.o: $(obj)/xdp_sample.bpf.o



$(obj)/%.bpf.o: $(src)/%.bpf.c $(obj)/vmlinux.h $(src)/xdp_sample.bpf.h $(src)/xdp_sample_shared.h

	@echo "  CLANG-BPF " $@

	$(Q)$(CLANG) -g -O2 -target bpf -D__TARGET_ARCH_$(SRCARCH) \

		-Wno-compare-distinct-pointer-types -I$(srctree)/include \

		-I$(srctree)/samples/bpf -I$(srctree)/tools/include \

		-I$(srctree)/tools/lib $(CLANG_SYS_INCLUDES) \

		-c $(filter %.bpf.c,$^) -o $@



LINKED_SKELS := xdp_redirect_cpu.skel.h xdp_redirect_map_multi.skel.h \

		xdp_redirect_map.skel.h xdp_redirect.skel.h xdp_monitor.skel.h

clean-files += $(LINKED_SKELS)



xdp_redirect_cpu.skel.h-deps := xdp_redirect_cpu.bpf.o xdp_sample.bpf.o

xdp_redirect_map_multi.skel.h-deps := xdp_redirect_map_multi.bpf.o xdp_sample.bpf.o

xdp_redirect_map.skel.h-deps := xdp_redirect_map.bpf.o xdp_sample.bpf.o

xdp_redirect.skel.h-deps := xdp_redirect.bpf.o xdp_sample.bpf.o

xdp_monitor.skel.h-deps := xdp_monitor.bpf.o xdp_sample.bpf.o



LINKED_BPF_SRCS := $(patsubst %.bpf.o,%.bpf.c,$(foreach skel,$(LINKED_SKELS),$($(skel)-deps)))



BPF_SRCS_LINKED := $(notdir $(wildcard $(src)/*.bpf.c))

BPF_OBJS_LINKED := $(patsubst %.bpf.c,$(obj)/%.bpf.o, $(BPF_SRCS_LINKED))

BPF_SKELS_LINKED := $(addprefix $(obj)/,$(LINKED_SKELS))



$(BPF_SKELS_LINKED): $(BPF_OBJS_LINKED) $(BPFTOOL)

	@echo "  BPF GEN-OBJ " $(@:.skel.h=)

	$(Q)$(BPFTOOL) gen object $(@:.skel.h=.lbpf.o) $(addprefix $(obj)/,$($(@F)-deps))

	@echo "  BPF GEN-SKEL" $(@:.skel.h=)

	$(Q)$(BPFTOOL) gen skeleton $(@:.skel.h=.lbpf.o) name $(notdir $(@:.skel.h=)) > $@



# asm/sysreg.h - inline assembly used by it is incompatible with llvm.

# But, there is no easy way to fix it, so just exclude it since it is

# useless for BPF samples.

samples/bpf/Makefile.target

+11 −0
Original line number Diff line number Diff line
@@ -73,3 +73,14 @@ quiet_cmd_tprog-cobjs = CC $@ 
      cmd_tprog-cobjs	= $(CC) $(tprogc_flags) -c -o $@ $<

$(tprog-cobjs): $(obj)/%.o: $(src)/%.c FORCE

	$(call if_changed_dep,tprog-cobjs)



# Override includes for xdp_sample_user.o because $(srctree)/usr/include in

# TPROGS_CFLAGS causes conflicts

XDP_SAMPLE_CFLAGS += -Wall -O2 -lm \

		     -I./tools/include \

		     -I./tools/include/uapi \

		     -I./tools/lib \

		     -I./tools/testing/selftests/bpf

$(obj)/xdp_sample_user.o: $(src)/xdp_sample_user.c \

	$(src)/xdp_sample_user.h $(src)/xdp_sample_shared.h

	$(CC) $(XDP_SAMPLE_CFLAGS) -c -o $@ $<

samples/bpf/cookie_uid_helper_example.c

+8 −3
Original line number Diff line number Diff line
@@ -167,7 +167,7 @@ static void prog_load(void) 
static void prog_attach_iptables(char *file)

{

	int ret;

	char rules[100];

	char rules[256];



	if (bpf_obj_pin(prog_fd, file))

		error(1, errno, "bpf_obj_pin");
@@ -175,8 +175,13 @@ static void prog_attach_iptables(char *file) 
		printf("file path too long: %s\n", file);

		exit(1);

	}

	sprintf(rules, "iptables -A OUTPUT -m bpf --object-pinned %s -j ACCEPT",

	ret = snprintf(rules, sizeof(rules),

		       "iptables -A OUTPUT -m bpf --object-pinned %s -j ACCEPT",

		       file);

	if (ret < 0 || ret >= sizeof(rules)) {

		printf("error constructing iptables command\n");

		exit(1);

	}

	ret = system(rules);

	if (ret < 0) {

		printf("iptables rule update failed: %d/n", WEXITSTATUS(ret));

samples/bpf/tracex4_user.c

+1 −1
Original line number Diff line number Diff line
@@ -32,7 +32,7 @@ static void print_old_objects(int fd) 
	__u64 key, next_key;

	struct pair v;



	key = write(1, "\e[1;1H\e[2J", 12); /* clear screen */

	key = write(1, "\e[1;1H\e[2J", 11); /* clear screen */



	key = -1;

	while (bpf_map_get_next_key(fd, &key, &next_key) == 0) {

samples/bpf/xdp_monitor.bpf.c

0 → 100644
+8 −0
Original line number Diff line number Diff line 
// SPDX-License-Identifier: GPL-2.0

/*  Copyright(c) 2017-2018 Jesper Dangaard Brouer, Red Hat Inc.

 *

 * XDP monitor tool, based on tracepoints

 */

#include "xdp_sample.bpf.h"



char _license[] SEC("license") = "GPL";