Commit b77fb25d authored by Kenny Yu's avatar Kenny Yu Committed by Alexei Starovoitov
Browse files

bpf: Add support for bpf iterator programs to use sleepable helpers 



This patch allows bpf iterator programs to use sleepable helpers by
changing `bpf_iter_run_prog` to use the appropriate synchronization.
With sleepable bpf iterator programs, we can no longer use
`rcu_read_lock()` and must use `rcu_read_lock_trace()` instead
to protect the bpf program.

Signed-off-by: default avatarKenny Yu <kennyyu@fb.com>
Acked-by: default avatarAndrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/r/20220124185403.468466-2-kennyyu@fb.com


Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
parent caaba961

kernel/bpf/bpf_iter.c

+15 −5
Original line number Diff line number Diff line
@@ -5,6 +5,7 @@ 
#include <linux/anon_inodes.h>

#include <linux/filter.h>

#include <linux/bpf.h>

#include <linux/rcupdate_trace.h>



struct bpf_iter_target_info {

	struct list_head list;
@@ -684,11 +685,20 @@ int bpf_iter_run_prog(struct bpf_prog *prog, void *ctx) 
{

	int ret;



	if (prog->aux->sleepable) {

		rcu_read_lock_trace();

		migrate_disable();

		might_fault();

		ret = bpf_prog_run(prog, ctx);

		migrate_enable();

		rcu_read_unlock_trace();

	} else {

		rcu_read_lock();

		migrate_disable();

		ret = bpf_prog_run(prog, ctx);

		migrate_enable();

		rcu_read_unlock();

	}



	/* bpf program can only return 0 or 1:

	 *  0 : okay