Commit ceea991a authored by Jiri Olsa's avatar Jiri Olsa Committed by Daniel Borkmann
Browse files

bpf: Move bpf_dispatcher function out of ftrace locations 

The dispatcher function is attached/detached to trampoline by
dispatcher update function. At the same time it's available as
ftrace attachable function.

After discussion [1] the proposed solution is to use compiler
attributes to alter bpf_dispatcher_##name##_func function:

  - remove it from being instrumented with __no_instrument_function__
    attribute, so ftrace has no track of it

  - but still generate 5 nop instructions with patchable_function_entry(5)
    attribute, which are expected by bpf_arch_text_poke used by
    dispatcher update function

Enabling HAVE_DYNAMIC_FTRACE_NO_PATCHABLE option for x86, so
__patchable_function_entries functions are not part of ftrace/mcount
locations.

Adding attributes to bpf_dispatcher_XXX function on x86_64 so it's
kept out of ftrace locations and has 5 byte nop generated at entry.

These attributes need to be arch specific as pointed out by Ilya
Leoshkevic in here [2].

The dispatcher image is generated only for x86_64 arch, so the
code can stay as is for other archs.

  [1] https://lore.kernel.org/bpf/20220722110811.124515-1-jolsa@kernel.org/
  [2] https://lore.kernel.org/bpf/969a14281a7791c334d476825863ee449964dd0c.camel@linux.ibm.com/



Suggested-by: default avatarPeter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: default avatarJiri Olsa <jolsa@kernel.org>
Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
Acked-by: default avatarPeter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lore.kernel.org/bpf/20220903131154.420467-3-jolsa@kernel.org
parent 9440155c

arch/x86/Kconfig

+1 −0
Original line number Diff line number Diff line
@@ -284,6 +284,7 @@ config X86 
	select PROC_PID_ARCH_STATUS		if PROC_FS

	select HAVE_ARCH_NODE_DEV_GROUP		if X86_SGX

	imply IMA_SECURE_AND_OR_TRUSTED_BOOT    if EFI

	select HAVE_DYNAMIC_FTRACE_NO_PATCHABLE



config INSTRUCTION_DECODER

	def_bool y

include/linux/bpf.h

+7 −0
Original line number Diff line number Diff line
@@ -977,7 +977,14 @@ int arch_prepare_bpf_dispatcher(void *image, s64 *funcs, int num_funcs); 
	},							\

}



#ifdef CONFIG_X86_64

#define BPF_DISPATCHER_ATTRIBUTES __attribute__((patchable_function_entry(5)))

#else

#define BPF_DISPATCHER_ATTRIBUTES

#endif



#define DEFINE_BPF_DISPATCHER(name)					\

	notrace BPF_DISPATCHER_ATTRIBUTES				\

	noinline __nocfi unsigned int bpf_dispatcher_##name##_func(	\

		const void *ctx,					\

		const struct bpf_insn *insnsi,				\