Commit d352bd88 authored by Alexei Starovoitov's avatar Alexei Starovoitov
Browse files

Merge branch 'bpf: Add 64bit enum value support' 



Yonghong Song says:

====================

Currently, btf only supports upto 32bit enum value with BTF_KIND_ENUM.
But in kernel, some enum has 64bit values, e.g., in uapi bpf.h, we have
  enum {
        BPF_F_INDEX_MASK                = 0xffffffffULL,
        BPF_F_CURRENT_CPU               = BPF_F_INDEX_MASK,
        BPF_F_CTXLEN_MASK               = (0xfffffULL << 32),
  };
With BTF_KIND_ENUM, the value for BPF_F_CTXLEN_MASK will be encoded
as 0 which is incorrect.

To solve this problem, BTF_KIND_ENUM64 is proposed in this patch set
to support enum 64bit values. Also, since sometimes there is a need
to generate C code from btf, e.g., vmlinux.h, btf kflag support
is also added for BTF_KIND_ENUM and BTF_KIND_ENUM64 to indicate
signedness, helping proper value printout.

Changelog:
  v4 -> v5:
    - skip newly-added enum64 C test if clang version <= 14.
  v3 -> v4:
    - rename btf_type_is_any_enum() to btf_is_any_enum() to favor
      consistency in libbpf.
    - fix sign extension issue in btf_dump_get_enum_value().
    - fix BPF_CORE_FIELD_SIGNED signedness issue in bpf_core_calc_field_relo().
  v2 -> v3:
    - Implement separate btf_equal_enum()/btf_equal_enum64() and
      btf_compat_enum()/btf_compat_enum64().
    - Add a new enum64 placeholder type dynamicly for enum64 sanitization.
    - For bpftool output and unit selftest, printed out signed/unsigned
      encoding as well.
    - fix some issues with BTF_KIND_ENUM is doc and clarified sign extension
      rules for enum values.
  v1 -> v2:
    - Changed kflag default from signed to unsigned
    - Fixed sanitization issue
    - Broke down libbpf related patches for easier review
    - Added more tests
    - More code refactorization
    - Corresponding llvm patch (to support enum64) is also updated
====================

Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
parents 02f4afeb 61dbd598

Documentation/bpf/btf.rst

+37 −6
Original line number Diff line number Diff line
@@ -74,7 +74,7 @@ sequentially and type id is assigned to each recognized type starting from id 
    #define BTF_KIND_ARRAY          3       /* Array        */

    #define BTF_KIND_STRUCT         4       /* Struct       */

    #define BTF_KIND_UNION          5       /* Union        */

    #define BTF_KIND_ENUM           6       /* Enumeration  */

    #define BTF_KIND_ENUM           6       /* Enumeration up to 32-bit values */

    #define BTF_KIND_FWD            7       /* Forward      */

    #define BTF_KIND_TYPEDEF        8       /* Typedef      */

    #define BTF_KIND_VOLATILE       9       /* Volatile     */
@@ -87,6 +87,7 @@ sequentially and type id is assigned to each recognized type starting from id 
    #define BTF_KIND_FLOAT          16      /* Floating point       */

    #define BTF_KIND_DECL_TAG       17      /* Decl Tag     */

    #define BTF_KIND_TYPE_TAG       18      /* Type Tag     */

    #define BTF_KIND_ENUM64         19      /* Enumeration up to 64-bit values */



Note that the type section encodes debug info, not just pure types.

``BTF_KIND_FUNC`` is not a type, and it represents a defined subprogram.
@@ -101,10 +102,10 @@ Each type contains the following common data:: 
         * bits 24-28: kind (e.g. int, ptr, array...etc)

         * bits 29-30: unused

         * bit     31: kind_flag, currently used by

         *             struct, union and fwd

         *             struct, union, fwd, enum and enum64.

         */

        __u32 info;

        /* "size" is used by INT, ENUM, STRUCT and UNION.

        /* "size" is used by INT, ENUM, STRUCT, UNION and ENUM64.

         * "size" tells the size of the type it is describing.

         *

         * "type" is used by PTR, TYPEDEF, VOLATILE, CONST, RESTRICT,
@@ -281,10 +282,10 @@ modes exist: 


``struct btf_type`` encoding requirement:

  * ``name_off``: 0 or offset to a valid C identifier

  * ``info.kind_flag``: 0

  * ``info.kind_flag``: 0 for unsigned, 1 for signed

  * ``info.kind``: BTF_KIND_ENUM

  * ``info.vlen``: number of enum values

  * ``size``: 4

  * ``size``: 1/2/4/8



``btf_type`` is followed by ``info.vlen`` number of ``struct btf_enum``.::
@@ -297,6 +298,10 @@ The ``btf_enum`` encoding: 
  * ``name_off``: offset to a valid C identifier

  * ``val``: any value



If the original enum value is signed and the size is less than 4,

that value will be sign extended into 4 bytes. If the size is 8,

the value will be truncated into 4 bytes.



2.2.7 BTF_KIND_FWD

~~~~~~~~~~~~~~~~~~
@@ -493,7 +498,7 @@ the attribute is applied to a ``struct``/``union`` member or 
a ``func`` argument, and ``btf_decl_tag.component_idx`` should be a

valid index (starting from 0) pointing to a member or an argument.



2.2.17 BTF_KIND_TYPE_TAG

2.2.18 BTF_KIND_TYPE_TAG

~~~~~~~~~~~~~~~~~~~~~~~~



``struct btf_type`` encoding requirement:
@@ -516,6 +521,32 @@ type_tag, then zero or more const/volatile/restrict/typedef 
and finally the base type. The base type is one of

int, ptr, array, struct, union, enum, func_proto and float types.



2.2.19 BTF_KIND_ENUM64

~~~~~~~~~~~~~~~~~~~~~~



``struct btf_type`` encoding requirement:

  * ``name_off``: 0 or offset to a valid C identifier

  * ``info.kind_flag``: 0 for unsigned, 1 for signed

  * ``info.kind``: BTF_KIND_ENUM64

  * ``info.vlen``: number of enum values

  * ``size``: 1/2/4/8



``btf_type`` is followed by ``info.vlen`` number of ``struct btf_enum64``.::



    struct btf_enum64 {

        __u32   name_off;

        __u32   val_lo32;

        __u32   val_hi32;

    };



The ``btf_enum64`` encoding:

  * ``name_off``: offset to a valid C identifier

  * ``val_lo32``: lower 32-bit value for a 64-bit value

  * ``val_hi32``: high 32-bit value for a 64-bit value



If the original enum value is signed and the size is less than 8,

that value will be sign extended into 8 bytes.



3. BTF Kernel API

=================

include/linux/btf.h

+28 −0
Original line number Diff line number Diff line
@@ -177,6 +177,19 @@ static inline bool btf_type_is_enum(const struct btf_type *t) 
	return BTF_INFO_KIND(t->info) == BTF_KIND_ENUM;

}



static inline bool btf_is_any_enum(const struct btf_type *t)

{

	return BTF_INFO_KIND(t->info) == BTF_KIND_ENUM ||

	       BTF_INFO_KIND(t->info) == BTF_KIND_ENUM64;

}



static inline bool btf_kind_core_compat(const struct btf_type *t1,

					const struct btf_type *t2)

{

	return BTF_INFO_KIND(t1->info) == BTF_INFO_KIND(t2->info) ||

	       (btf_is_any_enum(t1) && btf_is_any_enum(t2));

}



static inline bool str_is_empty(const char *s)

{

	return !s || !s[0];
@@ -192,6 +205,16 @@ static inline bool btf_is_enum(const struct btf_type *t) 
	return btf_kind(t) == BTF_KIND_ENUM;

}



static inline bool btf_is_enum64(const struct btf_type *t)

{

	return btf_kind(t) == BTF_KIND_ENUM64;

}



static inline u64 btf_enum64_value(const struct btf_enum64 *e)

{

	return ((u64)e->val_hi32 << 32) | e->val_lo32;

}



static inline bool btf_is_composite(const struct btf_type *t)

{

	u16 kind = btf_kind(t);
@@ -332,6 +355,11 @@ static inline struct btf_enum *btf_enum(const struct btf_type *t) 
	return (struct btf_enum *)(t + 1);

}



static inline struct btf_enum64 *btf_enum64(const struct btf_type *t)

{

	return (struct btf_enum64 *)(t + 1);

}



static inline const struct btf_var_secinfo *btf_type_var_secinfo(

		const struct btf_type *t)

{

include/uapi/linux/btf.h

+14 −3
Original line number Diff line number Diff line
@@ -36,10 +36,10 @@ struct btf_type { 
	 * bits 24-28: kind (e.g. int, ptr, array...etc)

	 * bits 29-30: unused

	 * bit     31: kind_flag, currently used by

	 *             struct, union and fwd

	 *             struct, union, enum, fwd and enum64

	 */

	__u32 info;

	/* "size" is used by INT, ENUM, STRUCT, UNION and DATASEC.

	/* "size" is used by INT, ENUM, STRUCT, UNION, DATASEC and ENUM64.

	 * "size" tells the size of the type it is describing.

	 *

	 * "type" is used by PTR, TYPEDEF, VOLATILE, CONST, RESTRICT,
@@ -63,7 +63,7 @@ enum { 
	BTF_KIND_ARRAY		= 3,	/* Array	*/

	BTF_KIND_STRUCT		= 4,	/* Struct	*/

	BTF_KIND_UNION		= 5,	/* Union	*/

	BTF_KIND_ENUM		= 6,	/* Enumeration	*/

	BTF_KIND_ENUM		= 6,	/* Enumeration up to 32-bit values */

	BTF_KIND_FWD		= 7,	/* Forward	*/

	BTF_KIND_TYPEDEF	= 8,	/* Typedef	*/

	BTF_KIND_VOLATILE	= 9,	/* Volatile	*/
@@ -76,6 +76,7 @@ enum { 
	BTF_KIND_FLOAT		= 16,	/* Floating point	*/

	BTF_KIND_DECL_TAG	= 17,	/* Decl Tag */

	BTF_KIND_TYPE_TAG	= 18,	/* Type Tag */

	BTF_KIND_ENUM64		= 19,	/* Enumeration up to 64-bit values */



	NR_BTF_KINDS,

	BTF_KIND_MAX		= NR_BTF_KINDS - 1,
@@ -186,4 +187,14 @@ struct btf_decl_tag { 
       __s32   component_idx;

};



/* BTF_KIND_ENUM64 is followed by multiple "struct btf_enum64".

 * The exact number of btf_enum64 is stored in the vlen (of the

 * info in "struct btf_type").

 */

struct btf_enum64 {

	__u32	name_off;

	__u32	val_lo32;

	__u32	val_hi32;

};



#endif /* _UAPI__LINUX_BTF_H__ */

kernel/bpf/btf.c

+128 −14
Original line number Diff line number Diff line
@@ -309,6 +309,7 @@ static const char * const btf_kind_str[NR_BTF_KINDS] = { 
	[BTF_KIND_FLOAT]	= "FLOAT",

	[BTF_KIND_DECL_TAG]	= "DECL_TAG",

	[BTF_KIND_TYPE_TAG]	= "TYPE_TAG",

	[BTF_KIND_ENUM64]	= "ENUM64",

};



const char *btf_type_str(const struct btf_type *t)
@@ -666,6 +667,7 @@ static bool btf_type_has_size(const struct btf_type *t) 
	case BTF_KIND_ENUM:

	case BTF_KIND_DATASEC:

	case BTF_KIND_FLOAT:

	case BTF_KIND_ENUM64:

		return true;

	}
@@ -711,6 +713,11 @@ static const struct btf_decl_tag *btf_type_decl_tag(const struct btf_type *t) 
	return (const struct btf_decl_tag *)(t + 1);

}



static const struct btf_enum64 *btf_type_enum64(const struct btf_type *t)

{

	return (const struct btf_enum64 *)(t + 1);

}



static const struct btf_kind_operations *btf_type_ops(const struct btf_type *t)

{

	return kind_ops[BTF_INFO_KIND(t->info)];
@@ -1019,6 +1026,7 @@ static const char *btf_show_name(struct btf_show *show) 
			parens = "{";

		break;

	case BTF_KIND_ENUM:

	case BTF_KIND_ENUM64:

		prefix = "enum";

		break;

	default:
@@ -1834,6 +1842,7 @@ __btf_resolve_size(const struct btf *btf, const struct btf_type *type, 
		case BTF_KIND_UNION:

		case BTF_KIND_ENUM:

		case BTF_KIND_FLOAT:

		case BTF_KIND_ENUM64:

			size = type->size;

			goto resolved;
@@ -3670,6 +3679,7 @@ static s32 btf_enum_check_meta(struct btf_verifier_env *env, 
{

	const struct btf_enum *enums = btf_type_enum(t);

	struct btf *btf = env->btf;

	const char *fmt_str;

	u16 i, nr_enums;

	u32 meta_needed;
@@ -3683,11 +3693,6 @@ static s32 btf_enum_check_meta(struct btf_verifier_env *env, 
		return -EINVAL;

	}



	if (btf_type_kflag(t)) {

		btf_verifier_log_type(env, t, "Invalid btf_info kind_flag");

		return -EINVAL;

	}



	if (t->size > 8 || !is_power_of_2(t->size)) {

		btf_verifier_log_type(env, t, "Unexpected size");

		return -EINVAL;
@@ -3718,7 +3723,8 @@ static s32 btf_enum_check_meta(struct btf_verifier_env *env, 


		if (env->log.level == BPF_LOG_KERNEL)

			continue;

		btf_verifier_log(env, "\t%s val=%d\n",

		fmt_str = btf_type_kflag(t) ? "\t%s val=%d\n" : "\t%s val=%u\n";

		btf_verifier_log(env, fmt_str,

				 __btf_name_by_offset(btf, enums[i].name_off),

				 enums[i].val);

	}
@@ -3759,7 +3765,10 @@ static void btf_enum_show(const struct btf *btf, const struct btf_type *t, 
		return;

	}



	if (btf_type_kflag(t))

		btf_show_type_value(show, "%d", v);

	else

		btf_show_type_value(show, "%u", v);

	btf_show_end_type(show);

}
@@ -3772,6 +3781,109 @@ static struct btf_kind_operations enum_ops = { 
	.show = btf_enum_show,

};



static s32 btf_enum64_check_meta(struct btf_verifier_env *env,

				 const struct btf_type *t,

				 u32 meta_left)

{

	const struct btf_enum64 *enums = btf_type_enum64(t);

	struct btf *btf = env->btf;

	const char *fmt_str;

	u16 i, nr_enums;

	u32 meta_needed;



	nr_enums = btf_type_vlen(t);

	meta_needed = nr_enums * sizeof(*enums);



	if (meta_left < meta_needed) {

		btf_verifier_log_basic(env, t,

				       "meta_left:%u meta_needed:%u",

				       meta_left, meta_needed);

		return -EINVAL;

	}



	if (t->size > 8 || !is_power_of_2(t->size)) {

		btf_verifier_log_type(env, t, "Unexpected size");

		return -EINVAL;

	}



	/* enum type either no name or a valid one */

	if (t->name_off &&

	    !btf_name_valid_identifier(env->btf, t->name_off)) {

		btf_verifier_log_type(env, t, "Invalid name");

		return -EINVAL;

	}



	btf_verifier_log_type(env, t, NULL);



	for (i = 0; i < nr_enums; i++) {

		if (!btf_name_offset_valid(btf, enums[i].name_off)) {

			btf_verifier_log(env, "\tInvalid name_offset:%u",

					 enums[i].name_off);

			return -EINVAL;

		}



		/* enum member must have a valid name */

		if (!enums[i].name_off ||

		    !btf_name_valid_identifier(btf, enums[i].name_off)) {

			btf_verifier_log_type(env, t, "Invalid name");

			return -EINVAL;

		}



		if (env->log.level == BPF_LOG_KERNEL)

			continue;



		fmt_str = btf_type_kflag(t) ? "\t%s val=%lld\n" : "\t%s val=%llu\n";

		btf_verifier_log(env, fmt_str,

				 __btf_name_by_offset(btf, enums[i].name_off),

				 btf_enum64_value(enums + i));

	}



	return meta_needed;

}



static void btf_enum64_show(const struct btf *btf, const struct btf_type *t,

			    u32 type_id, void *data, u8 bits_offset,

			    struct btf_show *show)

{

	const struct btf_enum64 *enums = btf_type_enum64(t);

	u32 i, nr_enums = btf_type_vlen(t);

	void *safe_data;

	s64 v;



	safe_data = btf_show_start_type(show, t, type_id, data);

	if (!safe_data)

		return;



	v = *(u64 *)safe_data;



	for (i = 0; i < nr_enums; i++) {

		if (v != btf_enum64_value(enums + i))

			continue;



		btf_show_type_value(show, "%s",

				    __btf_name_by_offset(btf,

							 enums[i].name_off));



		btf_show_end_type(show);

		return;

	}



	if (btf_type_kflag(t))

		btf_show_type_value(show, "%lld", v);

	else

		btf_show_type_value(show, "%llu", v);

	btf_show_end_type(show);

}



static struct btf_kind_operations enum64_ops = {

	.check_meta = btf_enum64_check_meta,

	.resolve = btf_df_resolve,

	.check_member = btf_enum_check_member,

	.check_kflag_member = btf_enum_check_kflag_member,

	.log_details = btf_enum_log,

	.show = btf_enum64_show,

};



static s32 btf_func_proto_check_meta(struct btf_verifier_env *env,

				     const struct btf_type *t,

				     u32 meta_left)
@@ -4438,6 +4550,7 @@ static const struct btf_kind_operations * const kind_ops[NR_BTF_KINDS] = { 
	[BTF_KIND_FLOAT] = &float_ops,

	[BTF_KIND_DECL_TAG] = &decl_tag_ops,

	[BTF_KIND_TYPE_TAG] = &modifier_ops,

	[BTF_KIND_ENUM64] = &enum64_ops,

};



static s32 btf_check_meta(struct btf_verifier_env *env,
@@ -5299,7 +5412,7 @@ bool btf_ctx_access(int off, int size, enum bpf_access_type type, 
	/* skip modifiers */

	while (btf_type_is_modifier(t))

		t = btf_type_by_id(btf, t->type);

	if (btf_type_is_small_int(t) || btf_type_is_enum(t))

	if (btf_type_is_small_int(t) || btf_is_any_enum(t))

		/* accessing a scalar */

		return true;

	if (!btf_type_is_ptr(t)) {
@@ -5763,7 +5876,7 @@ static int __get_type_size(struct btf *btf, u32 btf_id, 
	if (btf_type_is_ptr(t))

		/* kernel size of pointer. Not BPF's size of pointer*/

		return sizeof(void *);

	if (btf_type_is_int(t) || btf_type_is_enum(t))

	if (btf_type_is_int(t) || btf_is_any_enum(t))

		return t->size;

	*bad_type = t;

	return -EINVAL;
@@ -5911,7 +6024,7 @@ static int btf_check_func_type_match(struct bpf_verifier_log *log, 
		 * to context only. And only global functions can be replaced.

		 * Hence type check only those types.

		 */

		if (btf_type_is_int(t1) || btf_type_is_enum(t1))

		if (btf_type_is_int(t1) || btf_is_any_enum(t1))

			continue;

		if (!btf_type_is_ptr(t1)) {

			bpf_log(log,
@@ -6408,7 +6521,7 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog, 
	t = btf_type_by_id(btf, t->type);

	while (btf_type_is_modifier(t))

		t = btf_type_by_id(btf, t->type);

	if (!btf_type_is_int(t) && !btf_type_is_enum(t)) {

	if (!btf_type_is_int(t) && !btf_is_any_enum(t)) {

		bpf_log(log,

			"Global function %s() doesn't return scalar. Only those are supported.\n",

			tname);
@@ -6423,7 +6536,7 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog, 
		t = btf_type_by_id(btf, args[i].type);

		while (btf_type_is_modifier(t))

			t = btf_type_by_id(btf, t->type);

		if (btf_type_is_int(t) || btf_type_is_enum(t)) {

		if (btf_type_is_int(t) || btf_is_any_enum(t)) {

			reg->type = SCALAR_VALUE;

			continue;

		}
@@ -7335,6 +7448,7 @@ int __bpf_core_types_are_compat(const struct btf *local_btf, __u32 local_id, 
	case BTF_KIND_UNION:

	case BTF_KIND_ENUM:

	case BTF_KIND_FWD:

	case BTF_KIND_ENUM64:

		return 1;

	case BTF_KIND_INT:

		/* just reject deprecated bitfield-like integers; all other
@@ -7387,10 +7501,10 @@ int __bpf_core_types_are_compat(const struct btf *local_btf, __u32 local_id, 
 * field-based relocations. This function assumes that root types were already

 * checked for name match. Beyond that initial root-level name check, names

 * are completely ignored. Compatibility rules are as follows:

 *   - any two STRUCTs/UNIONs/FWDs/ENUMs/INTs are considered compatible, but

 *   - any two STRUCTs/UNIONs/FWDs/ENUMs/INTs/ENUM64s are considered compatible, but

 *     kind should match for local and target types (i.e., STRUCT is not

 *     compatible with UNION);

 *   - for ENUMs, the size is ignored;

 *   - for ENUMs/ENUM64s, the size is ignored;

 *   - for INT, size and signedness are ignored;

 *   - for ARRAY, dimensionality is ignored, element types are checked for

 *     compatibility recursively;

kernel/bpf/verifier.c

+1 −1
Original line number Diff line number Diff line
@@ -10901,7 +10901,7 @@ static int check_btf_func(struct bpf_verifier_env *env, 
			goto err_free;

		ret_type = btf_type_skip_modifiers(btf, func_proto->type, NULL);

		scalar_return =

			btf_type_is_small_int(ret_type) || btf_type_is_enum(ret_type);

			btf_type_is_small_int(ret_type) || btf_is_any_enum(ret_type);

		if (i && !scalar_return && env->subprog_info[i].has_ld_abs) {

			verbose(env, "LD_ABS is only allowed in functions that return 'int'.\n");

			goto err_free;