Commit fa4447cb authored by Hou Tao's avatar Hou Tao Committed by Alexei Starovoitov
Browse files

bpf: Use __llist_del_all() whenever possbile during memory draining 



Except for waiting_for_gp list, there are no concurrent operations on
free_by_rcu, free_llist and free_llist_extra lists, so use
__llist_del_all() instead of llist_del_all(). waiting_for_gp list can be
deleted by RCU callback concurrently, so still use llist_del_all().

Acked-by: default avatarStanislav Fomichev <sdf@google.com>
Signed-off-by: default avatarHou Tao <houtao1@huawei.com>
Link: https://lore.kernel.org/r/20221021114913.60508-3-houtao@huaweicloud.com


Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
parent 3d058187

kernel/bpf/memalloc.c

+5 −2
Original line number Diff line number Diff line
@@ -418,14 +418,17 @@ static void drain_mem_cache(struct bpf_mem_cache *c) 
	/* No progs are using this bpf_mem_cache, but htab_map_free() called

	 * bpf_mem_cache_free() for all remaining elements and they can be in

	 * free_by_rcu or in waiting_for_gp lists, so drain those lists now.

	 *

	 * Except for waiting_for_gp list, there are no concurrent operations

	 * on these lists, so it is safe to use __llist_del_all().

	 */

	llist_for_each_safe(llnode, t, __llist_del_all(&c->free_by_rcu))

		free_one(c, llnode);

	llist_for_each_safe(llnode, t, llist_del_all(&c->waiting_for_gp))

		free_one(c, llnode);

	llist_for_each_safe(llnode, t, llist_del_all(&c->free_llist))

	llist_for_each_safe(llnode, t, __llist_del_all(&c->free_llist))

		free_one(c, llnode);

	llist_for_each_safe(llnode, t, llist_del_all(&c->free_llist_extra))

	llist_for_each_safe(llnode, t, __llist_del_all(&c->free_llist_extra))

		free_one(c, llnode);

}