Commit 35b6d770 authored by Mel Gorman's avatar Mel Gorman Committed by Linus Torvalds
Browse files

mm/page_alloc: track range of active PCP lists during bulk free 

free_pcppages_bulk() frees pages in a round-robin fashion.  Originally,
this was dealing only with migratetypes but storing high-order pages
means that there can be many more empty lists that are uselessly
checked.  Track the minimum and maximum active pindex to reduce the
search space.

Link: https://lkml.kernel.org/r/20220217002227.5739-3-mgorman@techsingularity.net


Signed-off-by: default avatarMel Gorman <mgorman@techsingularity.net>
Reviewed-by: default avatarVlastimil Babka <vbabka@suse.cz>
Tested-by: default avatarAaron Lu <aaron.lu@intel.com>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: Jesper Dangaard Brouer <brouer@redhat.com>
Cc: Michal Hocko <mhocko@kernel.org>
Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
parent ca7b59b1

mm/page_alloc.c

+13 −4
Original line number Diff line number Diff line
@@ -1447,6 +1447,8 @@ static void free_pcppages_bulk(struct zone *zone, int count, 
					struct per_cpu_pages *pcp)

{

	int pindex = 0;

	int min_pindex = 0;

	int max_pindex = NR_PCP_LISTS - 1;

	int batch_free = 0;

	int nr_freed = 0;

	unsigned int order;
@@ -1472,13 +1474,20 @@ static void free_pcppages_bulk(struct zone *zone, int count, 
		 */

		do {

			batch_free++;

			if (++pindex == NR_PCP_LISTS)

				pindex = 0;

			if (++pindex > max_pindex)

				pindex = min_pindex;

			list = &pcp->lists[pindex];

		} while (list_empty(list));

			if (!list_empty(list))

				break;



			if (pindex == max_pindex)

				max_pindex--;

			if (pindex == min_pindex)

				min_pindex++;

		} while (1);



		/* This is the only non-empty list. Free them all. */

		if (batch_free == NR_PCP_LISTS)

		if (batch_free >= max_pindex - min_pindex)

			batch_free = count;



		order = pindex_to_order(pindex);