Commit 8a41dfcd authored by Vincent Guittot's avatar Vincent Guittot Committed by Peter Zijlstra
Browse files

sched/fair: Don't set LBF_ALL_PINNED unnecessarily 



Setting LBF_ALL_PINNED during active load balance is only valid when there
is only 1 running task on the rq otherwise this ends up increasing the
balance interval whereas other tasks could migrate after the next interval
once they become cache-cold as an example.

LBF_ALL_PINNED flag is now always set it by default. It is then cleared
when we find one task that can be pulled when calling detach_tasks() or
during active migration.

Signed-off-by: default avatarVincent Guittot <vincent.guittot@linaro.org>
Signed-off-by: default avatarPeter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: default avatarValentin Schneider <valentin.schneider@arm.com>
Acked-by: default avatarMel Gorman <mgorman@suse.de>
Link: https://lkml.kernel.org/r/20210107103325.30851-3-vincent.guittot@linaro.org
parent fc488ffd

kernel/sched/fair.c

+5 −2
Original line number Original line Diff line number Diff line
@@ -9639,6 +9639,8 @@ static int load_balance(int this_cpu, struct rq *this_rq, 
	env.src_rq = busiest;

	env.src_rq = busiest;





	ld_moved = 0;

	ld_moved = 0;

	/* Clear this flag as soon as we find a pullable task */

	env.flags |= LBF_ALL_PINNED;

	if (busiest->nr_running > 1) {

	if (busiest->nr_running > 1) {

		/*

		/*

		 * Attempt to move tasks. If find_busiest_group has found

		 * Attempt to move tasks. If find_busiest_group has found
@@ -9646,7 +9648,6 @@ static int load_balance(int this_cpu, struct rq *this_rq, 
		 * still unbalanced. ld_moved simply stays zero, so it is

		 * still unbalanced. ld_moved simply stays zero, so it is

		 * correctly treated as an imbalance.

		 * correctly treated as an imbalance.

		 */

		 */

		env.flags |= LBF_ALL_PINNED;

		env.loop_max  = min(sysctl_sched_nr_migrate, busiest->nr_running);

		env.loop_max  = min(sysctl_sched_nr_migrate, busiest->nr_running);





more_balance:

more_balance:
@@ -9772,10 +9773,12 @@ static int load_balance(int this_cpu, struct rq *this_rq, 
			if (!cpumask_test_cpu(this_cpu, busiest->curr->cpus_ptr)) {

			if (!cpumask_test_cpu(this_cpu, busiest->curr->cpus_ptr)) {

				raw_spin_unlock_irqrestore(&busiest->lock,

				raw_spin_unlock_irqrestore(&busiest->lock,

							    flags);

							    flags);

				env.flags |= LBF_ALL_PINNED;

				goto out_one_pinned;

				goto out_one_pinned;

			}

			}





			/* Record that we found at least one task that could run on this_cpu */

			env.flags &= ~LBF_ALL_PINNED;



			/*

			/*

			 * ->active_balance synchronizes accesses to

			 * ->active_balance synchronizes accesses to

			 * ->active_balance_work.  Once set, it's cleared

			 * ->active_balance_work.  Once set, it's cleared