Commit 9ae606bc authored by Will Deacon's avatar Will Deacon Committed by Peter Zijlstra
Browse files

sched: Introduce task_cpu_possible_mask() to limit fallback rq selection 



Asymmetric systems may not offer the same level of userspace ISA support
across all CPUs, meaning that some applications cannot be executed by
some CPUs. As a concrete example, upcoming arm64 big.LITTLE designs do
not feature support for 32-bit applications on both clusters.

On such a system, we must take care not to migrate a task to an
unsupported CPU when forcefully moving tasks in select_fallback_rq()
in response to a CPU hot-unplug operation.

Introduce a task_cpu_possible_mask() hook which, given a task argument,
allows an architecture to return a cpumask of CPUs that are capable of
executing that task. The default implementation returns the
cpu_possible_mask, since sane machines do not suffer from per-cpu ISA
limitations that affect scheduling. The new mask is used when selecting
the fallback runqueue as a last resort before forcing a migration to the
first active CPU.

Signed-off-by: default avatarWill Deacon <will@kernel.org>
Signed-off-by: default avatarPeter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: default avatarValentin Schneider <Valentin.Schneider@arm.com>
Reviewed-by: default avatarQuentin Perret <qperret@google.com>
Link: https://lore.kernel.org/r/20210730112443.23245-2-will@kernel.org
parent 30400039

include/linux/mmu_context.h

+14 −0
Original line number Diff line number Diff line
@@ -14,4 +14,18 @@ 
static inline void leave_mm(int cpu) { }

#endif



/*

 * CPUs that are capable of running user task @p. Must contain at least one

 * active CPU. It is assumed that the kernel can run on all CPUs, so calling

 * this for a kernel thread is pointless.

 *

 * By default, we assume a sane, homogeneous system.

 */

#ifndef task_cpu_possible_mask

# define task_cpu_possible_mask(p)	cpu_possible_mask

# define task_cpu_possible(cpu, p)	true

#else

# define task_cpu_possible(cpu, p)	cpumask_test_cpu((cpu), task_cpu_possible_mask(p))

#endif



#endif

kernel/sched/core.c

+3 −6
Original line number Diff line number Diff line
@@ -2173,7 +2173,7 @@ static inline bool is_cpu_allowed(struct task_struct *p, int cpu) 


	/* Non kernel threads are not allowed during either online or offline. */

	if (!(p->flags & PF_KTHREAD))

		return cpu_active(cpu);

		return cpu_active(cpu) && task_cpu_possible(cpu, p);



	/* KTHREAD_IS_PER_CPU is always allowed. */

	if (kthread_is_per_cpu(p))
@@ -3124,9 +3124,7 @@ static int select_fallback_rq(int cpu, struct task_struct *p) 


		/* Look for allowed, online CPU in same node. */

		for_each_cpu(dest_cpu, nodemask) {

			if (!cpu_active(dest_cpu))

				continue;

			if (cpumask_test_cpu(dest_cpu, p->cpus_ptr))

			if (is_cpu_allowed(p, dest_cpu))

				return dest_cpu;

		}

	}
@@ -3156,10 +3154,9 @@ static int select_fallback_rq(int cpu, struct task_struct *p) 
			 *

			 * More yuck to audit.

			 */

			do_set_cpus_allowed(p, cpu_possible_mask);

			do_set_cpus_allowed(p, task_cpu_possible_mask(p));

			state = fail;

			break;



		case fail:

			BUG();

			break;