Commit 6343402a authored by Pingfan Liu's avatar Pingfan Liu Committed by Paul E. McKenney
Browse files

rcu: Synchronize ->qsmaskinitnext in rcu_boost_kthread_setaffinity() 



Once either rcutree_online_cpu() or rcutree_dead_cpu() is invoked
concurrently, the following rcu_boost_kthread_setaffinity() race can
occur:

        CPU 1                               CPU2
mask = rcu_rnp_online_cpus(rnp);
...

                                   mask = rcu_rnp_online_cpus(rnp);
                                   ...
                                   set_cpus_allowed_ptr(t, cm);

set_cpus_allowed_ptr(t, cm);

This results in CPU2's update being overwritten by that of CPU1, and
thus the possibility of ->boost_kthread_task continuing to run on a
to-be-offlined CPU.

This commit therefore eliminates this race by relying on the pre-existing
acquisition of ->boost_kthread_mutex to serialize the full process of
changing the affinity of ->boost_kthread_task.

Signed-off-by: default avatarPingfan Liu <kernelfans@gmail.com>
Cc: David Woodhouse <dwmw@amazon.co.uk>
Cc: Frederic Weisbecker <frederic@kernel.org>
Cc: Neeraj Upadhyay <quic_neeraju@quicinc.com>
Cc: Josh Triplett <josh@joshtriplett.org>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Lai Jiangshan <jiangshanlai@gmail.com>
Cc: Joel Fernandes <joel@joelfernandes.org>
Cc: "Jason A. Donenfeld" <Jason@zx2c4.com>
Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
parent d6fd907a

kernel/rcu/tree_plugin.h

+4 −1
Original line number Diff line number Diff line
@@ -1221,11 +1221,13 @@ static void rcu_spawn_one_boost_kthread(struct rcu_node *rnp) 
 * We don't include outgoingcpu in the affinity set, use -1 if there is

 * no outgoing CPU.  If there are no CPUs left in the affinity set,

 * this function allows the kthread to execute on any CPU.

 *

 * Any future concurrent calls are serialized via ->boost_kthread_mutex.

 */

static void rcu_boost_kthread_setaffinity(struct rcu_node *rnp, int outgoingcpu)

{

	struct task_struct *t = rnp->boost_kthread_task;

	unsigned long mask = rcu_rnp_online_cpus(rnp);

	unsigned long mask;

	cpumask_var_t cm;

	int cpu;
@@ -1234,6 +1236,7 @@ static void rcu_boost_kthread_setaffinity(struct rcu_node *rnp, int outgoingcpu) 
	if (!zalloc_cpumask_var(&cm, GFP_KERNEL))

		return;

	mutex_lock(&rnp->boost_kthread_mutex);

	mask = rcu_rnp_online_cpus(rnp);

	for_each_leaf_node_possible_cpu(rnp, cpu)

		if ((mask & leaf_node_cpu_bit(rnp, cpu)) &&

		    cpu != outgoingcpu)