Commit 2525d77c authored Aug 04, 2022 by Frederic Weisbecker Committed by Zheng Zengkai Aug 04, 2022

rcu: Apply callbacks processing time limit only on softirq

stable inclusion
from stable-v5.10.115
commit a22d66eb518f444bd66b38dbf8235d98e40f37e1
category: bugfix
bugzilla: https://gitee.com/openeuler/kernel/issues/I5IZ9C

Reference: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=a22d66eb518f444bd66b38dbf8235d98e40f37e1



--------------------------------

commit a554ba28 upstream.

Time limit only makes sense when callbacks are serviced in softirq mode
because:

_ In case we need to get back to the scheduler,
  cond_resched_tasks_rcu_qs() is called after each callback.

_ In case some other softirq vector needs the CPU, the call to
  local_bh_enable() before cond_resched_tasks_rcu_qs() takes care about
  them via a call to do_softirq().

Therefore, make sure the time limit only applies to softirq mode.

Reviewed-by: Valentin Schneider <valentin.schneider@arm.com>
Tested-by: Valentin Schneider <valentin.schneider@arm.com>
Tested-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Cc: Valentin Schneider <valentin.schneider@arm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Cc: Josh Triplett <josh@joshtriplett.org>
Cc: Joel Fernandes <joel@joelfernandes.org>
Cc: Boqun Feng <boqun.feng@gmail.com>
Cc: Neeraj Upadhyay <neeraju@codeaurora.org>
Cc: Uladzislau Rezki <urezki@gmail.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
[UR: backport to 5.10-stable]
Signed-off-by: Uladzislau Rezki (Sony) <urezki@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Zheng Zengkai <zhengzengkai@huawei.com>
Acked-by: Xie XiuQi <xiexiuqi@huawei.com>

parent 8daec20c

kernel/rcu/tree.c

+13 −13

Original line number	Diff line number	Diff line
		@@ -2478,7 +2478,7 @@ static void rcu_do_batch(struct rcu_data *rdp)
		div = READ_ONCE(rcu_divisor);
		div = div < 0 ? 7 : div > sizeof(long) * 8 - 2 ? sizeof(long) * 8 - 2 : div;
		bl = max(rdp->blimit, pending >> div);
		if (unlikely(bl > 100)) {
		if (in_serving_softirq() && unlikely(bl > 100)) {
		long rrn = READ_ONCE(rcu_resched_ns);

		rrn = rrn < NSEC_PER_MSEC ? NSEC_PER_MSEC : rrn > NSEC_PER_SEC ? NSEC_PER_SEC : rrn;
		@@ -2516,13 +2516,6 @@ static void rcu_do_batch(struct rcu_data *rdp)
		if (-rcl.len >= bl && (need_resched() \|\|
		(!is_idle_task(current) && !rcu_is_callbacks_kthread())))
		break;
		} else {
		local_bh_enable();
		lockdep_assert_irqs_enabled();
		cond_resched_tasks_rcu_qs();
		lockdep_assert_irqs_enabled();
		local_bh_disable();
		}

		/*
		* Make sure we don't spend too much time here and deprive other
		@@ -2535,6 +2528,13 @@ static void rcu_do_batch(struct rcu_data *rdp)
		/* Exceeded the time limit, so leave. */
		break;
		}
		} else {
		local_bh_enable();
		lockdep_assert_irqs_enabled();
		cond_resched_tasks_rcu_qs();
		lockdep_assert_irqs_enabled();
		local_bh_disable();
		}
		}

		local_irq_save(flags);