Commit 2e9bafbf authored by Frederic Weisbecker's avatar Frederic Weisbecker Committed by Jialin Zhang
Browse files

timers/nohz: Last resort update jiffies on nohz_full IRQ entry 

mainline inclusion
from mainline-v5.16-rc4
commit 53e87e3c
category: bugfix
bugzilla: https://gitee.com/openeuler/kernel/issues/I6WCC1
CVE: NA

Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=53e87e3cdc155f20c3417b689df8d2ac88d79576



--------------------------------

When at least one CPU runs in nohz_full mode, a dedicated timekeeper CPU
is guaranteed to stay online and to never stop its tick.

Meanwhile on some rare case, the dedicated timekeeper may be running
with interrupts disabled for a while, such as in stop_machine.

If jiffies stop being updated, a nohz_full CPU may end up endlessly
programming the next tick in the past, taking the last jiffies update
monotonic timestamp as a stale base, resulting in an tick storm.

Here is a scenario where it matters:

0) CPU 0 is the timekeeper and CPU 1 a nohz_full CPU.

1) A stop machine callback is queued to execute somewhere.

2) CPU 0 reaches MULTI_STOP_DISABLE_IRQ while CPU 1 is still in
   MULTI_STOP_PREPARE. Hence CPU 0 can't do its timekeeping duty. CPU 1
   can still take IRQs.

3) CPU 1 receives an IRQ which queues a timer callback one jiffy forward.

4) On IRQ exit, CPU 1 schedules the tick one jiffy forward, taking
   last_jiffies_update as a base. But last_jiffies_update hasn't been
   updated for 2 jiffies since the timekeeper has interrupts disabled.

5) clockevents_program_event(), which relies on ktime_get(), observes
   that the expiration is in the past and therefore programs the min
   delta event on the clock.

6) The tick fires immediately, goto 3)

7) Tick storm, the nohz_full CPU is drown and takes ages to reach
   MULTI_STOP_DISABLE_IRQ, which is the only way out of this situation.

Solve this with unconditionally updating jiffies if the value is stale
on nohz_full IRQ entry. IRQs and other disturbances are expected to be
rare enough on nohz_full for the unconditional call to ktime_get() to
actually matter.

Reported-by: default avatarPaul E. McKenney <paulmck@kernel.org>
Signed-off-by: default avatarFrederic Weisbecker <frederic@kernel.org>
Signed-off-by: default avatarThomas Gleixner <tglx@linutronix.de>
Tested-by: default avatarPaul E. McKenney <paulmck@kernel.org>
Link: https://lore.kernel.org/r/20211026141055.57358-2-frederic@kernel.org



 Conflicts:
	kernel/softirq.c

Signed-off-by: default avatarYu Liao <liaoyu15@huawei.com>
Reviewed-by: default avatarXiongfeng Wang <wangxiongfeng2@huawei.com>
Signed-off-by: default avatarJialin Zhang <zhangjialin11@huawei.com>
parent 9b5e83a1

kernel/softirq.c

+2 −1
Original line number Diff line number Diff line
@@ -350,7 +350,8 @@ asmlinkage __visible void do_softirq(void) 
 */

void irq_enter_rcu(void)

{

	if (is_idle_task(current) && !in_interrupt()) {

	if (tick_nohz_full_cpu(smp_processor_id()) ||

		(is_idle_task(current) && !in_interrupt())) {

		/*

		 * Prevent raise_softirq from needlessly waking up ksoftirqd

		 * here, as softirq will be serviced on return from interrupt.

kernel/time/tick-sched.c

+7 −0
Original line number Diff line number Diff line
@@ -1347,6 +1347,13 @@ static inline void tick_nohz_irq_enter(void) 
	now = ktime_get();

	if (ts->idle_active)

		tick_nohz_stop_idle(ts, now);

	/*

	 * If all CPUs are idle. We may need to update a stale jiffies value.

	 * Note nohz_full is a special case: a timekeeper is guaranteed to stay

	 * alive but it might be busy looping with interrupts disabled in some

	 * rare case (typically stop machine). So we must make sure we have a

	 * last resort.

	 */

	if (ts->tick_stopped)

		tick_nohz_update_jiffies(now);

}