Skip to content
  1. May 30, 2013
  2. May 29, 2013
  3. May 28, 2013
  4. May 16, 2013
  5. May 15, 2013
  6. May 14, 2013
    • Tirupathi Reddy's avatar
      timer: Don't reinitialize the cpu base lock during CPU_UP_PREPARE · 42a5cf46
      Tirupathi Reddy authored
      
      
      An inactive timer's base can refer to a offline cpu's base.
      
      In the current code, cpu_base's lock is blindly reinitialized each
      time a CPU is brought up. If a CPU is brought online during the period
      that another thread is trying to modify an inactive timer on that CPU
      with holding its timer base lock, then the lock will be reinitialized
      under its feet. This leads to following SPIN_BUG().
      
      <0> BUG: spinlock already unlocked on CPU#3, kworker/u:3/1466
      <0> lock: 0xe3ebe000, .magic: dead4ead, .owner: kworker/u:3/1466, .owner_cpu: 1
      <4> [<c0013dc4>] (unwind_backtrace+0x0/0x11c) from [<c026e794>] (do_raw_spin_unlock+0x40/0xcc)
      <4> [<c026e794>] (do_raw_spin_unlock+0x40/0xcc) from [<c076c160>] (_raw_spin_unlock+0x8/0x30)
      <4> [<c076c160>] (_raw_spin_unlock+0x8/0x30) from [<c009b858>] (mod_timer+0x294/0x310)
      <4> [<c009b858>] (mod_timer+0x294/0x310) from [<c00a5e04>] (queue_delayed_work_on+0x104/0x120)
      <4> [<c00a5e04>] (queue_delayed_work_on+0x104/0x120) from [<c04eae00>] (sdhci_msm_bus_voting+0x88/0x9c)
      <4> [<c04eae00>] (sdhci_msm_bus_voting+0x88/0x9c) from [<c04d8780>] (sdhci_disable+0x40/0x48)
      <4> [<c04d8780>] (sdhci_disable+0x40/0x48) from [<c04bf300>] (mmc_release_host+0x4c/0xb0)
      <4> [<c04bf300>] (mmc_release_host+0x4c/0xb0) from [<c04c7aac>] (mmc_sd_detect+0x90/0xfc)
      <4> [<c04c7aac>] (mmc_sd_detect+0x90/0xfc) from [<c04c2504>] (mmc_rescan+0x7c/0x2c4)
      <4> [<c04c2504>] (mmc_rescan+0x7c/0x2c4) from [<c00a6a7c>] (process_one_work+0x27c/0x484)
      <4> [<c00a6a7c>] (process_one_work+0x27c/0x484) from [<c00a6e94>] (worker_thread+0x210/0x3b0)
      <4> [<c00a6e94>] (worker_thread+0x210/0x3b0) from [<c00aad9c>] (kthread+0x80/0x8c)
      <4> [<c00aad9c>] (kthread+0x80/0x8c) from [<c000ea80>] (kernel_thread_exit+0x0/0x8)
      
      As an example, this particular crash occurred when CPU #3 is executing
      mod_timer() on an inactive timer whose base is refered to offlined CPU
      #2.  The code locked the timer_base corresponding to CPU #2. Before it
      could proceed, CPU #2 came online and reinitialized the spinlock
      corresponding to its base. Thus now CPU #3 held a lock which was
      reinitialized. When CPU #3 finally ended up unlocking the old cpu_base
      corresponding to CPU #2, we hit the above SPIN_BUG().
      
      CPU #0		CPU #3				       CPU #2
      ------		-------				       -------
      .....		 ......				      <Offline>
      		mod_timer()
      		 lock_timer_base
      		   spin_lock_irqsave(&base->lock)
      
      cpu_up(2)	 .....				        ......
      							init_timers_cpu()
      ....		 .....				    	spin_lock_init(&base->lock)
      .....		   spin_unlock_irqrestore(&base->lock)  ......
      		   <spin_bug>
      
      Allocation of per_cpu timer vector bases is done only once under
      "tvec_base_done[]" check. In the current code, spinlock_initialization
      of base->lock isn't under this check. When a CPU is up each time the
      base lock is reinitialized. Move base spinlock initialization under
      the check.
      
      Signed-off-by: default avatarTirupathi Reddy <tirupath@codeaurora.org>
      Cc: stable@vger.kernel.org
      Link: http://lkml.kernel.org/r/1368520142-4136-1-git-send-email-tirupath@codeaurora.org
      
      
      Signed-off-by: default avatarThomas Gleixner <tglx@linutronix.de>
      42a5cf46
    • Srivatsa S. Bhat's avatar
      rcu/idle: Wrap cpu-idle poll mode within rcu_idle_enter/exit · b47430d3
      Srivatsa S. Bhat authored
      
      
      Bjørn Mork reported the following warning when running powertop.
      
      [   49.289034] ------------[ cut here ]------------
      [   49.289055] WARNING: at kernel/rcutree.c:502 rcu_eqs_exit_common.isra.48+0x3d/0x125()
      [   49.289244] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 3.10.0-bisect-rcu-warn+ #107
      [   49.289251]  ffffffff8157d8c8 ffffffff81801e28 ffffffff8137e4e3 ffffffff81801e68
      [   49.289260]  ffffffff8103094f ffffffff81801e68 0000000000000000 ffff88023afcd9b0
      [   49.289268]  0000000000000000 0140000000000000 ffff88023bee7700 ffffffff81801e78
      [   49.289276] Call Trace:
      [   49.289285]  [<ffffffff8137e4e3>] dump_stack+0x19/0x1b
      [   49.289293]  [<ffffffff8103094f>] warn_slowpath_common+0x62/0x7b
      [   49.289300]  [<ffffffff8103097d>] warn_slowpath_null+0x15/0x17
      [   49.289306]  [<ffffffff810a9006>] rcu_eqs_exit_common.isra.48+0x3d/0x125
      [   49.289314]  [<ffffffff81079b49>] ? trace_hardirqs_off_caller+0x37/0xa6
      [   49.289320]  [<ffffffff810a9692>] rcu_idle_exit+0x85/0xa8
      [   49.289327]  [<ffffffff8107076e>] trace_cpu_idle_rcuidle+0xae/0xff
      [   49.289334]  [<ffffffff810708b1>] cpu_startup_entry+0x72/0x115
      [   49.289341]  [<ffffffff813689e5>] rest_init+0x149/0x150
      [   49.289347]  [<ffffffff8136889c>] ? csum_partial_copy_generic+0x16c/0x16c
      [   49.289355]  [<ffffffff81a82d34>] start_kernel+0x3f0/0x3fd
      [   49.289362]  [<ffffffff81a8274c>] ? repair_env_string+0x5a/0x5a
      [   49.289368]  [<ffffffff81a82481>] x86_64_start_reservations+0x2a/0x2c
      [   49.289375]  [<ffffffff81a82550>] x86_64_start_kernel+0xcd/0xd1
      [   49.289379] ---[ end trace 07a1cc95e29e9036 ]---
      
      The warning is that 'rdtp->dynticks' has an unexpected value, which roughly
      translates to - the calls to rcu_idle_enter() and rcu_idle_exit() were not
      made in the correct order, or otherwise messed up.
      
      And Bjørn's painstaking debugging indicated that this happens when the idle
      loop enters the poll mode. Looking at the poll function cpu_idle_poll(), and
      the implementation of trace_cpu_idle_rcuidle(), the problem becomes very clear:
      cpu_idle_poll() lacks calls to rcu_idle_enter/exit(), and trace_cpu_idle_rcuidle()
      calls them in the reverse order - first rcu_idle_exit(), and then rcu_idle_enter().
      Hence the even/odd alternative sequencing of rdtp->dynticks goes for a toss.
      
      And powertop readily triggers this because powertop uses the idle-tracing
      infrastructure extensively.
      
      So, to fix this, wrap the code in cpu_idle_poll() within rcu_idle_enter/exit(),
      so that it blends properly with the calls inside trace_cpu_idle_rcuidle() and
      thus get the function ordering right.
      
      Reported-and-tested-by: default avatarBjørn Mork <bjorn@mork.no>
      Cc: Paul McKenney <paulmck@linux.vnet.ibm.com>
      Cc: Steven Rostedt <rostedt@goodmis.org>
      Cc: Dipankar Sarma <dipankar@in.ibm.com>
      Signed-off-by: default avatarSrivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
      Link: http://lkml.kernel.org/r/519169BF.4080208@linux.vnet.ibm.com
      
      
      Signed-off-by: default avatarThomas Gleixner <tglx@linutronix.de>
      b47430d3