Commit 9ab03be4 authored by Valentin Schneider's avatar Valentin Schneider Committed by Tejun Heo
Browse files

workqueue: Don't hold any lock while rcuwait'ing for !POOL_MANAGER_ACTIVE 



put_unbound_pool() currently passes wq_manager_inactive() as exit condition
to rcuwait_wait_event(), which grabs pool->lock to check for

  pool->flags & POOL_MANAGER_ACTIVE

A later patch will require destroy_worker() to be invoked with
wq_pool_attach_mutex held, which needs to be acquired before
pool->lock. A mutex cannot be acquired within rcuwait_wait_event(), as
it could clobber the task state set by rcuwait_wait_event()

Instead, restructure the waiting logic to acquire any necessary lock
outside of rcuwait_wait_event().

Since further work cannot be inserted into unbound pwqs that have reached
->refcnt==0, this is bound to make forward progress as eventually the
worklist will be drained and need_more_worker(pool) will remain false,
preventing any worker from stealing the manager position from us.

Suggested-by: default avatarTejun Heo <tj@kernel.org>
Signed-off-by: default avatarValentin Schneider <vschneid@redhat.com>
Signed-off-by: default avatarTejun Heo <tj@kernel.org>
parent 3f959aa3

kernel/workqueue.c

+19 −17
Original line number Diff line number Diff line
@@ -3608,18 +3608,6 @@ static void rcu_free_pool(struct rcu_head *rcu) 
	kfree(pool);

}



/* This returns with the lock held on success (pool manager is inactive). */

static bool wq_manager_inactive(struct worker_pool *pool)

{

	raw_spin_lock_irq(&pool->lock);



	if (pool->flags & POOL_MANAGER_ACTIVE) {

		raw_spin_unlock_irq(&pool->lock);

		return false;

	}

	return true;

}



/**

 * put_unbound_pool - put a worker_pool

 * @pool: worker_pool to put
@@ -3655,12 +3643,26 @@ static void put_unbound_pool(struct worker_pool *pool) 
	 * Become the manager and destroy all workers.  This prevents

	 * @pool's workers from blocking on attach_mutex.  We're the last

	 * manager and @pool gets freed with the flag set.

	 * Because of how wq_manager_inactive() works, we will hold the

	 * spinlock after a successful wait.

	 *

	 * Having a concurrent manager is quite unlikely to happen as we can

	 * only get here with

	 *   pwq->refcnt == pool->refcnt == 0

	 * which implies no work queued to the pool, which implies no worker can

	 * become the manager. However a worker could have taken the role of

	 * manager before the refcnts dropped to 0, since maybe_create_worker()

	 * drops pool->lock

	 */

	rcuwait_wait_event(&manager_wait, wq_manager_inactive(pool),

	while (true) {

		rcuwait_wait_event(&manager_wait,

				   !(pool->flags & POOL_MANAGER_ACTIVE),

				   TASK_UNINTERRUPTIBLE);

		raw_spin_lock_irq(&pool->lock);

		if (!(pool->flags & POOL_MANAGER_ACTIVE)) {

			pool->flags |= POOL_MANAGER_ACTIVE;

			break;

		}

		raw_spin_unlock_irq(&pool->lock);

	}



	while ((worker = first_idle_worker(pool)))

		destroy_worker(worker);