Commit 48c03331 authored by Jan Kara's avatar Jan Kara Committed by Jens Axboe
Browse files

sbitmap: Avoid leaving waitqueue in invalid state in __sbq_wake_up() 



When __sbq_wake_up() decrements wait_cnt to 0 but races with someone
else waking the waiter on the waitqueue (so the waitqueue becomes
empty), it exits without reseting wait_cnt to wake_batch number. Once
wait_cnt is 0, nobody will ever reset the wait_cnt or wake the new
waiters resulting in possible deadlocks or busyloops. Fix the problem by
making sure we reset wait_cnt even if we didn't wake up anybody in the
end.

Fixes: 040b83fc ("sbitmap: fix possible io hung due to lost wakeup")
Reported-by: default avatarKeith Busch <kbusch@kernel.org>
Signed-off-by: default avatarJan Kara <jack@suse.cz>
Link: https://lore.kernel.org/r/20220908130937.2795-1-jack@suse.cz


Signed-off-by: default avatarJens Axboe <axboe@kernel.dk>
parent 8807707d

lib/sbitmap.c

+15 −3
Original line number Diff line number Diff line
@@ -604,6 +604,7 @@ static bool __sbq_wake_up(struct sbitmap_queue *sbq) 
	struct sbq_wait_state *ws;

	unsigned int wake_batch;

	int wait_cnt;

	bool ret;



	ws = sbq_wake_ptr(sbq);

	if (!ws)
@@ -614,12 +615,23 @@ static bool __sbq_wake_up(struct sbitmap_queue *sbq) 
	 * For concurrent callers of this, callers should call this function

	 * again to wakeup a new batch on a different 'ws'.

	 */

	if (wait_cnt < 0 || !waitqueue_active(&ws->wait))

	if (wait_cnt < 0)

		return true;



	/*

	 * If we decremented queue without waiters, retry to avoid lost

	 * wakeups.

	 */

	if (wait_cnt > 0)

		return false;

		return !waitqueue_active(&ws->wait);



	/*

	 * When wait_cnt == 0, we have to be particularly careful as we are

	 * responsible to reset wait_cnt regardless whether we've actually

	 * woken up anybody. But in case we didn't wakeup anybody, we still

	 * need to retry.

	 */

	ret = !waitqueue_active(&ws->wait);

	wake_batch = READ_ONCE(sbq->wake_batch);



	/*
@@ -648,7 +660,7 @@ static bool __sbq_wake_up(struct sbitmap_queue *sbq) 
	sbq_index_atomic_inc(&sbq->wake_index);

	atomic_set(&ws->wait_cnt, wake_batch);



	return false;

	return ret;

}



void sbitmap_queue_wake_up(struct sbitmap_queue *sbq)