Commit 649bb75d authored by Almog Khaikin's avatar Almog Khaikin Committed by Jens Axboe
Browse files

io_uring: fix memory ordering when SQPOLL thread goes to sleep 



Without a full memory barrier between the store to the flags and the
load of the SQ tail the two operations can be reordered and this can
lead to a situation where the SQPOLL thread goes to sleep while the
application writes to the SQ tail and doesn't see the wakeup flag.
This memory barrier pairs with a full memory barrier in the application
between its store to the SQ tail and its load of the flags.

Signed-off-by: default avatarAlmog Khaikin <almogkh@gmail.com>
Link: https://lore.kernel.org/r/20220321090059.46313-1-almogkh@gmail.com


Signed-off-by: default avatarJens Axboe <axboe@kernel.dk>
parent f63cf519

fs/io_uring.c

+7 −0
Original line number Diff line number Diff line
@@ -8042,6 +8042,13 @@ static int io_sq_thread(void *data) 
					needs_sched = false;

					break;

				}



				/*

				 * Ensure the store of the wakeup flag is not

				 * reordered with the load of the SQ tail

				 */

				smp_mb();



				if (io_sqring_entries(ctx)) {

					needs_sched = false;

					break;