Commit bd99c71b authored by Hao Xu's avatar Hao Xu Committed by Jens Axboe
Browse files

io_uring: fix race between poll completion and cancel_hash insertion 



If poll arming and poll completion runs in parallel, there maybe races.
For instance, run io_poll_add in iowq and io_poll_task_func in original
context, then:

  iowq                                      original context
  io_poll_add
    vfs_poll
     (interruption happens
      tw queued to original
      context)                              io_poll_task_func
                                              generate cqe
                                              del from cancel_hash[]
    if !poll.done
      insert to cancel_hash[]

The entry left in cancel_hash[], similar case for fast poll.
Fix it by set poll.done = true when del from cancel_hash[].

Fixes: 5082620f ("io_uring: terminate multishot poll for CQ ring overflow")
Signed-off-by: default avatarHao Xu <haoxu@linux.alibaba.com>
Link: https://lore.kernel.org/r/20210922101238.7177-2-haoxu@linux.alibaba.com


Signed-off-by: default avatarJens Axboe <axboe@kernel.dk>
parent 87c16966

fs/io_uring.c

+3 −3
Original line number Diff line number Diff line
@@ -5337,10 +5337,8 @@ static bool __io_poll_complete(struct io_kiocb *req, __poll_t mask) 
	}

	if (req->poll.events & EPOLLONESHOT)

		flags = 0;

	if (!io_cqring_fill_event(ctx, req->user_data, error, flags)) {

		req->poll.done = true;

	if (!io_cqring_fill_event(ctx, req->user_data, error, flags))

		flags = 0;

	}

	if (flags & IORING_CQE_F_MORE)

		ctx->cq_extra++;
@@ -5371,6 +5369,7 @@ static void io_poll_task_func(struct io_kiocb *req, bool *locked) 
		if (done) {

			io_poll_remove_double(req);

			hash_del(&req->hash_node);

			req->poll.done = true;

		} else {

			req->result = 0;

			add_wait_queue(req->poll.head, &req->poll.wait);
@@ -5508,6 +5507,7 @@ static void io_async_task_func(struct io_kiocb *req, bool *locked) 


	hash_del(&req->hash_node);

	io_poll_remove_double(req);

	apoll->poll.done = true;

	spin_unlock(&ctx->completion_lock);



	if (!READ_ONCE(apoll->poll.canceled))