Commit 59409373 authored Jan 07, 2022 by Matthew Wilcox (Oracle)

mm/gup: Handle page split race more efficiently



If we hit the page split race, the current code returns NULL which will
presumably trigger a retry under the mmap_lock.  This isn't necessary;
we can just retry the compound_head() lookup.  This is a very minor
optimisation of an unlikely path, but conceptually it matches (eg)
the page cache RCU-protected lookup.

Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: John Hubbard <jhubbard@nvidia.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Reviewed-by: William Kucharski <william.kucharski@oracle.com>

parent 4c654229

mm/gup.c

+5 −2

Original line number	Diff line number	Diff line
		@@ -68,7 +68,10 @@ static void put_page_refs(struct page *page, int refs)
		*/
		static inline struct page try_get_compound_head(struct page page, int refs)
		{
		struct page *head = compound_head(page);
		struct page *head;

		retry:
		head = compound_head(page);

		if (WARN_ON_ONCE(page_ref_count(head) < 0))
		return NULL;
		@@ -86,7 +89,7 @@ static inline struct page try_get_compound_head(struct page page, int refs)
		*/
		if (unlikely(compound_head(page) != head)) {
		put_page_refs(head, refs);
		return NULL;
		goto retry;
		}

		return head;