Commit 7e692df3 authored by Eric Dumazet's avatar Eric Dumazet Committed by David S. Miller
Browse files

tcp: fix skb_copy_ubufs() vs BIG TCP 



David Ahern reported crashes in skb_copy_ubufs() caused by TCP tx zerocopy
using hugepages, and skb length bigger than ~68 KB.

skb_copy_ubufs() assumed it could copy all payload using up to
MAX_SKB_FRAGS order-0 pages.

This assumption broke when BIG TCP was able to put up to 512 KB per skb.

We did not hit this bug at Google because we use CONFIG_MAX_SKB_FRAGS=45
and limit gso_max_size to 180000.

A solution is to use higher order pages if needed.

v2: add missing __GFP_COMP, or we leak memory.

Fixes: 7c4e983c ("net: allow gso_max_size to exceed 65536")
Reported-by: default avatarDavid Ahern <dsahern@kernel.org>
Link: https://lore.kernel.org/netdev/c70000f6-baa4-4a05-46d0-4b3e0dc1ccc8@gmail.com/T/


Signed-off-by: default avatarEric Dumazet <edumazet@google.com>
Cc: Xin Long <lucien.xin@gmail.com>
Cc: Willem de Bruijn <willemb@google.com>
Cc: Coco Li <lixiaoyan@google.com>
Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
parent 6f75cd16

net/core/skbuff.c

+14 −6
Original line number Diff line number Diff line
@@ -1758,7 +1758,7 @@ int skb_copy_ubufs(struct sk_buff *skb, gfp_t gfp_mask) 
{

	int num_frags = skb_shinfo(skb)->nr_frags;

	struct page *page, *head = NULL;

	int i, new_frags;

	int i, order, psize, new_frags;

	u32 d_off;



	if (skb_shared(skb) || skb_unclone(skb, gfp_mask))
@@ -1767,9 +1767,17 @@ int skb_copy_ubufs(struct sk_buff *skb, gfp_t gfp_mask) 
	if (!num_frags)

		goto release;



	new_frags = (__skb_pagelen(skb) + PAGE_SIZE - 1) >> PAGE_SHIFT;

	/* We might have to allocate high order pages, so compute what minimum

	 * page order is needed.

	 */

	order = 0;

	while ((PAGE_SIZE << order) * MAX_SKB_FRAGS < __skb_pagelen(skb))

		order++;

	psize = (PAGE_SIZE << order);



	new_frags = (__skb_pagelen(skb) + psize - 1) >> (PAGE_SHIFT + order);

	for (i = 0; i < new_frags; i++) {

		page = alloc_page(gfp_mask);

		page = alloc_pages(gfp_mask | __GFP_COMP, order);

		if (!page) {

			while (head) {

				struct page *next = (struct page *)page_private(head);
@@ -1796,11 +1804,11 @@ int skb_copy_ubufs(struct sk_buff *skb, gfp_t gfp_mask) 
			vaddr = kmap_atomic(p);



			while (done < p_len) {

				if (d_off == PAGE_SIZE) {

				if (d_off == psize) {

					d_off = 0;

					page = (struct page *)page_private(page);

				}

				copy = min_t(u32, PAGE_SIZE - d_off, p_len - done);

				copy = min_t(u32, psize - d_off, p_len - done);

				memcpy(page_address(page) + d_off,

				       vaddr + p_off + done, copy);

				done += copy;
@@ -1816,7 +1824,7 @@ int skb_copy_ubufs(struct sk_buff *skb, gfp_t gfp_mask) 


	/* skb frags point to kernel buffers */

	for (i = 0; i < new_frags - 1; i++) {

		__skb_fill_page_desc(skb, i, head, 0, PAGE_SIZE);

		__skb_fill_page_desc(skb, i, head, 0, psize);

		head = (struct page *)page_private(head);

	}

	__skb_fill_page_desc(skb, new_frags - 1, head, 0, d_off);