Commit 5cc5924d authored by Alexei Starovoitov's avatar Alexei Starovoitov
Browse files

Merge branch 'xdp-grow-tail' 

Jesper Dangaard Brouer says:

====================
V4:
- Fixup checkpatch.pl issues
- Collected more ACKs

V3:
- Fix issue on virtio_net patch spotted by Jason Wang
- Adjust name for variable in mlx5 patch
- Collected more ACKs

V2:
- Fix bug in mlx5 for XDP_PASS case
- Collected nitpicks and ACKs from mailing list

V1:
- Fix bug in dpaa2

XDP have evolved to support several frame sizes, but xdp_buff was not
updated with this information. This have caused the side-effect that
XDP frame data hard end is unknown. This have limited the BPF-helper
bpf_xdp_adjust_tail to only shrink the packet. This patchset address
this and add packet tail extend/grow.

The purpose of the patchset is ALSO to reserve a memory area that can be
used for storing extra information, specifically for extending XDP with
multi-buffer support. One proposal is to use same layout as
skb_shared_info, which is why this area is currently 320 bytes.

When converting xdp_frame to SKB (veth and cpumap), the full tailroom
area can now be used and SKB truesize is now correct. For most
drivers this result in a much larger tailroom in SKB "head" data
area. The network stack can now take advantage of this when doing SKB
coalescing. Thus, a good driver test is to use xdp_redirect_cpu from
samples/bpf/ and do some TCP stream testing.

Use-cases for tail grow/extend:
(1) IPsec / XFRM needs a tail extend[1][2].
(2) DNS-cache responses in XDP.
(3) HAProxy ALOHA would need it to convert to XDP.
(4) Add tail info e.g. timestamp and collect via tcpdump

[1] http://vger.kernel.org/netconf2019_files/xfrm_xdp.pdf
[2] http://vger.kernel.org/netconf2019.html

Examples on howto access the tail area of an XDP packet is shown in the
XDP-tutorial example[3].

[3] https://github.com/xdp-project/xdp-tutorial/blob/master/experiment01-tailgrow/


====================

Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
parents d00f26b6 7ae2e00e

drivers/net/ethernet/amazon/ena/ena_netdev.c

+1 −0
Original line number Diff line number Diff line
@@ -1606,6 +1606,7 @@ static int ena_clean_rx_irq(struct ena_ring *rx_ring, struct napi_struct *napi, 
		  "%s qid %d\n", __func__, rx_ring->qid);

	res_budget = budget;

	xdp.rxq = &rx_ring->xdp_rxq;

	xdp.frame_sz = ENA_PAGE_SIZE;



	do {

		xdp_verdict = XDP_PASS;

drivers/net/ethernet/amazon/ena/ena_netdev.h

+3 −2
Original line number Diff line number Diff line
@@ -152,7 +152,8 @@ 
 */



#define ENA_XDP_MAX_MTU (ENA_PAGE_SIZE - ETH_HLEN - ETH_FCS_LEN -	\

				VLAN_HLEN - XDP_PACKET_HEADROOM)

			 VLAN_HLEN - XDP_PACKET_HEADROOM -		\

			 SKB_DATA_ALIGN(sizeof(struct skb_shared_info)))



#define ENA_IS_XDP_INDEX(adapter, index) (((index) >= (adapter)->xdp_first_ring) && \

	((index) < (adapter)->xdp_first_ring + (adapter)->xdp_num_queues))

drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c

+1 −0
Original line number Diff line number Diff line
@@ -138,6 +138,7 @@ bool bnxt_rx_xdp(struct bnxt *bp, struct bnxt_rx_ring_info *rxr, u16 cons, 
	xdp_set_data_meta_invalid(&xdp);

	xdp.data_end = *data_ptr + *len;

	xdp.rxq = &rxr->xdp_rxq;

	xdp.frame_sz = PAGE_SIZE; /* BNXT_RX_PAGE_MODE(bp) when XDP enabled */

	orig_data = xdp.data;



	rcu_read_lock();

drivers/net/ethernet/cavium/thunder/nicvf_main.c

+1 −0
Original line number Diff line number Diff line
@@ -552,6 +552,7 @@ static inline bool nicvf_xdp_rx(struct nicvf *nic, struct bpf_prog *prog, 
	xdp_set_data_meta_invalid(&xdp);

	xdp.data_end = xdp.data + len;

	xdp.rxq = &rq->xdp_rxq;

	xdp.frame_sz = RCV_FRAG_LEN + XDP_PACKET_HEADROOM;

	orig_data = xdp.data;



	rcu_read_lock();

drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c

+7 −0
Original line number Diff line number Diff line
@@ -331,6 +331,9 @@ static u32 run_xdp(struct dpaa2_eth_priv *priv, 
	xdp_set_data_meta_invalid(&xdp);

	xdp.rxq = &ch->xdp_rxq;



	xdp.frame_sz = DPAA2_ETH_RX_BUF_RAW_SIZE -

		(dpaa2_fd_get_offset(fd) - XDP_PACKET_HEADROOM);



	xdp_act = bpf_prog_run_xdp(xdp_prog, &xdp);



	/* xdp.data pointer may have changed */
@@ -366,7 +369,11 @@ static u32 run_xdp(struct dpaa2_eth_priv *priv, 
		dma_unmap_page(priv->net_dev->dev.parent, addr,

			       DPAA2_ETH_RX_BUF_SIZE, DMA_BIDIRECTIONAL);

		ch->buf_count--;



		/* Allow redirect use of full headroom */

		xdp.data_hard_start = vaddr;

		xdp.frame_sz = DPAA2_ETH_RX_BUF_RAW_SIZE;



		err = xdp_do_redirect(priv->net_dev, &xdp, xdp_prog);

		if (unlikely(err))

			ch->stats.xdp_drop++;