Commit 5a519c8f authored by Andrei Vagin's avatar Andrei Vagin Committed by Linus Torvalds
Browse files

fs/pipe: use kvcalloc to allocate a pipe_buffer array 

Right now, kcalloc is used to allocate a pipe_buffer array.  The size of
the pipe_buffer struct is 40 bytes.  kcalloc allows allocating reliably
chunks with sizes less or equal to PAGE_ALLOC_COSTLY_ORDER (3).  It
means that the maximum pipe size is 3.2MB in this case.

In CRIU, we use pipes to dump processes memory.  CRIU freezes a target
process, injects a parasite code into it and then this code splices
memory into pipes.  If a maximum pipe size is small, we need to do many
iterations or create many pipes.

kvcalloc attempt to allocate physically contiguous memory, but upon
failure, fall back to non-contiguous (vmalloc) allocation and so it
isn't limited by PAGE_ALLOC_COSTLY_ORDER.

The maximum pipe size for non-root users is limited by the
/proc/sys/fs/pipe-max-size sysctl that is 1MB by default, so only the
root user will be able to trigger vmalloc allocations.

Link: https://lkml.kernel.org/r/20220104171058.22580-1-avagin@gmail.com


Signed-off-by: default avatarAndrei Vagin <avagin@gmail.com>
Reviewed-by: default avatarDmitry Safonov <0x7f454c46@gmail.com>
Cc: Alexander Viro <viro@zeniv.linux.org.uk>
Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
parent f9a40b08

fs/pipe.c

+4 −5
Original line number Diff line number Diff line
@@ -804,7 +804,7 @@ struct pipe_inode_info *alloc_pipe_info(void) 
	if (too_many_pipe_buffers_hard(user_bufs) && pipe_is_unprivileged_user())

		goto out_revert_acct;



	pipe->bufs = kcalloc(pipe_bufs, sizeof(struct pipe_buffer),

	pipe->bufs = kvcalloc(pipe_bufs, sizeof(struct pipe_buffer),

			     GFP_KERNEL_ACCOUNT);



	if (pipe->bufs) {
@@ -849,7 +849,7 @@ void free_pipe_info(struct pipe_inode_info *pipe) 
#endif

	if (pipe->tmp_page)

		__free_page(pipe->tmp_page);

	kfree(pipe->bufs);

	kvfree(pipe->bufs);

	kfree(pipe);

}
@@ -1264,8 +1264,7 @@ int pipe_resize_ring(struct pipe_inode_info *pipe, unsigned int nr_slots) 
	if (nr_slots < n)

		return -EBUSY;



	bufs = kcalloc(nr_slots, sizeof(*bufs),

		       GFP_KERNEL_ACCOUNT | __GFP_NOWARN);

	bufs = kvcalloc(nr_slots, sizeof(*bufs), GFP_KERNEL_ACCOUNT);

	if (unlikely(!bufs))

		return -ENOMEM;
@@ -1292,7 +1291,7 @@ int pipe_resize_ring(struct pipe_inode_info *pipe, unsigned int nr_slots) 
	head = n;

	tail = 0;



	kfree(pipe->bufs);

	kvfree(pipe->bufs);

	pipe->bufs = bufs;

	pipe->ring_size = nr_slots;

	if (pipe->max_usage > nr_slots)