Unverified Commit 9c8467e9 authored by openeuler-ci-bot's avatar openeuler-ci-bot Committed by Gitee
Browse files

!1802 zram: Support multiple compression streams

Merge Pull Request from: @ci-robot 
 
PR sync from: Jinjiang Tu <tujinjiang@huawei.com>
https://mailweb.openeuler.org/hyperkitty/list/kernel@openeuler.org/message/UAQZCBQFUA5LYCLXXVHHJGAK2GNRA6LC/ 
Support multiple compression streams for zram.

Alexey Romanov (1):
  zram: add size class equals check into recompression

Andy Shevchenko (1):
  lib/cmdline: Export next_arg() for being used in modules

Ming Lei (1):
  zram: fix race between zram_reset_device() and disksize_store()

Sergey Senozhatsky (11):
  zram: preparation for multi-zcomp support
  zram: add recompression algorithm sysfs knob
  zram: factor out WB and non-WB zram read functions
  zram: introduce recompress sysfs knob
  zram: add recompress flag to read_block_state()
  zram: clarify writeback_store() comment
  zram: remove redundant checks from zram_recompress()
  zram: add algo parameter support to zram_recompress()
  documentation: add zram recompression documentation
  zram: add incompressible writeback
  zram: add incompressible flag to read_block_state()


-- 
2.25.1
 
https://gitee.com/openeuler/kernel/issues/I7TWVA 
 
Link:https://gitee.com/openeuler/kernel/pulls/1802

 

Reviewed-by: default avatarKefeng Wang <wangkefeng.wang@huawei.com>
Signed-off-by: default avatarJialin Zhang <zhangjialin11@huawei.com>
parents a5ef7db5 2ff365b3
Loading
Loading
Loading
Loading
+95 −3
Original line number Diff line number Diff line
@@ -334,6 +334,11 @@ Admin can request writeback of those idle pages at right timing via::

With the command, zram writeback idle pages from memory to the storage.

If a user chooses to writeback only incompressible pages (pages that none of
algorithms can compress) this can be accomplished with::

	echo incompressible > /sys/block/zramX/writeback

If there are lots of write IO with flash device, potentially, it has
flash wearout problem so that admin needs to design write limitation
to guarantee storage health for entire product life.
@@ -382,6 +387,87 @@ budget in next setting is user's job.
If admin wants to measure writeback count in a certain period, he could
know it via /sys/block/zram0/bd_stat's 3rd column.

recompression
-------------

With CONFIG_ZRAM_MULTI_COMP, zram can recompress pages using alternative
(secondary) compression algorithms. The basic idea is that alternative
compression algorithm can provide better compression ratio at a price of
(potentially) slower compression/decompression speeds. Alternative compression
algorithm can, for example, be more successful compressing huge pages (those
that default algorithm failed to compress). Another application is idle pages
recompression - pages that are cold and sit in the memory can be recompressed
using more effective algorithm and, hence, reduce zsmalloc memory usage.

With CONFIG_ZRAM_MULTI_COMP, zram supports up to 4 compression algorithms:
one primary and up to 3 secondary ones. Primary zram compressor is explained
in "3) Select compression algorithm", secondary algorithms are configured
using recomp_algorithm device attribute.

Example:::

	#show supported recompression algorithms
	cat /sys/block/zramX/recomp_algorithm
	#1: lzo lzo-rle lz4 lz4hc [zstd]
	#2: lzo lzo-rle lz4 [lz4hc] zstd

Alternative compression algorithms are sorted by priority. In the example
above, zstd is used as the first alternative algorithm, which has priority
of 1, while lz4hc is configured as a compression algorithm with priority 2.
Alternative compression algorithm's priority is provided during algorithms
configuration:::

	#select zstd recompression algorithm, priority 1
	echo "algo=zstd priority=1" > /sys/block/zramX/recomp_algorithm

	#select deflate recompression algorithm, priority 2
	echo "algo=deflate priority=2" > /sys/block/zramX/recomp_algorithm

Another device attribute that CONFIG_ZRAM_MULTI_COMP enables is recompress,
which controls recompression.

Examples:::

	#IDLE pages recompression is activated by `idle` mode
	echo "type=idle" > /sys/block/zramX/recompress

	#HUGE pages recompression is activated by `huge` mode
	echo "type=huge" > /sys/block/zram0/recompress

	#HUGE_IDLE pages recompression is activated by `huge_idle` mode
	echo "type=huge_idle" > /sys/block/zramX/recompress

The number of idle pages can be significant, so user-space can pass a size
threshold (in bytes) to the recompress knob: zram will recompress only pages
of equal or greater size:::

	#recompress all pages larger than 3000 bytes
	echo "threshold=3000" > /sys/block/zramX/recompress

	#recompress idle pages larger than 2000 bytes
	echo "type=idle threshold=2000" > /sys/block/zramX/recompress

Recompression of idle pages requires memory tracking.

During re-compression for every page, that matches re-compression criteria,
ZRAM iterates the list of registered alternative compression algorithms in
order of their priorities. ZRAM stops either when re-compression was
successful (re-compressed object is smaller in size than the original one)
and matches re-compression criteria (e.g. size threshold) or when there are
no secondary algorithms left to try. If none of the secondary algorithms can
successfully re-compressed the page such a page is marked as incompressible,
so ZRAM will not attempt to re-compress it in the future.

This re-compression behaviour, when it iterates through the list of
registered compression algorithms, increases our chances of finding the
algorithm that successfully compresses a particular page. Sometimes, however,
it is convenient (and sometimes even necessary) to limit recompression to
only one particular algorithm so that it will not try any other algorithms.
This can be achieved by providing a algo=NAME parameter:::

	#use zstd algorithm only (if registered)
	echo "type=huge algo=zstd" > /sys/block/zramX/recompress

memory tracking
===============

@@ -392,9 +478,11 @@ pages of the process with*pagemap.
If you enable the feature, you could see block state via
/sys/kernel/debug/zram/zram0/block_state". The output is as follows::

	  300    75.033841 .wh.
	  301    63.806904 s...
	  302    63.806919 ..hi
	  300    75.033841 .wh...
	  301    63.806904 s.....
	  302    63.806919 ..hi..
	  303    62.801919 ....r.
	  304   146.781902 ..hi.n

First column
	zram's block index.
@@ -411,6 +499,10 @@ Third column
		huge page
	i:
		idle page
	r:
		recompressed page (secondary compression algorithm)
	n:
		none (including secondary) of algorithms could compress it

First line of above example says 300th block is accessed at 75.033841sec
and the block's state is huge so it is written back to the backing
+9 −0
Original line number Diff line number Diff line
@@ -37,3 +37,12 @@ config ZRAM_MEMORY_TRACKING
	  /sys/kernel/debug/zram/zramX/block_state.

	  See Documentation/admin-guide/blockdev/zram.rst for more information.

config ZRAM_MULTI_COMP
	bool "Enable multiple compression streams"
	depends on ZRAM
	help
	  This will enable multi-compression streams, so that ZRAM can
	  re-compress pages using a potentially slower but more effective
	  compression algorithm. Note, that IDLE page recompression
	  requires ZRAM_MEMORY_TRACKING.
+3 −3
Original line number Diff line number Diff line
@@ -204,7 +204,7 @@ void zcomp_destroy(struct zcomp *comp)
 * case of allocation error, or any other error potentially
 * returned by zcomp_init().
 */
struct zcomp *zcomp_create(const char *compress)
struct zcomp *zcomp_create(const char *alg)
{
	struct zcomp *comp;
	int error;
@@ -214,14 +214,14 @@ struct zcomp *zcomp_create(const char *compress)
	 * is not loaded yet. We must do it here, otherwise we are about to
	 * call /sbin/modprobe under CPU hot-plug lock.
	 */
	if (!zcomp_available_algorithm(compress))
	if (!zcomp_available_algorithm(alg))
		return ERR_PTR(-EINVAL);

	comp = kzalloc(sizeof(struct zcomp), GFP_KERNEL);
	if (!comp)
		return ERR_PTR(-ENOMEM);

	comp->name = compress;
	comp->name = alg;
	error = zcomp_init(comp);
	if (error) {
		kfree(comp);
+1 −1
Original line number Diff line number Diff line
@@ -27,7 +27,7 @@ int zcomp_cpu_dead(unsigned int cpu, struct hlist_node *node);
ssize_t zcomp_available_show(const char *comp, char *buf);
bool zcomp_available_algorithm(const char *comp);

struct zcomp *zcomp_create(const char *comp);
struct zcomp *zcomp_create(const char *alg);
void zcomp_destroy(struct zcomp *comp);

struct zcomp_strm *zcomp_stream_get(struct zcomp *comp);
+530 −64

File changed.

Preview size limit exceeded, changes collapsed.

Loading