mm/dynamic_hugetlb: add Document for dynamic hugetlb feature

.. _dynamic_hugetlb:

===============

Dynamic Hugetlb

===============

Overview

========

Dynamic hugetlb is a self-developed feature based on the hugetlb and memcontrol.

It supports to split huge page dynamically in a memory cgroup. There is a new structure

dhugetlb_pool in every mem_cgroup to manage the pages configured to the mem_cgroup.

For the mem_cgroup configured with dhugetlb_pool, processes in the mem_cgroup will

preferentially use the pages in dhugetlb_pool.

Dynamic hugetlb supports three types of pages, including 1G/2M huge pages and 4K pages.

For the mem_cgroup configured with dhugetlb_pool, processes will be limited to alloc

1G/2M huge pages only from dhugetlb_pool. But there is no such constraint for 4K pages.

If there are insufficient 4K pages in the dhugetlb_pool, pages can also be allocated from

buddy system. So before using dynamic hugetlb, user must know how many huge pages they

need.

Conflict

========

1. Conflict with THP

--------------------

When THP is enabled, the allocation of a page(order=0) may be converted to

an allocation of pages(order>0). In this case, the allocation will skip the

dhugetlb_pool. When we want to use dynamic hugetlb feature, we have to

disable THP for now.

2. Conflict with hugetlb_vmemmap

--------------------------------

The dynamic_hugetlb feature need to split and merge pages frequently.

hugetlb_vmemmap will affects the perforemance of page split and merge.

If want to use dynamic hugetlb, please disable hugetlb_vmemmap.

Usage

=====

1) Add 'dynamic_hugetlb=on' in cmdline to enable dynamic hugetlb feature.

2) Prealloc some 1G hugepages through hugetlb.

3) Create a mem_cgroup and configure dhugetlb_pool to mem_cgroup.

4) Configure the count of 1G/2M hugepages, and the remaining pages in dhugetlb_pool will

   be used as basic pages.

5) Bound the process to mem_cgroup. then the memory for it will be allocated from dhugetlb_pool.

User control

============

1. dynamic_hugetlb=

-------------------

Add ``dynamic_hugtlb=on`` in cmdline to enable dynamic hugetlb feature.

By default, the feature si disabled.

2. dhugetlb.nr_pages

--------------------

In each memory cgroup, there is a ``dhugetlb.nr_pages`` interface used to create and configure dynamic

hugetlb. If this interface is not configured, the original functions are not affected. If configured,

then the memory used by processes in this memory cgroup will be allocated from corresponding hpool.

Usage:

	``echo <nid> <nr_pages> > /sys/fs/cgroup/memory/<memory cgroup>/dhugetlb.nr_pages``:

	Create a dynamic hugetlb pool and add <nr_pages> 1G hugepages from numa node <nid> to the pool.

	``cat /sys/fs/cgroup/memory/<memory cgroup>/dhugetlb.nr_pages``:

	Reads the memory information in the hpool, include the free amount and used amount of huge pages and

	normal pages.

3. dhugetlb.1G.reserved_pages

-----------------------------

In each memory cgroup, there is a ``dhugetlb.nr_pages`` interface used to reserved 1G huge pages.

By default, all memory configured to a dynamic hugetlb pool can be used only as normal pages, if want to use

it as 1G huge pages, need to configure the number of 1G huge pages by this interface firstly.

Usage:

	``echo <nr_pages> > /sys/fs/cgroup/memory/<memory cgroup>/dhugetlb.1G.reserved_pages``

4. dhugetlb.2M.reserved_pages

-----------------------------

Similar to the previous interface, this is used to configure the number of 2M huge pages.

Usage:

	``echo <nr_pages> > /sys/fs/cgroup/memory/<memory cgroup>/dhugetlb.2M.reserved_pages``

5. dhugetlb.normal_pages_disabled

---------------------------------

The dynamic hugetlb pool can be used as just a huge pages pool. This interface is used to disable allocation

of normal pages from the dynamic hugetlb pool.

Usage:

	``echo 1 > /sys/fs/cgroup/memory/<memory cgroup>/dhugetlb.normal_pages_disabled``

---

Liu Shixin, Jan 2022