Commit df4e817b authored by Pasha Tatashin's avatar Pasha Tatashin Committed by Linus Torvalds
Browse files

mm: page table check 

Check user page table entries at the time they are added and removed.

Allows to synchronously catch memory corruption issues related to double
mapping.

When a pte for an anonymous page is added into page table, we verify
that this pte does not already point to a file backed page, and vice
versa if this is a file backed page that is being added we verify that
this page does not have an anonymous mapping

We also enforce that read-only sharing for anonymous pages is allowed
(i.e.  cow after fork).  All other sharing must be for file pages.

Page table check allows to protect and debug cases where "struct page"
metadata became corrupted for some reason.  For example, when refcnt or
mapcount become invalid.

Link: https://lkml.kernel.org/r/20211221154650.1047963-4-pasha.tatashin@soleen.com


Signed-off-by: default avatarPasha Tatashin <pasha.tatashin@soleen.com>
Cc: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: David Rientjes <rientjes@google.com>
Cc: Frederic Weisbecker <frederic@kernel.org>
Cc: Greg Thelen <gthelen@google.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Hugh Dickins <hughd@google.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Jiri Slaby <jirislaby@kernel.org>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Kees Cook <keescook@chromium.org>
Cc: Masahiro Yamada <masahiroy@kernel.org>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Muchun Song <songmuchun@bytedance.com>
Cc: Paul Turner <pjt@google.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Sami Tolvanen <samitolvanen@google.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Wei Xu <weixugc@google.com>
Cc: Will Deacon <will@kernel.org>
Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
parent 08d5b29e

Documentation/vm/index.rst

+1 −0
Original line number Diff line number Diff line
@@ -31,6 +31,7 @@ algorithms. If you are looking for advice on simply allocating memory, see the 
   page_migration

   page_frags

   page_owner

   page_table_check

   remap_file_pages

   slub

   split_page_table_lock

Documentation/vm/page_table_check.rst

0 → 100644
+56 −0
Original line number Diff line number Diff line 
.. SPDX-License-Identifier: GPL-2.0



.. _page_table_check:



================

Page Table Check

================



Introduction

============



Page table check allows to hardern the kernel by ensuring that some types of

the memory corruptions are prevented.



Page table check performs extra verifications at the time when new pages become

accessible from the userspace by getting their page table entries (PTEs PMDs

etc.) added into the table.



In case of detected corruption, the kernel is crashed. There is a small

performance and memory overhead associated with the page table check. Therefore,

it is disabled by default, but can be optionally enabled on systems where the

extra hardening outweighs the performance costs. Also, because page table check

is synchronous, it can help with debugging double map memory corruption issues,

by crashing kernel at the time wrong mapping occurs instead of later which is

often the case with memory corruptions bugs.



Double mapping detection logic

==============================



+-------------------+-------------------+-------------------+------------------+

| Current Mapping   | New mapping       | Permissions       | Rule             |

+===================+===================+===================+==================+

| Anonymous         | Anonymous         | Read              | Allow            |

+-------------------+-------------------+-------------------+------------------+

| Anonymous         | Anonymous         | Read / Write      | Prohibit         |

+-------------------+-------------------+-------------------+------------------+

| Anonymous         | Named             | Any               | Prohibit         |

+-------------------+-------------------+-------------------+------------------+

| Named             | Anonymous         | Any               | Prohibit         |

+-------------------+-------------------+-------------------+------------------+

| Named             | Named             | Any               | Allow            |

+-------------------+-------------------+-------------------+------------------+



Enabling Page Table Check

=========================



Build kernel with:



- PAGE_TABLE_CHECK=y

  Note, it can only be enabled on platforms where ARCH_SUPPORTS_PAGE_TABLE_CHECK

  is available.



- Boot with 'page_table_check=on' kernel parameter.



Optionally, build kernel with PAGE_TABLE_CHECK_ENFORCED in order to have page

table support without extra kernel parameter.

MAINTAINERS

+9 −0
Original line number Diff line number Diff line
@@ -14387,6 +14387,15 @@ F: include/net/page_pool.h 
F:	include/trace/events/page_pool.h

F:	net/core/page_pool.c









PAGE TABLE CHECK









M:	Pasha Tatashin <pasha.tatashin@soleen.com>









M:	Andrew Morton <akpm@linux-foundation.org>









L:	linux-mm@kvack.org









S:	Maintained









F:	Documentation/vm/page_table_check.rst









F:	include/linux/page_table_check.h









F:	mm/page_table_check.c

















PANASONIC LAPTOP ACPI EXTRAS DRIVER









M:	Kenneth Chan <kenneth.t.chan@gmail.com>









L:	platform-driver-x86@vger.kernel.org
































arch/Kconfig



+3
−0






























Original line number
Diff line number
Diff line







@@ -1297,6 +1297,9 @@ config HAVE_ARCH_PFN_VALID







config ARCH_SUPPORTS_DEBUG_PAGEALLOC









	bool



















config ARCH_SUPPORTS_PAGE_TABLE_CHECK









	bool



















config ARCH_SPLIT_ARG64









	bool









	help
































include/linux/page_table_check.h
0 → 100644



+147
−0






























Original line number
Diff line number
Diff line









/* SPDX-License-Identifier: GPL-2.0 */



















/*









 * Copyright (c) 2021, Google LLC.









 * Pasha Tatashin <pasha.tatashin@soleen.com>









 */









#ifndef __LINUX_PAGE_TABLE_CHECK_H









#define __LINUX_PAGE_TABLE_CHECK_H



















#ifdef CONFIG_PAGE_TABLE_CHECK









#include <linux/jump_label.h>



















extern struct static_key_true page_table_check_disabled;









extern struct page_ext_operations page_table_check_ops;



















void __page_table_check_zero(struct page *page, unsigned int order);









void __page_table_check_pte_clear(struct mm_struct *mm, unsigned long addr,









				  pte_t pte);









void __page_table_check_pmd_clear(struct mm_struct *mm, unsigned long addr,









				  pmd_t pmd);









void __page_table_check_pud_clear(struct mm_struct *mm, unsigned long addr,









				  pud_t pud);









void __page_table_check_pte_set(struct mm_struct *mm, unsigned long addr,









				pte_t *ptep, pte_t pte);









void __page_table_check_pmd_set(struct mm_struct *mm, unsigned long addr,









				pmd_t *pmdp, pmd_t pmd);









void __page_table_check_pud_set(struct mm_struct *mm, unsigned long addr,









				pud_t *pudp, pud_t pud);



















static inline void page_table_check_alloc(struct page *page, unsigned int order)









{









	if (static_branch_likely(&page_table_check_disabled))









		return;



















	__page_table_check_zero(page, order);









}



















static inline void page_table_check_free(struct page *page, unsigned int order)









{









	if (static_branch_likely(&page_table_check_disabled))









		return;



















	__page_table_check_zero(page, order);









}



















static inline void page_table_check_pte_clear(struct mm_struct *mm,









					      unsigned long addr, pte_t pte)









{









	if (static_branch_likely(&page_table_check_disabled))









		return;



















	__page_table_check_pte_clear(mm, addr, pte);









}



















static inline void page_table_check_pmd_clear(struct mm_struct *mm,









					      unsigned long addr, pmd_t pmd)









{









	if (static_branch_likely(&page_table_check_disabled))









		return;



















	__page_table_check_pmd_clear(mm, addr, pmd);









}



















static inline void page_table_check_pud_clear(struct mm_struct *mm,









					      unsigned long addr, pud_t pud)









{









	if (static_branch_likely(&page_table_check_disabled))









		return;



















	__page_table_check_pud_clear(mm, addr, pud);









}



















static inline void page_table_check_pte_set(struct mm_struct *mm,









					    unsigned long addr, pte_t *ptep,









					    pte_t pte)









{









	if (static_branch_likely(&page_table_check_disabled))









		return;



















	__page_table_check_pte_set(mm, addr, ptep, pte);









}



















static inline void page_table_check_pmd_set(struct mm_struct *mm,









					    unsigned long addr, pmd_t *pmdp,









					    pmd_t pmd)









{









	if (static_branch_likely(&page_table_check_disabled))









		return;



















	__page_table_check_pmd_set(mm, addr, pmdp, pmd);









}



















static inline void page_table_check_pud_set(struct mm_struct *mm,









					    unsigned long addr, pud_t *pudp,









					    pud_t pud)









{









	if (static_branch_likely(&page_table_check_disabled))









		return;



















	__page_table_check_pud_set(mm, addr, pudp, pud);









}



















#else



















static inline void page_table_check_alloc(struct page *page, unsigned int order)









{









}



















static inline void page_table_check_free(struct page *page, unsigned int order)









{









}



















static inline void page_table_check_pte_clear(struct mm_struct *mm,









					      unsigned long addr, pte_t pte)









{









}



















static inline void page_table_check_pmd_clear(struct mm_struct *mm,









					      unsigned long addr, pmd_t pmd)









{









}



















static inline void page_table_check_pud_clear(struct mm_struct *mm,









					      unsigned long addr, pud_t pud)









{









}



















static inline void page_table_check_pte_set(struct mm_struct *mm,









					    unsigned long addr, pte_t *ptep,









					    pte_t pte)









{









}



















static inline void page_table_check_pmd_set(struct mm_struct *mm,









					    unsigned long addr, pmd_t *pmdp,









					    pmd_t pmd)









{









}



















static inline void page_table_check_pud_set(struct mm_struct *mm,









					    unsigned long addr, pud_t *pudp,









					    pud_t pud)









{









}



















#endif /* CONFIG_PAGE_TABLE_CHECK */









#endif /* __LINUX_PAGE_TABLE_CHECK_H */