Loading Documentation/x86/sgx.rst +41 −0 Original line number Diff line number Diff line Loading @@ -209,3 +209,44 @@ An application may be loaded into a container enclave which is specially configured with a library OS and run-time which permits the application to run. The enclave run-time and library OS work together to execute the application when a thread enters the enclave. Impact of Potential Kernel SGX Bugs =================================== EPC leaks --------- When EPC page leaks happen, a WARNING like this is shown in dmesg: "EREMOVE returned ... and an EPC page was leaked. SGX may become unusable..." This is effectively a kernel use-after-free of an EPC page, and due to the way SGX works, the bug is detected at freeing. Rather than adding the page back to the pool of available EPC pages, the kernel intentionally leaks the page to avoid additional errors in the future. When this happens, the kernel will likely soon leak more EPC pages, and SGX will likely become unusable because the memory available to SGX is limited. However, while this may be fatal to SGX, the rest of the kernel is unlikely to be impacted and should continue to work. As a result, when this happpens, user should stop running any new SGX workloads, (or just any new workloads), and migrate all valuable workloads. Although a machine reboot can recover all EPC memory, the bug should be reported to Linux developers. Virtual EPC =========== The implementation has also a virtual EPC driver to support SGX enclaves in guests. Unlike the SGX driver, an EPC page allocated by the virtual EPC driver doesn't have a specific enclave associated with it. This is because KVM doesn't track how a guest uses EPC pages. As a result, the SGX core page reclaimer doesn't support reclaiming EPC pages allocated to KVM guests through the virtual EPC driver. If the user wants to deploy SGX applications both on the host and in guests on the same machine, the user should reserve enough EPC (by taking out total virtual EPC size of all SGX VMs from the physical EPC size) for host SGX applications so they can run with acceptable performance. MAINTAINERS +1 −0 Original line number Diff line number Diff line Loading @@ -9274,6 +9274,7 @@ Q: https://patchwork.kernel.org/project/intel-sgx/list/ T: git git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git x86/sgx F: Documentation/x86/sgx.rst F: arch/x86/entry/vdso/vsgx.S F: arch/x86/include/asm/sgx.h F: arch/x86/include/uapi/asm/sgx.h F: arch/x86/kernel/cpu/sgx/* F: tools/testing/selftests/sgx/* Loading arch/x86/Kconfig +1 −0 Original line number Diff line number Diff line Loading @@ -1931,6 +1931,7 @@ config X86_SGX depends on CRYPTO_SHA256=y select SRCU select MMU_NOTIFIER select NUMA_KEEP_MEMINFO if NUMA help Intel(R) Software Guard eXtensions (SGX) is a set of CPU instructions that can be used by applications to set aside private regions of code Loading arch/x86/include/asm/cpufeatures.h +2 −0 Original line number Diff line number Diff line Loading @@ -290,6 +290,8 @@ #define X86_FEATURE_FENCE_SWAPGS_KERNEL (11*32+ 5) /* "" LFENCE in kernel entry SWAPGS path */ #define X86_FEATURE_SPLIT_LOCK_DETECT (11*32+ 6) /* #AC for split lock */ #define X86_FEATURE_PER_THREAD_MBA (11*32+ 7) /* "" Per-thread Memory Bandwidth Allocation */ #define X86_FEATURE_SGX1 (11*32+ 8) /* "" Basic SGX */ #define X86_FEATURE_SGX2 (11*32+ 9) /* "" SGX Enclave Dynamic Memory Management (EDMM) */ /* Intel-defined CPU features, CPUID level 0x00000007:1 (EAX), word 12 */ #define X86_FEATURE_AVX_VNNI (12*32+ 4) /* AVX VNNI instructions */ Loading arch/x86/kernel/cpu/sgx/arch.h→arch/x86/include/asm/sgx.h +45 −5 Original line number Diff line number Diff line Loading @@ -2,15 +2,20 @@ /** * Copyright(c) 2016-20 Intel Corporation. * * Contains data structures defined by the SGX architecture. Data structures * defined by the Linux software stack should not be placed here. * Intel Software Guard Extensions (SGX) support. */ #ifndef _ASM_X86_SGX_ARCH_H #define _ASM_X86_SGX_ARCH_H #ifndef _ASM_X86_SGX_H #define _ASM_X86_SGX_H #include <linux/bits.h> #include <linux/types.h> /* * This file contains both data structures defined by SGX architecture and Linux * defined software data structures and functions. The two should not be mixed * together for better readibility. The architectural definitions come first. */ /* The SGX specific CPUID function. */ #define SGX_CPUID 0x12 /* EPC enumeration. */ Loading @@ -22,16 +27,36 @@ /* The bitmask for the EPC section type. */ #define SGX_CPUID_EPC_MASK GENMASK(3, 0) enum sgx_encls_function { ECREATE = 0x00, EADD = 0x01, EINIT = 0x02, EREMOVE = 0x03, EDGBRD = 0x04, EDGBWR = 0x05, EEXTEND = 0x06, ELDU = 0x08, EBLOCK = 0x09, EPA = 0x0A, EWB = 0x0B, ETRACK = 0x0C, EAUG = 0x0D, EMODPR = 0x0E, EMODT = 0x0F, }; /** * enum sgx_return_code - The return code type for ENCLS, ENCLU and ENCLV * %SGX_NOT_TRACKED: Previous ETRACK's shootdown sequence has not * been completed yet. * %SGX_CHILD_PRESENT SECS has child pages present in the EPC. * %SGX_INVALID_EINITTOKEN: EINITTOKEN is invalid and enclave signer's * public key does not match IA32_SGXLEPUBKEYHASH. * %SGX_UNMASKED_EVENT: An unmasked event, e.g. INTR, was received */ enum sgx_return_code { SGX_NOT_TRACKED = 11, SGX_CHILD_PRESENT = 13, SGX_INVALID_EINITTOKEN = 16, SGX_UNMASKED_EVENT = 128, }; Loading Loading @@ -335,4 +360,19 @@ struct sgx_sigstruct { #define SGX_LAUNCH_TOKEN_SIZE 304 #endif /* _ASM_X86_SGX_ARCH_H */ /* * Do not put any hardware-defined SGX structure representations below this * comment! */ #ifdef CONFIG_X86_SGX_KVM int sgx_virt_ecreate(struct sgx_pageinfo *pageinfo, void __user *secs, int *trapnr); int sgx_virt_einit(void __user *sigstruct, void __user *token, void __user *secs, u64 *lepubkeyhash, int *trapnr); #endif int sgx_set_attribute(unsigned long *allowed_attributes, unsigned int attribute_fd); #endif /* _ASM_X86_SGX_H */ Loading
Documentation/x86/sgx.rst +41 −0 Original line number Diff line number Diff line Loading @@ -209,3 +209,44 @@ An application may be loaded into a container enclave which is specially configured with a library OS and run-time which permits the application to run. The enclave run-time and library OS work together to execute the application when a thread enters the enclave. Impact of Potential Kernel SGX Bugs =================================== EPC leaks --------- When EPC page leaks happen, a WARNING like this is shown in dmesg: "EREMOVE returned ... and an EPC page was leaked. SGX may become unusable..." This is effectively a kernel use-after-free of an EPC page, and due to the way SGX works, the bug is detected at freeing. Rather than adding the page back to the pool of available EPC pages, the kernel intentionally leaks the page to avoid additional errors in the future. When this happens, the kernel will likely soon leak more EPC pages, and SGX will likely become unusable because the memory available to SGX is limited. However, while this may be fatal to SGX, the rest of the kernel is unlikely to be impacted and should continue to work. As a result, when this happpens, user should stop running any new SGX workloads, (or just any new workloads), and migrate all valuable workloads. Although a machine reboot can recover all EPC memory, the bug should be reported to Linux developers. Virtual EPC =========== The implementation has also a virtual EPC driver to support SGX enclaves in guests. Unlike the SGX driver, an EPC page allocated by the virtual EPC driver doesn't have a specific enclave associated with it. This is because KVM doesn't track how a guest uses EPC pages. As a result, the SGX core page reclaimer doesn't support reclaiming EPC pages allocated to KVM guests through the virtual EPC driver. If the user wants to deploy SGX applications both on the host and in guests on the same machine, the user should reserve enough EPC (by taking out total virtual EPC size of all SGX VMs from the physical EPC size) for host SGX applications so they can run with acceptable performance.
MAINTAINERS +1 −0 Original line number Diff line number Diff line Loading @@ -9274,6 +9274,7 @@ Q: https://patchwork.kernel.org/project/intel-sgx/list/ T: git git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git x86/sgx F: Documentation/x86/sgx.rst F: arch/x86/entry/vdso/vsgx.S F: arch/x86/include/asm/sgx.h F: arch/x86/include/uapi/asm/sgx.h F: arch/x86/kernel/cpu/sgx/* F: tools/testing/selftests/sgx/* Loading
arch/x86/Kconfig +1 −0 Original line number Diff line number Diff line Loading @@ -1931,6 +1931,7 @@ config X86_SGX depends on CRYPTO_SHA256=y select SRCU select MMU_NOTIFIER select NUMA_KEEP_MEMINFO if NUMA help Intel(R) Software Guard eXtensions (SGX) is a set of CPU instructions that can be used by applications to set aside private regions of code Loading
arch/x86/include/asm/cpufeatures.h +2 −0 Original line number Diff line number Diff line Loading @@ -290,6 +290,8 @@ #define X86_FEATURE_FENCE_SWAPGS_KERNEL (11*32+ 5) /* "" LFENCE in kernel entry SWAPGS path */ #define X86_FEATURE_SPLIT_LOCK_DETECT (11*32+ 6) /* #AC for split lock */ #define X86_FEATURE_PER_THREAD_MBA (11*32+ 7) /* "" Per-thread Memory Bandwidth Allocation */ #define X86_FEATURE_SGX1 (11*32+ 8) /* "" Basic SGX */ #define X86_FEATURE_SGX2 (11*32+ 9) /* "" SGX Enclave Dynamic Memory Management (EDMM) */ /* Intel-defined CPU features, CPUID level 0x00000007:1 (EAX), word 12 */ #define X86_FEATURE_AVX_VNNI (12*32+ 4) /* AVX VNNI instructions */ Loading
arch/x86/kernel/cpu/sgx/arch.h→arch/x86/include/asm/sgx.h +45 −5 Original line number Diff line number Diff line Loading @@ -2,15 +2,20 @@ /** * Copyright(c) 2016-20 Intel Corporation. * * Contains data structures defined by the SGX architecture. Data structures * defined by the Linux software stack should not be placed here. * Intel Software Guard Extensions (SGX) support. */ #ifndef _ASM_X86_SGX_ARCH_H #define _ASM_X86_SGX_ARCH_H #ifndef _ASM_X86_SGX_H #define _ASM_X86_SGX_H #include <linux/bits.h> #include <linux/types.h> /* * This file contains both data structures defined by SGX architecture and Linux * defined software data structures and functions. The two should not be mixed * together for better readibility. The architectural definitions come first. */ /* The SGX specific CPUID function. */ #define SGX_CPUID 0x12 /* EPC enumeration. */ Loading @@ -22,16 +27,36 @@ /* The bitmask for the EPC section type. */ #define SGX_CPUID_EPC_MASK GENMASK(3, 0) enum sgx_encls_function { ECREATE = 0x00, EADD = 0x01, EINIT = 0x02, EREMOVE = 0x03, EDGBRD = 0x04, EDGBWR = 0x05, EEXTEND = 0x06, ELDU = 0x08, EBLOCK = 0x09, EPA = 0x0A, EWB = 0x0B, ETRACK = 0x0C, EAUG = 0x0D, EMODPR = 0x0E, EMODT = 0x0F, }; /** * enum sgx_return_code - The return code type for ENCLS, ENCLU and ENCLV * %SGX_NOT_TRACKED: Previous ETRACK's shootdown sequence has not * been completed yet. * %SGX_CHILD_PRESENT SECS has child pages present in the EPC. * %SGX_INVALID_EINITTOKEN: EINITTOKEN is invalid and enclave signer's * public key does not match IA32_SGXLEPUBKEYHASH. * %SGX_UNMASKED_EVENT: An unmasked event, e.g. INTR, was received */ enum sgx_return_code { SGX_NOT_TRACKED = 11, SGX_CHILD_PRESENT = 13, SGX_INVALID_EINITTOKEN = 16, SGX_UNMASKED_EVENT = 128, }; Loading Loading @@ -335,4 +360,19 @@ struct sgx_sigstruct { #define SGX_LAUNCH_TOKEN_SIZE 304 #endif /* _ASM_X86_SGX_ARCH_H */ /* * Do not put any hardware-defined SGX structure representations below this * comment! */ #ifdef CONFIG_X86_SGX_KVM int sgx_virt_ecreate(struct sgx_pageinfo *pageinfo, void __user *secs, int *trapnr); int sgx_virt_einit(void __user *sigstruct, void __user *token, void __user *secs, u64 *lepubkeyhash, int *trapnr); #endif int sgx_set_attribute(unsigned long *allowed_attributes, unsigned int attribute_fd); #endif /* _ASM_X86_SGX_H */
