Skip to content
  1. Dec 04, 2017
  3. Nov 17, 2017
  5. Nov 16, 2017
  7. Nov 15, 2017
  9. Nov 14, 2017
    • Dave Martin's avatar
      arm64: Make ARMV8_DEPRECATED depend on SYSCTL · 6cfa7cc4
      Dave Martin authored 
      

If CONFIG_SYSCTL=n and CONFIG_ARMV8_DEPRECATED=y, the deprecated
instruction emulation code currently leaks some memory at boot
time, and won't have any runtime control interface.  This does
not feel like useful or intended behaviour...

This patch adds a dependency on CONFIG_SYSCTL, so that such a
kernel can't be built in the first place.

It's probably not worth adding the error-handling / cleanup code
that would be needed to deal with this otherwise: people who
desperately need the emulation can still enable SYSCTL.

Acked-by: default avatarArnd Bergmann <arnd@arndb.de>
Signed-off-by: default avatarDave Martin <Dave.Martin@arm.com>
Signed-off-by: default avatarWill Deacon <will.deacon@arm.com>
      6cfa7cc4
    • Jason A. Donenfeld's avatar
      arm64: Implement __lshrti3 library function · 9bfe7553
      Jason A. Donenfeld authored 
      Commit fb872273

 ("arm64: support __int128 on gcc 5+") added support
for the __int128 data type, but this breaks the build in some configurations
where GCC ends up emitting calls to the __lshrti3 helper in libgcc, which
results in a link error:

  kernel/sched/fair.o: In function `__calc_delta':
  fair.c:(.text+0xca0): undefined reference to `__lshrti3'
  kernel/time/timekeeping.o: In function `timekeeping_resume':
  timekeeping.c:(.text+0x3f60): undefined reference to `__lshrti3'
  make: *** [vmlinux] Error 1

Fix the build by providing an implementation of __lshrti3, like we do
already for __ashlti3 and __ashrti3.

Reported-by: default avatarArnd Bergmann <arnd@arndb.de>
Signed-off-by: default avatarJason A. Donenfeld <Jason@zx2c4.com>
Signed-off-by: default avatarWill Deacon <will.deacon@arm.com>
      9bfe7553
  11. Nov 10, 2017
    • Marc Zyngier's avatar
      KVM: arm/arm64: GICv4: Add init/teardown of the per-VM vPE irq domain · 74fe55dc
      Marc Zyngier authored 
      

In order to control the GICv4 view of virtual CPUs, we rely
on an irqdomain allocated for that purpose. Let's add a couple
of helpers to that effect.

At the same time, the vgic data structures gain new fields to
track all this... erm... wonderful stuff.

The way we hook into the vgic init is slightly convoluted. We
need the vgic to be initialized (in order to guarantee that
the number of vcpus is now fixed), and we must have a vITS
(otherwise this is all very pointless). So we end-up calling
the init from both vgic_init and vgic_its_create.

Reviewed-by: default avatarChristoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: default avatarMarc Zyngier <marc.zyngier@arm.com>
Signed-off-by: default avatarChristoffer Dall <christoffer.dall@linaro.org>
      74fe55dc
    • Masahiro Yamada's avatar
      kbuild: handle dtb-y and CONFIG_OF_ALL_DTBS natively in Makefile.lib · 7e7962dd
      Masahiro Yamada authored 
      

If CONFIG_OF_ALL_DTBS is enabled, "make ARCH=arm64 dtbs" compiles each
DTB twice; one from arch/arm64/boot/dts/*/Makefile and the other from
the dtb-$(CONFIG_OF_ALL_DTBS) line in arch/arm64/boot/dts/Makefile.
It could be a race problem when building DTBS in parallel.

Another minor issue is CONFIG_OF_ALL_DTBS covers only *.dts in vendor
sub-directories, so this broke when Broadcom added one more hierarchy
in arch/arm64/boot/dts/broadcom/<soc>/.

One idea to fix the issues in a clean way is to move DTB handling
to Kbuild core scripts.  Makefile.dtbinst already recognizes dtb-y
natively, so it should not hurt to do so.

Add $(dtb-y) to extra-y, and $(dtb-) as well if CONFIG_OF_ALL_DTBS is
enabled.  All clutter things in Makefiles go away.

As a bonus clean-up, I also removed dts-dirs.  Just use subdir-y
directly to traverse sub-directories.

Signed-off-by: default avatarMasahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: default avatarArnd Bergmann <arnd@arndb.de>
[robh: corrected BUILTIN_DTB to CONFIG_BUILTIN_DTB]
Signed-off-by: default avatarRob Herring <robh@kernel.org>
      7e7962dd
  13. Nov 09, 2017
  15. Nov 07, 2017
  17. Nov 06, 2017
  19. Nov 04, 2017
  21. Nov 03, 2017
    • Jason A. Donenfeld's avatar
      arm64: support __int128 on gcc 5+ · fb872273
      Jason A. Donenfeld authored 
      Versions of gcc prior to gcc 5 emitted a __multi3 function call when
dealing with TI types, resulting in failures when trying to link to
libgcc, and more generally, bad performance. However, since gcc 5,
the compiler supports actually emitting fast instructions, which means
we can at long last enable this option and receive the speedups.

The gcc commit that added proper Aarch64 support is:
https://gcc.gnu.org/git/?p=gcc.git;a=commitdiff;h=d1ae7bb994f49316f6f63e6173f2931e837a351d


This commit appears to be part of the gcc 5 release.

There are still a few instructions, __ashlti3 and __ashrti3, which
require libgcc, which is fine. Rather than linking to libgcc, we
simply provide them ourselves, since they're not that complicated.

Signed-off-by: default avatarJason A. Donenfeld <Jason@zx2c4.com>
Signed-off-by: default avatarWill Deacon <will.deacon@arm.com>
      fb872273
    • Dave Martin's avatar
      arm64/sve: Add documentation · ce699081
      Dave Martin authored 
      

This patch adds basic documentation of the user/kernel interface
provided by the for SVE.

Signed-off-by: default avatarDave Martin <Dave.Martin@arm.com>
Reviewed-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
Cc: Alan Hayward <alan.hayward@arm.com>
Cc: Alex Bennée <alex.bennee@linaro.org>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Michael Kerrisk <mtk.manpages@gmail.com>
Cc: Szabolcs Nagy <szabolcs.nagy@arm.com>
Cc: linux-api@vger.kernel.org
Signed-off-by: default avatarWill Deacon <will.deacon@arm.com>
      ce699081
    • Dave Martin's avatar
      arm64/sve: Detect SVE and activate runtime support · 43994d82
      Dave Martin authored 
      

This patch enables detection of hardware SVE support via the
cpufeatures framework, and reports its presence to the kernel and
userspace via the new ARM64_SVE cpucap and HWCAP_SVE hwcap
respectively.

Userspace can also detect SVE using ID_AA64PFR0_EL1, using the
cpufeatures MRS emulation.

When running on hardware that supports SVE, this enables runtime
kernel support for SVE, and allows user tasks to execute SVE
instructions and make of the of the SVE-specific user/kernel
interface extensions implemented by this series.

Signed-off-by: default avatarDave Martin <Dave.Martin@arm.com>
Reviewed-by: default avatarSuzuki K Poulose <suzuki.poulose@arm.com>
Reviewed-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: default avatarWill Deacon <will.deacon@arm.com>
      43994d82
    • Dave Martin's avatar
      arm64/sve: KVM: Hide SVE from CPU features exposed to guests · 07d79fe7
      Dave Martin authored 
      

KVM guests cannot currently use SVE, because SVE is always
configured to trap to EL2.

However, a guest that sees SVE reported as present in
ID_AA64PFR0_EL1 may legitimately expect that SVE works and try to
use it.  Instead of working, the guest will receive an injected
undef exception, which may cause the guest to oops or go into a
spin.

To avoid misleading the guest into believing that SVE will work,
this patch masks out the SVE field from ID_AA64PFR0_EL1 when a
guest attempts to read this register.  No support is explicitly
added for ID_AA64ZFR0_EL1 either, so that is still emulated as
reading as zero, which is consistent with SVE not being
implemented.

This is a temporary measure, and will be removed in a later series
when full KVM support for SVE is implemented.

Signed-off-by: default avatarDave Martin <Dave.Martin@arm.com>
Reviewed-by: default avatarAlex Bennée <alex.bennee@linaro.org>
Acked-by: default avatarMarc Zyngier <marc.zyngier@arm.com>
Acked-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
Acked-by: default avatarChristoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: default avatarWill Deacon <will.deacon@arm.com>
      07d79fe7
    • Dave Martin's avatar
      arm64/sve: KVM: Treat guest SVE use as undefined instruction execution · aac45ffd
      Dave Martin authored 
      

When trapping forbidden attempts by a guest to use SVE, we want the
guest to see a trap consistent with SVE not being implemented.

This patch injects an undefined instruction exception into the
guest in response to such an exception.

Signed-off-by: default avatarDave Martin <Dave.Martin@arm.com>
Reviewed-by: default avatarAlex Bennée <alex.bennee@linaro.org>
Acked-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
Acked-by: default avatarChristoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: default avatarWill Deacon <will.deacon@arm.com>
      aac45ffd
    • Dave Martin's avatar
      arm64/sve: KVM: Prevent guests from using SVE · 17eed27b
      Dave Martin authored 
      

Until KVM has full SVE support, guests must not be allowed to
execute SVE instructions.

This patch enables the necessary traps, and also ensures that the
traps are disabled again on exit from the guest so that the host
can still use SVE if it wants to.

On guest exit, high bits of the SVE Zn registers may have been
clobbered as a side-effect the execution of FPSIMD instructions in
the guest.  The existing KVM host FPSIMD restore code is not
sufficient to restore these bits, so this patch explicitly marks
the CPU as not containing cached vector state for any task, thus
forcing a reload on the next return to userspace.  This is an
interim measure, in advance of adding full SVE awareness to KVM.

This marking of cached vector state in the CPU as invalid is done
using __this_cpu_write(fpsimd_last_state, NULL) in fpsimd.c.  Due
to the repeated use of this rather obscure operation, it makes
sense to factor it out as a separate helper with a clearer name.
This patch factors it out as fpsimd_flush_cpu_state(), and ports
all callers to use it.

As a side effect of this refactoring, a this_cpu_write() in
fpsimd_cpu_pm_notifier() is changed to __this_cpu_write().  This
should be fine, since cpu_pm_enter() is supposed to be called only
with interrupts disabled.

Signed-off-by: default avatarDave Martin <Dave.Martin@arm.com>
Reviewed-by: default avatarAlex Bennée <alex.bennee@linaro.org>
Reviewed-by: default avatarChristoffer Dall <christoffer.dall@linaro.org>
Acked-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
Acked-by: default avatarMarc Zyngier <marc.zyngier@arm.com>
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Signed-off-by: default avatarWill Deacon <will.deacon@arm.com>
      17eed27b
    • Dave Martin's avatar
      arm64/sve: Add sysctl to set the default vector length for new processes · 4ffa09a9
      Dave Martin authored 
      

Because of the effect of SVE on the size of the signal frame, the
default vector length used for new processes involves a tradeoff
between performance of SVE-enabled software on the one hand, and
reliability of non-SVE-aware software on the other hand.

For this reason, the best choice depends on the repertoire of
userspace software in use and is thus best left up to distro
maintainers, sysadmins and developers.

If CONFIG_SYSCTL and CONFIG_PROC_SYSCTL are enabled, this patch
exposes the default vector length in
/proc/sys/abi/sve_default_vector_length, where boot scripts or the
adventurous can poke it.

In common with other arm64 ABI sysctls, this control is currently
global: setting it requires CAP_SYS_ADMIN in the root user
namespace, but the value set is effective for subsequent execs in
all namespaces.  The control only affects _new_ processes, however:
changing it does not affect the vector length of any existing
process.

The intended usage model is that if userspace is known to be fully
SVE-tolerant (or a developer is curious to find out) then this
parameter can be cranked up during system startup.

Signed-off-by: default avatarDave Martin <Dave.Martin@arm.com>
Reviewed-by: default avatarAlex Bennée <alex.bennee@linaro.org>
Reviewed-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: default avatarWill Deacon <will.deacon@arm.com>
      4ffa09a9
    • Dave Martin's avatar
      arm64/sve: Add prctl controls for userspace vector length management · 2d2123bc
      Dave Martin authored 
      

This patch adds two arm64-specific prctls, to permit userspace to
control its vector length:

 * PR_SVE_SET_VL: set the thread's SVE vector length and vector
   length inheritance mode.

 * PR_SVE_GET_VL: get the same information.

Although these prctls resemble instruction set features in the SVE
architecture, they provide additional control: the vector length
inheritance mode is Linux-specific and nothing to do with the
architecture, and the architecture does not permit EL0 to set its
own vector length directly.  Both can be used in portable tools
without requiring the use of SVE instructions.

Signed-off-by: default avatarDave Martin <Dave.Martin@arm.com>
Reviewed-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
Cc: Alex Bennée <alex.bennee@linaro.org>
[will: Fixed up prctl constants to avoid clash with PDEATHSIG]
Signed-off-by: default avatarWill Deacon <will.deacon@arm.com>
      2d2123bc
    • Dave Martin's avatar
      arm64/sve: ptrace and ELF coredump support · 43d4da2c
      Dave Martin authored 
      

This patch defines and implements a new regset NT_ARM_SVE, which
describes a thread's SVE register state.  This allows a debugger to
manipulate the SVE state, as well as being included in ELF
coredumps for post-mortem debugging.

Because the regset size and layout are dependent on the thread's
current vector length, it is not possible to define a C struct to
describe the regset contents as is done for existing regsets.
Instead, and for the same reasons, NT_ARM_SVE is based on the
freeform variable-layout approach used for the SVE signal frame.

Additionally, to reduce debug overhead when debugging threads that
might or might not have live SVE register state, NT_ARM_SVE may be
presented in one of two different formats: the old struct
user_fpsimd_state format is embedded for describing the state of a
thread with no live SVE state, whereas a new variable-layout
structure is embedded for describing live SVE state.  This avoids a
debugger needing to poll NT_PRFPREG in addition to NT_ARM_SVE, and
allows existing userspace code to handle the non-SVE case without
too much modification.

For this to work, NT_ARM_SVE is defined with a fixed-format header
of type struct user_sve_header, which the recipient can use to
figure out the content, size and layout of the reset of the regset.
Accessor macros are defined to allow the vector-length-dependent
parts of the regset to be manipulated.

Signed-off-by: default avatarAlan Hayward <alan.hayward@arm.com>
Signed-off-by: default avatarDave Martin <Dave.Martin@arm.com>
Reviewed-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
Cc: Alex Bennée <alex.bennee@linaro.org>
Cc: Okamoto Takayuki <tokamoto@jp.fujitsu.com>
Signed-off-by: default avatarWill Deacon <will.deacon@arm.com>
      43d4da2c
    • Dave Martin's avatar
      arm64/sve: Preserve SVE registers around EFI runtime service calls · fdfa976c
      Dave Martin authored 
      

The EFI runtime services ABI allows EFI to make free use of the
FPSIMD registers during EFI runtime service calls, subject to the
callee-save requirements of the AArch64 procedure call standard.

However, the SVE architecture allows upper bits of the SVE vector
registers to be zeroed as a side-effect of FPSIMD V-register
writes.  This means that the SVE vector registers must be saved in
their entirety in order to avoid data loss: non-SVE-aware EFI
implementations cannot restore them correctly.

The non-IRQ case is already handled gracefully by
kernel_neon_begin().  For the IRQ case, this patch allocates a
suitable per-CPU stash buffer for the full SVE register state and
uses it to preserve the affected registers around EFI calls.  It is
currently unclear how the EFI runtime services ABI will be
clarified with respect to SVE, so it safest to assume that the
predicate registers and FFR must be saved and restored too.

No attempt is made to restore the restore the vector length after
a call, for now.  It is deemed rather insane for EFI to change it,
and contemporary EFI implementations certainly won't.

Signed-off-by: default avatarDave Martin <Dave.Martin@arm.com>
Reviewed-by: default avatarAlex Bennée <alex.bennee@linaro.org>
Reviewed-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Signed-off-by: default avatarWill Deacon <will.deacon@arm.com>
      fdfa976c
    • Dave Martin's avatar
      arm64/sve: Preserve SVE registers around kernel-mode NEON use · 1bd3f936
      Dave Martin authored 
      

Kernel-mode NEON will corrupt the SVE vector registers, due to the
way they alias the FPSIMD vector registers in the hardware.

This patch ensures that any live SVE register content for the task
is saved by kernel_neon_begin().  The data will be restored in the
usual way on return to userspace.

Signed-off-by: default avatarDave Martin <Dave.Martin@arm.com>
Reviewed-by: default avatarArd Biesheuvel <ard.biesheuvel@linaro.org>
Reviewed-by: default avatarAlex Bennée <alex.bennee@linaro.org>
Reviewed-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: default avatarWill Deacon <will.deacon@arm.com>
      1bd3f936
    • Dave Martin's avatar
      arm64/sve: Probe SVE capabilities and usable vector lengths · 2e0f2478
      Dave Martin authored 
      

This patch uses the cpufeatures framework to determine common SVE
capabilities and vector lengths, and configures the runtime SVE
support code appropriately.

ZCR_ELx is not really a feature register, but it is convenient to
use it as a template for recording the maximum vector length
supported by a CPU, using the LEN field.  This field is similar to
a feature field in that it is a contiguous bitfield for which we
want to determine the minimum system-wide value.  This patch adds
ZCR as a pseudo-register in cpuinfo/cpufeatures, with appropriate
custom code to populate it.  Finding the minimum supported value of
the LEN field is left to the cpufeatures framework in the usual
way.

The meaning of ID_AA64ZFR0_EL1 is not architecturally defined yet,
so for now we just require it to be zero.

Note that much of this code is dormant and SVE still won't be used
yet, since system_supports_sve() remains hardwired to false.

Signed-off-by: default avatarDave Martin <Dave.Martin@arm.com>
Reviewed-by: default avatarSuzuki K Poulose <suzuki.poulose@arm.com>
Reviewed-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
Cc: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: default avatarWill Deacon <will.deacon@arm.com>
      2e0f2478
    • Dave Martin's avatar
      arm64: cpufeature: Move sys_caps_initialised declarations · 8f1eec57
      Dave Martin authored 
      

update_cpu_features() currently cannot tell whether it is being
called during early or late secondary boot.  This doesn't
desperately matter for anything it currently does.

However, SVE will need to know here whether the set of available
vector lengths is known or still to be determined when booting a
CPU, so that it can be updated appropriately.

This patch simply moves the sys_caps_initialised stuff to the top
of the file so that it can be used more widely.  There doesn't seem
to be a more obvious place to put it.

Signed-off-by: default avatarDave Martin <Dave.Martin@arm.com>
Reviewed-by: default avatarAlex Bennée <alex.bennee@linaro.org>
Reviewed-by: default avatarSuzuki K Poulose <suzuki.poulose@arm.com>
Acked-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: default avatarWill Deacon <will.deacon@arm.com>
      8f1eec57
    • Dave Martin's avatar
      arm64/sve: Backend logic for setting the vector length · 7582e220
      Dave Martin authored 
      

This patch implements the core logic for changing a task's vector
length on request from userspace.  This will be used by the ptrace
and prctl frontends that are implemented in later patches.

The SVE architecture permits, but does not require, implementations
to support vector lengths that are not a power of two.  To handle
this, logic is added to check a requested vector length against a
possibly sparse bitmap of available vector lengths at runtime, so
that the best supported value can be chosen.

Signed-off-by: default avatarDave Martin <Dave.Martin@arm.com>
Reviewed-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
Cc: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: default avatarWill Deacon <will.deacon@arm.com>
      7582e220
    • Dave Martin's avatar
      arm64/sve: Signal handling support · 8cd969d2
      Dave Martin authored 
      

This patch implements support for saving and restoring the SVE
registers around signals.

A fixed-size header struct sve_context is always included in the
signal frame encoding the thread's vector length at the time of
signal delivery, optionally followed by a variable-layout structure
encoding the SVE registers.

Because of the need to preserve backwards compatibility, the FPSIMD
view of the SVE registers is always dumped as a struct
fpsimd_context in the usual way, in addition to any sve_context.

The SVE vector registers are dumped in full, including bits 127:0
of each register which alias the corresponding FPSIMD vector
registers in the hardware.  To avoid any ambiguity about which
alias to restore during sigreturn, the kernel always restores bits
127:0 of each SVE vector register from the fpsimd_context in the
signal frame (which must be present): userspace needs to take this
into account if it wants to modify the SVE vector register contents
on return from a signal.

FPSR and FPCR, which are used by both FPSIMD and SVE, are not
included in sve_context because they are always present in
fpsimd_context anyway.

For signal delivery, a new helper
fpsimd_signal_preserve_current_state() is added to update _both_
the FPSIMD and SVE views in the task struct, to make it easier to
populate this information into the signal frame.  Because of the
redundancy between the two views of the state, only one is updated
otherwise.

Reviewed-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: default avatarDave Martin <Dave.Martin@arm.com>
Cc: Alex Bennée <alex.bennee@linaro.org>
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: default avatarWill Deacon <will.deacon@arm.com>
      8cd969d2
    • Dave Martin's avatar
      arm64/sve: Support vector length resetting for new processes · 79ab047c
      Dave Martin authored 
      

It's desirable to be able to reset the vector length to some sane
default for new processes, since the new binary and its libraries
may or may not be SVE-aware.

This patch tracks the desired post-exec vector length (if any) in a
new thread member sve_vl_onexec, and adds a new thread flag
TIF_SVE_VL_INHERIT to control whether to inherit or reset the
vector length.  Currently these are inactive.  Subsequent patches
will provide the capability to configure them.

Signed-off-by: default avatarDave Martin <Dave.Martin@arm.com>
Reviewed-by: default avatarAlex Bennée <alex.bennee@linaro.org>
Reviewed-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: default avatarWill Deacon <will.deacon@arm.com>
      79ab047c