Skip to content
  1. May 19, 2015
    • Ingo Molnar's avatar
      x86/fpu: Use 'struct fpu' in fpu_lazy_restore() · 66ddc2cb
      Ingo Molnar authored
      
      
      Also rename it to fpu_want_lazy_restore(), to better indicate that
      this function just tests whether we can do a lazy restore. (The old
      name suggested that it was doing the lazy restore, which is not
      the case.)
      
      Reviewed-by: default avatarBorislav Petkov <bp@alien8.de>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: H. Peter Anvin <hpa@zytor.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
      66ddc2cb
    • Ingo Molnar's avatar
      x86/fpu: Remove task_disable_lazy_fpu_restore() · eb6a3251
      Ingo Molnar authored
      
      
      Replace task_disable_lazy_fpu_restore() with easier to read
      open-coded uses: we already update the fpu->last_cpu field
      explicitly in other cases.
      
      (This also removes yet another task_struct using FPU method.)
      
      Better explain the fpu::last_cpu field in the structure definition.
      
      Reviewed-by: default avatarBorislav Petkov <bp@alien8.de>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: H. Peter Anvin <hpa@zytor.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
      eb6a3251
    • Ingo Molnar's avatar
      x86/fpu: Remove 'struct task_struct' usage from drop_fpu() · ca6787ba
      Ingo Molnar authored
      
      
      Migrate this function to pure 'struct fpu' usage.
      
      Reviewed-by: default avatarBorislav Petkov <bp@alien8.de>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: H. Peter Anvin <hpa@zytor.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
      ca6787ba
    • Ingo Molnar's avatar
      x86/fpu: Get rid of PF_USED_MATH usage, convert it to fpu->fpstate_active · c5bedc68
      Ingo Molnar authored
      
      
      Introduce a simple fpu->fpstate_active flag in the fpu context data structure
      and use that instead of PF_USED_MATH in task->flags.
      
      Testing for this flag byte should be slightly more efficient than
      testing a bit in a bitmask, but the main advantage is that most
      FPU functions can now be performed on a 'struct fpu' alone, they
      don't need access to 'struct task_struct' anymore.
      
      There's a slight linecount increase, mostly due to the 'fpu' local
      variables and due to extra comments. The local variables will go away
      once we move most of the FPU methods to pure 'struct fpu' parameters.
      
      Reviewed-by: default avatarBorislav Petkov <bp@alien8.de>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: H. Peter Anvin <hpa@zytor.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
      c5bedc68
    • Ingo Molnar's avatar
      x86/fpu: Document fpu__unlazy_stopped() · af7f8721
      Ingo Molnar authored
      
      
      Explain its usage and also document a TODO item.
      
      Reviewed-by: default avatarBorislav Petkov <bp@alien8.de>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: H. Peter Anvin <hpa@zytor.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
      af7f8721
    • Ingo Molnar's avatar
      x86/fpu: Open code PF_USED_MATH usages · 4c138410
      Ingo Molnar authored
      
      
      PF_USED_MATH is used directly, but also in a handful of helper inlines.
      
      To ease the elimination of PF_USED_MATH, convert all inline helpers
      to open-coded PF_USED_MATH usage.
      
      Reviewed-by: default avatarBorislav Petkov <bp@alien8.de>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: H. Peter Anvin <hpa@zytor.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
      4c138410
    • Ingo Molnar's avatar
      x86/fpu: Remove 'struct task_struct' usage from __thread_fpu_begin() · 4540d3fa
      Ingo Molnar authored
      
      
      Migrate this function to pure 'struct fpu' usage.
      
      Reviewed-by: default avatarBorislav Petkov <bp@alien8.de>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: H. Peter Anvin <hpa@zytor.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
      4540d3fa
    • Ingo Molnar's avatar
      x86/fpu: Remove 'struct task_struct' usage from __thread_fpu_end() · 35191e3f
      Ingo Molnar authored
      
      
      Migrate this function to pure 'struct fpu' usage.
      
      Reviewed-by: default avatarBorislav Petkov <bp@alien8.de>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: H. Peter Anvin <hpa@zytor.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
      35191e3f
    • Ingo Molnar's avatar
      x86/fpu: Remove 'struct task_struct' usage from __thread_set_has_fpu() · c0311f63
      Ingo Molnar authored
      
      
      Migrate this function to pure 'struct fpu' usage.
      
      Reviewed-by: default avatarBorislav Petkov <bp@alien8.de>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: H. Peter Anvin <hpa@zytor.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
      c0311f63
    • Ingo Molnar's avatar
      x86/fpu: Change fpu_owner_task to fpu_fpregs_owner_ctx · 36b544dc
      Ingo Molnar authored
      
      
      Track the FPU owner context instead of the owner task: this change,
      together with other changes, will allow in subsequent patches the
      elimination of 'struct task_struct' usage in various FPU code:
      we'll be able to use 'struct fpu' only.
      
      There's no change in code size:
      
            text           data     bss      dec            hex filename
        13066467        2545248 1626112 17237827        1070743 vmlinux.before
        13066467        2545248 1626112 17237827        1070743 vmlinux.after
      
      Reviewed-by: default avatarBorislav Petkov <bp@alien8.de>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: H. Peter Anvin <hpa@zytor.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
      36b544dc
    • Ingo Molnar's avatar
      x86/fpu: Move 'PER_CPU(fpu_owner_task)' to fpu/core.c · b0c050c5
      Ingo Molnar authored
      
      
      Move it closer to other per-cpu FPU data structures.
      
      This also unifies the 32-bit and 64-bit code.
      
      Reviewed-by: default avatarBorislav Petkov <bp@alien8.de>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: H. Peter Anvin <hpa@zytor.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
      b0c050c5
    • Ingo Molnar's avatar
      x86/fpu: Change __thread_clear_has_fpu() to 'struct fpu' parameter · 36fe6175
      Ingo Molnar authored
      
      
      We do this to make the code more readable, and also to be able to eliminate
      task_struct usage from most of the FPU code.
      
      Reviewed-by: default avatarBorislav Petkov <bp@alien8.de>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: H. Peter Anvin <hpa@zytor.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
      36fe6175
    • Ingo Molnar's avatar
      x86/fpu: Eliminate the __thread_has_fpu() wrapper · 276983f8
      Ingo Molnar authored
      
      
      Start migrating FPU methods towards using 'struct fpu *fpu'
      directly. __thread_has_fpu() is just a trivial wrapper around
      fpu->has_fpu, eliminate it.
      
      Reviewed-by: default avatarBorislav Petkov <bp@alien8.de>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: H. Peter Anvin <hpa@zytor.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
      276983f8
    • Ingo Molnar's avatar
      x86/fpu: Print out whether we are doing lazy/eager FPU context switches · 9a89b029
      Ingo Molnar authored
      
      
      Ever since the kernel started defaulting to eager FPU switches on modern Intel
      CPUs it's not been obvious whether a given system is using the lazy or the eager
      FPU context switching logic.
      
      So generate a boot message about which mode the FPU code is in:
      
        x86/fpu: Using 'lazy' FPU context switches.
      
      or:
      
        x86/fpu: Using 'eager' FPU context switches.
      
      Reviewed-by: default avatarBorislav Petkov <bp@alien8.de>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: H. Peter Anvin <hpa@zytor.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
      9a89b029
    • Ingo Molnar's avatar
      x86/fpu: Add debugging check to fpu_copy() · bfd6fc05
      Ingo Molnar authored
      
      
      Also add a bit of documentation.
      
      Reviewed-by: default avatarBorislav Petkov <bp@alien8.de>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: H. Peter Anvin <hpa@zytor.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
      bfd6fc05
    • Ingo Molnar's avatar
      x86/fpu: Move fpu_copy() to fpu/core.c · e102f30f
      Ingo Molnar authored
      
      
      Move fpu_copy() where its only user is.
      
      Beyond readability this also speeds up compilation, as fpu-internal.h
      is included in over a dozen .c files.
      
      Reviewed-by: default avatarBorislav Petkov <bp@alien8.de>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: H. Peter Anvin <hpa@zytor.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
      e102f30f
    • Ingo Molnar's avatar
      x86/fpu: Remove __save_init_fpu() · 6522d783
      Ingo Molnar authored
      
      
      __save_init_fpu() is just a trivial wrapper around fpu_save_init().
      
      Remove the extra layer of obfuscation.
      
      Reviewed-by: default avatarBorislav Petkov <bp@alien8.de>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: H. Peter Anvin <hpa@zytor.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
      6522d783
    • Ingo Molnar's avatar
      x86/fpu: Add kernel_fpu_disabled() · 085cc281
      Ingo Molnar authored
      
      
      Instead of open-coded in_kernel_fpu access, Use kernel_fpu_disabled() in
      interrupted_kernel_fpu_idle(), matching the other kernel_fpu_*() methods.
      
      Also add some documentation for in_kernel_fpu.
      
      Reviewed-by: default avatarBorislav Petkov <bp@alien8.de>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: H. Peter Anvin <hpa@zytor.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
      085cc281
    • Ingo Molnar's avatar
      x86/fpu: Add debug check to kernel_fpu_disable() · 3103ae3a
      Ingo Molnar authored
      
      
      We are not supposed to call kernel_fpu_disable() if we have not
      previously enabled it.
      
      Also use kernel_fpu_disable()/enable() in the __kernel_fpu_begin/end()
      primitives, instead of writing to in_kernel_fpu directly,
      so that we get the debugging checks.
      
      Reviewed-by: default avatarBorislav Petkov <bp@alien8.de>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: H. Peter Anvin <hpa@zytor.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
      3103ae3a
    • Ingo Molnar's avatar
      x86/fpu: Make kernel_fpu_disable/enable() static · 416d49ac
      Ingo Molnar authored
      
      
      This allows the compiler to inline them and to eliminate them:
      
         arch/x86/kernel/fpu/core.o:
      
         text    data     bss     dec     hex filename
         6741       4       8    6753    1a61 core.o.before
         6716       4       8    6728    1a48 core.o.after
      
      Reviewed-by: default avatarBorislav Petkov <bp@alien8.de>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: H. Peter Anvin <hpa@zytor.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
      416d49ac
    • Ingo Molnar's avatar
      x86/fpu: Make task_xstate_cachep static · f55f88e2
      Ingo Molnar authored
      
      
      It's now local to fpu/core.c, make it static.
      
      Reviewed-by: default avatarBorislav Petkov <bp@alien8.de>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: H. Peter Anvin <hpa@zytor.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
      f55f88e2
    • Ingo Molnar's avatar
      x86/fpu: Uninline fpstate_free() and move it next to the allocation function · 5a12bf63
      Ingo Molnar authored
      
      
      Reviewed-by: default avatarBorislav Petkov <bp@alien8.de>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: H. Peter Anvin <hpa@zytor.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
      5a12bf63
    • Ingo Molnar's avatar
      x86/fpu: Factor out fpu__copy() · a752b53d
      Ingo Molnar authored
      
      
      Introduce fpu__copy() and use it in arch_dup_task_struct(),
      thus moving another chunk of FPU logic to fpu/core.c.
      
      Reviewed-by: default avatarBorislav Petkov <bp@alien8.de>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: H. Peter Anvin <hpa@zytor.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
      a752b53d
    • Ingo Molnar's avatar
      x86/fpu: Move task_xstate_cachep handling to core.c · 8ffb53ab
      Ingo Molnar authored
      
      
      This code was historically in process.c, now we have FPU core internals in
      fpu/core.c instead - move it there.
      
      Reviewed-by: default avatarBorislav Petkov <bp@alien8.de>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: H. Peter Anvin <hpa@zytor.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
      8ffb53ab
    • Ingo Molnar's avatar
      x86/fpu: Remove fpu_xsave() · 0afc4a94
      Ingo Molnar authored
      
      
      It's a pointless wrapper now - use xsave_state().
      
      Reviewed-by: default avatarBorislav Petkov <bp@alien8.de>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: H. Peter Anvin <hpa@zytor.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
      0afc4a94
    • Ingo Molnar's avatar
      x86/fpu: Simplify the xsave_state*() methods · 3e261c14
      Ingo Molnar authored
      
      
      These functions (xsave_state() and xsave_state_booting()) have a 'mask'
      argument that is always -1.
      
      Propagate this into the functions instead and eliminate the extra argument.
      
      Does not change the generated code, because these were inlined functions.
      
      Reviewed-by: default avatarBorislav Petkov <bp@alien8.de>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: H. Peter Anvin <hpa@zytor.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
      3e261c14
    • Ingo Molnar's avatar
      x86/fpu: Factor out the FPU bug detection code into fpu__init_check_bugs() · 4d164092
      Ingo Molnar authored
      
      
      Move the boot-time FPU bug detection code to the other FPU boot time
      init code in fpu/init.c.
      
      No change in code size:
      
         text    data     bss     dec     hex filename
         13044568        1884440 1130496 16059504         f50c70 vmlinux.before
         13044568        1884440 1130496 16059504         f50c70 vmlinux.after
      
      Reviewed-by: default avatarBorislav Petkov <bp@alien8.de>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: H. Peter Anvin <hpa@zytor.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
      4d164092
    • Ingo Molnar's avatar
      x86/fpu: Rename math_state_restore() to fpu__restore() · 3a0aee48
      Ingo Molnar authored
      
      
      Move to the new fpu__*() namespace.
      
      Reviewed-by: default avatarBorislav Petkov <bp@alien8.de>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: H. Peter Anvin <hpa@zytor.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
      3a0aee48
    • Ingo Molnar's avatar
      x86/fpu: Move math_state_restore() to fpu/core.c · 93b90712
      Ingo Molnar authored
      
      
      It's another piece of FPU internals that is better off close to
      the other FPU internals.
      
      Reviewed-by: default avatarBorislav Petkov <bp@alien8.de>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: H. Peter Anvin <hpa@zytor.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
      93b90712
    • Ingo Molnar's avatar
      x86/fpu: Factor out fpu__flush_thread() from flush_thread() · 81683cc8
      Ingo Molnar authored
      
      
      flush_thread() open codes a lot of FPU internals - create a separate
      function for it in fpu/core.c.
      
      Turns out that this does not hurt performance:
      
         text    data     bss     dec     hex filename
         11843039        1884440 1130496 14857975         e2b6f7 vmlinux.before
         11843039        1884440 1130496 14857975         e2b6f7 vmlinux.after
      
      and since this is a slowpath clarity comes first anyway.
      
      We can reconsider inlining decisions after the FPU code has been cleaned up.
      
      Reviewed-by: default avatarBorislav Petkov <bp@alien8.de>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: H. Peter Anvin <hpa@zytor.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
      81683cc8
    • Ingo Molnar's avatar
      x86/fpu: Remove the free_thread_xstate() complication · 11ad1927
      Ingo Molnar authored
      
      
      Use fpstate_free() directly to manage FPU state.
      
      Only process.c was using this method, so this is a speedup as well,
      as it removes the extra function call and related clobbers.
      
      Reviewed-by: default avatarBorislav Petkov <bp@alien8.de>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: H. Peter Anvin <hpa@zytor.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
      11ad1927
    • Ingo Molnar's avatar
      x86/fpu: Move the no_387 handling and FPU detection code into init.c · 146ed598
      Ingo Molnar authored
      
      
      Both no_387() and fpu__detect() run at boot time, so they belong
      into init.c.
      
      Reviewed-by: default avatarBorislav Petkov <bp@alien8.de>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: H. Peter Anvin <hpa@zytor.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
      146ed598
    • Ingo Molnar's avatar
      x86/fpu: Remove unnecessary includes from core.c · 4445e6e9
      Ingo Molnar authored
      
      
      fpu/core.c includes a lot of files for mostly historic reasons.
      
      It only needs fpu-internal.h, which already includes all
      the required headers.
      
      Reviewed-by: default avatarBorislav Petkov <bp@alien8.de>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: H. Peter Anvin <hpa@zytor.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
      4445e6e9
    • Ingo Molnar's avatar
      x86/fpu: Split out the boot time FPU init code into fpu/init.c · 0c867537
      Ingo Molnar authored
      
      
      Move boot time FPU initialization code into init.c, to better
      isolate it into its own domain.
      
      Reviewed-by: default avatarBorislav Petkov <bp@alien8.de>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: H. Peter Anvin <hpa@zytor.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
      0c867537
    • Ingo Molnar's avatar
      x86/fpu: Fix header file dependencies of fpu-internal.h · f89e32e0
      Ingo Molnar authored
      
      
      Fix a minor header file dependency bug in asm/fpu-internal.h: it
      relies on i387.h but does not include it. All users of fpu-internal.h
      included it explicitly.
      
      Also remove unnecessary includes, to reduce compilation time.
      
      This also makes it easier to use it as a standalone header file
      for FPU internals, such as an upcoming C module in arch/x86/kernel/fpu/.
      
      Reviewed-by: default avatarBorislav Petkov <bp@alien8.de>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: H. Peter Anvin <hpa@zytor.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
      f89e32e0
    • Ingo Molnar's avatar
      x86/fpu: Move i387.c and xsave.c to arch/x86/kernel/fpu/ · ce4c4c26
      Ingo Molnar authored
      
      
      Create a new subdirectory for the FPU support code in arch/x86/kernel/fpu/.
      
      Rename 'i387.c' to 'core.c' - as this really collects the core FPU support
      code, nothing i387 specific.
      
      We'll better organize this directory in later patches.
      
      Reviewed-by: default avatarBorislav Petkov <bp@alien8.de>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: H. Peter Anvin <hpa@zytor.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
      ce4c4c26
    • Ingo Molnar's avatar
      x86/fpu: Clean up asm/fpu/types.h · 47bc5106
      Ingo Molnar authored
      
      
       - add header guards
      
       - standardize vertical alignment
      
       - add comments about MPX
      
      No code changed.
      
      Reviewed-by: default avatarBorislav Petkov <bp@alien8.de>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: H. Peter Anvin <hpa@zytor.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
      47bc5106
    • Ingo Molnar's avatar
      x86/fpu: Move FPU data structures to asm/fpu_types.h · 14b9675a
      Ingo Molnar authored
      
      
      Move the FPU details to asm/fpu_types.h, to further factor out the
      FPU code.
      
      ( As an added bonus, the 'struct orig_ist' definition now moves
        next to its other data types - the FPU definitions were
        slapped in the middle of them for some mysterious reason. )
      
      No code changed.
      
      Reviewed-by: default avatarBorislav Petkov <bp@alien8.de>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: H. Peter Anvin <hpa@zytor.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
      14b9675a
    • Ingo Molnar's avatar
      x86/fpu: Improve the comment for the fpu::counter field · 12600999
      Ingo Molnar authored
      
      
      This was pretty hard to read, improve it.
      
      Reviewed-by: default avatarBorislav Petkov <bp@alien8.de>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: H. Peter Anvin <hpa@zytor.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
      12600999
    • Ingo Molnar's avatar
      x86/fpu: Move thread_info::fpu_counter into thread_info::fpu.counter · c0c2803d
      Ingo Molnar authored
      
      
      This field is kept separate from the main FPU state structure for
      no good reason.
      
      Reviewed-by: default avatarBorislav Petkov <bp@alien8.de>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: H. Peter Anvin <hpa@zytor.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
      c0c2803d