linux/arch/x86/kernel
Thomas Gleixner 130a1d76ee x86/fpu: Make init_fpstate correct with optimized XSAVE
commit f9dfb5e390 upstream.

The XSAVE init code initializes all enabled and supported components with
XRSTOR(S) to init state. Then it XSAVEs the state of the components back
into init_fpstate which is used in several places to fill in the init state
of components.

This works correctly with XSAVE, but not with XSAVEOPT and XSAVES because
those use the init optimization and skip writing state of components which
are in init state. So init_fpstate.xsave still contains all zeroes after
this operation.

There are two ways to solve that:

   1) Use XSAVE unconditionally, but that requires to reshuffle the buffer when
      XSAVES is enabled because XSAVES uses compacted format.

   2) Save the components which are known to have a non-zero init state by other
      means.

Looking deeper, #2 is the right thing to do because all components the
kernel supports have all-zeroes init state except the legacy features (FP,
SSE). Those cannot be hard coded because the states are not identical on all
CPUs, but they can be saved with FXSAVE which avoids all conditionals.

Use FXSAVE to save the legacy FP/SSE components in init_fpstate along with
a BUILD_BUG_ON() which reminds developers to validate that a newly added
component has all zeroes init state. As a bonus remove the now unused
copy_xregs_to_kernel_booting() crutch.

The XSAVE and reshuffle method can still be implemented in the unlikely
case that components are added which have a non-zero init state and no
other means to save them. For now, FXSAVE is just simple and good enough.

  [ bp: Fix a typo or two in the text. ]

Fixes: 6bad06b768 ("x86, xsave: Use xsaveopt in context-switch path when supported")
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Borislav Petkov <bp@suse.de>
Reviewed-by: Borislav Petkov <bp@suse.de>
Cc: stable@vger.kernel.org
Link: https://lkml.kernel.org/r/20210618143444.587311343@linutronix.de
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
History 2021-06-30 08:47:26 -04:00
..
acpi ACPI: tables: x86: Reserve memory occupied by ACPI tables 2021-04-07 15:00:08 +02:00
apic x86/apic: Mark _all_ legacy interrupts when IO/APIC is missing 2021-06-10 13:39:27 +02:00
cpu x86/nmi_watchdog: Fix old-style NMI watchdog regression on old Intel CPUs 2021-06-16 12:01:46 +02:00
fpu x86/fpu: Make init_fpstate correct with optimized XSAVE 2021-06-30 08:47:26 -04:00
kprobes x86/kprobes: Fix to check non boostable prefixes correctly 2021-05-14 09:50:24 +02:00
.gitignore .gitignore: add SPDX License Identifier 2020-03-25 11:50:48 +01:00
alternative.c A couple of x86 fixes which missed rc1 due to my stupidity: 2020-10-27 14:39:29 -07:00
amd_gart_64.c dma-mapping: split <linux/dma-mapping.h> 2020-10-06 07:07:03 +02:00
amd_nb.c x86/amd_nb: Add AMD family 17h model 60h PCI IDs 2020-05-22 18:24:40 +02:00
apb_timer.c x86/apb_timer: Drop unused TSC calibration 2020-05-27 13:05:59 +02:00
aperture_64.c x86/gart: Exclude GART aperture from kcore 2019-03-23 12:11:49 +01:00
apm_32.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 118 2019-05-24 17:39:02 +02:00
asm-offsets.c x86: remove address space overrides using set_fs() 2020-09-08 22:21:36 -04:00
asm-offsets_32.c x86 entry code updates: 2020-03-30 19:14:28 -07:00
asm-offsets_64.c x86/entry: Remove DBn stacks 2020-06-11 15:15:23 +02:00
audit_64.c x86/audit: Fix a -Wmissing-prototypes warning for ia32_classify_syscall() 2020-05-19 18:03:07 +02:00
bootflag.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
check.c x86/headers: Fix -Wmissing-prototypes warning 2018-11-23 07:59:59 +01:00
cpuid.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 142 2019-05-30 11:25:17 -07:00
crash.c x86/crash: Fix crash_setup_memmap_entries() out-of-bounds access 2021-04-28 13:40:02 +02:00
crash_core_32.c mm: reorder includes after introduction of linux/pgtable.h 2020-06-09 09:39:13 -07:00
crash_core_64.c mm: reorder includes after introduction of linux/pgtable.h 2020-06-09 09:39:13 -07:00
crash_dump_32.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
crash_dump_64.c fs/core/vmcore: Move sev_active() reference to x86 arch code 2019-08-09 22:52:10 +10:00
devicetree.c x86_ioapic_Consolidate_IOAPIC_allocation 2020-09-16 16:52:32 +02:00
doublefault_32.c x86/entry: Convert double fault exception to IDTENTRY_DF 2020-06-11 15:15:03 +02:00
dumpstack.c x86/dumpstack: Do not try to access user space code of other tasks 2020-11-18 12:56:29 +01:00
dumpstack_32.c x86/32: Remove CONFIG_DOUBLEFAULT 2020-04-14 14:24:05 +02:00
dumpstack_64.c x86/dumpstack/64: Add noinstr version of get_stack_info() 2020-09-09 11:33:19 +02:00
e820.c PM: hibernate: x86: Use crc32 instead of md5 for hibernation e820 integrity check 2021-05-14 09:50:21 +02:00
early-quirks.c x86/gpu: add RKL stolen memory support 2020-05-20 08:35:22 -07:00
early_printk.c mm: reorder includes after introduction of linux/pgtable.h 2020-06-09 09:39:13 -07:00
ebda.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
eisa.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 243 2019-06-19 17:09:07 +02:00
espfix_64.c mm: introduce include/linux/pgtable.h 2020-06-09 09:39:13 -07:00
ftrace.c x86/ftrace: Do not jump to direct code in created trampolines 2020-06-29 11:42:48 -04:00
ftrace_32.S x86: Change {JMP,CALL}_NOSPEC argument 2020-04-30 20:14:34 +02:00
ftrace_64.S x86/ftrace: Do not jump to direct code in created trampolines 2020-06-29 11:42:48 -04:00
head32.c x86/boot: Mostly revert commit ae7e1238e6 ("Add ACPI RSDP address to setup_header") 2018-11-20 09:43:10 +01:00
head64.c treewide: Convert macro and uses of __section(foo) to __section("foo") 2020-10-25 14:51:49 -07:00
head_32.S x86/xen: remove 32-bit Xen PV guest support 2020-08-11 08:26:48 +02:00
head_64.S x86/head/64: Check SEV encryption before switching to kernel page-table 2020-10-29 18:09:59 +01:00
hpet.c remove ioremap_nocache and devm_ioremap_nocache 2020-01-06 09:45:59 +01:00
hw_breakpoint.c x86/debug: Prevent data breakpoints on cpu_dr7 2021-02-10 09:29:22 +01:00
i8237.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
i8253.c x86/timer: Skip PIT initialization on modern chipsets 2019-06-29 11:35:35 +02:00
i8259.c x86/i8259: Use printk_deferred() to prevent deadlock 2020-07-29 16:27:16 +02:00
idt.c This feature enhances the current guest memory encryption support 2020-10-14 10:21:34 -07:00
ima_arch.c EFI updates for v5.7: 2020-02-26 15:21:22 +01:00
io_delay.c x86/io_delay: Define IO_DELAY macros in C instead of Kconfig 2019-05-24 08:46:06 +02:00
ioport.c x86/ioperm: Prevent a memory leak when fork fails 2020-05-28 21:36:20 +02:00
irq.c x86/irq: Make run_on_irqstack_cond() typesafe 2020-09-22 22:13:34 +02:00
irq_32.c x86/irq: Rework handle_irq() for 64-bit 2020-06-11 15:15:12 +02:00
irq_64.c x86/irq: Make run_on_irqstack_cond() typesafe 2020-09-22 22:13:34 +02:00
irq_work.c x86/entry: Convert various system vectors 2020-06-11 15:15:14 +02:00
irqflags.S x86/asm: Change all ENTRY+ENDPROC to SYM_FUNC_* 2019-10-18 11:58:33 +02:00
irqinit.c x86/headers: Remove APIC headers from <asm/smp.h> 2020-08-06 16:13:09 +02:00
itmt.c sysctl: pass kernel pointers to ->proc_handler 2020-04-27 02:07:40 -04:00
jailhouse.c locking/seqlock, headers: Untangle the spaghetti monster 2020-08-06 16:13:13 +02:00
jump_label.c x86/jump_label: Move 'inline' keyword placement 2020-03-27 11:05:41 +01:00
kdebugfs.c x86/boot: Introduce setup_indirect 2019-11-12 16:21:15 +01:00
kexec-bzimage64.c x86/kexec: Use up-to-dated screen_info copy to fill boot params 2020-10-14 17:05:03 +02:00
kgdb.c x86/debug: Change thread.debugreg6 to thread.virtual_dr6 2020-09-04 15:12:58 +02:00
ksysfs.c x86/boot: Introduce setup_indirect 2019-11-12 16:21:15 +01:00
kvm.c x86/kvm: Disable all PV features on crash 2021-06-10 13:39:29 +02:00
kvmclock.c x86/kvm: Disable all PV features on crash 2021-06-10 13:39:29 +02:00
ldt.c x86/ldt: use "pr_info_once()" instead of open-coding it badly 2020-07-05 12:50:20 -07:00
machine_kexec_32.c mm: don't include asm/pgtable.h if linux/mm.h is already included 2020-06-09 09:39:13 -07:00
machine_kexec_64.c mm: don't include asm/pgtable.h if linux/mm.h is already included 2020-06-09 09:39:13 -07:00
Makefile x86/head/64: Disable stack protection for head$(BITS).o 2020-10-19 13:11:00 +02:00
mmconf-fam10h_64.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
module.c x86/build: Treat R_386_PLT32 relocation as R_386_PC32 2021-03-07 12:34:04 +01:00
mpparse.c Surgery of the MSI interrupt handling to prepare the support of upcoming 2020-10-12 11:40:41 -07:00
msr.c x86/MSR: Filter MSR writes through X86_IOC_WRMSR_REGS ioctl too 2021-03-04 11:37:26 +01:00
nmi.c KVM/VMX: Invoke NMI non-IST entry instead of IST entry 2021-05-19 10:12:51 +02:00
nmi_selftest.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
paravirt-spinlocks.c x86/paravirt: Use a single ops structure 2018-09-03 16:50:35 +02:00
paravirt.c x86/paravirt: Remove set_pte_at() pv-op 2020-08-15 13:52:12 +02:00
paravirt_patch.c x86/paravirt: Remove 32-bit support from CONFIG_PARAVIRT_XXL 2020-08-15 13:52:11 +02:00
pci-dma.c dma-mapping: move dma-debug.h to kernel/dma/ 2020-10-06 07:07:05 +02:00
pci-iommu_table.c x86/iommu: Use NULL instead of 0 2018-08-02 14:33:19 +02:00
pci-swiotlb.c dma-mapping: fix filename references 2019-09-03 08:36:30 +02:00
pcspeaker.c x86/platform/pcspeaker: Use PTR_ERR_OR_ZERO() to fix ptr_ret.cocci warning 2018-07-24 09:46:42 +02:00
perf_regs.c perf/arch: Remove perf_sample_data::regs_user_copy 2020-11-09 18:12:34 +01:00
platform-quirks.c x86/i8237: Register device based on FADT legacy boot flag 2018-04-27 16:44:29 +02:00
pmem.c resource: Provide resource struct in resource walk callback 2017-11-07 15:35:57 +01:00
probe_roms.c maccess: make get_kernel_nofault() check for minimal type compatibility 2020-06-18 12:10:37 -07:00
process.c sched/idle: Fix arch_cpu_idle() vs tracing 2020-11-24 16:47:35 +01:00
process.h x86: Use the correct SPDX License Identifier in headers 2019-10-01 20:31:35 +02:00
process_32.c x86/dumpstack: Add log_lvl to __show_regs() 2020-07-22 23:56:53 +02:00
process_64.c x86/fsgsbase: Replace static_cpu_has() with boot_cpu_has() 2020-08-24 18:18:32 +02:00
ptrace.c x86/debug: Change thread.debugreg6 to thread.virtual_dr6 2020-09-04 15:12:58 +02:00
pvclock.c x86/vdso: Use generic VDSO clock mode storage 2020-02-17 14:40:23 +01:00
quirks.c x86, powerpc: Rename memcpy_mcsafe() to copy_mc_to_{user, kernel}() 2020-10-06 11:18:04 +02:00
reboot.c x86/reboot: Add Zotac ZBOX CI327 nano PCI reboot quirk 2021-03-07 12:34:09 +01:00
reboot_fixups_32.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
relocate_kernel_32.S x86/asm: Annotate relocate_kernel_{32,64}.c 2019-10-18 09:53:19 +02:00
relocate_kernel_64.S x86/kexec: Make relocate_kernel_64.S objtool clean 2020-03-25 18:28:28 +01:00
resource.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
rtc.c x86: Convert x86_platform_ops to timespec64 2018-05-19 14:03:14 +02:00
setup.c ACPI: x86: Call acpi_boot_table_init() after acpi_table_upgrade() 2021-04-21 13:00:51 +02:00
setup_percpu.c x86/mm: remove vmalloc faulting 2020-06-02 10:59:12 -07:00
sev-es-shared.c x86/sev-es: Invalidate the GHCB after completing VMGEXIT 2021-05-26 12:06:52 +02:00
sev-es.c x86/sev-es: Forward page-faults which happen during emulation 2021-05-26 12:06:53 +02:00
sev_verify_cbit.S x86/boot/compressed/64: Check SEV encryption in 64-bit boot-path 2020-10-29 18:06:52 +01:00
signal.c x86: Introduce TS_COMPAT_RESTART to fix get_nr_restart_syscall() 2021-03-25 09:04:17 +01:00
signal_compat.c arm64: mte: Add specific SIGSEGV codes 2020-09-04 12:46:06 +01:00
smp.c x86/entry: Convert reschedule interrupt to IDTENTRY_SYSVEC_SIMPLE 2020-06-11 15:15:16 +02:00
smpboot.c x86, sched: Treat Intel SNC topology as default, COD as exception 2021-05-14 09:49:59 +02:00
stacktrace.c stacktrace: Remove reliable argument from arch_stack_walk() callback 2020-09-18 14:24:16 +01:00
static_call.c static_call: Allow early init 2020-09-01 09:58:06 +02:00
step.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
sys_ia32.c x86: switch to kernel_clone() 2020-08-20 13:12:58 +02:00
sys_x86_64.c x86: Remove unneeded includes 2020-03-21 16:03:25 +01:00
sysfb.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
sysfb_efi.c x86/sysfb_efi: Add quirks for some devices with swapped width and height 2019-07-22 10:47:11 +02:00
sysfb_simplefb.c x86/sysfb: Fix check for bad VRAM size 2020-01-20 10:57:53 +01:00
tboot.c mm/gup: prevent gup_fast from racing with COW during fork 2020-12-30 11:53:54 +01:00
time.c A set of fixes and updates for x86: 2020-06-11 15:54:31 -07:00
tls.c x86: switch to ->regset_get() 2020-07-27 14:31:07 -04:00
tls.h x86: switch to ->regset_get() 2020-07-27 14:31:07 -04:00
topology.c x86/headers: Remove APIC headers from <asm/smp.h> 2020-08-06 16:13:09 +02:00
trace_clock.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
tracepoint.c x86/entry: Convert reschedule interrupt to IDTENTRY_SYSVEC_SIMPLE 2020-06-11 15:15:16 +02:00
traps.c x86/entry: Move nmi entry/exit into common code 2021-03-17 17:06:36 +01:00
tsc.c x86/tsc: Use seqcount_latch_t 2020-09-10 11:19:29 +02:00
tsc_msr.c Misc fixes and small updates all around the place: 2020-08-15 10:38:03 -07:00
tsc_sync.c x86: Fix a handful of typos 2020-02-16 20:58:06 +01:00
umip.c x86/umip: Factor out instruction decoding 2020-09-07 19:45:24 +02:00
unwind_frame.c fork-v5.9 2020-08-04 14:47:45 -07:00
unwind_guess.c treewide: Add SPDX license identifier for missed files 2019-05-21 10:50:45 +02:00
unwind_orc.c x86/unwind/orc: Disable KASAN checking in the ORC unwinder, part 2 2021-03-17 17:06:35 +01:00
uprobes.c x86/uprobes: Do not use prefixes.nbytes when looping over prefixes.bytes 2020-12-06 09:58:13 +01:00
verify_cpu.S x86/asm: Annotate local pseudo-functions 2019-10-18 10:04:04 +02:00
vm86_32.c mmap locking API: use coccinelle to convert mmap_sem rwsem call sites 2020-06-09 09:39:14 -07:00
vmlinux.lds.S This tree introduces static_call(), which is the idea of static_branch() 2020-10-12 13:58:15 -07:00
vsmp_64.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 346 2019-06-05 17:37:08 +02:00
x86_init.c x86/irq: Cleanup the arch_*_msi_irqs() leftovers 2020-09-16 16:52:38 +02:00