linux/include/asm-generic
Richard Fellner f9a1666f97 KAISER: Kernel Address Isolation
This patch introduces our implementation of KAISER (Kernel Address Isolation to
have Side-channels Efficiently Removed), a kernel isolation technique to close
hardware side channels on kernel address information.

More information about the patch can be found on:

        https://github.com/IAIK/KAISER

From: Richard Fellner <richard.fellner@student.tugraz.at>
From: Daniel Gruss <daniel.gruss@iaik.tugraz.at>
Subject: [RFC, PATCH] x86_64: KAISER - do not map kernel in user mode
Date: Thu, 4 May 2017 14:26:50 +0200
Link: http://marc.info/?l=linux-kernel&m=149390087310405&w=2
Kaiser-4.10-SHA1: c4b1831d44c6144d3762ccc72f0c4e71a0c713e5

To: <linux-kernel@vger.kernel.org>
To: <kernel-hardening@lists.openwall.com>
Cc: <clementine.maurice@iaik.tugraz.at>
Cc: <moritz.lipp@iaik.tugraz.at>
Cc: Michael Schwarz <michael.schwarz@iaik.tugraz.at>
Cc: Richard Fellner <richard.fellner@student.tugraz.at>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: <kirill.shutemov@linux.intel.com>
Cc: <anders.fogh@gdata-adan.de>

After several recent works [1,2,3] KASLR on x86_64 was basically
considered dead by many researchers. We have been working on an
efficient but effective fix for this problem and found that not mapping
the kernel space when running in user mode is the solution to this
problem [4] (the corresponding paper [5] will be presented at ESSoS17).

With this RFC patch we allow anybody to configure their kernel with the
flag CONFIG_KAISER to add our defense mechanism.

If there are any questions we would love to answer them.
We also appreciate any comments!

Cheers,
Daniel (+ the KAISER team from Graz University of Technology)

[1] http://www.ieee-security.org/TC/SP2013/papers/4977a191.pdf
[2] https://www.blackhat.com/docs/us-16/materials/us-16-Fogh-Using-Undocumented-CPU-Behaviour-To-See-Into-Kernel-Mode-And-Break-KASLR-In-The-Process.pdf
[3] https://www.blackhat.com/docs/us-16/materials/us-16-Jang-Breaking-Kernel-Address-Space-Layout-Randomization-KASLR-With-Intel-TSX.pdf
[4] https://github.com/IAIK/KAISER
[5] https://gruss.cc/files/kaiser.pdf

(cherry picked from Change-Id: I0eb000c33290af01fc4454ca0c701d00f1d30b1d)

Conflicts:
arch/x86/entry/entry_64.S (not in this tree)
arch/x86/kernel/entry_64.S (patched instead of that)
arch/x86/entry/entry_64_compat.S (not in this tree)
arch/x86/ia32/ia32entry.S (patched instead of that)
arch/x86/include/asm/hw_irq.h
arch/x86/include/asm/pgtable_types.h
arch/x86/include/asm/processor.h
arch/x86/kernel/irqinit.c
arch/x86/kernel/process.c
arch/x86/mm/Makefile
arch/x86/mm/pgtable.c
init/main.c

Signed-off-by: Hugh Dickins <hughd@google.com>
[bwh: Folded in the follow-up patches from Hugh:
 - kaiser: merged update
 - kaiser: do not set _PAGE_NX on pgd_none
 - kaiser: stack map PAGE_SIZE at THREAD_SIZE-PAGE_SIZE
 - kaiser: fix build and FIXME in alloc_ldt_struct()
 - kaiser: KAISER depends on SMP
 - kaiser: fix regs to do_nmi() ifndef CONFIG_KAISER
 - kaiser: fix perf crashes
 - kaiser: ENOMEM if kaiser_pagetable_walk() NULL
 - kaiser: tidied up asm/kaiser.h somewhat
 - kaiser: tidied up kaiser_add/remove_mapping slightly
 - kaiser: kaiser_remove_mapping() move along the pgd
 - kaiser: align addition to x86/mm/Makefile
 - kaiser: cleanups while trying for gold link
 - kaiser: name that 0x1000 KAISER_SHADOW_PGD_OFFSET
 - kaiser: delete KAISER_REAL_SWITCH option
 - kaiser: vmstat show NR_KAISERTABLE as nr_overhead
 - kaiser: enhanced by kernel and user PCIDs
 - kaiser: load_new_mm_cr3() let SWITCH_USER_CR3 flush user
 - kaiser: PCID 0 for kernel and 128 for user
 - kaiser: x86_cr3_pcid_noflush and x86_cr3_pcid_user
 - kaiser: paranoid_entry pass cr3 need to paranoid_exit
 - kaiser: _pgd_alloc() without __GFP_REPEAT to avoid stalls
 - kaiser: fix unlikely error in alloc_ldt_struct()
 - kaiser: drop is_atomic arg to kaiser_pagetable_walk()
 Backported to 3.16:
 - Add missing #include in arch/x86/mm/kaiser.c
 - Use variable PEBS buffer size since we have "perf/x86/intel: Use PAGE_SIZE
   for PEBS buffer size on Core2"
 - Renumber X86_FEATURE_INVPCID_SINGLE to avoid collision
 - Adjust context]
Signed-off-by: Ben Hutchings <ben@decadent.org.uk>
History 2018-01-09 00:35:13 +00:00
..
bitops bitops: Do not default to __clear_bit() for __clear_bit_unlock() 2016-05-01 00:06:03 +02:00
4level-fixup.h mm: Pass virtual address to [__]p{te,ud,md}_free_tlb() 2009-07-27 12:10:38 -07:00
atomic-long.h asm-generic: merge branch 'master' of torvalds/linux-2.6 2009-06-12 11:32:58 +02:00
atomic.h arch: Prepare for smp_mb__{before,after}_atomic() 2014-04-18 11:40:30 +02:00
atomic64.h lib: Provide generic atomic64_t implementation 2009-06-15 13:27:38 +10:00
audit_change_attr.h audit: Modify a set of system calls in audit class definitions 2014-01-17 17:01:46 -05:00
audit_dir_write.h audit: support the "standard" <asm-generic/unistd.h> 2011-05-04 14:41:28 -04:00
audit_read.h audit: support the "standard" <asm-generic/unistd.h> 2011-05-04 14:41:28 -04:00
audit_signal.h [PATCH] add SIGNAL syscall class (v3) 2007-05-11 05:38:25 -04:00
audit_write.h audit: Modify a set of system calls in audit class definitions 2014-01-17 17:01:46 -05:00
barrier.h arch: Prepare for smp_mb__{before,after}_atomic() 2014-04-18 11:40:30 +02:00
bitops.h arch: Prepare for smp_mb__{before,after}_atomic() 2014-04-18 11:40:30 +02:00
bitsperlong.h UAPI: (Scripted) Disintegrate include/asm-generic 2012-10-04 18:20:15 +01:00
bug.h bug: Make BUG() always stop the machine 2014-04-07 16:36:10 -07:00
bugs.h asm-generic: add generic versions of common headers 2009-06-11 21:02:37 +02:00
cache.h asm-generic: add generic NOMMU versions of some headers 2009-06-11 21:02:50 +02:00
cacheflush.h asm-generic/cacheflush.h: flush icache when copying to user pages 2011-05-25 08:39:37 -07:00
checksum.h asm-generic headers: Allow yet more arch overrides in checksum.h 2013-02-11 20:00:33 +05:30
clkdev.h asm-generic: Add default clkdev.h 2012-10-03 21:33:53 +02:00
cmpxchg-local.h LLVMLinux: Remove warning about returning an uninitialized variable 2014-04-09 13:44:35 -07:00
cmpxchg.h asm-generic: move cmpxchg*_local defs to cmpxchg.h 2013-03-13 06:11:05 +01:00
cputime.h cputime: Generic on-demand virtual cputime accounting 2013-01-27 19:23:27 +01:00
cputime_jiffies.h cputime: Bring cputime -> nsecs conversion 2014-03-13 15:56:44 +01:00
cputime_nsecs.h cputime: Prevent 32bit overflow in time[val|spec]_to_cputime() 2016-02-25 00:18:42 +00:00
current.h asm-generic: add generic versions of common headers 2009-06-11 21:02:37 +02:00
delay.h asm-generic: delay.h fix udelay and ndelay for 8 bit args 2011-07-22 18:45:33 +02:00
device.h Driver Core: Add platform device arch data V3 2009-07-22 00:28:38 +02:00
div64.h rename div64_64 to div64_u64 2008-05-01 08:03:58 -07:00
dma-coherent.h DMA-API: Change dma_declare_coherent_memory() CPU address to phys_addr_t 2014-05-20 16:55:23 -06:00
dma-mapping-broken.h asm-generic/dma-mapping-broken.h: Provide dma_alloc_attrs()/dma_free_attrs() 2012-12-25 20:14:54 +01:00
dma-mapping-common.h common: dma-mapping: introduce dma_get_sgtable() function 2012-07-30 12:25:46 +02:00
dma.h asm-generic: add legacy I/O header files 2009-06-11 21:02:42 +02:00
early_ioremap.h mm: create generic early_ioremap() support 2014-04-07 16:36:15 -07:00
emergency-restart.h [PATCH] Add emergency_restart() 2005-07-26 14:35:41 -07:00
exec.h Split arch_align_stack() out from asm-generic/system.h 2012-03-28 18:30:03 +01:00
fb.h asm-generic: add generic versions of common headers 2009-06-11 21:02:37 +02:00
fixmap.h arm64: fixmap: fix missing sub-page offset for earlyprintk 2014-05-03 22:20:31 +01:00
ftrace.h asm-generic headers: add ftrace.h 2011-03-17 09:19:04 +08:00
futex.h futex: Sanitize futex ops argument types 2011-03-11 12:23:31 +01:00
getorder.h bitops: Add missing parentheses to new get_order macro 2012-02-24 10:39:27 -08:00
gpio.h gpio: make gpiod_direction_output take a logical value 2014-02-07 09:47:02 +01:00
hardirq.h Fix IRQ flag handling naming 2010-10-07 14:08:55 +01:00
hash.h lib: hash: follow-up fixups for arch hash 2013-12-19 00:14:53 -05:00
hugetlb.h mm: Fix generic hugetlb pte check return type. 2013-10-02 20:02:35 -04:00
hw_irq.h asm-generic: add legacy I/O header files 2009-06-11 21:02:42 +02:00
ide_iops.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
int-ll64.h UAPI: (Scripted) Disintegrate include/asm-generic 2012-10-04 18:20:15 +01:00
io-64-nonatomic-hi-lo.h asm-generic: architecture independent readq/writeq for 32bit environment 2012-02-21 16:47:28 -08:00
io-64-nonatomic-lo-hi.h asm-generic: architecture independent readq/writeq for 32bit environment 2012-02-21 16:47:28 -08:00
io.h Kconfig: rename HAS_IOPORT to HAS_IOPORT_MAP 2014-04-07 16:36:11 -07:00
ioctl.h include/asm-generic/ioctl.h: fix _IOC_TYPECHECK sparse error 2014-06-06 16:08:13 -07:00
iomap.h Kconfig: rename HAS_IOPORT to HAS_IOPORT_MAP 2014-04-07 16:36:11 -07:00
irq.h asm-generic: add generic versions of common headers 2009-06-11 21:02:37 +02:00
irq_regs.h core: Replace __get_cpu_var with __this_cpu_read if not used for an address. 2010-12-17 15:07:19 +01:00
irqflags.h Fix IRQ flag handling naming 2010-10-07 14:08:55 +01:00
Kbuild.asm UAPI: Set up uapi/asm/Kbuild.asm 2012-10-02 18:01:56 +01:00
kdebug.h asm-generic: kdebug.h: Checkpatch cleanup 2010-10-09 21:51:44 +02:00
kmap_types.h asm-generic: remove km_type definitions 2012-07-24 15:27:30 +08:00
kvm_para.h KVM: add kvm_para_available to asm-generic/kvm_para.h 2013-06-05 13:21:29 +03:00
libata-portmap.h libata-portmap: Remove unused definitions 2007-10-12 14:55:37 -04:00
linkage.h asm-generic: add generic versions of common headers 2009-06-11 21:02:37 +02:00
local.h atomic: use <linux/atomic.h> 2011-07-26 16:49:47 -07:00
local64.h atomic: use <linux/atomic.h> 2011-07-26 16:49:47 -07:00
mcs_spinlock.h locking/mcs: Allow architecture specific asm files to be used for contended case 2014-02-09 21:18:52 +01:00
memory_model.h __page_to_pfn: Fix typo in comment 2013-10-14 15:28:29 +02:00
mm_hooks.h [PATCH] x86: PARAVIRT: add hooks to intercept mm creation and destruction 2007-05-02 19:27:14 +02:00
mmu.h asm-generic/mmu.h: Add support for FDPIC 2012-12-09 23:14:14 +01:00
mmu_context.h asm-generic: add generic NOMMU versions of some headers 2009-06-11 21:02:50 +02:00
module.h Make most arch asm/module.h files use asm-generic/module.h 2012-09-28 14:31:03 +09:30
mutex-dec.h arch: Make __mutex_fastpath_lock_retval return whether fastpath succeeded or not 2013-06-26 12:10:55 +02:00
mutex-null.h arch: Make __mutex_fastpath_lock_retval return whether fastpath succeeded or not 2013-06-26 12:10:55 +02:00
mutex-xchg.h arch: Make __mutex_fastpath_lock_retval return whether fastpath succeeded or not 2013-06-26 12:10:55 +02:00
mutex.h asm-generic: add generic versions of common headers 2009-06-11 21:02:37 +02:00
page.h The following changes since commit 3ee72ca992 2012-01-10 17:39:40 -08:00
param.h UAPI: (Scripted) Disintegrate include/asm-generic 2012-10-04 18:20:15 +01:00
parport.h include: remove __dev* attributes. 2013-01-03 15:57:16 -08:00
pci-bridge.h PCI: work around Stratus ftServer broken PCIe hierarchy 2012-04-30 15:21:02 -06:00
pci-dma-compat.h dma-mapping: pci: move pci_set_dma_mask and pci_set_consistent_dma_mask to pci-dma-compat.h 2010-03-12 15:52:42 -08:00
pci.h PCI: collapse pcibios_resource_to_bus 2012-02-23 20:19:04 -07:00
pci_iomap.h [PARISC] fix compile break caused by iomap: make IOPORT/PCI mapping functions conditional 2012-02-27 09:43:30 -06:00
percpu.h percpu: add raw_cpu_ops 2014-04-07 16:36:13 -07:00
pgalloc.h asm-generic: add generic NOMMU versions of some headers 2009-06-11 21:02:50 +02:00
pgtable-nopmd.h mm: Pass virtual address to [__]p{te,ud,md}_free_tlb() 2009-07-27 12:10:38 -07:00
pgtable-nopud.h mm: Pass virtual address to [__]p{te,ud,md}_free_tlb() 2009-07-27 12:10:38 -07:00
pgtable.h x86: define _PAGE_NUMA by reusing software bits on the PMD and PTE levels 2014-06-04 16:53:55 -07:00
preempt.h sched/preempt: Fix preempt_count manipulations 2016-08-22 22:38:00 +01:00
ptrace.h asm-generic/ptrace.h: start a common low level ptrace helper 2011-05-26 17:12:36 -07:00
qrwlock.h locking/rwlocks: Introduce 'qrwlocks' - fair, queued rwlocks 2014-06-06 07:58:28 +02:00
qrwlock_types.h locking/rwlocks: Introduce 'qrwlocks' - fair, queued rwlocks 2014-06-06 07:58:28 +02:00
resource.h asm-generic: remove _STK_LIM_MAX 2014-05-15 00:32:09 +01:00
rtc.h asm-generic: make get_rtc_time overridable 2009-06-11 21:02:18 +02:00
rwsem.h asm-generic: rwsem: de-PPCify rwsem.h 2014-03-14 18:02:08 +00:00
scatterlist.h asm-generic: remove ARCH_HAS_SG_CHAIN in scatterlist.h 2010-05-27 09:12:54 -07:00
sections.h asm/sections: add helpers to check for section data 2017-11-26 13:50:15 +00:00
segment.h asm-generic: add generic NOMMU versions of some headers 2009-06-11 21:02:50 +02:00
serial.h asm-generic: add legacy I/O header files 2009-06-11 21:02:42 +02:00
siginfo.h constify copy_siginfo_to_user{,32}() 2013-11-09 00:16:29 -05:00
signal.h unify default ptrace_signal_deliver 2012-11-29 00:01:23 -05:00
simd.h crypto: create generic version of ablk_helper 2013-09-24 06:02:24 +10:00
sizes.h ARM: 7430/1: sizes.h: move from asm-generic to <linux/sizes.h> 2012-06-28 17:14:34 +01:00
spinlock.h asm-generic: add generic versions of common headers 2009-06-11 21:02:37 +02:00
statfs.h UAPI: (Scripted) Disintegrate include/asm-generic 2012-10-04 18:20:15 +01:00
string.h asm-generic: add generic versions of common headers 2009-06-11 21:02:37 +02:00
switch_to.h Split the switch_to() wrapper out of asm-generic/system.h 2012-03-28 18:30:03 +01:00
syscall.h syscall_get_arch: remove useless function arguments 2014-03-20 10:11:59 -04:00
syscalls.h burying unused conditionals 2013-02-14 09:21:15 -05:00
termios-base.h asm-generic: rename termios.h, signal.h and mman.h 2009-06-11 21:01:52 +02:00
termios.h UAPI: (Scripted) Disintegrate include/asm-generic 2012-10-04 18:20:15 +01:00
timex.h asm-generic: add legacy I/O header files 2009-06-11 21:02:42 +02:00
tlb.h Fix TLB gather virtual address range invalidation corner cases 2013-08-16 08:52:46 -07:00
tlbflush.h BUG: headers with BUG/BUG_ON etc. need linux/bug.h 2012-03-04 17:54:34 -05:00
topology.h cpumask: fix spurious cpumask_of_node() on non-NUMA multi-node configs 2017-11-11 13:33:32 +00:00
trace_clock.h tracing,x86: Add a TSC trace_clock 2012-11-13 15:48:27 -05:00
uaccess-unaligned.h asm-generic: rename page.h and uaccess.h 2009-06-11 21:02:17 +02:00
uaccess.h asm-generic: make get_user() clear the destination on errors 2016-11-20 01:17:26 +00:00
unaligned.h asm-generic: allow generic unaligned access if the arch supports it 2014-05-08 10:22:23 +02:00
unistd.h We get rid of the general module prefix confusion with a binary config option, 2013-05-05 10:58:06 -07:00
user.h asm-generic/user.h: Fix spelling in comment 2011-03-01 15:49:39 +01:00
vga.h asm-generic: add legacy I/O header files 2009-06-11 21:02:42 +02:00
vmlinux.lds.h KAISER: Kernel Address Isolation 2018-01-09 00:35:13 +00:00
vtime.h include/asm-generic/vtime.h: avoid zero-length file 2013-09-30 14:31:02 -07:00
word-at-a-time.h word-at-a-time: simplify big-endian zero_bytemask macro 2014-05-01 08:57:44 -07:00
xor.h asm-generic: xor: mark static functions as __maybe_unused 2012-10-03 21:21:06 +02:00