linux/mm
Richard Fellner f9a1666f97 KAISER: Kernel Address Isolation
This patch introduces our implementation of KAISER (Kernel Address Isolation to
have Side-channels Efficiently Removed), a kernel isolation technique to close
hardware side channels on kernel address information.

More information about the patch can be found on:

        https://github.com/IAIK/KAISER

From: Richard Fellner <richard.fellner@student.tugraz.at>
From: Daniel Gruss <daniel.gruss@iaik.tugraz.at>
Subject: [RFC, PATCH] x86_64: KAISER - do not map kernel in user mode
Date: Thu, 4 May 2017 14:26:50 +0200
Link: http://marc.info/?l=linux-kernel&m=149390087310405&w=2
Kaiser-4.10-SHA1: c4b1831d44c6144d3762ccc72f0c4e71a0c713e5

To: <linux-kernel@vger.kernel.org>
To: <kernel-hardening@lists.openwall.com>
Cc: <clementine.maurice@iaik.tugraz.at>
Cc: <moritz.lipp@iaik.tugraz.at>
Cc: Michael Schwarz <michael.schwarz@iaik.tugraz.at>
Cc: Richard Fellner <richard.fellner@student.tugraz.at>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: <kirill.shutemov@linux.intel.com>
Cc: <anders.fogh@gdata-adan.de>

After several recent works [1,2,3] KASLR on x86_64 was basically
considered dead by many researchers. We have been working on an
efficient but effective fix for this problem and found that not mapping
the kernel space when running in user mode is the solution to this
problem [4] (the corresponding paper [5] will be presented at ESSoS17).

With this RFC patch we allow anybody to configure their kernel with the
flag CONFIG_KAISER to add our defense mechanism.

If there are any questions we would love to answer them.
We also appreciate any comments!

Cheers,
Daniel (+ the KAISER team from Graz University of Technology)

[1] http://www.ieee-security.org/TC/SP2013/papers/4977a191.pdf
[2] https://www.blackhat.com/docs/us-16/materials/us-16-Fogh-Using-Undocumented-CPU-Behaviour-To-See-Into-Kernel-Mode-And-Break-KASLR-In-The-Process.pdf
[3] https://www.blackhat.com/docs/us-16/materials/us-16-Jang-Breaking-Kernel-Address-Space-Layout-Randomization-KASLR-With-Intel-TSX.pdf
[4] https://github.com/IAIK/KAISER
[5] https://gruss.cc/files/kaiser.pdf

(cherry picked from Change-Id: I0eb000c33290af01fc4454ca0c701d00f1d30b1d)

Conflicts:
arch/x86/entry/entry_64.S (not in this tree)
arch/x86/kernel/entry_64.S (patched instead of that)
arch/x86/entry/entry_64_compat.S (not in this tree)
arch/x86/ia32/ia32entry.S (patched instead of that)
arch/x86/include/asm/hw_irq.h
arch/x86/include/asm/pgtable_types.h
arch/x86/include/asm/processor.h
arch/x86/kernel/irqinit.c
arch/x86/kernel/process.c
arch/x86/mm/Makefile
arch/x86/mm/pgtable.c
init/main.c

Signed-off-by: Hugh Dickins <hughd@google.com>
[bwh: Folded in the follow-up patches from Hugh:
 - kaiser: merged update
 - kaiser: do not set _PAGE_NX on pgd_none
 - kaiser: stack map PAGE_SIZE at THREAD_SIZE-PAGE_SIZE
 - kaiser: fix build and FIXME in alloc_ldt_struct()
 - kaiser: KAISER depends on SMP
 - kaiser: fix regs to do_nmi() ifndef CONFIG_KAISER
 - kaiser: fix perf crashes
 - kaiser: ENOMEM if kaiser_pagetable_walk() NULL
 - kaiser: tidied up asm/kaiser.h somewhat
 - kaiser: tidied up kaiser_add/remove_mapping slightly
 - kaiser: kaiser_remove_mapping() move along the pgd
 - kaiser: align addition to x86/mm/Makefile
 - kaiser: cleanups while trying for gold link
 - kaiser: name that 0x1000 KAISER_SHADOW_PGD_OFFSET
 - kaiser: delete KAISER_REAL_SWITCH option
 - kaiser: vmstat show NR_KAISERTABLE as nr_overhead
 - kaiser: enhanced by kernel and user PCIDs
 - kaiser: load_new_mm_cr3() let SWITCH_USER_CR3 flush user
 - kaiser: PCID 0 for kernel and 128 for user
 - kaiser: x86_cr3_pcid_noflush and x86_cr3_pcid_user
 - kaiser: paranoid_entry pass cr3 need to paranoid_exit
 - kaiser: _pgd_alloc() without __GFP_REPEAT to avoid stalls
 - kaiser: fix unlikely error in alloc_ldt_struct()
 - kaiser: drop is_atomic arg to kaiser_pagetable_walk()
 Backported to 3.16:
 - Add missing #include in arch/x86/mm/kaiser.c
 - Use variable PEBS buffer size since we have "perf/x86/intel: Use PAGE_SIZE
   for PEBS buffer size on Core2"
 - Renumber X86_FEATURE_INVPCID_SINGLE to avoid collision
 - Adjust context]
Signed-off-by: Ben Hutchings <ben@decadent.org.uk>
History 2018-01-09 00:35:13 +00:00
..
backing-dev.c block: fix bdi vs gendisk lifetime mismatch 2016-11-20 01:17:05 +00:00
balloon_compaction.c mm/balloon_compaction: fix deflation when compaction is disabled 2016-06-15 21:29:35 +01:00
bootmem.c mem-hotplug: reset node managed pages when hot-adding a new pgdat 2014-11-27 11:21:50 +00:00
cleancache.c mm: dump page when hitting a VM_BUG_ON using VM_BUG_ON_PAGE 2014-01-23 16:36:50 -08:00
compaction.c mm/balloon_compaction: redesign ballooned pages management 2016-06-15 21:29:35 +01:00
debug-pagealloc.c mm, x86: Remove debug_pagealloc_enabled 2011-12-06 09:24:07 +01:00
dmapool.c Fix unbalanced mutex in dma_pool_create(). 2014-10-05 13:41:09 -07:00
early_ioremap.c mm: create generic early_ioremap() support 2014-04-07 16:36:15 -07:00
fadvise.c teach SYSCALL_DEFINE<n> how to deal with long long/unsigned long long 2013-03-03 22:46:22 -05:00
failslab.c switch debugfs to umode_t 2012-01-03 22:54:56 -05:00
filemap.c mm: do not access page->mapping directly on page_endio 2017-06-05 21:17:16 +01:00
filemap_xip.c seqcount: Add lockdep functionality to seqcount/seqlock structures 2013-11-06 12:40:26 +01:00
fremap.c mm: mark remap_file_pages() syscall as deprecated 2014-06-06 16:08:17 -07:00
frontswap.c mm: frontswap: invalidate expired data on a dup-store failure 2014-12-10 17:49:41 +00:00
gup.c mm: larger stack guard gap, between vmas 2017-07-02 17:13:02 +01:00
highmem.c Some nice cleanups, and even a patch my wife did as a "live" demo for 2012-12-20 08:37:05 -08:00
huge_memory.c mm, thp: Do not make page table dirty unconditionally in touch_p[mu]d() 2018-01-01 20:52:09 +00:00
hugetlb.c mm/hugetlb: improve locking in dissolve_free_huge_pages() 2017-11-11 13:33:43 +00:00
hugetlb_cgroup.c cgroup: remove css_parent() 2014-05-16 13:22:48 -04:00
hwpoison-inject.c mm/hwpoison: add '#' to hwpoison_inject 2014-01-21 16:19:48 -08:00
init-mm.c mm: Add a user_ns owner to mm_struct and fix ptrace permission checks 2018-01-01 20:52:10 +00:00
internal.h mm, compaction: properly signal and act upon lock and need_sched() contention 2014-06-04 16:54:11 -07:00
interval_tree.c mm: add CONFIG_DEBUG_VM_RB build option 2012-10-09 16:22:42 +09:00
iov_iter.c Fix thinko in iov_iter_single_seg_count 2014-11-27 11:21:48 +00:00
Kconfig mm/zsmalloc: make zsmalloc module-buildable 2014-06-04 16:54:14 -07:00
Kconfig.debug mm: more intensive memory corruption debugging 2012-01-10 16:30:42 -08:00
kmemcheck.c kmemcheck: Fix build errors due to missing slab.h 2010-03-30 22:02:32 +09:00
kmemleak-test.c mm/kmemleak-test.c: use pr_fmt for logging 2014-06-06 16:08:18 -07:00
kmemleak.c mm: kmemleak: allow safe memory scanning during kmemleak disabling 2015-07-15 10:00:54 +01:00
ksm.c mm,ksm: fix endless looping in allocating memory when ksm enable 2016-11-20 01:17:35 +00:00
list_lru.c mm: keep page cache radix tree nodes in check 2014-04-03 16:21:01 -07:00
maccess.c mm: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
madvise.c mm: madvise: fix MADV_WILLNEED on shmem swapouts 2014-05-23 09:37:29 -07:00
Makefile mm: move get_user_pages()-related code to separate file 2014-06-04 16:54:04 -07:00
memblock.c memblock, memhotplug: fix wrong type in memblock_find_in_range_node(). 2014-10-05 13:41:03 -07:00
memcontrol.c memcg: only free spare array when readers are done 2016-02-02 19:09:49 +00:00
memory-failure.c mm/memory-failure.c: use compound_head() flags for huge pages 2017-09-15 18:30:12 +01:00
memory.c ptrace: Don't allow accessing an undumpable mm 2018-01-01 20:52:11 +00:00
memory_hotplug.c mm/memory_hotplug: define find_{smallest|biggest}_section_pfn as unsigned long 2018-01-01 20:51:48 +00:00
mempolicy.c mm/mempolicy: fix use after free when calling get_mempolicy 2017-11-11 13:33:22 +00:00
mempool.c mm/mempool.c: update the kmemleak stack trace for mempool allocations 2014-06-06 16:08:17 -07:00
migrate.c Sanitize 'move_pages()' permission checks 2017-09-15 18:30:18 +01:00
mincore.c mm + fs: prepare for non-page entries in page cache radix trees 2014-04-03 16:21:00 -07:00
mlock.c mlock: fix mlock count can not decrease in race condition 2017-09-15 18:29:55 +01:00
mm_init.c mm: bring back /sys/kernel/mm 2014-01-27 21:02:39 -08:00
mmap.c mm: fix overflow check in expand_upwards() 2017-10-12 15:28:06 +01:00
mmu_context.c mm/mmu_context, sched/core: Fix mmu_context.h assumption 2018-01-09 00:35:08 +00:00
mmu_notifier.c mm: audit/fix non-modular users of module_init in core code 2014-01-23 16:36:52 -08:00
mmzone.c mm: numa: Change page last {nid,pid} into {cpu,pid} 2013-10-09 14:47:45 +02:00
mprotect.c mm: migrate: prevent racy access to tlb_flush_pending 2017-11-11 13:33:19 +00:00
mremap.c mm, thp: close race between mremap() and split_huge_page() 2014-05-11 17:55:48 +09:00
msync.c msync: fix incorrect fstart calculation 2014-07-03 09:21:53 -07:00
nobootmem.c mem-hotplug: reset node managed pages when hot-adding a new pgdat 2014-11-27 11:21:50 +00:00
nommu.c ptrace: Don't allow accessing an undumpable mm 2018-01-01 20:52:11 +00:00
oom_kill.c OOM, PM: OOM killed task shouldn't escape PM suspend 2014-11-05 11:43:09 +00:00
page-writeback.c writeback: use |1 instead of +1 to protect against div by zero 2015-05-20 13:25:57 +01:00
page_alloc.c mm/init: fix zone boundary creation 2017-11-11 13:33:41 +00:00
page_cgroup.c swap: cond_resched in swap_cgroup_prepare() 2017-09-15 18:30:12 +01:00
page_io.c fix __swap_writepage() compile failure on old gcc versions 2014-06-14 19:30:48 -05:00
page_isolation.c mm: fix invalid node in alloc_migrate_target() 2016-05-01 00:06:08 +02:00
pagewalk.c mm: pagewalk: call pte_hole() for VM_PFNMAP during walk_page_range 2015-02-10 13:38:47 +00:00
percpu-km.c percpu: clear memory allocated with the km allocator 2010-10-02 10:28:42 +03:00
percpu-vm.c percpu: perform tlb flush after pcpu_map_pages() failure 2014-10-05 13:41:06 -07:00
percpu.c Revert "percpu: free percpu allocation info for uniprocessor system" 2014-11-14 15:20:06 +00:00
pgtable-generic.c mm: fix TLB flush race between migration, and change_protection_range 2013-12-18 19:04:51 -08:00
process_vm_access.c ptrace: use fsuid, fsgid, effective creds for fs access checks 2017-09-15 18:30:17 +01:00
quicklist.c mm: delete various needless include <linux/module.h> 2011-10-31 09:20:11 -04:00
readahead.c mm/readahead.c: inline ra_submit 2014-04-07 16:35:58 -07:00
rmap.c mm: fix anon_vma->degree underflow in anon_vma endless growing prevention 2015-04-10 10:04:05 +01:00
shmem.c fs: Give dentry to inode_change_ok() instead of inode 2017-02-23 03:53:52 +00:00
slab.c cpuset: PF_SPREAD_PAGE and PF_SPREAD_SLAB should be atomic flags 2017-10-12 15:28:22 +01:00
slab.h memcg, slab: merge memcg_{bind,release}_pages to memcg_{un}charge_slab 2014-06-04 16:54:01 -07:00
slab_common.c Merge branch 'slab/urgent' of git://git.kernel.org/pub/scm/linux/kernel/git/penberg/linux into for-3.16-rcX 2014-07-22 18:38:27 -04:00
slob.c slab: get_online_mems for kmem_cache_{create,destroy,shrink} 2014-06-04 16:53:59 -07:00
slub.c slub/memcg: cure the brainless abuse of sysfs attributes 2017-09-15 18:29:54 +01:00
sparse-vmemmap.c mm/sparse: use memblock apis for early memory allocations 2014-01-21 16:19:47 -08:00
sparse.c mm: use macros from compiler.h instead of __attribute__((...)) 2014-04-07 16:35:54 -07:00
swap.c mm/swap.c: flush lru pvecs on compound page arrival 2017-10-12 15:28:18 +01:00
swap_state.c mm: page_alloc: convert hot/cold parameter and immediate callers to bool 2014-06-04 16:54:09 -07:00
swapfile.c swapfile: fix memory corruption via malformed swapfile 2017-02-23 03:54:32 +00:00
truncate.c mm: Remove false WARN_ON from pagecache_isize_extended() 2014-11-13 11:49:03 +00:00
util.c vm_is_stack: use for_each_thread() rather then buggy while_each_thread() 2014-09-05 16:36:43 -07:00
vmacache.c mm,vmacache: optimize overflow system-wide flushing 2014-06-04 16:53:57 -07:00
vmalloc.c mm/vmalloc.c: export unmap_kernel_range() 2014-06-04 16:54:14 -07:00
vmpressure.c mm: vmpressure: fix sending wrong events on underflow 2017-06-05 21:17:16 +01:00
vmscan.c vmscan: fix increasing nr_isolated incurred by putback unevictable pages 2015-09-29 16:44:51 +01:00
vmstat.c KAISER: Kernel Address Isolation 2018-01-09 00:35:13 +00:00
workingset.c mm: workingset: fix crash in shadow node shrinker caused by replace_page_cache_page() 2016-11-20 01:17:36 +00:00
zbud.c mm/zbud.c: make size unsigned like unique callsite 2014-06-04 16:54:13 -07:00
zsmalloc.c zsmalloc: fixup trivial zs size classes value in comments 2014-06-04 16:54:13 -07:00
zswap.c mm/zswap: NUMA aware allocation for zswap_dstmem 2014-06-04 16:54:14 -07:00