commit 5ec5ac4ca2 upstream.
kasan_init_generic() indicates that kasan is fully initialized, so it
should be put at end of kasan_init().
Otherwise bringing up the primary CPU failed when CONFIG_KASAN is set
on PTW-enabled systems, here are the call chains:
kernel_entry()
start_kernel()
setup_arch()
kasan_init()
kasan_init_generic()
The reason is PTW-enabled systems have speculative accesses which means
memory accesses to the shadow memory after kasan_init() may be executed
by hardware before. However, accessing shadow memory is safe only after
kasan fully initialized because kasan_init() uses a temporary PGD table
until we have populated all levels of shadow page tables and writen the
PGD register. Moving kasan_init_generic() later can defer the occasion
of kasan_enabled(), so as to avoid speculative accesses on shadow pages.
After moving kasan_init_generic() to the end, kasan_init() can no longer
call kasan_mem_to_shadow() for shadow address conversion because it will
always return kasan_early_shadow_page. On the other hand, we should keep
the current logic of kasan_mem_to_shadow() for both the early and final
stage because there may be instrumentation before kasan_init().
To solve this, we factor out a new mem_to_shadow() function from current
kasan_mem_to_shadow() for the shadow address conversion in kasan_init().
Cc: stable@vger.kernel.org
Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn>
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
[ Huacai: To backport from upstream to 6.6 & 6.12, kasan_enabled() is
replaced with kasan_arch_is_ready() and kasan_init_generic()
is replaced with "kasan_early_stage = false". ]
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
56db97af7c