Unregistering an s390dbf debug area while one of the associated debugfs
files is being written to can cause a deadlock:
$ echo >.../vmur/level $ rmmod vmur
===================================================
debugfs write
debugfs_file_get()
debug_unregister()
mutex_lock(debug_mutex)
debugfs_remove()
wait for debugfs_file_put()
debug_file_ops.write()
debug_input()
mutex_lock(debug_mutex) ==> DEADLOCK
Fix this by splitting debug_unregister() into an s390dbf and debugfs
part, and running only the s390dbf part with debug_mutex locked.
Fixes: 9372a82892 ("s390/debug: fix debug area life cycle")
Signed-off-by: Peter Oberparleiter <oberpar@linux.ibm.com>
Reviewed-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
The command 'perf stat -e cycles -- <command>' crashes the kernel
when CPUs are hotplug added during that run.
Root cause is the allocation of struct cpu_cf_events at first
event initialization. The allocation is dynamic and the first
event that has task context creates such a structure for
each online CPU. This is not sufficient. CPUs may be offline
during event creation and can be set online during the
perf run time. For example commands
# echo 0 > /sys/devices/system/cpu/cpu1/online
# perf stat -e cycles -i -- stress-ng -t10s --matrix X
# sleep 1
# echo 1 > /sys/devices/system/cpu/cpu1/online
create an event for CPUs 0,2-X. Since the events are created with
task-context, the scheduler will eventually schedule the program
on CPU1. This CPU has not created and initialized any per
CPU event infrastructure as that CPU was not online at the time
of the perf invocation. Thus when the scheduler runs stress-ng
on CPU1, the function cpumf_pmu_add() refers to a NULL pointer:
struct cpu_cf_events *cpuhw = this_cpu_cfhw();
This function call is invoked after the task stress-ng has been
made runnable on CPU1. And this_cpu_cfhw() returns NULL.
The result is a panic:
Unable to handle kernel pointer dereference in virtual kernel address space
Failing address: 0000000000000000 TEID: 0000000000000483
....
Krnl PSW : 0404d00180000000 000003ef8291fd0c (cpumf_pmu_add+0x3c/0x80)
....
Call Trace:
[<000003ef8291fd0c>] cpumf_pmu_add+0x3c/0x80
[<000003ef82bb5e3e>] event_sched_in+0xae/0x190
[<000003ef82bb60d6>] merge_sched_in+0x1b6/0x390
[<000003ef82bb65b8>] visit_groups_merge.constprop.0.isra.0+0x308/0x5b0
[<000003ef82bb689a>] pmu_groups_sched_in+0x3a/0x50
[<000003ef82bb6a30>] ctx_sched_in+0x180/0x260
[<000003ef82bb780c>] perf_event_context_sched_in+0x11c/0x2d0
[<000003ef82bb79ee>] __perf_event_task_sched_in+0x2e/0xc0
[<000003ef82994834>] finish_task_switch.isra.0+0x1a4/0x250
....
Last Breaking-Event-Address:
[<000003ef8291f1d8>] this_cpu_cfhw+0x38/0x40
The issue arises only in per-task context when the CPUMF facility is
used and the scheduler picks a random CPU for such a process to run on.
The scheduler enables the CPUMF infrastructure via PMU callback
functions pmu::add() and pmu::del().
Introduce a CPU hotplug prepare/dead callback pair which creates and
removes the per CPU counter data while the CPU is offline. Count the
users which track every CPU (cpu == -1), that is perf_event_open()
events with task context and /dev/hwctr device sessions, in the new
counter cpu_cf_root::tskcnt, protected by pmc_reserve_mutex.
This ensures the infrastructure is available when
new CPU is selected to run the per-task context process.
In cpum_cf_free_root() and cpum_cf_free_cpu() ensure the reference
pointer to data structures is set to NULL before the data is freed
to prevent interrupt handlers to access stale data.
[gor@linux.ibm.com: change commit message]
Fixes: 9b9cf3c77e ("s390/cpum_cf: rework PER_CPU_DEFINE of struct cpu_cf_events")
Cc: stable@vger.kernel.org # v6.5+
Suggested-by: Heiko Carstens <hca@linux.ibm.com>
Suggested-by: Christian Borntraeger <borntraeger@linux.ibm.com>
Assisted-by: Claude:claude-sonnet-5
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Acked-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Enable CONTEXT_ANALYSIS for various directories which do not generate
any warnings (anymore).
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Disable context analysis for various gmap helper functions to get rid
of a few warnings:
arch/s390/mm/gmap_helpers.c:80:1: warning: spinlock 'ptl' is not held on every path through here
arch/s390/mm/gmap_helpers.c:116:2: warning: releasing spinlock 'ptl' that was not held
arch/s390/mm/gmap_helpers.c:186:2: warning: releasing spinlock 'ptl' that was not held
Use __context_unsafe() to give a short comment why for function context
analysis is disabled.
try_get_locked_pte() is disabled since it may return a nonull value
regardless if it returns with a lock held or not.
This cannot be reflected with the context analysis attributes. It is
however possible to workaround this e.g. by adding a another `contended`
function parameter, however this would lead to the next problem:
pte_unmap_unlock() is a macro and therefore doesn't come with the
required context analysis attribute to address this.
For that reason also disable context analysis for
gmap_helper_zap_one_page() and gmap_helper_try_set_pte_unused()
until this has been addressed.
Acked-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Compiling do_secure_storage_access() with context analysis enabled
results in this warning:
arch/s390/mm/fault.c:472:3: warning: releasing spinlock 'fw.ptl' that was not held
472 | folio_walk_end(&fw, vma);
Problem is that folio_walk_end() comes without the required context
analysis attribute. Also the proper attribute cannot be added easily,
since folio_walk_end() is a macro, and not a function.
For the time being disable context analysis only for
do_secure_storage_access() until this is resolved.
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Add context analysis attributes to service_level_start() and
service_level_stop() to specify that those functions only
acquire or release a lock.
Addresses the following warnings:
arch/s390/kernel/sysinfo.c:331:1: warning: rw_semaphore 'service_level_sem' is still held at the end of function
arch/s390/kernel/sysinfo.c:329:2: note: rw_semaphore acquired here
329 | down_read(&service_level_sem);
arch/s390/kernel/sysinfo.c:340:2: warning: releasing rw_semaphore 'service_level_sem' that was not held
340 | up_read(&service_level_sem);
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Add __must_hold() attribute to vfio_ccw_sch_quiesce() in order to let
clang's context analysis know that sch->lock must be held on function
entry. This can also be easily verified when inspecting the function.
Without this annotation this leads to a valid warning when context
analysis is enabled:
drivers/s390/cio/vfio_ccw_drv.c:55:9: warning:
expecting spinlock 'sch->lock' to be held at start of each loop [-Wthread-safety-analysis]
55 | ret = cio_cancel_halt_clear(sch, &iretry);
| ^
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Acked-by: Christian Borntraeger <borntraeger@linux.ibm.com>
Reviewed-by: Matthew Rosato <mjrosato@linux.ibm.com>
Signed-off-by: Christian Borntraeger <borntraeger@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Clang's compiler based static context analysis does not work with locks
that are conditionally taken like in __zpci_event_availability():
arch/s390/pci/pci_event.c:402:10: warning: mutex 'get_zdev_by_fid(ccdf->fid).state_lock'
is not held on every path through here [-Wthread-safety-analysis]
Given that code which takes locks conditionally can be considered
suboptimal rework __zpci_event_availability() to get rid of this.
Reviewed-by: Niklas Schnelle <schnelle@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Clang's compiler based static context analysis does not work with
locks that are conditionally taken like in __zpci_event_error():
arch/s390/pci/pci_event.c:320:2: warning: mutex 'get_zdev_by_fid(ccdf->fid).state_lock'
is not held on every path through here [-Wthread-safety-analysis]
Given that code which takes locks conditionally can be considered
suboptimal rework __zpci_event_error() to get rid of this.
Reviewed-by: Niklas Schnelle <schnelle@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Add __must_hold() attribute to raw3215_make_room() in order to let
clang's context analysis know that "get_ccwdev_lock(raw->cdev)" must be
held on function entry. This can also be easily verified when inspecting
the function.
Without this annotation this leads to a valid warning when context
analysis is enabled:
drivers/s390/char/con3215.c:485:9: warning:
expecting spinlock 'raw->cdev->ccwlock' to be held at start of each loop [-Wthread-safety-analysis]
485 | while (RAW3215_BUFFER_SIZE - raw->count < length) {
Reviewed-by: Sven Schnelle <svens@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
When the TAPQ subfunction of PQAP returns a value between 0 and 3 in the
ml field, the Principles of Operation states that the Maximum AP Message
Length (MAPML) must be 12 KB.
The previous code incorrectly calculated smaller sizes for ml values 1-2:
- ml=1: 4 KB instead of 12 KB
- ml=2: 8 KB instead of 12 KB
While no current hardware reports ml=1 or ml=2, this fix ensures correct
behavior for specification compliance and future-proofing.
The fix ensures all ml values 0-3 correctly use the 12 KB default, with
only values 4+ using the formula ml * 4096.
Reviewed-by: Harald Freudenberger <freude@linux.ibm.com>
Signed-off-by: Finn Callies <fcallies@linux.ibm.com>
Acked-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Since [1] cond_resched() is a no-op on s390. Remove all calls.
[1] commit 7dadeaa6e8 ("sched: Further restrict the preemption modes")
Reviewed-by: Peter Oberparleiter <oberpar@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Since [1] cond_resched() is a no-op on s390. Remove all calls.
[1] commit 7dadeaa6e8 ("sched: Further restrict the preemption modes")
Reviewed-by: Vasily Gorbik <gor@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Since [1] cond_resched() is a no-op on s390. Remove all calls.
This also entirely removes uv_call_sched() and replaces all call sites
with uv_call(), since both functions are identical after the removal
of cond_resched().
[1] commit 7dadeaa6e8 ("sched: Further restrict the preemption modes")
Reviewed-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
With [1] cond_resched() is always compiled away and becomes a no-op.
The comments for all cond_resched() calls in crypto code however indicate
that the current process should be scheduled away to avoid instant
re-invocation of a callback. This is not what cond_resched() would do or
did.
Instead of just removing the cond_resched() calls, replace them with
msleep() calls, as suggested by Holger Dengler. This forces the current
task to be scheduled away (sleeps) like originally intended.
Holger provided information why this intended [2]:
"
Let me give a bit of background here: The protected key can only get
invalid, if the Linux instance (z/VM or KVM guest) is moved to another
hypervisor on a different machine (aka life guest relocation). In such a
case, the crypto accelerator card and the host has to exchange the "real
key", which is wrapped by the host and handed back to the guest as the
re-newed protected key. Unfortunately there is no asynchronous trigger
on completion, you have to re-try (and maybe get another "in progress"
return).
And as if that weren't bad enough, if this key exchange between card and
host is the first one, card and host has to instantiate a secure
communication channel (including a key exchange for the transport layer).
"
[1] commit 7dadeaa6e8 ("sched: Further restrict the preemption modes")
[2] https://lore.kernel.org/all/39570813-27b0-40f9-89c5-8e2dce05e2f0@linux.ibm.com/
[3] https://lore.kernel.org/all/20260731084027.GE776954@noisy.programming.kicks-ass.net/
[hca@linux.ibm.com: took Peter's patch [3] and provided commit message]
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Reviewed-by: Holger Dengler <dengler@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
With all prerequisites in place select ARCH_SUPPORTS_CFI.
Note that this support is supposed to work with the generic kCFI support
which is provided by clang. This comes with a couple of limitations:
The generic kCFI implementation does not generate a .kcfi_traps section,
nor is a special instruction used in case a checksum mismatch is detected.
This means in case of checksum mismatch the kernel just crashes. It should
be quite easy to tell by the surrounding code that a crash happened because
of a checksum mismatch.
If clang and/or gcc provide a .kcfi_traps section it will be possible to
print proper CFI messages instead of just crashing the kernel (enable
ARCH_USES_CFI_TRAPS).
In addition this also means that CFI_PERMISSIVE does not work. Even if the
option is selected the kernel will crash in case of checksum mismatch.
However it seems to be acceptable to enable kCFI support to the kernel now
even if it is not perfect. Later clang and gcc extensions are required to
improve this.
As of now a crash caused by a CFI failure looks like this:
illegal operation: 0001 ilc:1 [#1]SMP
Modules linked in: bpf_testmod(OE)
CPU: 0 UID: 0 PID: 92 Comm: test_progs Tainted: G OE 7.2.0-rc4-00021-gc35ed7a1ca22-dirty #3 PREEMPTLAZY
Tainted: [O]=OOT_MODULE, [E]=UNSIGNED_MODULE
Hardware name: IBM 3931 A01 703 (KVM/Linux)
Krnl PSW : 0704e00180000000 00000166d4853a0a (bpf_task_work_callback+0x176/0x290)
R:0 T:1 IO:1 EX:1 Key:0 M:1 W:0 P:0 AS:3 CC:2 PM:0 RI:0 EA:3
Krnl GPRS: 0400000069b02e96 000001665471856c 0000000084dc1000 000000008084da58
000000008084da60 000000005ff492bf 0000000000000000 00000000809de300
fffffffffff7ffff 00000000000a0337 00000000809e4d00 000000008437b100
00000000801bc288 00000000801bc280 00000166d48538fc 000000e6d502ba90
Krnl Code: 00000166d48539fa: e320c0400004 lg %r2,64(%r12)
00000166d4853a00: e340c0480004 lg %r4,72(%r12)
*00000166d4853a06: a7640001 brc 6,00000166d4853a08
>00000166d4853a0a: 0de1 basr %r14,%r1
00000166d4853a0c: e300b6580095 llh %r0,1624(%r11)
00000166d4853a12: ec040009027e cij %r0,2,4,00000166d4853a24
00000166d4853a18: a70affff ahi %r0,-1
00000166d4853a1c: 4000b658 sth %r0,1624(%r11)
Call Trace:
[<00000166d4853a0a>] bpf_task_work_callback+0x176/0x290
...
The disassembly contains the brc instruction (branch into the instruction
itself with two byte offset to execute an illegal instruction, and the
following basr instruction used for an indirect branch.
Reviewed-by: Jens Remus <jremus@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Tested-by: Nathan Chancellor <nathan@kernel.org>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Use SYM_TYPED_FUNC_START to generate __kcfi_typeid_ symbols for assembler
functions which are called indirectly. All assembler functions contained in
text_amode31.S are called indirectly and require such annotations.
Reviewed-by: Jens Remus <jremus@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Tested-by: Nathan Chancellor <nathan@kernel.org>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
This is the s390 variant of commit f3a0c23f25 ("riscv: Add
ftrace_stub_graph"):
"Commit 883bbbffa5 ("ftrace,kcfi: Separate ftrace_stub() and
ftrace_stub_graph()") added a separate ftrace_stub_graph function for
CFI_CLANG. Add the stub to fix FUNCTION_GRAPH_TRACER compatibility
with CFI."
Reviewed-by: Jens Remus <jremus@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Tested-by: Nathan Chancellor <nathan@kernel.org>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
This is the s390 variant of commit ca7e10bff1 ("x86/tools/relocs: Ignore
__kcfi_typeid_ relocations"):
"The compiler generates __kcfi_typeid_ symbols for annotating assembly
functions with type information. These are constants that can be referenced
in assembly code and are resolved by the linker. Ignore them in relocs."
Reviewed-by: Jens Remus <jremus@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Tested-by: Nathan Chancellor <nathan@kernel.org>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Pass the symbol symbol name which corresponds to a relocation to
do_relocs(). This is preparation for kCFI support.
Given that the s390 specific relocs tool is a stripped down version of the
x86 version, add more code from the x86 version to the s390 version, while
keeping coding style, etc. in order to add the required functionality.
Reviewed-by: Jens Remus <jremus@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Tested-by: Nathan Chancellor <nathan@kernel.org>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
On s390, CPUs can be in a state where it is not possible to hotplug
them online before certain prequisite steps. For example the CPUs which
get introduced during runtime of a system can posses a "deconfigured"
state which prevents them from being hotplugged online before they get
configured. Another case is when users set the configured state of CPUs
themselves via "chcpu" or sysfs attributes.
On s390 available CPUs are being registered as new devices via
smp_add_core() either during boot or after a CPU rescan (for newly added
CPUs during runtime). Registered CPUs are marked as enabled without
considering the configure states. Add necessary checks to smp_add_core()
and userspace configure attribute handler. Reflect the configured CPUs
to cpu_enabled_mask to correctly represent which CPUs can be hotplugged
online.
Signed-off-by: Mete Durlu <meted@linux.ibm.com>
Acked-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Add SCLP Action Qualifiers used by the Spyre stack for reporting of the
card's initialization status, recoverable errors, and telemetry data.
Co-developed-by: Andreas Krebbel <krebbel@linux.ibm.com>
Signed-off-by: Andreas Krebbel <krebbel@linux.ibm.com>
Signed-off-by: Niklas Schnelle <schnelle@linux.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
The queue depth field is defined as a 5 bit field in the Z architecture
instead of a 4 bit field.
The queue depth (qd) can be in range 0-31 and is reported in bits 59-63
of the TAPQ response. Currently this has no effect as all CEX generations
report a queue depth of 7, which fits into 4 bits. However, future CEX
generations reporting a value >15 would not be properly reflected by the
ap bus and therefore all user space applications relying on it.
Reviewed-by: Harald Freudenberger <freude@linux.ibm.com>
Signed-off-by: Finn Callies <fcallies@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
With btrfs and other complex code paths the message
BUG_MAX_LOCKDEP_CHAIN_HLOCKS_TOO_LOW is triggered.
To avoid disabling lockdep increase CONFIG_LOCKDEP_CHAINS_BITS to 18.
Acked-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Christian Borntraeger <borntraeger@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Implement load_unaligned_zeropad() and enable DCACHE_WORD_ACCESS to
speed up string operations in fs/dcache.c and fs/namei.c.
With the secure storage access exception cases addressed, add support
for DCACHE_WORD_ACCESS again.
Tested-by: Christian Borntraeger <borntraeger@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Reviewed-by: Sven Schnelle <svens@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Make do_secure_storage_access() look more like the do_exception()
by using a goto statement to get rid of an extra open coded
mmap_read_unlock() call.
Just cosmetic, no functional change.
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Reviewed-by: Christian Borntraeger <borntraeger@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Similar to the normal exception handler use handle_fault_error() which
handles accesses from user space and kernel space differently.
This also addresses failing kernel accesses via e.g. copy_from_user() to
user space: instead of an endless loop, the fixup handling is performed.
Even though this could only happen if arch_make_folio_accessible() would
permanently fail, which is not realistic.
Therefore this is rather a cleanup patch.
Acked-by: Vasily Gorbik <gor@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Remove folio handling for secure storage access exceptions that happen
via the kernel mapping. Using folio_try_get() for exceptions via the
kernel mapping is racy: for example such an exception may happen on one
CPU, while a different CPU frees the corresponding page. Subsequently the
page can be part of a folio of different size, before the faulting CPU
executes folio_try_get(). This may lead to data corruption.
As of now this scenario does not seem to be possible, however with the
upcoming load_unaligned_zeropad() this is possible.
As a side effect PG_arch_1 is not cleared anymore for such faults via the
kernel mapping. However given that this bit is over-indicating it
shouldn't matter.
Tested-by: Christian Borntraeger <borntraeger@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Acked-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
Reviewed-by: Christian Borntraeger <borntraeger@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Secure storage access exceptions in vmalloc area need to be handled
separately since virt_to_phys() does not work on such addresses. Given
that there cannot be a valid access let such exceptions fail
immediately. The kernel will either fixup or crash.
Tested-by: Christian Borntraeger <borntraeger@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Reviewed-by: Christian Borntraeger <borntraeger@linux.ibm.com>> ---
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
do_secure_storage_access() uses find_vma() without verifying that the
faulting address is within the returned vma. Add this missing check by
converting to lock_mm_and_find_vma().
This is not a critical fix, since the worst that could happen is a
WARN_ON_ONCE() in folio_walk_start().
Reported-by: sashiko-bot <sashiko-bot@kernel.org>
Closes: https://lore.kernel.org/all/20260717093904.E4A421F00A3E@smtp.kernel.org/
Link: https://lore.kernel.org/all/20260717093904.E4A421F00A3E@smtp.kernel.org/
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Reviewed-by: Christian Borntraeger <borntraeger@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Similar like the normal exception handler the secure storage access
exception handler needs to verify if mm is not NULL, which may happen
with kernel threads.
Reported-by: sashiko-bot <sashiko-bot@kernel.org>
Closes: https://lore.kernel.org/all/20260717132345.539A11F00A3A@smtp.kernel.org/
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Reviewed-by: Christian Borntraeger <borntraeger@linux.ibm.com>> ---
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
The guest variable storage area is allocated with vmalloc and then
donated to the ultravisor. Any kernel access to that area will result
in a secure storage access exception (aka fault).
This is a problem if such a memory area is read via /proc/kcore. This
causes an exception via vread_iter() and results in an unexpected short
read. Avoid this by allocating a custom VM_SPARSE area. If such an area
is read, vread_iter() returns zeroes for the entire area.
Note that the function which frees the area does not update ptes. This
is intentional to allow for deferred / lazy pte updates and TLB flushing
like the generic vfree() code is doing that. See vunmap_pte_range().
This assumes that s390 will gain full support for lazy_mmu_mode_enable()
and lazy_mmu_mode_disable() in the future, since as of now the used
ptep_get_and_clear() in vunmap_pte_range() does indeed invalidate and
flush every single pte entry, but only for s390.
Tested-by: Christian Borntraeger <borntraeger@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Reviewed-by: Christian Borntraeger <borntraeger@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Instrument arch_spin_lock_wait() with trace_contention_begin() and
trace_contention_end().
These tracepoints are used by lock contention analysis tools such as
perf lock contention to identify contended locks and measure wait times.
Both the generic implementation and powerpc emit the same events from
their spinlock slow path.
Place the tracepoints in arch_spin_lock_wait(), which is only entered
when lock acquisition falls back to the contention path.
Acked-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Ciunas Bennett <ciunas@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Use explicit decleration on all shutdown_action/shutdown_trigger
declerations and reformat shutdown_actions_list decleration to
improve readability. No functional changes.
Signed-off-by: Mete Durlu <meted@linux.ibm.com>
Acked-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Use ARRAY_SIZE macro instead of reimplementing it.
Signed-off-by: Mete Durlu <meted@linux.ibm.com>
Acked-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Follow the pattern established by commit c33c794828 ("mm:
ptep_get() conversion") and use proper PTE accessors instead
of a direct pointer dereference in memcpy_real_iter().
Signed-off-by: Alexander Gordeev <agordeev@linux.ibm.com>
Reviewed-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
With the pkey rework there was the suggestion to rework the error and
free paths of the pkey ioctl functions. The complain was especially to
rewrite the failure handling with goto instead of all repeat the
nearly same code (kfree(), kfree_sensitive(), memzero_explicit()) for
each path. This patch removes all this duplicated code and introduces
one code block at the end of the functions which is jumped into via
goto out or executed on regular exit. As some helper functions return
an error pointer value (which is NOT NULL) make sure on the error path
there is not by accident kfree() or similar called on such ptr values.
Suggested-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Harald Freudenberger <freude@linux.ibm.com>
Reviewed-by: Finn Callies <fcallies@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
A deep dive into the AP bus code and zcrypt device driver about the
usage of mutex locking showed that there is one questionable call in
ap_bus.c in function ap_bus_force_rescan(). This function may be
called in kernel and process context. In both contexts only one info
is important: was there a AP bus scan running and did it result in
some updates on the AP devices. So only true/false is returned but no
info like -EINTR. But still the mutex lock call should be
interruptible to be able to kill a user space program blocked forever
on this. So this patch replaces mutex_lock_interruptible() with
mutex_lock_killable() to be able to handle SIGKILL especially in user
space process context.
Suggested-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Harald Freudenberger <freude@linux.ibm.com>
Reviewed-by: Finn Callies <fcallies@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
While at it explicitly specify GNU_EH_FRAME PHDR to be read-only.
Inspired by x86 commit 8717b02b8c ("x86/entry/vdso: Include
GNU_PROPERTY and GNU_STACK PHDRs").
Reviewed-by: Ilya Leoshkevich <iii@linux.ibm.com>
Acked-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Jens Remus <jremus@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Commit 2b2a25845d ("s390/vdso: Use $(LD) instead of $(CC) to link
vDSO") accidentally broke the GNU_EH_FRAME program table entry in
the vDSO, causing it to be empty:
$ readelf --program-headers arch/s390/kernel/vdso/vdso.so
...
Program Headers:
Type Offset VirtAddr PhysAddr
FileSiz MemSiz Flags Align
...
GNU_EH_FRAME 0x0000000000000000 0x0000000000000000 0x0000000000000000
0x0000000000000000 0x0000000000000000 0x8
...
Originally, the compiler would implicitly add --eh-frame-hdr when
invoking the linker, but when this Makefile was converted from invoking
the linker via the compiler, to invoking it directly, the option was
missed.
This is the s390 variant of x86 commit cd01544a26 ("x86/vdso: Pass
--eh-frame-hdr to the linker").
Fixes: 2b2a25845d ("s390/vdso: Use $(LD) instead of $(CC) to link vDSO")
Reviewed-by: Ilya Leoshkevich <iii@linux.ibm.com>
Acked-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Jens Remus <jremus@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Make the code a bit easier to read by defining SYSCALL_PER_TRAP
instead of passing '1' to __do_syscall().
Suggested-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Sven Schnelle <svens@linux.ibm.com>
Reviewed-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
PIF_GUEST_FAULT is only used to pass information whether a fault was
caused when executing SIE or when executing host code. Instead of
using ptregs for this, just pass the flag directly as argument to
__do_pgm_check(). This also saves the time required to read the flag
from ptregs, although this likely isn't much as it is already in the
data cache.
Signed-off-by: Sven Schnelle <svens@linux.ibm.com>
Reviewed-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
unistd_32.h is not build anymore since commit 4ac286c4a8
("s390/syscalls: Switch to generic system call table generation").
Thus drop the superfluous line from the Kbuild file now, too.
Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
SWIOTLB_DYNAMIC allows the SWIOTLB to grow on demand when the initial
bounce-buffer area is exhausted, instead of relying only on the pool
reserved at boot. This improves behavior under sustained I/O pressure.
In testing, exhaustion of the initial SWIOTLB area was observed under
real workloads such as large file transfers and package installation
workloads, where continued availability of bounce buffering was needed
to avoid guest-visible I/O stalls or failures.
When the current SWIOTLB capacity is exhausted, the dynamic expansion
path schedules asynchronous pool growth and then attempts immediate
non-blocking allocation of transient bounce-buffer memory.
In that path, the current implementation uses the generic DMA coherent
atomic pool for transient bounce-buffer allocations. Selecting
DMA_COHERENT_POOL together with SWIOTLB_DYNAMIC on s390 makes that
atomic fallback path available.
The coherent atomic pool is a generic DMA facility. By default, its size
is 128 KiB per 1 GiB of memory, with a minimum default size of 128 KiB,
and it can be adjusted with the coherent_pool= kernel parameter. The pool
itself can be exhausted, so this improves behavior under pressure, but
does not guarantee that DMA mappings will always succeed.
Select both options from s390 Kconfig so this support is available by
default on s390 systems.
Signed-off-by: Jaehoon Kim <jhkim@linux.ibm.com>
Acked-by: Niklas Schnelle <schnelle@linux.ibm.com>
Reviewed-by: Halil Pasic <pasic@linux.ibm.com>
Tested-by: Joshua Daley <jdaley@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
