linux/include
Liao Chang 83179cd678 uprobes: Remove the spinlock within handle_singlestep()
This patch introduces a flag to track TIF_SIGPENDING is suppress
temporarily during the uprobe single-step. Upon uprobe singlestep is
handled and the flag is confirmed, it could resume the TIF_SIGPENDING
directly without acquiring the siglock in most case, then reducing
contention and improving overall performance.

I've use the script developed by Andrii in [1] to run benchmark. The CPU
used was Kunpeng916 (Hi1616), 4 NUMA nodes, 64 cores@2.4GHz running the
kernel on next tree + the optimization for get_xol_insn_slot() [2].

before-opt
----------
uprobe-nop      ( 1 cpus):    0.907 ± 0.003M/s  (  0.907M/s/cpu)
uprobe-nop      ( 2 cpus):    1.676 ± 0.008M/s  (  0.838M/s/cpu)
uprobe-nop      ( 4 cpus):    3.210 ± 0.003M/s  (  0.802M/s/cpu)
uprobe-nop      ( 8 cpus):    4.457 ± 0.003M/s  (  0.557M/s/cpu)
uprobe-nop      (16 cpus):    3.724 ± 0.011M/s  (  0.233M/s/cpu)
uprobe-nop      (32 cpus):    2.761 ± 0.003M/s  (  0.086M/s/cpu)
uprobe-nop      (64 cpus):    1.293 ± 0.015M/s  (  0.020M/s/cpu)

uprobe-push     ( 1 cpus):    0.883 ± 0.001M/s  (  0.883M/s/cpu)
uprobe-push     ( 2 cpus):    1.642 ± 0.005M/s  (  0.821M/s/cpu)
uprobe-push     ( 4 cpus):    3.086 ± 0.002M/s  (  0.771M/s/cpu)
uprobe-push     ( 8 cpus):    3.390 ± 0.003M/s  (  0.424M/s/cpu)
uprobe-push     (16 cpus):    2.652 ± 0.005M/s  (  0.166M/s/cpu)
uprobe-push     (32 cpus):    2.713 ± 0.005M/s  (  0.085M/s/cpu)
uprobe-push     (64 cpus):    1.313 ± 0.009M/s  (  0.021M/s/cpu)

uprobe-ret      ( 1 cpus):    1.774 ± 0.000M/s  (  1.774M/s/cpu)
uprobe-ret      ( 2 cpus):    3.350 ± 0.001M/s  (  1.675M/s/cpu)
uprobe-ret      ( 4 cpus):    6.604 ± 0.000M/s  (  1.651M/s/cpu)
uprobe-ret      ( 8 cpus):    6.706 ± 0.005M/s  (  0.838M/s/cpu)
uprobe-ret      (16 cpus):    5.231 ± 0.001M/s  (  0.327M/s/cpu)
uprobe-ret      (32 cpus):    5.743 ± 0.003M/s  (  0.179M/s/cpu)
uprobe-ret      (64 cpus):    4.726 ± 0.016M/s  (  0.074M/s/cpu)

after-opt
---------
uprobe-nop      ( 1 cpus):    0.985 ± 0.002M/s  (  0.985M/s/cpu)
uprobe-nop      ( 2 cpus):    1.773 ± 0.005M/s  (  0.887M/s/cpu)
uprobe-nop      ( 4 cpus):    3.304 ± 0.001M/s  (  0.826M/s/cpu)
uprobe-nop      ( 8 cpus):    5.328 ± 0.002M/s  (  0.666M/s/cpu)
uprobe-nop      (16 cpus):    6.475 ± 0.002M/s  (  0.405M/s/cpu)
uprobe-nop      (32 cpus):    4.831 ± 0.082M/s  (  0.151M/s/cpu)
uprobe-nop      (64 cpus):    2.564 ± 0.053M/s  (  0.040M/s/cpu)

uprobe-push     ( 1 cpus):    0.964 ± 0.001M/s  (  0.964M/s/cpu)
uprobe-push     ( 2 cpus):    1.766 ± 0.002M/s  (  0.883M/s/cpu)
uprobe-push     ( 4 cpus):    3.290 ± 0.009M/s  (  0.823M/s/cpu)
uprobe-push     ( 8 cpus):    4.670 ± 0.002M/s  (  0.584M/s/cpu)
uprobe-push     (16 cpus):    5.197 ± 0.004M/s  (  0.325M/s/cpu)
uprobe-push     (32 cpus):    5.068 ± 0.161M/s  (  0.158M/s/cpu)
uprobe-push     (64 cpus):    2.605 ± 0.026M/s  (  0.041M/s/cpu)

uprobe-ret      ( 1 cpus):    1.833 ± 0.001M/s  (  1.833M/s/cpu)
uprobe-ret      ( 2 cpus):    3.384 ± 0.003M/s  (  1.692M/s/cpu)
uprobe-ret      ( 4 cpus):    6.677 ± 0.004M/s  (  1.669M/s/cpu)
uprobe-ret      ( 8 cpus):    6.854 ± 0.005M/s  (  0.857M/s/cpu)
uprobe-ret      (16 cpus):    6.508 ± 0.006M/s  (  0.407M/s/cpu)
uprobe-ret      (32 cpus):    5.793 ± 0.009M/s  (  0.181M/s/cpu)
uprobe-ret      (64 cpus):    4.743 ± 0.016M/s  (  0.074M/s/cpu)

Above benchmark results demonstrates a obivious improvement in the
scalability of trig-uprobe-nop and trig-uprobe-push, the peak throughput
of which are from 4.5M/s to 6.4M/s and 3.3M/s to 5.1M/s individually.

[1] https://lore.kernel.org/all/20240731214256.3588718-1-andrii@kernel.org
[2] https://lore.kernel.org/all/20240727094405.1362496-1-liaochang1@huawei.com

Acked-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Acked-by: Oleg Nesterov <oleg@redhat.com>
Signed-off-by: Liao Chang <liaochang1@huawei.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lkml.kernel.org/r/20250124093826.2123675-3-liaochang1@huawei.com
History 2025-02-05 10:29:12 +01:00
..
acpi LoongArch changes for v6.14 2025-01-28 08:52:01 -08:00
asm-generic The various patchsets are summarized below. Plus of course many 2025-01-26 18:36:23 -08:00
clocksource KVM/arm64 updates for 6.14 2025-01-28 09:01:36 -08:00
crypto crypto: skcipher - document skcipher_walk_done() and rename some vars 2025-01-14 11:38:32 +08:00
cxl cxl/events: Update Memory Module Event Record to CXL spec rev 3.1 2025-01-13 09:33:21 -07:00
drm drm/bridge: fix documentation for the hdmi_audio_prepare() callback 2025-01-15 12:32:47 +02:00
dt-bindings Char/Misc/IIO driver updates for 6.14-rc1 2025-01-27 16:51:51 -08:00
hyperv hyperv: Add new Hyper-V headers in include/hyperv 2025-01-10 00:54:20 +00:00
keys keys: drop shadowing dead prototype 2025-01-21 11:25:23 +02:00
kunit linux_kselftest-kunit-6.14-rc1 2025-01-22 12:32:39 -08:00
kvm Merge branch kvm-arm64/pkvm-memshare-declutter into kvmarm-master/next 2025-01-17 11:05:18 +00:00
linux uprobes: Remove the spinlock within handle_singlestep() 2025-02-05 10:29:12 +01:00
math-emu powerpc/math-emu: Remove -w build flag and fix warnings 2022-09-05 17:28:25 +10:00
media media: cec: include linux/debugfs.h and linux/seq_file.h where needed 2025-01-07 12:21:38 +01:00
memory memory: renesas-rpc-if: Remove Runtime PM wrappers 2023-01-23 11:48:00 +01:00
misc cxl: fix typo in comment 2022-11-24 23:12:19 +11:00
net ipsec-2025-01-27 2025-01-27 15:15:12 -08:00
pcmcia ARM: pxa/sa1100: move I/O space to PCI_IOBASE 2022-05-07 22:56:17 +02:00
ras mm/memory-failure: remove obsolete MF_MSG_DIFFERENT_COMPOUND 2024-07-12 15:52:22 -07:00
rdma RDMA/core: Support link status events dispatching 2024-12-24 05:22:18 -05:00
rv rv: Reset per-task monitors also for idle tasks 2025-01-23 12:16:04 -05:00
scsi Driver core and debugfs updates 2025-01-28 12:25:12 -08:00
soc soc: driver updates for 6.14 2025-01-24 14:56:59 -08:00
sound sound updates for 6.14-rc1 2025-01-24 07:54:34 -08:00
target scsi: target: iscsi: switch to using the crc32c library 2024-12-01 17:23:02 -08:00
trace f2fs-for-6.14-rc1 2025-01-27 20:58:58 -08:00
uapi This pull request contains updates for UBI and UBIFS: 2025-01-30 18:27:02 -08:00
ufs scsi: Revert "scsi: ufs: core: Probe for EXT_IID support" 2025-01-10 16:38:21 -05:00
vdso vdso: Correct typo in PAGE_SHIFT comment 2025-01-15 11:07:08 +01:00
video - Improved handling of LCD power states and interactions with the fbdev subsystem. 2024-11-22 16:29:57 -08:00
xen xen: Remove dependency between pciback and privcmd 2024-10-18 11:59:04 +02:00