kernel/linux
0
Fork 0
mirror of https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/ synced 2026-10-20 19:23:40 +00:00
Code Issues Releases Packages Activity
No description
1,077,327 commits 117 branches 5,799 tags 5.6 GiB
  • C 96.9%
  • Assembly 0.9%
  • Rust 0.6%
  • Shell 0.6%
  • Python 0.5%
  • Other 0.3%
Find a file
Alexander Grest 88813b736c iommu/arm-smmu-v3: Improve CMDQ lock fairness and efficiency 
[ Upstream commit df180b1a4c ]

The SMMU CMDQ lock is highly contentious when there are multiple CPUs
issuing commands and the queue is nearly full.

The lock has the following states:
 - 0:		Unlocked
 - >0:		Shared lock held with count
 - INT_MIN+N:	Exclusive lock held, where N is the # of shared waiters
 - INT_MIN:	Exclusive lock held, no shared waiters

When multiple CPUs are polling for space in the queue, they attempt to
grab the exclusive lock to update the cons pointer from the hardware. If
they fail to get the lock, they will spin until either the cons pointer
is updated by another CPU.

The current code allows the possibility of shared lock starvation
if there is a constant stream of CPUs trying to grab the exclusive lock.
This leads to severe latency issues and soft lockups.

Consider the following scenario where CPU1's attempt to acquire the
shared lock is starved by CPU2 and CPU0 contending for the exclusive
lock.

CPU0 (exclusive)  | CPU1 (shared)     | CPU2 (exclusive)    | `cmdq->lock`
--------------------------------------------------------------------------
trylock() //takes |                   |                     | 0
                  | shared_lock()     |                     | INT_MIN
                  | fetch_inc()       |                     | INT_MIN
                  | no return         |                     | INT_MIN + 1
                  | spins // VAL >= 0 |                     | INT_MIN + 1
unlock()          | spins...          |                     | INT_MIN + 1
set_release(0)    | spins...          |                     | 0 see[NOTE]
(done)            | (sees 0)          | trylock() // takes  | 0
                  | *exits loop*      | cmpxchg(0, INT_MIN) | 0
                  |                   | *cuts in*           | INT_MIN
                  | cmpxchg(0, 1)     |                     | INT_MIN
                  | fails // != 0     |                     | INT_MIN
                  | spins // VAL >= 0 |                     | INT_MIN
                  | *starved*         |                     | INT_MIN

[NOTE] The current code resets the exclusive lock to 0 regardless of the
state of the lock. This causes two problems:
1. It opens the possibility of back-to-back exclusive locks and the
   downstream effect of starving shared lock.
2. The count of shared lock waiters are lost.

To mitigate this, we release the exclusive lock by only clearing the sign
bit while retaining the shared lock waiter count as a way to avoid
starving the shared lock waiters.

Also deleted cmpxchg loop while trying to acquire the shared lock as it
is not needed. The waiters can see the positive lock count and proceed
immediately after the exclusive lock is released.

Exclusive lock is not starved in that submitters will try exclusive lock
first when new spaces become available.

Reviewed-by: Mostafa Saleh <smostafa@google.com>
Reviewed-by: Nicolin Chen <nicolinc@nvidia.com>
Signed-off-by: Alexander Grest <Alexander.Grest@microsoft.com>
Signed-off-by: Jacob Pan <jacob.pan@linux.microsoft.com>
Signed-off-by: Will Deacon <will@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
History 2026-03-04 07:20:07 -05:00
arch parisc: Prevent interrupts during reboot 2026-03-04 07:20:01 -05:00
block blk-mq-debugfs: add missing debugfs_mutex in blk_mq_debugfs_register_hctxs() 2026-03-04 07:19:59 -05:00
certs certs/blacklist_hashes.c: fix const confusion in certs blacklist 2022-06-22 14:22:01 +02:00
crypto crypto: authencesn - reject too-short AAD (assoclen<8) to match ESP/ESN spec 2026-02-06 16:41:53 +01:00
Documentation PCI: endpoint: Automatically create a function specific attributes group 2026-02-19 16:21:46 +01:00
drivers iommu/arm-smmu-v3: Improve CMDQ lock fairness and efficiency 2026-03-04 07:20:07 -05:00
fs jfs: nlink overflow in jfs_rename 2026-03-04 07:20:07 -05:00
include virt: vbox: uapi: Mark inner unions in packed structs as packed 2026-03-04 07:20:05 -05:00
init init: handle bootloader identifier in kernel parameters 2025-10-19 16:21:45 +02:00
io_uring io_uring: fix filename leak in __io_openat_prep() 2026-01-19 13:09:54 +01:00
ipc ipc: fix to protect IPCS lookups using RCU 2025-06-27 11:05:26 +01:00
kernel bpf: verifier improvement in 32bit shift sign extension pattern 2026-03-04 07:20:00 -05:00
lib lib/crypto: aes: Fix missing MMU protection for AES S-box 2026-01-19 13:10:13 +01:00
LICENSES LICENSES/dual/CC-BY-4.0: Git rid of "smart quotes" 2021-07-15 06:31:24 -06:00
mm mm/kfence: randomize the freelist on initialization 2026-02-11 13:35:27 +01:00
net SUNRPC: fix gss_auth kref leak in gss_alloc_msg error path 2026-03-04 07:19:55 -05:00
samples samples: mei: Fix building on musl libc 2025-08-28 16:24:07 +02:00
scripts modpost: Amend ppc64 save/restfpr symnames for -Os build 2026-03-04 07:20:06 -05:00
security apparmor: fix invalid deref of rawdata when export_binary is unset 2026-03-04 07:19:53 -05:00
sound ALSA: usb-audio: Add iface reset and delay quirk for AB13X USB Audio 2026-03-04 07:20:07 -05:00
tools tools/power cpupower: Reset errno before strtoull() 2026-03-04 07:19:57 -05:00
usr kbuild: hdrcheck: fix cross build with clang 2025-06-27 11:05:22 +01:00
virt KVM: Don't clobber irqfd routing type when deassigning irqfd 2026-02-11 13:35:27 +01:00
.clang-format clang-format: Update with the latest for_each macro list 2021-05-12 23:32:39 +02:00
.cocciconfig scripts: add Linux .cocciconfig for coccinelle 2016-07-22 12:13:39 +02:00
.get_maintainer.ignore Opt out of scripts/get_maintainer.pl 2019-05-16 10:53:40 -07:00
.gitattributes .gitattributes: use 'dts' diff driver for dts files 2019-12-04 19:44:11 -08:00
.gitignore Remove *.orig pattern from .gitignore 2024-10-17 15:11:10 +02:00
.mailmap mailmap: add Andrej Shadura 2021-10-18 20:22:03 -10:00
COPYING COPYING: state that all contributions really are covered by this file 2020-02-10 13:32:20 -08:00
CREDITS MAINTAINERS: Move Daniel Drake to credits 2021-09-21 08:34:58 +03:00
Kbuild kbuild: rename hostprogs-y/always to hostprogs/always-y 2020-02-04 01:53:07 +09:00
Kconfig kbuild: ensure full rebuild when the compiler is updated 2020-05-12 13:28:33 +09:00
MAINTAINERS staging: rtl8712: Remove driver using deprecated API wext 2025-12-07 06:09:36 +09:00
Makefile Linux 5.15.201 2026-02-19 16:21:47 +01:00
README Drop all 00-INDEX files from Documentation/ 2018-09-09 15:08:58 -06:00

README 

Linux kernel
============

There are several guides for kernel developers and users. These guides can
be rendered in a number of formats, like HTML and PDF. Please read
Documentation/admin-guide/README.rst first.

In order to build the documentation, use ``make htmldocs`` or
``make pdfdocs``.  The formatted documentation can also be read online at:

    https://www.kernel.org/doc/html/latest/

There are various text files in the Documentation/ subdirectory,
several of them using the Restructured Text markup notation.

Please read the Documentation/process/changes.rst file, as it contains the
requirements for building and running the kernel, and information about
the problems which may result by upgrading your kernel.