Commit graph

1,482,976 commits

Author SHA1 Message Date
Chuck Lever
218c56ddf6 SUNRPC: Remove legacy skcipher/ahash handles from krb5_ctx
Previous patches switched all per-message crypto operations
(encrypt, decrypt, get_mic, verify_mic) from the internal
skcipher/ahash primitives to crypto/krb5 AEAD and shash
handles. The old crypto_sync_skcipher and crypto_ahash fields in
struct krb5_ctx are no longer referenced at runtime.

Remove the ten legacy handle fields from struct krb5_ctx
along with the key derivation and handle allocation code in
gss_krb5_import_ctx_v2() that populated them. Context import
now prepares only the four crypto/krb5 handles (two AEAD for
encryption, two shash for checksums). The corresponding cleanup
in gss_krb5_delete_sec_context() and the error path is likewise
reduced.

The krb5_derive_key() inline wrapper, gss_krb5_alloc_cipher_v2(),
and gss_krb5_alloc_hash_v2() become unused and are removed.
The per-enctype encrypt/decrypt functions (gss_krb5_aes_encrypt,
gss_krb5_aes_decrypt, krb5_etm_encrypt, krb5_etm_decrypt) that
were the sole remaining consumers of these fields are also removed;
their function-pointer call sites were already deleted in earlier
patches.

Assisted-by: Claude:claude-opus-4-6
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Acked-by: Anna Schumaker <anna.schumaker@hammerspace.com>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2026-06-09 16:32:59 -04:00
Chuck Lever
5f02e76061 SUNRPC: Remove encrypt/decrypt function pointers from enctype table
All enctypes now route through gss_krb5_aead_encrypt() and
gss_krb5_aead_decrypt(). The per-enctype .encrypt and .decrypt
function pointers served the same purpose as .get_mic and
.wrap before them: dispatching v1 versus v2 implementations.
With v1 support long removed and the Camellia decrypt path
migrated in a preceding patch, every table entry points to
the same pair of functions.

Call gss_krb5_aead_encrypt() and gss_krb5_aead_decrypt()
directly from gss_krb5_wrap_v2() and gss_krb5_unwrap_v2(),
and drop the function pointers from struct gss_krb5_enctype.

While here, propagate the GSS status code returned by
gss_krb5_aead_decrypt() instead of discarding it.
The old indirect call sites returned GSS_S_FAILURE
unconditionally, losing the distinction between an
integrity failure (GSS_S_BAD_SIG) and a structural
error (GSS_S_DEFECTIVE_TOKEN).

Assisted-by: Claude:claude-opus-4-6
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Acked-by: Anna Schumaker <anna.schumaker@hammerspace.com>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2026-06-09 16:32:59 -04:00
Chuck Lever
5fc7721223 SUNRPC: Remove wrap/unwrap function pointers from enctype table
Every enctype points .wrap and .unwrap at gss_krb5_wrap_v2()
and gss_krb5_unwrap_v2(). As with get_mic/verify_mic, the
indirection dates from when v1 enctypes had different wrap
implementations. Call the functions directly and remove the
pointers from struct gss_krb5_enctype.

Assisted-by: Claude:claude-opus-4-6
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Acked-by: Anna Schumaker <anna.schumaker@hammerspace.com>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2026-06-09 16:32:59 -04:00
Chuck Lever
0308f69427 SUNRPC: Remove get_mic/verify_mic function pointers from enctype table
Every enctype in the table points .get_mic and .verify_mic at
the same pair of functions. The indirection served no purpose
after the v1 enctype support was removed. Call
gss_krb5_get_mic_v2() and gss_krb5_verify_mic_v2() directly
from the GSS mechanism dispatch and drop the function pointers
from struct gss_krb5_enctype.

Assisted-by: Claude:claude-opus-4-6
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Acked-by: Anna Schumaker <anna.schumaker@hammerspace.com>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2026-06-09 16:32:59 -04:00
Chuck Lever
d4f6bd01c5 SUNRPC: Switch MIC token verification to crypto/krb5
gss_krb5_verify_mic_v2() currently recomputes a checksum using
gss_krb5_checksum() and then compares it against the received
checksum with memcmp(). Replace this with a call to
crypto_krb5_verify_mic(), which performs the hash, comparison,
and offset/length adjustment in a single operation through the
crypto/krb5 library.

The scatterlist layout required by RFC 4121 Section 4.2.4 is
constructed via gss_krb5_mic_build_sg(), the shared helper
introduced in the preceding commit. The received checksum
occupies the first scatterlist entry, pointing directly into
the token buffer.

The errno result from crypto_krb5_verify_mic() is mapped to a
GSS major status code via gss_krb5_errno_to_status(), which
returns GSS_S_BAD_SIG for -EBADMSG (checksum mismatch).

Assisted-by: Claude:claude-opus-4-6
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Acked-by: Anna Schumaker <anna.schumaker@hammerspace.com>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2026-06-09 16:32:59 -04:00
Chuck Lever
a9b9eb0858 SUNRPC: Switch MIC token generation to crypto/krb5
gss_krb5_get_mic_v2() currently computes the MIC checksum by
driving a crypto_ahash directly, calling gss_krb5_checksum()
with the message body and GSS token header. Replace this with
a call to crypto_krb5_get_mic(), which performs the same keyed
hash operation through the crypto/krb5 library.

RFC 4121 Section 4.2.4 specifies that the checksum covers the
message body followed by the token header. Because the
crypto/krb5 metadata parameter is hashed before the data, the
GSS header cannot be passed as metadata. Instead, the header
is appended to the scatterlist after the body data, producing
the correct hash input ordering without using the metadata
parameter.

The scatterlist layout is:
  [checksum_output | message_body | gss_header]

The first scatterlist entry points directly into the
token buffer, so the checksum is written in place.

A shared helper, gss_krb5_mic_build_sg(), is introduced in
gss_krb5_crypto.c to construct this scatterlist layout. The
helper handles overflow allocation and scatterlist chaining
for large xdr_buf page arrays. It is reused by the verify_mic
counterpart in the following commit.

Assisted-by: Claude:claude-opus-4-6
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Acked-by: Anna Schumaker <anna.schumaker@hammerspace.com>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2026-06-09 16:32:59 -04:00
Chuck Lever
315fc6f2a1 SUNRPC: Switch Camellia decrypt to crypto/krb5
The Camellia enctypes (RFC 6803) use the same MtE authenticated
encryption construction as AES-SHA1 (RFC 3962), implemented in
crypto/krb5 by the rfc3961_simplified profile. The encrypt path
already uses gss_krb5_aead_encrypt() for Camellia, but the decrypt
path was left on the old gss_krb5_aes_decrypt() code when the AES
enctypes were migrated.

Switch the Camellia .decrypt callback to gss_krb5_aead_decrypt() to
complete the AEAD migration for all enctypes. The conf_len and
cksum_len values in crypto/krb5's Camellia enctype descriptors match
the block size and checksum length that gss_krb5_aes_decrypt() was
using, so the headskip and tailskip returned to the unwrap layer are
unchanged.

Assisted-by: Claude:claude-opus-4-6
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Acked-by: Anna Schumaker <anna.schumaker@hammerspace.com>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2026-06-09 16:32:59 -04:00
Chuck Lever
598de25dea SUNRPC: Switch wrap token decryption to crypto/krb5
Replace the per-enctype .decrypt callbacks (gss_krb5_aes_decrypt
and krb5_etm_decrypt) with a single gss_krb5_aead_decrypt()
wrapper that delegates to crypto_krb5_decrypt().

The new wrapper builds a scatterlist covering the secured
region (confounder through checksum), passes it to the AEAD
decrypt operation, and derives the confounder and checksum
lengths from the data offset and length that
crypto_krb5_decrypt() reports. The caller's token header
verification and buffer adjustment logic is unchanged.

Assisted-by: Claude:claude-opus-4-6
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Acked-by: Anna Schumaker <anna.schumaker@hammerspace.com>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2026-06-09 16:32:59 -04:00
Chuck Lever
f1965a3bcb SUNRPC: Switch wrap token encryption to crypto/krb5
Replace the per-enctype .encrypt callbacks (gss_krb5_aes_encrypt and
krb5_etm_encrypt) with a single gss_krb5_aead_encrypt() wrapper that
delegates to crypto_krb5_encrypt().

The xdr_buf setup -- GSS header insertion, confounder space
allocation, and token header copy -- remains unchanged. The
difference is that the CBC-CTS encryption and HMAC computation are
now a single AEAD operation through the crypto/krb5 library. Both
the MtE construction (RFC 3962) and the EtM construction (RFC 8009)
are handled transparently by the AEAD transform.

The plaintext page data must be copied from the page cache pages to
the scratch output pages before building the scatterlist, since the
AEAD operates in-place rather than using separate input and output
scatterlists.

Assisted-by: Claude:claude-opus-4-6
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Acked-by: Anna Schumaker <anna.schumaker@hammerspace.com>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2026-06-09 16:32:59 -04:00
Chuck Lever
25ccbefc9f SUNRPC: Prepare crypto/krb5 encryption and checksum handles
Allocate crypto_aead handles for encryption (one per direction)
and crypto_shash handles for checksumming (one per direction)
using the crypto/krb5 library's key preparation functions.

These four handles derive their subkeys from the session key
and the RFC 4121 usage numbers and are ready for use in
encrypt, decrypt, get_mic, and verify_mic operations.

The existing crypto_sync_skcipher and crypto_ahash handles
remain in place for now; subsequent patches switch the
per-message operations to the new handles and then remove
the old ones.

Assisted-by: Claude:claude-opus-4-6
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Acked-by: Anna Schumaker <anna.schumaker@hammerspace.com>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2026-06-09 16:32:59 -04:00
Chuck Lever
6e5e0c58db SUNRPC: Add errno-to-GSS status conversion helper
The crypto/krb5 library returns standard negative errno values,
but the GSS mechanism layer reports results as GSS_S_* major
status codes. A translation is needed at each call site that
will be switched to the new library.

Rather than open-coding the mapping in every wrapper, provide a
single helper function.

Assisted-by: Claude:claude-opus-4-6
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Acked-by: Anna Schumaker <anna.schumaker@hammerspace.com>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2026-06-09 16:32:59 -04:00
Chuck Lever
e9be933959 SUNRPC: Add helpers to convert xdr_buf byte ranges to scatterlists
The crypto/krb5 library accepts data in scatterlist form, but
the GSS-API layer presents RPC payloads as struct xdr_buf.
Bridge that gap with a pair of helper functions:

  xdr_buf_to_sg()        - populate a caller-supplied scatterlist
                           array from a byte range
  xdr_buf_to_sg_alloc()  - populate a caller-supplied inline
                           scatterlist, chaining to a heap-
                           allocated overflow for large payloads

The inline array (typically stack-allocated at eight entries)
covers the common case of small RPCs with no heap allocation
on the encrypt/decrypt path. Only buffers spanning many pages
incur a kmalloc for the chained extension.

The segment-walking logic follows the same head, page array,
tail traversal as xdr_process_buf(), but populates a
scatterlist directly rather than invoking a per-segment
callback. sg_next() traversal makes the walker safe for
chained scatterlists. Once subsequent patches reroute all
per-message crypto operations through crypto/krb5,
xdr_process_buf() loses its last callers and is removed.

Assisted-by: Claude:claude-opus-4-6
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Acked-by: Anna Schumaker <anna.schumaker@hammerspace.com>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2026-06-09 16:32:59 -04:00
Chuck Lever
f8b942fc38 SUNRPC: Add crypto/krb5 enctype lookup to krb5_ctx
Each krb5_ctx currently points to a gss_krb5_enctype, the
rpcsec_gss_krb5 module's own enctype descriptor. To begin
using the common crypto/krb5 library, store a pointer to the
corresponding struct krb5_enctype (from <crypto/krb5.h>) as
well.

The lookup is performed in gss_import_v2_context() immediately
after the existing gss_krb5_lookup_enctype() call. If
crypto_krb5_find_enctype() cannot find a matching enctype the
context import fails, ensuring the module never operates with
a partially-initialized krb5_ctx.

Assisted-by: Claude:claude-opus-4-6
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Acked-by: Anna Schumaker <anna.schumaker@hammerspace.com>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2026-06-09 16:32:59 -04:00
Chuck Lever
bf8672b994 SUNRPC: Add Kconfig dependency on CRYPTO_KRB5
The rpcsec_gss_krb5 module currently contains its own Kerberos 5
crypto implementation (key derivation, encryption, checksumming)
that duplicates functionality available in the common crypto/krb5
library. As a first step toward migrating to that library, add a
Kconfig select so that building rpcsec_gss_krb5 pulls in the
common Kerberos 5 crypto support.

The per-enctype Kconfig options (AES_SHA1, CAMELLIA, AES_SHA2)
remain: they continue to gate which encryption types are offered
by the GSS mechanism. The individual crypto algorithm selects
they carry become redundant once the migration is complete, since
CRYPTO_KRB5 already selects all needed ciphers and hashes.

Assisted-by: Claude:claude-opus-4-6
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Acked-by: Anna Schumaker <anna.schumaker@hammerspace.com>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2026-06-09 16:32:59 -04:00
Chuck Lever
ae9a0cce07 NFSD: Increase the default max_block_size to 4MB
Commit 8a81f16de6 ("NFSD: Add a "default" block size") introduced
NFSSVC_DEFBLKSIZE at 1MB, well below the 4MB NFSSVC_MAXBLKSIZE
ceiling, with the stated intent that a later change would raise the
default.

Raising the default reduces per-RPC overhead on fast networks by
amortizing header processing and scheduling costs across larger
payloads. The halving loop in nfsd_get_default_max_blksize()
constrains the returned value to 1/4096 of available RAM, so the
new 4MB default takes effect only on systems with at least 16GB of
RAM. Smaller machines continue to receive the same computed value
as before. Administrators can still override the computed value
through /proc/fs/nfsd/max_block_size.

On systems where the new default takes effect,
svc_sock_setbufsize() sizes each service socket's send and receive
buffers as nreqs * max_mesg * 2. Quadrupling max_mesg therefore
quadruples the per-socket buffer reservation at a fixed thread
count, which operators tuning large thread pools should account
for.

Note well: Your NFS client implementation must support large read
and write size settings to benefit from this change.

Reviewed-by: Jeff Layton <jlayton@kernel.org>
Reviewed-by: Roland Mainz <roland.mainz@nrubsig.org>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2026-06-09 16:32:59 -04:00
Chuck Lever
9870b5cb92 NFSD: Close cached file handles when revoking export state
When NFSD_CMD_UNLOCK_EXPORT revokes NFSv4 state for an export path,
GC-managed nfsd_file entries for files under that path may remain
in the file cache.  These cached handles hold the underlying
filesystem busy, preventing a subsequent unmount.

Add nfsd_file_close_export(), which walks the nfsd_file hash table
and closes GC-eligible entries whose underlying file resides on the
same filesystem and is a descendant of the export path.  Because
nfsd_file entries do not carry an export reference, the ancestry
check uses is_subdir() on the file's dentry.  False positives --
closing a cached handle that did not originate from the target
export -- are harmless; the handle is simply reopened on the next
access.

The handler calls nfsd_file_close_export() before revoking NFSv4
state, mirroring the order used by NFSD_CMD_UNLOCK_FILESYSTEM
(which cancels copies and releases NLM locks before revoking
state).  Both calls run under nfsd_mutex.

Reviewed-by: Jeff Layton <jlayton@kernel.org>
Tested-by: Dai Ngo <dai.ngo@oracle.com>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2026-06-09 16:32:59 -04:00
Chuck Lever
2eac189bb0 NFSD: Add NFSD_CMD_UNLOCK_EXPORT netlink command
When a filesystem is exported to NFS clients, NFSv4 state
(opens, locks, delegations, layouts) holds references that
prevent the underlying filesystem from being unmounted.
NFSD_CMD_UNLOCK_FILESYSTEM addresses this at superblock
granularity, but administrators unexporting a single path on a
shared filesystem (e.g., one of several exports on the same device)
need finer control.

Add NFSD_CMD_UNLOCK_EXPORT, which revokes NFSv4 state acquired
through exports of a specific path.  Matching is by path identity
(dentry + vfsmount) via the sc_export field on each nfs4_stid,
so multiple svc_export objects for the same path -- one per
auth_domain -- are handled correctly without requiring the caller
to name a specific client.

The command takes a single "path" attribute.  Userspace (exportfs
-u) sends this after removing the last client for a given path,
enabling the underlying filesystem to be unmounted.  When multiple
clients share an export path, individual unexports do not trigger
state revocation; only the final one does.

Reviewed-by: Jeff Layton <jlayton@kernel.org>
Tested-by: Dai Ngo <dai.ngo@oracle.com>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2026-06-09 16:32:59 -04:00
Chuck Lever
ba0cde5dc8 NFSD: Track svc_export in nfs4_stid
Add an sc_export field to struct nfs4_stid so that each stateid
records the export under which it was acquired.  The export
reference is taken via exp_get() at stateid creation and released
via exp_put() in nfs4_put_stid().

Open stateids record the export from current_fh->fh_export.
Lock stateids and delegations inherit the export from their
parent open stateid. Layout stateids inherit from their
parent stateid. Directory delegations record the export from
cstate->current_fh.

A subsequent commit uses sc_export to scope state revocation to a
specific export, avoiding the need to walk inode dentry aliases at
revocation time.

Reviewed-by: Jeff Layton <jlayton@kernel.org>
Tested-by: Dai Ngo <dai.ngo@oracle.com>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2026-06-09 16:32:59 -04:00
Chuck Lever
8fc2748835 NFSD: Replace idr_for_each_entry_ul in find_one_sb_stid()
Replace idr_for_each_entry_ul() with a while loop over
idr_get_next_ul() for consistency with find_one_export_stid(),
added in a subsequent commit.

No change in behavior.

Reviewed-by: Jeff Layton <jlayton@kernel.org>
Tested-by: Dai Ngo <dai.ngo@oracle.com>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2026-06-09 16:32:59 -04:00
Chuck Lever
327c5168ef NFSD: Add NFSD_CMD_UNLOCK_FILESYSTEM netlink command
Add NFSD_CMD_UNLOCK_FILESYSTEM as a dedicated netlink command for
revoking NFS state under a filesystem path, providing a netlink
equivalent of /proc/fs/nfsd/unlock_fs.

The command requires a "path" string attribute containing the
filesystem path whose state should be released. The handler
resolves the path to its superblock, then cancels async copies,
releases NLM locks, and revokes NFSv4 state on that superblock.

Reviewed-by: Jeff Layton <jlayton@kernel.org>
Tested-by: Dai Ngo <dai.ngo@oracle.com>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2026-06-09 16:32:59 -04:00
Chuck Lever
978cda83de NFSD: Add NFSD_CMD_UNLOCK_IP netlink command
The existing write_unlock_ip procfs interface releases NLM file
locks held by a specific client IP address, but procfs provides
no structured way to extend that operation to other scopes such as
revoking NFSv4 state.

Add NFSD_CMD_UNLOCK_IP as a dedicated netlink command for
releasing NLM locks by client address. The command accepts a
binary sockaddr_in or sockaddr_in6 in its address attribute.
The handler validates the address family and length, then calls
nlmsvc_unlock_all_by_ip() to release matching NLM locks.  Because
lockd is a single global instance, that call operates across
all network namespaces regardless of which namespace the caller
inhabits.

A separate netlink command for filesystem-scoped unlock is added in
a subsequent commit.

The nfsd_ctl_unlock_ip tracepoint is updated from string-based
address logging to __sockaddr, which stores the binary sockaddr
and formats it with %pISpc. This affects both the new netlink path
and the existing procfs write_unlock_ip path, giving consistent
structured output in both cases.

Reviewed-by: Jeff Layton <jlayton@kernel.org>
Tested-by: Dai Ngo <dai.ngo@oracle.com>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2026-06-09 16:32:59 -04:00
Chuck Lever
0a7c2aa584 NFSD: Extract revoke_one_stid() utility function
The per-stateid revocation logic in nfsd4_revoke_states() handles
four stateid types in a deeply nested switch. Extract two helpers:

revoke_ol_stid() performs admin-revocation of an open or lock
stateid with st_mutex already held: marks the stateid as
SC_STATUS_ADMIN_REVOKED, closes POSIX locks for lock stateids,
and releases file access.

revoke_one_stid() dispatches by sc_type, acquires st_mutex with
the appropriate lockdep class for open and lock stateids, and
handles delegation unhash and layout close inline.

No functional change. Preparation for adding export-scoped state
revocation which reuses revoke_one_stid().

Reviewed-by: Jeff Layton <jlayton@kernel.org>
Tested-by: Dai Ngo <dai.ngo@oracle.com>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2026-06-09 16:32:59 -04:00
Chuck Lever
86b9898920 NFSD: Handle layout stid in nfsd4_drop_revoked_stid()
nfsd4_drop_revoked_stid() has no SC_TYPE_LAYOUT case, so when a
client sends FREE_STATEID for an admin-revoked layout stid, the
default branch releases cl_lock and returns without unhashing or
releasing the stid.  The stid remains in the IDR and on the
per-client list until the client is destroyed.

Remove the layout stid from the per-client list and call
nfs4_put_stid() to drop the creation reference.  When the
refcount reaches zero, nfsd4_free_layout_stateid() handles the
remaining cleanup: cancelling the fence worker, removing from
the per-file list, and freeing the slab object.

Fixes: 1e33e1414b ("nfsd: allow layout state to be admin-revoked.")
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Tested-by: Dai Ngo <dai.ngo@oracle.com>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2026-06-09 16:32:59 -04:00
Chuck Lever
c8ae4aef6d NFSD: Update my maintainer email addresses
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2026-06-09 16:32:56 -04:00
Jason Gunthorpe
6ec91df8af iommu/dma: Do not try to iommu_map a 0 length region in swiotlb
iommu_dma_iova_link_swiotlb() processes a mapping that is unaligned in three
parts, the head, middle and trailer. If the middle is empty because there
are no aligned pages it will call down to iommu_map() with a 0 size
which the iommupt implementation will fail as illegal.

It then tries to do an error unwind and starts from the wrong spot
corrupting the mapping so the eventual destruction triggers a WARN_ON.

Check for 0 length and avoid mapping and use offset not 0 as the starting
point to unlink.

This is frequently triggered by using some kinds of thunderbolt NVMe
drives that trigger forced SWIOTLB for unaligned memory. NVMe seems to
pass in oddly aligned buffers for the passthrough commands from smartctl
that hit this condition.

Cc: stable@vger.kernel.org
Fixes: 433a76207d ("dma-mapping: Implement link/unlink ranges API")
Reported-by: Mark Lord <mlord@pobox.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Leon Romanovsky <leonro@nvidia.com>
Reviewed-by: Samiullah Khawaja <skhawaja@google.com>
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
Link: https://lore.kernel.org/r/0-v1-8536728bc89f+469-swiotlb_warn_jgg@nvidia.com
2026-06-09 22:23:42 +02:00
Nicolas Frattaroli
7a921d1118 drm/connector: Update docs of "colorspace" for color format prop
The colorspace property's documentation states that BT2020_RGB and
BT2020_YCC are equivalent, and the output format depends on the driver.

Now that there is a "color format" property that userspace can use to
explicitly set a format, update the colorspace docs to mention this.

The behaviour here is not changed for userspace that doesn't know about
the color format property yet, as the color format property defaults to
"AUTO", where the choice of output format is left up to drivers.

Reviewed-by: Daniel Stone <daniel@fooishbar.org>
Signed-off-by: Nicolas Frattaroli <nicolas.frattaroli@collabora.com>
Link: https://patch.msgid.link/20260609-color-format-v17-24-35739b5782cc@collabora.com
Signed-off-by: Daniel Stone <daniels@collabora.com>
2026-06-09 21:03:51 +01:00
Nicolas Frattaroli
860a13a91c drm/bridge: Document bridge chain format selection
The bridge chain format selection behaviour was, until now,
undocumented. With the addition of the "color format" DRM property, it's
not sufficiently complex enough that documentation is warranted,
especially for driver authors trying to do the right thing.

Add a high-level overview of how the process is supposed to work, and
mention what the display driver is supposed to do if it wants to make
use of this functionality.

Reviewed-by: Maxime Ripard <mripard@kernel.org>
Reviewed-by: Daniel Stone <daniel@fooishbar.org>
Signed-off-by: Nicolas Frattaroli <nicolas.frattaroli@collabora.com>
Link: https://patch.msgid.link/20260609-color-format-v17-23-35739b5782cc@collabora.com
Signed-off-by: Daniel Stone <daniels@collabora.com>
2026-06-09 21:03:51 +01:00
Nicolas Frattaroli
ce1d0139ad drm/tests: bridge: Add test for HDMI output bus formats helper
The common atomic_get_output_bus_fmts helper for HDMI bridge connectors,
called drm_atomic_helper_bridge_get_hdmi_output_bus_fmts, should return
an array of output bus formats depending on the supported formats of the
connector, and the current output BPC.

Add a test to exercise some of this helper.

Reviewed-by: Maxime Ripard <mripard@kernel.org>
Reviewed-by: Daniel Stone <daniel@fooishbar.org>
Signed-off-by: Nicolas Frattaroli <nicolas.frattaroli@collabora.com>
Link: https://patch.msgid.link/20260609-color-format-v17-22-35739b5782cc@collabora.com
Signed-off-by: Daniel Stone <daniels@collabora.com>
2026-06-09 21:03:51 +01:00
Nicolas Frattaroli
082fbc179c drm/tests: bridge: Add KUnit tests for bridge chain format selection
With the "color format" property, the bridge chain format selection has
gained increased complexity. Instead of simply finding any sequence of
bus formats that works, the bridge chain format selection needs to pick
a sequence that results in the requested color format.

Add KUnit tests for this new logic. These take the form of some pleasant
preprocessor macros to make it less cumbersome to define test bridges
with a set of possible input and output formats.

The input and output formats are defined for bridges in the form of
tuples, where the first member defines the input format, and the second
member defines the output format that can be produced from this input
format. This means the tests can construct scenarios in which not all
inputs can be converted to all outputs.

Some tests are added to test interesting scenarios to exercise the bus
format selection in the presence of a specific color format request.

Furthermore, tests are added to verify that bridge chains that end in an
HDMI connector will always prefer RGB when the color format is
DRM_CONNECTOR_COLOR_FORMAT_AUTO, as is the behaviour in the HDMI state
helpers.

Reviewed-by: Maxime Ripard <mripard@kernel.org>
Reviewed-by: Daniel Stone <daniel@fooishbar.org>
Signed-off-by: Nicolas Frattaroli <nicolas.frattaroli@collabora.com>
Link: https://patch.msgid.link/20260609-color-format-v17-21-35739b5782cc@collabora.com
Signed-off-by: Daniel Stone <daniels@collabora.com>
2026-06-09 21:03:51 +01:00
Nicolas Frattaroli
64b92c9f5f drm/tests: hdmi: Add tests for HDMI helper's mode_valid
Add some KUnit tests to verify that the HDMI state helper's mode_valid
implementation does not improperly reject chroma subsampled modes on the
basis of their clock rate not being satisfiable in RGB.

Reviewed-by: Maxime Ripard <mripard@kernel.org>
Reviewed-by: Daniel Stone <daniel@fooishbar.org>
Signed-off-by: Nicolas Frattaroli <nicolas.frattaroli@collabora.com>
Link: https://patch.msgid.link/20260609-color-format-v17-20-35739b5782cc@collabora.com
Signed-off-by: Daniel Stone <daniels@collabora.com>
2026-06-09 21:03:51 +01:00
Nicolas Frattaroli
f7235a4805 drm/tests: hdmi: Add tests for the color_format property
Add some KUnit tests to check the color_format property is working as
expected with the HDMI state helper.

Existing tests are extended to also test the
DRM_CONNECTOR_COLOR_FORMAT_AUTO case, in order to avoid duplicating test
cases. For the explicitly selected color format cases, parameterized
tests are added.

Reviewed-by: Maxime Ripard <mripard@kernel.org>
Reviewed-by: Daniel Stone <daniel@fooishbar.org>
Signed-off-by: Nicolas Frattaroli <nicolas.frattaroli@collabora.com>
Link: https://patch.msgid.link/20260609-color-format-v17-19-35739b5782cc@collabora.com
Signed-off-by: Daniel Stone <daniels@collabora.com>
2026-06-09 21:03:51 +01:00
Nicolas Frattaroli
d84f9fd6f6 drm/connector: Register color format property on HDMI connectors
The drmm_connector_hdmi_init function can figure out what DRM color
formats are supported by a particular connector based on the supported
HDMI format bitmask that's passed in.

Use it to register the drm color format property.

Reviewed-by: Maxime Ripard <mripard@kernel.org>
Reviewed-by: Dmitry Baryshkov <dmitry.baryshkov@oss.qualcomm.com>
Reviewed-by: Daniel Stone <daniel@fooishbar.org>
Signed-off-by: Nicolas Frattaroli <nicolas.frattaroli@collabora.com>
Link: https://patch.msgid.link/20260609-color-format-v17-18-35739b5782cc@collabora.com
Signed-off-by: Daniel Stone <daniels@collabora.com>
2026-06-09 21:03:51 +01:00
Nicolas Frattaroli
894fc3e3f6 drm/rockchip: dw_hdmi_qp: Set supported_formats platdata
With the introduction of the supported_formats member in the
dw-hdmi-qp platform data struct, drivers that have access to this
information should now set it.

Set it in the rockchip dw_hdmi_qp glue driver.

This allows this information to be passed down to the dw-hdmi-qp core,
which sets it in the bridge it creates, and consequently will allow the
common HDMI bridge code to act on it.

Reviewed-by: Cristian Ciocaltea <cristian.ciocaltea@collabora.com>
Reviewed-by: Daniel Stone <daniel@fooishbar.org>
Signed-off-by: Nicolas Frattaroli <nicolas.frattaroli@collabora.com>
Link: https://patch.msgid.link/20260609-color-format-v17-17-35739b5782cc@collabora.com
Signed-off-by: Daniel Stone <daniels@collabora.com>
2026-06-09 21:03:51 +01:00
Nicolas Frattaroli
a18aa0dd7a drm/rockchip: dw_hdmi_qp: Implement "color format" DRM property
Switch between requested color formats by setting the right bus formats,
configuring the VO GRF registers, and setting the right output mode.

To do this, the encoder's atomic_check queries the bus format of the
first bridge, which was determined by the bridge chain recursive format
selection. Pick the input format if it's !FIXED, otherwise, pick the
output format.

The previously unused GRF register color format defines are redone as
well. Both RK3588 and RK3576 use the same defines; it didn't look like
this as there was a typo in the previously (unused) definition.

Reviewed-by: Cristian Ciocaltea <cristian.ciocaltea@collabora.com>
Reviewed-by: Daniel Stone <daniel@fooishbar.org>
Signed-off-by: Nicolas Frattaroli <nicolas.frattaroli@collabora.com>
Link: https://patch.msgid.link/20260609-color-format-v17-16-35739b5782cc@collabora.com
Signed-off-by: Daniel Stone <daniels@collabora.com>
2026-06-09 21:03:51 +01:00
Nicolas Frattaroli
433d8ad4eb drm/bridge: dw-hdmi-qp: Use common HDMI output bus fmts helper
Make use of the common drm_bridge_funcs.atomic_get_output_bus_fmts
helper for HDMI bridge connectors.

This allows dw-hdmi-qp HDMI bridges to participate in recursive bus
format selection in a meaningful way.

Reviewed-by: Cristian Ciocaltea <cristian.ciocaltea@collabora.com>
Reviewed-by: Daniel Stone <daniel@fooishbar.org>
Signed-off-by: Nicolas Frattaroli <nicolas.frattaroli@collabora.com>
Link: https://patch.msgid.link/20260609-color-format-v17-15-35739b5782cc@collabora.com
Signed-off-by: Daniel Stone <daniels@collabora.com>
2026-06-09 21:03:51 +01:00
Nicolas Frattaroli
6937769f04 drm/rockchip: vop2: Set correct output format for RK3576 YUV422
For RK3576 to be able to output YUV422 signals, it first needs to be
able to pick the right output mode in the display controller to do so.

The RK3576 hardware specifies different output formats depending on the
used display protocol.

Adjust the written register value based on the SoC and connector, so
other users of vcstate->output_mode don't have to care about this.

Reviewed-by: Andy Yan <andyshrk@163.com>
Reviewed-by: Daniel Stone <daniel@fooishbar.org>
Signed-off-by: Nicolas Frattaroli <nicolas.frattaroli@collabora.com>
Link: https://patch.msgid.link/20260609-color-format-v17-14-35739b5782cc@collabora.com
Signed-off-by: Daniel Stone <daniels@collabora.com>
2026-06-09 21:03:51 +01:00
Nicolas Frattaroli
c1bfe8dac0 drm/rockchip: vop2: Recognise 10-bit YUV422 as YUV format
The Rockchip VOP2 video output driver has a "is_yuv_output" function,
which returns true when a given bus format is a YUV format, and false
otherwise.

This switch statement is lacking the bus format used for YUV422 10-bit.

Add the two component orderings of the YUV422 10-bit bus formats to the
switch statement.

Fixes: 604be85547 ("drm/rockchip: Add VOP2 driver")
Reviewed-by: Cristian Ciocaltea <cristian.ciocaltea@collabora.com>
Reviewed-by: Daniel Stone <daniel@fooishbar.org>
Signed-off-by: Nicolas Frattaroli <nicolas.frattaroli@collabora.com>
Link: https://patch.msgid.link/20260609-color-format-v17-13-35739b5782cc@collabora.com
Signed-off-by: Daniel Stone <daniels@collabora.com>
2026-06-09 21:03:51 +01:00
Nicolas Frattaroli
ae4a4e6938 drm/rockchip: vop2: Add RK3576 to the RG swap special case
Much like RK3588, RK3576 requires an RG swap to be performed for YUV444
8-bit and YUV444 10-bit bus formats.

Add its version to the already existing check for RK3588, so that YUV444
output is correct on this platform.

Fixes: 944757a4cb ("drm/rockchip: vop2: Add support for rk3576")
Reviewed-by: Andy Yan <andyshrk@163.com>
Reviewed-by: Daniel Stone <daniel@fooishbar.org>
Signed-off-by: Nicolas Frattaroli <nicolas.frattaroli@collabora.com>
Link: https://patch.msgid.link/20260609-color-format-v17-12-35739b5782cc@collabora.com
Signed-off-by: Daniel Stone <daniels@collabora.com>
2026-06-09 21:03:51 +01:00
Nicolas Frattaroli
f424f307de drm/rockchip: Add YUV422 output mode constants for VOP2
The Rockchip display controller has a general YUV422 output mode, and
some SoC-specific connector-specific output modes for RK3576.

Add them, based on the values in downstream and the TRM (dsp_out_mode in
RK3576 TRM Part 2, register POST*_CTRL_POST_DSP_CTRL).

Reviewed-by: Andy Yan <andyshrk@163.com>
Reviewed-by: Daniel Stone <daniel@fooishbar.org>
Signed-off-by: Nicolas Frattaroli <nicolas.frattaroli@collabora.com>
Link: https://patch.msgid.link/20260609-color-format-v17-11-35739b5782cc@collabora.com
Signed-off-by: Daniel Stone <daniels@collabora.com>
2026-06-09 21:03:51 +01:00
Nicolas Frattaroli
8a0343b03b drm/amdgpu: Implement "color format" DRM property
The "color format" DRM property allows userspace to explicitly pick a
color format to use. If an unsupported color format is requested,
userspace will be given an error instead of silently having its request
disobeyed.

The default case, which is AUTO, picks YCbCr 4:2:0 if it's a 4:2:0-only
mode, and RGB in all other cases.

Co-developed-by: Werner Sembach <wse@tuxedocomputers.com>
Signed-off-by: Werner Sembach <wse@tuxedocomputers.com>
Co-developed-by: Andri Yngvason <andri@yngvason.is>
Signed-off-by: Andri Yngvason <andri@yngvason.is>
Co-developed-by: Marius Vlad <marius.vlad@collabora.com>
Signed-off-by: Marius Vlad <marius.vlad@collabora.com>
Reviewed-by: Daniel Stone <daniel@fooishbar.org>
Signed-off-by: Nicolas Frattaroli <nicolas.frattaroli@collabora.com>
Link: https://patch.msgid.link/20260609-color-format-v17-10-35739b5782cc@collabora.com
Signed-off-by: Daniel Stone <daniels@collabora.com>
2026-06-09 21:03:51 +01:00
Nicolas Frattaroli
f532dc0ea5 drm/display: hdmi-state-helper: Try subsampling in mode_valid
drm_hdmi_connector_mode_valid assumes modes are only valid if they work
with RGB. The reality is more complex however: YCbCr 4:2:0
chroma-subsampled modes only require half the pixel clock that the same
mode would require in RGB.

This leads to drm_hdmi_connector_mode_valid rejecting perfectly valid
420-only or 420-also modes.

Fix this by checking whether the mode is 420-capable first. If so, then
proceed by checking it with DRM_OUTPUT_COLOR_FORMAT_YCBCR420 so long as
the connector has legalized 420, otherwise error out. If the mode is not
420-capable, check with RGB as was previously always the case.

Fixes: 47368ab437 ("drm/display: hdmi: add generic mode_valid helper")
Reviewed-by: Maxime Ripard <mripard@kernel.org>
Reviewed-by: Dmitry Baryshkov <dmitry.baryshkov@oss.qualcomm.com>
Reviewed-by: Daniel Stone <daniel@fooishbar.org>
Signed-off-by: Nicolas Frattaroli <nicolas.frattaroli@collabora.com>
Link: https://patch.msgid.link/20260609-color-format-v17-9-35739b5782cc@collabora.com
Signed-off-by: Daniel Stone <daniels@collabora.com>
2026-06-09 21:03:51 +01:00
Nicolas Frattaroli
cb5c6b7e7b drm/display: hdmi-state-helper: Act on color format DRM property
With the introduction of the "color format" DRM property, which allows
userspace to request a specific color format, the HDMI state helper
should implement this.

Implement it by translating the requested drm_connector_color_format to
a drm_output_color_format enum value as per the logic HDMI should use
for this: Auto is translated to RGB, and a fallback to YUV420 is only
performed if the original color format was auto.

Reviewed-by: Dmitry Baryshkov <dmitry.baryshkov@oss.qualcomm.com>
Reviewed-by: Maxime Ripard <mripard@kernel.org>
Reviewed-by: Daniel Stone <daniel@fooishbar.org>
Signed-off-by: Nicolas Frattaroli <nicolas.frattaroli@collabora.com>
Link: https://patch.msgid.link/20260609-color-format-v17-8-35739b5782cc@collabora.com
Signed-off-by: Daniel Stone <daniels@collabora.com>
2026-06-09 21:03:50 +01:00
Nicolas Frattaroli
3f0f770ccc drm/atomic-helper: Add HDMI bridge output bus formats helper
The drm_bridge_funcs atomic_get_output_bus_fmts operation should be the
same for likely every HDMI connector bridge, unless such an HDMI
connector bridge has some special hardware restrictions that I cannot
envision yet.

To avoid code duplication and standardize on a set of media bus formats
that the HDMI output color formats translate to, add a common helper
function that implements this operation to the drm bridge helpers.

The function returns a list of output bus formats based on the HDMI
bridge's current output bits-per-component, and its bitmask of supported
color formats.

To guard against future expansion of DRM_OUTPUT_COLOR_FORMAT outgrowing
the hweight8 call, add a BUILD_BUG_ON statement where it's used that
checks for DRM_OUTPUT_COLOR_FORMAT_COUNT. The justification for not
using hweight32 in all cases is that not all ISAs have a popcount
instruction, and will benefit from a smaller/faster software
implementation that doesn't have to operate across all bits.

The justification for not defining an hweight_color depending on the
value of DRM_OUTPUT_COLOR_FORMAT_COUNT is that this count enum value is
only known at compile time, not at preprocessor time.

Reviewed-by: Dmitry Baryshkov <dmitry.baryshkov@oss.qualcomm.com>
Reviewed-by: Daniel Stone <daniel@fooishbar.org>
Signed-off-by: Nicolas Frattaroli <nicolas.frattaroli@collabora.com>
Link: https://patch.msgid.link/20260609-color-format-v17-7-35739b5782cc@collabora.com
Signed-off-by: Daniel Stone <daniels@collabora.com>
2026-06-09 21:03:50 +01:00
Nicolas Frattaroli
f5873533bf drm/bridge: Act on the DRM color format property
The new DRM color format property allows userspace to request a specific
color format on a connector. In turn, this fills the connector state's
color_format member to switch color formats.

Make drm_bridges consider the color_format set in the connector state
during the atomic bridge check. Call into the connector function to get
the connector state's connector color format. For bridge connectors
including an HDMI bridge, this will make use of whatever the HDMI
implementation set as output formats, and AUTO will never be part of the
rejection logic.

Reject any output bus formats that do not correspond to the requested
color format. DRM_CONNECTOR_COLOR_FORMAT_AUTO is always accepted as a
matching color format for a bus format, meaning that non-HDMI bridge
chains will end up picking the first bus format choice that works, as
has already been the case previously.

Reviewed-by: Dmitry Baryshkov <dmitry.baryshkov@oss.qualcomm.com>
Reviewed-by: Daniel Stone <daniel@fooishbar.org>
Signed-off-by: Nicolas Frattaroli <nicolas.frattaroli@collabora.com>
Link: https://patch.msgid.link/20260609-color-format-v17-6-35739b5782cc@collabora.com
Signed-off-by: Daniel Stone <daniels@collabora.com>
2026-06-09 21:03:50 +01:00
Nicolas Frattaroli
9743ab2c18 drm/display: bridge_connector: Use HDMI color format for HDMI conns
For bridge connectors which contain an HDMI bridge at some stage, the
HDMI state helpers' format selection logic should be involved.

Add an implementation for the drm_bridge_funcs color_format function,
which translates from the HDMI state's output format to a connector
format for bridge connectors involving an HDMI bridge, but return the
connector state's color_format member unchanged otherwise.

Reviewed-by: Dmitry Baryshkov <dmitry.baryshkov@oss.qualcomm.com>
Reviewed-by: Daniel Stone <daniel@fooishbar.org>
Signed-off-by: Nicolas Frattaroli <nicolas.frattaroli@collabora.com>
Link: https://patch.msgid.link/20260609-color-format-v17-5-35739b5782cc@collabora.com
Signed-off-by: Daniel Stone <daniels@collabora.com>
2026-06-09 21:03:50 +01:00
Nicolas Frattaroli
c761e89065 drm/connector: Let connectors have a say in their color format
Add a function to get the connector color format from a connector state,
and a new function pointer in drm_connector_funcs to allow connectors to
override what connector color format it returns.

This is useful for the bridge chain recursive bus format selection code,
which does not wish to implement connector implementation specific
checks like whether it involves HDMI.

Reviewed-by: Dmitry Baryshkov <dmitry.baryshkov@oss.qualcomm.com>
Reviewed-by: Daniel Stone <daniel@fooishbar.org>
Signed-off-by: Nicolas Frattaroli <nicolas.frattaroli@collabora.com>
Link: https://patch.msgid.link/20260609-color-format-v17-4-35739b5782cc@collabora.com
Signed-off-by: Daniel Stone <daniels@collabora.com>
2026-06-09 21:03:50 +01:00
Nicolas Frattaroli
1b61bea237 drm: Add new general DRM property "color format"
Add a new general DRM property named "color format" which can be used by
userspace to request the display driver to output a particular color
format.

Possible string values for the new enum property are:
 - "AUTO" (setup by default, driver internally picks the color format)
 - "RGB"
 - "YUV 4:4:4"
 - "YUV 4:2:2"
 - "YUV 4:2:0"

Drivers should advertise from this list the formats they support in an
optimistic best-case scenario. EDID data from the sink can then be used
in the kernel's atomic check phase to restrict this set of formats, as
well as by userspace to make a correct choice in the first place.

Co-developed-by: Werner Sembach <wse@tuxedocomputers.com>
Signed-off-by: Werner Sembach <wse@tuxedocomputers.com>
Co-developed-by: Andri Yngvason <andri@yngvason.is>
Signed-off-by: Andri Yngvason <andri@yngvason.is>
Signed-off-by: Marius Vlad <marius.vlad@collabora.com>
Reviewed-by: Maxime Ripard <mripard@kernel.org>
Reviewed-by: Daniel Stone <daniel@fooishbar.org>
Signed-off-by: Nicolas Frattaroli <nicolas.frattaroli@collabora.com>
Link: https://patch.msgid.link/20260609-color-format-v17-3-35739b5782cc@collabora.com
Signed-off-by: Daniel Stone <daniels@collabora.com>
2026-06-09 21:03:50 +01:00
Nicolas Frattaroli
851ae5e3be drm/display: hdmi-state-helper: Use default case for unsupported formats
Switch statements that do not handle all possible values of an
enumeration will generate a warning during compilation. In preparation
for adding a COUNT value to the end of the enum, this needs to be dealt
with.

Add a default case to sink_supports_format_bpc's DRM_OUTPUT_COLOR_FORMAT
switch statement, and move the log-and-return unknown pixel format
handling into it.

No functional change.

Reviewed-by: Maxime Ripard <mripard@kernel.org>
Reviewed-by: Cristian Ciocaltea <cristian.ciocaltea@collabora.com>
Reviewed-by: Daniel Stone <daniel@fooishbar.org>
Signed-off-by: Nicolas Frattaroli <nicolas.frattaroli@collabora.com>
Link: https://patch.msgid.link/20260609-color-format-v17-2-35739b5782cc@collabora.com
Signed-off-by: Daniel Stone <daniels@collabora.com>
2026-06-09 21:03:50 +01:00
Werner Sembach
93db402ac7 drm/amd/display: Remove unnecessary SIGNAL_TYPE_HDMI_TYPE_A check
Remove unnecessary SIGNAL_TYPE_HDMI_TYPE_A check that was performed in the
drm_mode_is_420_only() case, but not in the drm_mode_is_420_also() &&
force_yuv420_output case.

Without further knowledge if YCbCr 4:2:0 is supported outside of HDMI,
there is no reason to use RGB when the display
reports drm_mode_is_420_only() even on a non HDMI connection.

This patch also moves both checks in the same if-case. This  eliminates an
extra else-if-case.

Signed-off-by: Werner Sembach <wse@tuxedocomputers.com>
Signed-off-by: Andri Yngvason <andri@yngvason.is>
Tested-by: Andri Yngvason <andri@yngvason.is>
Reviewed-by: Daniel Stone <daniel@fooishbar.org>
Signed-off-by: Nicolas Frattaroli <nicolas.frattaroli@collabora.com>
Link: https://patch.msgid.link/20260609-color-format-v17-1-35739b5782cc@collabora.com
Signed-off-by: Daniel Stone <daniels@collabora.com>
2026-06-09 21:03:50 +01:00
Alexei Starovoitov
c15261b1bb Merge branch 'bpf-lpm_trie-allow-sleepable-bpf-programs-to-use-lpm-tries'
Vlad Poenaru says:

====================
bpf, lpm_trie: Allow sleepable BPF programs to use LPM tries

trie_lookup_elem() annotates its rcu_dereference_check() walks with only
rcu_read_lock_bh_held(), so a sleepable BPF program that touches an LPM
trie (e.g. a sleepable LSM hook calling bpf_map_lookup_elem()) trips a
"suspicious RCU usage" lockdep splat on debug kernels: it holds only
rcu_read_lock_trace(), which that annotation does not accept.

Patch 1 relaxes the rcu_dereference annotations in the trie walks so they
no longer trip lockdep from the Tasks Trace context, including the
trie_update_elem()/trie_delete_elem() writer walks (protected by
trie->lock). Patch 2 adds BPF_MAP_TYPE_LPM_TRIE to the verifier's
sleepable map whitelist so sleepable programs can reference an LPM trie
directly, not just as the inner map of a map-of-maps. LPM trie nodes are
reclaimed via bpf_mem_cache_free_rcu(), which chains a regular RCU grace
period into a Tasks Trace grace period before freeing -- the same
discipline BPF_MAP_TYPE_HASH relies on for sleepable access.

Changes since v1:
- Split into a 2-patch series.
- Patch 1 now also converts the trie_update_elem()/trie_delete_elem()
  walks from rcu_dereference() to rcu_dereference_protected(*p, 1),
  addressing review feedback that v1 only fixed the lookup path and left
  the same splat on the writer paths.
- New patch 2 adds the verifier whitelist entry so the fix is actually
  reachable for directly-referenced LPM tries.
- Retitled v1 ("Allow lookups from sleepable BPF programs").

v1: https://lore.kernel.org/all/20260529174233.2954240-1-vlad.wing@gmail.com/
====================

Link: https://patch.msgid.link/20260609135558.193287-1-vlad.wing@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2026-06-09 12:42:50 -07:00