AE QP fatal events and iWARP CM paths load QPs from dev->qp_xa
and then use or reference them outside the xarray lock.
erdma_destroy_qp() can drop the destroy-path reference and free QP
resources while such a lookup is in flight.
Add erdma_qp_get_by_qpn() to acquire a kref under the xarray
lock with kref_get_unless_zero(). Remove the QP from the xarray
before dropping the destroy-path reference so no new lookup can acquire
it while destruction waits for existing users.
Fixes: 1550557717 ("RDMA/erdma: Add verbs implementation")
Signed-off-by: Cheng Xu <chengyou@linux.alibaba.com>
Link: https://patch.msgid.link/20260730124357.12976-2-chengyou@linux.alibaba.com
Signed-off-by: Leon Romanovsky <leon@kernel.org>
EQ handlers look up CQs from dev->cq_xa and invoke CQ completion or
error callbacks outside the xarray lock. erdma_destroy_cq() can erase the
CQ from the xarray and free its queue buffer and doorbell record while a
previously scheduled EQ handler is still using the CQ.
Add a CQ refcount and take a reference under the xarray lock with
refcount_inc_not_zero(). Remove the CQ from the xarray before dropping
the destroy-path reference, then wait for in-flight EQ users before
releasing CQ resources.
Fixes: 1550557717 ("RDMA/erdma: Add verbs implementation")
Signed-off-by: Cheng Xu <chengyou@linux.alibaba.com>
Link: https://patch.msgid.link/20260730124357.12976-1-chengyou@linux.alibaba.com
Signed-off-by: Leon Romanovsky <leon@kernel.org>
Implement UC QP creation in the RNIC HW for user API. An UC QP is exposed
as three work queues: send, receive, and memory management. The latter is
used for bind and invalidate WQEs to support memory windows.
Signed-off-by: Konstantin Taranov <kotaranov@microsoft.com>
Link: https://patch.msgid.link/20260723115955.1859519-3-kotaranov@linux.microsoft.com
Reviewed-by: Long Li <longli@microsoft.com>
Signed-off-by: Leon Romanovsky <leon@kernel.org>
Add helpers to retrieve the send and receive queues of a QP.
Use these helpers when storing queue IDs in the lookup table.
MANA queue IDs are 2-bit aligned, allowing the two least
significant bits to be omitted when storing and looking up
queue IDs.
Signed-off-by: Konstantin Taranov <kotaranov@microsoft.com>
Link: https://patch.msgid.link/20260723115955.1859519-2-kotaranov@linux.microsoft.com
Reviewed-by: Long Li <longli@microsoft.com>
Signed-off-by: Leon Romanovsky <leon@kernel.org>
Now that ib_umem_check_rereg entirely blocks rereg for dmabuf
umems, the extra logic can be removed from irdma.
Signed-off-by: Jacob Moroni <jmoroni@google.com>
Link: https://patch.msgid.link/20260728141501.1425737-2-jmoroni@google.com
Signed-off-by: Leon Romanovsky <leon@kernel.org>
The rereg_mr method has always been explicitly blocked in
rdma-core for dmabuf MRs anyway, so add a check to the
ib_umem_check_rereg helper so that each driver doesn't need
to handle it.
Depending on how the driver handled rereg_mr, this also
has the benefit of preventing rereg_mr from being used
to add the IB_ACCESS_MW_BIND flag to a dmabuf MR. This
flag is not allowed during registration, so it seems
sensible to prevent it from being added back with rereg_mr.
Preventing IB_ACCESS_MW_BIND is important for drivers that
support revocable dmabufs and implement "revoke" by issuing
a dereg_mr command to the HW because most(?) HW will reject
this command if the MR has windows bound to it, and a failure
to revoke is supposed to trigger a function reset.
Signed-off-by: Jacob Moroni <jmoroni@google.com>
Link: https://patch.msgid.link/20260728141501.1425737-1-jmoroni@google.com
Signed-off-by: Leon Romanovsky <leon@kernel.org>
Add a leading "struct" keyword to struct descriptions:
Warning: ../include/uapi/rdma/ib_user_mad.h:72 cannot understand function prototype: 'struct ib_user_mad_hdr_old'
Warning: ../include/uapi/rdma/ib_user_mad.h:116 cannot understand function prototype: 'struct ib_user_mad_hdr'
Warning: ../include/uapi/rdma/ib_user_mad.h:143 cannot understand function prototype: 'struct ib_user_mad'
Warning: ../include/uapi/rdma/ib_user_mad.h:184 cannot understand function prototype: 'struct ib_user_mad_reg_req'
Warning: ../include/uapi/rdma/ib_user_mad.h:216 cannot understand function prototype: 'enum'
One enum and #define are moved out of line so that the struct description
and the struct block are not split (separated).
When this is done, more warnings are exposed:
Warning: include/uapi/rdma/ib_user_mad.h:89 struct member 'length' not described in 'ib_user_mad_hdr_old'
Warning: include/uapi/rdma/ib_user_mad.h:135 struct member 'length' not described in 'ib_user_mad_hdr'
Warning: include/uapi/rdma/ib_user_mad.h:135 struct member 'reserved' not described in 'ib_user_mad_hdr'
Warning: include/uapi/rdma/ib_user_mad.h:227 struct member 'id' not described in 'ib_user_mad_reg_req2'
Warning: include/uapi/rdma/ib_user_mad.h:227 struct member 'qpn' not described in 'ib_user_mad_reg_req2'
Warning: include/uapi/rdma/ib_user_mad.h:227 struct member 'mgmt_class' not described in 'ib_user_mad_reg_req2'
Warning: include/uapi/rdma/ib_user_mad.h:227 struct member 'mgmt_class_version' not described in 'ib_user_mad_reg_req2'
Warning: include/uapi/rdma/ib_user_mad.h:227 struct member 'res' not described in 'ib_user_mad_reg_req2'
Warning: include/uapi/rdma/ib_user_mad.h:227 struct member 'flags' not described in 'ib_user_mad_reg_req2'
Warning: include/uapi/rdma/ib_user_mad.h:227 struct member 'method_mask' not described in 'ib_user_mad_reg_req2'
Warning: include/uapi/rdma/ib_user_mad.h:227 struct member 'oui' not described in 'ib_user_mad_reg_req2'
Warning: include/uapi/rdma/ib_user_mad.h:227 struct member 'rmpp_version' not described in 'ib_user_mad_reg_req2'
Warning: include/uapi/rdma/ib_user_mad.h:227 struct member 'reserved' not described in 'ib_user_mad_reg_req2'
These are all repaired by using ':' as the separator between the struct
member @name and its description (instead of '-').
Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
Link: https://patch.msgid.link/20260728183403.1136827-1-rdunlap@infradead.org
Signed-off-by: Leon Romanovsky <leon@kernel.org>
isert_login_recv_done() records how many bytes the HCA actually placed in
the login buffer, but nothing compares that against the length the login
PDU's BHS declares. isert_rx_login_req() copies min(login_req_len,
MAX_KEY_VALUE_PAIRS) bytes into login->req_buf, and the login code then
reads the declared length back out of that buffer - for the first PDU in
iscsi_target_locate_portal(),
payload_length = ntoh24(login_req->dlength);
tmpbuf = kmemdup_nul(login->req_buf, payload_length, GFP_KERNEL);
and for the ones after it in iscsi_decode_text_input(), reached from
iscsi_target_do_login().
login->req_buf is a fixed MAX_KEY_VALUE_PAIRS (8192) byte allocation, so
an initiator that declares more than it sends reads off the end of it,
before authentication and with the length under its control:
BUG: KASAN: slab-out-of-bounds in kmemdup_nul+0x43/0x80
Read of size 8193 at addr ffff8881056a8000 by task iscsi_np/167
__asan_memcpy+0x23/0x60
kmemdup_nul+0x43/0x80
iscsi_target_locate_portal+0x48d/0x1180
iscsi_target_login_thread+0x19a9/0x3350
Allocated by task 167:
__kmalloc_cache_noprof+0x158/0x370
iscsi_target_login_thread+0x971/0x3350
which belongs to the cache kmalloc-8k of size 8192
allocated 8192-byte region
Falsifying the second login PDU instead reaches the other reader, on the
same buffer:
BUG: KASAN: slab-out-of-bounds in kmemdup_nul+0x43/0x80
Read of size 8193 at addr ffff888104d10000 by task kworker/1:1/50
Workqueue: isert_login_wq iscsi_target_do_login_rx
__asan_memcpy+0x23/0x60
kmemdup_nul+0x43/0x80
iscsi_decode_text_input+0xc6/0x11c0
iscsi_target_do_login+0x261/0x1470
iscsi_target_do_login_rx+0x51d/0x7d0
iscsit over TCP is not exposed: iscsit_get_login_rx() validates the
declared length with iscsi_target_check_login_request() and then reads
exactly that many bytes off the socket, so the declared length governs
how much arrives rather than how much is copied out of an already-filled
buffer. isert does not call iscsi_target_check_login_request() at all.
Reject a login PDU whose declared DataSegmentLength exceeds what was
received, in both paths that reach isert_rx_login_req():
isert_get_login_rx() for the first login PDU and isert_login_recv_done()
for the ones after it. dlength <= login_req_len is allowed because the
received count can include up to three bytes of iSCSI padding.
Once the check is in place the copy out can no longer exceed the copy in:
the posted login SGE is ISER_RX_PAYLOAD_SIZE, so login_req_len cannot
exceed MAX_KEY_VALUE_PAIRS and the min() in isert_rx_login_req() is
login_req_len.
Like the existing short-PDU check added by 29e7b925ae, the reject in
isert_login_recv_done() returns without completing login_req_comp, so a
malformed subsequent PDU leaves the login to be torn down by the login
timer rather than failing immediately. The first-PDU path returns an
error and fails straight away.
Reproduced on 7.2.0-rc4 with soft-RoCE (rdma_rxe) under KASAN, using an
initiator that sends the real key=value payload while declaring 8193 in
the BHS, on the first login PDU and on the second in separate runs. The
reported read size tracks the declared value exactly; 16384 and 61440
behave the same. Unpatched 3 of 3 runs report on each of the two paths,
patched 0 of 3 on both, run alternately in a single session, and a normal
login still completes on the patched build.
Fixes: b8d26b3be8 ("iser-target: Add iSCSI Extensions for RDMA (iSER) target driver")
Suggested-by: Leon Romanovsky <leonro@nvidia.com>
Signed-off-by: Yehyeong Lee <yhlee@isslab.korea.ac.kr>
Link: https://patch.msgid.link/20260726163931.971063-3-yhlee@isslab.korea.ac.kr
Signed-off-by: Leon Romanovsky <leon@kernel.org>
isert_recv_done() hands each received PDU to the opcode handlers without
ever looking at wc->byte_len, the number of bytes the HCA actually placed
in the receive descriptor. The handlers then copy that many bytes - the
data-segment length the initiator declared in the BHS
(ntoh24(hdr->dlength), via the derived unsol_data_len / imm_data_len) -
out of the fixed-size descriptor:
isert_handle_iscsi_dataout():
sg_copy_from_buffer(sg_start, sg_nents, isert_get_data(rx_desc),
unsol_data_len);
isert_handle_scsi_cmd():
sg_copy_from_buffer(cmd->se_cmd.t_data_sg, sg_nents,
isert_get_data(rx_desc), imm_data_len);
Because the declared length is never checked against wc->byte_len, an
initiator can declare a data segment larger than the bytes it actually
sent (and larger than the descriptor) and cause an out-of-bounds read of
the receive buffer.
Nothing upstream of isert closes this door:
- __iscsit_check_dataout_hdr() bounds the inbound payload against
conn_ops->MaxXmitDataSegmentLength (MXDSL) - a transmit parameter,
used here for the inbound check.
- iscsi_set_connection_parameters() sets
ops->MaxXmitDataSegmentLength = ops->TargetRecvDataSegmentLength;
and TARGETRECVDATASEGMENTLENGTH is absent from the min()-clamp list in
iscsi_check_acceptor_state(), so the value the initiator declares is
adopted verbatim (type range 512..16777215). The initiator effectively
raises its own ceiling.
- isert never clamps the negotiated value to its own fixed receive
descriptor (ISER_RX_SIZE, 9216 bytes), so the target core's bound and
the descriptor size are unrelated.
The imm_data_len == data_len path is more than an over-read: it aliases
the receive descriptor via sg_set_buf() and passes it to the backend as
the data source for the SCSI WRITE, so an over-declared length causes heap
contents past the descriptor to be written through the backend to the
backing store. The backend is the victim of the oversized scatterlist
isert hands it, not the cause; no read-back of the written bytes was
demonstrated.
Trigger: after login completes (full feature phase), an initiator that has
declared a large TargetRecvDataSegmentLength and a FirstBurstLength that
permits unsolicited/immediate data sends a PDU whose declared data-segment
length exceeds what was received. With KASAN:
BUG: KASAN: slab-out-of-bounds in sg_copy_buffer+0x150/0x1c0
Read of size 4096 at addr ffff888109720800 by task kworker/1:0H/25
Workqueue: ib-comp-wq ib_cq_poll_work
Call Trace:
sg_copy_buffer+0x150/0x1c0
isert_recv_done+0xba6/0x2390
__ib_process_cq+0xe1/0x390
ib_cq_poll_work+0x46/0x150
isert_recv_done+0xba6 resolves to isert_handle_iscsi_dataout()
(ib_isert.c:1160), inlined through isert_rx_opcode().
Validate wc->byte_len against the framing in isert_recv_done() before the
PDU reaches any handler, and reinstate the connection if it is short.
Because the test compares without subtracting the header length, it also
rejects PDUs shorter than the iSER and iSCSI headers, which would otherwise
be parsed out of stale descriptor contents. The login handler rejects PDUs
shorter than ISER_HEADERS_LEN (commit 29e7b925ae ("IB/isert: Reject login
PDUs shorter than ISER_HEADERS_LEN")) but does not bound the declared
length either; that is fixed in the next patch. The data handlers had no
length check at all.
isert reads the data segment from a fixed offset: isert_get_data()
returns the iSER header plus ISER_HEADERS_LEN and makes no adjustment for
an AHS. The bytes the handlers touch are therefore exactly
[ISER_HEADERS_LEN, ISER_HEADERS_LEN + dlength), and comparing that sum
against wc->byte_len bounds precisely the region that is read. An AHS
term would only make the test stricter without bounding anything further,
and cannot cause a false reject: a PDU carrying an AHS is longer, not
shorter.
This is a memory-safety fix that verifies the bytes that were actually
received; it does not touch RFC 7145 length negotiation and is not the
MaxXmitDataSegmentLength negotiation redesign raised in the 2017 "[Query]
iSER-Target: QP errors observed on increasing MaxXmitDataSegmentLength"
discussion. That redesign is explicitly out of scope here.
The patched kernel rejects the malformed DataOut PDU and both
immediate-data variants with "PDU declares ... bytes were received" and
continues to pass normal traffic with no regression.
Reproduced with soft-RoCE (rdma_rxe) and a raw rdma_cm/ibv initiator; no
kernel-side test hooks were needed.
Fixes: b8d26b3be8 ("iser-target: Add iSCSI Extensions for RDMA (iSER) target driver")
Signed-off-by: Yehyeong Lee <yhlee@isslab.korea.ac.kr>
Link: https://patch.msgid.link/20260726163931.971063-2-yhlee@isslab.korea.ac.kr
Signed-off-by: Leon Romanovsky <leon@kernel.org>
write_tpt_entry() allocates a new STAG index with c4iw_get_resource() and
bumps stats.stag.cur before programming the entry. When
write_adapter_mem() fails, it returns the error without releasing the index
or reversing the statistic. No MR is inserted into rhp->mrs, so
deregistration never reclaims it, leaking the index until device teardown.
Record whether this call allocated the index and, on a failed write, return
it to tpt_table and decrement stats.stag.cur. Key the rollback on both the
write error and that flag, not the error alone: a non-reset update carries
a caller-owned STAG that this call did not allocate and must not free.
Fixes: ec3eead217 ("RDMA/cxgb4: Remove kfifo usage")
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
mlx5_ib_set_cc_params() resolves the port-specific mlx5_core_dev via
mlx5_ib_get_native_port_mdev() but issued MLX5_CMD_OP_MODIFY_CONG_PARAMS
through dev->mdev. On an affiliated secondary RoCE port those pointers
refer to different devices, so a write to the secondary port's cc_params
debugfs file either altered the master port or failed with a master-side
command error, while the read path already used the resolved mdev and
returned the unchanged secondary value.
Issue the command to the resolved mdev, the same device whose capabilities
were checked when its debugfs directory was created. It is already
referenced by the get/put pair, so its lifetime is safe.
Fixes: 31578defe4 ("RDMA/mlx5: Update mlx5_ib to use new cmd interface")
Link: https://patch.msgid.link/20260726-mlx5-ib-set-cc-params-applies-conges-v1-1-a253edafe1f3@nvidia.com
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
get_param() reads a congestion parameter as a u32 but formats it with the
signed "%d" into an 11-byte stack buffer. A value with bit 31 set, such as
0x80000000, renders as "-2147483648\n" whose full length is 12. snprintf()
stores only 11 bytes yet returns 12, so simple_read_from_buffer() treats 12
bytes as valid and reads one byte past lbuf[].
Size the buffer for the widest unsigned decimal, format with "%u" to match
the u32, and use scnprintf() so the length passed to
simple_read_from_buffer() reflects the bytes actually stored.
Fixes: 4a2da0b8c0 ("IB/mlx5: Add debug control parameters for congestion control")
Link: https://patch.msgid.link/20260726-get-param-leaks-kernel-stack-memory-v1-1-d61a4d39662d@nvidia.com
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
When the device is in the fatal error state, write_tpt_entry() returns -EIO
before handing the caller's preallocated skb to the transmit path; its
allocation-failure returns do the same. c4iw_dereg_mr() ignores the error
and frees mhp, leaking mhp->dereg_skb. c4iw_get_dma_mr() instead frees the
skb a second time after dereg_mem() already consumed it, a double free.
Make write_tpt_entry() the sole owner of a non-NULL skb, freeing it on
every return preceding handoff to c4iw_ofld_send(): fatal error, tpt and
stag allocation failure. c4iw_ofld_send() consumes the skb on success and
error alike, so drop the redundant kfree_skb() in c4iw_get_dma_mr() after
dereg_mem().
Fixes: 0f8ab0b6e9 ("RDMA/iw_cxgb4: Low resource fixes for Memory registration")
Link: https://patch.msgid.link/20260726-leaked-mhp-dereg-skb-in-c4iw-dereg-m-v1-1-ebd6df364d53@nvidia.com
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
On register MR, when creating the PBL, if it's an indirect PBL we create
a chunk list to hold the PBL pages pointers. Each chunk is 4KB in size
and can hold 510 addresses (EFA_PTRS_PER_CHUNK) and has a 12-byte
control buffer at the end of it holding the next chunk's pointer and its
length.
If the PBL number of pages is a multiple of EFA_PTRS_PER_CHUNK, the
calculated last chunk length is wrongly computed as 0, even though that
chunk is fully populated with 510 real page pointers. This wrong length
is used both to DMA map the chunk and is propagated to the device,
causing the device to see the chunk as empty and reject the memory
registration.
Fix the calculation so it will be performed only if the number of pages
isn't a multiple of EFA_PTRS_PER_CHUNK, if it is, its already handled in
the above loop correctly.
Also prevent out-of-bounds reach in the chunks array in such scenario.
Fixes: 40909f664d ("RDMA/efa: Add EFA verbs implementation")
Reviewed-by: Firas Jahjah <firasj@amazon.com>
Reviewed-by: Michael Margolin <mrgolin@amazon.com>
Signed-off-by: Yonatan Nachum <ynachum@amazon.com>
Link: https://patch.msgid.link/20260727090255.1175120-1-ynachum@amazon.com
Signed-off-by: Leon Romanovsky <leon@kernel.org>
rxe_odp_mr_init_user() stores &umem_odp->umem in mr->umem before
calling rxe_odp_init_pages(). If rxe_odp_init_pages() fails,
rxe_odp_mr_init_user() releases umem_odp and returns an error.
rxe_reg_user_mr() then unwinds the error through rxe_cleanup(),
rxe_mr_cleanup(), ib_umem_release(mr->umem). There is an
IS_ERR_OR_NULL(umem) check at the start of ib_umem_release().
But since mr->umem is NOT reset to NULL in the error handling
path of rxe_odp_mr_init_user(), the check passes and it reads
already-freed fields like umem->is_dmabuf, causing UAF.
Fix the UAF by clearing mr->umem after releasing the failed
ODP umem so the MR cleanup path does not release it again.
Fixes: d03fb5c659 ("RDMA/rxe: Allow registering MRs for On-Demand Paging")
Assisted-by: Codex:gpt-5.6-sol
Signed-off-by: Peiyang He <peiyang_he@smail.nju.edu.cn>
Link: https://patch.msgid.link/70CB6DBCB19624C7+20260727050659.1543627-1-peiyang_he@smail.nju.edu.cn
Signed-off-by: Leon Romanovsky <leon@kernel.org>
The current GET_TOGGLE_MEM ioctl requires the caller to supply
a type enum and a raw hardware queue ID (RES_ID). The kernel
looks up the CQ or SRQ by that ID without verifying that the
caller owns the resource.
Add a new, preferred code path that accepts standard uverbs
object handles (BNXT_RE_TOGGLE_MEM_CQ_HANDLE /
BNXT_RE_TOGGLE_MEM_SRQ_HANDLE) instead. The uverbs core validates
that the handle belongs to the calling context as part of resolving
it, so this path no longer needs the driver's own XArray lookup for
ownership checking. As with the legacy path, the toggle_entry's own
mmap-entry refcount (not a CQ/SRQ uobject reference) is what pins
the toggle page for the life of the GET_TOGGLE_MEM handle.
Only newer rdma-core versions support this path, if the
driver reports the supported resp mask
(BNXT_RE_UCNTX_CMASK_TOGGLE_MEM_UOBJ_SUPPORT).
The existing TYPE + RES_ID path is retained for backward
compatibility with older rdma-core.
Suggested-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Selvin Xavier <selvin.xavier@broadcom.com>
Signed-off-by: Leon Romanovsky <leon@kernel.org>
Fix the page lifetime by making the rdma_user_mmap_entry the sole owner
of the toggle page allocation. Creating the rdma_user_mmap_entry and page
during the CQ/SRQ creation time. Freeing the page is handled when the
mmap free is called. Introduce struct bnxt_re_toggle_mem to carry
the mmap_offset for the lifetime of the GET_TOGGLE_MEM uobject handle.
bnxt_re_destroy_cq/srq can erase the entry from the XArray and call
rdma_user_mmap_entry_remove() on the toggle_entry concurrently with
the caller's xa_load() and its subsequent use of that toggle_entry.
Guard against this by taking an extra kref directly on the
toggle_entry's rdma_user_mmap_entry while the GET_TOGGLE_MEM handle
exists, released when the handle is destroyed. This pins exactly the
resource that GET_TOGGLE_MEM hands out (the mmap offset/page),
independent of the CQ/SRQ's own lifetime.
Signed-off-by: Selvin Xavier <selvin.xavier@broadcom.com>
Signed-off-by: Leon Romanovsky <leon@kernel.org>
The CQ and SRQ hash tables (cq_hash, srq_hash) on struct bnxt_re_dev
were used exclusively to look up a toggle-page pointer from a
user-space-supplied hardware queue ID in the GET_TOGGLE_MEM
ioctl handler. This approach has couple of problems. First,
because the tables are per-device, any user can look up another
user's CQ or SRQ by guessing the hardware queue ID. Second,
concurrent add and remove operations on the hash table are not
protected by any lock, leaving a race window.
The correct fix is to retrieve the CQ and SRQ objects via the uverbs
object handle, which gives built-in ownership verification and reference
pinning for the duration of the ioctl. That is added in a later patch of
this series.
To maintain backward compatibility with older rdma-core versions that
do not send a uverbs object handle, the driver must continue to support
the existing TYPE + RES_ID lookup path. This patch replaces the per-device
hash tables with per-ucontext XArrays (cq_xa and srq_xa on struct
bnxt_re_ucontext), which narrows the lookup scope to the calling context,
eliminating the cross-user visibility. Also adds Xarray locking mechanism
for synchronization.
The GET_TOGGLE_MEM ioctl handler is updated to call xa_load()
in place of the now-removed bnxt_re_search_for_cq()/
bnxt_re_search_for_srq() helpers. No ABI changes are required.
bnxt_re_create_user_cq()/bnxt_re_create_srq() publish the uobject into
cq_xa/srq_xa before returning to the uverbs core, but the core only
sets uobject->object once the create callback has returned success.
Guard the lookup against this so a concurrent GET_TOGGLE_MEM racing an
in-progress create cannot feed a NULL ->object into container_of().
Signed-off-by: Selvin Xavier <selvin.xavier@broadcom.com>
Signed-off-by: Leon Romanovsky <leon@kernel.org>
New HW no longer implements a separate class for latency-sensitive QPs and
advertises this by a new cap bit.
REG_UMR is the only QP that used this feature, so gate its usage of the
feature on this cap.
Signed-off-by: Michael Guralnik <michaelgur@nvidia.com>
Reviewed-by: Patrisious Haddad <phaddad@nvidia.com>
Signed-off-by: Edward Srouji <edwards@nvidia.com>
Link: https://patch.msgid.link/20260726-deprecate-lat-qps-v2-2-5e0c2ee55046@nvidia.com
Signed-off-by: Leon Romanovsky <leon@kernel.org>
New HW no longer implements a separate class for latency-sensitive QPs
and advertises this by a new cap bit.
Signed-off-by: Leon Romanovsky <leon@kernel.org>
Introduce the qp_latency_sensitive_disable capability bit to indicate
that the device no longer implements a separate class for
latency-sensitive QPs, so drivers can stop programming the
latency_sensitive QPC field and skip allocating the dedicated
fast-path bfreg on such HW.
Signed-off-by: Michael Guralnik <michaelgur@nvidia.com>
Reviewed-by: Patrisious Haddad <phaddad@nvidia.com>
Signed-off-by: Edward Srouji <edwards@nvidia.com>
Link: https://patch.msgid.link/20260726-deprecate-lat-qps-v2-1-5e0c2ee55046@nvidia.com
Signed-off-by: Leon Romanovsky <leon@kernel.org>
set_user_buf_size() computes the QP buffer size by left-shifting the
user-supplied rq.wqe_cnt and rq.wqe_shift values as signed integers.
A sufficiently large rq.wqe_cnt causes signed integer overflow, which
is undefined behavior, and yields a small or negative buf_size, causing
ib_umem_get() to map a buffer smaller than the hardware will actually
write into.
Replace the shifts and addition with check_shl_overflow() and
check_add_overflow(), rejecting invalid user inputs.
Moreover, guard the identical shift computing qp->sq.offset in
_create_user_qp() before set_user_buf_size() is reached.
Fixes: e126ba97db ("mlx5: Add driver for Mellanox Connect-IB adapters")
Signed-off-by: Maher Sanalla <msanalla@nvidia.com>
Signed-off-by: Edward Srouji <edwards@nvidia.com>
Link: https://patch.msgid.link/20260723-fix-qp-buf-size-overflow-v1-1-ccb05ee43a7b@nvidia.com
Signed-off-by: Leon Romanovsky <leon@kernel.org>
Decouple RoCE acceleration counters exposure from the roce_accl device
cap. The device cap is intended to protect the access to the roce_accl
register and was disabled on VFs for that purpose.
Reading the acceleration counters, however, does not involve that
register, the counters are read-only statistics that carry no
configuration risk. Gating their exposure on the capability therefore
needlessly hides useful diagnostic data on VFs.
Expose the counters on all functions regardless of the capability.
Signed-off-by: Michael Gur <michaelgur@nvidia.com>
Reviewed-by: Chiara Meiohas <cmeiohas@nvidia.com>
Signed-off-by: Edward Srouji <edwards@nvidia.com>
Link: https://patch.msgid.link/20260723-expose-roce-accl-counters-v1-1-967618b550cd@nvidia.com
Signed-off-by: Leon Romanovsky <leon@kernel.org>
alloc_skb() with the __GFP_NOFAIL flag will never return NULL,
so the subsequent NULL checks and error handling are unreachable
dead code. Remove them.
Signed-off-by: Gou Hao <gouhao@uniontech.com>
Link: https://patch.msgid.link/20260724022851.466017-7-gouhao@uniontech.com
Reviewed-by: Potnuri Bharat Teja <bharat@chelsio.com>
Signed-off-by: Leon Romanovsky <leon@kernel.org>
Implement an empty disassociate_ucontext() callback so the RDMA core
can move rxe devices between net namespaces. The core requires this
callback to reset user contexts without waiting for userspace.
rxe needs no teardown here: its user-mapped queues live in
reference-counted vmalloc memory (see rxe_mmap.c) that stays valid
while userspace holds the mappings.
Signed-off-by: Jiri Pirko <jiri@nvidia.com>
Link: https://patch.msgid.link/20260716132316.1495242-14-jiri@resnulli.us
Reviewed-by: Yanjun Zhu <yanjun.zhu@linux.dev>
Signed-off-by: Leon Romanovsky <leon@kernel.org>
Prepare queue mappings for asynchronous ucontext disassociation during
device disable. Rely on the VMA page references to preserve mapped
memory until the final unmap.
Signed-off-by: Jiri Pirko <jiri@nvidia.com>
Link: https://patch.msgid.link/20260716132316.1495242-13-jiri@resnulli.us
Signed-off-by: Leon Romanovsky <leon@kernel.org>
Use rdma_dev_access_netns() to scope RDMA device name lookup and "%d" name
allocation to the relevant net namespace. Keep shared mode and
CONFIG_NET_NS=n behaviour system-wide.
Signed-off-by: Jiri Pirko <jiri@nvidia.com>
Link: https://patch.msgid.link/20260716132316.1495242-12-jiri@resnulli.us
Signed-off-by: Leon Romanovsky <leon@kernel.org>
Document that SELinux ibendport labels use a global (device name, port)
key, so same-named RDMA devices in different net namespaces share a label.
Signed-off-by: Jiri Pirko <jiri@nvidia.com>
Link: https://patch.msgid.link/20260716132316.1495242-11-jiri@resnulli.us
Signed-off-by: Leon Romanovsky <leon@kernel.org>
Document the rdma_cm configfs limitation: configfs is global, so same-named
RDMA devices in different net namespaces cannot both be represented there.
Signed-off-by: Jiri Pirko <jiri@nvidia.com>
Link: https://patch.msgid.link/20260716132316.1495242-10-jiri@resnulli.us
Signed-off-by: Leon Romanovsky <leon@kernel.org>
RDMA device names are unique only within a network namespace, but an
RDMA cgroup can account resources for devices from multiple namespaces.
Duplicate names therefore make cgroup output ambiguous and can cause
limit writes to select the wrong device.
Use the system-wide RDMA device index to distinguish duplicate names
while preserving the existing UAPI for unique names. Reject ambiguous
name-only writes with -ENOTUNIQ and expose a complete device view to
administrators.
Signed-off-by: Jiri Pirko <jiri@nvidia.com>
Link: https://patch.msgid.link/20260716132316.1495242-9-jiri@resnulli.us
Signed-off-by: Leon Romanovsky <leon@kernel.org>
Tag srp_class by the RDMA device's net namespace so SRP hosts derived from
same-named RDMA devices can coexist across namespaces.
Signed-off-by: Jiri Pirko <jiri@nvidia.com>
Link: https://patch.msgid.link/20260716132316.1495242-8-jiri@resnulli.us
Signed-off-by: Leon Romanovsky <leon@kernel.org>
Scope smc_pnet_find_ib() to the caller's net namespace so pnetid setup
cannot bind to a same-named RDMA device from another namespace once names
become per-netns.
Signed-off-by: Jiri Pirko <jiri@nvidia.com>
Link: https://patch.msgid.link/20260716132316.1495242-7-jiri@resnulli.us
Signed-off-by: Leon Romanovsky <leon@kernel.org>
Accept RDMA_NLDEV_ATTR_DEV_NAME together with RDMA_NLDEV_NET_NS_FD so a
netlink move can rename the device in the destination namespace. Keep the
name semantics aligned with the existing RDMA rename path.
Signed-off-by: Jiri Pirko <jiri@nvidia.com>
Link: https://patch.msgid.link/20260716132316.1495242-6-jiri@resnulli.us
Signed-off-by: Leon Romanovsky <leon@kernel.org>
Thread extack through the existing net namespace move helper and report the
main failure reasons from the core path. Keep the existing move UAPI shape
unchanged.
Signed-off-by: Jiri Pirko <jiri@nvidia.com>
Link: https://patch.msgid.link/20260716132316.1495242-5-jiri@resnulli.us
Signed-off-by: Leon Romanovsky <leon@kernel.org>
Allow namespace moves to request a destination device name. Keep requested
names on the same literal-name path as the existing RDMA rename operation,
and keep teardown fallback naming on the trusted kernel-controlled path.
Signed-off-by: Jiri Pirko <jiri@nvidia.com>
Link: https://patch.msgid.link/20260716132316.1495242-4-jiri@resnulli.us
Signed-off-by: Leon Romanovsky <leon@kernel.org>
Prepare namespace moves for per-netns names. Check user-initiated moves for
destination-name conflicts before disabling the device, keep same-netns
moves as no-ops, and make teardown moves detach from the exiting namespace
even if fallback naming fails.
Signed-off-by: Jiri Pirko <jiri@nvidia.com>
Link: https://patch.msgid.link/20260716132316.1495242-3-jiri@resnulli.us
Signed-off-by: Leon Romanovsky <leon@kernel.org>
Prepare for per-netns RDMA device names by passing the target net
namespace through the name lookup and allocation helpers. Keep current
global uniqueness behaviour.
Signed-off-by: Jiri Pirko <jiri@nvidia.com>
Link: https://patch.msgid.link/20260716132316.1495242-2-jiri@resnulli.us
Signed-off-by: Leon Romanovsky <leon@kernel.org>
Add an optional mlx5 driver-namespace UMEM attribute on SRQ create so
userspace can supply the doorbell record umem explicitly, symmetric to
the CQ and QP sides. Resolve it inside mlx5_ib_db_map_user() and use it
as a private DBR page when present; otherwise take the existing UHW
share-or-pin path that preserves per-page DBR sharing across CQ/QP/SRQ
in the same process.
Add mlx5's first UVERBS_OBJECT_SRQ UAPI definition chain to attach the
new attr.
Signed-off-by: Jiri Pirko <jiri@nvidia.com>
Link: https://patch.msgid.link/20260701124015.64350-4-jiri@resnulli.us
Signed-off-by: Leon Romanovsky <leon@kernel.org>
Use the per-attribute UMEM helper to pin the SRQ buffer umem on demand.
ib_umem_get_attr_or_va() resolves the new CREATE_SRQ_BUF_UMEM attribute
when present and otherwise falls back to the existing UHW ucmd->buf_addr
VA, preserving the legacy behavior.
Signed-off-by: Jiri Pirko <jiri@nvidia.com>
Link: https://patch.msgid.link/20260701124015.64350-3-jiri@resnulli.us
Signed-off-by: Leon Romanovsky <leon@kernel.org>
Apply the per-attribute UMEM model to the SRQ create method. Add an
optional UMEM attribute that backs the SRQ WQE buffer, so userspace can
supply it as either a VA or a dma-buf through a single descriptor,
consistent with the CQ and QP create methods.
mlx5 is the only driver that pins an SRQ WQE buffer via umem; it maps a
single ucmd->buf_addr region through this attribute. No other driver
implements a user SRQ buffer, so none of them use the attribute.
Signed-off-by: Jiri Pirko <jiri@nvidia.com>
Link: https://patch.msgid.link/20260701124015.64350-2-jiri@resnulli.us
Signed-off-by: Leon Romanovsky <leon@kernel.org>
Implement returning to userspace applications PDNs of created PDs.
The PDN is used by applications that build work requests outside of the
rdma-core code base. The PDN is used to build work requests that require
mentioning the PD. The HW still ensures PD isolation using PDN attached
to MRs and WRs, therefore the PDN mentioned in the work request must match
the PDN of the used work queue. The work requests can fit only 16 bit PDNs.
Allow users to request short PDNs which are 16 bits.
The capability to request short PDN and get PDN is encoded in the response
of alloc user context IOCTL.
Signed-off-by: Konstantin Taranov <kotaranov@microsoft.com>
Link: https://patch.msgid.link/20260717200839.495327-1-kotaranov@linux.microsoft.com
Signed-off-by: Leon Romanovsky <leon@kernel.org>
erdma_destroy_qp(), erdma_destroy_cq(), erdma_dereg_mr(), and
erdma_destroy_ah() returned early when erdma_post_cmd_wait() failed,
leaking the queue buffers, MTTs, doorbells and the STAG, QPN, CQN and AHN
identifiers. A command timeout clears ERDMA_CMDQ_STATE_OK_BIT and
permanently disables the command queue, so no retry can succeed; the RDMA
core keeps the object after a failed destructor and forced uverbs cleanup
then nulls the pointers, making the resources unreachable.
Warn on failure but release every software-owned resource and return
success, since during terminal destruction the hardware command result is
only diagnostic.
Fixes: 1550557717 ("RDMA/erdma: Add verbs implementation")
Link: https://patch.msgid.link/20260722-b4-qp-and-cq-memory-are-leaked-if-the-d-v1-1-97e223dc1c96@nvidia.com
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
Acked-by: Cheng Xu <chengyou@linux.alibaba.com>
Since reset is actually controlled by the boolean flag rather than
long, switch to 'kstrtobool_from_user()' and use the latter for an
overall simplification of 'ocrdma_dbgfs_ops_write()'.
Signed-off-by: Dmitry Antipov <dmantipov@yandex.ru>
Link: https://patch.msgid.link/20260723071845.568718-1-dmantipov@yandex.ru
Signed-off-by: Leon Romanovsky <leon@kernel.org>
Simplify 'bnxt_re_cc_config_set()' and 'cq_coal_cfg_write()'
by using the convenient 'kstrtou32_from_user()'.
Signed-off-by: Dmitry Antipov <dmantipov@yandex.ru>
Link: https://patch.msgid.link/20260723071629.568675-1-dmantipov@yandex.ru
Signed-off-by: Leon Romanovsky <leon@kernel.org>
Currently EFA WRs support 16-bit request ID, this requires EFA to manage
a translation table to translate the IB WR ID from 64-bits to 16-bits
and translating it back on CQ completion.
Expose a new device capability to handle 64-bit request ID for SQ WRs
allowing userspace to directly post the 64-bit ID to the device.
Reviewed-by: Michael Margolin <mrgolin@amazon.com>
Reviewed-by: Tom Sela <tomsela@amazon.com>
Signed-off-by: Yonatan Nachum <ynachum@amazon.com>
Link: https://patch.msgid.link/20260722113331.2515247-3-ynachum@amazon.com
Signed-off-by: Leon Romanovsky <leon@kernel.org>
Add the support needed to propagate the user requested flags to config
the CQ/QP to support 64-bit SQ request ID to the device.
Reviewed-by: Michael Margolin <mrgolin@amazon.com>
Reviewed-by: Tom Sela <tomsela@amazon.com>
Signed-off-by: Yonatan Nachum <ynachum@amazon.com>
Link: https://patch.msgid.link/20260722113331.2515247-2-ynachum@amazon.com
Signed-off-by: Leon Romanovsky <leon@kernel.org>
bnxt_re_mmap() rejects VM_WRITE for the DBR_PAGE and TOGGLE_PAGE mmap
flags, but a read-only mapping can still retain VM_MAYWRITE. nd later
be upgraded with mprotect(PROT_WRITE). This can bypass the write check
that only runs at mmap time.
Clear VM_MAYWRITE before vm_insert_page() in the shared DBR/toggle-page
branch, matching the existing policy that userspace writes are not
expected for these pages.
Fixes: ea22248578 ("RDMA/bnxt_re: Update alloc_page uapi for pacing")
Suggested-by: Yousef Alhouseen <alhouseenyousef@gmail.com>
Signed-off-by: Selvin Xavier <selvin.xavier@broadcom.com>
Link: https://patch.msgid.link/20260721115440.24021-5-selvin.xavier@broadcom.com
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
