svcrdma: Relieve contention on sc_send_lock.

Browse source

/proc/lock_stat indicates the the sc_send_lock is heavily
contended when the server is under load from a single client.

To address this, convert the send_ctxt free list to an llist.
Returning an item to the send_ctxt cache is now waitless, which
reduces the instruction path length in the single-threaded Send
handler (svc_rdma_wc_send).

The goal is to enable the ib_comp_wq worker to handle a higher
RPC/RDMA Send completion rate given the same CPU resources. This
change reduces CPU utilization of Send completion by 2-3% on my
server.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
Reviewed-By: Tom Talpey <tom@talpey.com>

...

This commit is contained in:

Chuck Lever 2021-02-09 10:32:20 -05:00

parent 6c8c84f525

commit b6c2bfea09

3 changed files with 11 additions and 18 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download patch file Download diff file Expand all files Collapse all files

4

include/linux/sunrpc/svc_rdma.h

View file

23

net/sunrpc/xprtrdma/svc_rdma_sendto.c

View file

2

net/sunrpc/xprtrdma/svc_rdma_transport.c

View file