xprtrdma: Split rb_lock

Browse source

/proc/lock_stat showed contention between rpcrdma_buffer_get/put
and the MR allocation functions during I/O intensive workloads.

Now that MRs are no longer allocated in rpcrdma_buffer_get(),
there's no reason the rb_mws list has to be managed using the
same lock as the send/receive buffers. Split that lock. The
new lock does not need to disable interrupts because buffer
get/put is never called in an interrupt context.

struct rpcrdma_buffer is re-arranged to ensure rb_mwlock and rb_mws
are always in a different cacheline than rb_lock and the buffer
pointers.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
Reviewed-by: Steve Wise <swise@opengridcomputing.com>
Reviewed-by: Sagi Grimberg <sagig@mellanox.com>
Tested-By: Devesh Sharma <devesh.sharma@avagotech.com>
Reviewed-by: Doug Ledford <dledford@redhat.com>
Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>

...

This commit is contained in:

Chuck Lever 2015-05-26 11:53:13 -04:00 • committed by Anna Schumaker

parent 7e53df111b

commit 58d1dcf5a8

4 changed files with 15 additions and 13 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download patch file Download diff file Expand all files Collapse all files

1

net/sunrpc/xprtrdma/fmr_ops.c

View file

1

net/sunrpc/xprtrdma/frwr_ops.c

View file

10

net/sunrpc/xprtrdma/verbs.c

View file

16

net/sunrpc/xprtrdma/xprt_rdma.h

View file