linux/net/core
Eric Dumazet 16c610162d net: call cond_resched() less often in __release_sock()
While stress testing TCP I had unexpected retransmits and sack packets
when a single cpu receives data from multiple high-throughput flows.

super_netperf 4 -H srv -T,10 -l 3000 &

Tcpdump extract:

 00:00:00.000007 IP6 clnt > srv: Flags [.], seq 26062848:26124288, ack 1, win 66, options [nop,nop,TS val 651460834 ecr 3100749131], length 61440
 00:00:00.000006 IP6 clnt > srv: Flags [.], seq 26124288:26185728, ack 1, win 66, options [nop,nop,TS val 651460834 ecr 3100749131], length 61440
 00:00:00.000005 IP6 clnt > srv: Flags [P.], seq 26185728:26243072, ack 1, win 66, options [nop,nop,TS val 651460834 ecr 3100749131], length 57344
 00:00:00.000006 IP6 clnt > srv: Flags [.], seq 26243072:26304512, ack 1, win 66, options [nop,nop,TS val 651460844 ecr 3100749141], length 61440
 00:00:00.000005 IP6 clnt > srv: Flags [.], seq 26304512:26365952, ack 1, win 66, options [nop,nop,TS val 651460844 ecr 3100749141], length 61440
 00:00:00.000007 IP6 clnt > srv: Flags [P.], seq 26365952:26423296, ack 1, win 66, options [nop,nop,TS val 651460844 ecr 3100749141], length 57344
 00:00:00.000006 IP6 clnt > srv: Flags [.], seq 26423296:26484736, ack 1, win 66, options [nop,nop,TS val 651460853 ecr 3100749150], length 61440
 00:00:00.000005 IP6 clnt > srv: Flags [.], seq 26484736:26546176, ack 1, win 66, options [nop,nop,TS val 651460853 ecr 3100749150], length 61440
 00:00:00.000005 IP6 clnt > srv: Flags [P.], seq 26546176:26603520, ack 1, win 66, options [nop,nop,TS val 651460853 ecr 3100749150], length 57344
 00:00:00.003932 IP6 clnt > srv: Flags [P.], seq 26603520:26619904, ack 1, win 66, options [nop,nop,TS val 651464844 ecr 3100753141], length 16384
 00:00:00.006602 IP6 clnt > srv: Flags [.], seq 24862720:24866816, ack 1, win 66, options [nop,nop,TS val 651471419 ecr 3100759716], length 4096
 00:00:00.013000 IP6 clnt > srv: Flags [.], seq 24862720:24866816, ack 1, win 66, options [nop,nop,TS val 651484421 ecr 3100772718], length 4096
 00:00:00.000416 IP6 srv > clnt: Flags [.], ack 26619904, win 1393, options [nop,nop,TS val 3100773185 ecr 651484421,nop,nop,sack 1 {24862720:24866816}], length 0

After analysis, it appears this is because of the cond_resched()
call from  __release_sock().

When current thread is yielding, while still holding the TCP socket lock,
it might regain the cpu after a very long time.

Other peer TLP/RTO is firing (multiple times) and packets are retransmit,
while the initial copy is waiting in the socket backlog or receive queue.

In this patch, I call cond_resched() only once every 16 packets.

Modern TCP stack now spends less time per packet in the backlog,
especially because ACK are no longer sent (commit 133c4c0d37
"tcp: defer regular ACK while processing socket backlog")

Before:

clnt:/# nstat -n;sleep 10;nstat|egrep "TcpOutSegs|TcpRetransSegs|TCPFastRetrans|TCPTimeouts|Probes|TCPSpuriousRTOs|DSACK"
TcpOutSegs                      19046186           0.0
TcpRetransSegs                  1471               0.0
TcpExtTCPTimeouts               1397               0.0
TcpExtTCPLossProbes             1356               0.0
TcpExtTCPDSACKRecv              1352               0.0
TcpExtTCPSpuriousRTOs           114                0.0
TcpExtTCPDSACKRecvSegs          1352               0.0

After:

clnt:/# nstat -n;sleep 10;nstat|egrep "TcpOutSegs|TcpRetransSegs|TCPFastRetrans|TCPTimeouts|Probes|TCPSpuriousRTOs|DSACK"
TcpOutSegs                      19218936           0.0

Signed-off-by: Eric Dumazet <edumazet@google.com>
Reviewed-by: Kuniyuki Iwashima <kuniyu@google.com>
Link: https://patch.msgid.link/20250903174811.1930820-1-edumazet@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
History 2025-09-04 19:16:51 -07:00
..
bpf_sk_storage.c bpf: Remove unnecessary BTF lookups in bpf_sk_storage_tracing_allowed 2025-01-29 08:51:51 -08:00
datagram.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net 2025-08-29 11:48:01 -07:00
dev.c net: Cache hash and flow_id to avoid recalculation 2025-08-27 18:24:20 -07:00
dev.h net: prevent deadlocks when enabling NAPIs with mixed kthread config 2025-08-12 14:43:05 +02:00
dev_addr_lists.c net: s/dev_pre_changeaddr_notify/netif_pre_changeaddr_notify/ 2025-07-18 17:27:47 -07:00
dev_addr_lists_test.c net: dev_addr_lists: move locking out of init/exit in kunit 2024-04-15 10:26:35 +01:00
dev_api.c net: define an enum for the napi threaded state 2025-07-24 18:34:55 -07:00
dev_ioctl.c net: s/dev_get_flags/netif_get_flags/ 2025-07-18 17:27:47 -07:00
devmem.c net: devmem: pull out dma_dev out of net_devmem_bind_dmabuf 2025-08-28 16:05:32 -07:00
devmem.h net: devmem: pull out dma_dev out of net_devmem_bind_dmabuf 2025-08-28 16:05:32 -07:00
drop_monitor.c treewide, timers: Rename from_timer() to timer_container_of() 2025-06-08 09:07:37 +02:00
dst.c net: dst: introduce dst->dev_rcu 2025-08-29 19:36:31 -07:00
dst_cache.c net: dst: annotate data-races around dst->obsolete 2025-07-02 14:32:29 -07:00
failover.c net: failover: use IFF_NO_ADDRCONF flag to prevent ipv6 addrconf 2022-12-12 15:18:25 -08:00
fib_notifier.c net: do not acquire rtnl in fib_seq_sum() 2024-10-11 15:35:05 -07:00
fib_rules.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net 2025-04-17 12:26:50 -07:00
filter.c ipv4: Convert ->flowi4_tos to dscp_t. 2025-08-26 17:34:31 -07:00
flow_dissector.c net: remove '__' from __skb_flow_get_ports() 2025-02-24 14:27:53 -08:00
flow_offload.c tc: flower: Enable offload support IPSEC SPI field. 2023-08-02 10:09:32 +01:00
gen_estimator.c net_sched: gen_estimator: fix est_timer() vs CONFIG_PREEMPT_RT=y 2025-08-29 19:04:20 -07:00
gen_stats.c net: Remove the obsolte u64_stats_fetch_*_irq() users (net). 2022-10-28 20:13:54 -07:00
gro.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net 2025-02-27 10:20:58 -08:00
gro_cells.c net: move netdev_max_backlog to net_hotdata 2024-03-07 21:12:42 -08:00
gso.c net: introduce struct net_hotdata 2024-03-07 21:12:41 -08:00
hotdata.c tcp: move tcp_memory_allocated into net_aligned_data 2025-07-02 14:22:02 -07:00
hwbm.c net: hwbm: Make the hwbm_pool lock a mutex 2019-06-09 19:40:10 -07:00
ieee8021q_helpers.c net: ieee8021q: fix insufficient table-size assertion 2025-07-01 12:55:49 +02:00
link_watch.c net: hold instance lock during NETDEV_CHANGE 2025-04-07 11:13:39 -07:00
lock_debug.c netdev: fix the locking for netdev notifications 2025-04-17 18:55:14 -07:00
lwt_bpf.c ipv4: Convert ->flowi4_tos to dscp_t. 2025-08-26 17:34:31 -07:00
lwtunnel.c inet: Remove rtnl_is_held arg of lwtunnel_valid_encap_type(_attr)?(). 2025-05-20 19:18:24 -07:00
Makefile queue_api: add support for fetching per queue DMA dev 2025-08-28 16:05:31 -07:00
mp_dmabuf_devmem.h memory-provider: dmabuf devmem memory provider 2024-09-11 20:44:31 -07:00
neighbour.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net 2025-07-26 11:49:45 -07:00
net-procfs.c net: add data-race annotations in softnet_seq_show() 2025-04-08 12:30:55 -07:00
net-sysfs.c net: Prevent RPS table overwrite of active flows 2025-08-27 18:24:13 -07:00
net-sysfs.h net: remove RTNL use for /proc/sys/net/core/rps_default_mask 2025-07-07 18:42:12 -07:00
net-traces.c move asm/unaligned.h to linux/unaligned.h 2024-10-02 17:23:23 -04:00
net_namespace.c Networking changes for 6.17. 2025-07-30 08:58:55 -07:00
net_test.c pfcp: always set pfcp metadata 2024-04-01 10:49:28 +01:00
netclassid_cgroup.c net, bpf: Fix RCU usage in task_cls_state() for BPF programs 2025-06-11 21:30:29 +02:00
netdev-genl-gen.c net: define an enum for the napi threaded state 2025-07-24 18:34:55 -07:00
netdev-genl-gen.h net: devmem: TCP tx netlink api 2025-05-13 11:12:48 +02:00
netdev-genl.c net: devmem: allow binding on rx queues with same DMA devices 2025-08-28 16:05:32 -07:00
netdev_queues.c queue_api: add support for fetching per queue DMA dev 2025-08-28 16:05:31 -07:00
netdev_rx_queue.c net: add helper to pre-check if PP for an Rx queue will be unreadable 2025-09-04 10:19:17 +02:00
netevent.c net: core: Correct function name netevent_unregister_notifier() in the kerneldoc 2021-03-28 17:56:56 -07:00
netmem_priv.h page_pool: Track DMA-mapped pages and unmap them when destroying the pool 2025-04-14 16:30:29 -07:00
netpoll.c netpoll: prevent hanging NAPI when netcons gets enabled 2025-07-30 18:05:52 -07:00
netprio_cgroup.c bpf, cgroups: Fix cgroup v2 fallback on v1/v2 mixed mode 2021-09-13 16:35:58 -07:00
of_net.c net: Explicitly include correct DT includes 2023-07-27 20:33:16 -07:00
page_pool.c page_pool: fix incorrect mp_ops error handling 2025-08-22 15:52:02 -07:00
page_pool_priv.h net: page_pool: don't try to stash the napi id 2025-01-27 14:37:41 -08:00
page_pool_user.c net: use napi_id_valid helper 2025-02-17 16:43:04 -08:00
pktgen.c net: pktgen: Use min()/min_t() to improve pktgen_finalize_skb() 2025-08-21 10:12:11 +02:00
ptp_classifier.c ptp: Add generic PTP is_sync() function 2022-03-07 11:31:34 +00:00
request_sock.c tcp: make sure init the accept_queue's spinlocks once 2024-01-19 21:13:25 -08:00
rtnetlink.c net: s/dev_get_flags/netif_get_flags/ 2025-07-18 17:27:47 -07:00
scm.c af_unix: enable handing out pidfds for reaped tasks in SCM_PIDFD 2025-07-04 09:32:35 +02:00
secure_seq.c net: Retire DCCP socket. 2025-04-11 18:58:10 -07:00
selftests.c net: selftests: add PHY-loopback test for bad TCP checksums 2025-07-18 17:19:46 -07:00
skb_fault_injection.c net: Implement fault injection forcing skb reallocation 2024-11-12 12:05:33 +01:00
skbuff.c net: avoid one loop iteration in __skb_splice_bits 2025-08-20 19:24:17 -07:00
skmsg.c bpf, sockmap: Fix psock incorrectly pointing to sk 2025-06-10 18:16:15 +02:00
sock.c net: call cond_resched() less often in __release_sock() 2025-09-04 19:16:51 -07:00
sock_destructor.h skb_expand_head() adjust skb->truesize incorrectly 2021-10-22 12:35:51 -07:00
sock_diag.c net: Retire DCCP socket. 2025-04-11 18:58:10 -07:00
sock_map.c bpf: Remove attach_type in sockmap_link 2025-07-11 10:51:55 -07:00
sock_reuseport.c net: core: annotate socks of struct sock_reuseport with __counted_by 2024-08-02 17:16:59 -07:00
stream.c net: stream: add description for sk_stream_write_space() 2025-07-18 16:57:21 -07:00
sysctl_net_core.c net: remove RTNL use for /proc/sys/net/core/rps_default_mask 2025-07-07 18:42:12 -07:00
timestamping.c net: Add the possibility to support a selected hwtstamp in netdevice 2024-12-16 12:51:40 +00:00
tso.c move asm/unaligned.h to linux/unaligned.h 2024-10-02 17:23:23 -04:00
utils.c net: Fix checksum update for ILA adj-transport 2025-05-30 19:53:51 -07:00
xdp.c xsk: add missing virtual address conversion for page 2025-05-27 11:46:47 +02:00