Merge branch 'net-group-together-hot-data'

Eric Dumazet says: ==================== net: group together hot data While our recent structure reorganizations were focused on increasing max throughput, there is still an area where improvements are much needed. In many cases, a cpu handles one packet at a time, instead of a nice batch. Hardware interrupt. -> Software interrupt. -> Network/Protocol stacks. If the cpu was idle or busy in other layers, it has to pull many cache lines. This series adds a new net_hotdata structure, where some critical (and read-mostly) data used in rx and tx path is packed in a small number of cache lines. Synthetic benchmarks will not see much difference, but latency of single packet should improve. net_hodata current size on 64bit is 416 bytes, but might grow in the future. Also move RPS definitions to a new include file. ==================== Link: https://lore.kernel.org/r/20240306160031.874438-1-edumazet@google.com Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2026-09-28 19:20:30 +00:00 · 2024-03-07 21:12:45 -08:00 · 2024-03-07 21:12:45 -08:00 · e8bb2ccff7
commit e8bb2ccff7
parent d3423ed9b8 ce7f49ab74
44 changed files with 391 additions and 320 deletions
--- a/drivers/net/ethernet/intel/ice/ice_arfs.c
+++ b/drivers/net/ethernet/intel/ice/ice_arfs.c
--- a/drivers/net/ethernet/mellanox/mlx4/en_netdev.c
+++ b/drivers/net/ethernet/mellanox/mlx4/en_netdev.c
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_arfs.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_arfs.c
--- a/drivers/net/ethernet/sfc/rx_common.c
+++ b/drivers/net/ethernet/sfc/rx_common.c
--- a/drivers/net/ethernet/sfc/siena/rx_common.c
+++ b/drivers/net/ethernet/sfc/siena/rx_common.c
--- a/drivers/net/tun.c
+++ b/drivers/net/tun.c
--- a/include/linux/netdevice.h
+++ b/include/linux/netdevice.h
--- a/include/linux/skbuff.h
+++ b/include/linux/skbuff.h
--- a/include/net/gro.h
+++ b/include/net/gro.h
--- a/include/net/hotdata.h
+++ b/include/net/hotdata.h
--- a/include/net/protocol.h
+++ b/include/net/protocol.h
--- a/include/net/rps.h
+++ b/include/net/rps.h
--- a/include/net/sock.h
+++ b/include/net/sock.h
--- a/kernel/bpf/cpumap.c
+++ b/kernel/bpf/cpumap.c
--- a/net/bpf/test_run.c
+++ b/net/bpf/test_run.c
--- a/net/core/Makefile
+++ b/net/core/Makefile
--- a/net/core/dev.c
+++ b/net/core/dev.c
--- a/net/core/dev.h
+++ b/net/core/dev.h
--- a/net/core/gro.c
+++ b/net/core/gro.c
--- a/net/core/gro_cells.c
+++ b/net/core/gro_cells.c
--- a/net/core/gso.c
+++ b/net/core/gso.c
--- a/net/core/hotdata.c
+++ b/net/core/hotdata.c
--- a/net/core/net-procfs.c
+++ b/net/core/net-procfs.c
--- a/net/core/net-sysfs.c
+++ b/net/core/net-sysfs.c
--- a/net/core/skbuff.c
+++ b/net/core/skbuff.c
--- a/net/core/sysctl_net_core.c
+++ b/net/core/sysctl_net_core.c
--- a/net/core/xdp.c
+++ b/net/core/xdp.c
--- a/net/ipv4/af_inet.c
+++ b/net/ipv4/af_inet.c
--- a/net/ipv4/inet_hashtables.c
+++ b/net/ipv4/inet_hashtables.c
--- a/net/ipv4/tcp.c
+++ b/net/ipv4/tcp.c
--- a/net/ipv4/tcp_offload.c
+++ b/net/ipv4/tcp_offload.c
--- a/net/ipv4/udp.c
+++ b/net/ipv4/udp.c
--- a/net/ipv4/udp_offload.c
+++ b/net/ipv4/udp_offload.c
--- a/net/ipv6/af_inet6.c
+++ b/net/ipv6/af_inet6.c
--- a/net/ipv6/inet6_hashtables.c
+++ b/net/ipv6/inet6_hashtables.c
--- a/net/ipv6/ip6_offload.c
+++ b/net/ipv6/ip6_offload.c
--- a/net/ipv6/tcp_ipv6.c
+++ b/net/ipv6/tcp_ipv6.c
--- a/net/ipv6/tcpv6_offload.c
+++ b/net/ipv6/tcpv6_offload.c
--- a/net/ipv6/udp.c
+++ b/net/ipv6/udp.c
--- a/net/ipv6/udp_offload.c
+++ b/net/ipv6/udp_offload.c
--- a/net/sched/sch_generic.c
+++ b/net/sched/sch_generic.c
--- a/net/sctp/socket.c
+++ b/net/sctp/socket.c
--- a/net/xfrm/espintcp.c
+++ b/net/xfrm/espintcp.c
--- a/net/xfrm/xfrm_input.c
+++ b/net/xfrm/xfrm_input.c