linux/drivers/net/vrf.c

1969 lines
45 KiB

ipv4: Convert ->flowi4_tos to dscp_t. Convert the ->flowic_tos field of struct flowi_common from __u8 to dscp_t, rename it ->flowic_dscp and propagate these changes to struct flowi and struct flowi4. We've had several bugs in the past where ECN bits could interfere with IPv4 routing, because these bits were not properly cleared when setting ->flowi4_tos. These bugs should be fixed now and the dscp_t type has been introduced to ensure that variables carrying DSCP values don't accidentally have any ECN bits set. Several variables and structure fields have been converted to dscp_t already, but the main IPv4 routing structure, struct flowi4, is still using a __u8. To avoid any future regression, this patch converts it to dscp_t. There are many users to convert at once. Fortunately, around half of ->flowi4_tos users already have a dscp_t value at hand, which they currently convert to __u8 using inet_dscp_to_dsfield(). For all of these users, we just need to drop that conversion. But, although we try to do the __u8 <-> dscp_t conversions at the boundaries of the network or of user space, some places still store TOS/DSCP variables as __u8 in core networking code. Those can hardly be converted either because the data structure is part of UAPI or because the same variable or field is also used for handling ECN in other parts of the code. In all of these cases where we don't have a dscp_t variable at hand, we need to use inet_dsfield_to_dscp() when interacting with ->flowi4_dscp. Changes since v1: * Fix space alignment in __bpf_redirect_neigh_v4() (Ido). Signed-off-by: Guillaume Nault <gnault@redhat.com> Reviewed-by: Ido Schimmel <idosch@nvidia.com> Link: https://patch.msgid.link/29acecb45e911d17446b9a3dbdb1ab7b821ea371.1756128932.git.gnault@redhat.com Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2025-08-25 15:37:43 +02:00
vrf: Remove unnecessary RCU protection around dst entries During initialization of a VRF device, the VRF driver creates two dst entries (for IPv4 and IPv6). They are attached to locally generated packets that are transmitted out of the VRF ports (via the l3mdev_l3_out() hook). Their purpose is to redirect packets towards the VRF device instead of having the packets egress directly out of the VRF ports. This is useful, for example, when a queuing discipline is configured on the VRF device. In order to avoid a NULL pointer dereference, commit b0e95ccdd775 ("net: vrf: protect changes to private data with rcu") made the pointers to the dst entries RCU protected. As far as I can tell, this was needed because back then the dst entries were released (and the pointers reset to NULL) before removing the VRF ports. Later on, commit f630c38ef0d7 ("vrf: fix bug_on triggered by rx when destroying a vrf") moved the removal of the VRF ports to the VRF device's dellink() callback. As such, the tear down sequence of a VRF device looks as follows: 1. VRF ports are removed. 2. VRF device is unregistered. a. Device is closed. b. An RCU grace period passes. c. ndo_uninit() is called. i. dst entries are released. Given the above, the Tx path will always see the same fully initialized dst entries and will never race with the ndo_uninit() callback. Therefore, there is no need to make the pointers to the dst entries RCU protected. Remove it as well as the unnecessary NULL checks in the Tx path. Signed-off-by: Ido Schimmel <idosch@nvidia.com> Reviewed-by: David Ahern <dsahern@kernel.org> Link: https://patch.msgid.link/20260326203233.1128554-4-idosch@nvidia.com Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2026-03-26 22:32:33 +02:00
net: vrf: performance improvements for IPv4 The VRF driver allows users to implement device based features for an entire domain. For example, a qdisc or netfilter rules can be attached to a VRF device or tcpdump can be used to view packets for all devices in the L3 domain. The device-based features come with a performance penalty, most notably in the Tx path. The VRF driver uses the l3mdev_l3_out hook to switch the dst on an skb to its private dst. This allows the skb to traverse the xmit stack with the device set to the VRF device which in turn enables the netfilter and qdisc features. The VRF driver then performs the FIB lookup again and reinserts the packet. This patch avoids the redirect for IPv4 packets if a qdisc has not been attached to a VRF device which is the default config. In this case the netfilter hooks and network taps are directly traversed in the l3mdev_l3_out handler. If a qdisc is attached to a VRF device, then the redirect using the vrf dst is done. Additional overhead is removed by only checking packet taps if a socket is open on the device (vrf_dev->ptype_all list is not empty). Packet sockets bound to any device will still get a copy of the packet via the real ingress or egress interface. The end result of this change is a decrease in the overhead of VRF for the default, baseline case (ie., no netfilter rules, no packet sockets, no qdisc) to ~3% for UDP which has a lookup per packet and < 1% overhead for connected sockets that leverage early demux and avoid FIB lookups. Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-03-20 11:19:44 -07:00
net: sched: generalize check for no-queue qdisc on TX queue The "noqueue" qdisc can either be directly attached, or get default attached if net_device priv_flags has IFF_NO_QUEUE. In both cases, the allocated Qdisc structure gets it's enqueue function pointer reset to NULL by noqueue_init() via noqueue_qdisc_ops. This is a common case for software virtual net_devices. For these devices with no-queue, the transmission path in __dev_queue_xmit() will bypass the qdisc layer. Directly invoking device drivers ndo_start_xmit (via dev_hard_start_xmit). In this mode the device driver is not allowed to ask for packets to be queued (either via returning NETDEV_TX_BUSY or stopping the TXQ). The simplest and most reliable way to identify this no-queue case is by checking if enqueue == NULL. The vrf driver currently open-codes this check (!qdisc->enqueue). While functionally correct, this low-level detail is better encapsulated in a dedicated helper for clarity and long-term maintainability. To make this behavior more explicit and reusable, this patch introduce a new helper: qdisc_txq_has_no_queue(). Helper will also be used by the veth driver in the next patch, which introduces optional qdisc-based backpressure. This is a non-functional change. Reviewed-by: David Ahern <dsahern@kernel.org> Reviewed-by: Toke Høiland-Jørgensen <toke@redhat.com> Signed-off-by: Jesper Dangaard Brouer <hawk@kernel.org> Link: https://patch.msgid.link/174559293172.827981.7583862632045264175.stgit@firesoul Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2025-04-25 16:55:31 +02:00
net: vrf: performance improvements for IPv4 The VRF driver allows users to implement device based features for an entire domain. For example, a qdisc or netfilter rules can be attached to a VRF device or tcpdump can be used to view packets for all devices in the L3 domain. The device-based features come with a performance penalty, most notably in the Tx path. The VRF driver uses the l3mdev_l3_out hook to switch the dst on an skb to its private dst. This allows the skb to traverse the xmit stack with the device set to the VRF device which in turn enables the netfilter and qdisc features. The VRF driver then performs the FIB lookup again and reinserts the packet. This patch avoids the redirect for IPv4 packets if a qdisc has not been attached to a VRF device which is the default config. In this case the netfilter hooks and network taps are directly traversed in the l3mdev_l3_out handler. If a qdisc is attached to a VRF device, then the redirect using the vrf dst is done. Additional overhead is removed by only checking packet taps if a socket is open on the device (vrf_dev->ptype_all list is not empty). Packet sockets bound to any device will still get a copy of the packet via the real ingress or egress interface. The end result of this change is a decrease in the overhead of VRF for the default, baseline case (ie., no netfilter rules, no packet sockets, no qdisc) to ~3% for UDP which has a lookup per packet and < 1% overhead for connected sockets that leverage early demux and avoid FIB lookups. Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-03-20 11:19:44 -07:00
net: vrf: ipv4 support for local traffic to local addresses Add support for locally originated traffic to VRF-local addresses. If destination device for an skb is the loopback or VRF device then set its dst to a local version of the VRF cached dst_entry and call netif_rx to insert the packet onto the rx queue - similar to what is done for loopback. This patch handles IPv4 support; follow on patch handles IPv6. With this patch, ping, tcp and udp packets to a local IPv4 address are successfully routed: $ ip addr show dev eth1 4: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master red state UP group default qlen 1000 link/ether 02:e0:f9:1c:b9:74 brd ff:ff:ff:ff:ff:ff inet 10.100.1.1/24 brd 10.100.1.255 scope global eth1 valid_lft forever preferred_lft forever inet6 2100:1::1/120 scope global valid_lft forever preferred_lft forever inet6 fe80::e0:f9ff:fe1c:b974/64 scope link valid_lft forever preferred_lft forever $ ping -c1 -I red 10.100.1.1 ping: Warning: source address might be selected on device other than red. PING 10.100.1.1 (10.100.1.1) from 10.100.1.1 red: 56(84) bytes of data. 64 bytes from 10.100.1.1: icmp_seq=1 ttl=64 time=0.057 ms This patch also enables use of IPv4 loopback address on the VRF device: $ ip addr add dev red 127.0.0.1/8 $ ping -c1 -I red 127.0.0.1 PING 127.0.0.1 (127.0.0.1) from 127.0.0.1 red: 56(84) bytes of data. 64 bytes from 127.0.0.1: icmp_seq=1 ttl=64 time=0.058 ms Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2016-06-06 20:50:39 -07:00
net: vrf: ipv4 support for local traffic to local addresses Add support for locally originated traffic to VRF-local addresses. If destination device for an skb is the loopback or VRF device then set its dst to a local version of the VRF cached dst_entry and call netif_rx to insert the packet onto the rx queue - similar to what is done for loopback. This patch handles IPv4 support; follow on patch handles IPv6. With this patch, ping, tcp and udp packets to a local IPv4 address are successfully routed: $ ip addr show dev eth1 4: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master red state UP group default qlen 1000 link/ether 02:e0:f9:1c:b9:74 brd ff:ff:ff:ff:ff:ff inet 10.100.1.1/24 brd 10.100.1.255 scope global eth1 valid_lft forever preferred_lft forever inet6 2100:1::1/120 scope global valid_lft forever preferred_lft forever inet6 fe80::e0:f9ff:fe1c:b974/64 scope link valid_lft forever preferred_lft forever $ ping -c1 -I red 10.100.1.1 ping: Warning: source address might be selected on device other than red. PING 10.100.1.1 (10.100.1.1) from 10.100.1.1 red: 56(84) bytes of data. 64 bytes from 10.100.1.1: icmp_seq=1 ttl=64 time=0.057 ms This patch also enables use of IPv4 loopback address on the VRF device: $ ip addr add dev red 127.0.0.1/8 $ ping -c1 -I red 127.0.0.1 PING 127.0.0.1 (127.0.0.1) from 127.0.0.1 red: 56(84) bytes of data. 64 bytes from 127.0.0.1: icmp_seq=1 ttl=64 time=0.058 ms Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2016-06-06 20:50:39 -07:00
net: vrf: ipv4 support for local traffic to local addresses Add support for locally originated traffic to VRF-local addresses. If destination device for an skb is the loopback or VRF device then set its dst to a local version of the VRF cached dst_entry and call netif_rx to insert the packet onto the rx queue - similar to what is done for loopback. This patch handles IPv4 support; follow on patch handles IPv6. With this patch, ping, tcp and udp packets to a local IPv4 address are successfully routed: $ ip addr show dev eth1 4: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master red state UP group default qlen 1000 link/ether 02:e0:f9:1c:b9:74 brd ff:ff:ff:ff:ff:ff inet 10.100.1.1/24 brd 10.100.1.255 scope global eth1 valid_lft forever preferred_lft forever inet6 2100:1::1/120 scope global valid_lft forever preferred_lft forever inet6 fe80::e0:f9ff:fe1c:b974/64 scope link valid_lft forever preferred_lft forever $ ping -c1 -I red 10.100.1.1 ping: Warning: source address might be selected on device other than red. PING 10.100.1.1 (10.100.1.1) from 10.100.1.1 red: 56(84) bytes of data. 64 bytes from 10.100.1.1: icmp_seq=1 ttl=64 time=0.057 ms This patch also enables use of IPv4 loopback address on the VRF device: $ ip addr add dev red 127.0.0.1/8 $ ping -c1 -I red 127.0.0.1 PING 127.0.0.1 (127.0.0.1) from 127.0.0.1 red: 56(84) bytes of data. 64 bytes from 127.0.0.1: icmp_seq=1 ttl=64 time=0.058 ms Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2016-06-06 20:50:39 -07:00
net: Add l3mdev index to flow struct and avoid oif reset for port devices The fundamental premise of VRF and l3mdev core code is binding a socket to a device (l3mdev or netdev with an L3 domain) to indicate L3 scope. Legacy code resets flowi_oif to the l3mdev losing any original port device binding. Ben (among others) has demonstrated use cases where the original port device binding is important and needs to be retained. This patch handles that by adding a new entry to the common flow struct that can indicate the l3mdev index for later rule and table matching avoiding the need to reset flowi_oif. In addition to allowing more use cases that require port device binds, this patch brings a few datapath simplications: 1. l3mdev_fib_rule_match is only called when walking fib rules and always after l3mdev_update_flow. That allows an optimization to bail early for non-VRF type uses cases when flowi_l3mdev is not set. Also, only that index needs to be checked for the FIB table id. 2. l3mdev_update_flow can be called with flowi_oif set to a l3mdev (e.g., VRF) device. By resetting flowi_oif only for this case the FLOWI_FLAG_SKIP_NH_OIF flag is not longer needed and can be removed, removing several checks in the datapath. The flowi_iif path can be simplified to only be called if the it is not loopback (loopback can not be assigned to an L3 domain) and the l3mdev index is not already set. 3. Avoid another device lookup in the output path when the fib lookup returns a reject failure. Note: 2 functional tests for local traffic with reject fib rules are updated to reflect the new direct failure at FIB lookup time for ping rather than the failure on packet path. The current code fails like this: HINT: Fails since address on vrf device is out of device scope COMMAND: ip netns exec ns-A ping -c1 -w1 -I eth1 172.16.3.1 ping: Warning: source address might be selected on device other than: eth1 PING 172.16.3.1 (172.16.3.1) from 172.16.3.1 eth1: 56(84) bytes of data. --- 172.16.3.1 ping statistics --- 1 packets transmitted, 0 received, 100% packet loss, time 0ms where the test now directly fails: HINT: Fails since address on vrf device is out of device scope COMMAND: ip netns exec ns-A ping -c1 -w1 -I eth1 172.16.3.1 ping: connect: No route to host Signed-off-by: David Ahern <dsahern@kernel.org> Tested-by: Ben Greear <greearb@candelatech.com> Link: https://lore.kernel.org/r/20220314204551.16369-1-dsahern@kernel.org Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2022-03-14 14:45:51 -06:00
net: Add l3mdev index to flow struct and avoid oif reset for port devices The fundamental premise of VRF and l3mdev core code is binding a socket to a device (l3mdev or netdev with an L3 domain) to indicate L3 scope. Legacy code resets flowi_oif to the l3mdev losing any original port device binding. Ben (among others) has demonstrated use cases where the original port device binding is important and needs to be retained. This patch handles that by adding a new entry to the common flow struct that can indicate the l3mdev index for later rule and table matching avoiding the need to reset flowi_oif. In addition to allowing more use cases that require port device binds, this patch brings a few datapath simplications: 1. l3mdev_fib_rule_match is only called when walking fib rules and always after l3mdev_update_flow. That allows an optimization to bail early for non-VRF type uses cases when flowi_l3mdev is not set. Also, only that index needs to be checked for the FIB table id. 2. l3mdev_update_flow can be called with flowi_oif set to a l3mdev (e.g., VRF) device. By resetting flowi_oif only for this case the FLOWI_FLAG_SKIP_NH_OIF flag is not longer needed and can be removed, removing several checks in the datapath. The flowi_iif path can be simplified to only be called if the it is not loopback (loopback can not be assigned to an L3 domain) and the l3mdev index is not already set. 3. Avoid another device lookup in the output path when the fib lookup returns a reject failure. Note: 2 functional tests for local traffic with reject fib rules are updated to reflect the new direct failure at FIB lookup time for ping rather than the failure on packet path. The current code fails like this: HINT: Fails since address on vrf device is out of device scope COMMAND: ip netns exec ns-A ping -c1 -w1 -I eth1 172.16.3.1 ping: Warning: source address might be selected on device other than: eth1 PING 172.16.3.1 (172.16.3.1) from 172.16.3.1 eth1: 56(84) bytes of data. --- 172.16.3.1 ping statistics --- 1 packets transmitted, 0 received, 100% packet loss, time 0ms where the test now directly fails: HINT: Fails since address on vrf device is out of device scope COMMAND: ip netns exec ns-A ping -c1 -w1 -I eth1 172.16.3.1 ping: connect: No route to host Signed-off-by: David Ahern <dsahern@kernel.org> Tested-by: Ben Greear <greearb@candelatech.com> Link: https://lore.kernel.org/r/20220314204551.16369-1-dsahern@kernel.org Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2022-03-14 14:45:51 -06:00
ipv4: Convert ->flowi4_tos to dscp_t. Convert the ->flowic_tos field of struct flowi_common from __u8 to dscp_t, rename it ->flowic_dscp and propagate these changes to struct flowi and struct flowi4. We've had several bugs in the past where ECN bits could interfere with IPv4 routing, because these bits were not properly cleared when setting ->flowi4_tos. These bugs should be fixed now and the dscp_t type has been introduced to ensure that variables carrying DSCP values don't accidentally have any ECN bits set. Several variables and structure fields have been converted to dscp_t already, but the main IPv4 routing structure, struct flowi4, is still using a __u8. To avoid any future regression, this patch converts it to dscp_t. There are many users to convert at once. Fortunately, around half of ->flowi4_tos users already have a dscp_t value at hand, which they currently convert to __u8 using inet_dscp_to_dsfield(). For all of these users, we just need to drop that conversion. But, although we try to do the __u8 <-> dscp_t conversions at the boundaries of the network or of user space, some places still store TOS/DSCP variables as __u8 in core networking code. Those can hardly be converted either because the data structure is part of UAPI or because the same variable or field is also used for handling ECN in other parts of the code. In all of these cases where we don't have a dscp_t variable at hand, we need to use inet_dsfield_to_dscp() when interacting with ->flowi4_dscp. Changes since v1: * Fix space alignment in __bpf_redirect_neigh_v4() (Ido). Signed-off-by: Guillaume Nault <gnault@redhat.com> Reviewed-by: Ido Schimmel <idosch@nvidia.com> Link: https://patch.msgid.link/29acecb45e911d17446b9a3dbdb1ab7b821ea371.1756128932.git.gnault@redhat.com Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2025-08-25 15:37:43 +02:00
net: Add l3mdev index to flow struct and avoid oif reset for port devices The fundamental premise of VRF and l3mdev core code is binding a socket to a device (l3mdev or netdev with an L3 domain) to indicate L3 scope. Legacy code resets flowi_oif to the l3mdev losing any original port device binding. Ben (among others) has demonstrated use cases where the original port device binding is important and needs to be retained. This patch handles that by adding a new entry to the common flow struct that can indicate the l3mdev index for later rule and table matching avoiding the need to reset flowi_oif. In addition to allowing more use cases that require port device binds, this patch brings a few datapath simplications: 1. l3mdev_fib_rule_match is only called when walking fib rules and always after l3mdev_update_flow. That allows an optimization to bail early for non-VRF type uses cases when flowi_l3mdev is not set. Also, only that index needs to be checked for the FIB table id. 2. l3mdev_update_flow can be called with flowi_oif set to a l3mdev (e.g., VRF) device. By resetting flowi_oif only for this case the FLOWI_FLAG_SKIP_NH_OIF flag is not longer needed and can be removed, removing several checks in the datapath. The flowi_iif path can be simplified to only be called if the it is not loopback (loopback can not be assigned to an L3 domain) and the l3mdev index is not already set. 3. Avoid another device lookup in the output path when the fib lookup returns a reject failure. Note: 2 functional tests for local traffic with reject fib rules are updated to reflect the new direct failure at FIB lookup time for ping rather than the failure on packet path. The current code fails like this: HINT: Fails since address on vrf device is out of device scope COMMAND: ip netns exec ns-A ping -c1 -w1 -I eth1 172.16.3.1 ping: Warning: source address might be selected on device other than: eth1 PING 172.16.3.1 (172.16.3.1) from 172.16.3.1 eth1: 56(84) bytes of data. --- 172.16.3.1 ping statistics --- 1 packets transmitted, 0 received, 100% packet loss, time 0ms where the test now directly fails: HINT: Fails since address on vrf device is out of device scope COMMAND: ip netns exec ns-A ping -c1 -w1 -I eth1 172.16.3.1 ping: connect: No route to host Signed-off-by: David Ahern <dsahern@kernel.org> Tested-by: Ben Greear <greearb@candelatech.com> Link: https://lore.kernel.org/r/20220314204551.16369-1-dsahern@kernel.org Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2022-03-14 14:45:51 -06:00
net: vrf: ipv4 support for local traffic to local addresses Add support for locally originated traffic to VRF-local addresses. If destination device for an skb is the loopback or VRF device then set its dst to a local version of the VRF cached dst_entry and call netif_rx to insert the packet onto the rx queue - similar to what is done for loopback. This patch handles IPv4 support; follow on patch handles IPv6. With this patch, ping, tcp and udp packets to a local IPv4 address are successfully routed: $ ip addr show dev eth1 4: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master red state UP group default qlen 1000 link/ether 02:e0:f9:1c:b9:74 brd ff:ff:ff:ff:ff:ff inet 10.100.1.1/24 brd 10.100.1.255 scope global eth1 valid_lft forever preferred_lft forever inet6 2100:1::1/120 scope global valid_lft forever preferred_lft forever inet6 fe80::e0:f9ff:fe1c:b974/64 scope link valid_lft forever preferred_lft forever $ ping -c1 -I red 10.100.1.1 ping: Warning: source address might be selected on device other than red. PING 10.100.1.1 (10.100.1.1) from 10.100.1.1 red: 56(84) bytes of data. 64 bytes from 10.100.1.1: icmp_seq=1 ttl=64 time=0.057 ms This patch also enables use of IPv4 loopback address on the VRF device: $ ip addr add dev red 127.0.0.1/8 $ ping -c1 -I red 127.0.0.1 PING 127.0.0.1 (127.0.0.1) from 127.0.0.1 red: 56(84) bytes of data. 64 bytes from 127.0.0.1: icmp_seq=1 ttl=64 time=0.058 ms Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2016-06-06 20:50:39 -07:00
net: vrf: ipv4 support for local traffic to local addresses Add support for locally originated traffic to VRF-local addresses. If destination device for an skb is the loopback or VRF device then set its dst to a local version of the VRF cached dst_entry and call netif_rx to insert the packet onto the rx queue - similar to what is done for loopback. This patch handles IPv4 support; follow on patch handles IPv6. With this patch, ping, tcp and udp packets to a local IPv4 address are successfully routed: $ ip addr show dev eth1 4: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master red state UP group default qlen 1000 link/ether 02:e0:f9:1c:b9:74 brd ff:ff:ff:ff:ff:ff inet 10.100.1.1/24 brd 10.100.1.255 scope global eth1 valid_lft forever preferred_lft forever inet6 2100:1::1/120 scope global valid_lft forever preferred_lft forever inet6 fe80::e0:f9ff:fe1c:b974/64 scope link valid_lft forever preferred_lft forever $ ping -c1 -I red 10.100.1.1 ping: Warning: source address might be selected on device other than red. PING 10.100.1.1 (10.100.1.1) from 10.100.1.1 red: 56(84) bytes of data. 64 bytes from 10.100.1.1: icmp_seq=1 ttl=64 time=0.057 ms This patch also enables use of IPv4 loopback address on the VRF device: $ ip addr add dev red 127.0.0.1/8 $ ping -c1 -I red 127.0.0.1 PING 127.0.0.1 (127.0.0.1) from 127.0.0.1 red: 56(84) bytes of data. 64 bytes from 127.0.0.1: icmp_seq=1 ttl=64 time=0.058 ms Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2016-06-06 20:50:39 -07:00
net: vrf: ipv4 support for local traffic to local addresses Add support for locally originated traffic to VRF-local addresses. If destination device for an skb is the loopback or VRF device then set its dst to a local version of the VRF cached dst_entry and call netif_rx to insert the packet onto the rx queue - similar to what is done for loopback. This patch handles IPv4 support; follow on patch handles IPv6. With this patch, ping, tcp and udp packets to a local IPv4 address are successfully routed: $ ip addr show dev eth1 4: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master red state UP group default qlen 1000 link/ether 02:e0:f9:1c:b9:74 brd ff:ff:ff:ff:ff:ff inet 10.100.1.1/24 brd 10.100.1.255 scope global eth1 valid_lft forever preferred_lft forever inet6 2100:1::1/120 scope global valid_lft forever preferred_lft forever inet6 fe80::e0:f9ff:fe1c:b974/64 scope link valid_lft forever preferred_lft forever $ ping -c1 -I red 10.100.1.1 ping: Warning: source address might be selected on device other than red. PING 10.100.1.1 (10.100.1.1) from 10.100.1.1 red: 56(84) bytes of data. 64 bytes from 10.100.1.1: icmp_seq=1 ttl=64 time=0.057 ms This patch also enables use of IPv4 loopback address on the VRF device: $ ip addr add dev red 127.0.0.1/8 $ ping -c1 -I red 127.0.0.1 PING 127.0.0.1 (127.0.0.1) from 127.0.0.1 red: 56(84) bytes of data. 64 bytes from 127.0.0.1: icmp_seq=1 ttl=64 time=0.058 ms Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2016-06-06 20:50:39 -07:00
net: vrf: performance improvements for IPv4 The VRF driver allows users to implement device based features for an entire domain. For example, a qdisc or netfilter rules can be attached to a VRF device or tcpdump can be used to view packets for all devices in the L3 domain. The device-based features come with a performance penalty, most notably in the Tx path. The VRF driver uses the l3mdev_l3_out hook to switch the dst on an skb to its private dst. This allows the skb to traverse the xmit stack with the device set to the VRF device which in turn enables the netfilter and qdisc features. The VRF driver then performs the FIB lookup again and reinserts the packet. This patch avoids the redirect for IPv4 packets if a qdisc has not been attached to a VRF device which is the default config. In this case the netfilter hooks and network taps are directly traversed in the l3mdev_l3_out handler. If a qdisc is attached to a VRF device, then the redirect using the vrf dst is done. Additional overhead is removed by only checking packet taps if a socket is open on the device (vrf_dev->ptype_all list is not empty). Packet sockets bound to any device will still get a copy of the packet via the real ingress or egress interface. The end result of this change is a decrease in the overhead of VRF for the default, baseline case (ie., no netfilter rules, no packet sockets, no qdisc) to ~3% for UDP which has a lookup per packet and < 1% overhead for connected sockets that leverage early demux and avoid FIB lookups. Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-03-20 11:19:44 -07:00
net: vrf: performance improvements for IPv4 The VRF driver allows users to implement device based features for an entire domain. For example, a qdisc or netfilter rules can be attached to a VRF device or tcpdump can be used to view packets for all devices in the L3 domain. The device-based features come with a performance penalty, most notably in the Tx path. The VRF driver uses the l3mdev_l3_out hook to switch the dst on an skb to its private dst. This allows the skb to traverse the xmit stack with the device set to the VRF device which in turn enables the netfilter and qdisc features. The VRF driver then performs the FIB lookup again and reinserts the packet. This patch avoids the redirect for IPv4 packets if a qdisc has not been attached to a VRF device which is the default config. In this case the netfilter hooks and network taps are directly traversed in the l3mdev_l3_out handler. If a qdisc is attached to a VRF device, then the redirect using the vrf dst is done. Additional overhead is removed by only checking packet taps if a socket is open on the device (vrf_dev->ptype_all list is not empty). Packet sockets bound to any device will still get a copy of the packet via the real ingress or egress interface. The end result of this change is a decrease in the overhead of VRF for the default, baseline case (ie., no netfilter rules, no packet sockets, no qdisc) to ~3% for UDP which has a lookup per packet and < 1% overhead for connected sockets that leverage early demux and avoid FIB lookups. Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-03-20 11:19:44 -07:00
vrf: revert "vrf: Remove unnecessary RCU-bh critical section" This reverts commit 504fc6f4f7f681d2a03aa5f68aad549d90eab853. dev_queue_xmit_nit is expected to be called with BH disabled. __dev_queue_xmit has the following: /* Disable soft irqs for various locks below. Also * stops preemption for RCU. */ rcu_read_lock_bh(); VRF must follow this invariant. The referenced commit removed this protection. Which triggered a lockdep warning: ================================ WARNING: inconsistent lock state 6.11.0 #1 Tainted: G W -------------------------------- inconsistent {IN-SOFTIRQ-W} -> {SOFTIRQ-ON-W} usage. btserver/134819 [HC0[0]:SC0[0]:HE1:SE1] takes: ffff8882da30c118 (rlock-AF_PACKET){+.?.}-{2:2}, at: tpacket_rcv+0x863/0x3b30 {IN-SOFTIRQ-W} state was registered at: lock_acquire+0x19a/0x4f0 _raw_spin_lock+0x27/0x40 packet_rcv+0xa33/0x1320 __netif_receive_skb_core.constprop.0+0xcb0/0x3a90 __netif_receive_skb_list_core+0x2c9/0x890 netif_receive_skb_list_internal+0x610/0xcc0 [...] other info that might help us debug this: Possible unsafe locking scenario: CPU0 ---- lock(rlock-AF_PACKET); <Interrupt> lock(rlock-AF_PACKET); *** DEADLOCK *** Call Trace: <TASK> dump_stack_lvl+0x73/0xa0 mark_lock+0x102e/0x16b0 __lock_acquire+0x9ae/0x6170 lock_acquire+0x19a/0x4f0 _raw_spin_lock+0x27/0x40 tpacket_rcv+0x863/0x3b30 dev_queue_xmit_nit+0x709/0xa40 vrf_finish_direct+0x26e/0x340 [vrf] vrf_l3_out+0x5f4/0xe80 [vrf] __ip_local_out+0x51e/0x7a0 [...] Fixes: 504fc6f4f7f6 ("vrf: Remove unnecessary RCU-bh critical section") Link: https://lore.kernel.org/netdev/20240925185216.1990381-1-greearb@candelatech.com/ Reported-by: Ben Greear <greearb@candelatech.com> Signed-off-by: Willem de Bruijn <willemb@google.com> Cc: stable@vger.kernel.org Reviewed-by: Ido Schimmel <idosch@nvidia.com> Tested-by: Ido Schimmel <idosch@nvidia.com> Reviewed-by: David Ahern <dsahern@kernel.org> Link: https://patch.msgid.link/20240929061839.1175300-1-willemdebruijn.kernel@gmail.com Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2024-09-29 02:18:20 -04:00
net: vrf: performance improvements for IPv4 The VRF driver allows users to implement device based features for an entire domain. For example, a qdisc or netfilter rules can be attached to a VRF device or tcpdump can be used to view packets for all devices in the L3 domain. The device-based features come with a performance penalty, most notably in the Tx path. The VRF driver uses the l3mdev_l3_out hook to switch the dst on an skb to its private dst. This allows the skb to traverse the xmit stack with the device set to the VRF device which in turn enables the netfilter and qdisc features. The VRF driver then performs the FIB lookup again and reinserts the packet. This patch avoids the redirect for IPv4 packets if a qdisc has not been attached to a VRF device which is the default config. In this case the netfilter hooks and network taps are directly traversed in the l3mdev_l3_out handler. If a qdisc is attached to a VRF device, then the redirect using the vrf dst is done. Additional overhead is removed by only checking packet taps if a socket is open on the device (vrf_dev->ptype_all list is not empty). Packet sockets bound to any device will still get a copy of the packet via the real ingress or egress interface. The end result of this change is a decrease in the overhead of VRF for the default, baseline case (ie., no netfilter rules, no packet sockets, no qdisc) to ~3% for UDP which has a lookup per packet and < 1% overhead for connected sockets that leverage early demux and avoid FIB lookups. Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-03-20 11:19:44 -07:00
vrf: revert "vrf: Remove unnecessary RCU-bh critical section" This reverts commit 504fc6f4f7f681d2a03aa5f68aad549d90eab853. dev_queue_xmit_nit is expected to be called with BH disabled. __dev_queue_xmit has the following: /* Disable soft irqs for various locks below. Also * stops preemption for RCU. */ rcu_read_lock_bh(); VRF must follow this invariant. The referenced commit removed this protection. Which triggered a lockdep warning: ================================ WARNING: inconsistent lock state 6.11.0 #1 Tainted: G W -------------------------------- inconsistent {IN-SOFTIRQ-W} -> {SOFTIRQ-ON-W} usage. btserver/134819 [HC0[0]:SC0[0]:HE1:SE1] takes: ffff8882da30c118 (rlock-AF_PACKET){+.?.}-{2:2}, at: tpacket_rcv+0x863/0x3b30 {IN-SOFTIRQ-W} state was registered at: lock_acquire+0x19a/0x4f0 _raw_spin_lock+0x27/0x40 packet_rcv+0xa33/0x1320 __netif_receive_skb_core.constprop.0+0xcb0/0x3a90 __netif_receive_skb_list_core+0x2c9/0x890 netif_receive_skb_list_internal+0x610/0xcc0 [...] other info that might help us debug this: Possible unsafe locking scenario: CPU0 ---- lock(rlock-AF_PACKET); <Interrupt> lock(rlock-AF_PACKET); *** DEADLOCK *** Call Trace: <TASK> dump_stack_lvl+0x73/0xa0 mark_lock+0x102e/0x16b0 __lock_acquire+0x9ae/0x6170 lock_acquire+0x19a/0x4f0 _raw_spin_lock+0x27/0x40 tpacket_rcv+0x863/0x3b30 dev_queue_xmit_nit+0x709/0xa40 vrf_finish_direct+0x26e/0x340 [vrf] vrf_l3_out+0x5f4/0xe80 [vrf] __ip_local_out+0x51e/0x7a0 [...] Fixes: 504fc6f4f7f6 ("vrf: Remove unnecessary RCU-bh critical section") Link: https://lore.kernel.org/netdev/20240925185216.1990381-1-greearb@candelatech.com/ Reported-by: Ben Greear <greearb@candelatech.com> Signed-off-by: Willem de Bruijn <willemb@google.com> Cc: stable@vger.kernel.org Reviewed-by: Ido Schimmel <idosch@nvidia.com> Tested-by: Ido Schimmel <idosch@nvidia.com> Reviewed-by: David Ahern <dsahern@kernel.org> Link: https://patch.msgid.link/20240929061839.1175300-1-willemdebruijn.kernel@gmail.com Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2024-09-29 02:18:20 -04:00
net: vrf: performance improvements for IPv4 The VRF driver allows users to implement device based features for an entire domain. For example, a qdisc or netfilter rules can be attached to a VRF device or tcpdump can be used to view packets for all devices in the L3 domain. The device-based features come with a performance penalty, most notably in the Tx path. The VRF driver uses the l3mdev_l3_out hook to switch the dst on an skb to its private dst. This allows the skb to traverse the xmit stack with the device set to the VRF device which in turn enables the netfilter and qdisc features. The VRF driver then performs the FIB lookup again and reinserts the packet. This patch avoids the redirect for IPv4 packets if a qdisc has not been attached to a VRF device which is the default config. In this case the netfilter hooks and network taps are directly traversed in the l3mdev_l3_out handler. If a qdisc is attached to a VRF device, then the redirect using the vrf dst is done. Additional overhead is removed by only checking packet taps if a socket is open on the device (vrf_dev->ptype_all list is not empty). Packet sockets bound to any device will still get a copy of the packet via the real ingress or egress interface. The end result of this change is a decrease in the overhead of VRF for the default, baseline case (ie., no netfilter rules, no packet sockets, no qdisc) to ~3% for UDP which has a lookup per packet and < 1% overhead for connected sockets that leverage early demux and avoid FIB lookups. Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-03-20 11:19:44 -07:00
net: vrf: performance improvements for IPv4 The VRF driver allows users to implement device based features for an entire domain. For example, a qdisc or netfilter rules can be attached to a VRF device or tcpdump can be used to view packets for all devices in the L3 domain. The device-based features come with a performance penalty, most notably in the Tx path. The VRF driver uses the l3mdev_l3_out hook to switch the dst on an skb to its private dst. This allows the skb to traverse the xmit stack with the device set to the VRF device which in turn enables the netfilter and qdisc features. The VRF driver then performs the FIB lookup again and reinserts the packet. This patch avoids the redirect for IPv4 packets if a qdisc has not been attached to a VRF device which is the default config. In this case the netfilter hooks and network taps are directly traversed in the l3mdev_l3_out handler. If a qdisc is attached to a VRF device, then the redirect using the vrf dst is done. Additional overhead is removed by only checking packet taps if a socket is open on the device (vrf_dev->ptype_all list is not empty). Packet sockets bound to any device will still get a copy of the packet via the real ingress or egress interface. The end result of this change is a decrease in the overhead of VRF for the default, baseline case (ie., no netfilter rules, no packet sockets, no qdisc) to ~3% for UDP which has a lookup per packet and < 1% overhead for connected sockets that leverage early demux and avoid FIB lookups. Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-03-20 11:19:44 -07:00
vrf: Fix lockdep splat in output path Cited commit converted the neighbour code to use the standard RCU variant instead of the RCU-bh variant, but the VRF code still uses rcu_read_lock_bh() / rcu_read_unlock_bh() around the neighbour lookup code in its IPv4 and IPv6 output paths, resulting in lockdep splats [1][2]. Can be reproduced using [3]. Fix by switching to rcu_read_lock() / rcu_read_unlock(). [1] ============================= WARNING: suspicious RCU usage 6.5.0-rc1-custom-g9c099e6dbf98 #403 Not tainted ----------------------------- include/net/neighbour.h:302 suspicious rcu_dereference_check() usage! other info that might help us debug this: rcu_scheduler_active = 2, debug_locks = 1 2 locks held by ping/183: #0: ffff888105ea1d80 (sk_lock-AF_INET){+.+.}-{0:0}, at: raw_sendmsg+0xc6c/0x33c0 #1: ffffffff85b46820 (rcu_read_lock_bh){....}-{1:2}, at: vrf_output+0x2e3/0x2030 stack backtrace: CPU: 0 PID: 183 Comm: ping Not tainted 6.5.0-rc1-custom-g9c099e6dbf98 #403 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.2-1.fc37 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0xc1/0xf0 lockdep_rcu_suspicious+0x211/0x3b0 vrf_output+0x1380/0x2030 ip_push_pending_frames+0x125/0x2a0 raw_sendmsg+0x200d/0x33c0 inet_sendmsg+0xa2/0xe0 __sys_sendto+0x2aa/0x420 __x64_sys_sendto+0xe5/0x1c0 do_syscall_64+0x38/0x80 entry_SYSCALL_64_after_hwframe+0x63/0xcd [2] ============================= WARNING: suspicious RCU usage 6.5.0-rc1-custom-g9c099e6dbf98 #403 Not tainted ----------------------------- include/net/neighbour.h:302 suspicious rcu_dereference_check() usage! other info that might help us debug this: rcu_scheduler_active = 2, debug_locks = 1 2 locks held by ping6/182: #0: ffff888114b63000 (sk_lock-AF_INET6){+.+.}-{0:0}, at: rawv6_sendmsg+0x1602/0x3e50 #1: ffffffff85b46820 (rcu_read_lock_bh){....}-{1:2}, at: vrf_output6+0xe9/0x1310 stack backtrace: CPU: 0 PID: 182 Comm: ping6 Not tainted 6.5.0-rc1-custom-g9c099e6dbf98 #403 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.2-1.fc37 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0xc1/0xf0 lockdep_rcu_suspicious+0x211/0x3b0 vrf_output6+0xd32/0x1310 ip6_local_out+0xb4/0x1a0 ip6_send_skb+0xbc/0x340 ip6_push_pending_frames+0xe5/0x110 rawv6_sendmsg+0x2e6e/0x3e50 inet_sendmsg+0xa2/0xe0 __sys_sendto+0x2aa/0x420 __x64_sys_sendto+0xe5/0x1c0 do_syscall_64+0x38/0x80 entry_SYSCALL_64_after_hwframe+0x63/0xcd [3] #!/bin/bash ip link add name vrf-red up numtxqueues 2 type vrf table 10 ip link add name swp1 up master vrf-red type dummy ip address add 192.0.2.1/24 dev swp1 ip address add 2001:db8:1::1/64 dev swp1 ip neigh add 192.0.2.2 lladdr 00:11:22:33:44:55 nud perm dev swp1 ip neigh add 2001:db8:1::2 lladdr 00:11:22:33:44:55 nud perm dev swp1 ip vrf exec vrf-red ping 192.0.2.2 -c 1 &> /dev/null ip vrf exec vrf-red ping6 2001:db8:1::2 -c 1 &> /dev/null Fixes: 09eed1192cec ("neighbour: switch to standard rcu, instead of rcu_bh") Reported-by: Naresh Kamboju <naresh.kamboju@linaro.org> Link: https://lore.kernel.org/netdev/CA+G9fYtEr-=GbcXNDYo3XOkwR+uYgehVoDjsP0pFLUpZ_AZcyg@mail.gmail.com/ Signed-off-by: Ido Schimmel <idosch@nvidia.com> Reviewed-by: David Ahern <dsahern@kernel.org> Reviewed-by: Eric Dumazet <edumazet@google.com> Link: https://lore.kernel.org/r/20230715153605.4068066-1-idosch@nvidia.com Signed-off-by: Paolo Abeni <pabeni@redhat.com>
2023-07-15 18:36:05 +03:00
vrf: Fix lockdep splat in output path Cited commit converted the neighbour code to use the standard RCU variant instead of the RCU-bh variant, but the VRF code still uses rcu_read_lock_bh() / rcu_read_unlock_bh() around the neighbour lookup code in its IPv4 and IPv6 output paths, resulting in lockdep splats [1][2]. Can be reproduced using [3]. Fix by switching to rcu_read_lock() / rcu_read_unlock(). [1] ============================= WARNING: suspicious RCU usage 6.5.0-rc1-custom-g9c099e6dbf98 #403 Not tainted ----------------------------- include/net/neighbour.h:302 suspicious rcu_dereference_check() usage! other info that might help us debug this: rcu_scheduler_active = 2, debug_locks = 1 2 locks held by ping/183: #0: ffff888105ea1d80 (sk_lock-AF_INET){+.+.}-{0:0}, at: raw_sendmsg+0xc6c/0x33c0 #1: ffffffff85b46820 (rcu_read_lock_bh){....}-{1:2}, at: vrf_output+0x2e3/0x2030 stack backtrace: CPU: 0 PID: 183 Comm: ping Not tainted 6.5.0-rc1-custom-g9c099e6dbf98 #403 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.2-1.fc37 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0xc1/0xf0 lockdep_rcu_suspicious+0x211/0x3b0 vrf_output+0x1380/0x2030 ip_push_pending_frames+0x125/0x2a0 raw_sendmsg+0x200d/0x33c0 inet_sendmsg+0xa2/0xe0 __sys_sendto+0x2aa/0x420 __x64_sys_sendto+0xe5/0x1c0 do_syscall_64+0x38/0x80 entry_SYSCALL_64_after_hwframe+0x63/0xcd [2] ============================= WARNING: suspicious RCU usage 6.5.0-rc1-custom-g9c099e6dbf98 #403 Not tainted ----------------------------- include/net/neighbour.h:302 suspicious rcu_dereference_check() usage! other info that might help us debug this: rcu_scheduler_active = 2, debug_locks = 1 2 locks held by ping6/182: #0: ffff888114b63000 (sk_lock-AF_INET6){+.+.}-{0:0}, at: rawv6_sendmsg+0x1602/0x3e50 #1: ffffffff85b46820 (rcu_read_lock_bh){....}-{1:2}, at: vrf_output6+0xe9/0x1310 stack backtrace: CPU: 0 PID: 182 Comm: ping6 Not tainted 6.5.0-rc1-custom-g9c099e6dbf98 #403 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.2-1.fc37 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0xc1/0xf0 lockdep_rcu_suspicious+0x211/0x3b0 vrf_output6+0xd32/0x1310 ip6_local_out+0xb4/0x1a0 ip6_send_skb+0xbc/0x340 ip6_push_pending_frames+0xe5/0x110 rawv6_sendmsg+0x2e6e/0x3e50 inet_sendmsg+0xa2/0xe0 __sys_sendto+0x2aa/0x420 __x64_sys_sendto+0xe5/0x1c0 do_syscall_64+0x38/0x80 entry_SYSCALL_64_after_hwframe+0x63/0xcd [3] #!/bin/bash ip link add name vrf-red up numtxqueues 2 type vrf table 10 ip link add name swp1 up master vrf-red type dummy ip address add 192.0.2.1/24 dev swp1 ip address add 2001:db8:1::1/64 dev swp1 ip neigh add 192.0.2.2 lladdr 00:11:22:33:44:55 nud perm dev swp1 ip neigh add 2001:db8:1::2 lladdr 00:11:22:33:44:55 nud perm dev swp1 ip vrf exec vrf-red ping 192.0.2.2 -c 1 &> /dev/null ip vrf exec vrf-red ping6 2001:db8:1::2 -c 1 &> /dev/null Fixes: 09eed1192cec ("neighbour: switch to standard rcu, instead of rcu_bh") Reported-by: Naresh Kamboju <naresh.kamboju@linaro.org> Link: https://lore.kernel.org/netdev/CA+G9fYtEr-=GbcXNDYo3XOkwR+uYgehVoDjsP0pFLUpZ_AZcyg@mail.gmail.com/ Signed-off-by: Ido Schimmel <idosch@nvidia.com> Reviewed-by: David Ahern <dsahern@kernel.org> Reviewed-by: Eric Dumazet <edumazet@google.com> Link: https://lore.kernel.org/r/20230715153605.4068066-1-idosch@nvidia.com Signed-off-by: Paolo Abeni <pabeni@redhat.com>
2023-07-15 18:36:05 +03:00
vrf: Fix lockdep splat in output path Cited commit converted the neighbour code to use the standard RCU variant instead of the RCU-bh variant, but the VRF code still uses rcu_read_lock_bh() / rcu_read_unlock_bh() around the neighbour lookup code in its IPv4 and IPv6 output paths, resulting in lockdep splats [1][2]. Can be reproduced using [3]. Fix by switching to rcu_read_lock() / rcu_read_unlock(). [1] ============================= WARNING: suspicious RCU usage 6.5.0-rc1-custom-g9c099e6dbf98 #403 Not tainted ----------------------------- include/net/neighbour.h:302 suspicious rcu_dereference_check() usage! other info that might help us debug this: rcu_scheduler_active = 2, debug_locks = 1 2 locks held by ping/183: #0: ffff888105ea1d80 (sk_lock-AF_INET){+.+.}-{0:0}, at: raw_sendmsg+0xc6c/0x33c0 #1: ffffffff85b46820 (rcu_read_lock_bh){....}-{1:2}, at: vrf_output+0x2e3/0x2030 stack backtrace: CPU: 0 PID: 183 Comm: ping Not tainted 6.5.0-rc1-custom-g9c099e6dbf98 #403 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.2-1.fc37 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0xc1/0xf0 lockdep_rcu_suspicious+0x211/0x3b0 vrf_output+0x1380/0x2030 ip_push_pending_frames+0x125/0x2a0 raw_sendmsg+0x200d/0x33c0 inet_sendmsg+0xa2/0xe0 __sys_sendto+0x2aa/0x420 __x64_sys_sendto+0xe5/0x1c0 do_syscall_64+0x38/0x80 entry_SYSCALL_64_after_hwframe+0x63/0xcd [2] ============================= WARNING: suspicious RCU usage 6.5.0-rc1-custom-g9c099e6dbf98 #403 Not tainted ----------------------------- include/net/neighbour.h:302 suspicious rcu_dereference_check() usage! other info that might help us debug this: rcu_scheduler_active = 2, debug_locks = 1 2 locks held by ping6/182: #0: ffff888114b63000 (sk_lock-AF_INET6){+.+.}-{0:0}, at: rawv6_sendmsg+0x1602/0x3e50 #1: ffffffff85b46820 (rcu_read_lock_bh){....}-{1:2}, at: vrf_output6+0xe9/0x1310 stack backtrace: CPU: 0 PID: 182 Comm: ping6 Not tainted 6.5.0-rc1-custom-g9c099e6dbf98 #403 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.2-1.fc37 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0xc1/0xf0 lockdep_rcu_suspicious+0x211/0x3b0 vrf_output6+0xd32/0x1310 ip6_local_out+0xb4/0x1a0 ip6_send_skb+0xbc/0x340 ip6_push_pending_frames+0xe5/0x110 rawv6_sendmsg+0x2e6e/0x3e50 inet_sendmsg+0xa2/0xe0 __sys_sendto+0x2aa/0x420 __x64_sys_sendto+0xe5/0x1c0 do_syscall_64+0x38/0x80 entry_SYSCALL_64_after_hwframe+0x63/0xcd [3] #!/bin/bash ip link add name vrf-red up numtxqueues 2 type vrf table 10 ip link add name swp1 up master vrf-red type dummy ip address add 192.0.2.1/24 dev swp1 ip address add 2001:db8:1::1/64 dev swp1 ip neigh add 192.0.2.2 lladdr 00:11:22:33:44:55 nud perm dev swp1 ip neigh add 2001:db8:1::2 lladdr 00:11:22:33:44:55 nud perm dev swp1 ip vrf exec vrf-red ping 192.0.2.2 -c 1 &> /dev/null ip vrf exec vrf-red ping6 2001:db8:1::2 -c 1 &> /dev/null Fixes: 09eed1192cec ("neighbour: switch to standard rcu, instead of rcu_bh") Reported-by: Naresh Kamboju <naresh.kamboju@linaro.org> Link: https://lore.kernel.org/netdev/CA+G9fYtEr-=GbcXNDYo3XOkwR+uYgehVoDjsP0pFLUpZ_AZcyg@mail.gmail.com/ Signed-off-by: Ido Schimmel <idosch@nvidia.com> Reviewed-by: David Ahern <dsahern@kernel.org> Reviewed-by: Eric Dumazet <edumazet@google.com> Link: https://lore.kernel.org/r/20230715153605.4068066-1-idosch@nvidia.com Signed-off-by: Paolo Abeni <pabeni@redhat.com>
2023-07-15 18:36:05 +03:00
net: vrf: performance improvements for IPv6 The VRF driver allows users to implement device based features for an entire domain. For example, a qdisc or netfilter rules can be attached to a VRF device or tcpdump can be used to view packets for all devices in the L3 domain. The device-based features come with a performance penalty, most notably in the Tx path. The VRF driver uses the l3mdev_l3_out hook to switch the dst on an skb to its private dst. This allows the skb to traverse the xmit stack with the device set to the VRF device which in turn enables the netfilter and qdisc features. The VRF driver then performs the FIB lookup again and reinserts the packet. This patch avoids the redirect for IPv6 packets if a qdisc has not been attached to a VRF device which is the default config. In this case the netfilter hooks and network taps are directly traversed in the l3mdev_l3_out handler. If a qdisc is attached to a VRF device, then the redirect using the vrf dst is done. Additional overhead is removed by only checking packet taps if a socket is open on the device (vrf_dev->ptype_all list is not empty). Packet sockets bound to any device will still get a copy of the packet via the real ingress or egress interface. The end result of this change is a decrease in the overhead of VRF for the default, baseline case (ie., no netfilter rules, no packet sockets, no qdisc) from a +3% improvement for UDP which has a lookup per packet (VRF being better than no l3mdev) to ~2% loss for TCP_CRR which connects a socket for each request-response. Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-03-20 11:19:45 -07:00
vrf: Remove unnecessary RCU protection around dst entries During initialization of a VRF device, the VRF driver creates two dst entries (for IPv4 and IPv6). They are attached to locally generated packets that are transmitted out of the VRF ports (via the l3mdev_l3_out() hook). Their purpose is to redirect packets towards the VRF device instead of having the packets egress directly out of the VRF ports. This is useful, for example, when a queuing discipline is configured on the VRF device. In order to avoid a NULL pointer dereference, commit b0e95ccdd775 ("net: vrf: protect changes to private data with rcu") made the pointers to the dst entries RCU protected. As far as I can tell, this was needed because back then the dst entries were released (and the pointers reset to NULL) before removing the VRF ports. Later on, commit f630c38ef0d7 ("vrf: fix bug_on triggered by rx when destroying a vrf") moved the removal of the VRF ports to the VRF device's dellink() callback. As such, the tear down sequence of a VRF device looks as follows: 1. VRF ports are removed. 2. VRF device is unregistered. a. Device is closed. b. An RCU grace period passes. c. ndo_uninit() is called. i. dst entries are released. Given the above, the Tx path will always see the same fully initialized dst entries and will never race with the ndo_uninit() callback. Therefore, there is no need to make the pointers to the dst entries RCU protected. Remove it as well as the unnecessary NULL checks in the Tx path. Signed-off-by: Ido Schimmel <idosch@nvidia.com> Reviewed-by: David Ahern <dsahern@kernel.org> Link: https://patch.msgid.link/20260326203233.1128554-4-idosch@nvidia.com Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2026-03-26 22:32:33 +02:00
vrf: Remove unnecessary RCU protection around dst entries During initialization of a VRF device, the VRF driver creates two dst entries (for IPv4 and IPv6). They are attached to locally generated packets that are transmitted out of the VRF ports (via the l3mdev_l3_out() hook). Their purpose is to redirect packets towards the VRF device instead of having the packets egress directly out of the VRF ports. This is useful, for example, when a queuing discipline is configured on the VRF device. In order to avoid a NULL pointer dereference, commit b0e95ccdd775 ("net: vrf: protect changes to private data with rcu") made the pointers to the dst entries RCU protected. As far as I can tell, this was needed because back then the dst entries were released (and the pointers reset to NULL) before removing the VRF ports. Later on, commit f630c38ef0d7 ("vrf: fix bug_on triggered by rx when destroying a vrf") moved the removal of the VRF ports to the VRF device's dellink() callback. As such, the tear down sequence of a VRF device looks as follows: 1. VRF ports are removed. 2. VRF device is unregistered. a. Device is closed. b. An RCU grace period passes. c. ndo_uninit() is called. i. dst entries are released. Given the above, the Tx path will always see the same fully initialized dst entries and will never race with the ndo_uninit() callback. Therefore, there is no need to make the pointers to the dst entries RCU protected. Remove it as well as the unnecessary NULL checks in the Tx path. Signed-off-by: Ido Schimmel <idosch@nvidia.com> Reviewed-by: David Ahern <dsahern@kernel.org> Link: https://patch.msgid.link/20260326203233.1128554-4-idosch@nvidia.com Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2026-03-26 22:32:33 +02:00
net: vrf: performance improvements for IPv6 The VRF driver allows users to implement device based features for an entire domain. For example, a qdisc or netfilter rules can be attached to a VRF device or tcpdump can be used to view packets for all devices in the L3 domain. The device-based features come with a performance penalty, most notably in the Tx path. The VRF driver uses the l3mdev_l3_out hook to switch the dst on an skb to its private dst. This allows the skb to traverse the xmit stack with the device set to the VRF device which in turn enables the netfilter and qdisc features. The VRF driver then performs the FIB lookup again and reinserts the packet. This patch avoids the redirect for IPv6 packets if a qdisc has not been attached to a VRF device which is the default config. In this case the netfilter hooks and network taps are directly traversed in the l3mdev_l3_out handler. If a qdisc is attached to a VRF device, then the redirect using the vrf dst is done. Additional overhead is removed by only checking packet taps if a socket is open on the device (vrf_dev->ptype_all list is not empty). Packet sockets bound to any device will still get a copy of the packet via the real ingress or egress interface. The end result of this change is a decrease in the overhead of VRF for the default, baseline case (ie., no netfilter rules, no packet sockets, no qdisc) from a +3% improvement for UDP which has a lookup per packet (VRF being better than no l3mdev) to ~2% loss for TCP_CRR which connects a socket for each request-response. Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-03-20 11:19:45 -07:00
net: vrf: performance improvements for IPv6 The VRF driver allows users to implement device based features for an entire domain. For example, a qdisc or netfilter rules can be attached to a VRF device or tcpdump can be used to view packets for all devices in the L3 domain. The device-based features come with a performance penalty, most notably in the Tx path. The VRF driver uses the l3mdev_l3_out hook to switch the dst on an skb to its private dst. This allows the skb to traverse the xmit stack with the device set to the VRF device which in turn enables the netfilter and qdisc features. The VRF driver then performs the FIB lookup again and reinserts the packet. This patch avoids the redirect for IPv6 packets if a qdisc has not been attached to a VRF device which is the default config. In this case the netfilter hooks and network taps are directly traversed in the l3mdev_l3_out handler. If a qdisc is attached to a VRF device, then the redirect using the vrf dst is done. Additional overhead is removed by only checking packet taps if a socket is open on the device (vrf_dev->ptype_all list is not empty). Packet sockets bound to any device will still get a copy of the packet via the real ingress or egress interface. The end result of this change is a decrease in the overhead of VRF for the default, baseline case (ie., no netfilter rules, no packet sockets, no qdisc) from a +3% improvement for UDP which has a lookup per packet (VRF being better than no l3mdev) to ~2% loss for TCP_CRR which connects a socket for each request-response. Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-03-20 11:19:45 -07:00
net: vrf: performance improvements for IPv6 The VRF driver allows users to implement device based features for an entire domain. For example, a qdisc or netfilter rules can be attached to a VRF device or tcpdump can be used to view packets for all devices in the L3 domain. The device-based features come with a performance penalty, most notably in the Tx path. The VRF driver uses the l3mdev_l3_out hook to switch the dst on an skb to its private dst. This allows the skb to traverse the xmit stack with the device set to the VRF device which in turn enables the netfilter and qdisc features. The VRF driver then performs the FIB lookup again and reinserts the packet. This patch avoids the redirect for IPv6 packets if a qdisc has not been attached to a VRF device which is the default config. In this case the netfilter hooks and network taps are directly traversed in the l3mdev_l3_out handler. If a qdisc is attached to a VRF device, then the redirect using the vrf dst is done. Additional overhead is removed by only checking packet taps if a socket is open on the device (vrf_dev->ptype_all list is not empty). Packet sockets bound to any device will still get a copy of the packet via the real ingress or egress interface. The end result of this change is a decrease in the overhead of VRF for the default, baseline case (ie., no netfilter rules, no packet sockets, no qdisc) from a +3% improvement for UDP which has a lookup per packet (VRF being better than no l3mdev) to ~2% loss for TCP_CRR which connects a socket for each request-response. Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-03-20 11:19:45 -07:00
net: vrf: performance improvements for IPv6 The VRF driver allows users to implement device based features for an entire domain. For example, a qdisc or netfilter rules can be attached to a VRF device or tcpdump can be used to view packets for all devices in the L3 domain. The device-based features come with a performance penalty, most notably in the Tx path. The VRF driver uses the l3mdev_l3_out hook to switch the dst on an skb to its private dst. This allows the skb to traverse the xmit stack with the device set to the VRF device which in turn enables the netfilter and qdisc features. The VRF driver then performs the FIB lookup again and reinserts the packet. This patch avoids the redirect for IPv6 packets if a qdisc has not been attached to a VRF device which is the default config. In this case the netfilter hooks and network taps are directly traversed in the l3mdev_l3_out handler. If a qdisc is attached to a VRF device, then the redirect using the vrf dst is done. Additional overhead is removed by only checking packet taps if a socket is open on the device (vrf_dev->ptype_all list is not empty). Packet sockets bound to any device will still get a copy of the packet via the real ingress or egress interface. The end result of this change is a decrease in the overhead of VRF for the default, baseline case (ie., no netfilter rules, no packet sockets, no qdisc) from a +3% improvement for UDP which has a lookup per packet (VRF being better than no l3mdev) to ~2% loss for TCP_CRR which connects a socket for each request-response. Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-03-20 11:19:45 -07:00
net: vrf: performance improvements for IPv6 The VRF driver allows users to implement device based features for an entire domain. For example, a qdisc or netfilter rules can be attached to a VRF device or tcpdump can be used to view packets for all devices in the L3 domain. The device-based features come with a performance penalty, most notably in the Tx path. The VRF driver uses the l3mdev_l3_out hook to switch the dst on an skb to its private dst. This allows the skb to traverse the xmit stack with the device set to the VRF device which in turn enables the netfilter and qdisc features. The VRF driver then performs the FIB lookup again and reinserts the packet. This patch avoids the redirect for IPv6 packets if a qdisc has not been attached to a VRF device which is the default config. In this case the netfilter hooks and network taps are directly traversed in the l3mdev_l3_out handler. If a qdisc is attached to a VRF device, then the redirect using the vrf dst is done. Additional overhead is removed by only checking packet taps if a socket is open on the device (vrf_dev->ptype_all list is not empty). Packet sockets bound to any device will still get a copy of the packet via the real ingress or egress interface. The end result of this change is a decrease in the overhead of VRF for the default, baseline case (ie., no netfilter rules, no packet sockets, no qdisc) from a +3% improvement for UDP which has a lookup per packet (VRF being better than no l3mdev) to ~2% loss for TCP_CRR which connects a socket for each request-response. Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-03-20 11:19:45 -07:00
net: vrf: performance improvements for IPv6 The VRF driver allows users to implement device based features for an entire domain. For example, a qdisc or netfilter rules can be attached to a VRF device or tcpdump can be used to view packets for all devices in the L3 domain. The device-based features come with a performance penalty, most notably in the Tx path. The VRF driver uses the l3mdev_l3_out hook to switch the dst on an skb to its private dst. This allows the skb to traverse the xmit stack with the device set to the VRF device which in turn enables the netfilter and qdisc features. The VRF driver then performs the FIB lookup again and reinserts the packet. This patch avoids the redirect for IPv6 packets if a qdisc has not been attached to a VRF device which is the default config. In this case the netfilter hooks and network taps are directly traversed in the l3mdev_l3_out handler. If a qdisc is attached to a VRF device, then the redirect using the vrf dst is done. Additional overhead is removed by only checking packet taps if a socket is open on the device (vrf_dev->ptype_all list is not empty). Packet sockets bound to any device will still get a copy of the packet via the real ingress or egress interface. The end result of this change is a decrease in the overhead of VRF for the default, baseline case (ie., no netfilter rules, no packet sockets, no qdisc) from a +3% improvement for UDP which has a lookup per packet (VRF being better than no l3mdev) to ~2% loss for TCP_CRR which connects a socket for each request-response. Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-03-20 11:19:45 -07:00
net: vrf: performance improvements for IPv6 The VRF driver allows users to implement device based features for an entire domain. For example, a qdisc or netfilter rules can be attached to a VRF device or tcpdump can be used to view packets for all devices in the L3 domain. The device-based features come with a performance penalty, most notably in the Tx path. The VRF driver uses the l3mdev_l3_out hook to switch the dst on an skb to its private dst. This allows the skb to traverse the xmit stack with the device set to the VRF device which in turn enables the netfilter and qdisc features. The VRF driver then performs the FIB lookup again and reinserts the packet. This patch avoids the redirect for IPv6 packets if a qdisc has not been attached to a VRF device which is the default config. In this case the netfilter hooks and network taps are directly traversed in the l3mdev_l3_out handler. If a qdisc is attached to a VRF device, then the redirect using the vrf dst is done. Additional overhead is removed by only checking packet taps if a socket is open on the device (vrf_dev->ptype_all list is not empty). Packet sockets bound to any device will still get a copy of the packet via the real ingress or egress interface. The end result of this change is a decrease in the overhead of VRF for the default, baseline case (ie., no netfilter rules, no packet sockets, no qdisc) from a +3% improvement for UDP which has a lookup per packet (VRF being better than no l3mdev) to ~2% loss for TCP_CRR which connects a socket for each request-response. Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-03-20 11:19:45 -07:00
vrf: Remove unnecessary RCU protection around dst entries During initialization of a VRF device, the VRF driver creates two dst entries (for IPv4 and IPv6). They are attached to locally generated packets that are transmitted out of the VRF ports (via the l3mdev_l3_out() hook). Their purpose is to redirect packets towards the VRF device instead of having the packets egress directly out of the VRF ports. This is useful, for example, when a queuing discipline is configured on the VRF device. In order to avoid a NULL pointer dereference, commit b0e95ccdd775 ("net: vrf: protect changes to private data with rcu") made the pointers to the dst entries RCU protected. As far as I can tell, this was needed because back then the dst entries were released (and the pointers reset to NULL) before removing the VRF ports. Later on, commit f630c38ef0d7 ("vrf: fix bug_on triggered by rx when destroying a vrf") moved the removal of the VRF ports to the VRF device's dellink() callback. As such, the tear down sequence of a VRF device looks as follows: 1. VRF ports are removed. 2. VRF device is unregistered. a. Device is closed. b. An RCU grace period passes. c. ndo_uninit() is called. i. dst entries are released. Given the above, the Tx path will always see the same fully initialized dst entries and will never race with the ndo_uninit() callback. Therefore, there is no need to make the pointers to the dst entries RCU protected. Remove it as well as the unnecessary NULL checks in the Tx path. Signed-off-by: Ido Schimmel <idosch@nvidia.com> Reviewed-by: David Ahern <dsahern@kernel.org> Link: https://patch.msgid.link/20260326203233.1128554-4-idosch@nvidia.com Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2026-03-26 22:32:33 +02:00
net: vrf: Fix crash when IPv6 is disabled at boot time Frank Kellermann reported a kernel crash with 4.5.0 when IPv6 is disabled at boot using the kernel option ipv6.disable=1. Using current net-next with the boot option: $ ip link add red type vrf table 1001 Generates: [12210.919584] BUG: unable to handle kernel NULL pointer dereference at 0000000000000748 [12210.921341] IP: [<ffffffff814b30e3>] fib6_get_table+0x2c/0x5a [12210.922537] PGD b79e3067 PUD bb32b067 PMD 0 [12210.923479] Oops: 0000 [#1] SMP [12210.924001] Modules linked in: ipvlan 8021q garp mrp stp llc [12210.925130] CPU: 3 PID: 1177 Comm: ip Not tainted 4.7.0-rc1+ #235 [12210.926168] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.7.5-20140531_083030-gandalf 04/01/2014 [12210.928065] task: ffff8800b9ac4640 ti: ffff8800bacac000 task.ti: ffff8800bacac000 [12210.929328] RIP: 0010:[<ffffffff814b30e3>] [<ffffffff814b30e3>] fib6_get_table+0x2c/0x5a [12210.930697] RSP: 0018:ffff8800bacaf888 EFLAGS: 00010202 [12210.931563] RAX: 0000000000000748 RBX: ffffffff81a9e280 RCX: ffff8800b9ac4e28 [12210.932688] RDX: 00000000000000e9 RSI: 0000000000000002 RDI: 0000000000000286 [12210.933820] RBP: ffff8800bacaf898 R08: ffff8800b9ac4df0 R09: 000000000052001b [12210.934941] R10: 00000000657c0000 R11: 000000000000c649 R12: 00000000000003e9 [12210.936032] R13: 00000000000003e9 R14: ffff8800bace7800 R15: ffff8800bb3ec000 [12210.937103] FS: 00007faa1766c700(0000) GS:ffff88013ac00000(0000) knlGS:0000000000000000 [12210.938321] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [12210.939166] CR2: 0000000000000748 CR3: 00000000b79d6000 CR4: 00000000000406e0 [12210.940278] Stack: [12210.940603] ffff8800bb3ec000 ffffffff81a9e280 ffff8800bacaf8c8 ffffffff814b3135 [12210.941818] ffff8800bb3ec000 ffffffff81a9e280 ffffffff81a9e280 ffff8800bace7800 [12210.943040] ffff8800bacaf8f0 ffffffff81397c88 ffff8800bb3ec000 ffffffff81a9e280 [12210.944288] Call Trace: [12210.944688] [<ffffffff814b3135>] fib6_new_table+0x24/0x8a [12210.945516] [<ffffffff81397c88>] vrf_dev_init+0xd4/0x162 [12210.946328] [<ffffffff814091e1>] register_netdevice+0x100/0x396 [12210.947209] [<ffffffff8139823d>] vrf_newlink+0x40/0xb3 [12210.948001] [<ffffffff814187f0>] rtnl_newlink+0x5d3/0x6d5 ... The problem above is due to the fact that the fib hash table is not allocated when IPv6 is disabled at boot. As for the VRF driver it should not do any IPv6 initializations if IPv6 is disabled, so it needs to know if IPv6 is disabled at boot. The disable parameter is private to the IPv6 module, so provide an accessor for modules to determine if IPv6 was disabled at boot time. Fixes: 35402e3136634 ("net: Add IPv6 support to VRF device") Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2016-06-09 10:21:00 -07:00
vrf: Remove unnecessary RCU protection around dst entries During initialization of a VRF device, the VRF driver creates two dst entries (for IPv4 and IPv6). They are attached to locally generated packets that are transmitted out of the VRF ports (via the l3mdev_l3_out() hook). Their purpose is to redirect packets towards the VRF device instead of having the packets egress directly out of the VRF ports. This is useful, for example, when a queuing discipline is configured on the VRF device. In order to avoid a NULL pointer dereference, commit b0e95ccdd775 ("net: vrf: protect changes to private data with rcu") made the pointers to the dst entries RCU protected. As far as I can tell, this was needed because back then the dst entries were released (and the pointers reset to NULL) before removing the VRF ports. Later on, commit f630c38ef0d7 ("vrf: fix bug_on triggered by rx when destroying a vrf") moved the removal of the VRF ports to the VRF device's dellink() callback. As such, the tear down sequence of a VRF device looks as follows: 1. VRF ports are removed. 2. VRF device is unregistered. a. Device is closed. b. An RCU grace period passes. c. ndo_uninit() is called. i. dst entries are released. Given the above, the Tx path will always see the same fully initialized dst entries and will never race with the ndo_uninit() callback. Therefore, there is no need to make the pointers to the dst entries RCU protected. Remove it as well as the unnecessary NULL checks in the Tx path. Signed-off-by: Ido Schimmel <idosch@nvidia.com> Reviewed-by: David Ahern <dsahern@kernel.org> Link: https://patch.msgid.link/20260326203233.1128554-4-idosch@nvidia.com Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2026-03-26 22:32:33 +02:00
ipv4: Add helpers for neigh lookup for nexthop A common theme in the output path is looking up a neigh entry for a nexthop, either the gateway in an rtable or a fallback to the daddr in the skb: nexthop = (__force u32)rt_nexthop(rt, ip_hdr(skb)->daddr); neigh = __ipv4_neigh_lookup_noref(dev, nexthop); if (unlikely(!neigh)) neigh = __neigh_create(&arp_tbl, &nexthop, dev, false); To allow the nexthop to be an IPv6 address we need to consider the family of the nexthop and then call __ipv{4,6}_neigh_lookup_noref based on it. To make this simpler, add a ip_neigh_gw4 helper similar to ip_neigh_gw6 added in an earlier patch which handles: neigh = __ipv4_neigh_lookup_noref(dev, nexthop); if (unlikely(!neigh)) neigh = __neigh_create(&arp_tbl, &nexthop, dev, false); And then add a second one, ip_neigh_for_gw, that calls either ip_neigh_gw4 or ip_neigh_gw6 based on the address family of the gateway. Update the output paths in the VRF driver and core v4 code to use ip_neigh_for_gw simplifying the family based lookup and making both ready for a v6 nexthop. ipv4_neigh_lookup has a different need - the potential to resolve a passed in address in addition to any gateway in the rtable or skb. Since this is a one-off, add ip_neigh_gw4 and ip_neigh_gw6 diectly. The difference between __neigh_create used by the helpers and neigh_create called by ipv4_neigh_lookup is taking a refcount, so add rcu_read_lock_bh and bump the refcnt on the neigh entry. Signed-off-by: David Ahern <dsahern@gmail.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2019-04-05 16:30:34 -07:00
vrf: Fix lockdep splat in output path Cited commit converted the neighbour code to use the standard RCU variant instead of the RCU-bh variant, but the VRF code still uses rcu_read_lock_bh() / rcu_read_unlock_bh() around the neighbour lookup code in its IPv4 and IPv6 output paths, resulting in lockdep splats [1][2]. Can be reproduced using [3]. Fix by switching to rcu_read_lock() / rcu_read_unlock(). [1] ============================= WARNING: suspicious RCU usage 6.5.0-rc1-custom-g9c099e6dbf98 #403 Not tainted ----------------------------- include/net/neighbour.h:302 suspicious rcu_dereference_check() usage! other info that might help us debug this: rcu_scheduler_active = 2, debug_locks = 1 2 locks held by ping/183: #0: ffff888105ea1d80 (sk_lock-AF_INET){+.+.}-{0:0}, at: raw_sendmsg+0xc6c/0x33c0 #1: ffffffff85b46820 (rcu_read_lock_bh){....}-{1:2}, at: vrf_output+0x2e3/0x2030 stack backtrace: CPU: 0 PID: 183 Comm: ping Not tainted 6.5.0-rc1-custom-g9c099e6dbf98 #403 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.2-1.fc37 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0xc1/0xf0 lockdep_rcu_suspicious+0x211/0x3b0 vrf_output+0x1380/0x2030 ip_push_pending_frames+0x125/0x2a0 raw_sendmsg+0x200d/0x33c0 inet_sendmsg+0xa2/0xe0 __sys_sendto+0x2aa/0x420 __x64_sys_sendto+0xe5/0x1c0 do_syscall_64+0x38/0x80 entry_SYSCALL_64_after_hwframe+0x63/0xcd [2] ============================= WARNING: suspicious RCU usage 6.5.0-rc1-custom-g9c099e6dbf98 #403 Not tainted ----------------------------- include/net/neighbour.h:302 suspicious rcu_dereference_check() usage! other info that might help us debug this: rcu_scheduler_active = 2, debug_locks = 1 2 locks held by ping6/182: #0: ffff888114b63000 (sk_lock-AF_INET6){+.+.}-{0:0}, at: rawv6_sendmsg+0x1602/0x3e50 #1: ffffffff85b46820 (rcu_read_lock_bh){....}-{1:2}, at: vrf_output6+0xe9/0x1310 stack backtrace: CPU: 0 PID: 182 Comm: ping6 Not tainted 6.5.0-rc1-custom-g9c099e6dbf98 #403 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.2-1.fc37 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0xc1/0xf0 lockdep_rcu_suspicious+0x211/0x3b0 vrf_output6+0xd32/0x1310 ip6_local_out+0xb4/0x1a0 ip6_send_skb+0xbc/0x340 ip6_push_pending_frames+0xe5/0x110 rawv6_sendmsg+0x2e6e/0x3e50 inet_sendmsg+0xa2/0xe0 __sys_sendto+0x2aa/0x420 __x64_sys_sendto+0xe5/0x1c0 do_syscall_64+0x38/0x80 entry_SYSCALL_64_after_hwframe+0x63/0xcd [3] #!/bin/bash ip link add name vrf-red up numtxqueues 2 type vrf table 10 ip link add name swp1 up master vrf-red type dummy ip address add 192.0.2.1/24 dev swp1 ip address add 2001:db8:1::1/64 dev swp1 ip neigh add 192.0.2.2 lladdr 00:11:22:33:44:55 nud perm dev swp1 ip neigh add 2001:db8:1::2 lladdr 00:11:22:33:44:55 nud perm dev swp1 ip vrf exec vrf-red ping 192.0.2.2 -c 1 &> /dev/null ip vrf exec vrf-red ping6 2001:db8:1::2 -c 1 &> /dev/null Fixes: 09eed1192cec ("neighbour: switch to standard rcu, instead of rcu_bh") Reported-by: Naresh Kamboju <naresh.kamboju@linaro.org> Link: https://lore.kernel.org/netdev/CA+G9fYtEr-=GbcXNDYo3XOkwR+uYgehVoDjsP0pFLUpZ_AZcyg@mail.gmail.com/ Signed-off-by: Ido Schimmel <idosch@nvidia.com> Reviewed-by: David Ahern <dsahern@kernel.org> Reviewed-by: Eric Dumazet <edumazet@google.com> Link: https://lore.kernel.org/r/20230715153605.4068066-1-idosch@nvidia.com Signed-off-by: Paolo Abeni <pabeni@redhat.com>
2023-07-15 18:36:05 +03:00
ipv4: Add helpers for neigh lookup for nexthop A common theme in the output path is looking up a neigh entry for a nexthop, either the gateway in an rtable or a fallback to the daddr in the skb: nexthop = (__force u32)rt_nexthop(rt, ip_hdr(skb)->daddr); neigh = __ipv4_neigh_lookup_noref(dev, nexthop); if (unlikely(!neigh)) neigh = __neigh_create(&arp_tbl, &nexthop, dev, false); To allow the nexthop to be an IPv6 address we need to consider the family of the nexthop and then call __ipv{4,6}_neigh_lookup_noref based on it. To make this simpler, add a ip_neigh_gw4 helper similar to ip_neigh_gw6 added in an earlier patch which handles: neigh = __ipv4_neigh_lookup_noref(dev, nexthop); if (unlikely(!neigh)) neigh = __neigh_create(&arp_tbl, &nexthop, dev, false); And then add a second one, ip_neigh_for_gw, that calls either ip_neigh_gw4 or ip_neigh_gw6 based on the address family of the gateway. Update the output paths in the VRF driver and core v4 code to use ip_neigh_for_gw simplifying the family based lookup and making both ready for a v6 nexthop. ipv4_neigh_lookup has a different need - the potential to resolve a passed in address in addition to any gateway in the rtable or skb. Since this is a one-off, add ip_neigh_gw4 and ip_neigh_gw6 diectly. The difference between __neigh_create used by the helpers and neigh_create called by ipv4_neigh_lookup is taking a refcount, so add rcu_read_lock_bh and bump the refcnt on the neigh entry. Signed-off-by: David Ahern <dsahern@gmail.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2019-04-05 16:30:34 -07:00
ipv4: Add helpers for neigh lookup for nexthop A common theme in the output path is looking up a neigh entry for a nexthop, either the gateway in an rtable or a fallback to the daddr in the skb: nexthop = (__force u32)rt_nexthop(rt, ip_hdr(skb)->daddr); neigh = __ipv4_neigh_lookup_noref(dev, nexthop); if (unlikely(!neigh)) neigh = __neigh_create(&arp_tbl, &nexthop, dev, false); To allow the nexthop to be an IPv6 address we need to consider the family of the nexthop and then call __ipv{4,6}_neigh_lookup_noref based on it. To make this simpler, add a ip_neigh_gw4 helper similar to ip_neigh_gw6 added in an earlier patch which handles: neigh = __ipv4_neigh_lookup_noref(dev, nexthop); if (unlikely(!neigh)) neigh = __neigh_create(&arp_tbl, &nexthop, dev, false); And then add a second one, ip_neigh_for_gw, that calls either ip_neigh_gw4 or ip_neigh_gw6 based on the address family of the gateway. Update the output paths in the VRF driver and core v4 code to use ip_neigh_for_gw simplifying the family based lookup and making both ready for a v6 nexthop. ipv4_neigh_lookup has a different need - the potential to resolve a passed in address in addition to any gateway in the rtable or skb. Since this is a one-off, add ip_neigh_gw4 and ip_neigh_gw6 diectly. The difference between __neigh_create used by the helpers and neigh_create called by ipv4_neigh_lookup is taking a refcount, so add rcu_read_lock_bh and bump the refcnt on the neigh entry. Signed-off-by: David Ahern <dsahern@gmail.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2019-04-05 16:30:34 -07:00
vrf: Fix lockdep splat in output path Cited commit converted the neighbour code to use the standard RCU variant instead of the RCU-bh variant, but the VRF code still uses rcu_read_lock_bh() / rcu_read_unlock_bh() around the neighbour lookup code in its IPv4 and IPv6 output paths, resulting in lockdep splats [1][2]. Can be reproduced using [3]. Fix by switching to rcu_read_lock() / rcu_read_unlock(). [1] ============================= WARNING: suspicious RCU usage 6.5.0-rc1-custom-g9c099e6dbf98 #403 Not tainted ----------------------------- include/net/neighbour.h:302 suspicious rcu_dereference_check() usage! other info that might help us debug this: rcu_scheduler_active = 2, debug_locks = 1 2 locks held by ping/183: #0: ffff888105ea1d80 (sk_lock-AF_INET){+.+.}-{0:0}, at: raw_sendmsg+0xc6c/0x33c0 #1: ffffffff85b46820 (rcu_read_lock_bh){....}-{1:2}, at: vrf_output+0x2e3/0x2030 stack backtrace: CPU: 0 PID: 183 Comm: ping Not tainted 6.5.0-rc1-custom-g9c099e6dbf98 #403 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.2-1.fc37 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0xc1/0xf0 lockdep_rcu_suspicious+0x211/0x3b0 vrf_output+0x1380/0x2030 ip_push_pending_frames+0x125/0x2a0 raw_sendmsg+0x200d/0x33c0 inet_sendmsg+0xa2/0xe0 __sys_sendto+0x2aa/0x420 __x64_sys_sendto+0xe5/0x1c0 do_syscall_64+0x38/0x80 entry_SYSCALL_64_after_hwframe+0x63/0xcd [2] ============================= WARNING: suspicious RCU usage 6.5.0-rc1-custom-g9c099e6dbf98 #403 Not tainted ----------------------------- include/net/neighbour.h:302 suspicious rcu_dereference_check() usage! other info that might help us debug this: rcu_scheduler_active = 2, debug_locks = 1 2 locks held by ping6/182: #0: ffff888114b63000 (sk_lock-AF_INET6){+.+.}-{0:0}, at: rawv6_sendmsg+0x1602/0x3e50 #1: ffffffff85b46820 (rcu_read_lock_bh){....}-{1:2}, at: vrf_output6+0xe9/0x1310 stack backtrace: CPU: 0 PID: 182 Comm: ping6 Not tainted 6.5.0-rc1-custom-g9c099e6dbf98 #403 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.2-1.fc37 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0xc1/0xf0 lockdep_rcu_suspicious+0x211/0x3b0 vrf_output6+0xd32/0x1310 ip6_local_out+0xb4/0x1a0 ip6_send_skb+0xbc/0x340 ip6_push_pending_frames+0xe5/0x110 rawv6_sendmsg+0x2e6e/0x3e50 inet_sendmsg+0xa2/0xe0 __sys_sendto+0x2aa/0x420 __x64_sys_sendto+0xe5/0x1c0 do_syscall_64+0x38/0x80 entry_SYSCALL_64_after_hwframe+0x63/0xcd [3] #!/bin/bash ip link add name vrf-red up numtxqueues 2 type vrf table 10 ip link add name swp1 up master vrf-red type dummy ip address add 192.0.2.1/24 dev swp1 ip address add 2001:db8:1::1/64 dev swp1 ip neigh add 192.0.2.2 lladdr 00:11:22:33:44:55 nud perm dev swp1 ip neigh add 2001:db8:1::2 lladdr 00:11:22:33:44:55 nud perm dev swp1 ip vrf exec vrf-red ping 192.0.2.2 -c 1 &> /dev/null ip vrf exec vrf-red ping6 2001:db8:1::2 -c 1 &> /dev/null Fixes: 09eed1192cec ("neighbour: switch to standard rcu, instead of rcu_bh") Reported-by: Naresh Kamboju <naresh.kamboju@linaro.org> Link: https://lore.kernel.org/netdev/CA+G9fYtEr-=GbcXNDYo3XOkwR+uYgehVoDjsP0pFLUpZ_AZcyg@mail.gmail.com/ Signed-off-by: Ido Schimmel <idosch@nvidia.com> Reviewed-by: David Ahern <dsahern@kernel.org> Reviewed-by: Eric Dumazet <edumazet@google.com> Link: https://lore.kernel.org/r/20230715153605.4068066-1-idosch@nvidia.com Signed-off-by: Paolo Abeni <pabeni@redhat.com>
2023-07-15 18:36:05 +03:00
vrf: Fix lockdep splat in output path Cited commit converted the neighbour code to use the standard RCU variant instead of the RCU-bh variant, but the VRF code still uses rcu_read_lock_bh() / rcu_read_unlock_bh() around the neighbour lookup code in its IPv4 and IPv6 output paths, resulting in lockdep splats [1][2]. Can be reproduced using [3]. Fix by switching to rcu_read_lock() / rcu_read_unlock(). [1] ============================= WARNING: suspicious RCU usage 6.5.0-rc1-custom-g9c099e6dbf98 #403 Not tainted ----------------------------- include/net/neighbour.h:302 suspicious rcu_dereference_check() usage! other info that might help us debug this: rcu_scheduler_active = 2, debug_locks = 1 2 locks held by ping/183: #0: ffff888105ea1d80 (sk_lock-AF_INET){+.+.}-{0:0}, at: raw_sendmsg+0xc6c/0x33c0 #1: ffffffff85b46820 (rcu_read_lock_bh){....}-{1:2}, at: vrf_output+0x2e3/0x2030 stack backtrace: CPU: 0 PID: 183 Comm: ping Not tainted 6.5.0-rc1-custom-g9c099e6dbf98 #403 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.2-1.fc37 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0xc1/0xf0 lockdep_rcu_suspicious+0x211/0x3b0 vrf_output+0x1380/0x2030 ip_push_pending_frames+0x125/0x2a0 raw_sendmsg+0x200d/0x33c0 inet_sendmsg+0xa2/0xe0 __sys_sendto+0x2aa/0x420 __x64_sys_sendto+0xe5/0x1c0 do_syscall_64+0x38/0x80 entry_SYSCALL_64_after_hwframe+0x63/0xcd [2] ============================= WARNING: suspicious RCU usage 6.5.0-rc1-custom-g9c099e6dbf98 #403 Not tainted ----------------------------- include/net/neighbour.h:302 suspicious rcu_dereference_check() usage! other info that might help us debug this: rcu_scheduler_active = 2, debug_locks = 1 2 locks held by ping6/182: #0: ffff888114b63000 (sk_lock-AF_INET6){+.+.}-{0:0}, at: rawv6_sendmsg+0x1602/0x3e50 #1: ffffffff85b46820 (rcu_read_lock_bh){....}-{1:2}, at: vrf_output6+0xe9/0x1310 stack backtrace: CPU: 0 PID: 182 Comm: ping6 Not tainted 6.5.0-rc1-custom-g9c099e6dbf98 #403 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.2-1.fc37 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0xc1/0xf0 lockdep_rcu_suspicious+0x211/0x3b0 vrf_output6+0xd32/0x1310 ip6_local_out+0xb4/0x1a0 ip6_send_skb+0xbc/0x340 ip6_push_pending_frames+0xe5/0x110 rawv6_sendmsg+0x2e6e/0x3e50 inet_sendmsg+0xa2/0xe0 __sys_sendto+0x2aa/0x420 __x64_sys_sendto+0xe5/0x1c0 do_syscall_64+0x38/0x80 entry_SYSCALL_64_after_hwframe+0x63/0xcd [3] #!/bin/bash ip link add name vrf-red up numtxqueues 2 type vrf table 10 ip link add name swp1 up master vrf-red type dummy ip address add 192.0.2.1/24 dev swp1 ip address add 2001:db8:1::1/64 dev swp1 ip neigh add 192.0.2.2 lladdr 00:11:22:33:44:55 nud perm dev swp1 ip neigh add 2001:db8:1::2 lladdr 00:11:22:33:44:55 nud perm dev swp1 ip vrf exec vrf-red ping 192.0.2.2 -c 1 &> /dev/null ip vrf exec vrf-red ping6 2001:db8:1::2 -c 1 &> /dev/null Fixes: 09eed1192cec ("neighbour: switch to standard rcu, instead of rcu_bh") Reported-by: Naresh Kamboju <naresh.kamboju@linaro.org> Link: https://lore.kernel.org/netdev/CA+G9fYtEr-=GbcXNDYo3XOkwR+uYgehVoDjsP0pFLUpZ_AZcyg@mail.gmail.com/ Signed-off-by: Ido Schimmel <idosch@nvidia.com> Reviewed-by: David Ahern <dsahern@kernel.org> Reviewed-by: Eric Dumazet <edumazet@google.com> Link: https://lore.kernel.org/r/20230715153605.4068066-1-idosch@nvidia.com Signed-off-by: Paolo Abeni <pabeni@redhat.com>
2023-07-15 18:36:05 +03:00
2015-09-15 20:04:16 -05:00
2015-09-15 20:04:16 -05:00
net: vrf: performance improvements for IPv4 The VRF driver allows users to implement device based features for an entire domain. For example, a qdisc or netfilter rules can be attached to a VRF device or tcpdump can be used to view packets for all devices in the L3 domain. The device-based features come with a performance penalty, most notably in the Tx path. The VRF driver uses the l3mdev_l3_out hook to switch the dst on an skb to its private dst. This allows the skb to traverse the xmit stack with the device set to the VRF device which in turn enables the netfilter and qdisc features. The VRF driver then performs the FIB lookup again and reinserts the packet. This patch avoids the redirect for IPv4 packets if a qdisc has not been attached to a VRF device which is the default config. In this case the netfilter hooks and network taps are directly traversed in the l3mdev_l3_out handler. If a qdisc is attached to a VRF device, then the redirect using the vrf dst is done. Additional overhead is removed by only checking packet taps if a socket is open on the device (vrf_dev->ptype_all list is not empty). Packet sockets bound to any device will still get a copy of the packet via the real ingress or egress interface. The end result of this change is a decrease in the overhead of VRF for the default, baseline case (ie., no netfilter rules, no packet sockets, no qdisc) to ~3% for UDP which has a lookup per packet and < 1% overhead for connected sockets that leverage early demux and avoid FIB lookups. Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-03-20 11:19:44 -07:00
vrf: Remove unnecessary RCU protection around dst entries During initialization of a VRF device, the VRF driver creates two dst entries (for IPv4 and IPv6). They are attached to locally generated packets that are transmitted out of the VRF ports (via the l3mdev_l3_out() hook). Their purpose is to redirect packets towards the VRF device instead of having the packets egress directly out of the VRF ports. This is useful, for example, when a queuing discipline is configured on the VRF device. In order to avoid a NULL pointer dereference, commit b0e95ccdd775 ("net: vrf: protect changes to private data with rcu") made the pointers to the dst entries RCU protected. As far as I can tell, this was needed because back then the dst entries were released (and the pointers reset to NULL) before removing the VRF ports. Later on, commit f630c38ef0d7 ("vrf: fix bug_on triggered by rx when destroying a vrf") moved the removal of the VRF ports to the VRF device's dellink() callback. As such, the tear down sequence of a VRF device looks as follows: 1. VRF ports are removed. 2. VRF device is unregistered. a. Device is closed. b. An RCU grace period passes. c. ndo_uninit() is called. i. dst entries are released. Given the above, the Tx path will always see the same fully initialized dst entries and will never race with the ndo_uninit() callback. Therefore, there is no need to make the pointers to the dst entries RCU protected. Remove it as well as the unnecessary NULL checks in the Tx path. Signed-off-by: Ido Schimmel <idosch@nvidia.com> Reviewed-by: David Ahern <dsahern@kernel.org> Link: https://patch.msgid.link/20260326203233.1128554-4-idosch@nvidia.com Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2026-03-26 22:32:33 +02:00
vrf: Remove unnecessary RCU protection around dst entries During initialization of a VRF device, the VRF driver creates two dst entries (for IPv4 and IPv6). They are attached to locally generated packets that are transmitted out of the VRF ports (via the l3mdev_l3_out() hook). Their purpose is to redirect packets towards the VRF device instead of having the packets egress directly out of the VRF ports. This is useful, for example, when a queuing discipline is configured on the VRF device. In order to avoid a NULL pointer dereference, commit b0e95ccdd775 ("net: vrf: protect changes to private data with rcu") made the pointers to the dst entries RCU protected. As far as I can tell, this was needed because back then the dst entries were released (and the pointers reset to NULL) before removing the VRF ports. Later on, commit f630c38ef0d7 ("vrf: fix bug_on triggered by rx when destroying a vrf") moved the removal of the VRF ports to the VRF device's dellink() callback. As such, the tear down sequence of a VRF device looks as follows: 1. VRF ports are removed. 2. VRF device is unregistered. a. Device is closed. b. An RCU grace period passes. c. ndo_uninit() is called. i. dst entries are released. Given the above, the Tx path will always see the same fully initialized dst entries and will never race with the ndo_uninit() callback. Therefore, there is no need to make the pointers to the dst entries RCU protected. Remove it as well as the unnecessary NULL checks in the Tx path. Signed-off-by: Ido Schimmel <idosch@nvidia.com> Reviewed-by: David Ahern <dsahern@kernel.org> Link: https://patch.msgid.link/20260326203233.1128554-4-idosch@nvidia.com Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2026-03-26 22:32:33 +02:00
net: vrf: performance improvements for IPv4 The VRF driver allows users to implement device based features for an entire domain. For example, a qdisc or netfilter rules can be attached to a VRF device or tcpdump can be used to view packets for all devices in the L3 domain. The device-based features come with a performance penalty, most notably in the Tx path. The VRF driver uses the l3mdev_l3_out hook to switch the dst on an skb to its private dst. This allows the skb to traverse the xmit stack with the device set to the VRF device which in turn enables the netfilter and qdisc features. The VRF driver then performs the FIB lookup again and reinserts the packet. This patch avoids the redirect for IPv4 packets if a qdisc has not been attached to a VRF device which is the default config. In this case the netfilter hooks and network taps are directly traversed in the l3mdev_l3_out handler. If a qdisc is attached to a VRF device, then the redirect using the vrf dst is done. Additional overhead is removed by only checking packet taps if a socket is open on the device (vrf_dev->ptype_all list is not empty). Packet sockets bound to any device will still get a copy of the packet via the real ingress or egress interface. The end result of this change is a decrease in the overhead of VRF for the default, baseline case (ie., no netfilter rules, no packet sockets, no qdisc) to ~3% for UDP which has a lookup per packet and < 1% overhead for connected sockets that leverage early demux and avoid FIB lookups. Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-03-20 11:19:44 -07:00
net: vrf: performance improvements for IPv4 The VRF driver allows users to implement device based features for an entire domain. For example, a qdisc or netfilter rules can be attached to a VRF device or tcpdump can be used to view packets for all devices in the L3 domain. The device-based features come with a performance penalty, most notably in the Tx path. The VRF driver uses the l3mdev_l3_out hook to switch the dst on an skb to its private dst. This allows the skb to traverse the xmit stack with the device set to the VRF device which in turn enables the netfilter and qdisc features. The VRF driver then performs the FIB lookup again and reinserts the packet. This patch avoids the redirect for IPv4 packets if a qdisc has not been attached to a VRF device which is the default config. In this case the netfilter hooks and network taps are directly traversed in the l3mdev_l3_out handler. If a qdisc is attached to a VRF device, then the redirect using the vrf dst is done. Additional overhead is removed by only checking packet taps if a socket is open on the device (vrf_dev->ptype_all list is not empty). Packet sockets bound to any device will still get a copy of the packet via the real ingress or egress interface. The end result of this change is a decrease in the overhead of VRF for the default, baseline case (ie., no netfilter rules, no packet sockets, no qdisc) to ~3% for UDP which has a lookup per packet and < 1% overhead for connected sockets that leverage early demux and avoid FIB lookups. Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-03-20 11:19:44 -07:00
net: vrf: performance improvements for IPv4 The VRF driver allows users to implement device based features for an entire domain. For example, a qdisc or netfilter rules can be attached to a VRF device or tcpdump can be used to view packets for all devices in the L3 domain. The device-based features come with a performance penalty, most notably in the Tx path. The VRF driver uses the l3mdev_l3_out hook to switch the dst on an skb to its private dst. This allows the skb to traverse the xmit stack with the device set to the VRF device which in turn enables the netfilter and qdisc features. The VRF driver then performs the FIB lookup again and reinserts the packet. This patch avoids the redirect for IPv4 packets if a qdisc has not been attached to a VRF device which is the default config. In this case the netfilter hooks and network taps are directly traversed in the l3mdev_l3_out handler. If a qdisc is attached to a VRF device, then the redirect using the vrf dst is done. Additional overhead is removed by only checking packet taps if a socket is open on the device (vrf_dev->ptype_all list is not empty). Packet sockets bound to any device will still get a copy of the packet via the real ingress or egress interface. The end result of this change is a decrease in the overhead of VRF for the default, baseline case (ie., no netfilter rules, no packet sockets, no qdisc) to ~3% for UDP which has a lookup per packet and < 1% overhead for connected sockets that leverage early demux and avoid FIB lookups. Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-03-20 11:19:44 -07:00
net: vrf: performance improvements for IPv4 The VRF driver allows users to implement device based features for an entire domain. For example, a qdisc or netfilter rules can be attached to a VRF device or tcpdump can be used to view packets for all devices in the L3 domain. The device-based features come with a performance penalty, most notably in the Tx path. The VRF driver uses the l3mdev_l3_out hook to switch the dst on an skb to its private dst. This allows the skb to traverse the xmit stack with the device set to the VRF device which in turn enables the netfilter and qdisc features. The VRF driver then performs the FIB lookup again and reinserts the packet. This patch avoids the redirect for IPv4 packets if a qdisc has not been attached to a VRF device which is the default config. In this case the netfilter hooks and network taps are directly traversed in the l3mdev_l3_out handler. If a qdisc is attached to a VRF device, then the redirect using the vrf dst is done. Additional overhead is removed by only checking packet taps if a socket is open on the device (vrf_dev->ptype_all list is not empty). Packet sockets bound to any device will still get a copy of the packet via the real ingress or egress interface. The end result of this change is a decrease in the overhead of VRF for the default, baseline case (ie., no netfilter rules, no packet sockets, no qdisc) to ~3% for UDP which has a lookup per packet and < 1% overhead for connected sockets that leverage early demux and avoid FIB lookups. Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-03-20 11:19:44 -07:00
net: vrf: performance improvements for IPv4 The VRF driver allows users to implement device based features for an entire domain. For example, a qdisc or netfilter rules can be attached to a VRF device or tcpdump can be used to view packets for all devices in the L3 domain. The device-based features come with a performance penalty, most notably in the Tx path. The VRF driver uses the l3mdev_l3_out hook to switch the dst on an skb to its private dst. This allows the skb to traverse the xmit stack with the device set to the VRF device which in turn enables the netfilter and qdisc features. The VRF driver then performs the FIB lookup again and reinserts the packet. This patch avoids the redirect for IPv4 packets if a qdisc has not been attached to a VRF device which is the default config. In this case the netfilter hooks and network taps are directly traversed in the l3mdev_l3_out handler. If a qdisc is attached to a VRF device, then the redirect using the vrf dst is done. Additional overhead is removed by only checking packet taps if a socket is open on the device (vrf_dev->ptype_all list is not empty). Packet sockets bound to any device will still get a copy of the packet via the real ingress or egress interface. The end result of this change is a decrease in the overhead of VRF for the default, baseline case (ie., no netfilter rules, no packet sockets, no qdisc) to ~3% for UDP which has a lookup per packet and < 1% overhead for connected sockets that leverage early demux and avoid FIB lookups. Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-03-20 11:19:44 -07:00
net: vrf: performance improvements for IPv4 The VRF driver allows users to implement device based features for an entire domain. For example, a qdisc or netfilter rules can be attached to a VRF device or tcpdump can be used to view packets for all devices in the L3 domain. The device-based features come with a performance penalty, most notably in the Tx path. The VRF driver uses the l3mdev_l3_out hook to switch the dst on an skb to its private dst. This allows the skb to traverse the xmit stack with the device set to the VRF device which in turn enables the netfilter and qdisc features. The VRF driver then performs the FIB lookup again and reinserts the packet. This patch avoids the redirect for IPv4 packets if a qdisc has not been attached to a VRF device which is the default config. In this case the netfilter hooks and network taps are directly traversed in the l3mdev_l3_out handler. If a qdisc is attached to a VRF device, then the redirect using the vrf dst is done. Additional overhead is removed by only checking packet taps if a socket is open on the device (vrf_dev->ptype_all list is not empty). Packet sockets bound to any device will still get a copy of the packet via the real ingress or egress interface. The end result of this change is a decrease in the overhead of VRF for the default, baseline case (ie., no netfilter rules, no packet sockets, no qdisc) to ~3% for UDP which has a lookup per packet and < 1% overhead for connected sockets that leverage early demux and avoid FIB lookups. Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-03-20 11:19:44 -07:00
net: vrf: performance improvements for IPv4 The VRF driver allows users to implement device based features for an entire domain. For example, a qdisc or netfilter rules can be attached to a VRF device or tcpdump can be used to view packets for all devices in the L3 domain. The device-based features come with a performance penalty, most notably in the Tx path. The VRF driver uses the l3mdev_l3_out hook to switch the dst on an skb to its private dst. This allows the skb to traverse the xmit stack with the device set to the VRF device which in turn enables the netfilter and qdisc features. The VRF driver then performs the FIB lookup again and reinserts the packet. This patch avoids the redirect for IPv4 packets if a qdisc has not been attached to a VRF device which is the default config. In this case the netfilter hooks and network taps are directly traversed in the l3mdev_l3_out handler. If a qdisc is attached to a VRF device, then the redirect using the vrf dst is done. Additional overhead is removed by only checking packet taps if a socket is open on the device (vrf_dev->ptype_all list is not empty). Packet sockets bound to any device will still get a copy of the packet via the real ingress or egress interface. The end result of this change is a decrease in the overhead of VRF for the default, baseline case (ie., no netfilter rules, no packet sockets, no qdisc) to ~3% for UDP which has a lookup per packet and < 1% overhead for connected sockets that leverage early demux and avoid FIB lookups. Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-03-20 11:19:44 -07:00
net: vrf: performance improvements for IPv4 The VRF driver allows users to implement device based features for an entire domain. For example, a qdisc or netfilter rules can be attached to a VRF device or tcpdump can be used to view packets for all devices in the L3 domain. The device-based features come with a performance penalty, most notably in the Tx path. The VRF driver uses the l3mdev_l3_out hook to switch the dst on an skb to its private dst. This allows the skb to traverse the xmit stack with the device set to the VRF device which in turn enables the netfilter and qdisc features. The VRF driver then performs the FIB lookup again and reinserts the packet. This patch avoids the redirect for IPv4 packets if a qdisc has not been attached to a VRF device which is the default config. In this case the netfilter hooks and network taps are directly traversed in the l3mdev_l3_out handler. If a qdisc is attached to a VRF device, then the redirect using the vrf dst is done. Additional overhead is removed by only checking packet taps if a socket is open on the device (vrf_dev->ptype_all list is not empty). Packet sockets bound to any device will still get a copy of the packet via the real ingress or egress interface. The end result of this change is a decrease in the overhead of VRF for the default, baseline case (ie., no netfilter rules, no packet sockets, no qdisc) to ~3% for UDP which has a lookup per packet and < 1% overhead for connected sockets that leverage early demux and avoid FIB lookups. Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-03-20 11:19:44 -07:00
vrf: Remove unnecessary RCU protection around dst entries During initialization of a VRF device, the VRF driver creates two dst entries (for IPv4 and IPv6). They are attached to locally generated packets that are transmitted out of the VRF ports (via the l3mdev_l3_out() hook). Their purpose is to redirect packets towards the VRF device instead of having the packets egress directly out of the VRF ports. This is useful, for example, when a queuing discipline is configured on the VRF device. In order to avoid a NULL pointer dereference, commit b0e95ccdd775 ("net: vrf: protect changes to private data with rcu") made the pointers to the dst entries RCU protected. As far as I can tell, this was needed because back then the dst entries were released (and the pointers reset to NULL) before removing the VRF ports. Later on, commit f630c38ef0d7 ("vrf: fix bug_on triggered by rx when destroying a vrf") moved the removal of the VRF ports to the VRF device's dellink() callback. As such, the tear down sequence of a VRF device looks as follows: 1. VRF ports are removed. 2. VRF device is unregistered. a. Device is closed. b. An RCU grace period passes. c. ndo_uninit() is called. i. dst entries are released. Given the above, the Tx path will always see the same fully initialized dst entries and will never race with the ndo_uninit() callback. Therefore, there is no need to make the pointers to the dst entries RCU protected. Remove it as well as the unnecessary NULL checks in the Tx path. Signed-off-by: Ido Schimmel <idosch@nvidia.com> Reviewed-by: David Ahern <dsahern@kernel.org> Link: https://patch.msgid.link/20260326203233.1128554-4-idosch@nvidia.com Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2026-03-26 22:32:33 +02:00
net: vrf: ipv4 support for local traffic to local addresses Add support for locally originated traffic to VRF-local addresses. If destination device for an skb is the loopback or VRF device then set its dst to a local version of the VRF cached dst_entry and call netif_rx to insert the packet onto the rx queue - similar to what is done for loopback. This patch handles IPv4 support; follow on patch handles IPv6. With this patch, ping, tcp and udp packets to a local IPv4 address are successfully routed: $ ip addr show dev eth1 4: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master red state UP group default qlen 1000 link/ether 02:e0:f9:1c:b9:74 brd ff:ff:ff:ff:ff:ff inet 10.100.1.1/24 brd 10.100.1.255 scope global eth1 valid_lft forever preferred_lft forever inet6 2100:1::1/120 scope global valid_lft forever preferred_lft forever inet6 fe80::e0:f9ff:fe1c:b974/64 scope link valid_lft forever preferred_lft forever $ ping -c1 -I red 10.100.1.1 ping: Warning: source address might be selected on device other than red. PING 10.100.1.1 (10.100.1.1) from 10.100.1.1 red: 56(84) bytes of data. 64 bytes from 10.100.1.1: icmp_seq=1 ttl=64 time=0.057 ms This patch also enables use of IPv4 loopback address on the VRF device: $ ip addr add dev red 127.0.0.1/8 $ ping -c1 -I red 127.0.0.1 PING 127.0.0.1 (127.0.0.1) from 127.0.0.1 red: 56(84) bytes of data. 64 bytes from 127.0.0.1: icmp_seq=1 ttl=64 time=0.058 ms Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2016-06-06 20:50:39 -07:00
vrf: Remove unnecessary RCU protection around dst entries During initialization of a VRF device, the VRF driver creates two dst entries (for IPv4 and IPv6). They are attached to locally generated packets that are transmitted out of the VRF ports (via the l3mdev_l3_out() hook). Their purpose is to redirect packets towards the VRF device instead of having the packets egress directly out of the VRF ports. This is useful, for example, when a queuing discipline is configured on the VRF device. In order to avoid a NULL pointer dereference, commit b0e95ccdd775 ("net: vrf: protect changes to private data with rcu") made the pointers to the dst entries RCU protected. As far as I can tell, this was needed because back then the dst entries were released (and the pointers reset to NULL) before removing the VRF ports. Later on, commit f630c38ef0d7 ("vrf: fix bug_on triggered by rx when destroying a vrf") moved the removal of the VRF ports to the VRF device's dellink() callback. As such, the tear down sequence of a VRF device looks as follows: 1. VRF ports are removed. 2. VRF device is unregistered. a. Device is closed. b. An RCU grace period passes. c. ndo_uninit() is called. i. dst entries are released. Given the above, the Tx path will always see the same fully initialized dst entries and will never race with the ndo_uninit() callback. Therefore, there is no need to make the pointers to the dst entries RCU protected. Remove it as well as the unnecessary NULL checks in the Tx path. Signed-off-by: Ido Schimmel <idosch@nvidia.com> Reviewed-by: David Ahern <dsahern@kernel.org> Link: https://patch.msgid.link/20260326203233.1128554-4-idosch@nvidia.com Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2026-03-26 22:32:33 +02:00
vrf: Fix a potential NPD when removing a port from a VRF RCU readers that identified a net device as a VRF port using netif_is_l3_slave() assume that a subsequent call to netdev_master_upper_dev_get_rcu() will return a VRF device. They then continue to dereference its l3mdev operations. This assumption is not always correct and can result in a NPD [1]. There is no RCU synchronization when removing a port from a VRF, so it is possible for an RCU reader to see a new master device (e.g., a bridge) that does not have l3mdev operations. Fix by adding RCU synchronization after clearing the IFF_L3MDEV_SLAVE flag. Skip this synchronization when a net device is removed from a VRF as part of its deletion and when the VRF device itself is deleted. In the latter case an RCU grace period will pass by the time RTNL is released. [1] BUG: kernel NULL pointer dereference, address: 0000000000000000 [...] RIP: 0010:l3mdev_fib_table_rcu (net/l3mdev/l3mdev.c:181) [...] Call Trace: <TASK> l3mdev_fib_table_by_index (net/l3mdev/l3mdev.c:201 net/l3mdev/l3mdev.c:189) __inet_bind (net/ipv4/af_inet.c:499 (discriminator 3)) inet_bind_sk (net/ipv4/af_inet.c:469) __sys_bind (./include/linux/file.h:62 (discriminator 1) ./include/linux/file.h:83 (discriminator 1) net/socket.c:1951 (discriminator 1)) __x64_sys_bind (net/socket.c:1969 (discriminator 1) net/socket.c:1967 (discriminator 1) net/socket.c:1967 (discriminator 1)) do_syscall_64 (arch/x86/entry/syscall_64.c:63 (discriminator 1) arch/x86/entry/syscall_64.c:94 (discriminator 1)) entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:130) Fixes: fdeea7be88b1 ("net: vrf: Set slave's private flag before linking") Reported-by: Haoze Xie <royenheart@gmail.com> Reported-by: Yifan Wu <yifanwucs@gmail.com> Reported-by: Juefei Pu <tomapufckgml@gmail.com> Reported-by: Yuan Tan <yuantan098@gmail.com> Closes: https://lore.kernel.org/netdev/20260419145332.3988923-1-n05ec@lzu.edu.cn/ Signed-off-by: Ido Schimmel <idosch@nvidia.com> Reviewed-by: David Ahern <dsahern@kernel.org> Link: https://patch.msgid.link/20260423063607.1208202-1-idosch@nvidia.com Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2026-04-23 09:36:07 +03:00
vrf: Fix a potential NPD when removing a port from a VRF RCU readers that identified a net device as a VRF port using netif_is_l3_slave() assume that a subsequent call to netdev_master_upper_dev_get_rcu() will return a VRF device. They then continue to dereference its l3mdev operations. This assumption is not always correct and can result in a NPD [1]. There is no RCU synchronization when removing a port from a VRF, so it is possible for an RCU reader to see a new master device (e.g., a bridge) that does not have l3mdev operations. Fix by adding RCU synchronization after clearing the IFF_L3MDEV_SLAVE flag. Skip this synchronization when a net device is removed from a VRF as part of its deletion and when the VRF device itself is deleted. In the latter case an RCU grace period will pass by the time RTNL is released. [1] BUG: kernel NULL pointer dereference, address: 0000000000000000 [...] RIP: 0010:l3mdev_fib_table_rcu (net/l3mdev/l3mdev.c:181) [...] Call Trace: <TASK> l3mdev_fib_table_by_index (net/l3mdev/l3mdev.c:201 net/l3mdev/l3mdev.c:189) __inet_bind (net/ipv4/af_inet.c:499 (discriminator 3)) inet_bind_sk (net/ipv4/af_inet.c:469) __sys_bind (./include/linux/file.h:62 (discriminator 1) ./include/linux/file.h:83 (discriminator 1) net/socket.c:1951 (discriminator 1)) __x64_sys_bind (net/socket.c:1969 (discriminator 1) net/socket.c:1967 (discriminator 1) net/socket.c:1967 (discriminator 1)) do_syscall_64 (arch/x86/entry/syscall_64.c:63 (discriminator 1) arch/x86/entry/syscall_64.c:94 (discriminator 1)) entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:130) Fixes: fdeea7be88b1 ("net: vrf: Set slave's private flag before linking") Reported-by: Haoze Xie <royenheart@gmail.com> Reported-by: Yifan Wu <yifanwucs@gmail.com> Reported-by: Juefei Pu <tomapufckgml@gmail.com> Reported-by: Yuan Tan <yuantan098@gmail.com> Closes: https://lore.kernel.org/netdev/20260419145332.3988923-1-n05ec@lzu.edu.cn/ Signed-off-by: Ido Schimmel <idosch@nvidia.com> Reviewed-by: David Ahern <dsahern@kernel.org> Link: https://patch.msgid.link/20260423063607.1208202-1-idosch@nvidia.com Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2026-04-23 09:36:07 +03:00
vrf: Fix a potential NPD when removing a port from a VRF RCU readers that identified a net device as a VRF port using netif_is_l3_slave() assume that a subsequent call to netdev_master_upper_dev_get_rcu() will return a VRF device. They then continue to dereference its l3mdev operations. This assumption is not always correct and can result in a NPD [1]. There is no RCU synchronization when removing a port from a VRF, so it is possible for an RCU reader to see a new master device (e.g., a bridge) that does not have l3mdev operations. Fix by adding RCU synchronization after clearing the IFF_L3MDEV_SLAVE flag. Skip this synchronization when a net device is removed from a VRF as part of its deletion and when the VRF device itself is deleted. In the latter case an RCU grace period will pass by the time RTNL is released. [1] BUG: kernel NULL pointer dereference, address: 0000000000000000 [...] RIP: 0010:l3mdev_fib_table_rcu (net/l3mdev/l3mdev.c:181) [...] Call Trace: <TASK> l3mdev_fib_table_by_index (net/l3mdev/l3mdev.c:201 net/l3mdev/l3mdev.c:189) __inet_bind (net/ipv4/af_inet.c:499 (discriminator 3)) inet_bind_sk (net/ipv4/af_inet.c:469) __sys_bind (./include/linux/file.h:62 (discriminator 1) ./include/linux/file.h:83 (discriminator 1) net/socket.c:1951 (discriminator 1)) __x64_sys_bind (net/socket.c:1969 (discriminator 1) net/socket.c:1967 (discriminator 1) net/socket.c:1967 (discriminator 1)) do_syscall_64 (arch/x86/entry/syscall_64.c:63 (discriminator 1) arch/x86/entry/syscall_64.c:94 (discriminator 1)) entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:130) Fixes: fdeea7be88b1 ("net: vrf: Set slave's private flag before linking") Reported-by: Haoze Xie <royenheart@gmail.com> Reported-by: Yifan Wu <yifanwucs@gmail.com> Reported-by: Juefei Pu <tomapufckgml@gmail.com> Reported-by: Yuan Tan <yuantan098@gmail.com> Closes: https://lore.kernel.org/netdev/20260419145332.3988923-1-n05ec@lzu.edu.cn/ Signed-off-by: Ido Schimmel <idosch@nvidia.com> Reviewed-by: David Ahern <dsahern@kernel.org> Link: https://patch.msgid.link/20260423063607.1208202-1-idosch@nvidia.com Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2026-04-23 09:36:07 +03:00
vrf: Fix a potential NPD when removing a port from a VRF RCU readers that identified a net device as a VRF port using netif_is_l3_slave() assume that a subsequent call to netdev_master_upper_dev_get_rcu() will return a VRF device. They then continue to dereference its l3mdev operations. This assumption is not always correct and can result in a NPD [1]. There is no RCU synchronization when removing a port from a VRF, so it is possible for an RCU reader to see a new master device (e.g., a bridge) that does not have l3mdev operations. Fix by adding RCU synchronization after clearing the IFF_L3MDEV_SLAVE flag. Skip this synchronization when a net device is removed from a VRF as part of its deletion and when the VRF device itself is deleted. In the latter case an RCU grace period will pass by the time RTNL is released. [1] BUG: kernel NULL pointer dereference, address: 0000000000000000 [...] RIP: 0010:l3mdev_fib_table_rcu (net/l3mdev/l3mdev.c:181) [...] Call Trace: <TASK> l3mdev_fib_table_by_index (net/l3mdev/l3mdev.c:201 net/l3mdev/l3mdev.c:189) __inet_bind (net/ipv4/af_inet.c:499 (discriminator 3)) inet_bind_sk (net/ipv4/af_inet.c:469) __sys_bind (./include/linux/file.h:62 (discriminator 1) ./include/linux/file.h:83 (discriminator 1) net/socket.c:1951 (discriminator 1)) __x64_sys_bind (net/socket.c:1969 (discriminator 1) net/socket.c:1967 (discriminator 1) net/socket.c:1967 (discriminator 1)) do_syscall_64 (arch/x86/entry/syscall_64.c:63 (discriminator 1) arch/x86/entry/syscall_64.c:94 (discriminator 1)) entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:130) Fixes: fdeea7be88b1 ("net: vrf: Set slave's private flag before linking") Reported-by: Haoze Xie <royenheart@gmail.com> Reported-by: Yifan Wu <yifanwucs@gmail.com> Reported-by: Juefei Pu <tomapufckgml@gmail.com> Reported-by: Yuan Tan <yuantan098@gmail.com> Closes: https://lore.kernel.org/netdev/20260419145332.3988923-1-n05ec@lzu.edu.cn/ Signed-off-by: Ido Schimmel <idosch@nvidia.com> Reviewed-by: David Ahern <dsahern@kernel.org> Link: https://patch.msgid.link/20260423063607.1208202-1-idosch@nvidia.com Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2026-04-23 09:36:07 +03:00
vrf: do not push non-ND strict packets with a source LLA through packet taps again Non-ND strict packets with a source LLA go through the packet taps again, while non-ND strict packets with other source addresses do not, and we can see a clone of those packets on the vrf interface (we should not). This is due to a series of changes: Commit 6f12fa775530[1] made non-ND strict packets not being pushed again in the packet taps. This changed with commit 205704c618af[2] for those packets having a source LLA, as they need a lookup with the orig_iif. The issue now is those packets do not skip the 'vrf_ip6_rcv' function to the end (as the ones without a source LLA) and go through the check to call packet taps again. This check was changed by commit 6f12fa775530[1] and do not exclude non-strict packets anymore. Packets matching 'need_strict && !is_ndisc && is_ll_src' are now being sent through the packet taps again. This can be seen by dumping packets on the vrf interface. Fix this by having the same code path for all non-ND strict packets and selectively lookup with the orig_iif for those with a source LLA. This has the effect to revert to the pre-205704c618af[2] condition, which should also be easier to maintain. [1] 6f12fa775530 ("vrf: mark skb for multicast or link-local as enslaved to VRF") [2] 205704c618af ("vrf: packets with lladdr src needs dst at input with orig_iif when needs strict") Fixes: 205704c618af ("vrf: packets with lladdr src needs dst at input with orig_iif when needs strict") Cc: Stephen Suryaputra <ssuryaextr@gmail.com> Reported-by: Paolo Abeni <pabeni@redhat.com> Signed-off-by: Antoine Tenart <atenart@kernel.org> Reviewed-by: David Ahern <dsahern@kernel.org> Signed-off-by: David S. Miller <davem@davemloft.net>
2021-06-18 17:15:53 +02:00
net: Require exact match for TCP socket lookups if dif is l3mdev Currently, socket lookups for l3mdev (vrf) use cases can match a socket that is bound to a port but not a device (ie., a global socket). If the sysctl tcp_l3mdev_accept is not set this leads to ack packets going out based on the main table even though the packet came in from an L3 domain. The end result is that the connection does not establish creating confusion for users since the service is running and a socket shows in ss output. Fix by requiring an exact dif to sk_bound_dev_if match if the skb came through an interface enslaved to an l3mdev device and the tcp_l3mdev_accept is not set. skb's through an l3mdev interface are marked by setting a flag in inet{6}_skb_parm. The IPv6 variant is already set; this patch adds the flag for IPv4. Using an skb flag avoids a device lookup on the dif. The flag is set in the VRF driver using the IP{6}CB macros. For IPv4, the inet_skb_parm struct is moved in the cb per commit 971f10eca186, so the match function in the TCP stack needs to use TCP_SKB_CB. For IPv6, the move is done after the socket lookup, so IP6CB is used. The flags field in inet_skb_parm struct needs to be increased to add another flag. There is currently a 1-byte hole following the flags, so it can be expanded to u16 without increasing the size of the struct. Fixes: 193125dbd8eb ("net: Introduce VRF device driver") Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2016-10-16 20:02:52 -07:00
vrf: do not push non-ND strict packets with a source LLA through packet taps again Non-ND strict packets with a source LLA go through the packet taps again, while non-ND strict packets with other source addresses do not, and we can see a clone of those packets on the vrf interface (we should not). This is due to a series of changes: Commit 6f12fa775530[1] made non-ND strict packets not being pushed again in the packet taps. This changed with commit 205704c618af[2] for those packets having a source LLA, as they need a lookup with the orig_iif. The issue now is those packets do not skip the 'vrf_ip6_rcv' function to the end (as the ones without a source LLA) and go through the check to call packet taps again. This check was changed by commit 6f12fa775530[1] and do not exclude non-strict packets anymore. Packets matching 'need_strict && !is_ndisc && is_ll_src' are now being sent through the packet taps again. This can be seen by dumping packets on the vrf interface. Fix this by having the same code path for all non-ND strict packets and selectively lookup with the orig_iif for those with a source LLA. This has the effect to revert to the pre-205704c618af[2] condition, which should also be easier to maintain. [1] 6f12fa775530 ("vrf: mark skb for multicast or link-local as enslaved to VRF") [2] 205704c618af ("vrf: packets with lladdr src needs dst at input with orig_iif when needs strict") Fixes: 205704c618af ("vrf: packets with lladdr src needs dst at input with orig_iif when needs strict") Cc: Stephen Suryaputra <ssuryaextr@gmail.com> Reported-by: Paolo Abeni <pabeni@redhat.com> Signed-off-by: Antoine Tenart <atenart@kernel.org> Reviewed-by: David Ahern <dsahern@kernel.org> Signed-off-by: David S. Miller <davem@davemloft.net>
2021-06-18 17:15:53 +02:00
vrf: do not push non-ND strict packets with a source LLA through packet taps again Non-ND strict packets with a source LLA go through the packet taps again, while non-ND strict packets with other source addresses do not, and we can see a clone of those packets on the vrf interface (we should not). This is due to a series of changes: Commit 6f12fa775530[1] made non-ND strict packets not being pushed again in the packet taps. This changed with commit 205704c618af[2] for those packets having a source LLA, as they need a lookup with the orig_iif. The issue now is those packets do not skip the 'vrf_ip6_rcv' function to the end (as the ones without a source LLA) and go through the check to call packet taps again. This check was changed by commit 6f12fa775530[1] and do not exclude non-strict packets anymore. Packets matching 'need_strict && !is_ndisc && is_ll_src' are now being sent through the packet taps again. This can be seen by dumping packets on the vrf interface. Fix this by having the same code path for all non-ND strict packets and selectively lookup with the orig_iif for those with a source LLA. This has the effect to revert to the pre-205704c618af[2] condition, which should also be easier to maintain. [1] 6f12fa775530 ("vrf: mark skb for multicast or link-local as enslaved to VRF") [2] 205704c618af ("vrf: packets with lladdr src needs dst at input with orig_iif when needs strict") Fixes: 205704c618af ("vrf: packets with lladdr src needs dst at input with orig_iif when needs strict") Cc: Stephen Suryaputra <ssuryaextr@gmail.com> Reported-by: Paolo Abeni <pabeni@redhat.com> Signed-off-by: Antoine Tenart <atenart@kernel.org> Reviewed-by: David Ahern <dsahern@kernel.org> Signed-off-by: David S. Miller <davem@davemloft.net>
2021-06-18 17:15:53 +02:00
net: vrf: performance improvements for IPv6 The VRF driver allows users to implement device based features for an entire domain. For example, a qdisc or netfilter rules can be attached to a VRF device or tcpdump can be used to view packets for all devices in the L3 domain. The device-based features come with a performance penalty, most notably in the Tx path. The VRF driver uses the l3mdev_l3_out hook to switch the dst on an skb to its private dst. This allows the skb to traverse the xmit stack with the device set to the VRF device which in turn enables the netfilter and qdisc features. The VRF driver then performs the FIB lookup again and reinserts the packet. This patch avoids the redirect for IPv6 packets if a qdisc has not been attached to a VRF device which is the default config. In this case the netfilter hooks and network taps are directly traversed in the l3mdev_l3_out handler. If a qdisc is attached to a VRF device, then the redirect using the vrf dst is done. Additional overhead is removed by only checking packet taps if a socket is open on the device (vrf_dev->ptype_all list is not empty). Packet sockets bound to any device will still get a copy of the packet via the real ingress or egress interface. The end result of this change is a decrease in the overhead of VRF for the default, baseline case (ie., no netfilter rules, no packet sockets, no qdisc) from a +3% improvement for UDP which has a lookup per packet (VRF being better than no l3mdev) to ~2% loss for TCP_CRR which connects a socket for each request-response. Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-03-20 11:19:45 -07:00
net: vrf: performance improvements for IPv6 The VRF driver allows users to implement device based features for an entire domain. For example, a qdisc or netfilter rules can be attached to a VRF device or tcpdump can be used to view packets for all devices in the L3 domain. The device-based features come with a performance penalty, most notably in the Tx path. The VRF driver uses the l3mdev_l3_out hook to switch the dst on an skb to its private dst. This allows the skb to traverse the xmit stack with the device set to the VRF device which in turn enables the netfilter and qdisc features. The VRF driver then performs the FIB lookup again and reinserts the packet. This patch avoids the redirect for IPv6 packets if a qdisc has not been attached to a VRF device which is the default config. In this case the netfilter hooks and network taps are directly traversed in the l3mdev_l3_out handler. If a qdisc is attached to a VRF device, then the redirect using the vrf dst is done. Additional overhead is removed by only checking packet taps if a socket is open on the device (vrf_dev->ptype_all list is not empty). Packet sockets bound to any device will still get a copy of the packet via the real ingress or egress interface. The end result of this change is a decrease in the overhead of VRF for the default, baseline case (ie., no netfilter rules, no packet sockets, no qdisc) from a +3% improvement for UDP which has a lookup per packet (VRF being better than no l3mdev) to ~2% loss for TCP_CRR which connects a socket for each request-response. Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-03-20 11:19:45 -07:00
net: Require exact match for TCP socket lookups if dif is l3mdev Currently, socket lookups for l3mdev (vrf) use cases can match a socket that is bound to a port but not a device (ie., a global socket). If the sysctl tcp_l3mdev_accept is not set this leads to ack packets going out based on the main table even though the packet came in from an L3 domain. The end result is that the connection does not establish creating confusion for users since the service is running and a socket shows in ss output. Fix by requiring an exact dif to sk_bound_dev_if match if the skb came through an interface enslaved to an l3mdev device and the tcp_l3mdev_accept is not set. skb's through an l3mdev interface are marked by setting a flag in inet{6}_skb_parm. The IPv6 variant is already set; this patch adds the flag for IPv4. Using an skb flag avoids a device lookup on the dif. The flag is set in the VRF driver using the IP{6}CB macros. For IPv4, the inet_skb_parm struct is moved in the cb per commit 971f10eca186, so the match function in the TCP stack needs to use TCP_SKB_CB. For IPv6, the move is done after the socket lookup, so IP6CB is used. The flags field in inet_skb_parm struct needs to be increased to add another flag. There is currently a 1-byte hole following the flags, so it can be expanded to u16 without increasing the size of the struct. Fixes: 193125dbd8eb ("net: Introduce VRF device driver") Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2016-10-16 20:02:52 -07:00
net: vrf: ipv4 support for local traffic to local addresses Add support for locally originated traffic to VRF-local addresses. If destination device for an skb is the loopback or VRF device then set its dst to a local version of the VRF cached dst_entry and call netif_rx to insert the packet onto the rx queue - similar to what is done for loopback. This patch handles IPv4 support; follow on patch handles IPv6. With this patch, ping, tcp and udp packets to a local IPv4 address are successfully routed: $ ip addr show dev eth1 4: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master red state UP group default qlen 1000 link/ether 02:e0:f9:1c:b9:74 brd ff:ff:ff:ff:ff:ff inet 10.100.1.1/24 brd 10.100.1.255 scope global eth1 valid_lft forever preferred_lft forever inet6 2100:1::1/120 scope global valid_lft forever preferred_lft forever inet6 fe80::e0:f9ff:fe1c:b974/64 scope link valid_lft forever preferred_lft forever $ ping -c1 -I red 10.100.1.1 ping: Warning: source address might be selected on device other than red. PING 10.100.1.1 (10.100.1.1) from 10.100.1.1 red: 56(84) bytes of data. 64 bytes from 10.100.1.1: icmp_seq=1 ttl=64 time=0.057 ms This patch also enables use of IPv4 loopback address on the VRF device: $ ip addr add dev red 127.0.0.1/8 $ ping -c1 -I red 127.0.0.1 PING 127.0.0.1 (127.0.0.1) from 127.0.0.1 red: 56(84) bytes of data. 64 bytes from 127.0.0.1: icmp_seq=1 ttl=64 time=0.058 ms Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2016-06-06 20:50:39 -07:00
net: vrf: performance improvements for IPv4 The VRF driver allows users to implement device based features for an entire domain. For example, a qdisc or netfilter rules can be attached to a VRF device or tcpdump can be used to view packets for all devices in the L3 domain. The device-based features come with a performance penalty, most notably in the Tx path. The VRF driver uses the l3mdev_l3_out hook to switch the dst on an skb to its private dst. This allows the skb to traverse the xmit stack with the device set to the VRF device which in turn enables the netfilter and qdisc features. The VRF driver then performs the FIB lookup again and reinserts the packet. This patch avoids the redirect for IPv4 packets if a qdisc has not been attached to a VRF device which is the default config. In this case the netfilter hooks and network taps are directly traversed in the l3mdev_l3_out handler. If a qdisc is attached to a VRF device, then the redirect using the vrf dst is done. Additional overhead is removed by only checking packet taps if a socket is open on the device (vrf_dev->ptype_all list is not empty). Packet sockets bound to any device will still get a copy of the packet via the real ingress or egress interface. The end result of this change is a decrease in the overhead of VRF for the default, baseline case (ie., no netfilter rules, no packet sockets, no qdisc) to ~3% for UDP which has a lookup per packet and < 1% overhead for connected sockets that leverage early demux and avoid FIB lookups. Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-03-20 11:19:44 -07:00
net: vrf: performance improvements for IPv4 The VRF driver allows users to implement device based features for an entire domain. For example, a qdisc or netfilter rules can be attached to a VRF device or tcpdump can be used to view packets for all devices in the L3 domain. The device-based features come with a performance penalty, most notably in the Tx path. The VRF driver uses the l3mdev_l3_out hook to switch the dst on an skb to its private dst. This allows the skb to traverse the xmit stack with the device set to the VRF device which in turn enables the netfilter and qdisc features. The VRF driver then performs the FIB lookup again and reinserts the packet. This patch avoids the redirect for IPv4 packets if a qdisc has not been attached to a VRF device which is the default config. In this case the netfilter hooks and network taps are directly traversed in the l3mdev_l3_out handler. If a qdisc is attached to a VRF device, then the redirect using the vrf dst is done. Additional overhead is removed by only checking packet taps if a socket is open on the device (vrf_dev->ptype_all list is not empty). Packet sockets bound to any device will still get a copy of the packet via the real ingress or egress interface. The end result of this change is a decrease in the overhead of VRF for the default, baseline case (ie., no netfilter rules, no packet sockets, no qdisc) to ~3% for UDP which has a lookup per packet and < 1% overhead for connected sockets that leverage early demux and avoid FIB lookups. Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-03-20 11:19:44 -07:00
net: vrf: ipv4 support for local traffic to local addresses Add support for locally originated traffic to VRF-local addresses. If destination device for an skb is the loopback or VRF device then set its dst to a local version of the VRF cached dst_entry and call netif_rx to insert the packet onto the rx queue - similar to what is done for loopback. This patch handles IPv4 support; follow on patch handles IPv6. With this patch, ping, tcp and udp packets to a local IPv4 address are successfully routed: $ ip addr show dev eth1 4: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master red state UP group default qlen 1000 link/ether 02:e0:f9:1c:b9:74 brd ff:ff:ff:ff:ff:ff inet 10.100.1.1/24 brd 10.100.1.255 scope global eth1 valid_lft forever preferred_lft forever inet6 2100:1::1/120 scope global valid_lft forever preferred_lft forever inet6 fe80::e0:f9ff:fe1c:b974/64 scope link valid_lft forever preferred_lft forever $ ping -c1 -I red 10.100.1.1 ping: Warning: source address might be selected on device other than red. PING 10.100.1.1 (10.100.1.1) from 10.100.1.1 red: 56(84) bytes of data. 64 bytes from 10.100.1.1: icmp_seq=1 ttl=64 time=0.057 ms This patch also enables use of IPv4 loopback address on the VRF device: $ ip addr add dev red 127.0.0.1/8 $ ping -c1 -I red 127.0.0.1 PING 127.0.0.1 (127.0.0.1) from 127.0.0.1 red: 56(84) bytes of data. 64 bytes from 127.0.0.1: icmp_seq=1 ttl=64 time=0.058 ms Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2016-06-06 20:50:39 -07:00
net: vrf: Fix crash when IPv6 is disabled at boot time Frank Kellermann reported a kernel crash with 4.5.0 when IPv6 is disabled at boot using the kernel option ipv6.disable=1. Using current net-next with the boot option: $ ip link add red type vrf table 1001 Generates: [12210.919584] BUG: unable to handle kernel NULL pointer dereference at 0000000000000748 [12210.921341] IP: [<ffffffff814b30e3>] fib6_get_table+0x2c/0x5a [12210.922537] PGD b79e3067 PUD bb32b067 PMD 0 [12210.923479] Oops: 0000 [#1] SMP [12210.924001] Modules linked in: ipvlan 8021q garp mrp stp llc [12210.925130] CPU: 3 PID: 1177 Comm: ip Not tainted 4.7.0-rc1+ #235 [12210.926168] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.7.5-20140531_083030-gandalf 04/01/2014 [12210.928065] task: ffff8800b9ac4640 ti: ffff8800bacac000 task.ti: ffff8800bacac000 [12210.929328] RIP: 0010:[<ffffffff814b30e3>] [<ffffffff814b30e3>] fib6_get_table+0x2c/0x5a [12210.930697] RSP: 0018:ffff8800bacaf888 EFLAGS: 00010202 [12210.931563] RAX: 0000000000000748 RBX: ffffffff81a9e280 RCX: ffff8800b9ac4e28 [12210.932688] RDX: 00000000000000e9 RSI: 0000000000000002 RDI: 0000000000000286 [12210.933820] RBP: ffff8800bacaf898 R08: ffff8800b9ac4df0 R09: 000000000052001b [12210.934941] R10: 00000000657c0000 R11: 000000000000c649 R12: 00000000000003e9 [12210.936032] R13: 00000000000003e9 R14: ffff8800bace7800 R15: ffff8800bb3ec000 [12210.937103] FS: 00007faa1766c700(0000) GS:ffff88013ac00000(0000) knlGS:0000000000000000 [12210.938321] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [12210.939166] CR2: 0000000000000748 CR3: 00000000b79d6000 CR4: 00000000000406e0 [12210.940278] Stack: [12210.940603] ffff8800bb3ec000 ffffffff81a9e280 ffff8800bacaf8c8 ffffffff814b3135 [12210.941818] ffff8800bb3ec000 ffffffff81a9e280 ffffffff81a9e280 ffff8800bace7800 [12210.943040] ffff8800bacaf8f0 ffffffff81397c88 ffff8800bb3ec000 ffffffff81a9e280 [12210.944288] Call Trace: [12210.944688] [<ffffffff814b3135>] fib6_new_table+0x24/0x8a [12210.945516] [<ffffffff81397c88>] vrf_dev_init+0xd4/0x162 [12210.946328] [<ffffffff814091e1>] register_netdevice+0x100/0x396 [12210.947209] [<ffffffff8139823d>] vrf_newlink+0x40/0xb3 [12210.948001] [<ffffffff814187f0>] rtnl_newlink+0x5d3/0x6d5 ... The problem above is due to the fact that the fib hash table is not allocated when IPv6 is disabled at boot. As for the VRF driver it should not do any IPv6 initializations if IPv6 is disabled, so it needs to know if IPv6 is disabled at boot. The disable parameter is private to the IPv6 module, so provide an accessor for modules to determine if IPv6 was disabled at boot time. Fixes: 35402e3136634 ("net: Add IPv6 support to VRF device") Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2016-06-09 10:21:00 -07:00
net: Fix inconsistent teardown and release of private netdev state. Network devices can allocate reasources and private memory using netdev_ops->ndo_init(). However, the release of these resources can occur in one of two different places. Either netdev_ops->ndo_uninit() or netdev->destructor(). The decision of which operation frees the resources depends upon whether it is necessary for all netdev refs to be released before it is safe to perform the freeing. netdev_ops->ndo_uninit() presumably can occur right after the NETDEV_UNREGISTER notifier completes and the unicast and multicast address lists are flushed. netdev->destructor(), on the other hand, does not run until the netdev references all go away. Further complicating the situation is that netdev->destructor() almost universally does also a free_netdev(). This creates a problem for the logic in register_netdevice(). Because all callers of register_netdevice() manage the freeing of the netdev, and invoke free_netdev(dev) if register_netdevice() fails. If netdev_ops->ndo_init() succeeds, but something else fails inside of register_netdevice(), it does call ndo_ops->ndo_uninit(). But it is not able to invoke netdev->destructor(). This is because netdev->destructor() will do a free_netdev() and then the caller of register_netdevice() will do the same. However, this means that the resources that would normally be released by netdev->destructor() will not be. Over the years drivers have added local hacks to deal with this, by invoking their destructor parts by hand when register_netdevice() fails. Many drivers do not try to deal with this, and instead we have leaks. Let's close this hole by formalizing the distinction between what private things need to be freed up by netdev->destructor() and whether the driver needs unregister_netdevice() to perform the free_netdev(). netdev->priv_destructor() performs all actions to free up the private resources that used to be freed by netdev->destructor(), except for free_netdev(). netdev->needs_free_netdev is a boolean that indicates whether free_netdev() should be done at the end of unregister_netdevice(). Now, register_netdevice() can sanely release all resources after ndo_ops->ndo_init() succeeds, by invoking both ndo_ops->ndo_uninit() and netdev->priv_destructor(). And at the end of unregister_netdevice(), we invoke netdev->priv_destructor() and optionally call free_netdev(). Signed-off-by: David S. Miller <davem@davemloft.net>
2017-05-08 12:52:56 -04:00
vrf: fix bug_on triggered by rx when destroying a vrf When destroying a VRF device we cleanup the slaves in its ndo_uninit() function, but that causes packets to be switched (skb->dev == vrf being destroyed) even though we're pass the point where the VRF should be receiving any packets while it is being dismantled. This causes a BUG_ON to trigger if we have raw sockets (trace below). The reason is that the inetdev of the VRF has been destroyed but we're still sending packets up the stack with it, so let's free the slaves in the dellink callback as David Ahern suggested. Note that this fix doesn't prevent packets from going up when the VRF device is admin down. [ 35.631371] ------------[ cut here ]------------ [ 35.631603] kernel BUG at net/ipv4/fib_frontend.c:285! [ 35.631854] invalid opcode: 0000 [#1] SMP [ 35.631977] Modules linked in: [ 35.632081] CPU: 2 PID: 22 Comm: ksoftirqd/2 Not tainted 4.12.0-rc7+ #45 [ 35.632247] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.7.5-20140531_083030-gandalf 04/01/2014 [ 35.632477] task: ffff88005ad68000 task.stack: ffff88005ad64000 [ 35.632632] RIP: 0010:fib_compute_spec_dst+0xfc/0x1ee [ 35.632769] RSP: 0018:ffff88005ad67978 EFLAGS: 00010202 [ 35.632910] RAX: 0000000000000001 RBX: ffff880059a7f200 RCX: 0000000000000000 [ 35.633084] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffffffff82274af0 [ 35.633256] RBP: ffff88005ad679f8 R08: 000000000001ef70 R09: 0000000000000046 [ 35.633430] R10: ffff88005ad679f8 R11: ffff880037731cb0 R12: 0000000000000001 [ 35.633603] R13: ffff8800599e3000 R14: 0000000000000000 R15: ffff8800599cb852 [ 35.634114] FS: 0000000000000000(0000) GS:ffff88005d900000(0000) knlGS:0000000000000000 [ 35.634306] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 35.634456] CR2: 00007f3563227095 CR3: 000000000201d000 CR4: 00000000000406e0 [ 35.634632] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 35.634865] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [ 35.635055] Call Trace: [ 35.635271] ? __lock_acquire+0xf0d/0x1117 [ 35.635522] ipv4_pktinfo_prepare+0x82/0x151 [ 35.635831] raw_rcv_skb+0x17/0x3c [ 35.636062] raw_rcv+0xe5/0xf7 [ 35.636287] raw_local_deliver+0x169/0x1d9 [ 35.636534] ip_local_deliver_finish+0x87/0x1c4 [ 35.636820] ip_local_deliver+0x63/0x7f [ 35.637058] ip_rcv_finish+0x340/0x3a1 [ 35.637295] ip_rcv+0x314/0x34a [ 35.637525] __netif_receive_skb_core+0x49f/0x7c5 [ 35.637780] ? lock_acquire+0x13f/0x1d7 [ 35.638018] ? lock_acquire+0x15e/0x1d7 [ 35.638259] __netif_receive_skb+0x1e/0x94 [ 35.638502] ? __netif_receive_skb+0x1e/0x94 [ 35.638748] netif_receive_skb_internal+0x74/0x300 [ 35.639002] ? dev_gro_receive+0x2ed/0x411 [ 35.639246] ? lock_is_held_type+0xc4/0xd2 [ 35.639491] napi_gro_receive+0x105/0x1a0 [ 35.639736] receive_buf+0xc32/0xc74 [ 35.639965] ? detach_buf+0x67/0x153 [ 35.640201] ? virtqueue_get_buf_ctx+0x120/0x176 [ 35.640453] virtnet_poll+0x128/0x1c5 [ 35.640690] net_rx_action+0x103/0x343 [ 35.640932] __do_softirq+0x1c7/0x4b7 [ 35.641171] run_ksoftirqd+0x23/0x5c [ 35.641403] smpboot_thread_fn+0x24f/0x26d [ 35.641646] ? sort_range+0x22/0x22 [ 35.641878] kthread+0x129/0x131 [ 35.642104] ? __list_add+0x31/0x31 [ 35.642335] ? __list_add+0x31/0x31 [ 35.642568] ret_from_fork+0x2a/0x40 [ 35.642804] Code: 05 bd 87 a3 00 01 e8 1f ef 98 ff 4d 85 f6 48 c7 c7 f0 4a 27 82 41 0f 94 c4 31 c9 31 d2 41 0f b6 f4 e8 04 71 a1 ff 45 84 e4 74 02 <0f> 0b 0f b7 93 c4 00 00 00 4d 8b a5 80 05 00 00 48 03 93 d0 00 [ 35.644342] RIP: fib_compute_spec_dst+0xfc/0x1ee RSP: ffff88005ad67978 Fixes: 193125dbd8eb ("net: Introduce VRF device driver") Reported-by: Chris Cormier <chriscormier@cumulusnetworks.com> Signed-off-by: Nikolay Aleksandrov <nikolay@cumulusnetworks.com> Acked-by: David Ahern <dsahern@gmail.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-07-06 15:24:40 +03:00
vrf: Fix a potential NPD when removing a port from a VRF RCU readers that identified a net device as a VRF port using netif_is_l3_slave() assume that a subsequent call to netdev_master_upper_dev_get_rcu() will return a VRF device. They then continue to dereference its l3mdev operations. This assumption is not always correct and can result in a NPD [1]. There is no RCU synchronization when removing a port from a VRF, so it is possible for an RCU reader to see a new master device (e.g., a bridge) that does not have l3mdev operations. Fix by adding RCU synchronization after clearing the IFF_L3MDEV_SLAVE flag. Skip this synchronization when a net device is removed from a VRF as part of its deletion and when the VRF device itself is deleted. In the latter case an RCU grace period will pass by the time RTNL is released. [1] BUG: kernel NULL pointer dereference, address: 0000000000000000 [...] RIP: 0010:l3mdev_fib_table_rcu (net/l3mdev/l3mdev.c:181) [...] Call Trace: <TASK> l3mdev_fib_table_by_index (net/l3mdev/l3mdev.c:201 net/l3mdev/l3mdev.c:189) __inet_bind (net/ipv4/af_inet.c:499 (discriminator 3)) inet_bind_sk (net/ipv4/af_inet.c:469) __sys_bind (./include/linux/file.h:62 (discriminator 1) ./include/linux/file.h:83 (discriminator 1) net/socket.c:1951 (discriminator 1)) __x64_sys_bind (net/socket.c:1969 (discriminator 1) net/socket.c:1967 (discriminator 1) net/socket.c:1967 (discriminator 1)) do_syscall_64 (arch/x86/entry/syscall_64.c:63 (discriminator 1) arch/x86/entry/syscall_64.c:94 (discriminator 1)) entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:130) Fixes: fdeea7be88b1 ("net: vrf: Set slave's private flag before linking") Reported-by: Haoze Xie <royenheart@gmail.com> Reported-by: Yifan Wu <yifanwucs@gmail.com> Reported-by: Juefei Pu <tomapufckgml@gmail.com> Reported-by: Yuan Tan <yuantan098@gmail.com> Closes: https://lore.kernel.org/netdev/20260419145332.3988923-1-n05ec@lzu.edu.cn/ Signed-off-by: Ido Schimmel <idosch@nvidia.com> Reviewed-by: David Ahern <dsahern@kernel.org> Link: https://patch.msgid.link/20260423063607.1208202-1-idosch@nvidia.com Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2026-04-23 09:36:07 +03:00
vrf: fix bug_on triggered by rx when destroying a vrf When destroying a VRF device we cleanup the slaves in its ndo_uninit() function, but that causes packets to be switched (skb->dev == vrf being destroyed) even though we're pass the point where the VRF should be receiving any packets while it is being dismantled. This causes a BUG_ON to trigger if we have raw sockets (trace below). The reason is that the inetdev of the VRF has been destroyed but we're still sending packets up the stack with it, so let's free the slaves in the dellink callback as David Ahern suggested. Note that this fix doesn't prevent packets from going up when the VRF device is admin down. [ 35.631371] ------------[ cut here ]------------ [ 35.631603] kernel BUG at net/ipv4/fib_frontend.c:285! [ 35.631854] invalid opcode: 0000 [#1] SMP [ 35.631977] Modules linked in: [ 35.632081] CPU: 2 PID: 22 Comm: ksoftirqd/2 Not tainted 4.12.0-rc7+ #45 [ 35.632247] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.7.5-20140531_083030-gandalf 04/01/2014 [ 35.632477] task: ffff88005ad68000 task.stack: ffff88005ad64000 [ 35.632632] RIP: 0010:fib_compute_spec_dst+0xfc/0x1ee [ 35.632769] RSP: 0018:ffff88005ad67978 EFLAGS: 00010202 [ 35.632910] RAX: 0000000000000001 RBX: ffff880059a7f200 RCX: 0000000000000000 [ 35.633084] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffffffff82274af0 [ 35.633256] RBP: ffff88005ad679f8 R08: 000000000001ef70 R09: 0000000000000046 [ 35.633430] R10: ffff88005ad679f8 R11: ffff880037731cb0 R12: 0000000000000001 [ 35.633603] R13: ffff8800599e3000 R14: 0000000000000000 R15: ffff8800599cb852 [ 35.634114] FS: 0000000000000000(0000) GS:ffff88005d900000(0000) knlGS:0000000000000000 [ 35.634306] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 35.634456] CR2: 00007f3563227095 CR3: 000000000201d000 CR4: 00000000000406e0 [ 35.634632] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 35.634865] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [ 35.635055] Call Trace: [ 35.635271] ? __lock_acquire+0xf0d/0x1117 [ 35.635522] ipv4_pktinfo_prepare+0x82/0x151 [ 35.635831] raw_rcv_skb+0x17/0x3c [ 35.636062] raw_rcv+0xe5/0xf7 [ 35.636287] raw_local_deliver+0x169/0x1d9 [ 35.636534] ip_local_deliver_finish+0x87/0x1c4 [ 35.636820] ip_local_deliver+0x63/0x7f [ 35.637058] ip_rcv_finish+0x340/0x3a1 [ 35.637295] ip_rcv+0x314/0x34a [ 35.637525] __netif_receive_skb_core+0x49f/0x7c5 [ 35.637780] ? lock_acquire+0x13f/0x1d7 [ 35.638018] ? lock_acquire+0x15e/0x1d7 [ 35.638259] __netif_receive_skb+0x1e/0x94 [ 35.638502] ? __netif_receive_skb+0x1e/0x94 [ 35.638748] netif_receive_skb_internal+0x74/0x300 [ 35.639002] ? dev_gro_receive+0x2ed/0x411 [ 35.639246] ? lock_is_held_type+0xc4/0xd2 [ 35.639491] napi_gro_receive+0x105/0x1a0 [ 35.639736] receive_buf+0xc32/0xc74 [ 35.639965] ? detach_buf+0x67/0x153 [ 35.640201] ? virtqueue_get_buf_ctx+0x120/0x176 [ 35.640453] virtnet_poll+0x128/0x1c5 [ 35.640690] net_rx_action+0x103/0x343 [ 35.640932] __do_softirq+0x1c7/0x4b7 [ 35.641171] run_ksoftirqd+0x23/0x5c [ 35.641403] smpboot_thread_fn+0x24f/0x26d [ 35.641646] ? sort_range+0x22/0x22 [ 35.641878] kthread+0x129/0x131 [ 35.642104] ? __list_add+0x31/0x31 [ 35.642335] ? __list_add+0x31/0x31 [ 35.642568] ret_from_fork+0x2a/0x40 [ 35.642804] Code: 05 bd 87 a3 00 01 e8 1f ef 98 ff 4d 85 f6 48 c7 c7 f0 4a 27 82 41 0f 94 c4 31 c9 31 d2 41 0f b6 f4 e8 04 71 a1 ff 45 84 e4 74 02 <0f> 0b 0f b7 93 c4 00 00 00 4d 8b a5 80 05 00 00 48 03 93 d0 00 [ 35.644342] RIP: fib_compute_spec_dst+0xfc/0x1ee RSP: ffff88005ad67978 Fixes: 193125dbd8eb ("net: Introduce VRF device driver") Reported-by: Chris Cormier <chriscormier@cumulusnetworks.com> Signed-off-by: Nikolay Aleksandrov <nikolay@cumulusnetworks.com> Acked-by: David Ahern <dsahern@gmail.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-07-06 15:24:40 +03:00
rtnetlink: Pack newlink() params into struct There are 4 net namespaces involved when creating links: - source netns - where the netlink socket resides, - target netns - where to put the device being created, - link netns - netns associated with the device (backend), - peer netns - netns of peer device. Currently, two nets are passed to newlink() callback - "src_net" parameter and "dev_net" (implicitly in net_device). They are set as follows, depending on netlink attributes in the request. +------------+-------------------+---------+---------+ | peer netns | IFLA_LINK_NETNSID | src_net | dev_net | +------------+-------------------+---------+---------+ | | absent | source | target | | absent +-------------------+---------+---------+ | | present | link | link | +------------+-------------------+---------+---------+ | | absent | peer | target | | present +-------------------+---------+---------+ | | present | peer | link | +------------+-------------------+---------+---------+ When IFLA_LINK_NETNSID is present, the device is created in link netns first and then moved to target netns. This has some side effects, including extra ifindex allocation, ifname validation and link events. These could be avoided if we create it in target netns from the beginning. On the other hand, the meaning of src_net parameter is ambiguous. It varies depending on how parameters are passed. It is the effective link (or peer netns) by design, but some drivers ignore it and use dev_net instead. To provide more netns context for drivers, this patch packs existing newlink() parameters, along with the source netns, link netns and peer netns, into a struct. The old "src_net" is renamed to "net" to avoid confusion with real source netns, and will be deprecated later. The use of src_net are converted to params->net trivially. Signed-off-by: Xiao Liang <shaw.leon@gmail.com> Reviewed-by: Kuniyuki Iwashima <kuniyu@amazon.com> Link: https://patch.msgid.link/20250219125039.18024-3-shaw.leon@gmail.com Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2025-02-19 20:50:28 +08:00
rtnetlink: Pack newlink() params into struct There are 4 net namespaces involved when creating links: - source netns - where the netlink socket resides, - target netns - where to put the device being created, - link netns - netns associated with the device (backend), - peer netns - netns of peer device. Currently, two nets are passed to newlink() callback - "src_net" parameter and "dev_net" (implicitly in net_device). They are set as follows, depending on netlink attributes in the request. +------------+-------------------+---------+---------+ | peer netns | IFLA_LINK_NETNSID | src_net | dev_net | +------------+-------------------+---------+---------+ | | absent | source | target | | absent +-------------------+---------+---------+ | | present | link | link | +------------+-------------------+---------+---------+ | | absent | peer | target | | present +-------------------+---------+---------+ | | present | peer | link | +------------+-------------------+---------+---------+ When IFLA_LINK_NETNSID is present, the device is created in link netns first and then moved to target netns. This has some side effects, including extra ifindex allocation, ifname validation and link events. These could be avoided if we create it in target netns from the beginning. On the other hand, the meaning of src_net parameter is ambiguous. It varies depending on how parameters are passed. It is the effective link (or peer netns) by design, but some drivers ignore it and use dev_net instead. To provide more netns context for drivers, this patch packs existing newlink() parameters, along with the source netns, link netns and peer netns, into a struct. The old "src_net" is renamed to "net" to avoid confusion with real source netns, and will be deprecated later. The use of src_net are converted to params->net trivially. Signed-off-by: Xiao Liang <shaw.leon@gmail.com> Reviewed-by: Kuniyuki Iwashima <kuniyu@amazon.com> Link: https://patch.msgid.link/20250219125039.18024-3-shaw.leon@gmail.com Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2025-02-19 20:50:28 +08:00
vrf: Fix a potential NPD when removing a port from a VRF RCU readers that identified a net device as a VRF port using netif_is_l3_slave() assume that a subsequent call to netdev_master_upper_dev_get_rcu() will return a VRF device. They then continue to dereference its l3mdev operations. This assumption is not always correct and can result in a NPD [1]. There is no RCU synchronization when removing a port from a VRF, so it is possible for an RCU reader to see a new master device (e.g., a bridge) that does not have l3mdev operations. Fix by adding RCU synchronization after clearing the IFF_L3MDEV_SLAVE flag. Skip this synchronization when a net device is removed from a VRF as part of its deletion and when the VRF device itself is deleted. In the latter case an RCU grace period will pass by the time RTNL is released. [1] BUG: kernel NULL pointer dereference, address: 0000000000000000 [...] RIP: 0010:l3mdev_fib_table_rcu (net/l3mdev/l3mdev.c:181) [...] Call Trace: <TASK> l3mdev_fib_table_by_index (net/l3mdev/l3mdev.c:201 net/l3mdev/l3mdev.c:189) __inet_bind (net/ipv4/af_inet.c:499 (discriminator 3)) inet_bind_sk (net/ipv4/af_inet.c:469) __sys_bind (./include/linux/file.h:62 (discriminator 1) ./include/linux/file.h:83 (discriminator 1) net/socket.c:1951 (discriminator 1)) __x64_sys_bind (net/socket.c:1969 (discriminator 1) net/socket.c:1967 (discriminator 1) net/socket.c:1967 (discriminator 1)) do_syscall_64 (arch/x86/entry/syscall_64.c:63 (discriminator 1) arch/x86/entry/syscall_64.c:94 (discriminator 1)) entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:130) Fixes: fdeea7be88b1 ("net: vrf: Set slave's private flag before linking") Reported-by: Haoze Xie <royenheart@gmail.com> Reported-by: Yifan Wu <yifanwucs@gmail.com> Reported-by: Juefei Pu <tomapufckgml@gmail.com> Reported-by: Yuan Tan <yuantan098@gmail.com> Closes: https://lore.kernel.org/netdev/20260419145332.3988923-1-n05ec@lzu.edu.cn/ Signed-off-by: Ido Schimmel <idosch@nvidia.com> Reviewed-by: David Ahern <dsahern@kernel.org> Link: https://patch.msgid.link/20260423063607.1208202-1-idosch@nvidia.com Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2026-04-23 09:36:07 +03:00
sysctl: treewide: constify the ctl_table argument of proc_handlers const qualify the struct ctl_table argument in the proc_handler function signatures. This is a prerequisite to moving the static ctl_table structs into .rodata data which will ensure that proc_handler function pointers cannot be modified. This patch has been generated by the following coccinelle script: ``` virtual patch @r1@ identifier ctl, write, buffer, lenp, ppos; identifier func !~ "appldata_(timer|interval)_handler|sched_(rt|rr)_handler|rds_tcp_skbuf_handler|proc_sctp_do_(hmac_alg|rto_min|rto_max|udp_port|alpha_beta|auth|probe_interval)"; @@ int func( - struct ctl_table *ctl + const struct ctl_table *ctl ,int write, void *buffer, size_t *lenp, loff_t *ppos); @r2@ identifier func, ctl, write, buffer, lenp, ppos; @@ int func( - struct ctl_table *ctl + const struct ctl_table *ctl ,int write, void *buffer, size_t *lenp, loff_t *ppos) { ... } @r3@ identifier func; @@ int func( - struct ctl_table * + const struct ctl_table * ,int , void *, size_t *, loff_t *); @r4@ identifier func, ctl; @@ int func( - struct ctl_table *ctl + const struct ctl_table *ctl ,int , void *, size_t *, loff_t *); @r5@ identifier func, write, buffer, lenp, ppos; @@ int func( - struct ctl_table * + const struct ctl_table * ,int write, void *buffer, size_t *lenp, loff_t *ppos); ``` * Code formatting was adjusted in xfs_sysctl.c to comply with code conventions. The xfs_stats_clear_proc_handler, xfs_panic_mask_proc_handler and xfs_deprecated_dointvec_minmax where adjusted. * The ctl_table argument in proc_watchdog_common was const qualified. This is called from a proc_handler itself and is calling back into another proc_handler, making it necessary to change it as part of the proc_handler migration. Co-developed-by: Thomas Weißschuh <linux@weissschuh.net> Signed-off-by: Thomas Weißschuh <linux@weissschuh.net> Co-developed-by: Joel Granados <j.granados@samsung.com> Signed-off-by: Joel Granados <j.granados@samsung.com>
2024-07-24 20:59:29 +02:00