linux/drivers/net/vrf.c

2072 lines
47 KiB

net: vrf: ipv4 support for local traffic to local addresses Add support for locally originated traffic to VRF-local addresses. If destination device for an skb is the loopback or VRF device then set its dst to a local version of the VRF cached dst_entry and call netif_rx to insert the packet onto the rx queue - similar to what is done for loopback. This patch handles IPv4 support; follow on patch handles IPv6. With this patch, ping, tcp and udp packets to a local IPv4 address are successfully routed: $ ip addr show dev eth1 4: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master red state UP group default qlen 1000 link/ether 02:e0:f9:1c:b9:74 brd ff:ff:ff:ff:ff:ff inet 10.100.1.1/24 brd 10.100.1.255 scope global eth1 valid_lft forever preferred_lft forever inet6 2100:1::1/120 scope global valid_lft forever preferred_lft forever inet6 fe80::e0:f9ff:fe1c:b974/64 scope link valid_lft forever preferred_lft forever $ ping -c1 -I red 10.100.1.1 ping: Warning: source address might be selected on device other than red. PING 10.100.1.1 (10.100.1.1) from 10.100.1.1 red: 56(84) bytes of data. 64 bytes from 10.100.1.1: icmp_seq=1 ttl=64 time=0.057 ms This patch also enables use of IPv4 loopback address on the VRF device: $ ip addr add dev red 127.0.0.1/8 $ ping -c1 -I red 127.0.0.1 PING 127.0.0.1 (127.0.0.1) from 127.0.0.1 red: 56(84) bytes of data. 64 bytes from 127.0.0.1: icmp_seq=1 ttl=64 time=0.058 ms Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2016-06-06 20:50:39 -07:00
net: vrf: ipv4 support for local traffic to local addresses Add support for locally originated traffic to VRF-local addresses. If destination device for an skb is the loopback or VRF device then set its dst to a local version of the VRF cached dst_entry and call netif_rx to insert the packet onto the rx queue - similar to what is done for loopback. This patch handles IPv4 support; follow on patch handles IPv6. With this patch, ping, tcp and udp packets to a local IPv4 address are successfully routed: $ ip addr show dev eth1 4: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master red state UP group default qlen 1000 link/ether 02:e0:f9:1c:b9:74 brd ff:ff:ff:ff:ff:ff inet 10.100.1.1/24 brd 10.100.1.255 scope global eth1 valid_lft forever preferred_lft forever inet6 2100:1::1/120 scope global valid_lft forever preferred_lft forever inet6 fe80::e0:f9ff:fe1c:b974/64 scope link valid_lft forever preferred_lft forever $ ping -c1 -I red 10.100.1.1 ping: Warning: source address might be selected on device other than red. PING 10.100.1.1 (10.100.1.1) from 10.100.1.1 red: 56(84) bytes of data. 64 bytes from 10.100.1.1: icmp_seq=1 ttl=64 time=0.057 ms This patch also enables use of IPv4 loopback address on the VRF device: $ ip addr add dev red 127.0.0.1/8 $ ping -c1 -I red 127.0.0.1 PING 127.0.0.1 (127.0.0.1) from 127.0.0.1 red: 56(84) bytes of data. 64 bytes from 127.0.0.1: icmp_seq=1 ttl=64 time=0.058 ms Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2016-06-06 20:50:39 -07:00
net: vrf: performance improvements for IPv4 The VRF driver allows users to implement device based features for an entire domain. For example, a qdisc or netfilter rules can be attached to a VRF device or tcpdump can be used to view packets for all devices in the L3 domain. The device-based features come with a performance penalty, most notably in the Tx path. The VRF driver uses the l3mdev_l3_out hook to switch the dst on an skb to its private dst. This allows the skb to traverse the xmit stack with the device set to the VRF device which in turn enables the netfilter and qdisc features. The VRF driver then performs the FIB lookup again and reinserts the packet. This patch avoids the redirect for IPv4 packets if a qdisc has not been attached to a VRF device which is the default config. In this case the netfilter hooks and network taps are directly traversed in the l3mdev_l3_out handler. If a qdisc is attached to a VRF device, then the redirect using the vrf dst is done. Additional overhead is removed by only checking packet taps if a socket is open on the device (vrf_dev->ptype_all list is not empty). Packet sockets bound to any device will still get a copy of the packet via the real ingress or egress interface. The end result of this change is a decrease in the overhead of VRF for the default, baseline case (ie., no netfilter rules, no packet sockets, no qdisc) to ~3% for UDP which has a lookup per packet and < 1% overhead for connected sockets that leverage early demux and avoid FIB lookups. Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-03-20 11:19:44 -07:00
net: vrf: ipv4 support for local traffic to local addresses Add support for locally originated traffic to VRF-local addresses. If destination device for an skb is the loopback or VRF device then set its dst to a local version of the VRF cached dst_entry and call netif_rx to insert the packet onto the rx queue - similar to what is done for loopback. This patch handles IPv4 support; follow on patch handles IPv6. With this patch, ping, tcp and udp packets to a local IPv4 address are successfully routed: $ ip addr show dev eth1 4: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master red state UP group default qlen 1000 link/ether 02:e0:f9:1c:b9:74 brd ff:ff:ff:ff:ff:ff inet 10.100.1.1/24 brd 10.100.1.255 scope global eth1 valid_lft forever preferred_lft forever inet6 2100:1::1/120 scope global valid_lft forever preferred_lft forever inet6 fe80::e0:f9ff:fe1c:b974/64 scope link valid_lft forever preferred_lft forever $ ping -c1 -I red 10.100.1.1 ping: Warning: source address might be selected on device other than red. PING 10.100.1.1 (10.100.1.1) from 10.100.1.1 red: 56(84) bytes of data. 64 bytes from 10.100.1.1: icmp_seq=1 ttl=64 time=0.057 ms This patch also enables use of IPv4 loopback address on the VRF device: $ ip addr add dev red 127.0.0.1/8 $ ping -c1 -I red 127.0.0.1 PING 127.0.0.1 (127.0.0.1) from 127.0.0.1 red: 56(84) bytes of data. 64 bytes from 127.0.0.1: icmp_seq=1 ttl=64 time=0.058 ms Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2016-06-06 20:50:39 -07:00
net: dev: Makes sure netif_rx() can be invoked in any context. Dave suggested a while ago (eleven years by now) "Let's make netif_rx() work in all contexts and get rid of netif_rx_ni()". Eric agreed and pointed out that modern devices should use netif_receive_skb() to avoid the overhead. In the meantime someone added another variant, netif_rx_any_context(), which behaves as suggested. netif_rx() must be invoked with disabled bottom halves to ensure that pending softirqs, which were raised within the function, are handled. netif_rx_ni() can be invoked only from process context (bottom halves must be enabled) because the function handles pending softirqs without checking if bottom halves were disabled or not. netif_rx_any_context() invokes on the former functions by checking in_interrupts(). netif_rx() could be taught to handle both cases (disabled and enabled bottom halves) by simply disabling bottom halves while invoking netif_rx_internal(). The local_bh_enable() invocation will then invoke pending softirqs only if the BH-disable counter drops to zero. Eric is concerned about the overhead of BH-disable+enable especially in regard to the loopback driver. As critical as this driver is, it will receive a shortcut to avoid the additional overhead which is not needed. Add a local_bh_disable() section in netif_rx() to ensure softirqs are handled if needed. Provide __netif_rx() which does not disable BH and has a lockdep assert to ensure that interrupts are disabled. Use this shortcut in the loopback driver and in drivers/net/*.c. Make netif_rx_ni() and netif_rx_any_context() invoke netif_rx() so they can be removed once they are no more users left. Link: https://lkml.kernel.org/r/20100415.020246.218622820.davem@davemloft.net Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> Reviewed-by: Eric Dumazet <edumazet@google.com> Reviewed-by: Toke Høiland-Jørgensen <toke@redhat.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2022-02-12 00:38:38 +01:00
net: vrf: ipv4 support for local traffic to local addresses Add support for locally originated traffic to VRF-local addresses. If destination device for an skb is the loopback or VRF device then set its dst to a local version of the VRF cached dst_entry and call netif_rx to insert the packet onto the rx queue - similar to what is done for loopback. This patch handles IPv4 support; follow on patch handles IPv6. With this patch, ping, tcp and udp packets to a local IPv4 address are successfully routed: $ ip addr show dev eth1 4: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master red state UP group default qlen 1000 link/ether 02:e0:f9:1c:b9:74 brd ff:ff:ff:ff:ff:ff inet 10.100.1.1/24 brd 10.100.1.255 scope global eth1 valid_lft forever preferred_lft forever inet6 2100:1::1/120 scope global valid_lft forever preferred_lft forever inet6 fe80::e0:f9ff:fe1c:b974/64 scope link valid_lft forever preferred_lft forever $ ping -c1 -I red 10.100.1.1 ping: Warning: source address might be selected on device other than red. PING 10.100.1.1 (10.100.1.1) from 10.100.1.1 red: 56(84) bytes of data. 64 bytes from 10.100.1.1: icmp_seq=1 ttl=64 time=0.057 ms This patch also enables use of IPv4 loopback address on the VRF device: $ ip addr add dev red 127.0.0.1/8 $ ping -c1 -I red 127.0.0.1 PING 127.0.0.1 (127.0.0.1) from 127.0.0.1 red: 56(84) bytes of data. 64 bytes from 127.0.0.1: icmp_seq=1 ttl=64 time=0.058 ms Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2016-06-06 20:50:39 -07:00
net: Add l3mdev index to flow struct and avoid oif reset for port devices The fundamental premise of VRF and l3mdev core code is binding a socket to a device (l3mdev or netdev with an L3 domain) to indicate L3 scope. Legacy code resets flowi_oif to the l3mdev losing any original port device binding. Ben (among others) has demonstrated use cases where the original port device binding is important and needs to be retained. This patch handles that by adding a new entry to the common flow struct that can indicate the l3mdev index for later rule and table matching avoiding the need to reset flowi_oif. In addition to allowing more use cases that require port device binds, this patch brings a few datapath simplications: 1. l3mdev_fib_rule_match is only called when walking fib rules and always after l3mdev_update_flow. That allows an optimization to bail early for non-VRF type uses cases when flowi_l3mdev is not set. Also, only that index needs to be checked for the FIB table id. 2. l3mdev_update_flow can be called with flowi_oif set to a l3mdev (e.g., VRF) device. By resetting flowi_oif only for this case the FLOWI_FLAG_SKIP_NH_OIF flag is not longer needed and can be removed, removing several checks in the datapath. The flowi_iif path can be simplified to only be called if the it is not loopback (loopback can not be assigned to an L3 domain) and the l3mdev index is not already set. 3. Avoid another device lookup in the output path when the fib lookup returns a reject failure. Note: 2 functional tests for local traffic with reject fib rules are updated to reflect the new direct failure at FIB lookup time for ping rather than the failure on packet path. The current code fails like this: HINT: Fails since address on vrf device is out of device scope COMMAND: ip netns exec ns-A ping -c1 -w1 -I eth1 172.16.3.1 ping: Warning: source address might be selected on device other than: eth1 PING 172.16.3.1 (172.16.3.1) from 172.16.3.1 eth1: 56(84) bytes of data. --- 172.16.3.1 ping statistics --- 1 packets transmitted, 0 received, 100% packet loss, time 0ms where the test now directly fails: HINT: Fails since address on vrf device is out of device scope COMMAND: ip netns exec ns-A ping -c1 -w1 -I eth1 172.16.3.1 ping: connect: No route to host Signed-off-by: David Ahern <dsahern@kernel.org> Tested-by: Ben Greear <greearb@candelatech.com> Link: https://lore.kernel.org/r/20220314204551.16369-1-dsahern@kernel.org Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2022-03-14 14:45:51 -06:00
net: Add l3mdev index to flow struct and avoid oif reset for port devices The fundamental premise of VRF and l3mdev core code is binding a socket to a device (l3mdev or netdev with an L3 domain) to indicate L3 scope. Legacy code resets flowi_oif to the l3mdev losing any original port device binding. Ben (among others) has demonstrated use cases where the original port device binding is important and needs to be retained. This patch handles that by adding a new entry to the common flow struct that can indicate the l3mdev index for later rule and table matching avoiding the need to reset flowi_oif. In addition to allowing more use cases that require port device binds, this patch brings a few datapath simplications: 1. l3mdev_fib_rule_match is only called when walking fib rules and always after l3mdev_update_flow. That allows an optimization to bail early for non-VRF type uses cases when flowi_l3mdev is not set. Also, only that index needs to be checked for the FIB table id. 2. l3mdev_update_flow can be called with flowi_oif set to a l3mdev (e.g., VRF) device. By resetting flowi_oif only for this case the FLOWI_FLAG_SKIP_NH_OIF flag is not longer needed and can be removed, removing several checks in the datapath. The flowi_iif path can be simplified to only be called if the it is not loopback (loopback can not be assigned to an L3 domain) and the l3mdev index is not already set. 3. Avoid another device lookup in the output path when the fib lookup returns a reject failure. Note: 2 functional tests for local traffic with reject fib rules are updated to reflect the new direct failure at FIB lookup time for ping rather than the failure on packet path. The current code fails like this: HINT: Fails since address on vrf device is out of device scope COMMAND: ip netns exec ns-A ping -c1 -w1 -I eth1 172.16.3.1 ping: Warning: source address might be selected on device other than: eth1 PING 172.16.3.1 (172.16.3.1) from 172.16.3.1 eth1: 56(84) bytes of data. --- 172.16.3.1 ping statistics --- 1 packets transmitted, 0 received, 100% packet loss, time 0ms where the test now directly fails: HINT: Fails since address on vrf device is out of device scope COMMAND: ip netns exec ns-A ping -c1 -w1 -I eth1 172.16.3.1 ping: connect: No route to host Signed-off-by: David Ahern <dsahern@kernel.org> Tested-by: Ben Greear <greearb@candelatech.com> Link: https://lore.kernel.org/r/20220314204551.16369-1-dsahern@kernel.org Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2022-03-14 14:45:51 -06:00
net: Add l3mdev index to flow struct and avoid oif reset for port devices The fundamental premise of VRF and l3mdev core code is binding a socket to a device (l3mdev or netdev with an L3 domain) to indicate L3 scope. Legacy code resets flowi_oif to the l3mdev losing any original port device binding. Ben (among others) has demonstrated use cases where the original port device binding is important and needs to be retained. This patch handles that by adding a new entry to the common flow struct that can indicate the l3mdev index for later rule and table matching avoiding the need to reset flowi_oif. In addition to allowing more use cases that require port device binds, this patch brings a few datapath simplications: 1. l3mdev_fib_rule_match is only called when walking fib rules and always after l3mdev_update_flow. That allows an optimization to bail early for non-VRF type uses cases when flowi_l3mdev is not set. Also, only that index needs to be checked for the FIB table id. 2. l3mdev_update_flow can be called with flowi_oif set to a l3mdev (e.g., VRF) device. By resetting flowi_oif only for this case the FLOWI_FLAG_SKIP_NH_OIF flag is not longer needed and can be removed, removing several checks in the datapath. The flowi_iif path can be simplified to only be called if the it is not loopback (loopback can not be assigned to an L3 domain) and the l3mdev index is not already set. 3. Avoid another device lookup in the output path when the fib lookup returns a reject failure. Note: 2 functional tests for local traffic with reject fib rules are updated to reflect the new direct failure at FIB lookup time for ping rather than the failure on packet path. The current code fails like this: HINT: Fails since address on vrf device is out of device scope COMMAND: ip netns exec ns-A ping -c1 -w1 -I eth1 172.16.3.1 ping: Warning: source address might be selected on device other than: eth1 PING 172.16.3.1 (172.16.3.1) from 172.16.3.1 eth1: 56(84) bytes of data. --- 172.16.3.1 ping statistics --- 1 packets transmitted, 0 received, 100% packet loss, time 0ms where the test now directly fails: HINT: Fails since address on vrf device is out of device scope COMMAND: ip netns exec ns-A ping -c1 -w1 -I eth1 172.16.3.1 ping: connect: No route to host Signed-off-by: David Ahern <dsahern@kernel.org> Tested-by: Ben Greear <greearb@candelatech.com> Link: https://lore.kernel.org/r/20220314204551.16369-1-dsahern@kernel.org Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2022-03-14 14:45:51 -06:00
net: vrf: ipv4 support for local traffic to local addresses Add support for locally originated traffic to VRF-local addresses. If destination device for an skb is the loopback or VRF device then set its dst to a local version of the VRF cached dst_entry and call netif_rx to insert the packet onto the rx queue - similar to what is done for loopback. This patch handles IPv4 support; follow on patch handles IPv6. With this patch, ping, tcp and udp packets to a local IPv4 address are successfully routed: $ ip addr show dev eth1 4: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master red state UP group default qlen 1000 link/ether 02:e0:f9:1c:b9:74 brd ff:ff:ff:ff:ff:ff inet 10.100.1.1/24 brd 10.100.1.255 scope global eth1 valid_lft forever preferred_lft forever inet6 2100:1::1/120 scope global valid_lft forever preferred_lft forever inet6 fe80::e0:f9ff:fe1c:b974/64 scope link valid_lft forever preferred_lft forever $ ping -c1 -I red 10.100.1.1 ping: Warning: source address might be selected on device other than red. PING 10.100.1.1 (10.100.1.1) from 10.100.1.1 red: 56(84) bytes of data. 64 bytes from 10.100.1.1: icmp_seq=1 ttl=64 time=0.057 ms This patch also enables use of IPv4 loopback address on the VRF device: $ ip addr add dev red 127.0.0.1/8 $ ping -c1 -I red 127.0.0.1 PING 127.0.0.1 (127.0.0.1) from 127.0.0.1 red: 56(84) bytes of data. 64 bytes from 127.0.0.1: icmp_seq=1 ttl=64 time=0.058 ms Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2016-06-06 20:50:39 -07:00
net: vrf: ipv4 support for local traffic to local addresses Add support for locally originated traffic to VRF-local addresses. If destination device for an skb is the loopback or VRF device then set its dst to a local version of the VRF cached dst_entry and call netif_rx to insert the packet onto the rx queue - similar to what is done for loopback. This patch handles IPv4 support; follow on patch handles IPv6. With this patch, ping, tcp and udp packets to a local IPv4 address are successfully routed: $ ip addr show dev eth1 4: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master red state UP group default qlen 1000 link/ether 02:e0:f9:1c:b9:74 brd ff:ff:ff:ff:ff:ff inet 10.100.1.1/24 brd 10.100.1.255 scope global eth1 valid_lft forever preferred_lft forever inet6 2100:1::1/120 scope global valid_lft forever preferred_lft forever inet6 fe80::e0:f9ff:fe1c:b974/64 scope link valid_lft forever preferred_lft forever $ ping -c1 -I red 10.100.1.1 ping: Warning: source address might be selected on device other than red. PING 10.100.1.1 (10.100.1.1) from 10.100.1.1 red: 56(84) bytes of data. 64 bytes from 10.100.1.1: icmp_seq=1 ttl=64 time=0.057 ms This patch also enables use of IPv4 loopback address on the VRF device: $ ip addr add dev red 127.0.0.1/8 $ ping -c1 -I red 127.0.0.1 PING 127.0.0.1 (127.0.0.1) from 127.0.0.1 red: 56(84) bytes of data. 64 bytes from 127.0.0.1: icmp_seq=1 ttl=64 time=0.058 ms Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2016-06-06 20:50:39 -07:00
net: vrf: ipv4 support for local traffic to local addresses Add support for locally originated traffic to VRF-local addresses. If destination device for an skb is the loopback or VRF device then set its dst to a local version of the VRF cached dst_entry and call netif_rx to insert the packet onto the rx queue - similar to what is done for loopback. This patch handles IPv4 support; follow on patch handles IPv6. With this patch, ping, tcp and udp packets to a local IPv4 address are successfully routed: $ ip addr show dev eth1 4: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master red state UP group default qlen 1000 link/ether 02:e0:f9:1c:b9:74 brd ff:ff:ff:ff:ff:ff inet 10.100.1.1/24 brd 10.100.1.255 scope global eth1 valid_lft forever preferred_lft forever inet6 2100:1::1/120 scope global valid_lft forever preferred_lft forever inet6 fe80::e0:f9ff:fe1c:b974/64 scope link valid_lft forever preferred_lft forever $ ping -c1 -I red 10.100.1.1 ping: Warning: source address might be selected on device other than red. PING 10.100.1.1 (10.100.1.1) from 10.100.1.1 red: 56(84) bytes of data. 64 bytes from 10.100.1.1: icmp_seq=1 ttl=64 time=0.057 ms This patch also enables use of IPv4 loopback address on the VRF device: $ ip addr add dev red 127.0.0.1/8 $ ping -c1 -I red 127.0.0.1 PING 127.0.0.1 (127.0.0.1) from 127.0.0.1 red: 56(84) bytes of data. 64 bytes from 127.0.0.1: icmp_seq=1 ttl=64 time=0.058 ms Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2016-06-06 20:50:39 -07:00
net: vrf: performance improvements for IPv4 The VRF driver allows users to implement device based features for an entire domain. For example, a qdisc or netfilter rules can be attached to a VRF device or tcpdump can be used to view packets for all devices in the L3 domain. The device-based features come with a performance penalty, most notably in the Tx path. The VRF driver uses the l3mdev_l3_out hook to switch the dst on an skb to its private dst. This allows the skb to traverse the xmit stack with the device set to the VRF device which in turn enables the netfilter and qdisc features. The VRF driver then performs the FIB lookup again and reinserts the packet. This patch avoids the redirect for IPv4 packets if a qdisc has not been attached to a VRF device which is the default config. In this case the netfilter hooks and network taps are directly traversed in the l3mdev_l3_out handler. If a qdisc is attached to a VRF device, then the redirect using the vrf dst is done. Additional overhead is removed by only checking packet taps if a socket is open on the device (vrf_dev->ptype_all list is not empty). Packet sockets bound to any device will still get a copy of the packet via the real ingress or egress interface. The end result of this change is a decrease in the overhead of VRF for the default, baseline case (ie., no netfilter rules, no packet sockets, no qdisc) to ~3% for UDP which has a lookup per packet and < 1% overhead for connected sockets that leverage early demux and avoid FIB lookups. Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-03-20 11:19:44 -07:00
net: vrf: performance improvements for IPv4 The VRF driver allows users to implement device based features for an entire domain. For example, a qdisc or netfilter rules can be attached to a VRF device or tcpdump can be used to view packets for all devices in the L3 domain. The device-based features come with a performance penalty, most notably in the Tx path. The VRF driver uses the l3mdev_l3_out hook to switch the dst on an skb to its private dst. This allows the skb to traverse the xmit stack with the device set to the VRF device which in turn enables the netfilter and qdisc features. The VRF driver then performs the FIB lookup again and reinserts the packet. This patch avoids the redirect for IPv4 packets if a qdisc has not been attached to a VRF device which is the default config. In this case the netfilter hooks and network taps are directly traversed in the l3mdev_l3_out handler. If a qdisc is attached to a VRF device, then the redirect using the vrf dst is done. Additional overhead is removed by only checking packet taps if a socket is open on the device (vrf_dev->ptype_all list is not empty). Packet sockets bound to any device will still get a copy of the packet via the real ingress or egress interface. The end result of this change is a decrease in the overhead of VRF for the default, baseline case (ie., no netfilter rules, no packet sockets, no qdisc) to ~3% for UDP which has a lookup per packet and < 1% overhead for connected sockets that leverage early demux and avoid FIB lookups. Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-03-20 11:19:44 -07:00
net: vrf: performance improvements for IPv4 The VRF driver allows users to implement device based features for an entire domain. For example, a qdisc or netfilter rules can be attached to a VRF device or tcpdump can be used to view packets for all devices in the L3 domain. The device-based features come with a performance penalty, most notably in the Tx path. The VRF driver uses the l3mdev_l3_out hook to switch the dst on an skb to its private dst. This allows the skb to traverse the xmit stack with the device set to the VRF device which in turn enables the netfilter and qdisc features. The VRF driver then performs the FIB lookup again and reinserts the packet. This patch avoids the redirect for IPv4 packets if a qdisc has not been attached to a VRF device which is the default config. In this case the netfilter hooks and network taps are directly traversed in the l3mdev_l3_out handler. If a qdisc is attached to a VRF device, then the redirect using the vrf dst is done. Additional overhead is removed by only checking packet taps if a socket is open on the device (vrf_dev->ptype_all list is not empty). Packet sockets bound to any device will still get a copy of the packet via the real ingress or egress interface. The end result of this change is a decrease in the overhead of VRF for the default, baseline case (ie., no netfilter rules, no packet sockets, no qdisc) to ~3% for UDP which has a lookup per packet and < 1% overhead for connected sockets that leverage early demux and avoid FIB lookups. Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-03-20 11:19:44 -07:00
net: vrf: performance improvements for IPv6 The VRF driver allows users to implement device based features for an entire domain. For example, a qdisc or netfilter rules can be attached to a VRF device or tcpdump can be used to view packets for all devices in the L3 domain. The device-based features come with a performance penalty, most notably in the Tx path. The VRF driver uses the l3mdev_l3_out hook to switch the dst on an skb to its private dst. This allows the skb to traverse the xmit stack with the device set to the VRF device which in turn enables the netfilter and qdisc features. The VRF driver then performs the FIB lookup again and reinserts the packet. This patch avoids the redirect for IPv6 packets if a qdisc has not been attached to a VRF device which is the default config. In this case the netfilter hooks and network taps are directly traversed in the l3mdev_l3_out handler. If a qdisc is attached to a VRF device, then the redirect using the vrf dst is done. Additional overhead is removed by only checking packet taps if a socket is open on the device (vrf_dev->ptype_all list is not empty). Packet sockets bound to any device will still get a copy of the packet via the real ingress or egress interface. The end result of this change is a decrease in the overhead of VRF for the default, baseline case (ie., no netfilter rules, no packet sockets, no qdisc) from a +3% improvement for UDP which has a lookup per packet (VRF being better than no l3mdev) to ~2% loss for TCP_CRR which connects a socket for each request-response. Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-03-20 11:19:45 -07:00
net: vrf: performance improvements for IPv6 The VRF driver allows users to implement device based features for an entire domain. For example, a qdisc or netfilter rules can be attached to a VRF device or tcpdump can be used to view packets for all devices in the L3 domain. The device-based features come with a performance penalty, most notably in the Tx path. The VRF driver uses the l3mdev_l3_out hook to switch the dst on an skb to its private dst. This allows the skb to traverse the xmit stack with the device set to the VRF device which in turn enables the netfilter and qdisc features. The VRF driver then performs the FIB lookup again and reinserts the packet. This patch avoids the redirect for IPv6 packets if a qdisc has not been attached to a VRF device which is the default config. In this case the netfilter hooks and network taps are directly traversed in the l3mdev_l3_out handler. If a qdisc is attached to a VRF device, then the redirect using the vrf dst is done. Additional overhead is removed by only checking packet taps if a socket is open on the device (vrf_dev->ptype_all list is not empty). Packet sockets bound to any device will still get a copy of the packet via the real ingress or egress interface. The end result of this change is a decrease in the overhead of VRF for the default, baseline case (ie., no netfilter rules, no packet sockets, no qdisc) from a +3% improvement for UDP which has a lookup per packet (VRF being better than no l3mdev) to ~2% loss for TCP_CRR which connects a socket for each request-response. Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-03-20 11:19:45 -07:00
net: vrf: performance improvements for IPv6 The VRF driver allows users to implement device based features for an entire domain. For example, a qdisc or netfilter rules can be attached to a VRF device or tcpdump can be used to view packets for all devices in the L3 domain. The device-based features come with a performance penalty, most notably in the Tx path. The VRF driver uses the l3mdev_l3_out hook to switch the dst on an skb to its private dst. This allows the skb to traverse the xmit stack with the device set to the VRF device which in turn enables the netfilter and qdisc features. The VRF driver then performs the FIB lookup again and reinserts the packet. This patch avoids the redirect for IPv6 packets if a qdisc has not been attached to a VRF device which is the default config. In this case the netfilter hooks and network taps are directly traversed in the l3mdev_l3_out handler. If a qdisc is attached to a VRF device, then the redirect using the vrf dst is done. Additional overhead is removed by only checking packet taps if a socket is open on the device (vrf_dev->ptype_all list is not empty). Packet sockets bound to any device will still get a copy of the packet via the real ingress or egress interface. The end result of this change is a decrease in the overhead of VRF for the default, baseline case (ie., no netfilter rules, no packet sockets, no qdisc) from a +3% improvement for UDP which has a lookup per packet (VRF being better than no l3mdev) to ~2% loss for TCP_CRR which connects a socket for each request-response. Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-03-20 11:19:45 -07:00
net: vrf: performance improvements for IPv6 The VRF driver allows users to implement device based features for an entire domain. For example, a qdisc or netfilter rules can be attached to a VRF device or tcpdump can be used to view packets for all devices in the L3 domain. The device-based features come with a performance penalty, most notably in the Tx path. The VRF driver uses the l3mdev_l3_out hook to switch the dst on an skb to its private dst. This allows the skb to traverse the xmit stack with the device set to the VRF device which in turn enables the netfilter and qdisc features. The VRF driver then performs the FIB lookup again and reinserts the packet. This patch avoids the redirect for IPv6 packets if a qdisc has not been attached to a VRF device which is the default config. In this case the netfilter hooks and network taps are directly traversed in the l3mdev_l3_out handler. If a qdisc is attached to a VRF device, then the redirect using the vrf dst is done. Additional overhead is removed by only checking packet taps if a socket is open on the device (vrf_dev->ptype_all list is not empty). Packet sockets bound to any device will still get a copy of the packet via the real ingress or egress interface. The end result of this change is a decrease in the overhead of VRF for the default, baseline case (ie., no netfilter rules, no packet sockets, no qdisc) from a +3% improvement for UDP which has a lookup per packet (VRF being better than no l3mdev) to ~2% loss for TCP_CRR which connects a socket for each request-response. Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-03-20 11:19:45 -07:00
net: vrf: performance improvements for IPv6 The VRF driver allows users to implement device based features for an entire domain. For example, a qdisc or netfilter rules can be attached to a VRF device or tcpdump can be used to view packets for all devices in the L3 domain. The device-based features come with a performance penalty, most notably in the Tx path. The VRF driver uses the l3mdev_l3_out hook to switch the dst on an skb to its private dst. This allows the skb to traverse the xmit stack with the device set to the VRF device which in turn enables the netfilter and qdisc features. The VRF driver then performs the FIB lookup again and reinserts the packet. This patch avoids the redirect for IPv6 packets if a qdisc has not been attached to a VRF device which is the default config. In this case the netfilter hooks and network taps are directly traversed in the l3mdev_l3_out handler. If a qdisc is attached to a VRF device, then the redirect using the vrf dst is done. Additional overhead is removed by only checking packet taps if a socket is open on the device (vrf_dev->ptype_all list is not empty). Packet sockets bound to any device will still get a copy of the packet via the real ingress or egress interface. The end result of this change is a decrease in the overhead of VRF for the default, baseline case (ie., no netfilter rules, no packet sockets, no qdisc) from a +3% improvement for UDP which has a lookup per packet (VRF being better than no l3mdev) to ~2% loss for TCP_CRR which connects a socket for each request-response. Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-03-20 11:19:45 -07:00
net: vrf: performance improvements for IPv6 The VRF driver allows users to implement device based features for an entire domain. For example, a qdisc or netfilter rules can be attached to a VRF device or tcpdump can be used to view packets for all devices in the L3 domain. The device-based features come with a performance penalty, most notably in the Tx path. The VRF driver uses the l3mdev_l3_out hook to switch the dst on an skb to its private dst. This allows the skb to traverse the xmit stack with the device set to the VRF device which in turn enables the netfilter and qdisc features. The VRF driver then performs the FIB lookup again and reinserts the packet. This patch avoids the redirect for IPv6 packets if a qdisc has not been attached to a VRF device which is the default config. In this case the netfilter hooks and network taps are directly traversed in the l3mdev_l3_out handler. If a qdisc is attached to a VRF device, then the redirect using the vrf dst is done. Additional overhead is removed by only checking packet taps if a socket is open on the device (vrf_dev->ptype_all list is not empty). Packet sockets bound to any device will still get a copy of the packet via the real ingress or egress interface. The end result of this change is a decrease in the overhead of VRF for the default, baseline case (ie., no netfilter rules, no packet sockets, no qdisc) from a +3% improvement for UDP which has a lookup per packet (VRF being better than no l3mdev) to ~2% loss for TCP_CRR which connects a socket for each request-response. Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-03-20 11:19:45 -07:00
net: vrf: performance improvements for IPv6 The VRF driver allows users to implement device based features for an entire domain. For example, a qdisc or netfilter rules can be attached to a VRF device or tcpdump can be used to view packets for all devices in the L3 domain. The device-based features come with a performance penalty, most notably in the Tx path. The VRF driver uses the l3mdev_l3_out hook to switch the dst on an skb to its private dst. This allows the skb to traverse the xmit stack with the device set to the VRF device which in turn enables the netfilter and qdisc features. The VRF driver then performs the FIB lookup again and reinserts the packet. This patch avoids the redirect for IPv6 packets if a qdisc has not been attached to a VRF device which is the default config. In this case the netfilter hooks and network taps are directly traversed in the l3mdev_l3_out handler. If a qdisc is attached to a VRF device, then the redirect using the vrf dst is done. Additional overhead is removed by only checking packet taps if a socket is open on the device (vrf_dev->ptype_all list is not empty). Packet sockets bound to any device will still get a copy of the packet via the real ingress or egress interface. The end result of this change is a decrease in the overhead of VRF for the default, baseline case (ie., no netfilter rules, no packet sockets, no qdisc) from a +3% improvement for UDP which has a lookup per packet (VRF being better than no l3mdev) to ~2% loss for TCP_CRR which connects a socket for each request-response. Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-03-20 11:19:45 -07:00
net: vrf: performance improvements for IPv6 The VRF driver allows users to implement device based features for an entire domain. For example, a qdisc or netfilter rules can be attached to a VRF device or tcpdump can be used to view packets for all devices in the L3 domain. The device-based features come with a performance penalty, most notably in the Tx path. The VRF driver uses the l3mdev_l3_out hook to switch the dst on an skb to its private dst. This allows the skb to traverse the xmit stack with the device set to the VRF device which in turn enables the netfilter and qdisc features. The VRF driver then performs the FIB lookup again and reinserts the packet. This patch avoids the redirect for IPv6 packets if a qdisc has not been attached to a VRF device which is the default config. In this case the netfilter hooks and network taps are directly traversed in the l3mdev_l3_out handler. If a qdisc is attached to a VRF device, then the redirect using the vrf dst is done. Additional overhead is removed by only checking packet taps if a socket is open on the device (vrf_dev->ptype_all list is not empty). Packet sockets bound to any device will still get a copy of the packet via the real ingress or egress interface. The end result of this change is a decrease in the overhead of VRF for the default, baseline case (ie., no netfilter rules, no packet sockets, no qdisc) from a +3% improvement for UDP which has a lookup per packet (VRF being better than no l3mdev) to ~2% loss for TCP_CRR which connects a socket for each request-response. Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-03-20 11:19:45 -07:00
net: vrf: Fix crash when IPv6 is disabled at boot time Frank Kellermann reported a kernel crash with 4.5.0 when IPv6 is disabled at boot using the kernel option ipv6.disable=1. Using current net-next with the boot option: $ ip link add red type vrf table 1001 Generates: [12210.919584] BUG: unable to handle kernel NULL pointer dereference at 0000000000000748 [12210.921341] IP: [<ffffffff814b30e3>] fib6_get_table+0x2c/0x5a [12210.922537] PGD b79e3067 PUD bb32b067 PMD 0 [12210.923479] Oops: 0000 [#1] SMP [12210.924001] Modules linked in: ipvlan 8021q garp mrp stp llc [12210.925130] CPU: 3 PID: 1177 Comm: ip Not tainted 4.7.0-rc1+ #235 [12210.926168] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.7.5-20140531_083030-gandalf 04/01/2014 [12210.928065] task: ffff8800b9ac4640 ti: ffff8800bacac000 task.ti: ffff8800bacac000 [12210.929328] RIP: 0010:[<ffffffff814b30e3>] [<ffffffff814b30e3>] fib6_get_table+0x2c/0x5a [12210.930697] RSP: 0018:ffff8800bacaf888 EFLAGS: 00010202 [12210.931563] RAX: 0000000000000748 RBX: ffffffff81a9e280 RCX: ffff8800b9ac4e28 [12210.932688] RDX: 00000000000000e9 RSI: 0000000000000002 RDI: 0000000000000286 [12210.933820] RBP: ffff8800bacaf898 R08: ffff8800b9ac4df0 R09: 000000000052001b [12210.934941] R10: 00000000657c0000 R11: 000000000000c649 R12: 00000000000003e9 [12210.936032] R13: 00000000000003e9 R14: ffff8800bace7800 R15: ffff8800bb3ec000 [12210.937103] FS: 00007faa1766c700(0000) GS:ffff88013ac00000(0000) knlGS:0000000000000000 [12210.938321] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [12210.939166] CR2: 0000000000000748 CR3: 00000000b79d6000 CR4: 00000000000406e0 [12210.940278] Stack: [12210.940603] ffff8800bb3ec000 ffffffff81a9e280 ffff8800bacaf8c8 ffffffff814b3135 [12210.941818] ffff8800bb3ec000 ffffffff81a9e280 ffffffff81a9e280 ffff8800bace7800 [12210.943040] ffff8800bacaf8f0 ffffffff81397c88 ffff8800bb3ec000 ffffffff81a9e280 [12210.944288] Call Trace: [12210.944688] [<ffffffff814b3135>] fib6_new_table+0x24/0x8a [12210.945516] [<ffffffff81397c88>] vrf_dev_init+0xd4/0x162 [12210.946328] [<ffffffff814091e1>] register_netdevice+0x100/0x396 [12210.947209] [<ffffffff8139823d>] vrf_newlink+0x40/0xb3 [12210.948001] [<ffffffff814187f0>] rtnl_newlink+0x5d3/0x6d5 ... The problem above is due to the fact that the fib hash table is not allocated when IPv6 is disabled at boot. As for the VRF driver it should not do any IPv6 initializations if IPv6 is disabled, so it needs to know if IPv6 is disabled at boot. The disable parameter is private to the IPv6 module, so provide an accessor for modules to determine if IPv6 was disabled at boot time. Fixes: 35402e3136634 ("net: Add IPv6 support to VRF device") Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2016-06-09 10:21:00 -07:00
ipv4: Add helpers for neigh lookup for nexthop A common theme in the output path is looking up a neigh entry for a nexthop, either the gateway in an rtable or a fallback to the daddr in the skb: nexthop = (__force u32)rt_nexthop(rt, ip_hdr(skb)->daddr); neigh = __ipv4_neigh_lookup_noref(dev, nexthop); if (unlikely(!neigh)) neigh = __neigh_create(&arp_tbl, &nexthop, dev, false); To allow the nexthop to be an IPv6 address we need to consider the family of the nexthop and then call __ipv{4,6}_neigh_lookup_noref based on it. To make this simpler, add a ip_neigh_gw4 helper similar to ip_neigh_gw6 added in an earlier patch which handles: neigh = __ipv4_neigh_lookup_noref(dev, nexthop); if (unlikely(!neigh)) neigh = __neigh_create(&arp_tbl, &nexthop, dev, false); And then add a second one, ip_neigh_for_gw, that calls either ip_neigh_gw4 or ip_neigh_gw6 based on the address family of the gateway. Update the output paths in the VRF driver and core v4 code to use ip_neigh_for_gw simplifying the family based lookup and making both ready for a v6 nexthop. ipv4_neigh_lookup has a different need - the potential to resolve a passed in address in addition to any gateway in the rtable or skb. Since this is a one-off, add ip_neigh_gw4 and ip_neigh_gw6 diectly. The difference between __neigh_create used by the helpers and neigh_create called by ipv4_neigh_lookup is taking a refcount, so add rcu_read_lock_bh and bump the refcnt on the neigh entry. Signed-off-by: David Ahern <dsahern@gmail.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2019-04-05 16:30:34 -07:00
ipv4: Add helpers for neigh lookup for nexthop A common theme in the output path is looking up a neigh entry for a nexthop, either the gateway in an rtable or a fallback to the daddr in the skb: nexthop = (__force u32)rt_nexthop(rt, ip_hdr(skb)->daddr); neigh = __ipv4_neigh_lookup_noref(dev, nexthop); if (unlikely(!neigh)) neigh = __neigh_create(&arp_tbl, &nexthop, dev, false); To allow the nexthop to be an IPv6 address we need to consider the family of the nexthop and then call __ipv{4,6}_neigh_lookup_noref based on it. To make this simpler, add a ip_neigh_gw4 helper similar to ip_neigh_gw6 added in an earlier patch which handles: neigh = __ipv4_neigh_lookup_noref(dev, nexthop); if (unlikely(!neigh)) neigh = __neigh_create(&arp_tbl, &nexthop, dev, false); And then add a second one, ip_neigh_for_gw, that calls either ip_neigh_gw4 or ip_neigh_gw6 based on the address family of the gateway. Update the output paths in the VRF driver and core v4 code to use ip_neigh_for_gw simplifying the family based lookup and making both ready for a v6 nexthop. ipv4_neigh_lookup has a different need - the potential to resolve a passed in address in addition to any gateway in the rtable or skb. Since this is a one-off, add ip_neigh_gw4 and ip_neigh_gw6 diectly. The difference between __neigh_create used by the helpers and neigh_create called by ipv4_neigh_lookup is taking a refcount, so add rcu_read_lock_bh and bump the refcnt on the neigh entry. Signed-off-by: David Ahern <dsahern@gmail.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2019-04-05 16:30:34 -07:00
ipv4: Add helpers for neigh lookup for nexthop A common theme in the output path is looking up a neigh entry for a nexthop, either the gateway in an rtable or a fallback to the daddr in the skb: nexthop = (__force u32)rt_nexthop(rt, ip_hdr(skb)->daddr); neigh = __ipv4_neigh_lookup_noref(dev, nexthop); if (unlikely(!neigh)) neigh = __neigh_create(&arp_tbl, &nexthop, dev, false); To allow the nexthop to be an IPv6 address we need to consider the family of the nexthop and then call __ipv{4,6}_neigh_lookup_noref based on it. To make this simpler, add a ip_neigh_gw4 helper similar to ip_neigh_gw6 added in an earlier patch which handles: neigh = __ipv4_neigh_lookup_noref(dev, nexthop); if (unlikely(!neigh)) neigh = __neigh_create(&arp_tbl, &nexthop, dev, false); And then add a second one, ip_neigh_for_gw, that calls either ip_neigh_gw4 or ip_neigh_gw6 based on the address family of the gateway. Update the output paths in the VRF driver and core v4 code to use ip_neigh_for_gw simplifying the family based lookup and making both ready for a v6 nexthop. ipv4_neigh_lookup has a different need - the potential to resolve a passed in address in addition to any gateway in the rtable or skb. Since this is a one-off, add ip_neigh_gw4 and ip_neigh_gw6 diectly. The difference between __neigh_create used by the helpers and neigh_create called by ipv4_neigh_lookup is taking a refcount, so add rcu_read_lock_bh and bump the refcnt on the neigh entry. Signed-off-by: David Ahern <dsahern@gmail.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2019-04-05 16:30:34 -07:00
2015-09-15 20:04:16 -05:00
2015-09-15 20:04:16 -05:00
net: vrf: performance improvements for IPv4 The VRF driver allows users to implement device based features for an entire domain. For example, a qdisc or netfilter rules can be attached to a VRF device or tcpdump can be used to view packets for all devices in the L3 domain. The device-based features come with a performance penalty, most notably in the Tx path. The VRF driver uses the l3mdev_l3_out hook to switch the dst on an skb to its private dst. This allows the skb to traverse the xmit stack with the device set to the VRF device which in turn enables the netfilter and qdisc features. The VRF driver then performs the FIB lookup again and reinserts the packet. This patch avoids the redirect for IPv4 packets if a qdisc has not been attached to a VRF device which is the default config. In this case the netfilter hooks and network taps are directly traversed in the l3mdev_l3_out handler. If a qdisc is attached to a VRF device, then the redirect using the vrf dst is done. Additional overhead is removed by only checking packet taps if a socket is open on the device (vrf_dev->ptype_all list is not empty). Packet sockets bound to any device will still get a copy of the packet via the real ingress or egress interface. The end result of this change is a decrease in the overhead of VRF for the default, baseline case (ie., no netfilter rules, no packet sockets, no qdisc) to ~3% for UDP which has a lookup per packet and < 1% overhead for connected sockets that leverage early demux and avoid FIB lookups. Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-03-20 11:19:44 -07:00
net: vrf: performance improvements for IPv4 The VRF driver allows users to implement device based features for an entire domain. For example, a qdisc or netfilter rules can be attached to a VRF device or tcpdump can be used to view packets for all devices in the L3 domain. The device-based features come with a performance penalty, most notably in the Tx path. The VRF driver uses the l3mdev_l3_out hook to switch the dst on an skb to its private dst. This allows the skb to traverse the xmit stack with the device set to the VRF device which in turn enables the netfilter and qdisc features. The VRF driver then performs the FIB lookup again and reinserts the packet. This patch avoids the redirect for IPv4 packets if a qdisc has not been attached to a VRF device which is the default config. In this case the netfilter hooks and network taps are directly traversed in the l3mdev_l3_out handler. If a qdisc is attached to a VRF device, then the redirect using the vrf dst is done. Additional overhead is removed by only checking packet taps if a socket is open on the device (vrf_dev->ptype_all list is not empty). Packet sockets bound to any device will still get a copy of the packet via the real ingress or egress interface. The end result of this change is a decrease in the overhead of VRF for the default, baseline case (ie., no netfilter rules, no packet sockets, no qdisc) to ~3% for UDP which has a lookup per packet and < 1% overhead for connected sockets that leverage early demux and avoid FIB lookups. Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-03-20 11:19:44 -07:00
net: vrf: performance improvements for IPv4 The VRF driver allows users to implement device based features for an entire domain. For example, a qdisc or netfilter rules can be attached to a VRF device or tcpdump can be used to view packets for all devices in the L3 domain. The device-based features come with a performance penalty, most notably in the Tx path. The VRF driver uses the l3mdev_l3_out hook to switch the dst on an skb to its private dst. This allows the skb to traverse the xmit stack with the device set to the VRF device which in turn enables the netfilter and qdisc features. The VRF driver then performs the FIB lookup again and reinserts the packet. This patch avoids the redirect for IPv4 packets if a qdisc has not been attached to a VRF device which is the default config. In this case the netfilter hooks and network taps are directly traversed in the l3mdev_l3_out handler. If a qdisc is attached to a VRF device, then the redirect using the vrf dst is done. Additional overhead is removed by only checking packet taps if a socket is open on the device (vrf_dev->ptype_all list is not empty). Packet sockets bound to any device will still get a copy of the packet via the real ingress or egress interface. The end result of this change is a decrease in the overhead of VRF for the default, baseline case (ie., no netfilter rules, no packet sockets, no qdisc) to ~3% for UDP which has a lookup per packet and < 1% overhead for connected sockets that leverage early demux and avoid FIB lookups. Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-03-20 11:19:44 -07:00
net: vrf: performance improvements for IPv4 The VRF driver allows users to implement device based features for an entire domain. For example, a qdisc or netfilter rules can be attached to a VRF device or tcpdump can be used to view packets for all devices in the L3 domain. The device-based features come with a performance penalty, most notably in the Tx path. The VRF driver uses the l3mdev_l3_out hook to switch the dst on an skb to its private dst. This allows the skb to traverse the xmit stack with the device set to the VRF device which in turn enables the netfilter and qdisc features. The VRF driver then performs the FIB lookup again and reinserts the packet. This patch avoids the redirect for IPv4 packets if a qdisc has not been attached to a VRF device which is the default config. In this case the netfilter hooks and network taps are directly traversed in the l3mdev_l3_out handler. If a qdisc is attached to a VRF device, then the redirect using the vrf dst is done. Additional overhead is removed by only checking packet taps if a socket is open on the device (vrf_dev->ptype_all list is not empty). Packet sockets bound to any device will still get a copy of the packet via the real ingress or egress interface. The end result of this change is a decrease in the overhead of VRF for the default, baseline case (ie., no netfilter rules, no packet sockets, no qdisc) to ~3% for UDP which has a lookup per packet and < 1% overhead for connected sockets that leverage early demux and avoid FIB lookups. Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-03-20 11:19:44 -07:00
net: vrf: performance improvements for IPv4 The VRF driver allows users to implement device based features for an entire domain. For example, a qdisc or netfilter rules can be attached to a VRF device or tcpdump can be used to view packets for all devices in the L3 domain. The device-based features come with a performance penalty, most notably in the Tx path. The VRF driver uses the l3mdev_l3_out hook to switch the dst on an skb to its private dst. This allows the skb to traverse the xmit stack with the device set to the VRF device which in turn enables the netfilter and qdisc features. The VRF driver then performs the FIB lookup again and reinserts the packet. This patch avoids the redirect for IPv4 packets if a qdisc has not been attached to a VRF device which is the default config. In this case the netfilter hooks and network taps are directly traversed in the l3mdev_l3_out handler. If a qdisc is attached to a VRF device, then the redirect using the vrf dst is done. Additional overhead is removed by only checking packet taps if a socket is open on the device (vrf_dev->ptype_all list is not empty). Packet sockets bound to any device will still get a copy of the packet via the real ingress or egress interface. The end result of this change is a decrease in the overhead of VRF for the default, baseline case (ie., no netfilter rules, no packet sockets, no qdisc) to ~3% for UDP which has a lookup per packet and < 1% overhead for connected sockets that leverage early demux and avoid FIB lookups. Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-03-20 11:19:44 -07:00
net: vrf: performance improvements for IPv4 The VRF driver allows users to implement device based features for an entire domain. For example, a qdisc or netfilter rules can be attached to a VRF device or tcpdump can be used to view packets for all devices in the L3 domain. The device-based features come with a performance penalty, most notably in the Tx path. The VRF driver uses the l3mdev_l3_out hook to switch the dst on an skb to its private dst. This allows the skb to traverse the xmit stack with the device set to the VRF device which in turn enables the netfilter and qdisc features. The VRF driver then performs the FIB lookup again and reinserts the packet. This patch avoids the redirect for IPv4 packets if a qdisc has not been attached to a VRF device which is the default config. In this case the netfilter hooks and network taps are directly traversed in the l3mdev_l3_out handler. If a qdisc is attached to a VRF device, then the redirect using the vrf dst is done. Additional overhead is removed by only checking packet taps if a socket is open on the device (vrf_dev->ptype_all list is not empty). Packet sockets bound to any device will still get a copy of the packet via the real ingress or egress interface. The end result of this change is a decrease in the overhead of VRF for the default, baseline case (ie., no netfilter rules, no packet sockets, no qdisc) to ~3% for UDP which has a lookup per packet and < 1% overhead for connected sockets that leverage early demux and avoid FIB lookups. Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-03-20 11:19:44 -07:00
net: vrf: performance improvements for IPv4 The VRF driver allows users to implement device based features for an entire domain. For example, a qdisc or netfilter rules can be attached to a VRF device or tcpdump can be used to view packets for all devices in the L3 domain. The device-based features come with a performance penalty, most notably in the Tx path. The VRF driver uses the l3mdev_l3_out hook to switch the dst on an skb to its private dst. This allows the skb to traverse the xmit stack with the device set to the VRF device which in turn enables the netfilter and qdisc features. The VRF driver then performs the FIB lookup again and reinserts the packet. This patch avoids the redirect for IPv4 packets if a qdisc has not been attached to a VRF device which is the default config. In this case the netfilter hooks and network taps are directly traversed in the l3mdev_l3_out handler. If a qdisc is attached to a VRF device, then the redirect using the vrf dst is done. Additional overhead is removed by only checking packet taps if a socket is open on the device (vrf_dev->ptype_all list is not empty). Packet sockets bound to any device will still get a copy of the packet via the real ingress or egress interface. The end result of this change is a decrease in the overhead of VRF for the default, baseline case (ie., no netfilter rules, no packet sockets, no qdisc) to ~3% for UDP which has a lookup per packet and < 1% overhead for connected sockets that leverage early demux and avoid FIB lookups. Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-03-20 11:19:44 -07:00
net: vrf: performance improvements for IPv4 The VRF driver allows users to implement device based features for an entire domain. For example, a qdisc or netfilter rules can be attached to a VRF device or tcpdump can be used to view packets for all devices in the L3 domain. The device-based features come with a performance penalty, most notably in the Tx path. The VRF driver uses the l3mdev_l3_out hook to switch the dst on an skb to its private dst. This allows the skb to traverse the xmit stack with the device set to the VRF device which in turn enables the netfilter and qdisc features. The VRF driver then performs the FIB lookup again and reinserts the packet. This patch avoids the redirect for IPv4 packets if a qdisc has not been attached to a VRF device which is the default config. In this case the netfilter hooks and network taps are directly traversed in the l3mdev_l3_out handler. If a qdisc is attached to a VRF device, then the redirect using the vrf dst is done. Additional overhead is removed by only checking packet taps if a socket is open on the device (vrf_dev->ptype_all list is not empty). Packet sockets bound to any device will still get a copy of the packet via the real ingress or egress interface. The end result of this change is a decrease in the overhead of VRF for the default, baseline case (ie., no netfilter rules, no packet sockets, no qdisc) to ~3% for UDP which has a lookup per packet and < 1% overhead for connected sockets that leverage early demux and avoid FIB lookups. Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-03-20 11:19:44 -07:00
net: vrf: performance improvements for IPv4 The VRF driver allows users to implement device based features for an entire domain. For example, a qdisc or netfilter rules can be attached to a VRF device or tcpdump can be used to view packets for all devices in the L3 domain. The device-based features come with a performance penalty, most notably in the Tx path. The VRF driver uses the l3mdev_l3_out hook to switch the dst on an skb to its private dst. This allows the skb to traverse the xmit stack with the device set to the VRF device which in turn enables the netfilter and qdisc features. The VRF driver then performs the FIB lookup again and reinserts the packet. This patch avoids the redirect for IPv4 packets if a qdisc has not been attached to a VRF device which is the default config. In this case the netfilter hooks and network taps are directly traversed in the l3mdev_l3_out handler. If a qdisc is attached to a VRF device, then the redirect using the vrf dst is done. Additional overhead is removed by only checking packet taps if a socket is open on the device (vrf_dev->ptype_all list is not empty). Packet sockets bound to any device will still get a copy of the packet via the real ingress or egress interface. The end result of this change is a decrease in the overhead of VRF for the default, baseline case (ie., no netfilter rules, no packet sockets, no qdisc) to ~3% for UDP which has a lookup per packet and < 1% overhead for connected sockets that leverage early demux and avoid FIB lookups. Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-03-20 11:19:44 -07:00
net: vrf: ipv4 support for local traffic to local addresses Add support for locally originated traffic to VRF-local addresses. If destination device for an skb is the loopback or VRF device then set its dst to a local version of the VRF cached dst_entry and call netif_rx to insert the packet onto the rx queue - similar to what is done for loopback. This patch handles IPv4 support; follow on patch handles IPv6. With this patch, ping, tcp and udp packets to a local IPv4 address are successfully routed: $ ip addr show dev eth1 4: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master red state UP group default qlen 1000 link/ether 02:e0:f9:1c:b9:74 brd ff:ff:ff:ff:ff:ff inet 10.100.1.1/24 brd 10.100.1.255 scope global eth1 valid_lft forever preferred_lft forever inet6 2100:1::1/120 scope global valid_lft forever preferred_lft forever inet6 fe80::e0:f9ff:fe1c:b974/64 scope link valid_lft forever preferred_lft forever $ ping -c1 -I red 10.100.1.1 ping: Warning: source address might be selected on device other than red. PING 10.100.1.1 (10.100.1.1) from 10.100.1.1 red: 56(84) bytes of data. 64 bytes from 10.100.1.1: icmp_seq=1 ttl=64 time=0.057 ms This patch also enables use of IPv4 loopback address on the VRF device: $ ip addr add dev red 127.0.0.1/8 $ ping -c1 -I red 127.0.0.1 PING 127.0.0.1 (127.0.0.1) from 127.0.0.1 red: 56(84) bytes of data. 64 bytes from 127.0.0.1: icmp_seq=1 ttl=64 time=0.058 ms Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2016-06-06 20:50:39 -07:00
net: vrf: ipv4 support for local traffic to local addresses Add support for locally originated traffic to VRF-local addresses. If destination device for an skb is the loopback or VRF device then set its dst to a local version of the VRF cached dst_entry and call netif_rx to insert the packet onto the rx queue - similar to what is done for loopback. This patch handles IPv4 support; follow on patch handles IPv6. With this patch, ping, tcp and udp packets to a local IPv4 address are successfully routed: $ ip addr show dev eth1 4: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master red state UP group default qlen 1000 link/ether 02:e0:f9:1c:b9:74 brd ff:ff:ff:ff:ff:ff inet 10.100.1.1/24 brd 10.100.1.255 scope global eth1 valid_lft forever preferred_lft forever inet6 2100:1::1/120 scope global valid_lft forever preferred_lft forever inet6 fe80::e0:f9ff:fe1c:b974/64 scope link valid_lft forever preferred_lft forever $ ping -c1 -I red 10.100.1.1 ping: Warning: source address might be selected on device other than red. PING 10.100.1.1 (10.100.1.1) from 10.100.1.1 red: 56(84) bytes of data. 64 bytes from 10.100.1.1: icmp_seq=1 ttl=64 time=0.057 ms This patch also enables use of IPv4 loopback address on the VRF device: $ ip addr add dev red 127.0.0.1/8 $ ping -c1 -I red 127.0.0.1 PING 127.0.0.1 (127.0.0.1) from 127.0.0.1 red: 56(84) bytes of data. 64 bytes from 127.0.0.1: icmp_seq=1 ttl=64 time=0.058 ms Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2016-06-06 20:50:39 -07:00
vrf: do not push non-ND strict packets with a source LLA through packet taps again Non-ND strict packets with a source LLA go through the packet taps again, while non-ND strict packets with other source addresses do not, and we can see a clone of those packets on the vrf interface (we should not). This is due to a series of changes: Commit 6f12fa775530[1] made non-ND strict packets not being pushed again in the packet taps. This changed with commit 205704c618af[2] for those packets having a source LLA, as they need a lookup with the orig_iif. The issue now is those packets do not skip the 'vrf_ip6_rcv' function to the end (as the ones without a source LLA) and go through the check to call packet taps again. This check was changed by commit 6f12fa775530[1] and do not exclude non-strict packets anymore. Packets matching 'need_strict && !is_ndisc && is_ll_src' are now being sent through the packet taps again. This can be seen by dumping packets on the vrf interface. Fix this by having the same code path for all non-ND strict packets and selectively lookup with the orig_iif for those with a source LLA. This has the effect to revert to the pre-205704c618af[2] condition, which should also be easier to maintain. [1] 6f12fa775530 ("vrf: mark skb for multicast or link-local as enslaved to VRF") [2] 205704c618af ("vrf: packets with lladdr src needs dst at input with orig_iif when needs strict") Fixes: 205704c618af ("vrf: packets with lladdr src needs dst at input with orig_iif when needs strict") Cc: Stephen Suryaputra <ssuryaextr@gmail.com> Reported-by: Paolo Abeni <pabeni@redhat.com> Signed-off-by: Antoine Tenart <atenart@kernel.org> Reviewed-by: David Ahern <dsahern@kernel.org> Signed-off-by: David S. Miller <davem@davemloft.net>
2021-06-18 17:15:53 +02:00
vrf: do not push non-ND strict packets with a source LLA through packet taps again Non-ND strict packets with a source LLA go through the packet taps again, while non-ND strict packets with other source addresses do not, and we can see a clone of those packets on the vrf interface (we should not). This is due to a series of changes: Commit 6f12fa775530[1] made non-ND strict packets not being pushed again in the packet taps. This changed with commit 205704c618af[2] for those packets having a source LLA, as they need a lookup with the orig_iif. The issue now is those packets do not skip the 'vrf_ip6_rcv' function to the end (as the ones without a source LLA) and go through the check to call packet taps again. This check was changed by commit 6f12fa775530[1] and do not exclude non-strict packets anymore. Packets matching 'need_strict && !is_ndisc && is_ll_src' are now being sent through the packet taps again. This can be seen by dumping packets on the vrf interface. Fix this by having the same code path for all non-ND strict packets and selectively lookup with the orig_iif for those with a source LLA. This has the effect to revert to the pre-205704c618af[2] condition, which should also be easier to maintain. [1] 6f12fa775530 ("vrf: mark skb for multicast or link-local as enslaved to VRF") [2] 205704c618af ("vrf: packets with lladdr src needs dst at input with orig_iif when needs strict") Fixes: 205704c618af ("vrf: packets with lladdr src needs dst at input with orig_iif when needs strict") Cc: Stephen Suryaputra <ssuryaextr@gmail.com> Reported-by: Paolo Abeni <pabeni@redhat.com> Signed-off-by: Antoine Tenart <atenart@kernel.org> Reviewed-by: David Ahern <dsahern@kernel.org> Signed-off-by: David S. Miller <davem@davemloft.net>
2021-06-18 17:15:53 +02:00
net: Require exact match for TCP socket lookups if dif is l3mdev Currently, socket lookups for l3mdev (vrf) use cases can match a socket that is bound to a port but not a device (ie., a global socket). If the sysctl tcp_l3mdev_accept is not set this leads to ack packets going out based on the main table even though the packet came in from an L3 domain. The end result is that the connection does not establish creating confusion for users since the service is running and a socket shows in ss output. Fix by requiring an exact dif to sk_bound_dev_if match if the skb came through an interface enslaved to an l3mdev device and the tcp_l3mdev_accept is not set. skb's through an l3mdev interface are marked by setting a flag in inet{6}_skb_parm. The IPv6 variant is already set; this patch adds the flag for IPv4. Using an skb flag avoids a device lookup on the dif. The flag is set in the VRF driver using the IP{6}CB macros. For IPv4, the inet_skb_parm struct is moved in the cb per commit 971f10eca186, so the match function in the TCP stack needs to use TCP_SKB_CB. For IPv6, the move is done after the socket lookup, so IP6CB is used. The flags field in inet_skb_parm struct needs to be increased to add another flag. There is currently a 1-byte hole following the flags, so it can be expanded to u16 without increasing the size of the struct. Fixes: 193125dbd8eb ("net: Introduce VRF device driver") Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2016-10-16 20:02:52 -07:00
vrf: do not push non-ND strict packets with a source LLA through packet taps again Non-ND strict packets with a source LLA go through the packet taps again, while non-ND strict packets with other source addresses do not, and we can see a clone of those packets on the vrf interface (we should not). This is due to a series of changes: Commit 6f12fa775530[1] made non-ND strict packets not being pushed again in the packet taps. This changed with commit 205704c618af[2] for those packets having a source LLA, as they need a lookup with the orig_iif. The issue now is those packets do not skip the 'vrf_ip6_rcv' function to the end (as the ones without a source LLA) and go through the check to call packet taps again. This check was changed by commit 6f12fa775530[1] and do not exclude non-strict packets anymore. Packets matching 'need_strict && !is_ndisc && is_ll_src' are now being sent through the packet taps again. This can be seen by dumping packets on the vrf interface. Fix this by having the same code path for all non-ND strict packets and selectively lookup with the orig_iif for those with a source LLA. This has the effect to revert to the pre-205704c618af[2] condition, which should also be easier to maintain. [1] 6f12fa775530 ("vrf: mark skb for multicast or link-local as enslaved to VRF") [2] 205704c618af ("vrf: packets with lladdr src needs dst at input with orig_iif when needs strict") Fixes: 205704c618af ("vrf: packets with lladdr src needs dst at input with orig_iif when needs strict") Cc: Stephen Suryaputra <ssuryaextr@gmail.com> Reported-by: Paolo Abeni <pabeni@redhat.com> Signed-off-by: Antoine Tenart <atenart@kernel.org> Reviewed-by: David Ahern <dsahern@kernel.org> Signed-off-by: David S. Miller <davem@davemloft.net>
2021-06-18 17:15:53 +02:00
vrf: do not push non-ND strict packets with a source LLA through packet taps again Non-ND strict packets with a source LLA go through the packet taps again, while non-ND strict packets with other source addresses do not, and we can see a clone of those packets on the vrf interface (we should not). This is due to a series of changes: Commit 6f12fa775530[1] made non-ND strict packets not being pushed again in the packet taps. This changed with commit 205704c618af[2] for those packets having a source LLA, as they need a lookup with the orig_iif. The issue now is those packets do not skip the 'vrf_ip6_rcv' function to the end (as the ones without a source LLA) and go through the check to call packet taps again. This check was changed by commit 6f12fa775530[1] and do not exclude non-strict packets anymore. Packets matching 'need_strict && !is_ndisc && is_ll_src' are now being sent through the packet taps again. This can be seen by dumping packets on the vrf interface. Fix this by having the same code path for all non-ND strict packets and selectively lookup with the orig_iif for those with a source LLA. This has the effect to revert to the pre-205704c618af[2] condition, which should also be easier to maintain. [1] 6f12fa775530 ("vrf: mark skb for multicast or link-local as enslaved to VRF") [2] 205704c618af ("vrf: packets with lladdr src needs dst at input with orig_iif when needs strict") Fixes: 205704c618af ("vrf: packets with lladdr src needs dst at input with orig_iif when needs strict") Cc: Stephen Suryaputra <ssuryaextr@gmail.com> Reported-by: Paolo Abeni <pabeni@redhat.com> Signed-off-by: Antoine Tenart <atenart@kernel.org> Reviewed-by: David Ahern <dsahern@kernel.org> Signed-off-by: David S. Miller <davem@davemloft.net>
2021-06-18 17:15:53 +02:00
net: vrf: performance improvements for IPv6 The VRF driver allows users to implement device based features for an entire domain. For example, a qdisc or netfilter rules can be attached to a VRF device or tcpdump can be used to view packets for all devices in the L3 domain. The device-based features come with a performance penalty, most notably in the Tx path. The VRF driver uses the l3mdev_l3_out hook to switch the dst on an skb to its private dst. This allows the skb to traverse the xmit stack with the device set to the VRF device which in turn enables the netfilter and qdisc features. The VRF driver then performs the FIB lookup again and reinserts the packet. This patch avoids the redirect for IPv6 packets if a qdisc has not been attached to a VRF device which is the default config. In this case the netfilter hooks and network taps are directly traversed in the l3mdev_l3_out handler. If a qdisc is attached to a VRF device, then the redirect using the vrf dst is done. Additional overhead is removed by only checking packet taps if a socket is open on the device (vrf_dev->ptype_all list is not empty). Packet sockets bound to any device will still get a copy of the packet via the real ingress or egress interface. The end result of this change is a decrease in the overhead of VRF for the default, baseline case (ie., no netfilter rules, no packet sockets, no qdisc) from a +3% improvement for UDP which has a lookup per packet (VRF being better than no l3mdev) to ~2% loss for TCP_CRR which connects a socket for each request-response. Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-03-20 11:19:45 -07:00
net: vrf: performance improvements for IPv6 The VRF driver allows users to implement device based features for an entire domain. For example, a qdisc or netfilter rules can be attached to a VRF device or tcpdump can be used to view packets for all devices in the L3 domain. The device-based features come with a performance penalty, most notably in the Tx path. The VRF driver uses the l3mdev_l3_out hook to switch the dst on an skb to its private dst. This allows the skb to traverse the xmit stack with the device set to the VRF device which in turn enables the netfilter and qdisc features. The VRF driver then performs the FIB lookup again and reinserts the packet. This patch avoids the redirect for IPv6 packets if a qdisc has not been attached to a VRF device which is the default config. In this case the netfilter hooks and network taps are directly traversed in the l3mdev_l3_out handler. If a qdisc is attached to a VRF device, then the redirect using the vrf dst is done. Additional overhead is removed by only checking packet taps if a socket is open on the device (vrf_dev->ptype_all list is not empty). Packet sockets bound to any device will still get a copy of the packet via the real ingress or egress interface. The end result of this change is a decrease in the overhead of VRF for the default, baseline case (ie., no netfilter rules, no packet sockets, no qdisc) from a +3% improvement for UDP which has a lookup per packet (VRF being better than no l3mdev) to ~2% loss for TCP_CRR which connects a socket for each request-response. Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-03-20 11:19:45 -07:00
net: Require exact match for TCP socket lookups if dif is l3mdev Currently, socket lookups for l3mdev (vrf) use cases can match a socket that is bound to a port but not a device (ie., a global socket). If the sysctl tcp_l3mdev_accept is not set this leads to ack packets going out based on the main table even though the packet came in from an L3 domain. The end result is that the connection does not establish creating confusion for users since the service is running and a socket shows in ss output. Fix by requiring an exact dif to sk_bound_dev_if match if the skb came through an interface enslaved to an l3mdev device and the tcp_l3mdev_accept is not set. skb's through an l3mdev interface are marked by setting a flag in inet{6}_skb_parm. The IPv6 variant is already set; this patch adds the flag for IPv4. Using an skb flag avoids a device lookup on the dif. The flag is set in the VRF driver using the IP{6}CB macros. For IPv4, the inet_skb_parm struct is moved in the cb per commit 971f10eca186, so the match function in the TCP stack needs to use TCP_SKB_CB. For IPv6, the move is done after the socket lookup, so IP6CB is used. The flags field in inet_skb_parm struct needs to be increased to add another flag. There is currently a 1-byte hole following the flags, so it can be expanded to u16 without increasing the size of the struct. Fixes: 193125dbd8eb ("net: Introduce VRF device driver") Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2016-10-16 20:02:52 -07:00
net: vrf: ipv4 support for local traffic to local addresses Add support for locally originated traffic to VRF-local addresses. If destination device for an skb is the loopback or VRF device then set its dst to a local version of the VRF cached dst_entry and call netif_rx to insert the packet onto the rx queue - similar to what is done for loopback. This patch handles IPv4 support; follow on patch handles IPv6. With this patch, ping, tcp and udp packets to a local IPv4 address are successfully routed: $ ip addr show dev eth1 4: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master red state UP group default qlen 1000 link/ether 02:e0:f9:1c:b9:74 brd ff:ff:ff:ff:ff:ff inet 10.100.1.1/24 brd 10.100.1.255 scope global eth1 valid_lft forever preferred_lft forever inet6 2100:1::1/120 scope global valid_lft forever preferred_lft forever inet6 fe80::e0:f9ff:fe1c:b974/64 scope link valid_lft forever preferred_lft forever $ ping -c1 -I red 10.100.1.1 ping: Warning: source address might be selected on device other than red. PING 10.100.1.1 (10.100.1.1) from 10.100.1.1 red: 56(84) bytes of data. 64 bytes from 10.100.1.1: icmp_seq=1 ttl=64 time=0.057 ms This patch also enables use of IPv4 loopback address on the VRF device: $ ip addr add dev red 127.0.0.1/8 $ ping -c1 -I red 127.0.0.1 PING 127.0.0.1 (127.0.0.1) from 127.0.0.1 red: 56(84) bytes of data. 64 bytes from 127.0.0.1: icmp_seq=1 ttl=64 time=0.058 ms Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2016-06-06 20:50:39 -07:00
net: vrf: performance improvements for IPv4 The VRF driver allows users to implement device based features for an entire domain. For example, a qdisc or netfilter rules can be attached to a VRF device or tcpdump can be used to view packets for all devices in the L3 domain. The device-based features come with a performance penalty, most notably in the Tx path. The VRF driver uses the l3mdev_l3_out hook to switch the dst on an skb to its private dst. This allows the skb to traverse the xmit stack with the device set to the VRF device which in turn enables the netfilter and qdisc features. The VRF driver then performs the FIB lookup again and reinserts the packet. This patch avoids the redirect for IPv4 packets if a qdisc has not been attached to a VRF device which is the default config. In this case the netfilter hooks and network taps are directly traversed in the l3mdev_l3_out handler. If a qdisc is attached to a VRF device, then the redirect using the vrf dst is done. Additional overhead is removed by only checking packet taps if a socket is open on the device (vrf_dev->ptype_all list is not empty). Packet sockets bound to any device will still get a copy of the packet via the real ingress or egress interface. The end result of this change is a decrease in the overhead of VRF for the default, baseline case (ie., no netfilter rules, no packet sockets, no qdisc) to ~3% for UDP which has a lookup per packet and < 1% overhead for connected sockets that leverage early demux and avoid FIB lookups. Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-03-20 11:19:44 -07:00
net: vrf: performance improvements for IPv4 The VRF driver allows users to implement device based features for an entire domain. For example, a qdisc or netfilter rules can be attached to a VRF device or tcpdump can be used to view packets for all devices in the L3 domain. The device-based features come with a performance penalty, most notably in the Tx path. The VRF driver uses the l3mdev_l3_out hook to switch the dst on an skb to its private dst. This allows the skb to traverse the xmit stack with the device set to the VRF device which in turn enables the netfilter and qdisc features. The VRF driver then performs the FIB lookup again and reinserts the packet. This patch avoids the redirect for IPv4 packets if a qdisc has not been attached to a VRF device which is the default config. In this case the netfilter hooks and network taps are directly traversed in the l3mdev_l3_out handler. If a qdisc is attached to a VRF device, then the redirect using the vrf dst is done. Additional overhead is removed by only checking packet taps if a socket is open on the device (vrf_dev->ptype_all list is not empty). Packet sockets bound to any device will still get a copy of the packet via the real ingress or egress interface. The end result of this change is a decrease in the overhead of VRF for the default, baseline case (ie., no netfilter rules, no packet sockets, no qdisc) to ~3% for UDP which has a lookup per packet and < 1% overhead for connected sockets that leverage early demux and avoid FIB lookups. Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-03-20 11:19:44 -07:00
net: vrf: ipv4 support for local traffic to local addresses Add support for locally originated traffic to VRF-local addresses. If destination device for an skb is the loopback or VRF device then set its dst to a local version of the VRF cached dst_entry and call netif_rx to insert the packet onto the rx queue - similar to what is done for loopback. This patch handles IPv4 support; follow on patch handles IPv6. With this patch, ping, tcp and udp packets to a local IPv4 address are successfully routed: $ ip addr show dev eth1 4: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master red state UP group default qlen 1000 link/ether 02:e0:f9:1c:b9:74 brd ff:ff:ff:ff:ff:ff inet 10.100.1.1/24 brd 10.100.1.255 scope global eth1 valid_lft forever preferred_lft forever inet6 2100:1::1/120 scope global valid_lft forever preferred_lft forever inet6 fe80::e0:f9ff:fe1c:b974/64 scope link valid_lft forever preferred_lft forever $ ping -c1 -I red 10.100.1.1 ping: Warning: source address might be selected on device other than red. PING 10.100.1.1 (10.100.1.1) from 10.100.1.1 red: 56(84) bytes of data. 64 bytes from 10.100.1.1: icmp_seq=1 ttl=64 time=0.057 ms This patch also enables use of IPv4 loopback address on the VRF device: $ ip addr add dev red 127.0.0.1/8 $ ping -c1 -I red 127.0.0.1 PING 127.0.0.1 (127.0.0.1) from 127.0.0.1 red: 56(84) bytes of data. 64 bytes from 127.0.0.1: icmp_seq=1 ttl=64 time=0.058 ms Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2016-06-06 20:50:39 -07:00
net: vrf: Fix crash when IPv6 is disabled at boot time Frank Kellermann reported a kernel crash with 4.5.0 when IPv6 is disabled at boot using the kernel option ipv6.disable=1. Using current net-next with the boot option: $ ip link add red type vrf table 1001 Generates: [12210.919584] BUG: unable to handle kernel NULL pointer dereference at 0000000000000748 [12210.921341] IP: [<ffffffff814b30e3>] fib6_get_table+0x2c/0x5a [12210.922537] PGD b79e3067 PUD bb32b067 PMD 0 [12210.923479] Oops: 0000 [#1] SMP [12210.924001] Modules linked in: ipvlan 8021q garp mrp stp llc [12210.925130] CPU: 3 PID: 1177 Comm: ip Not tainted 4.7.0-rc1+ #235 [12210.926168] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.7.5-20140531_083030-gandalf 04/01/2014 [12210.928065] task: ffff8800b9ac4640 ti: ffff8800bacac000 task.ti: ffff8800bacac000 [12210.929328] RIP: 0010:[<ffffffff814b30e3>] [<ffffffff814b30e3>] fib6_get_table+0x2c/0x5a [12210.930697] RSP: 0018:ffff8800bacaf888 EFLAGS: 00010202 [12210.931563] RAX: 0000000000000748 RBX: ffffffff81a9e280 RCX: ffff8800b9ac4e28 [12210.932688] RDX: 00000000000000e9 RSI: 0000000000000002 RDI: 0000000000000286 [12210.933820] RBP: ffff8800bacaf898 R08: ffff8800b9ac4df0 R09: 000000000052001b [12210.934941] R10: 00000000657c0000 R11: 000000000000c649 R12: 00000000000003e9 [12210.936032] R13: 00000000000003e9 R14: ffff8800bace7800 R15: ffff8800bb3ec000 [12210.937103] FS: 00007faa1766c700(0000) GS:ffff88013ac00000(0000) knlGS:0000000000000000 [12210.938321] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [12210.939166] CR2: 0000000000000748 CR3: 00000000b79d6000 CR4: 00000000000406e0 [12210.940278] Stack: [12210.940603] ffff8800bb3ec000 ffffffff81a9e280 ffff8800bacaf8c8 ffffffff814b3135 [12210.941818] ffff8800bb3ec000 ffffffff81a9e280 ffffffff81a9e280 ffff8800bace7800 [12210.943040] ffff8800bacaf8f0 ffffffff81397c88 ffff8800bb3ec000 ffffffff81a9e280 [12210.944288] Call Trace: [12210.944688] [<ffffffff814b3135>] fib6_new_table+0x24/0x8a [12210.945516] [<ffffffff81397c88>] vrf_dev_init+0xd4/0x162 [12210.946328] [<ffffffff814091e1>] register_netdevice+0x100/0x396 [12210.947209] [<ffffffff8139823d>] vrf_newlink+0x40/0xb3 [12210.948001] [<ffffffff814187f0>] rtnl_newlink+0x5d3/0x6d5 ... The problem above is due to the fact that the fib hash table is not allocated when IPv6 is disabled at boot. As for the VRF driver it should not do any IPv6 initializations if IPv6 is disabled, so it needs to know if IPv6 is disabled at boot. The disable parameter is private to the IPv6 module, so provide an accessor for modules to determine if IPv6 was disabled at boot time. Fixes: 35402e3136634 ("net: Add IPv6 support to VRF device") Signed-off-by: David Ahern <dsa@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2016-06-09 10:21:00 -07:00
net: Fix inconsistent teardown and release of private netdev state. Network devices can allocate reasources and private memory using netdev_ops->ndo_init(). However, the release of these resources can occur in one of two different places. Either netdev_ops->ndo_uninit() or netdev->destructor(). The decision of which operation frees the resources depends upon whether it is necessary for all netdev refs to be released before it is safe to perform the freeing. netdev_ops->ndo_uninit() presumably can occur right after the NETDEV_UNREGISTER notifier completes and the unicast and multicast address lists are flushed. netdev->destructor(), on the other hand, does not run until the netdev references all go away. Further complicating the situation is that netdev->destructor() almost universally does also a free_netdev(). This creates a problem for the logic in register_netdevice(). Because all callers of register_netdevice() manage the freeing of the netdev, and invoke free_netdev(dev) if register_netdevice() fails. If netdev_ops->ndo_init() succeeds, but something else fails inside of register_netdevice(), it does call ndo_ops->ndo_uninit(). But it is not able to invoke netdev->destructor(). This is because netdev->destructor() will do a free_netdev() and then the caller of register_netdevice() will do the same. However, this means that the resources that would normally be released by netdev->destructor() will not be. Over the years drivers have added local hacks to deal with this, by invoking their destructor parts by hand when register_netdevice() fails. Many drivers do not try to deal with this, and instead we have leaks. Let's close this hole by formalizing the distinction between what private things need to be freed up by netdev->destructor() and whether the driver needs unregister_netdevice() to perform the free_netdev(). netdev->priv_destructor() performs all actions to free up the private resources that used to be freed by netdev->destructor(), except for free_netdev(). netdev->needs_free_netdev is a boolean that indicates whether free_netdev() should be done at the end of unregister_netdevice(). Now, register_netdevice() can sanely release all resources after ndo_ops->ndo_init() succeeds, by invoking both ndo_ops->ndo_uninit() and netdev->priv_destructor(). And at the end of unregister_netdevice(), we invoke netdev->priv_destructor() and optionally call free_netdev(). Signed-off-by: David S. Miller <davem@davemloft.net>
2017-05-08 12:52:56 -04:00
vrf: fix bug_on triggered by rx when destroying a vrf When destroying a VRF device we cleanup the slaves in its ndo_uninit() function, but that causes packets to be switched (skb->dev == vrf being destroyed) even though we're pass the point where the VRF should be receiving any packets while it is being dismantled. This causes a BUG_ON to trigger if we have raw sockets (trace below). The reason is that the inetdev of the VRF has been destroyed but we're still sending packets up the stack with it, so let's free the slaves in the dellink callback as David Ahern suggested. Note that this fix doesn't prevent packets from going up when the VRF device is admin down. [ 35.631371] ------------[ cut here ]------------ [ 35.631603] kernel BUG at net/ipv4/fib_frontend.c:285! [ 35.631854] invalid opcode: 0000 [#1] SMP [ 35.631977] Modules linked in: [ 35.632081] CPU: 2 PID: 22 Comm: ksoftirqd/2 Not tainted 4.12.0-rc7+ #45 [ 35.632247] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.7.5-20140531_083030-gandalf 04/01/2014 [ 35.632477] task: ffff88005ad68000 task.stack: ffff88005ad64000 [ 35.632632] RIP: 0010:fib_compute_spec_dst+0xfc/0x1ee [ 35.632769] RSP: 0018:ffff88005ad67978 EFLAGS: 00010202 [ 35.632910] RAX: 0000000000000001 RBX: ffff880059a7f200 RCX: 0000000000000000 [ 35.633084] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffffffff82274af0 [ 35.633256] RBP: ffff88005ad679f8 R08: 000000000001ef70 R09: 0000000000000046 [ 35.633430] R10: ffff88005ad679f8 R11: ffff880037731cb0 R12: 0000000000000001 [ 35.633603] R13: ffff8800599e3000 R14: 0000000000000000 R15: ffff8800599cb852 [ 35.634114] FS: 0000000000000000(0000) GS:ffff88005d900000(0000) knlGS:0000000000000000 [ 35.634306] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 35.634456] CR2: 00007f3563227095 CR3: 000000000201d000 CR4: 00000000000406e0 [ 35.634632] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 35.634865] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [ 35.635055] Call Trace: [ 35.635271] ? __lock_acquire+0xf0d/0x1117 [ 35.635522] ipv4_pktinfo_prepare+0x82/0x151 [ 35.635831] raw_rcv_skb+0x17/0x3c [ 35.636062] raw_rcv+0xe5/0xf7 [ 35.636287] raw_local_deliver+0x169/0x1d9 [ 35.636534] ip_local_deliver_finish+0x87/0x1c4 [ 35.636820] ip_local_deliver+0x63/0x7f [ 35.637058] ip_rcv_finish+0x340/0x3a1 [ 35.637295] ip_rcv+0x314/0x34a [ 35.637525] __netif_receive_skb_core+0x49f/0x7c5 [ 35.637780] ? lock_acquire+0x13f/0x1d7 [ 35.638018] ? lock_acquire+0x15e/0x1d7 [ 35.638259] __netif_receive_skb+0x1e/0x94 [ 35.638502] ? __netif_receive_skb+0x1e/0x94 [ 35.638748] netif_receive_skb_internal+0x74/0x300 [ 35.639002] ? dev_gro_receive+0x2ed/0x411 [ 35.639246] ? lock_is_held_type+0xc4/0xd2 [ 35.639491] napi_gro_receive+0x105/0x1a0 [ 35.639736] receive_buf+0xc32/0xc74 [ 35.639965] ? detach_buf+0x67/0x153 [ 35.640201] ? virtqueue_get_buf_ctx+0x120/0x176 [ 35.640453] virtnet_poll+0x128/0x1c5 [ 35.640690] net_rx_action+0x103/0x343 [ 35.640932] __do_softirq+0x1c7/0x4b7 [ 35.641171] run_ksoftirqd+0x23/0x5c [ 35.641403] smpboot_thread_fn+0x24f/0x26d [ 35.641646] ? sort_range+0x22/0x22 [ 35.641878] kthread+0x129/0x131 [ 35.642104] ? __list_add+0x31/0x31 [ 35.642335] ? __list_add+0x31/0x31 [ 35.642568] ret_from_fork+0x2a/0x40 [ 35.642804] Code: 05 bd 87 a3 00 01 e8 1f ef 98 ff 4d 85 f6 48 c7 c7 f0 4a 27 82 41 0f 94 c4 31 c9 31 d2 41 0f b6 f4 e8 04 71 a1 ff 45 84 e4 74 02 <0f> 0b 0f b7 93 c4 00 00 00 4d 8b a5 80 05 00 00 48 03 93 d0 00 [ 35.644342] RIP: fib_compute_spec_dst+0xfc/0x1ee RSP: ffff88005ad67978 Fixes: 193125dbd8eb ("net: Introduce VRF device driver") Reported-by: Chris Cormier <chriscormier@cumulusnetworks.com> Signed-off-by: Nikolay Aleksandrov <nikolay@cumulusnetworks.com> Acked-by: David Ahern <dsahern@gmail.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-07-06 15:24:40 +03:00