2005-04-16 15:20:36 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2005-05-05 16:16:16 -07:00
|
|
|
|
2005-04-16 15:20:36 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-11-23 20:12:59 -05:00
|
|
|
|
2005-12-27 02:43:12 -02:00
|
|
|
|
2005-04-16 15:20:36 -07:00
|
|
|
|
|
|
|
|
|
2006-11-27 11:10:57 -08:00
|
|
|
|
|
|
|
|
|
2005-04-16 15:20:36 -07:00
|
|
|
|
2006-10-19 17:23:57 -04:00
|
|
|
|
2005-04-16 15:20:36 -07:00
|
|
|
|
2006-11-27 11:10:57 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-12-10 09:48:31 +00:00
|
|
|
|
2006-11-27 11:10:57 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2005-04-16 15:20:36 -07:00
|
|
|
|
2009-11-08 10:17:05 +00:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2008-10-29 01:41:45 -07:00
|
|
|
|
2016-04-01 08:52:13 -07:00
|
|
|
|
2009-11-08 10:17:05 +00:00
|
|
|
|
2008-10-29 01:41:45 -07:00
|
|
|
|
|
|
|
|
|
2009-10-07 00:37:59 +00:00
|
|
|
|
2009-11-08 10:17:58 +00:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2008-10-29 01:41:45 -07:00
|
|
|
|
2009-10-07 00:37:59 +00:00
|
|
|
|
2009-11-08 10:17:58 +00:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2008-10-29 01:41:45 -07:00
|
|
|
|
|
|
|
|
|
2013-09-23 11:33:36 -07:00
|
|
|
|
2009-10-07 00:37:59 +00:00
|
|
|
|
2012-04-15 05:58:06 +00:00
|
|
|
|
2009-10-07 00:37:59 +00:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2009-11-08 10:17:58 +00:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2005-04-16 15:20:36 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-11-09 23:24:26 +00:00
|
|
|
|
2007-12-31 00:29:24 -08:00
|
|
|
|
|
|
|
|
|
2010-11-09 23:24:26 +00:00
|
|
|
|
2007-12-31 00:29:24 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2005-12-27 02:43:12 -02:00
|
|
|
|
2005-04-16 15:20:36 -07:00
|
|
|
|
2006-11-27 11:10:57 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-11-14 21:40:42 -08:00
|
|
|
|
2006-11-27 11:10:57 -08:00
|
|
|
|
2014-06-14 23:24:20 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-11-27 11:10:57 -08:00
|
|
|
|
|
|
|
|
|
2006-11-20 18:06:37 -08:00
|
|
|
|
2006-11-27 11:10:57 -08:00
|
|
|
|
2007-04-09 11:59:39 -07:00
|
|
|
|
2006-11-27 11:10:57 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-11-14 21:35:48 -08:00
|
|
|
|
2006-11-27 11:10:57 -08:00
|
|
|
|
2007-04-25 18:04:18 -07:00
|
|
|
|
|
|
|
|
|
2006-11-27 11:10:57 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-03-01 02:36:48 +00:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-06-04 17:19:48 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2016-04-05 12:41:15 -04:00
|
|
|
|
|
|
|
|
|
2016-05-31 15:22:41 -07:00
|
|
|
|
|
|
|
|
|
2016-04-05 12:41:15 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2016-04-05 08:22:51 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-08-22 13:34:44 -07:00
|
|
|
|
2016-04-05 08:22:51 -07:00
|
|
|
|
|
|
|
|
|
2014-08-22 13:34:44 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-11-27 11:10:57 -08:00
|
|
|
|
2016-02-10 11:50:35 -05:00
|
|
|
|
2006-11-27 11:10:57 -08:00
|
|
|
|
|
|
|
|
|
2016-02-10 11:50:35 -05:00
|
|
|
|
2006-11-27 11:10:57 -08:00
|
|
|
|
|
|
|
|
|
2013-09-23 11:33:36 -07:00
|
|
|
|
|
|
|
|
|
2006-11-27 11:10:57 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-09-23 11:33:36 -07:00
|
|
|
|
|
|
|
|
|
2006-11-27 11:10:57 -08:00
|
|
|
|
2015-02-24 09:17:31 -08:00
|
|
|
|
|
|
|
|
|
2014-07-01 21:32:39 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-02-24 09:17:31 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-07-01 21:32:39 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-11-27 11:10:57 -08:00
|
|
|
|
2016-11-15 16:37:53 +01:00
|
|
|
|
udp: implement memory accounting helpers
Avoid using the generic helpers.
Use the receive queue spin lock to protect the memory
accounting operation, both on enqueue and on dequeue.
On dequeue perform partial memory reclaiming, trying to
leave a quantum of forward allocated memory.
On enqueue use a custom helper, to allow some optimizations:
- use a plain spin_lock() variant instead of the slightly
costly spin_lock_irqsave(),
- avoid dst_force check, since the calling code has already
dropped the skb dst
- avoid orphaning the skb, since skb_steal_sock() already did
the work for us
The above needs custom memory reclaiming on shutdown, provided
by the udp_destruct_sock().
v5 -> v6:
- don't orphan the skb on enqueue
v4 -> v5:
- replace the mem_lock with the receive queue spin lock
- ensure that the bh is always allowed to enqueue at least
a skb, even if sk_rcvbuf is exceeded
v3 -> v4:
- reworked memory accunting, simplifying the schema
- provide an helper for both memory scheduling and enqueuing
v1 -> v2:
- use a udp specific destrctor to perform memory reclaiming
- remove a couple of helpers, unneeded after the above cleanup
- do not reclaim memory on dequeue if not under memory
pressure
- reworked the fwd accounting schema to avoid potential
integer overflow
Acked-by: Hannes Frederic Sowa <hannes@stressinduktion.org>
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
Acked-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2016-10-21 13:55:46 +02:00
|
|
|
|
|
|
|
|
|
2016-11-04 11:28:59 +01:00
|
|
|
|
2017-05-16 11:20:14 +02:00
|
|
|
|
|
|
|
|
|
2016-11-04 11:28:59 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
udp: implement memory accounting helpers
Avoid using the generic helpers.
Use the receive queue spin lock to protect the memory
accounting operation, both on enqueue and on dequeue.
On dequeue perform partial memory reclaiming, trying to
leave a quantum of forward allocated memory.
On enqueue use a custom helper, to allow some optimizations:
- use a plain spin_lock() variant instead of the slightly
costly spin_lock_irqsave(),
- avoid dst_force check, since the calling code has already
dropped the skb dst
- avoid orphaning the skb, since skb_steal_sock() already did
the work for us
The above needs custom memory reclaiming on shutdown, provided
by the udp_destruct_sock().
v5 -> v6:
- don't orphan the skb on enqueue
v4 -> v5:
- replace the mem_lock with the receive queue spin lock
- ensure that the bh is always allowed to enqueue at least
a skb, even if sk_rcvbuf is exceeded
v3 -> v4:
- reworked memory accunting, simplifying the schema
- provide an helper for both memory scheduling and enqueuing
v1 -> v2:
- use a udp specific destrctor to perform memory reclaiming
- remove a couple of helpers, unneeded after the above cleanup
- do not reclaim memory on dequeue if not under memory
pressure
- reworked the fwd accounting schema to avoid potential
integer overflow
Acked-by: Hannes Frederic Sowa <hannes@stressinduktion.org>
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
Acked-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2016-10-21 13:55:46 +02:00
|
|
|
|
2017-09-28 15:51:36 +02:00
|
|
|
|
2017-08-25 14:31:01 +02:00
|
|
|
|
2013-09-23 11:33:36 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2016-08-23 21:06:33 -07:00
|
|
|
|
2015-03-02 15:37:48 +08:00
|
|
|
|
2013-09-23 11:33:36 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
udp: implement memory accounting helpers
Avoid using the generic helpers.
Use the receive queue spin lock to protect the memory
accounting operation, both on enqueue and on dequeue.
On dequeue perform partial memory reclaiming, trying to
leave a quantum of forward allocated memory.
On enqueue use a custom helper, to allow some optimizations:
- use a plain spin_lock() variant instead of the slightly
costly spin_lock_irqsave(),
- avoid dst_force check, since the calling code has already
dropped the skb dst
- avoid orphaning the skb, since skb_steal_sock() already did
the work for us
The above needs custom memory reclaiming on shutdown, provided
by the udp_destruct_sock().
v5 -> v6:
- don't orphan the skb on enqueue
v4 -> v5:
- replace the mem_lock with the receive queue spin lock
- ensure that the bh is always allowed to enqueue at least
a skb, even if sk_rcvbuf is exceeded
v3 -> v4:
- reworked memory accunting, simplifying the schema
- provide an helper for both memory scheduling and enqueuing
v1 -> v2:
- use a udp specific destrctor to perform memory reclaiming
- remove a couple of helpers, unneeded after the above cleanup
- do not reclaim memory on dequeue if not under memory
pressure
- reworked the fwd accounting schema to avoid potential
integer overflow
Acked-by: Hannes Frederic Sowa <hannes@stressinduktion.org>
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
Acked-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2016-10-21 13:55:46 +02:00
|
|
|
|
2016-10-20 09:39:40 -07:00
|
|
|
|
2013-09-23 11:33:36 -07:00
|
|
|
|
2017-07-03 00:01:49 -04:00
|
|
|
|
2013-09-23 11:33:36 -07:00
|
|
|
|
2014-09-29 20:22:29 -07:00
|
|
|
|
|
|
|
|
|
2013-09-23 11:33:36 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2017-08-07 08:44:16 -07:00
|
|
|
|
2016-01-04 17:41:47 -05:00
|
|
|
|
2016-04-05 08:22:50 -07:00
|
|
|
|
|
|
|
|
|
2013-09-23 11:33:36 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2017-08-07 08:44:20 -07:00
|
|
|
|
2016-01-04 17:41:47 -05:00
|
|
|
|
2016-04-05 08:22:50 -07:00
|
|
|
|
|
|
|
|
|
2008-10-01 07:48:10 -07:00
|
|
|
|
2017-06-26 19:01:50 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2017-07-25 17:57:47 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2017-06-26 19:01:50 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2017-07-25 17:57:47 +02:00
|
|
|
|
2017-06-26 19:01:50 +02:00
|
|
|
|
|
|
|
|
|
2017-07-25 17:57:47 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2017-06-26 19:01:50 +02:00
|
|
|
|
|
|
|
|
|
2017-07-25 17:57:47 +02:00
|
|
|
|
2017-06-26 19:01:50 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2017-07-25 17:57:47 +02:00
|
|
|
|
2017-06-26 19:01:50 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2017-07-25 17:57:47 +02:00
|
|
|
|
2017-06-26 19:01:50 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2017-08-14 21:31:38 +02:00
|
|
|
|
2017-06-26 19:01:50 +02:00
|
|
|
|
2017-08-14 21:31:38 +02:00
|
|
|
|
|
|
|
|
|
2017-06-26 19:01:50 +02:00
|
|
|
|
|
|
|
|
|
2017-08-14 21:31:38 +02:00
|
|
|
|
2017-06-26 19:01:50 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-11-27 11:10:57 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
net: snmp: kill various STATS_USER() helpers
In the old days (before linux-3.0), SNMP counters were duplicated,
one for user context, and one for BH context.
After commit 8f0ea0fe3a03 ("snmp: reduce percpu needs by 50%")
we have a single copy, and what really matters is preemption being
enabled or disabled, since we use this_cpu_inc() or __this_cpu_inc()
respectively.
We therefore kill SNMP_INC_STATS_USER(), SNMP_ADD_STATS_USER(),
NET_INC_STATS_USER(), NET_ADD_STATS_USER(), SCTP_INC_STATS_USER(),
SNMP_INC_STATS64_USER(), SNMP_ADD_STATS64_USER(), TCP_ADD_STATS_USER(),
UDP_INC_STATS_USER(), UDP6_INC_STATS_USER(), and XFRM_INC_STATS_USER()
Following patches will rename __BH helpers to make clear their
usage is not tied to BH being disabled.
Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2016-04-27 16:44:27 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2016-04-27 16:44:30 -07:00
|
|
|
|
2016-04-27 16:44:43 -07:00
|
|
|
|
|
|
|
|
|
2005-04-16 15:20:36 -07:00
|
|
|
|
2016-04-27 16:44:30 -07:00
|
|
|
|
2016-04-27 16:44:43 -07:00
|
|
|
|
|
|
|
|
|
2008-10-07 14:49:36 -07:00
|
|
|
|
net: snmp: kill various STATS_USER() helpers
In the old days (before linux-3.0), SNMP counters were duplicated,
one for user context, and one for BH context.
After commit 8f0ea0fe3a03 ("snmp: reduce percpu needs by 50%")
we have a single copy, and what really matters is preemption being
enabled or disabled, since we use this_cpu_inc() or __this_cpu_inc()
respectively.
We therefore kill SNMP_INC_STATS_USER(), SNMP_ADD_STATS_USER(),
NET_INC_STATS_USER(), NET_ADD_STATS_USER(), SCTP_INC_STATS_USER(),
SNMP_INC_STATS64_USER(), SNMP_ADD_STATS64_USER(), TCP_ADD_STATS_USER(),
UDP_INC_STATS_USER(), UDP6_INC_STATS_USER(), and XFRM_INC_STATS_USER()
Following patches will rename __BH helpers to make clear their
usage is not tied to BH being disabled.
Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2016-04-27 16:44:27 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2008-10-07 14:49:36 -07:00
|
|
|
|
2007-12-11 11:30:32 -08:00
|
|
|
|
2011-12-10 09:48:31 +00:00
|
|
|
|
2016-04-27 16:44:30 -07:00
|
|
|
|
2013-11-05 14:13:47 -08:00
|
|
|
|
|
|
|
|
|
2016-04-27 16:44:30 -07:00
|
|
|
|
2013-11-05 14:13:47 -08:00
|
|
|
|
2016-04-27 16:44:30 -07:00
|
|
|
|
2013-11-05 14:13:47 -08:00
|
|
|
|
2007-12-11 11:30:32 -08:00
|
|
|
|
2016-04-27 16:44:30 -07:00
|
|
|
|
2007-12-11 11:30:32 -08:00
|
|
|
|
|
|
|
|
|
2005-04-16 15:20:36 -07:00
|
|
|
|
2011-10-30 06:46:30 +00:00
|
|
|
|
|
|
|
|
|
2005-04-16 15:20:36 -07:00
|
|
|
|
2011-10-30 06:46:30 +00:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2005-04-16 15:20:36 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2008-03-28 18:23:33 -07:00
|
|
|
|
2005-04-16 15:20:36 -07:00
|
|
|
|
|
|
|
|
|
2008-10-29 01:41:45 -07:00
|
|
|
|
2005-04-16 15:20:36 -07:00
|
|
|
|
|
|
|
|
|
2005-08-16 02:18:02 -03:00
|
|
|
|
2013-09-23 11:33:36 -07:00
|
|
|
|
|
|
|
|
|
2005-08-16 02:18:02 -03:00
|
|
|
|
2013-09-23 11:33:36 -07:00
|
|
|
|
|
|
|
|
|
2005-08-16 02:18:02 -03:00
|
|
|
|
2007-12-31 00:29:24 -08:00
|
|
|
|
2013-09-23 11:33:36 -07:00
|
|
|
|
2013-06-08 12:56:03 +02:00
|
|
|
|
2013-09-23 11:33:36 -07:00
|
|
|
|
2009-07-09 08:09:47 +00:00
|
|
|
|
2013-09-23 11:33:36 -07:00
|
|
|
|
2012-04-27 08:24:08 +00:00
|
|
|
|
2013-09-23 11:33:36 -07:00
|
|
|
|
2012-04-27 08:24:08 +00:00
|
|
|
|
2017-03-23 13:34:16 -06:00
|
|
|
|
2005-04-16 15:20:36 -07:00
|
|
|
|