- 17 Mar, 2014 3 commits
-
-
Florian Westphal authored
With current match design every invocation of the connlimit_match function means we have to perform (number_of_conntracks % 256) lookups in the conntrack table [ to perform GC/delete stale entries ]. This is also the reason why ____nf_conntrack_find() in perf top has > 20% cpu time per core. This patch changes the storage to rbtree which cuts down the number of ct objects that need testing. When looking up a new tuple, we only test the connections of the host objects we visit while searching for the wanted host/network (or the leaf we need to insert at). The slot count is reduced to 32. Increasing slot count doesn't speed up things much because of rbtree nature. before patch (50kpps rx, 10kpps tx): + 20.95% ksoftirqd/0 [nf_conntrack] [k] ____nf_conntrack_find + 20.50% ksoftirqd/1 [nf_conntrack] [k] ____nf_conntrack_find + 20.27% ksoftirqd/2 [nf_conntrack] [k] ____nf_conntrack_find + 5.76% ksoftirqd/1 [nf_conntrack] [k] hash_conntrack_raw + 5.39% ksoftirqd/2 [nf_conntrack] [k] hash_conntrack_raw + 5.35% ksoftirqd/0 [nf_conntrack] [k] hash_conntrack_raw after (90kpps, 51kpps tx): + 17.24% swapper [nf_conntrack] [k] ____nf_conntrack_find + 6.60% ksoftirqd/2 [nf_conntrack] [k] ____nf_conntrack_find + 2.73% swapper [nf_conntrack] [k] hash_conntrack_raw + 2.36% swapper [xt_connlimit] [k] count_tree Obvious disadvantages to previous version are the increase in code complexity and the increased memory cost. Partially based on Eric Dumazets fq scheduler. Reviewed-by:
Jesper Dangaard Brouer <brouer@redhat.com> Signed-off-by:
Florian Westphal <fw@strlen.de> Signed-off-by:
Pablo Neira Ayuso <pablo@netfilter.org>
-
Florian Westphal authored
currently returns 1 if they're the same. Make it work like mem/strcmp so it can be used as rbtree search function. Reviewed-by:
-
Florian Westphal authored
connlimit currently suffers from spinlock contention, example for 4-core system with rps enabled: + 20.84% ksoftirqd/2 [kernel.kallsyms] [k] _raw_spin_lock_bh + 20.76% ksoftirqd/1 [kernel.kallsyms] [k] _raw_spin_lock_bh + 20.42% ksoftirqd/0 [kernel.kallsyms] [k] _raw_spin_lock_bh + 6.07% ksoftirqd/2 [nf_conntrack] [k] ____nf_conntrack_find + 6.07% ksoftirqd/1 [nf_conntrack] [k] ____nf_conntrack_find + 5.97% ksoftirqd/0 [nf_conntrack] [k] ____nf_conntrack_find + 2.47% ksoftirqd/2 [nf_conntrack] [k] hash_conntrack_raw + 2.45% ksoftirqd/0 [nf_conntrack] [k] hash_conntrack_raw + 2.44% ksoftirqd/1 [nf_conntrack] [k] hash_conntrack_raw May allow parallel lookup/insert/delete if the entry is hashed to another slot. With patch: + 20.95% ksoftirqd/0 [nf_conntrack] [k] ____nf_conntrack_find + 20.50% ksoftirqd/1 [nf_conntrack] [k] ____nf_conntrack_find + 20.27% ksoftirqd/2 [nf_conntrack] [k] ____nf_conntrack_find + 5.76% ksoftirqd/1 [nf_conntrack] [k] hash_conntrack_raw + 5.39% ksoftirqd/2 [nf_conntrack] [k] hash_conntrack_raw + 5.35% ksoftirqd/0 [nf_conntrack] [k] hash_conntrack_raw + 2.00% ksoftirqd/1 [kernel.kallsyms] [k] __rcu_read_unlock Improved rx processing rate from ~35kpps to ~50 kpps. Reviewed-by:
-
- 13 Mar, 2014 1 commit
-
-
Joe Perches authored
The use of __constant_<foo> has been unnecessary for quite awhile now. Make these uses consistent with the rest of the kernel. Signed-off-by:
Joe Perches <joe@perches.com> Acked-by:
Jozsef Kadlecsik <kadlec@blackhole.kfki.hu> Signed-off-by:
-
- 12 Mar, 2014 4 commits
-
-
Florian Westphal authored
We might allocate thousands of these (one object per connection). Use distinct kmem cache to permit simplte tracking on how many objects are currently used by the connlimit match via the sysfs. Reviewed-by:
-
Florian Westphal authored
Allows easier code-reuse in followup patches. Reviewed-by:
-
Florian Westphal authored
Instead of freeing the entry from our list and then adding it back again in the 'packet to closing connection' case just keep the matching entry around. Also drop the found_ct != NULL test as nf_ct_tuplehash_to_ctrack is just container_of(). Reviewed-by:
-
Florian Westphal authored
Simplifies followup patch that introduces separate locks for each of the hash slots. Reviewed-by:
-
- 08 Mar, 2014 5 commits
-
-
Patrick McHardy authored
The family in the NAT expression is basically completely useless since we have it available during runtime anyway. Nevertheless it is used to decide the NAT family, so at least validate it properly. As we don't support cross-family NAT, it needs to match the family of the table the expression exists in. Unfortunately we can't remove it completely since we need to dump it for userspace (*sigh*), so at least reduce the memory waste. Additionally clean up the module init function by removing useless temporary variables. Signed-off-by:
Patrick McHardy <kaber@trash.net> Signed-off-by:
-
Patrick McHardy authored
Since we have the context available during destruction again, we can remove the family from the private structure. Signed-off-by:
-
Patrick McHardy authored
Since we have the context available again, we can restore notifications for destruction of anonymous sets. Signed-off-by:
-
Patrick McHardy authored
In order to fix set destruction notifications and get rid of unnecessary members in private data structures, pass the context to expressions' destructor functions again. In order to do so, replace various members in the nft_rule_trans structure by the full context. Signed-off-by:
-
Patrick McHardy authored
The context argument logically comes first, and this is what every other function dealing with contexts does. Signed-off-by:
-
- 07 Mar, 2014 9 commits
-
-
Patrick McHardy authored
The hash set type is very broken and was never meant to be merged in this state. Missing RCU synchronization on element removal, leaking chain refcounts when used as a verdict map, races during lookups, a fixed table size are probably just some of the problems. Luckily it is currently never chosen by the kernel when the rbtree type is also available. Rewrite it to be usable. The new implementation supports automatic hash table resizing using RCU, based on Paul McKenney's and Josh Triplett's algorithm "Optimized Resizing For RCU-Protected Hash Tables" described in [1]. Resizing doesn't require a second list head in the elements, it works by chosing a hash function that remaps elements to a predictable set of buckets, only resizing by integral factors and - during expansion: linking new buckets to the old bucket that contains elements for any of the new buckets, thereby creating imprecise chains, then incrementally seperating the elements until the new buckets only contain elements that hash directly to them. - during shrinking: linking the hash chains of all old buckets that hash to the same new bucket to form a single chain. Expansion requires at most the number of elements in the longest hash chain grace periods, shrinking requires a single grace period. Due to the requirement of having hash chains/elements linked to multiple buckets during resizing, homemade single linked lists are used instead of the existing list helpers, that don't support this in a clean fashion. As a side effect, the amount of memory required per element is reduced by one pointer. Expansion is triggered when the load factors exceeds 75%, shrinking when the load factor goes below 30%. Both operations are allowed to fail and will be retried on the next insertion or removal if their respective conditions still hold. [1] http://dl.acm.org/citation.cfm?id=2002181.2002192Reviewed-by:
Josh Triplett <josh@joshtriplett.org> Reviewed-by:
Paul E. McKenney <paulmck@linux.vnet.ibm.com> Signed-off-by:
-
Jesper Dangaard Brouer authored
nf_conntrack_lock is a monolithic lock and suffers from huge contention on current generation servers (8 or more core/threads). Perf locking congestion is clear on base kernel: - 72.56% ksoftirqd/6 [kernel.kallsyms] [k] _raw_spin_lock_bh - _raw_spin_lock_bh + 25.33% init_conntrack + 24.86% nf_ct_delete_from_lists + 24.62% __nf_conntrack_confirm + 24.38% destroy_conntrack + 0.70% tcp_packet + 2.21% ksoftirqd/6 [kernel.kallsyms] [k] fib_table_lookup + 1.15% ksoftirqd/6 [kernel.kallsyms] [k] __slab_free + 0.77% ksoftirqd/6 [kernel.kallsyms] [k] inet_getpeer + 0.70% ksoftirqd/6 [nf_conntrack] [k] nf_ct_delete + 0.55% ksoftirqd/6 [ip_tables] [k] ipt_do_table This patch change conntrack locking and provides a huge performance improvement. SYN-flood attack tested on a 24-core E5-2695v2(ES) with 10Gbit/s ixgbe (with tool trafgen): Base kernel: 810.405 new conntrack/sec After patch: 2.233.876 new conntrack/sec Notice other floods attack (SYN+ACK or ACK) can easily be deflected using: # iptables -A INPUT -m state --state INVALID -j DROP # sysctl -w net/netfilter/nf_conntrack_tcp_loose=0 Use an array of hashed spinlocks to protect insertions/deletions of conntracks into the hash table. 1024 spinlocks seem to give good results, at minimal cost (4KB memory). Due to lockdep max depth, 1024 becomes 8 if CONFIG_LOCKDEP=y The hash resize is a bit tricky, because we need to take all locks in the array. A seqcount_t is used to synchronize the hash table users with the resizing process. Signed-off-by:Eric Dumazet <edumazet@google.com> Signed-off-by:
David S. Miller <davem@davemloft.net> Reviewed-by:
-
Jesper Dangaard Brouer authored
Netfilter expectations are protected with the same lock as conntrack entries (nf_conntrack_lock). This patch split out expectations locking to use it's own lock (nf_conntrack_expect_lock). Signed-off-by:
-
Jesper Dangaard Brouer authored
Preparation for disconnecting the nf_conntrack_lock from the expectations code. Once the nf_conntrack_lock is lifted, a race condition is exposed. The expectations master conntrack exp->master, can race with delete operations, as the refcnt increment happens too late in init_conntrack(). Race is against other CPUs invoking ->destroy() (destroy_conntrack()), or nf_ct_delete() (via timeout or early_drop()). Avoid this race in nf_ct_find_expectation() by using atomic_inc_not_zero(), and checking if nf_ct_is_dying() (path via nf_ct_delete()). Signed-off-by:
-
Jesper Dangaard Brouer authored
One spinlock per cpu to protect dying/unconfirmed/template special lists. (These lists are now per cpu, a bit like the untracked ct) Add a @cpu field to nf_conn, to make sure we hold the appropriate spinlock at removal time. Signed-off-by:
-
Jesper Dangaard Brouer authored
Changes while reading through the netfilter code. Added hint about how conntrack nf_conn refcnt is accessed. And renamed repl_hash to reply_hash for readability Signed-off-by:
-
git://git.kernel.org/pub/scm/linux/kernel/git/horms/ipvs-nextPablo Neira Ayuso authored
Via Simon Horman: ==================== * Whitespace cleanup spotted by checkpatch.pl from Tingwei Liu. * Section conflict cleanup, basically removal of one wrong __read_mostly, from Andi Kleen. ==================== Signed-off-by: -
Tingwei Liu authored
Add whitespace after operator and put open brace { on the previous line Cc: Tingwei Liu <liutingwei@hisense.com> Cc: lvs-devel@vger.kernel.org Signed-off-by:Tingwei Liu <tingw.liu@gmail.com> Signed-off-by:
Simon Horman <horms@verge.net.au>
-
Andi Kleen authored
const __read_mostly does not make any sense, because const data is already read-only. Remove the __read_mostly for the ipvs genl_ops. This avoids a LTO section conflict compile problem. Cc: Wensong Zhang <wensong@linux-vs.org> Cc: Simon Horman <horms@verge.net.au> Cc: Patrick McHardy <kaber@trash.net> Cc: lvs-devel@vger.kernel.org Signed-off-by:
Andi Kleen <ak@linux.intel.com> Signed-off-by:
-
- 06 Mar, 2014 8 commits
-
-
Josh Hunt authored
Adds a new property for hash set types, where if a set is created with the 'forceadd' option and the set becomes full the next addition to the set may succeed and evict a random entry from the set. To keep overhead low eviction is done very simply. It checks to see which bucket the new entry would be added. If the bucket's pos value is non-zero (meaning there's at least one entry in the bucket) it replaces the first entry in the bucket. If pos is zero, then it continues down the normal add process. This property is useful if you have a set for 'ban' lists where it may not matter if you release some entries from the set early. Signed-off-by:
Josh Hunt <johunt@akamai.com> Signed-off-by:
-
Ilia Mirkin authored
Commit 1785e8f4 ("netfiler: ipset: Add net namespace for ipset") moved the initialization print into net_init, which can get called a lot due to namespaces. Move it back into init, reduce to pr_info. Signed-off-by:
Ilia Mirkin <imirkin@alum.mit.edu> Signed-off-by:
-
Florian Westphal authored
commit 2dfb973c0dcc6d2211 (add markmask for hash:ip,mark data type) inserted IPSET_ATTR_MARKMASK in-between other enum values, i.e. changing values of all further attributes. This causes 'ipset list' segfault on existing kernels since ipset no longer finds IPSET_ATTR_MEMSIZE (it has a different value on kernel side). Jozsef points out it should be moved below IPSET_ATTR_MARK which works since there is some extra reserved space after that value. Signed-off-by:
-
Jozsef Kadlecsik authored
Signed-off-by: -
Vytas Dauksa authored
Introduce packet mark mask for hash:ip,mark data type. This allows to set mark bit filter for the ip set. Change-Id: Id8dd9ca7e64477c4f7b022a1d9c1a5b187f1c96e Signed-off-by: -
Vytas Dauksa authored
Introduce packet mark support with new ip,mark hash set. This includes userspace and kernelspace code, hash:ip,mark set tests and man page updates. The intended use of ip,mark set is similar to the ip:port type, but for protocols which don't use a predictable port number. Instead of port number it matches a firewall mark determined by a layer 7 filtering program like opendpi. As well as allowing or blocking traffic it will also be used for accounting packets and bytes sent for each protocol. Signed-off-by: -
Fengguang Wu authored
net/netfilter/ipset/ip_set_hash_netnet.c:115:8-9: WARNING: return of 0/1 in function 'hash_netnet4_data_list' with return type bool /c/kernel-tests/src/cocci/net/netfilter/ipset/ip_set_hash_netnet.c:338:8-9: WARNING: return of 0/1 in function 'hash_netnet6_data_list' with return type bool Return statements in functions returning bool should use true/false instead of 1/0. Generated by: coccinelle/misc/boolreturn.cocci Signed-off-by:
Fengguang Wu <fengguang.wu@intel.com> Signed-off-by:
-
Sergey Popovich authored
ipset(8) for list:set says: The match will try to find a matching entry in the sets and the target will try to add an entry to the first set to which it can be added. However real behavior is bit differ from described. Consider example: # ipset create test-1-v4 hash:ip family inet # ipset create test-1-v6 hash:ip family inet6 # ipset create test-1 list:set # ipset add test-1 test-1-v4 # ipset add test-1 test-1-v6 # iptables -A INPUT -p tcp --destination-port 25 -j SET --add-set test-1 src # ip6tables -A INPUT -p tcp --destination-port 25 -j SET --add-set test-1 src And then when iptables/ip6tables rule matches packet IPSET target tries to add src from packet to the list:set test-1 where first entry is test-1-v4 and the second one is test-1-v6. For IPv4, as it first entry in test-1 src added to test-1-v4 correctly, but for IPv6 src not added! Placing test-1-v6 to the first element of list:set makes behavior correct for IPv6, but brokes for IPv4. This is due to result, returned from ip_set_add() and ip_set_del() from net/netfilter/ipset/ip_set_core.c when set in list:set equires more parameters than given or address families do not match (which is this case). It seems wrong returning 0 from ip_set_add() and ip_set_del() in this case, as 0 should be returned only when an element successfuly added/deleted to/from the set, contrary to ip_set_test() which returns 0 when no entry exists and >0 when entry found in set. Signed-off-by:
Sergey Popovich <popovich_sergei@mail.ru> Signed-off-by:
-
- 27 Feb, 2014 1 commit
-
-
Pablo Neira Ayuso authored
This allows us to store user comment strings, but it could be also used to store any kind of information that the user application needs to link to the rule. Scratch 8 bits for the new ulen field that indicates the length the user data area. 4 bits from the handle (so it's 42 bits long, according to Patrick, it would last 139 years with 1000 new rules per second) and 4 bits from dlen (so the expression data area is 4K, which seems sufficient by now even considering the compatibility layer). Signed-off-by:
-
- 25 Feb, 2014 5 commits
-
-
Florian Westphal authored
Signed-off-by:
-
Patrick McHardy authored
Allow userspace to specify the queue number or the errno code for QUEUE and DROP verdicts. Signed-off-by:
-
Patrick McHardy authored
Signed-off-by:
-
Patrick McHardy authored
Add a lockdep_nfnl_is_held() function and a nfnl_dereference() macro for RCU dereferences protected by a NFNL subsystem mutex. Signed-off-by:
-
Patrick McHardy authored
The next patch will introduce a nfnl_dereference() macro that actually checks that the appropriate mutex is held and therefore needs a subsystem argument. Signed-off-by:
-
- 19 Feb, 2014 4 commits
-
-
Florian Westphal authored
This also adds NF_CT_LABELS_MAX_SIZE so it can be re-used as BUILD_BUG_ON in nft_ct. At this time, nft doesn't yet support writing to the label area; when this changes the label->words handling needs to be moved out of xt_connlabel.c into nf_conntrack_labels.c. Also removes a useless run-time check: words cannot grow beyond 4 (32 bit) or 2 (64bit) since xt_connlabel enforces a maximum of 128 labels. Signed-off-by:
-
Pablo Neira Ayuso authored
0-DAY kernel build testing backend reported: sparse warnings: (new ones prefixed by >>) >> >> net/netfilter/xt_ipcomp.c:63:26: sparse: restricted __be16 degrades to integer >> >> net/netfilter/xt_ipcomp.c:63:26: sparse: cast to restricted __be32 Fix this by using ntohs without shifting. Tested with: make C=1 CF=-D__CHECK_ENDIAN__ Signed-off-by:
Fan Du <fan.du@windriver.com> Signed-off-by:
-
stephen hemminger authored
This is C not shell script Signed-off-by:
Stephen Hemminger <stephen@networkplumber.org> Signed-off-by:
-
git://git.kernel.org/pub/scm/linux/kernel/git/davem/netDavid S. Miller authored
Conflicts: drivers/net/bonding/bond_3ad.h drivers/net/bonding/bond_main.c Two minor conflicts in bonding, both of which were overlapping changes. Signed-off-by:
-
