David Ahern says:

====================
selftests: fib_tests: simplifications, verbosity and a race

Improve efficiency of fib_tests.sh and make the test result more verbose,
from this summary:
$ fib_tests.sh is failing in a VM:
    $ fib_tests.sh
    Running netdev unregister tests
    PASS: unicast route test
    PASS: multipath route test
    Running netdev down tests
    PASS: unicast route test
    PASS: multipath route test
    Running netdev carrier change tests
    PASS: local route carrier test
    FAIL: unicast route carrier test

where a single entry actually corresponds to many checks to a much more
verbse output that clarifies test cases:
$fib_tests.sh
Single path route carrier test
    ....
    Carrier down
        IPv4 fibmatch                                         [ OK ]
        IPv6 fibmatch                                         [ OK ]
        IPv4 linkdown flag set                                [FAIL]
        IPv6 linkdown flag set                                [FAIL]
    Second address added with carrier down
        IPv4 fibmatch                                         [ OK ]
        IPv6 fibmatch                                         [ OK ]
        IPv4 linkdown flag set                                [FAIL]
        IPv6 linkdown flag set                                [ OK ]

And then fix the race in changing carrier down on dummy device to checking
the corresponding routes.
====================
Signed-off-by: David S. Miller <davem@davemloft.net>

sleep for a second after setting carrier down to allow linkwatch
to propagate the change to the routing stack via netdev_state_change.
As it stands there is a race setting carrier down on the dummy
device and then checking the linkdown flag in the routes.
Signed-off-by: David Ahern <dsahern@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

Move setup and teardown of testns and dummy0 to helpers.
Signed-off-by: David Ahern <dsahern@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

fib_tests.sh is failing in a VM:
    $ fib_tests.sh
    Running netdev unregister tests
    PASS: unicast route test
    PASS: multipath route test
    Running netdev down tests
    PASS: unicast route test
    PASS: multipath route test
    Running netdev carrier change tests
    PASS: local route carrier test
    FAIL: unicast route carrier test

The last test corresponds to fib_carrier_unicast_test which 12 places
that could be failing. Be more verbose in the output so a failure is
easier to track down and separate test setup failures with set -e and
set +e pairs.

With the verbose logging it is easier to see which checks are failing:
    $fib_tests.sh
    Single path route carrier test
        ....
        Carrier down
            IPv4 fibmatch                                         [ OK ]
            IPv6 fibmatch                                         [ OK ]
            IPv4 linkdown flag set                                [FAIL]
            IPv6 linkdown flag set                                [FAIL]
        Second address added with carrier down
            IPv4 fibmatch                                         [ OK ]
            IPv6 fibmatch                                         [ OK ]
            IPv4 linkdown flag set                                [FAIL]
            IPv6 linkdown flag set                                [ OK ]
Signed-off-by: David Ahern <dsahern@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

'ip netns exec testns ip' is more efficiently handled using 'ip -netns';
runs the ip command after switching the namespace and avoids an exec.
Signed-off-by: David Ahern <dsahern@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

Do not export fib_multipath_hash or fib_select_path; both are only used
by core ipv4 code.
Signed-off-by: David Ahern <dsahern@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

If flow oif is set and it is not an l3mdev, then fib_select_path
can jump to the source address check.
Signed-off-by: David Ahern <dsahern@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

Xin Long says:

====================
sctp: rename sctp diag file and add file comments for it

This patchset is to remove the sctp_ prefix for sctp diag file,
and also to add the missing file comments for it.

v1->v2:
  split them into two patches as Marcelo suggested.
====================
Acked-by: Neil Horman <nhorman@tuxdriver.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

This patch is to add the missing file comments for sctp diag file.
Signed-off-by: Xin Long <lucien.xin@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

Remove 'sctp_' prefix for diag file, to keep consistent with other
files' names.
Signed-off-by: Xin Long <lucien.xin@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

Use NL_SET_ERR_MSG_MOD helper which adds the module name instead
of specifying the prefix each time.
Signed-off-by: Arkadi Sharshevsky <arkadis@mellanox.com>
Signed-off-by: Jiri Pirko <jiri@mellanox.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

Jiri Pirko says:

====================
mlxsw: SPAN cleanups

In patch one of this short series, a misplaced pointer star is moved to
the correct place.

In the second patch, we observe that if SPAN entries carry their
reference count anyway, it's redundant to also carry a "used" flag.

In the third patch, SPAN support code is moved to a separate module.
====================
Signed-off-by: David S. Miller <davem@davemloft.net>

For the upcoming work on SPAN, it makes sense to move the current code
to a module of its own. It already has a well-defined API boundary to
the mirror management (which is used from matchall and ACL code). A
couple more functions need to be exported for the functions that
spectrum.c needs to use for MTU handling and subsystem init/fini.
Signed-off-by: Petr Machata <petrm@mellanox.com>
Signed-off-by: Jiri Pirko <jiri@mellanox.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

The member ref_count already determines whether a given SPAN entry is
used, and is as easy to use as a dedicated boolean.
Signed-off-by: Petr Machata <petrm@mellanox.com>
Signed-off-by: Jiri Pirko <jiri@mellanox.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

Signed-off-by: Petr Machata <petrm@mellanox.com>
Signed-off-by: Jiri Pirko <jiri@mellanox.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

Jiri Pirko says:

====================
mlxsw: IPIP cleanups

In the first patch, a forgotten #include is added. Even though the code
compiles as-is, the include is necessary for modules that should include
spectrum_ipip.h.

The second patch corrects an assumption that IPv6 tunnels use struct
ip_tunnel_parm to store tunnel parameters.
====================
Signed-off-by: David S. Miller <davem@davemloft.net>

struct ip_tunnel_parm, where GRE and several other tunnel types hold
information, is IPv4-specific. The current router / ipip code in mlxsw
however uses it as if it were generic.

Make it clear that it's not. Rename many functions from _params_ to
_params4_. mlxsw_sp_ipip_parms_saddr() and _daddr() take a proto
argument to dispatch on it. Move the dispatch logic to
mlxsw_sp_ipip_netdev_saddr() and _daddr(), and replace with
single-protocol functions.

In struct mlxsw_sp_ipip_entry, move the "parms" field to a (for the time
being, singleton) union. Update users throughout.
Signed-off-by: Petr Machata <petrm@mellanox.com>
Signed-off-by: Jiri Pirko <jiri@mellanox.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

struct ip_tunnel_parm, which is used in spectrum_ipip.h, is defined in
if_tunnel.h. However, the former neglects to include the latter.
Signed-off-by: Petr Machata <petrm@mellanox.com>
Reviewed-by: Ido Schimmel <idosch@mellanox.com>
Signed-off-by: Jiri Pirko <jiri@mellanox.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

The comment stated that a thread was started, but
that is not the case.
Signed-off-by: Jake Moroni <mail@jakemoroni.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

Kirill Tkhai says:

====================
Replacing net_mutex with rw_semaphore

this is the third version of the patchset introducing net_sem
instead of net_mutex. The patchset adds net_sem in addition
to net_mutex and allows pernet_operations to be "async". This
flag means, the pernet_operations methods are safe to be
executed with any other pernet_operations (un)initializing
another net.

If there are only async pernet_operations in the system,
net_mutex is not used either for setup_net() or for cleanup_net().

The pernet_operations converted in this patchset allow
to create minimal .config to have network working, and
the changes improve the performance like you may see
below:

    %for i in {1..10000}; do unshare -n bash -c exit; done

    *before*
    real 1m40,377s
    user 0m9,672s
    sys 0m19,928s

    *after*
    real 0m17,007s
    user 0m5,311s
    sys 0m11,779

    (5.8 times faster)

In the future, when all pernet_operations become async,
we'll just remove this "async" field tree-wide.

All the new logic is concentrated in patches [1-5/32].
The rest of patches converts specific operations:
review, rationale of they can be converted, and setting
of async flag.

Kirill

v3: Improved patches descriptions. Added comment into [5/32].
Added [32/32] converting netlink_tap_net_ops (new pernet operations
introduced in 2018).

v2: Single patch -> patchset with rationale of every conversion
====================
Signed-off-by: David S. Miller <davem@davemloft.net>

These pernet_operations init just allocated net memory,
and they obviously can be executed in parallel in any
others.

v3: New
Signed-off-by: Kirill Tkhai <ktkhai@virtuozzo.com>
Acked-by: Andrei Vagin <avagin@virtuozzo.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

These pernet operations just create and destroy netlink
socket. The socket is pernet and else operations don't
touch it.
Signed-off-by: Kirill Tkhai <ktkhai@virtuozzo.com>
Acked-by: Andrei Vagin <avagin@virtuozzo.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

These pernet operations consist of exit() and exit_batch() methods.

default_device_exit() moves not-local and virtual devices to init_net.
There is nothing exciting, because this may happen in any time
on a working system, and rtnl_lock() and synchronize_net() protect
us from all cases of external dereference.

The same for default_device_exit_batch(). Similar unregisteration
may happen in any time on a system. Here several lists (like todo_list),
which are accessed under rtnl_lock(). After rtnl_unlock() and
netdev_run_todo() all the devices are flushed.
Signed-off-by: Kirill Tkhai <ktkhai@virtuozzo.com>
Acked-by: Andrei Vagin <avagin@virtuozzo.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

These pernet_operations have only init() method. It allocates
memory for net_device, calls register_netdev() and assigns
net::loopback_dev.

register_netdev() is allowed be used without additional locks,
as it's synchronized on rtnl_lock(). There are many examples
of using this functon directly from ioctl().

The only difference, compared to ioctl(), is that net is not
completely alive at this moment. But it looks like, there is
no way for parallel pernet_operations to dereference
the net_device, as the most of struct net_device lists,
where it's linked, are related to net, and the net is not liked.

The exceptions are net_device::unreg_list, close_list, todo_list,
used for unregistration, and ::link_watch_list, where net_device
may be linked to global lists.

Unregistration of loopback_dev obviously can't happen, when
loopback_net_init() is executing, as the net as alive. It occurs
in default_device_ops, which currently requires net_mutex,
and it behaves as a barrier at the moment. It will be considered
in next patch.

Speaking about link_watch_list, it seems, there is no way
for loopback_dev at time of registration to be linked in lweventlist
and be available for another pernet_operations.
Signed-off-by: Kirill Tkhai <ktkhai@virtuozzo.com>
Acked-by: Andrei Vagin <avagin@virtuozzo.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

These pernet_operations (un)register sysctl, which
are not touched by anybody else.

So, it's safe to make them async.
Signed-off-by: Kirill Tkhai <ktkhai@virtuozzo.com>
Acked-by: Andrei Vagin <avagin@virtuozzo.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

These pernet_operations create and destroy sysctl,
which are not touched by anybody else.
Signed-off-by: Kirill Tkhai <ktkhai@virtuozzo.com>
Acked-by: Andrei Vagin <avagin@virtuozzo.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

These pernet_operations just create and destroy /proc entry,
and another operations do not touch it.

Also, nobody else are interested in foreign net::packet::sklist.
Signed-off-by: Kirill Tkhai <ktkhai@virtuozzo.com>
Acked-by: Andrei Vagin <avagin@virtuozzo.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

These pernet_operations are just create and destroy
/proc and sysctl entries, and are not touched by
foreign pernet_operations.

So, we are able to make them async.
Signed-off-by: Kirill Tkhai <ktkhai@virtuozzo.com>
Acked-by: Andrei Vagin <avagin@virtuozzo.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

arp_net_ops just addr/removes /proc entry.

devinet_ops allocates and frees duplicate of init_net tables
and (un)registers sysctl entries.

fib_net_ops allocates and frees pernet tables, creates/destroys
netlink socket and (un)initializes /proc entries. Foreign
pernet_operations do not touch them.

ip_rt_proc_ops only modifies pernet /proc entries.

xfrm_net_ops creates/destroys /proc entries, allocates/frees
pernet statistics, hashes and tables, and (un)initializes
sysctl files. These are not touched by foreigh pernet_operations

xfrm4_net_ops allocates/frees private pernet memory, and
configures sysctls.

sysctl_route_ops creates/destroys sysctls.

rt_genid_ops only initializes fields of just allocated net.

ipv4_inetpeer_ops allocated/frees net private memory.

igmp_net_ops just creates/destroys /proc files and socket,
noone else interested in.

tcp_sk_ops seems to be safe, because tcp_sk_init() does not
depend on any other pernet_operations modifications. Iteration
over hash table in inet_twsk_purge() is made under RCU lock,
and it's safe to iterate the table this way. Removing from
the table happen from inet_twsk_deschedule_put(), but this
function is safe without any extern locks, as it's synchronized
inside itself. There are many examples, it's used in different
context. So, it's safe to leave tcp_sk_exit_batch() unlocked.

tcp_net_metrics_ops is synchronized on tcp_metrics_lock and safe.

udplite4_net_ops only creates/destroys pernet /proc file.

icmp_sk_ops creates percpu sockets, not touched by foreign
pernet_operations.

ipmr_net_ops creates/destroys pernet fib tables, (un)registers
fib rules and /proc files. This seem to be safe to execute
in parallel with foreign pernet_operations.

af_inet_ops just sets up default parameters of newly created net.

ipv4_mib_ops creates and destroys pernet percpu statistics.

raw_net_ops, tcp4_net_ops, udp4_net_ops, ping_v4_net_ops
and ip_proc_ops only create/destroy pernet /proc files.

ip4_frags_ops creates and destroys sysctl file.

So, it's safe to make the pernet_operations async.
Signed-off-by: Kirill Tkhai <ktkhai@virtuozzo.com>
Acked-by: Andrei Vagin <avagin@virtuozzo.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

These pernet_operations register and destroy sysctl
directory, and it's not interesting for foreign
pernet_operations.
Signed-off-by: Kirill Tkhai <ktkhai@virtuozzo.com>
Acked-by: Andrei Vagin <avagin@virtuozzo.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

These pernet_operations initialize and purge net::wext_nlevents
queue, and are not touched by foreign pernet_operations.

Mark them async.
Signed-off-by: Kirill Tkhai <ktkhai@virtuozzo.com>
Acked-by: Andrei Vagin <avagin@virtuozzo.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

This pernet_operations create and destroy net::genl_sock.
Foreign pernet_operations don't touch it.
Signed-off-by: Kirill Tkhai <ktkhai@virtuozzo.com>
Acked-by: Andrei Vagin <avagin@virtuozzo.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

psched_net_ops only creates and destroyes /proc entry,
and safe to be executed in parallel with any foreigh
pernet_operations.

tcf_action_net_ops initializes and destructs tcf_action_net::egdev_ht,
which is not touched by foreign pernet_operations.

So, make them async.
Signed-off-by: Kirill Tkhai <ktkhai@virtuozzo.com>
Acked-by: Andrei Vagin <avagin@virtuozzo.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

Both of them create and initialize lists, which are not touched
by another foreing pernet_operations.
Signed-off-by: Kirill Tkhai <ktkhai@virtuozzo.com>
Acked-by: Andrei Vagin <avagin@virtuozzo.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

There are:
1)dev_proc_ops and dev_mc_net_ops, which create and destroy
pernet proc file and not interesting for another net namespaces;
2)netdev_net_ops, which creates pernet hashes, which are not
touched by another pernet_operations.

So, make them async.
Signed-off-by: Kirill Tkhai <ktkhai@virtuozzo.com>
Acked-by: Andrei Vagin <avagin@virtuozzo.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

This patch starts to convert pernet_subsys, registered
from subsys initcalls.

It seems safe to be executed in parallel with others,
as it's only creates/destoyes proc entry,
which nobody else is not interested in.
Signed-off-by: Kirill Tkhai <ktkhai@virtuozzo.com>
Acked-by: Andrei Vagin <avagin@virtuozzo.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

uevent_net_init() and uevent_net_exit() create and
destroy netlink socket, and these actions serialized
in netlink code.

Parallel execution with other pernet_operations
makes the socket disappear earlier from uevent_sock_list
on ->exit. As userspace can't be interested in broadcast
messages of dying net, and, as I see, no one in kernel
listen them, we may safely make uevent_net_ops async.
Signed-off-by: Kirill Tkhai <ktkhai@virtuozzo.com>
Acked-by: Andrei Vagin <avagin@virtuozzo.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

This patch starts to convert pernet_subsys, registered
from postcore initcalls.

audit_net_init() creates netlink socket, while audit_net_exit()
destroys it. The rest of the pernet_list are not interested
in the socket, so we make audit_net_ops async.
Signed-off-by: Kirill Tkhai <ktkhai@virtuozzo.com>
Acked-by: Andrei Vagin <avagin@virtuozzo.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

rtnetlink_net_init() and rtnetlink_net_exit()
create and destroy netlink socket net::rtnl.

The socket is used to send rtnl notification via
rtnl_net_notifyid(). There is no a problem
to create and destroy it in parallel with other
pernet operations, as we link net in setup_net()
after the socket is created, and destroy
in cleanup_net() after net is unhashed from all
the lists and there is no RCU references on it.
Signed-off-by: Kirill Tkhai <ktkhai@virtuozzo.com>
Acked-by: Andrei Vagin <avagin@virtuozzo.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

The methods of netlink_net_ops create and destroy "netlink"
file, which are not interesting for foreigh pernet_operations.
So, netlink_net_ops may safely be made async.
Signed-off-by: Kirill Tkhai <ktkhai@virtuozzo.com>
Acked-by: Andrei Vagin <avagin@virtuozzo.com>
Signed-off-by: David S. Miller <davem@davemloft.net>