1. 04 Aug, 2017 8 commits
    • Willem de Bruijn's avatar
      tcp: enable MSG_ZEROCOPY · f214f915
      Willem de Bruijn authored
      Enable support for MSG_ZEROCOPY to the TCP stack. TSO and GSO are
      both supported. Only data sent to remote destinations is sent without
      copying. Packets looped onto a local destination have their payload
      copied to avoid unbounded latency.
      
      Tested:
        A 10x TCP_STREAM between two hosts showed a reduction in netserver
        process cycles by up to 70%, depending on packet size. Systemwide,
        savings are of course much less pronounced, at up to 20% best case.
      
        msg_zerocopy.sh 4 tcp:
      
        without zerocopy
          tx=121792 (7600 MB) txc=0 zc=n
          rx=60458 (7600 MB)
      
        with zerocopy
          tx=286257 (17863 MB) txc=286257 zc=y
          rx=140022 (17863 MB)
      
        This test opens a pair of sockets over veth, one one calls send with
        64KB and optionally MSG_ZEROCOPY and on the other reads the initial
        bytes. The receiver truncates, so this is strictly an upper bound on
        what is achievable. It is more representative of sending data out of
        a physical NIC (when payload is not touched, either).
      Signed-off-by: default avatarWillem de Bruijn <willemb@google.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      f214f915
    • Willem de Bruijn's avatar
      sock: ulimit on MSG_ZEROCOPY pages · a91dbff5
      Willem de Bruijn authored
      Bound the number of pages that a user may pin.
      
      Follow the lead of perf tools to maintain a per-user bound on memory
      locked pages commit 789f90fc ("perf_counter: per user mlock gift")
      Signed-off-by: default avatarWillem de Bruijn <willemb@google.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      a91dbff5
    • Willem de Bruijn's avatar
      sock: MSG_ZEROCOPY notification coalescing · 4ab6c99d
      Willem de Bruijn authored
      In the simple case, each sendmsg() call generates data and eventually
      a zerocopy ready notification N, where N indicates the Nth successful
      invocation of sendmsg() with the MSG_ZEROCOPY flag on this socket.
      
      TCP and corked sockets can cause send() calls to append new data to an
      existing sk_buff and, thus, ubuf_info. In that case the notification
      must hold a range. odify ubuf_info to store a inclusive range [N..N+m]
      and add skb_zerocopy_realloc() to optionally extend an existing range.
      
      Also coalesce notifications in this common case: if a notification
      [1, 1] is about to be queued while [0, 0] is the queue tail, just modify
      the head of the queue to read [0, 1].
      
      Coalescing is limited to a few TSO frames worth of data to bound
      notification latency.
      Signed-off-by: default avatarWillem de Bruijn <willemb@google.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      4ab6c99d
    • Willem de Bruijn's avatar
      sock: enable MSG_ZEROCOPY · 1f8b977a
      Willem de Bruijn authored
      Prepare the datapath for refcounted ubuf_info. Clone ubuf_info with
      skb_zerocopy_clone() wherever needed due to skb split, merge, resize
      or clone.
      
      Split skb_orphan_frags into two variants. The split, merge, .. paths
      support reference counted zerocopy buffers, so do not do a deep copy.
      Add skb_orphan_frags_rx for paths that may loop packets to receive
      sockets. That is not allowed, as it may cause unbounded latency.
      Deep copy all zerocopy copy buffers, ref-counted or not, in this path.
      
      The exact locations to modify were chosen by exhaustively searching
      through all code that might modify skb_frag references and/or the
      the SKBTX_DEV_ZEROCOPY tx_flags bit.
      
      The changes err on the safe side, in two ways.
      
      (1) legacy ubuf_info paths virtio and tap are not modified. They keep
          a 1:1 ubuf_info to sk_buff relationship. Calls to skb_orphan_frags
          still call skb_copy_ubufs and thus copy frags in this case.
      
      (2) not all copies deep in the stack are addressed yet. skb_shift,
          skb_split and skb_try_coalesce can be refined to avoid copying.
          These are not in the hot path and this patch is hairy enough as
          is, so that is left for future refinement.
      Signed-off-by: default avatarWillem de Bruijn <willemb@google.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      1f8b977a
    • Willem de Bruijn's avatar
      sock: add SOCK_ZEROCOPY sockopt · 76851d12
      Willem de Bruijn authored
      The send call ignores unknown flags. Legacy applications may already
      unwittingly pass MSG_ZEROCOPY. Continue to ignore this flag unless a
      socket opts in to zerocopy.
      
      Introduce socket option SO_ZEROCOPY to enable MSG_ZEROCOPY processing.
      Processes can also query this socket option to detect kernel support
      for the feature. Older kernels will return ENOPROTOOPT.
      Signed-off-by: default avatarWillem de Bruijn <willemb@google.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      76851d12
    • Willem de Bruijn's avatar
      sock: add MSG_ZEROCOPY · 52267790
      Willem de Bruijn authored
      The kernel supports zerocopy sendmsg in virtio and tap. Expand the
      infrastructure to support other socket types. Introduce a completion
      notification channel over the socket error queue. Notifications are
      returned with ee_origin SO_EE_ORIGIN_ZEROCOPY. ee_errno is 0 to avoid
      blocking the send/recv path on receiving notifications.
      
      Add reference counting, to support the skb split, merge, resize and
      clone operations possible with SOCK_STREAM and other socket types.
      
      The patch does not yet modify any datapaths.
      Signed-off-by: default avatarWillem de Bruijn <willemb@google.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      52267790
    • Willem de Bruijn's avatar
      sock: skb_copy_ubufs support for compound pages · 3ece7826
      Willem de Bruijn authored
      Refine skb_copy_ubufs to support compound pages. With upcoming TCP
      zerocopy sendmsg, such fragments may appear.
      
      The existing code replaces each page one for one. Splitting each
      compound page into an independent number of regular pages can result
      in exceeding limit MAX_SKB_FRAGS if data is not exactly page aligned.
      
      Instead, fill all destination pages but the last to PAGE_SIZE.
      Split the existing alloc + copy loop into separate stages:
      1. compute bytelength and minimum number of pages to store this.
      2. allocate
      3. copy, filling each page except the last to PAGE_SIZE bytes
      4. update skb frag array
      Signed-off-by: default avatarWillem de Bruijn <willemb@google.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      3ece7826
    • Willem de Bruijn's avatar
      sock: allocate skbs from optmem · 98ba0bd5
      Willem de Bruijn authored
      Add sock_omalloc and sock_ofree to be able to allocate control skbs,
      for instance for looping errors onto sk_error_queue.
      
      The transmit budget (sk_wmem_alloc) is involved in transmit skb
      shaping, most notably in TCP Small Queues. Using this budget for
      control packets would impact transmission.
      Signed-off-by: default avatarWillem de Bruijn <willemb@google.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      98ba0bd5
  2. 03 Aug, 2017 32 commits
    • David S. Miller's avatar
      Merge branch 'mlxsw-Support-for-IPv6-UC-router' · 84b7187c
      David S. Miller authored
      Jiri Pirko says:
      
      ====================
      mlxsw: Support for IPv6 UC router
      
      Ido says:
      
      This set adds support for IPv6 unicast routes offload. The first four
      patches make the FIB notification chain generic so that it could be used
      by address families other than IPv4. This is done by having each address
      family register its callbacks with the common code, so that its FIB tables
      and rules could be dumped upon registration to the chain, while ensuring
      the integrity of the dump. The exact mechanics are explained in detail in
      the first patch.
      
      The next six patches build upon this work and add the necessary callbacks
      in IPv6 code. This allows listeners of the chain to receive notifications
      about IPv6 routes addition, deletion and replacement as well as FIB rules
      notifications.
      
      Unlike user space notifications for IPv6 multipath routes, the FIB
      notification chain notifies these on a per-nexthop basis. This allows
      us to keep the common code lean and is also unnecessary, as notifications
      are serialized by each table's lock whereas applications maintaining
      netlink caches may suffer from concurrent dumps and deletions / additions
      of routes.
      
      The next five patches audit the different code paths reading the route's
      reference count (rt6i_ref) and remove assumptions regarding its meaning.
      This is needed since non-FIB users need to be able to hold a reference on
      the route and a non-zero reference count no longer means the route is in
      the FIB.
      
      The last six patches enable the mlxsw driver to offload IPv6 unicast
      routes to the Spectrum ASIC. Without resorting to ACLs, lookup is done
      solely based on the destination IP, so the abort mechanism is invoked
      upon the addition of source-specific routes.
      
      Follow-up patch sets will increase the scale of gatewayed routes by
      consolidating identical nexthop groups to one adjacency entry in the
      device's adjacency table (as in IPv4), as well as add support for
      NH_{ADD,DEL} events which enable support for the
      'ignore_routes_with_linkdown' sysctl.
      
      Changes in v2:
      * Provide offload indication for individual nexthops (David Ahern).
      * Use existing route reference count instead of adding another one.
        This resulted in several new patches to remove assumptions regarding
        current semantics of the existing reference count (David Ahern).
      * Add helpers to allow non-FIB users to take a reference on route.
      * Remove use of tb6_lock in mlxsw (David Ahern).
      * Add IPv6 dependency to mlxsw.
      ====================
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      84b7187c
    • Ido Schimmel's avatar
      mlxsw: spectrum_router: Don't ignore IPv6 notifications · 65e65ec1
      Ido Schimmel authored
      We now have all the necessary IPv6 infrastructure in place, so stop
      ignoring these notifications.
      Signed-off-by: default avatarIdo Schimmel <idosch@mellanox.com>
      Signed-off-by: default avatarJiri Pirko <jiri@mellanox.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      65e65ec1
    • Ido Schimmel's avatar
      mlxsw: spectrum_router: Abort on source-specific routes · f36f5ac6
      Ido Schimmel authored
      Without resorting to ACLs, the device performs route lookup solely based
      on the destination IP address.
      
      In case source-specific routing is needed, an error is returned and the
      abort mechanism is activated, thus allowing the kernel to take over
      forwarding decisions.
      
      Instead of aborting, we can trap specific destination prefixes where
      source-specific routes are present, but this will result in a lot more
      code that is unlikely to ever be used.
      Signed-off-by: default avatarIdo Schimmel <idosch@mellanox.com>
      Signed-off-by: default avatarJiri Pirko <jiri@mellanox.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      f36f5ac6
    • Ido Schimmel's avatar
      mlxsw: spectrum_router: Add support for route replace · 0a7fd1ac
      Ido Schimmel authored
      In case we got a replace event, then the replaced route must exist. If
      the route isn't capable of multipath, then replace first matching
      non-multipath capable route.
      
      If the route is capable of multipath and matching multipath capable
      route is found, then replace it. Otherwise, replace first matching
      non-multipath capable route.
      
      The new route is inserted before the replaced one. In case the replaced
      route is currently offloaded, then it's overwritten in the device's table
      by the new route and later deleted, thus not impacting routed traffic.
      Signed-off-by: default avatarIdo Schimmel <idosch@mellanox.com>
      Signed-off-by: default avatarJiri Pirko <jiri@mellanox.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      0a7fd1ac
    • Ido Schimmel's avatar
      mlxsw: spectrum_router: Add support for IPv6 routes addition / deletion · 428b851f
      Ido Schimmel authored
      Allow directly connected and remote unicast IPv6 routes to be programmed
      to the device's tables.
      
      As with IPv4, identical routes - sharing the same destination prefix -
      are ordered in a FIB node according to their table ID and then the
      metric. While the kernel doesn't share the same trie for the local and
      main table, this does happen in the device, so ordering according to
      table ID is needed.
      
      Since individual nexthops can be added and deleted in IPv6, each FIB
      entry stores a linked list of the rt6_info structs it represents. Upon
      the addition or deletion of a nexthop, a new nexthop group is allocated
      according to the new configuration and the old one is destroyed.
      Identical groups aren't currently consolidated, but will be in a
      follow-up patchset.
      Signed-off-by: default avatarIdo Schimmel <idosch@mellanox.com>
      Signed-off-by: default avatarJiri Pirko <jiri@mellanox.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      428b851f
    • Ido Schimmel's avatar
      mlxsw: spectrum_router: Sanitize IPv6 FIB rules · 583419fd
      Ido Schimmel authored
      We only allow FIB offload in the presence of default rules or an l3mdev
      rule. In a similar fashion to IPv4 FIB rules, sanitize IPv6 rules.
      Signed-off-by: default avatarIdo Schimmel <idosch@mellanox.com>
      Signed-off-by: default avatarJiri Pirko <jiri@mellanox.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      583419fd
    • Ido Schimmel's avatar
      mlxsw: spectrum_router: Demultiplex FIB event based on family · 66a5763a
      Ido Schimmel authored
      The FIB notification block currently only handles IPv4 events, but we
      want to start handling IPv6 events soon, so lay the groundwork now.
      
      Do that by preparing the work item and process it according to the
      notified address family.
      Signed-off-by: default avatarIdo Schimmel <idosch@mellanox.com>
      Signed-off-by: default avatarJiri Pirko <jiri@mellanox.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      66a5763a
    • Ido Schimmel's avatar
      ipv6: fib: Add helpers to hold / drop a reference on rt6_info · a460aa83
      Ido Schimmel authored
      Similar to commit 1c677b3d ("ipv4: fib: Add fib_info_hold() helper")
      and commit b423cb10 ("ipv4: fib: Export free_fib_info()") add an
      helper to hold a reference on rt6_info and export rt6_release() to drop
      it and potentially release the route.
      
      This is needed so that drivers capable of FIB offload could hold a
      reference on the route before queueing it for offload and drop it after
      the route has been programmed to the device's tables.
      Signed-off-by: default avatarIdo Schimmel <idosch@mellanox.com>
      Signed-off-by: default avatarJiri Pirko <jiri@mellanox.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      a460aa83
    • Ido Schimmel's avatar
      ipv6: Regenerate host route according to node pointer upon interface up · fc882fcf
      Ido Schimmel authored
      When an interface is brought back up, the kernel tries to restore the
      host routes tied to its permanent addresses.
      
      However, if the host route was removed from the FIB, then we need to
      reinsert it. This is done by releasing the current dst and allocating a
      new, so as to not reuse a dst with obsolete values.
      
      Since this function is called under RTNL and using the same explanation
      from the previous patch, we can test if the route is in the FIB by
      checking its node pointer instead of its reference count.
      
      Tested using the following script and Andrey's reproducer mentioned
      in commit 8048ced9 ("net: ipv6: regenerate host route if moved to gc
      list") and linked below:
      
      $ ip link set dev lo up
      $ ip link add dummy1 type dummy
      $ ip -6 address add cafe::1/64 dev dummy1
      $ ip link set dev lo down	# cafe::1/128 is removed
      $ ip link set dev dummy1 up
      $ ip link set dev lo up
      
      The host route is correctly regenerated.
      Signed-off-by: default avatarIdo Schimmel <idosch@mellanox.com>
      Link: http://lkml.kernel.org/r/CAAeHK+zSe82vc5gCRgr_EoUwiALPnWVdWJBPwJZBpbxYz=kGJw@mail.gmail.comSigned-off-by: default avatarJiri Pirko <jiri@mellanox.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      fc882fcf
    • Ido Schimmel's avatar
      ipv6: Regenerate host route according to node pointer upon loopback up · 9217d8c2
      Ido Schimmel authored
      When the loopback device is brought back up we need to check if the host
      route attached to the address is still in the FIB and regenerate one in
      case it's not.
      
      Host routes using the loopback device are always inserted into and
      removed from the FIB under RTNL (under which this function is called),
      so we can test their node pointer instead of the reference count in
      order to check if the route is in the FIB or not.
      
      Tested using the following script from Nicolas mentioned in
      commit a220445f ("ipv6: correctly add local routes when lo goes up"):
      
      $ ip link add dummy1 type dummy
      $ ip link set dummy1 up
      $ ip link set lo down ; ip link set lo up
      
      The host route is correctly regenerated.
      Signed-off-by: default avatarIdo Schimmel <idosch@mellanox.com>
      Signed-off-by: default avatarJiri Pirko <jiri@mellanox.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      9217d8c2
    • Ido Schimmel's avatar
      ipv6: fib: Unlink replaced routes from their nodes · 7483cea7
      Ido Schimmel authored
      When a route is deleted its node pointer is set to NULL to indicate it's
      no longer linked to its node. Do the same for routes that are replaced.
      
      This will later allow us to test if a route is still in the FIB by
      checking its node pointer instead of its reference count.
      Signed-off-by: default avatarIdo Schimmel <idosch@mellanox.com>
      Signed-off-by: default avatarJiri Pirko <jiri@mellanox.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      7483cea7
    • Ido Schimmel's avatar
      ipv6: fib: Don't assume only nodes hold a reference on routes · c5b12410
      Ido Schimmel authored
      The code currently assumes that only FIB nodes can hold a reference on
      routes. Therefore, after fib6_purge_rt() has run and the route is no
      longer present in any intermediate nodes, it's assumed that its
      reference count would be 1 - taken by the node where it's currently
      stored.
      
      However, we're going to allow users other than the FIB to take a
      reference on a route, so this assumption is no longer valid and the
      BUG_ON() needs to be removed.
      
      Note that purging only takes place if the initial reference count is
      different than 1. I've left that check intact, as in the majority of
      systems (where routes are only referenced by the FIB), it does actually
      mean the route is present in intermediate nodes.
      Signed-off-by: default avatarIdo Schimmel <idosch@mellanox.com>
      Signed-off-by: default avatarJiri Pirko <jiri@mellanox.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      c5b12410
    • Ido Schimmel's avatar
      ipv6: fib: Add offload indication to routes · 61e4d01e
      Ido Schimmel authored
      Allow user space applications to see which routes are offloaded and
      which aren't by setting the RTNH_F_OFFLOAD flag when dumping them.
      
      To be consistent with IPv4, offload indication is provided on a
      per-nexthop basis.
      Signed-off-by: default avatarIdo Schimmel <idosch@mellanox.com>
      Signed-off-by: default avatarJiri Pirko <jiri@mellanox.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      61e4d01e
    • Ido Schimmel's avatar
      ipv6: fib: Dump tables during registration to FIB chain · e1ee0a5b
      Ido Schimmel authored
      Dump all the FIB tables in each net namespace upon registration to the
      FIB notification chain so that the callee will have a complete view of
      the tables.
      
      The integrity of the dump is ensured by a per-table sequence counter
      that is incremented (under write lock) whenever a route is added or
      deleted from the table.
      
      All the sequence counters are read (under each table's read lock) and
      summed, prior and after the dump. In case the counters differ, then the
      dump is either restarted or the registration fails.
      
      While it's possible for a table to be modified after its counter has
      been read, this isn't really a problem. In case it happened before it
      was read the second time, then the comparison at the end will fail. If
      it happened afterwards, then we're guaranteed to be notified about the
      change, as the notification block is registered prior to the second
      read.
      Signed-off-by: default avatarIdo Schimmel <idosch@mellanox.com>
      Signed-off-by: default avatarJiri Pirko <jiri@mellanox.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      e1ee0a5b
    • Ido Schimmel's avatar
      ipv6: fib_rules: Dump rules during registration to FIB chain · dcb18f76
      Ido Schimmel authored
      Allow users of the FIB notification chain to receive a complete view of
      the IPv6 FIB rules upon registration to the chain.
      
      The integrity of the dump is ensured by a per-family sequence counter
      that is incremented (under RTNL) whenever a rule is added or deleted.
      
      All the sequence counters are read (under RTNL) and summed, prior and
      after the dump. In case the counters differ, then the dump is either
      restarted or the registration fails.
      Signed-off-by: default avatarIdo Schimmel <idosch@mellanox.com>
      Signed-off-by: default avatarJiri Pirko <jiri@mellanox.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      dcb18f76
    • Ido Schimmel's avatar
      ipv6: fib: Add in-kernel notifications for route add / delete · df77fe4d
      Ido Schimmel authored
      As with IPv4, allow listeners of the FIB notification chain to receive
      notifications whenever a route is added, replaced or deleted. This is
      done by placing calls to the FIB notification chain in the two lowest
      level functions that end up performing these operations - namely,
      fib6_add_rt2node() and fib6_del_route().
      
      Unlike IPv4, APPEND notifications aren't sent as the kernel doesn't
      distinguish between "append" (NLM_F_CREATE|NLM_F_APPEND) and "prepend"
      (NLM_F_CREATE). If NLM_F_EXCL isn't set, duplicate routes are always
      added after the existing duplicate routes.
      Signed-off-by: default avatarIdo Schimmel <idosch@mellanox.com>
      Signed-off-by: default avatarJiri Pirko <jiri@mellanox.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      df77fe4d
    • Ido Schimmel's avatar
      ipv6: fib: Add FIB notifiers callbacks · 16ab6d7d
      Ido Schimmel authored
      We're about to add IPv6 FIB offload support, so implement the necessary
      callbacks in IPv6 code, which will later allow us to add routes and
      rules notifications.
      Signed-off-by: default avatarIdo Schimmel <idosch@mellanox.com>
      Signed-off-by: default avatarJiri Pirko <jiri@mellanox.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      16ab6d7d
    • Ido Schimmel's avatar
      ipv6: fib_rules: Check if rule is a default rule · e3ea9731
      Ido Schimmel authored
      As explained in commit 3c71006d ("ipv4: fib_rules: Check if rule is
      a default rule"), drivers supporting IPv6 FIB offload need to be able to
      sanitize the rules they don't support and potentially flush their
      tables.
      
      Add an IPv6 helper to check if a FIB rule is a default rule.
      Signed-off-by: default avatarIdo Schimmel <idosch@mellanox.com>
      Signed-off-by: default avatarJiri Pirko <jiri@mellanox.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      e3ea9731
    • Ido Schimmel's avatar
      net: fib_rules: Implement notification logic in core · 1b2a4440
      Ido Schimmel authored
      Unlike the routing tables, the FIB rules share a common core, so instead
      of replicating the same logic for each address family we can simply dump
      the rules and send notifications from the core itself.
      
      To protect the integrity of the dump, a rules-specific sequence counter
      is added for each address family and incremented whenever a rule is
      added or deleted (under RTNL).
      Signed-off-by: default avatarIdo Schimmel <idosch@mellanox.com>
      Signed-off-by: default avatarJiri Pirko <jiri@mellanox.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      1b2a4440
    • Ido Schimmel's avatar
      rocker: Ignore address families other than IPv4 · d371ac1e
      Ido Schimmel authored
      As in previous patch, ignore IPv6 notifications since the driver doesn't
      support these.
      Signed-off-by: default avatarIdo Schimmel <idosch@mellanox.com>
      Signed-off-by: default avatarJiri Pirko <jiri@mellanox.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      d371ac1e
    • Ido Schimmel's avatar
      mlxsw: spectrum_router: Ignore address families other than IPv4 · 64e5e825
      Ido Schimmel authored
      We're about to add IPv6 notifications in the FIB notification chain, but
      the driver currently doesn't support these, so ignore them.
      Signed-off-by: default avatarIdo Schimmel <idosch@mellanox.com>
      Signed-off-by: default avatarJiri Pirko <jiri@mellanox.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      64e5e825
    • Ido Schimmel's avatar
      net: core: Make the FIB notification chain generic · 04b1d4e5
      Ido Schimmel authored
      The FIB notification chain is currently soley used by IPv4 code.
      However, we're going to introduce IPv6 FIB offload support, which
      requires these notification as well.
      
      As explained in commit c3852ef7 ("ipv4: fib: Replay events when
      registering FIB notifier"), upon registration to the chain, the callee
      receives a full dump of the FIB tables and rules by traversing all the
      net namespaces. The integrity of the dump is ensured by a per-namespace
      sequence counter that is incremented whenever a change to the tables or
      rules occurs.
      
      In order to allow more address families to use the chain, each family is
      expected to register its fib_notifier_ops in its pernet init. These
      operations allow the common code to read the family's sequence counter
      as well as dump its tables and rules in the given net namespace.
      
      Additionally, a 'family' parameter is added to sent notifications, so
      that listeners could distinguish between the different families.
      
      Implement the common code that allows listeners to register to the chain
      and for address families to register their fib_notifier_ops. Subsequent
      patches will implement these operations in IPv6.
      
      In the future, ipmr and ip6mr will be extended to provide these
      notifications as well.
      Signed-off-by: default avatarIdo Schimmel <idosch@mellanox.com>
      Signed-off-by: default avatarJiri Pirko <jiri@mellanox.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      04b1d4e5
    • David S. Miller's avatar
      Merge branch 'mvpp2-add-TX-interrupts-support' · 80cf0b45
      David S. Miller authored
      Thomas Petazzoni says:
      
      ====================
      net: mvpp2: add TX interrupts support
      
      So far, the mvpp2 driver was using an hrtimer to handle TX
      completion. This patch series adds support for using TX interrupts
      (for each CPU) on PPv2.2, the variant of the IP used on Marvell Armada
      7K/8K.
      
      Dave: this version can be applied right away, it no longer depends on
      Antoine's patch series. Antoine series had some comments, so he will
      have to respin later on. Therefore, let's merge this smaller patch
      series first.
      
      Changes since v1:
      
       - Rebased on top of net-next, instead of on top of Antoine's series.
      
       - Removed the Device Tree patch, as it shouldn't go through the net
         tree.
      ====================
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      80cf0b45
    • Thomas Petazzoni's avatar
      dt-bindings: net: marvell-pp2: update interrupt-names with TX interrupts · 5d3ecb24
      Thomas Petazzoni authored
      The PPv2.2 unit has several interrupts used for TX completion
      notification. This commit updates the Device Tree binding describing
      this HW block to mention such interrupts.
      
      While at it, we update the example to use a recent Device Tree
      example, that uses interrupts going through the ICU, and not to the
      GIC directly.
      Signed-off-by: default avatarThomas Petazzoni <thomas.petazzoni@free-electrons.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      5d3ecb24
    • Thomas Petazzoni's avatar
      net: mvpp2: add support for TX interrupts and RX queue distribution modes · 213f428f
      Thomas Petazzoni authored
      This commit adds the support for two related features:
      
       - Support for TX interrupts, with one interrupt for each CPU
      
       - Support for different RX queue distribution modes
         MVPP2_QDIST_SINGLE_MODE where a single interrupt, shared by all
         CPUs, receives the RX events, and MVPP2_QDIST_MULTI_MODE, where the
         per-CPU interrupts used for TX events are also used for RX events.
      
      Since additional interrupts are needed, an update to the Device Tree
      binding is needed. However, backward compatibility is preserved with
      the old Device Tree binding, by gracefully degrading to the original
      behavior, with only one RX interrupt, and TX completion being handled
      by an hrtimer.
      Signed-off-by: default avatarThomas Petazzoni <thomas.petazzoni@free-electrons.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      213f428f
    • Thomas Petazzoni's avatar
      net: mvpp2: introduce queue_vector concept · 591f4cfa
      Thomas Petazzoni authored
      In preparation to the introduction of TX interrupts and improved RX
      queue distribution, this commit introduces the concept of "queue
      vector". A queue vector represents a number of RX and/or TX queues,
      and an associated NAPI instance and interrupt.
      
      This commit currently only creates a single queue_vector, so there are
      no changes in behavior, but it paves the way for additional
      queue_vector in the next commits.
      Signed-off-by: default avatarThomas Petazzoni <thomas.petazzoni@free-electrons.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      591f4cfa
    • Thomas Petazzoni's avatar
      net: mvpp2: move from cpu-centric naming to "software thread" naming · df089aa0
      Thomas Petazzoni authored
      The PPv2.2 IP has a concept of "software thread", with all registers
      of the PPv2.2 mapped 8 times, for concurrent accesses by 8 "software
      threads". In addition, interrupts on RX queues are associated to such
      "software thread".
      
      For most cases, we map a "software thread" to the more conventional
      concept of CPU, but we will soon have one exception: we will have a
      model where we have one TX interrupt per CPU (each using one software
      thread), and all RX events mapped to another software thread
      (associated to another interrupt).
      
      In preparation for this change, it makes sense to change the naming
      from MVPP2_MAX_CPUS to MVPP2_MAX_THREADS, and plan for 8 software
      threads instead of 4 currently.
      Signed-off-by: default avatarThomas Petazzoni <thomas.petazzoni@free-electrons.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      df089aa0
    • Thomas Petazzoni's avatar
      net: mvpp2: introduce per-port nrxqs/ntxqs variables · 09f83975
      Thomas Petazzoni authored
      Currently, the global variables rxq_number and txq_number hold the
      number of per-port TXQs and RXQs. Until now, such numbers were
      constant regardless of the driver configuration. As we are going to
      introduce different modes for TX and RX queues, these numbers will
      depend on the configuration (PPv2.1 vs. PPv2.2, exact queue
      distribution logic).
      
      Therefore, as a preparation, we move the number of RXQs and TXQs in
      the 'struct mvpp2_port' structure, next to the RXQs and TXQs
      descriptor arrays.
      
      For now, they remain initialized to the same default values as
      rxq_number/txq_number used to be initialized, but this will change in
      future commits.
      
      The only non-mechanical change in this patch is that the check to
      verify hardware constraints on the number of RXQs and TXQs is moved
      from mvpp2_probe() to mvpp2_port_probe(), since it's now in
      mvpp2_port_probe() that we initialize the per-port count of RXQ and
      TXQ.
      Signed-off-by: default avatarThomas Petazzoni <thomas.petazzoni@free-electrons.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      09f83975
    • Thomas Petazzoni's avatar
      net: mvpp2: remove RX queue group reset code · b5635ad2
      Thomas Petazzoni authored
      The RX queue group allocation is anyway re-done later in
      mvpp2_port_init(), so resetting it in mvpp2_init() is not very useful,
      and will be annoying as we are going to rework the RX queue group
      allocation logic.
      Signed-off-by: default avatarThomas Petazzoni <thomas.petazzoni@free-electrons.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      b5635ad2
    • Thomas Petazzoni's avatar
      net: mvpp2: fix MVPP21_ISR_RXQ_GROUP_REG definition · eb1e93a1
      Thomas Petazzoni authored
      The MVPP21_ISR_RXQ_GROUP_REG register is not indexed by rxq, but by
      port, so we fix the parameter name accordingly. There are no
      functional changes.
      Signed-off-by: default avatarThomas Petazzoni <thomas.petazzoni@free-electrons.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      eb1e93a1
    • Romain Perier's avatar
      net: arc_emac: Add support for ndo_do_ioctl net_device_ops operation · e11e8729
      Romain Perier authored
      This operation is required for handling ioctl commands like SIOCGMIIREG,
      when debugging MDIO registers from userspace.
      
      This commit adds support for this operation.
      Signed-off-by: default avatarRomain Perier <romain.perier@collabora.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      e11e8729
    • David S. Miller's avatar
      Merge branch 'hns3-ethernet-driver' · f4b2a420
      David S. Miller authored
      Salil Mehta says:
      
      ====================
      Hisilicon Network Subsystem 3 Ethernet Driver
      
      This patch-set contains the support of the HNS3 (Hisilicon Network Subsystem 3)
      Ethernet driver for hip08 family of SoCs and future upcoming SoCs.
      
      Hisilicon's new hip08 SoCs have integrated ethernet based on PCI Express and
      hence there was a need of new driver over the previous HNS driver which is
      already part of the Linux mainline. This new driver is NOT backward
      compatible with HNS.
      
      This current driver is meant to control the Physical Function and there would
      soon be a support of a separate driver for Virtual Function once this base PF
      driver has been accepted. Also, this driver is the ongoing development work and
      HNS3 Ethernet driver would be incrementally enhanced with more new features.
      
      High Level Architecture:
      
              [ Ethtool ]
      	   ^  |
                 |  |
           [Ethernet Client]  [ODP/UIO Client] . . . [ RoCE Client ]
                               |                            |
                         [ HNAE Device ]                    |
                               |                            |
          ---------------------------------------------     |
                               |                            |
           [ HNAE3 Framework (Register/unregister) ]        |
                               |                            |
          ---------------------------------------------     |
                               |                            |
                         [ HCLGE Layer]                     |
               ________________|_________________           |
              |                |                 |          |
          [ MDIO ]    [ Scheduler/Shaper ]  [ Debugfs* ]    |
              |                |                 |          |
              |________________|_________________|          |
                               |                            |
                   [ IMP command Interface ]                |
          ---------------------------------------------     |
                    HIP08  H A R D W A R E                  *
      
      Current patch-set broadly adds the support of the following PF functionality:
       1. Basic Rx and Tx functionality
       2. TSO support
       3. Ethtool support
       4. * Debugfs support -> this patch for now has been taken off.
       5. HNAE framework and hardware compatability layer
       6. Scheduler and Shaper support in transmit function
       7. MDIO support
      
      Change Log:
      V5->V6: Addressed below comments:
              * Andrew Lunn: Comments on MDIO and ethtool link mode
              * Leon Romanvosky: Some comments on HNAE layer tidy-up
              * Internal comments on redundant code removal, fixing error types etc.
      V4->V5: Addressed below concerns:
              * Florian Fanelli: Miscellaneous comments on ethtool & enet layer
              * Stephen Hemminger: comment of Netdev stats in ethool layer
              * Leon Romanvosky: Comments on Driver Version String, naming & Kconfig
              * Rochard Cochran: Redundant function prototype
      V3->V4: Addressed below comments:
              * Andrew Lunn: Various comments on MDIO, ethtool, ENET driver etc,
              * Stephen Hemminger: change access and updation to 64 but statistics
              * Bo You: some spelling mistakes and checkpatch.pl errors.
      V2->V3: Addressed comments
              * Yuval Mintz: Removal of redundant userprio-to-tc code
              * Stephen Hemminger: Ethtool & interuupt enable
              * Andrew Lunn: On C45/C22 PHy support, HNAE, ethtool
              * Florian Fainelli: C45/C22 and phy_connect/attach
              * Intel kbuild errors
      V1->V2: Addressed some comments by kbuild, Yuval MIntz, Andrew Lunn &
              Florian Fainelli in the following patches:
              * Add support of HNS3 Ethernet Driver for hip08 SoC
              * Add MDIO support to HNS3 Ethernet driver for hip08 SoC
              * Add support of debugfs interface to HNS3 driver
      ====================
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      f4b2a420