1. 29 Jan, 2021 1 commit
    • Jakub Kicinski's avatar
      Merge tag 'mlx5-updates-2021-01-13' of git://git.kernel.org/pub/scm/linux/kernel/git/saeed/linux · 24a790da
      Jakub Kicinski authored
      Saeed Mahameed says:
      
      ====================
      mlx5 subfunction support
      
      Parav Pandit says:
      
      This patchset introduces support for mlx5 subfunction (SF).
      
      A subfunction is a lightweight function that has a parent PCI function on
      which it is deployed. mlx5 subfunction has its own function capabilities
      and its own resources. This means a subfunction has its own dedicated
      queues(txq, rxq, cq, eq). These queues are neither shared nor stolen from
      the parent PCI function.
      
      When subfunction is RDMA capable, it has its own QP1, GID table and rdma
      resources neither shared nor stolen from the parent PCI function.
      
      A subfunction has dedicated window in PCI BAR space that is not shared
      with the other subfunctions or parent PCI function. This ensures that all
      class devices of the subfunction accesses only assigned PCI BAR space.
      
      A Subfunction supports eswitch representation through which it supports tc
      offloads. User must configure eswitch to send/receive packets from/to
      subfunction port.
      
      Subfunctions share PCI level resources such as PCI MSI-X IRQs with
      their other subfunctions and/or with its parent PCI function.
      
      Subfunction support is discussed in detail in RFC [1] and [2].
      RFC [1] and extension [2] describes requirements, design and proposed
      plumbing using devlink, auxiliary bus and sysfs for systemd/udev
      support. Functionality of this patchset is best explained using real
      examples further below.
      
      overview:
      --------
      A subfunction can be created and deleted by a user using devlink port
      add/delete interface.
      
      A subfunction can be configured using devlink port function attribute
      before its activated.
      
      When a subfunction is activated, it results in an auxiliary device on
      the host PCI device where it is deployed. A driver binds to the
      auxiliary device that further creates supported class devices.
      
      example subfunction usage sequence:
      -----------------------------------
      Change device to switchdev mode:
      $ devlink dev eswitch set pci/0000:06:00.0 mode switchdev
      
      Add a devlink port of subfunction flavour:
      $ devlink port add pci/0000:06:00.0 flavour pcisf pfnum 0 sfnum 88
      
      Configure mac address of the port function:
      $ devlink port function set ens2f0npf0sf88 hw_addr 00:00:00:00:88:88
      
      Now activate the function:
      $ devlink port function set ens2f0npf0sf88 state active
      
      Now use the auxiliary device and class devices:
      $ devlink dev show
      pci/0000:06:00.0
      auxiliary/mlx5_core.sf.4
      
      $ ip link show
      127: ens2f0np0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
          link/ether 24:8a:07:b3:d1:12 brd ff:ff:ff:ff:ff:ff
          altname enp6s0f0np0
      129: p0sf88: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
          link/ether 00:00:00:00:88:88 brd ff:ff:ff:ff:ff:ff
      
      $ rdma dev show
      43: rdmap6s0f0: node_type ca fw 16.29.0550 node_guid 248a:0703:00b3:d112 sys_image_guid 248a:0703:00b3:d112
      44: mlx5_0: node_type ca fw 16.29.0550 node_guid 0000:00ff:fe00:8888 sys_image_guid 248a:0703:00b3:d112
      
      After use inactivate the function:
      $ devlink port function set ens2f0npf0sf88 state inactive
      
      Now delete the subfunction port:
      $ devlink port del ens2f0npf0sf88
      
      [1] https://lore.kernel.org/netdev/20200519092258.GF4655@nanopsycho/
      [2] https://marc.info/?l=linux-netdev&m=158555928517777&w=2
      
      =================
      
      * tag 'mlx5-updates-2021-01-13' of git://git.kernel.org/pub/scm/linux/kernel/git/saeed/linux:
        net/mlx5: Add devlink subfunction port documentation
        devlink: Extend devlink port documentation for subfunctions
        devlink: Add devlink port documentation
        net/mlx5: SF, Port function state change support
        net/mlx5: SF, Add port add delete functionality
        net/mlx5: E-switch, Add eswitch helpers for SF vport
        net/mlx5: E-switch, Prepare eswitch to handle SF vport
        net/mlx5: SF, Add auxiliary device driver
        net/mlx5: SF, Add auxiliary device support
        net/mlx5: Introduce vhca state event notifier
        devlink: Support get and set state of port function
        devlink: Support add and delete devlink port
        devlink: Introduce PCI SF port flavour and port attribute
        devlink: Prepare code to fill multiple port function attributes
      ====================
      
      Link: https://lore.kernel.org/r/20210122193658.282884-1-saeed@kernel.orgSigned-off-by: default avatarJakub Kicinski <kuba@kernel.org>
      24a790da
  2. 28 Jan, 2021 39 commits