1. 18 Jan, 2014 33 commits
  2. 17 Jan, 2014 7 commits
    • John W. Linville's avatar
      Merge branch 'master' of... · 7916a075
      John W. Linville authored
      Merge branch 'master' of git://git.kernel.org/pub/scm/linux/kernel/git/linville/wireless-next into for-davem
      7916a075
    • David S. Miller's avatar
      Merge branch 'virtio_rx_merging' · cf84eb0b
      David S. Miller authored
      Michael Dalton says:
      
      ====================
      virtio-net: mergeable rx buffer size auto-tuning
      
      The virtio-net device currently uses aligned MTU-sized mergeable receive
      packet buffers. Network throughput for workloads with large average
      packet size can be improved by posting larger receive packet buffers.
      However, due to SKB truesize effects, posting large (e.g, PAGE_SIZE)
      buffers reduces the throughput of workloads that do not benefit from GRO
      and have no large inbound packets.
      
      This patchset introduces virtio-net mergeable buffer size auto-tuning,
      with buffer sizes ranging from aligned MTU-size to PAGE_SIZE. Packet
      buffer size is chosen based on a per-receive queue EWMA of incoming
      packet size.
      
      To unify mergeable receive buffer memory allocation and improve
      SKB frag coalescing, all mergeable buffer memory allocation is
      migrated to per-receive queue page frag allocators.
      
      The per-receive queue mergeable packet buffer size is exported via
      sysfs, and the network device sysfs layer has been extended to add
      support for device-specific per-receive queue sysfs attribute groups.
      ====================
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      cf84eb0b
    • Michael Dalton's avatar
      virtio-net: initial rx sysfs support, export mergeable rx buffer size · fbf28d78
      Michael Dalton authored
      Add initial support for per-rx queue sysfs attributes to virtio-net. If
      mergeable packet buffers are enabled, adds a read-only mergeable packet
      buffer size sysfs attribute for each RX queue.
      Suggested-by: default avatarMichael S. Tsirkin <mst@redhat.com>
      Acked-by: default avatarMichael S. Tsirkin <mst@redhat.com>
      Signed-off-by: default avatarMichael Dalton <mwdalton@google.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      fbf28d78
    • Michael Dalton's avatar
      lib: Ensure EWMA does not store wrong intermediate values · 03144b58
      Michael Dalton authored
      To ensure ewma_read() without a lock returns a valid but possibly
      out of date average, modify ewma_add() by using ACCESS_ONCE to prevent
      intermediate wrong values from being written to avg->internal.
      Suggested-by: default avatarEric Dumazet <eric.dumazet@gmail.com>
      Acked-by: default avatarMichael S. Tsirkin <mst@redhat.com>
      Acked-by: default avatarEric Dumazet <edumazet@google.com>
      Signed-off-by: default avatarMichael Dalton <mwdalton@google.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      03144b58
    • Michael Dalton's avatar
      net-sysfs: add support for device-specific rx queue sysfs attributes · a953be53
      Michael Dalton authored
      Extend existing support for netdevice receive queue sysfs attributes to
      permit a device-specific attribute group. Initial use case for this
      support will be to allow the virtio-net device to export per-receive
      queue mergeable receive buffer size.
      Signed-off-by: default avatarMichael Dalton <mwdalton@google.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      a953be53
    • Michael Dalton's avatar
      virtio-net: auto-tune mergeable rx buffer size for improved performance · ab7db917
      Michael Dalton authored
      Commit 2613af0e ("virtio_net: migrate mergeable rx buffers to page frag
      allocators") changed the mergeable receive buffer size from PAGE_SIZE to
      MTU-size, introducing a single-stream regression for benchmarks with large
      average packet size. There is no single optimal buffer size for all
      workloads.  For workloads with packet size <= MTU bytes, MTU + virtio-net
      header-sized buffers are preferred as larger buffers reduce the TCP window
      due to SKB truesize. However, single-stream workloads with large average
      packet sizes have higher throughput if larger (e.g., PAGE_SIZE) buffers
      are used.
      
      This commit auto-tunes the mergeable receiver buffer packet size by
      choosing the packet buffer size based on an EWMA of the recent packet
      sizes for the receive queue. Packet buffer sizes range from MTU_SIZE +
      virtio-net header len to PAGE_SIZE. This improves throughput for
      large packet workloads, as any workload with average packet size >=
      PAGE_SIZE will use PAGE_SIZE buffers.
      
      These optimizations interact positively with recent commit
      ba275241 ("virtio-net: coalesce rx frags when possible during rx"),
      which coalesces adjacent RX SKB fragments in virtio_net. The coalescing
      optimizations benefit buffers of any size.
      
      Benchmarks taken from an average of 5 netperf 30-second TCP_STREAM runs
      between two QEMU VMs on a single physical machine. Each VM has two VCPUs
      with all offloads & vhost enabled. All VMs and vhost threads run in a
      single 4 CPU cgroup cpuset, using cgroups to ensure that other processes
      in the system will not be scheduled on the benchmark CPUs. Trunk includes
      SKB rx frag coalescing.
      
      net-next w/ virtio_net before 2613af0e (PAGE_SIZE bufs): 14642.85Gb/s
      net-next (MTU-size bufs):  13170.01Gb/s
      net-next + auto-tune: 14555.94Gb/s
      
      Jason Wang also reported a throughput increase on mlx4 from 22Gb/s
      using MTU-sized buffers to about 26Gb/s using auto-tuning.
      Signed-off-by: default avatarMichael Dalton <mwdalton@google.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      ab7db917
    • Michael Dalton's avatar
      virtio-net: use per-receive queue page frag alloc for mergeable bufs · fb51879d
      Michael Dalton authored
      The virtio-net driver currently uses netdev_alloc_frag() for GFP_ATOMIC
      mergeable rx buffer allocations. This commit migrates virtio-net to use
      per-receive queue page frags for GFP_ATOMIC allocation. This change unifies
      mergeable rx buffer memory allocation, which now will use skb_refill_frag()
      for both atomic and GFP-WAIT buffer allocations.
      
      To address fragmentation concerns, if after buffer allocation there
      is too little space left in the page frag to allocate a subsequent
      buffer, the remaining space is added to the current allocated buffer
      so that the remaining space can be used to store packet data.
      Acked-by: default avatarMichael S. Tsirkin <mst@redhat.com>
      Signed-off-by: default avatarMichael Dalton <mwdalton@google.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      fb51879d