• brakmo's avatar
    bpf: Add support for fq's EDT to HBM · 71634d7f
    brakmo authored
    Adds support for fq's Earliest Departure Time to HBM (Host Bandwidth
    Manager). Includes a new BPF program supporting EDT, and also updates
    corresponding programs.
    
    It will drop packets with an EDT of more than 500us in the future
    unless the packet belongs to a flow with less than 2 packets in flight.
    This is done so each flow has at least 2 packets in flight, so they
    will not starve, and also to help prevent delayed ACK timeouts.
    
    It will also work with ECN enabled traffic, where the packets will be
    CE marked if their EDT is more than 50us in the future.
    
    The table below shows some performance numbers. The flows are back to
    back RPCS. One server sending to another, either 2 or 4 flows.
    One flow is a 10KB RPC, the rest are 1MB RPCs. When there are more
    than one flow of a given RPC size, the numbers represent averages.
    
    The rate limit applies to all flows (they are in the same cgroup).
    Tests ending with "-edt" ran with the new BPF program supporting EDT.
    Tests ending with "-hbt" ran on top HBT qdisc with the specified rate
    (i.e. no HBM). The other tests ran with the HBM BPF program included
    in the HBM patch-set.
    
    EDT has limited value when using DCTCP, but it helps in many cases when
    using Cubic. It usually achieves larger link utilization and lower
    99% latencies for the 1MB RPCs.
    HBM ends up queueing a lot of packets with its default parameter values,
    reducing the goodput of the 10KB RPCs and increasing their latency. Also,
    the RTTs seen by the flows are quite large.
    
                             Aggr              10K  10K  10K   1MB  1MB  1MB
             Limit           rate drops  RTT  rate  P90  P99  rate  P90  P99
    Test      rate  Flows    Mbps   %     us  Mbps   us   us  Mbps   ms   ms
    --------  ----  -----    ---- -----  ---  ---- ---- ----  ---- ---- ----
    cubic       1G    2       904  0.02  108   257  511  539   647 13.4 24.5
    cubic-edt   1G    2       982  0.01  156   239  656  967   743 14.0 17.2
    dctcp       1G    2       977  0.00  105   324  408  744   653 14.5 15.9
    dctcp-edt   1G    2       981  0.01  142   321  417  811   660 15.7 17.0
    cubic-htb   1G    2       919  0.00 1825    40 2822 4140   879  9.7  9.9
    
    cubic     200M    2       155  0.30  220    81  532  655    74  283  450
    cubic-edt 200M    2       188  0.02  222    87 1035 1095   101   84   85
    dctcp     200M    2       188  0.03  111    77  912  939   111   76  325
    dctcp-edt 200M    2       188  0.03  217    74 1416 1738   114   76   79
    cubic-htb 200M    2       188  0.00 5015     8 14ms 15ms   180   48   50
    
    cubic       1G    4       952  0.03  110   165  516  546   262   38  154
    cubic-edt   1G    4       973  0.01  190   111 1034 1314   287   65   79
    dctcp       1G    4       951  0.00  103   180  617  905   257   37   38
    dctcp-edt   1G    4       967  0.00  163   151  732 1126   272   43   55
    cubic-htb   1G    4       914  0.00 3249    13  7ms  8ms   300   29   34
    
    cubic       5G    4      4236  0.00  134   305  490  624  1310   10   17
    cubic-edt   5G    4      4865  0.00  156   306  425  759  1520   10   16
    dctcp       5G    4      4936  0.00  128   485  221  409  1484    7    9
    dctcp-edt   5G    4      4924  0.00  148   390  392  623  1508   11   26
    
    v1 -> v2: Incorporated Andrii's suggestions
    v2 -> v3: Incorporated Yonghong's suggestions
    v3 -> v4: Removed credit update that is not needed
    Signed-off-by: default avatarLawrence Brakmo <brakmo@fb.com>
    Acked-by: default avatarYonghong Song <yhs@fb.com>
    Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
    71634d7f
hbm_edt_kern.c 5.27 KB