• Jason Xing's avatar
    ixgbe: let the xdpdrv work with more than 64 cpus · 4fe81585
    Jason Xing authored
    Originally, ixgbe driver doesn't allow the mounting of xdpdrv if the
    server is equipped with more than 64 cpus online. So it turns out that
    the loading of xdpdrv causes the "NOMEM" failure.
    
    Actually, we can adjust the algorithm and then make it work through
    mapping the current cpu to some xdp ring with the protect of @tx_lock.
    
    Here are some numbers before/after applying this patch with xdp-example
    loaded on the eth0X:
    
    As client (tx path):
                         Before    After
    TCP_STREAM send-64   734.14    714.20
    TCP_STREAM send-128  1401.91   1395.05
    TCP_STREAM send-512  5311.67   5292.84
    TCP_STREAM send-1k   9277.40   9356.22 (not stable)
    TCP_RR     send-1    22559.75  21844.22
    TCP_RR     send-128  23169.54  22725.13
    TCP_RR     send-512  21670.91  21412.56
    
    As server (rx path):
                         Before    After
    TCP_STREAM send-64   1416.49   1383.12
    TCP_STREAM send-128  3141.49   3055.50
    TCP_STREAM send-512  9488.73   9487.44
    TCP_STREAM send-1k   9491.17   9356.22 (not stable)
    TCP_RR     send-1    23617.74  23601.60
    ...
    
    Notice: the TCP_RR mode is unstable as the official document explains.
    
    I tested many times with different parameters combined through netperf.
    Though the result is not that accurate, I cannot see much influence on
    this patch. The static key is places on the hot path, but it actually
    shouldn't cause a huge regression theoretically.
    Co-developed-by: default avatarShujin Li <lishujin@kuaishou.com>
    Signed-off-by: default avatarShujin Li <lishujin@kuaishou.com>
    Signed-off-by: default avatarJason Xing <xingwanli@kuaishou.com>
    Tested-by: default avatarSandeep Penigalapati <sandeep.penigalapati@intel.com>
    Signed-off-by: default avatarTony Nguyen <anthony.l.nguyen@intel.com>
    Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
    4fe81585
ixgbe_main.c 312 KB