• Eric Dumazet's avatar
    net: properly flush delay-freed skbs · f52dffe0
    Eric Dumazet authored
    Typical NAPI drivers use napi_consume_skb(skb) at TX completion time.
    This put skb in a percpu special queue, napi_alloc_cache, to get bulk
    frees.
    
    It turns out the queue is not flushed and hits the NAPI_SKB_CACHE_SIZE
    limit quite often, with skbs that were queued hundreds of usec earlier.
    I measured this can take ~6000 nsec to perform one flush.
    
    __kfree_skb_flush() can be called from two points right now :
    
    1) From net_tx_action(), but only for skbs that were queued to
    sd->completion_queue.
    
     -> Irrelevant for NAPI drivers in normal operation.
    
    2) From net_rx_action(), but only under high stress or if RPS/RFS has a
    pending action.
    
    This patch changes net_rx_action() to perform the flush in all cases and
    after more urgent operations happened (like kicking remote CPUS for
    RPS/RFS).
    Signed-off-by: default avatarEric Dumazet <edumazet@google.com>
    Cc: Jesper Dangaard Brouer <brouer@redhat.com>
    Cc: Alexander Duyck <alexander.h.duyck@intel.com>
    Acked-by: default avatarAlexander Duyck <alexander.h.duyck@intel.com>
    Acked-by: default avatarJesper Dangaard Brouer <brouer@redhat.com>
    Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
    f52dffe0
dev.c 209 KB