• Eric Dumazet's avatar
    net_sched: sch_fq: avoid touching f->next from fq_gc() · dde0a648
    Eric Dumazet authored
    A significant amount of cpu cycles is spent in fq_gc()
    
    When fq_gc() does its lookup in the rb-tree, it needs the
    following fields from struct fq_flow :
    
    f->sk       (lookup key in the rb-tree)
    f->fq_node  (anchor in the rb-tree)
    f->next     (used to determine if the flow is detached)
    f->age      (used to determine if the flow is candidate for gc)
    
    This unfortunately spans two cache lines (assuming 64 bytes cache lines)
    
    We can avoid using f->next, if we use the low order bit of f->{age|tail}
    
    This low order bit is 0, if f->tail points to an sk_buff.
    We set the low order bit to 1, if the union contains a jiffies value.
    
    Combined with the following patch, this makes sure we only need
    to bring into cpu caches one cache line per flow.
    Signed-off-by: default avatarEric Dumazet <edumazet@google.com>
    Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
    dde0a648
sch_fq.c 24.8 KB