• Willem de Bruijn's avatar
    sock: enable MSG_ZEROCOPY · 1f8b977a
    Willem de Bruijn authored
    Prepare the datapath for refcounted ubuf_info. Clone ubuf_info with
    skb_zerocopy_clone() wherever needed due to skb split, merge, resize
    or clone.
    
    Split skb_orphan_frags into two variants. The split, merge, .. paths
    support reference counted zerocopy buffers, so do not do a deep copy.
    Add skb_orphan_frags_rx for paths that may loop packets to receive
    sockets. That is not allowed, as it may cause unbounded latency.
    Deep copy all zerocopy copy buffers, ref-counted or not, in this path.
    
    The exact locations to modify were chosen by exhaustively searching
    through all code that might modify skb_frag references and/or the
    the SKBTX_DEV_ZEROCOPY tx_flags bit.
    
    The changes err on the safe side, in two ways.
    
    (1) legacy ubuf_info paths virtio and tap are not modified. They keep
        a 1:1 ubuf_info to sk_buff relationship. Calls to skb_orphan_frags
        still call skb_copy_ubufs and thus copy frags in this case.
    
    (2) not all copies deep in the stack are addressed yet. skb_shift,
        skb_split and skb_try_coalesce can be refined to avoid copying.
        These are not in the hot path and this patch is hairy enough as
        is, so that is left for future refinement.
    Signed-off-by: default avatarWillem de Bruijn <willemb@google.com>
    Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
    1f8b977a
dev.c 217 KB