• Filipe Manana's avatar
    btrfs: send: cache leaf to roots mapping during backref walking · 66d04209
    Filipe Manana authored
    During a send operation, when doing backref walking to determine which
    inodes/offsets/roots we can clone from, the most repetitive and expensive
    step is to map each leaf that has file extent items pointing to the target
    data extent to the IDs of the roots from which the leaves are accessible,
    which happens at iterate_extent_inodes(). That step requires finding every
    parent node of a leaf, then the parent of each parent, and so on until we
    reach a root node. So it's a naturally expensive operation, and repetitive
    because each leaf can have hundreds of file extent items (for a nodesize
    of 16K, that can be slightly over 200 file extent items). There's also
    temporal locality, as we process all file extent items from a leave before
    moving the next leaf.
    
    This change caches the mapping of leaves to root IDs, to avoid repeating
    those computations over and over again. The cache is limited to a maximum
    of 128 entries, with each entry being a struct with a size of 128 bytes,
    so the maximum cache size is 16K plus any nodes internally allocated by
    the maple tree that is used to index pointers to those structs. The cache
    is invalidated whenever we detect relocation happened since we started
    filling the cache, because if relocation happened then extent buffers for
    leaves and nodes of the trees used by a send operation may have been
    reallocated.
    
    This cache also allows for another important optimization that is
    introduced in the next patch in the series.
    
    This change is part of a patchset comprised of the following patches:
    
      01/17 btrfs: fix inode list leak during backref walking at resolve_indirect_refs()
      02/17 btrfs: fix inode list leak during backref walking at find_parent_nodes()
      03/17 btrfs: fix ulist leaks in error paths of qgroup self tests
      04/17 btrfs: remove pointless and double ulist frees in error paths of qgroup tests
      05/17 btrfs: send: avoid unnecessary path allocations when finding extent clone
      06/17 btrfs: send: update comment at find_extent_clone()
      07/17 btrfs: send: drop unnecessary backref context field initializations
      08/17 btrfs: send: avoid unnecessary backref lookups when finding clone source
      09/17 btrfs: send: optimize clone detection to increase extent sharing
      10/17 btrfs: use a single argument for extent offset in backref walking functions
      11/17 btrfs: use a structure to pass arguments to backref walking functions
      12/17 btrfs: reuse roots ulist on each leaf iteration for iterate_extent_inodes()
      13/17 btrfs: constify ulist parameter of ulist_next()
      14/17 btrfs: send: cache leaf to roots mapping during backref walking
      15/17 btrfs: send: skip unnecessary backref iterations
      16/17 btrfs: send: avoid double extent tree search when finding clone source
      17/17 btrfs: send: skip resolution of our own backref when finding clone source
    
    Performance test results are in the changelog of patch 17/17.
    Signed-off-by: default avatarFilipe Manana <fdmanana@suse.com>
    Signed-off-by: default avatarDavid Sterba <dsterba@suse.com>
    66d04209
backref.c 95.5 KB