• Darrick J. Wong's avatar
    xfs: define an in-memory btree for storing refcount bag info during repairs · 18a1e644
    Darrick J. Wong authored
    Create a new in-memory btree type so that we can store refcount bag info
    in a much more memory-efficient and performant format.  Recall that the
    refcount recordset regenerator computes the new recordset from browsing
    the rmap records.  Let's say that the rmap records are:
    
    {agbno: 10, length: 40, ...}
    {agbno: 11, length: 3, ...}
    {agbno: 12, length: 20, ...}
    {agbno: 15, length: 1, ...}
    
    It is convenient to have a data structure that could quickly tell us the
    refcount for an arbitrary agbno without wasting memory.  An array or a
    list could do that pretty easily.  List suck because of the pointer
    overhead.  xfarrays are a lot more compact, but we want to minimize
    sparse holes in the xfarray to constrain memory usage.  Maintaining any
    kind of record order isn't needed for correctness, so I created the
    "rcbag", which is shorthand for an unordered list of (excerpted) reverse
    mappings.
    
    So we add the first rmap to the rcbag, and it looks like:
    
    0: {agbno: 10, length: 40}
    
    The refcount for agbno 10 is 1.  Then we move on to block 11, so we add
    the second rmap:
    
    0: {agbno: 10, length: 40}
    1: {agbno: 11, length: 3}
    
    The refcount for agbno 11 is 2.  We move on to block 12, so we add the
    third:
    
    0: {agbno: 10, length: 40}
    1: {agbno: 11, length: 3}
    2: {agbno: 12, length: 20}
    
    The refcount for agbno 12 and 13 is 3.  We move on to block 14, and
    remove the second rmap:
    
    0: {agbno: 10, length: 40}
    1: NULL
    2: {agbno: 12, length: 20}
    
    The refcount for agbno 14 is 2.  We move on to block 15, and add the
    last rmap.  But we don't care where it is and we don't want to expand
    the array so we put it in slot 1:
    
    0: {agbno: 10, length: 40}
    1: {agbno: 15, length: 1}
    2: {agbno: 12, length: 20}
    
    The refcount for block 15 is 3.  Notice how order doesn't matter in this
    list?  That's why repair uses an unordered list, or "bag".  The data
    structure is not a set because it does not guarantee uniqueness.
    
    That said, adding and removing specific items is now an O(n) operation
    because we have no idea where that item might be in the list.  Overall,
    the runtime is O(n^2) which is bad.
    
    I realized that I could easily refactor the btree code and reimplement
    the refcount bag with an xfbtree.  Adding and removing is now O(log2 n),
    so the runtime is at least O(n log2 n), which is much faster.  In the
    end, the rcbag becomes a sorted list, but that's merely a detail of the
    implementation.  The repair code doesn't care.
    
    (Note: That horrible xfs_db bmap_inflate command can be used to exercise
    this sort of rcbag insanity by cranking up refcounts quickly.)
    Signed-off-by: default avatarDarrick J. Wong <djwong@kernel.org>
    Reviewed-by: default avatarChristoph Hellwig <hch@lst.de>
    18a1e644
xfs_stats.c 4.5 KB