• Kristian Nielsen's avatar
    MDEV-4991: GTID binlog indexing · d039346a
    Kristian Nielsen authored
    Improve the performance of slave connect using B+-Tree indexes on each binlog
    file. The index allows fast lookup of a GTID position to the corresponding
    offset in the binlog file, as well as lookup of a position to find the
    corresponding GTID position.
    
    This eliminates a costly sequential scan of the starting binlog file
    to find the GTID starting position when a slave connects. This is
    especially costly if the binlog file is not cached in memory (IO
    cost), or if it is encrypted or a lot of slaves connect simultaneously
    (CPU cost).
    
    The size of the index files is generally less than 1% of the binlog data, so
    not expected to be an issue.
    
    Most of the work writing the index is done as a background task, in
    the binlog background thread. This minimises the performance impact on
    transaction commit. A simple global mutex is used to protect index
    reads and (background) index writes; this is fine as slave connect is
    a relatively infrequent operation.
    
    Here are the user-visible options and status variables. The feature is on by
    default and is expected to need no tuning or configuration for most users.
    
    binlog_gtid_index
      On by default. Can be used to disable the indexes for testing purposes.
    
    binlog_gtid_index_page_size (default 4096)
      Page size to use for the binlog GTID index. This is the size of the nodes
      in the B+-tree used internally in the index. A very small page-size (64 is
      the minimum) will be less efficient, but can be used to stress the
      BTree-code during testing.
    
    binlog_gtid_index_span_min (default 65536)
      Control sparseness of the binlog GTID index. If set to N, at most one
      index record will be added for every N bytes of binlog file written.
      This can be used to reduce the number of records in the index, at
      the cost only of having to scan a few more events in the binlog file
      before finding the target position
    
    Two status variables are available to monitor the use of the GTID indexes:
    
      Binlog_gtid_index_hit
      Binlog_gtid_index_miss
    
    The "hit" status increments for each successful lookup in a GTID index.
    The "miss" increments when a lookup is not possible. This indicates that the
    index file is missing (eg. binlog written by old server version
    without GTID index support), or corrupt.
    Signed-off-by: default avatarKristian Nielsen <knielsen@knielsen-hq.org>
    d039346a
relaylog.result 11.8 KB