• unknown's avatar
    WL#3071 - Maria checkpoint · 77017191
    unknown authored
    - serializing calls to flush_pagecache_blocks_int() on the same file
    to avoid known concurrency bugs
    - having that, we can now enable the background thread, as the
    flushes it does are now supposedly safe in concurrent situations.
    - new type of flush FLUSH_KEEP_LAZY: when the background checkpoint
    thread is flushing a packet of dirty pages between two checkpoints,
    it uses this flush type, indeed if a file is already being flushed
    by another thread it's smarter to move on to the next file than wait.
    - maria_checkpoint_frequency renamed to maria_checkpoint_interval.
    
    
    include/my_sys.h:
      new type of flushing for the page cache: FLUSH_KEEP_LAZY
    mysql-test/r/maria.result:
      result update
    mysys/mf_keycache.c:
      indentation. No FLUSH_KEEP_LAZY support in key cache.
    storage/maria/ha_maria.cc:
      maria_checkpoint_frequency was somehow a hidden part of the
      Checkpoint API and that was not good. Now we have checkpoint_interval,
      local to ha_maria.cc, which serves as container for the user-visible
      maria_checkpoint_interval global variable; setting it calls
      update_checkpoint_interval which passes the new value to
      ma_checkpoint_init(). There is no hiding anymore.
      By default, enable background thread which does checkpoints
      every 30 seconds, and dirty page flush in between. That thread takes
      a checkpoint when it ends, so no need for maria_hton_panic to take one.
      The | is | and not ||, because maria_panic() must always be called.
      frequency->interval.
    storage/maria/ma_checkpoint.c:
      Use FLUSH_KEEP_LAZY for background thread when it flushes packets of
      dirty pages between two checkpoints: it is smarter to move on to
      the next file than wait for it to have been completely flushed, which
      may take long.
      Comments about flush concurrency bugs moved from ma_pagecache.c.
      Removing out-of-date comment.
      frequency->interval.
      create_background_thread -> (interval>0).
      In ma_checkpoint_background(), some variables need to be preserved
      between iterations.
    storage/maria/ma_checkpoint.h:
      new prototype
    storage/maria/ma_pagecache.c:
      - concurrent calls of flush_pagecache_blocks_int() on the same file
      cause bugs (see @note in that function); we fix them by serializing
      in this situation. For that we use a global hash of (file, wqueue).
      When flush_pagecache_blocks_int() starts it looks into the hash,
      using the file as key. If not found, it inserts (file,wqueue) into the
      hash, flushes the file, and finally removes itself from the hash and
      wakes up any waiter in the queue. If found, it adds itself to the
      wqueue and waits.
      - As a by-product, we can remove changed_blocks_is_incomplete
      and replace it by scanning the hash, replace the sleep() by a queue wait.
      - new type of flush FLUSH_KEEP_LAZY: when flushing a file, if it's
      already being flushed by another thread (even partially), return
      immediately.
    storage/maria/ma_pagecache.h:
      In pagecache, a hash of files currently being flushed (i.e. there
      is a call to flush_pagecache_blocks_int() for them).
    storage/maria/ma_recovery.c:
      new prototype
    storage/maria/ma_test1.c:
      new prototype
    storage/maria/ma_test2.c:
      new prototype
    77017191
my_sys.h 38.5 KB