• unknown's avatar
    WL#3072 - Maria recovery. · d0b9387b
    unknown authored
    * Recovery of the table's live checksum (CREATE TABLE ... CHECKSUM=1)
    is achieved in this patch. The table's live checksum
    (info->s->state.state.checksum) is updated in inwrite_rec_hook's
    under the log mutex when writing UNDO_ROW_INSERT|UPDATE|DELETE
    and REDO_DELETE_ALL. The checksum variation caused by the operation
    is stored in these UNDOs, so that the REDO phase, when it sees such
    UNDOs, can update the live checksum if it is older (state.is_of_lsn is
    lower) than the record. It is also used, as a nice add-on with no
    cost, to do less row checksum computation during the UNDO phase
    (as we have it in the record already).
    Doing this work, it became pressing to move in-write hooks
    (write_hook_for_redo() et al) to ma_blockrec.c.
    The 'parts' argument of inwrite_rec_hook is unpredictable (it comes
    mangled at this stage, for example by LSN compression) so it is
    replaced by a 'void* hook_arg', which is used to pass down information,
    currently only to write_hook_for_clr_end() (previous undo_lsn and
    type of undone record).
    * If from ha_maria, we print to stderr how many seconds (with one
    fractional digit) the REDO phase took, same for UNDO phase and for
    final table close. Just to give an indication for debugging and maybe
    also for Support.
    
    
    storage/maria/ha_maria.cc:
      question for Monty
    storage/maria/ma_blockrec.c:
      * log in-write hooks (write_hook_for_redo() etc) move from
      ma_loghandler.c to here; this is natural: the hooks are coupled
      to their callers (functions in ma_blockrec.c).
      * translog_write_record() now has a new argument "hook_arg";
      using it to pass down to write_hook_for_clr_end() the transaction's
      previous_undo_lsn and the type of the being undone record, and also
      to pass down to all UNDOs the live checksum variation caused by the
      operation.
      * If table has live checksum, store in UNDO_ROW_INSERT|UPDATE|DELETE
      and in CLR_END the checksum variation ("delta") caused by the
      operation. For example if a DELETE caused the table's live checksum
      to change from 123 to 456, we store in the UNDO_ROW_DELETE, in 4 bytes,
      the value 333 (456-123).
      * Instead of hard-coded "1" as length of the place where we store
      the undone record's type in CLR_END, use a symbol CLR_TYPE_STORE_SIZE;
      use macros clr_type_store and clr_type_korr.
      * write_block_record() has a new parameter 'old_record_checksum'
      which is the pre-computed checksum of old_record; that value is used
      to update the table's live checksum when writing UNDO_ROW_UPDATE|CLR_END.
      * In allocate_write_block_record(), if we are executing UNDO_ROW_DELETE
      the row's checksum is already computed.
      * _ma_update_block_record2() now expect the new row's checksum into
      cur_row.checksum (was already true) and the old row's checksum into
      new_row.checksum (that's new). Its two callers, maria_update() and
      _ma_apply_undo_row_update(), honour this.
      * When executing an UNDO_ROW_INSERT|UPDATE|DELETE in UNDO phase, pick
      up the checksum delta from the log record. It is then used to update
      the table's live checksum when writing CLR_END, and saves us a
      computation of record.
    storage/maria/ma_blockrec.h:
      in-write hooks move from ma_loghandler.c
    storage/maria/ma_check.c:
      more straightforward size of buffer
    storage/maria/ma_checkpoint.c:
      <= is enough
    storage/maria/ma_commit.c:
      new prototype of translog_write_record()
    storage/maria/ma_create.c:
      new prototype of translog_write_record()
    storage/maria/ma_delete.c:
      The row's checksum must be computed before calling(*delete_record)(),
      not after, because it must be known inside _ma_delete_block_record()
      (to update the table's live checksum when writing UNDO_ROW_DELETE).
      If deleting from a transactional table, live checksum was already updated
      when writing UNDO_ROW_DELETE.
    storage/maria/ma_delete_all.c:
      @todo is now done (in ma_loghandler.c)
    storage/maria/ma_delete_table.c:
      new prototype of translog_write_record()
    storage/maria/ma_loghandler.c:
      * in-write hooks move to ma_blockrec.c.
      * translog_write_record() gets a new argument 'hook_arg' which is
      passed down to pre|inwrite_rec_hook. It is more useful that 'parts'
      for those hooks, because when those hooks are called, 'parts' has
      possibly been mangled (like with LSN compression) and is so
      unpredictable.
      * fix for compiler warning (unused buffer_start when compiling without
      debug support)
      * Because checksum delta is stored into UNDO_ROW_INSERT|UPDATE|DELETE
      and CLR_END, but only if the table has live checksum, these records
      are not PSEUDOFIXEDLENGTH anymore, they are now VARIABLE_LENGTH (their
      length is X if no live checksum and X+4 otherwise).
      * add an inwrite_rec_hook for UNDO_ROW_UPDATE, which updates the
      table's live checksum. Update it also in hooks of UNDO_ROW_INSERT|
      DELETE and REDO_DELETE_ALL and CLR_END.
      * Bugfix: when reading a record in translog_read_record(), it happened
      that "length" became negative, because the function assumed that
      the record extended beyond the page's end, whereas it may be shorter.
    storage/maria/ma_loghandler.h:
      * Instead of hard-coded "1" and "4", use symbols and macros
      to store/retrieve the type of record which the CLR_END corresponds
      to, and the checksum variation caused by the operation which logs the
      record
      * translog_write_record() gets a new argument 'hook_arg' which is
      passed down to pre|inwrite_rec_hook. It is more useful that 'parts'
      for those hooks, because when those hooks are called, 'parts' has
      possibly been mangled (like with LSN compression) and is so
      unpredictable.
    storage/maria/ma_open.c:
      fix for "empty body in if() statement" (when compiling without safemutex)
    storage/maria/ma_pagecache.c:
      <= is enough
    storage/maria/ma_recovery.c:
      * print the time that each recovery phase (REDO/UNDO/flush) took;
      this is enabled only when recovering from ha_maria. Is it printed
      n seconds with a fractional part of one digit (like 123.4 seconds).
      * In the REDO phase, update the table's live checksum by using
      the checksum delta stored in UNDO_ROW_INSERT|DELETE|UPDATE and CLR_END.
      Update it too when seeing REDO_DELETE_ALL.
      * In the UNDO phase, when executing UNDO_ROW_INSERT, if the table does
      not have live checksum then reading the record's header (as done by
      the master loop of run_undo_phase()) is enough; otherwise we
      do a translog_read_record() to have the checksum delta ready
      for _ma_apply_undo_row_insert().
      * When at the end of the REDO phase we notice that there is an unfinished
      group of REDOs, don't assert in debug binaries, as I verified that it
      can happen in real life (with kill -9)
      * removing ' in #error as it confuses gcc3
    storage/maria/ma_rename.c:
      new prototype of translog_write_record()
    storage/maria/ma_test_recovery.expected:
      Change in output of ma_test_recovery: now all live checksums of
      original tables equal those of tables recreated by the REDO phase
      and those of tables fixed by the UNDO phase. I.e. recovery of
      the live checksum looks like working (which was after all the only
      goal of this changeset).
      I checked by hand that it's not just all live checksums which are
      now 0 and that's why they match. They are the old values like
      3757530372. maria.test has hard-coded checksum values in its result
      file so checks this too.
    storage/maria/ma_update.c:
      * It's useless to put up HA_STATE_CHANGED in 'key_changed',
      as we put up HA_STATE_CHANGED in info->update anyway.
      * We need to compute the old and new rows' checksum before calling
      (*update_record)(), as checksum delta must be known when logging
      UNDO_ROW_UPDATE which is done by _ma_update_block_record(). Note that
      some functions change the 'newrec' record (at least _ma_check_unique()
      does) so we cannot move the checksum computation too early in the
      function.
    storage/maria/ma_write.c:
      If inserting into a transactional table, live's checksum was
      already updated when writing UNDO_ROW_INSERT. The multiplication
      is a trick to save an if().
    storage/maria/unittest/ma_test_loghandler-t.c:
      new prototype of translog_write_record()
    storage/maria/unittest/ma_test_loghandler_first_lsn-t.c:
      new prototype of translog_write_record()
    storage/maria/unittest/ma_test_loghandler_max_lsn-t.c:
      new prototype of translog_write_record()
    storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
      new prototype of translog_write_record()
    storage/maria/unittest/ma_test_loghandler_multithread-t.c:
      new prototype of translog_write_record()
    storage/maria/unittest/ma_test_loghandler_noflush-t.c:
      new prototype of translog_write_record()
    storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
      new prototype of translog_write_record()
    storage/maria/unittest/ma_test_loghandler_purge-t.c:
      new prototype of translog_write_record()
    storage/myisam/sort.c:
      fix for compiler warnings in pushbuild (write_merge_key* functions
      didn't have their declaration match MARIA_HA::write_key).
    d0b9387b
ma_test_loghandler_purge-t.c 4.59 KB