• unknown's avatar
    UNDO of rows now puts back all part of the row on their original pages and positions · bfd5c273
    unknown authored
    Added variable _dbug_on_ to speed up execution when DBUG is not going to be used
    Added --debug-on option to mysqld (to be able to turn of DBUG with --debug-on=0)
    Fixed some bugs with 'non_flushable' marking of bitmap pages
    Don't use 'non_flushable' marking of bitmap pages for not transactional tables
    SHOW CREATE TABLE now shows if table was created with page checksums
    Fixed a lot of bugs with BLOB handling in case of update/REDO and UNDO
    More tests (especially for blobs) and DBUG_ASSERTS()
    More readable output from maria_read_log and maria_chk
    Fixed wrong shift that caused Maria to crash on files > 4G
    Mark tables as crashed of REDO fails
    
    
    dbug/dbug.c:
      Changed to use my_bool (allowed me to remove some windows specific code)
      Added variable _dbug_on_ to speed up execution when DBUG is not going to be used
      Removed initialization of variables if not needed
    include/my_dbug.h:
      Use my_bool for some functions that was defined as BOOLEAN in dbug.c code
      Added DBUGGER_ON/DEBUGGER_OFF to speed up execution when DBUG is not used
    include/my_global.h:
      Define my_bool early
      Increase MY_HOW_OFTEN_TO_WRITE as computers are now faster than 10 years ago
    mysql-test/mysql-test-run.pl:
      Added debug-on=0 to speed up tests
    mysql-test/r/maria-recovery.result:
      Added new test by Guilhem to test if UNDO_ROW_DELETE preserves rowid
    mysql-test/r/maria.result:
      Added testing of page checksums
    mysql-test/t/crash_commit_before-master.opt:
      Added --debug-on as test require DBUG to work
    mysql-test/t/maria-recovery-bitmap-master.opt:
      Added --debug-on as test require DBUG to work
    mysql-test/t/maria-recovery-master.opt:
      Added --debug-on as test require DBUG to work
    mysql-test/t/maria-recovery.test:
      Added new test by Guilhem to test if UNDO_ROW_DELETE preserves rowid
    mysql-test/t/maria.test:
      Added testing of page checksums
    sql/mysqld.cc:
      Added --debug-on option (to be able to turn of DBUG with --debug-on=0)
      Indentation fixes
      Removed end spaces
    sql/sql_show.cc:
      Allow update_create_info() to inform MySQL if PACK_KEYS, NO_PACK_KEYS, CHECKSUM, PAGE_CHECKSUM or DELAY_KEY_WRITE is used
    storage/maria/Makefile.am:
      Added ma_test_big.sh
    storage/maria/ha_maria.cc:
      Store in create_info if page checksums are used (For SHOW CREATE TABLE)
    storage/maria/ma_bitmap.c:
      Added _ma_bitmap_wait_or_flush() to cause reader of bitmap pages to wait with reading until bitmap is flushed.
      Use TAIL_PAGE_COUNT_MARKER for tail pages
      Set 'sub_blocks' for and only for the head page or for the first extent of a blob. This is needed for store_extent_info() to be able to set START_EXTENT_BIT's
      Don't allocate more than 0x3ffff pages in one extent (We need bit 0x4000 as a START_EXTENT_BIT)
      Increase the calculated 'head_length' with the number of bytes used for extents.
      Update row->space_on_head_page also in _ma_bitmap_find_new_place()
      Make _ma_bitmap_get_page_bits() global. (Needed for UNDO handling)
      Changed _ma_bitmap_flushable() to take MARIA_HA instead of MARIA_SHARE.
      This was needed to be able to mark the handler if we had a 'non_flushable' call pending or not.
      Don't use 'non_flushable' marking of bitmap pages for not transactional tables.
      Added BLOCKUSED_USE_ORG_BITMAP handling also for tail pages.
      Added more DBUG_ASSERT() to find possible errors in other code
      Some code simplications by adding new local variables
    storage/maria/ma_blockrec.c:
      UNDO of rows now puts back all part of the row on their original pages and positions.
      Changed UNDO of DELETE and UNDO of UPDATE to contain information about the original length of data on head block and also extent information
      This changes a lot of logic as now an insert of a row on a page may happen to any position (and not just to the first or next free)
      Use PAGE_COUNT to mark if an extent is the start of of a blob. (Needed for extent_to_bitmap_blocks())
      Added check_directory() for checking that directroy entries are correct.
      Added checking of row checksums when reading rows (with EXTRA_DEBUG)
      Added make_space_for_directory() and extend_directory() for doing expansion of directory
      Added get_rowpos_in_head_or_tail_page() to be able to store head/tail on original position in UNDO
      Added extent_to_bitmap_blocks() to be able to generate original bitmap blocks from UNDO entry
      Added _ma_update_at_original_place() for UNDO of DELETES
      Added row->min_length to hold minmum required space needed on head page
      Changed find_free_position() to use make_space_for_directory()
      Changed make_empty_page() to allow optional creation of directory entry
      Changed delete_head_or_tail() and _ma_apply_undo_row_isnert() to not copy pagecache block (speed optimization)
      Changed _ma_apply_redo_insert_row_head_or_tail() to be able to insert new row at any position on 'new' page
      Changed _ma_apply_undo_row_delete() and _ma_apply_undo_row_update() to put row in it's original position
      Ensure allocation of tail blocks are of at least MIN_TAIL_SIZE.
      Ensure we store pages in pinned pages even if read failed. (If not we will have pages pinned forever in page cache)
      Write original extent information in UNDO entry, not compacted ones (we need position to tails!)
      When setting BLOCKUSED_USED, don't clear other bits (we have to preserve BLOCKUSED_USE_ORG_BITMAP)
      Fixed som bugs in directory handling
      Fixed bug where we wrote wrong lsn to blob pages
      Added separate blob_buffer for fixing bug when updating row that had char/varchar that spanned several pages and also had blobs
      Ensure we call _ma_bitmap_flushable() also in case of errors
      When doing an update, first delete old entries, then search in bitmap for where to put new information
      Info->s -> share
      Rowid -> rowid
      More DBUG_ASSERT()
    storage/maria/ma_blockrec.h:
      Added START_EXTENT_BIT and TAIL_PAGE_COUNT_MARKER
      Added _ma_bitmap_wait_or_flush() and _ma_bitmap_get_page_bits()
    storage/maria/ma_check.c:
      Don't write extra empty line if there is no deleted blocks
      Ignore START_EXTENT_BIT's in page count
      Call _ma_fast_unlock_key_del() to free key_del link
    storage/maria/ma_close.c:
      Ensure that used_key_del is 0. (If not, someone forgot to call _ma_unlock_key_del())
    storage/maria/ma_create.c:
      Changed constant to macro
    storage/maria/ma_delete.c:
      For deleted keys, log also position to row
    storage/maria/ma_extra.c:
      Release blob buffer at maria_reset() if bigger than MARIA_SMALL_BLOB_BUFFER
    storage/maria/ma_key_recover.c:
      Added bzero() of LSN that confused paged cache in case of uninitialized block
      Mark file crashed if applying of index changes fails
      Added calls to _ma_fast_unlock_key_del() for protection of shared key_del link.
    storage/maria/ma_locking.c:
      Added usage of MARIA_FILE_OPEN_COUNT_OFFSET
      Added _ma_mark_file_crashed()
    storage/maria/ma_loghandler.c:
      Fixed bug where we logged uninitialized memory
    storage/maria/ma_open.c:
      Moved state->changed to be at start of state info on disk to allow one to easly mark files as crashed
    storage/maria/ma_page.c:
      Disable 'dummy' checksumming of pages as this gave false warnings.
      (Need to investigate if this is ever needed)
    storage/maria/ma_pagecache.c:
      Fixed wrong shift that caused Maria to crash on files > 4G
    storage/maria/ma_recovery.c:
      In case of errors, start writing on new line if we where in %## %## printing mode (Made errors more readable)
      Changed global variable name from warnings -> recovery_warnings
      Use MARIA_FILE_CREATE_RENAME_LSN_OFFSET instead of constant
      Removed special handling of row position for deleted keys. Keys now always includes row positions
      _ma_apply_undo_row_delete() now gets page and row position
      Added check that we don't loop forever when handling undo's (in case of bug in undo chain)
      Print name of failed REDO/UNDO
    storage/maria/ma_recovery.h:
      Removed old comment
    storage/maria/ma_static.c:
      Chaned version number of Maria files to not accidently use old ones (becasue of change of ordering of status variables)
    storage/maria/ma_test2.c:
      Added option -u to specify number of rows to update
      Changed old option -u to be -A, as for ma_test1
      Fixed bug in update of rows with blobs (before blobs was always reset to empty on update)
      First created blob is now of max blob length to ensure we have at least one big blob in the table
    storage/maria/ma_test_all.sh:
      More tests
    storage/maria/ma_test_recovery.expected:
      Updated results
    storage/maria/ma_test_recovery:
      Changed tests to use bigger blobs (not just 1K)
      Added new tests that tests recovery of update with blobs
      Removed comparision of .MAD file as it's not guranteed that recovery from scratch gives identical data file as original update
      (compact_page() may be called at different times during normal execution and during REDO)
    storage/maria/ma_update.c:
      Simplify code (changed * to if)
    storage/maria/maria_chk.c:
      Make output more readable
    storage/maria/maria_def.h:
      Changed 'changed' to int to prepare for more bits
      Added 2 more bytes to status information
      Added 'st_mara_row->min_length' for storing min length needed on head page
      Added 'st_mara_handler->blob_buff & blob_buff_size' for storing blobs
      Moved all tunning parameters into one block
      Added MARIA_SMALL_BLOB_BUFFER
      Added _ma_mark_file_crashed()
    storage/myisam/mi_test2.c:
      Fixed bug in update of rows with blobs (before blobs was always reset to empty on update)
    storage/maria/ma_test_big.sh:
      Testing of insert, update, delete, recovery and undo of rows with blobs
      Thanks to the random-ness of ma_test2 this is likely to find most bugs in the row handling
    bfd5c273
maria_chk.c 63.3 KB