An error occurred fetching the project authors.
  1. 16 Oct, 2008 1 commit
    • unknown's avatar
      Fixed ability to read without read lock acquiring. (BUG#39665 related) · b70f7317
      unknown authored
      storage/maria/ma_pagecache.c:
        Fixed ability to read without read lock acquiring.
      storage/maria/unittest/CMakeLists.txt:
        New unit test which tests simple read and prolonged writes consistency added.
      storage/maria/unittest/Makefile.am:
        New unit test which tests simple read and prolonged writes consistency added.
      storage/maria/unittest/ma_pagecache_rwconsist2.c:
        New unit test which tests simple read and prolonged writes consistency added.
      b70f7317
  2. 14 Oct, 2008 1 commit
    • Guilhem Bichot's avatar
      _ma_bitmap_unpin_all() needs to unpin not-locked pages which were pinned by other threads · 8ecda6cd
      Guilhem Bichot authored
      in write_changed_bitmap(), and page cache forbids that. Here we make the page
      cache more relaxed. Original patch by Sanja, simplified by me as limited to
      not-locked. See comment of ma_bitmap.c.
      With that, maria_stress.yy runs until hitting BUG 39665.
      
      storage/maria/ma_bitmap.c:
        A thread which unpins bitmap pages in _ma_bitmap_unpin_all() sometimes
        hit an assertion in the page cache (info!=0 in remove_pin()) which states
        that you can unpin/unlock only what *you* have pinned/locked.
        Fixed by setting the new last parameter of pagecache_unlock_by_link()
        to TRUE in _ma_bitmap_unpin_all().
      storage/maria/ma_blockrec.c:
        new prototype and splitting assertion in three (3rd one fires: BUG 39665)
      storage/maria/ma_check.c:
        new prototype
      storage/maria/ma_key_recover.c:
        new prototype
      storage/maria/ma_loghandler.c:
        new prototype
      storage/maria/ma_pagecache.c:
        Allow a thread to unpin, with pagecache_unlock_by_link(), a non-locked page pinned by others.
        This is a hack for _ma_bitmap_unpin_all() which needs to unpin pages which were
        pinned by other threads in write_changed_bitmap().
      storage/maria/ma_pagecache.h:
        new prototype
      storage/maria/ma_preload.c:
        new prototype
      storage/maria/unittest/ma_pagecache_rwconsist.c:
        new prototype
      storage/maria/unittest/ma_pagecache_single.c:
        new prototype
      8ecda6cd
  3. 25 Aug, 2008 2 commits
    • Michael Widenius's avatar
      Fixed some wrong usage of my_bool · 6629c764
      Michael Widenius authored
      Fixed compiler warning
      Ensure that pagecache returns correct error number
      
      
      storage/maria/ma_pagecache.c:
        Ensure that block->error contain error code from my_errno (not return value from pread/pwrite)
        Went through usage of my_pread/my_pwrite and ensured that result was threated correctly
        Fixed some wrong usage of my_bool
      6629c764
    • Michael Widenius's avatar
      Changed all file names in maria to LEX_STRING and removed some calls to strlen() · 1a5de5bc
      Michael Widenius authored
      Ensure that pagecache gives correct error number even if error for block happend
      
      
      mysys/my_pread.c:
        Indentation fix
      storage/maria/ha_maria.cc:
        filenames changed to be of type LEX_STRING
      storage/maria/ma_check.c:
        filenames changed to be of type LEX_STRING
      storage/maria/ma_checkpoint.c:
        filenames changed to be of type LEX_STRING
      storage/maria/ma_create.c:
        filenames changed to be of type LEX_STRING
      storage/maria/ma_dbug.c:
        filenames changed to be of type LEX_STRING
      storage/maria/ma_delete.c:
        filenames changed to be of type LEX_STRING
      storage/maria/ma_info.c:
        filenames changed to be of type LEX_STRING
      storage/maria/ma_keycache.c:
        filenames changed to be of type LEX_STRING
      storage/maria/ma_locking.c:
        filenames changed to be of type LEX_STRING
      storage/maria/ma_loghandler.c:
        filenames changed to be of type LEX_STRING
      storage/maria/ma_open.c:
        filenames changed to be of type LEX_STRING
      storage/maria/ma_pagecache.c:
        Store error number for last failed operation in the page block
        This should fix some asserts() when errno was not properly set after failure to read block in another thread
      storage/maria/ma_recovery.c:
        filenames changed to be of type LEX_STRING
      storage/maria/ma_update.c:
        filenames changed to be of type LEX_STRING
      storage/maria/ma_write.c:
        filenames changed to be of type LEX_STRING
      storage/maria/maria_def.h:
        filenames changed to be of type LEX_STRING
      storage/maria/maria_ftdump.c:
        filenames changed to be of type LEX_STRING
      storage/maria/maria_pack.c:
        filenames changed to be of type LEX_STRING
      1a5de5bc
  4. 24 Apr, 2008 1 commit
    • unknown's avatar
      WL#3072 - Maria Recovery · c9a82581
      unknown authored
      Recovery of R-tree and fulltext indices.
      Fix for BUG#35551 "Maria: crash in REPAIR TABLE/ENABLE KEYS if using
      repair-with-keycache method".
      Fix for bug (see ma_rt_index.c) where we could have a wrong
      page_link pointer causing wrong memory access during some R-tree
      index insert/delete.
      Making ma_rt_test work again (it had been neglected over time) and
      adding options (record type etc) to prepare it for integration into
      ma_test_all-t (but there is BUG#36321 about "ma_rt_test -M" crash)
      
      
      mysql-test/r/maria.result:
        correct result
      mysql-test/t/maria.test:
        now we get no error
      storage/maria/ma_blockrec.c:
        delete_dir_entry() and delete_head_or_tail() don't use info->keyread_buff.
        ma_get_length() does not change **packet, marking it with 'const' to
        remove some casts in callers of this function. The
        (const uchar**)&header casts will be removed when Monty changes 'header'
        to const uchar*.
        _ma_apply_redo_purge_row_head_or_tail() sets 'buff' from pagecache_read()
        so its initialization was superfluous.
      storage/maria/ma_check.c:
        Fix for BUG#35551 "Maria: crash in REPAIR TABLE/ENABLE KEYS if using repair-with-keycache method"
        (see comment in code)
      storage/maria/ma_create.c:
        FULLTEXT and SPATIAL indices have logging now, they are recoverable.
      storage/maria/ma_delete.c:
        Logging done by _ma_ck_delete() is moved to a function
        (_ma_write_undo_key_delete()), for reusal by R-tree logging.
        _ma_log_delete() is made non-static for same
        reason, and some of its parameters are made pointers to const.
        Removed wrong comment ("Note that for delete key" etc, contradicted by
        code and comment "Log also position to row" a few lines above)
      storage/maria/ma_ft_update.c:
        unneeded cast, comment for future
      storage/maria/ma_key_recover.c:
        Comment about possible deadlock.
        Write bad page to DBUG trace if KEY_OP_CHECK founds bad CRC.
        Support operation KEY_OP_MULTI_COPY.
        When we execute, in UNDO phase, UNDO_KEY_DELETE|INSERT, we must call
        the proper key insertion|deletion depending on if this is R-tree
        or B-tree.
        Explanation of of _ma_[un]lock_key_del() work, maybe useful for
        mortals like me.
      storage/maria/ma_key_recover.h:
        change of prototypes
      storage/maria/ma_loghandler.h:
        New operation which can be stored in REDO_INDEX log records: KEY_OP_MULTI_COPY
      storage/maria/ma_page.c:
        Comments
      storage/maria/ma_pagecache.c:
        typo
      storage/maria/ma_rt_index.c:
        Fix for bug: the page_link pointer in maria_rtree_insert_req()
        could be wrong when we set its 'changed' member; for the solution
        see ma_key_recover.h. It is needed only in cases when we manipulate
        several pages.
        Logging of changes done to pages by key insert/delete.
        maria_rtree_delete()'s main work is moved to a new function
        maria_rtree_real_delete(), which is used by maria_rtree_delete()
        and by applying of UNDO_KEY_INSERT.
      storage/maria/ma_rt_index.h:
        new prototypes and macros for ma_rt_index.c
      storage/maria/ma_rt_key.c:
        Logging of maria_rtree_add_key() and maria_rtree_delete_key().
        When inserting, split is necessary if there is not enough room for key:
        take checksum's occupied space in this calculation.
      storage/maria/ma_rt_key.h:
        new prototypes (those functions need to know the page's id
        because they do logging)
      storage/maria/ma_rt_mbr.c:
        Comments about what the functions change.
      storage/maria/ma_rt_split.c:
        maria_rtree_split_page() needs to know the page's id, because
        it does logging.
        Logging of what a split operation does to the split page (see
        comment of _ma_log_rt_split(): moves of keys inside the page,
        sometimes insertion of the new key, and shrinking of the page)
        and to the new page (receives some keys from split page, and
        sometimes the new key).
      storage/maria/ma_rt_test.c:
        ma_rt_test had been forgotten when maria_rkey() was changed some months ago
        (0->HA_WHOLE_KEY change), and when calls to maria_rnd(,,HA_OFFSET_ERROR)
        were rewritten to maria_scan() calls (which implies maria_scan_init()).
        The 'max_i' change is to adapt to the fact that maria_scan() does
        not return deleted records for BLOCK_RECORD but does so for other formats;
        the initial code assumed a certain number of deleted records would be
        returned, we change it to rather count only non-deleted ones.
        We also add more features to this test, like ma_test1 (the plan
        is to run ma_rt_test in ma_test_all-t):
        options to choose records' format, table checksum, transactions,
        checkpoints, end at specific stages, abort without committing,
        and debug trace.
      storage/maria/ma_test1.c:
        MY_INIT() does my_init().
      storage/maria/ma_write.c:
        Logging done by _ma_ck_write_btree_with_log() is moved to a function
        (_ma_write_undo_key_insert()), for reusal by R-tree logging.
        _ma_log_new() and _ma_log_change() are made non-static for same
        reason. Some parameters of logging functions are made pointers to const.
        If EXTRA_DEBUG_KEY_CHANGES, we now log CRC in _ma_log_change() too
        (better checks, bigger record).
      storage/maria/maria_read_log.c:
        Program takes no arguments, bail out if any, instead of silently discarding them
      storage/myisam/rt_test.c:
        rt_test had been forgotten when mi_rkey() was changed some months ago
        (0->HA_WHOLE_KEY change).
        The 'max_i' change is to make it symmetric with ma_rt_test.c
      mysql-test/r/maria-gis-rtree-dynamic.result:
        correct result
      mysql-test/r/maria-gis-rtree-trans.result:
        correct result
      mysql-test/r/maria-recovery-rtree-ft.result:
        almost correct result (hitting BUG# in the end)
      mysql-test/t/maria-gis-rtree-dynamic.test:
        test R-tree & dynamic row format
      mysql-test/t/maria-gis-rtree-trans.test:
        Test R-tree and page row format and transactional
      mysql-test/t/maria-recovery-rtree-ft-master.opt:
        usual options for recovery testing
      mysql-test/t/maria-recovery-rtree-ft.test:
        test of recovery of R-tree and fulltext indices.
      c9a82581
  5. 11 Mar, 2008 1 commit
    • unknown's avatar
      Fixed statistic calculation. · f0da1ea6
      unknown authored
      (BUG#35030 maria_pagecache_read* status
      variables updated at wrong moment)
      
      
      storage/maria/ma_pagecache.c:
        Fixed statistic calculation.
      f0da1ea6
  6. 05 Mar, 2008 1 commit
  7. 04 Mar, 2008 2 commits
    • unknown's avatar
      Fixed problem of deleting blocks which are being evicted at · 7a6342c2
      unknown authored
      the moment. (BUG#34634)
      Fixed potential bug in pinning schema.
      
      
      storage/maria/ma_pagecache.c:
        Fixed problem of deleting blocks which are being evicted at
        the moment.
        Fixed potential bug in pinning schema.
      7a6342c2
    • unknown's avatar
      copyright and license info · ea3ae3c4
      unknown authored
      storage/maria/Makefile.am:
        copyright
      storage/maria/ma_pagecache.c:
        copyright
      storage/maria/ma_pagecache.h:
        copyright
      storage/maria/ma_pagecaches.c:
        copyright
      storage/maria/ma_pagecrc.c:
        copyright
      storage/maria/unittest/Makefile.am:
        copyright
      storage/maria/unittest/lockman-t.c:
        copyright
      storage/maria/unittest/lockman1-t.c:
        copyright
      storage/maria/unittest/lockman2-t.c:
        copyright
      storage/maria/unittest/ma_control_file-t.c:
        copyright
      storage/maria/unittest/ma_loghandler_examples.c:
        copyright
      storage/maria/unittest/ma_maria_log_cleanup.c:
        copyright
      storage/maria/unittest/ma_pagecache_consist.c:
        copyright
      storage/maria/unittest/ma_pagecache_rwconsist.c:
        copyright
      storage/maria/unittest/ma_pagecache_single.c:
        copyright
      storage/maria/unittest/ma_test_loghandler-t.c:
        copyright
      storage/maria/unittest/ma_test_loghandler_first_lsn-t.c:
        copyright
      storage/maria/unittest/ma_test_loghandler_max_lsn-t.c:
        copyright
      storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
        copyright
      storage/maria/unittest/ma_test_loghandler_multithread-t.c:
        copyright
      storage/maria/unittest/ma_test_loghandler_noflush-t.c:
        copyright
      storage/maria/unittest/ma_test_loghandler_nologs-t.c:
        copyright
      storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
        copyright
      storage/maria/unittest/ma_test_loghandler_purge-t.c:
        copyright
      storage/maria/unittest/test_file.c:
        copyright
      storage/maria/unittest/test_file.h:
        copyright
      storage/maria/unittest/trnman-t.c:
        copyright
      ea3ae3c4
  8. 03 Mar, 2008 2 commits
  9. 26 Feb, 2008 1 commit
    • unknown's avatar
      WL#3072 Maria Recovery · b01c9528
      unknown authored
      fixes for ma_test_recovery.pl to work in release builds too:
      - bugfix in maria_zerofill_index()
      - applying of LOGREC_INSERT_ROW_BLOBS now zeroes unused end of non-full
      blob page (a mutation of tail page when it takes >75% of maria_block_size)
      like write_full_pages() does.
      
      
      storage/maria/ma_blockrec.c:
        When we write a non-full blob page at run-time, we zero the rest of
        it (see write_full_pages()). We now do the same in
        _ma_apply_redo_insert_row_blobs(): this is consistent and helps
        having log-applying produce the same page as run-time.
      storage/maria/ma_check.c:
        maria_zerofill_index() was wrong: it didn't zero certain bytes
        because it believed that the count of relevant bytes is
        _ma_get_page_used(share, buff) + share->keypage_header,
        whereas it's only the first term.
      storage/maria/ma_pagecache.c:
        typo
      storage/maria/ma_pagecache.h:
        typo
      storage/maria/maria_chk.c:
        enable --zerofill-keep-lsn in all builds
      storage/maria/unittest/ma_test_recovery.pl:
        now even release builds can run ma_test_recovery.pl, because zerofill
        makes tables of this test (made by run-time and by log-applying)
        identical.
      b01c9528
  10. 22 Feb, 2008 1 commit
    • unknown's avatar
      Locking for read and write made waiting for each other (with loose · 190de95f
      unknown authored
      scheme for the same thread locking).
      
      
      include/my_pthread.h:
        Added variable for lock diffirentiation.
      include/wqueue.h:
        New release call.
      mysys/wqueue.c:
        New release call in case of read/write lock.
      storage/maria/ma_pagecache.c:
        Locking for read and write made waitimg for each other.
      storage/maria/unittest/Makefile.am:
        New test added.
      storage/maria/unittest/ma_pagecache_consist.c:
        Fixed thread initialization in the test.
      storage/maria/unittest/ma_pagecache_rwconsist.c:
        New BitKeeper file ``storage/maria/unittest/ma_pagecache_rwconsist.c''
      190de95f
  11. 21 Feb, 2008 1 commit
    • unknown's avatar
      fix for bug in Solaris 10 Sparc 64 bit (misaligned buffer in the · e068ad62
      unknown authored
      log handler).
      
      
      storage/maria/ma_loghandler.c:
        Buffers on the stack must be properly aligned as pagecache_read()
        will use bmove512() on them. Fixes hang in maria-recovery.test
        on Solaris 10 Sparc 64 bit.
      storage/maria/ma_pagecache.c:
        comment
      e068ad62
  12. 08 Feb, 2008 1 commit
    • unknown's avatar
      BUG#34089 "Maria crash on LOAD INDEX after FLUSH TABLES" · c5abb8f8
      unknown authored
      a typo had not disable LOAD INDEX for Maria.
      BUG#34161 "freeing maria tmp table after fetching rows using prepared statements - crashes"
      running the testcase I was not able to repeat the same crash but
      found a misplaced assertion giving false positive, fixed here.
      
      
      storage/maria/ha_maria.cc:
        important typo
      storage/maria/ma_pagecache.c:
        The assertion on block's type should be after read_block(). Otherwise
        it can give false alarms.
      c5abb8f8
  13. 07 Feb, 2008 1 commit
    • unknown's avatar
      Mac compiler warnings fixed. · 311d8e89
      unknown authored
      include/maria.h:
        mac compiler warnings fixed.
      storage/maria/ma_bitmap.c:
        mac compiler warnings fixed.
      storage/maria/ma_blockrec.c:
        mac compiler warnings fixed.
      storage/maria/ma_check.c:
        mac compiler warnings fixed.
      storage/maria/ma_control_file.c:
        mac compiler warnings fixed.
      storage/maria/ma_create.c:
        mac compiler warnings fixed.
      storage/maria/ma_delete.c:
        mac compiler warnings fixed.
      storage/maria/ma_ft_boolean_search.c:
        mac compiler warnings fixed.
      storage/maria/ma_page.c:
        mac compiler warnings fixed.
      storage/maria/ma_pagecache.c:
        mac compiler warnings fixed.
      storage/maria/ma_recovery.c:
        mac compiler warning fixed.
      storage/maria/ma_rt_test.c:
        mac compiler warnings fixed.
      storage/maria/ma_search.c:
        mac compiler warning fixed.
      storage/maria/ma_write.c:
        mac compiler warnings fixed.
      storage/maria/unittest/ma_control_file-t.c:
        mac compiler warnings fixed.
      storage/maria/unittest/ma_pagecache_consist.c:
        mac compiler warnings fixed.
      storage/maria/unittest/ma_pagecache_single.c:
        mac compiler warnings fixed.
      storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
        mac compiler warning fixed.
      storage/maria/unittest/test_file.c:
        mac compiler warning fixed.
      311d8e89
  14. 06 Feb, 2008 1 commit
  15. 31 Jan, 2008 2 commits
    • unknown's avatar
      Minor changes. New description in SHOW ENGINES for Maria. · 7323df78
      unknown authored
      Test for BUG#34106 "auto_increment is reset to 1 when table is recovered from crash"
      (fixed by Monty yesterday)
      
      
      mysql-test/r/maria-recovery.result:
        result, which is correct (before pulling Monty's fix for BUG#34106,
        we got a warning about auto_increment in CHECK TABLE (done in
        maria-verify-recovery.inc), no AUTO_INCREMENT clause in
        SHOW CREATE TABLE, and a failure of the last INSERT.
      mysql-test/r/maria.result:
        result
      mysql-test/t/maria-recovery.test:
        Test for BUG#34106
      mysql-test/t/maria.test:
        look at what is reported in SHOW ENGINES
      mysys/my_pread.c:
        changed my mind: if Count argument is >4GB, we'll surely see a segfault
        in the pread() call when it tries to read 4GB from memory, so no need
        to print it in ulonglong format (saves a function call).
      mysys/my_read.c:
        changed my mind: if Count argument is >4GB, we'll surely see a segfault
        in the pread() call when it tries to read 4GB from memory, so no need
        to print it in ulonglong format (saves a function call).
      mysys/my_write.c:
        changed my mind: if Count argument is >4GB, we'll surely see a segfault
        in the pread() call when it tries to read 4GB from memory, so no need
        to print it in ulonglong format (saves a function call).
      storage/maria/ha_maria.cc:
        Description representing the current reality. This can be changed later
      storage/maria/ma_page.c:
        When reading the new key_del from a page on disk, if there is a bug
        (like BUG#34062) this key_del could be wrong, we try to catch if it's
        out of the key file.
      storage/maria/ma_pagecache.c:
        - no truncation of page's number in DBUG_PRINT (useful for BUG#34062)
        - page_korr instead of uint5korr
      storage/maria/ma_recovery.c:
        page_korr instead of uint5korr
      storage/maria/plug.in:
        Description representing the current reality. This can be changed later.
      7323df78
    • unknown's avatar
      Fixed portability issue with comparing thread id · 4d1cb991
      unknown authored
      Fixed bug where return value 'error' was not set in case of error in pagecache
      Documented the open LOAD INDEX bug
      
      
      KNOWN_BUGS.txt:
        Added the problem with LOAD INDEX as a known bugs. Will fix this bug later this week
      storage/maria/ma_pagecache.c:
        Fixed portability issue with comparing thread id
        Fixed bug where return value 'error' was not set in case of error
      4d1cb991
  16. 30 Jan, 2008 2 commits
    • unknown's avatar
      The compiler warning fixed. · b51c819c
      unknown authored
      b51c819c
    • unknown's avatar
      Fixed thread specific ID. · 069178d7
      unknown authored
      Added support for delete by link.
      Aded level ("hits") management functions.
      
      
      storage/maria/ma_pagecache.h:
        Added support for delete by link.
        Aded level ("hits") management functions.
      storage/maria/unittest/ma_pagecache_single.c:
        Test of delete by link.
      069178d7
  17. 29 Jan, 2008 2 commits
    • unknown's avatar
      Fix for BUG#34114 "maria_chk reports false error when several tables on · 2fcff898
      unknown authored
      command-line" and BUG#34062 "Maria table corruption on master".
      Use 5 bytes (instead of 4) to store page's number in the checkpoint
      record, to allow bigger table (1PB with maria-block-size=1kB).
      Help pushbuild not run out of memory by moving the portion of
      maria-recovery.test which generates lots of data into a -big.test.
      
      
      mysql-test/r/maria-recovery.result:
        result moved
      mysql-test/t/maria-recovery.test:
        piece which generates much data moved to maria-recovery-big.test
      mysys/my_pread.c:
        To fix BUG#34062, where a 1.1TB file was generated due to a wrong
        pwrite offset, it was useful to not lose precision on 'offset' in
        DBUG_PRINT, so that the crazy value is visible.
      mysys/my_read.c:
        To fix BUG#34062, where a 1.1TB file was generated due to a wrong
        pwrite offset, it was useful to not lose precision on 'offset' in
        DBUG_PRINT, so that the crazy value is visible.
      mysys/my_write.c:
        To fix BUG#34062, where a 1.1TB file was generated due to a wrong
        pwrite offset, it was useful to not lose precision on 'offset' in
        DBUG_PRINT, so that the crazy value is visible.
      storage/maria/ha_maria.cc:
        When starting a bulk insert, we throw away dirty index pages from the
        cache. Unique (non disabled) key insertions thus read out-of-date
        pages from the disk leading to BUG#34062 "Maria table corruption on
        master": a DELETE in procedure viewer_sp() had deleted all rows of
        viewer_tbl2 one by one, putting index page 1 into key_del; that page
        was thrown away at start of INSERT SELECT, then the INSERT SELECT
        needed a page to insert keys, looked at key_del, found 1, read page 1
        from disk, and its out-of-date content was used to set the new value of
        key_del (crazy value of 1TB), then a later insertion needed another
        index page, tried to read page at this crazy offset and failed, leading
        to corruption mark.
        The fix is to destroy out-of-date pages and make the state consistent
        with that, i.e. call maria_delete_all_rows().
      storage/maria/ma_blockrec.c:
        Special hook for UNDO_BULK_INSERT
      storage/maria/ma_blockrec.h:
        special hook for UNDO_BULK_INSERT
      storage/maria/ma_check.c:
        Fix for BUG#34114 "maria_chk reports false error when several tables on
        command-line": if the Nth (on the command line) table was BLOCK_RECORD
        it would start checks by using the param->record_checksum computed by
        checks of table N-1.
      storage/maria/ma_delete_all.c:
        comment
      storage/maria/ma_loghandler.c:
        special hook for UNDO_BULK_INSERT
      storage/maria/ma_page.c:
        comment
      storage/maria/ma_pagecache.c:
        page number is 5 bytes in checkpoint record now (allows bigger tables)
      storage/maria/ma_recovery.c:
        page number is 5 bytes in checkpoint record now
      storage/maria/ma_recovery_util.c:
        page number is 5 bytes now
      storage/maria/ma_write.c:
        typo
      mysql-test/r/maria-recovery-big.result:
        result is correct
      mysql-test/t/maria-recovery-big-master.opt:
        usual options for recovery tests
      mysql-test/t/maria-recovery-big.test:
        Moving out the big blob test to a -big test (it exhausts memory when
        using /dev/shm on certain machines)
      2fcff898
    • unknown's avatar
      Fixed unused branch of page write call. · e13538dc
      unknown authored
      e13538dc
  18. 20 Jan, 2008 1 commit
    • unknown's avatar
      - fix for segfault in rpl_trigger/rpl_found_rows with default engine=maria · 0825c485
      unknown authored
      (fix is keeping the real TRN through a disable_logging/reenable cycle)
      - fix for pagecache assertion failure in ps/type_ranges with default
      engine=maria (fix is in sql_insert.cc)
      - when reenabling logging we must either flush all dirty pages,
      or at least verify (in debug build) that there are none. For example
      a bulk insert with single UNDO_BULK_INSERT must flush them, no matter
      if it uses repair or not (bugfix)
      - UNDO_BULK_INSERT_WITH_REPAIR is also used with repair, changes name
      
      
      mysql-test/r/maria.result:
        tests for bugs fixed
      mysql-test/t/maria.test:
        tests for bugs fixed
      sql/sql_insert.cc:
        Bugfix: even if select_create::prepare() failed to create the 'table' object
        we still have to re-enable logging.
      storage/maria/ha_maria.cc:
        Bugfix: when a transactional table does a bulk insert without
        repair, it still sometimes skips logging of REDOs thus needs a full
        flush and sync at the end. Not if repair is done, as repair does
        it internally already (see end of maria_repair*()).
      storage/maria/ha_maria.h:
        variable now can have 3 states not 2
      storage/maria/ma_bitmap.c:
        name change
      storage/maria/ma_blockrec.c:
        name change
      storage/maria/ma_blockrec.h:
        name change
      storage/maria/ma_check.c:
        * When maria_repair() re-enables logging it does not need to ask for
        a flush&sync as it did it by itself already a few lines before.
        * the log record of bulk insert can be used even without repair
        * disable logging in maria_zerofill(): without that, it puts LSN pages
        in the cache, so when it flushes them it flushes the log; the change
        makes auto-ha_maria::zerofill-if-moved faster (no log flush).
      storage/maria/ma_key_recover.c:
        name change
      storage/maria/ma_loghandler.c:
        name change
      storage/maria/ma_loghandler.h:
        name change
      storage/maria/ma_pagecache.c:
        A function, to check in debug builds that no dirty pages exist for a file.
      storage/maria/ma_pagecache.h:
        new function (nothing in non-debug)
      storage/maria/ma_recovery.c:
        _ma_tmp_disable_logging() sets info->trn to dummy_transaction_object
        when needed now. The changes done here about info->trn are to allow
        a table to retain its original, real TRN through a disable/reenable
        cycle (see replication scenario in _ma_reenable_logging_for_table()).
        When we reenable, we offer the caller to flush and sync the table;
        if the caller doesn't accept our offer, we verify that it's ok
        (no REDOs => no dirty pages are allowed to exist).
      storage/maria/maria_chk.c:
        comment
      storage/maria/maria_def.h:
        new names
      mysql-test/suite/rpl/r/rpl_stm_maria.result:
        result (it used to crash)
      mysql-test/suite/rpl/t/rpl_stm_maria.test:
        Test of replication-specific Maria bug fixed
      0825c485
  19. 19 Jan, 2008 1 commit
    • unknown's avatar
      - fix for bug when creating bitmaps · cd15ea74
      unknown authored
      - fix for bug seen when running test "type_datetime" with Maria
      (wrong data_file_length in maria_repair())
      - fix for bug seen when running test "repair" with Maria
      (myisam_sort_buffer_size was influencing Maria)
      
      
      sql/handler.cc:
        Sounds illogical to store myisam_sort_buffer_size into a structure
        used by all engines. There are only MyISAM and Maria which used 
        sort_buffer_size: they can get their value from their respective
        system variable (myisam|maria_sort_buffer_size). Using MyISAM's value
        for all engines was wrong (myisam_sort_buffer_size influenced Maria).
      sql/handler.h:
        not needed
      storage/maria/ha_maria.cc:
        check_opt->sort_buffer_size was myisam_sort_buffer_size; Maria
        must use maria_sort_buffer_size instead.
      storage/maria/ma_bitmap.c:
        don't use my_chsize() now that Monty re-explained the problem to me :)
      storage/maria/ma_check.c:
        making maria_repair() work like maria_repair_by_sort(): sort_param.filepos
        must be set at start then possibly corrected by create_new_data_handle();
        in the opposite order, filepos is finally set to 0, and if the table
        has no records, it stays 0 and this causes state.data_file_length
        to be 0 which is incorrect for a BLOCK_RECORD table having always
        at least one bitmap page.
      storage/maria/ma_pagecache.c:
        Comments
      storage/myisam/ha_myisam.cc:
        check_opt->sort_buffer_size is gone
      cd15ea74
  20. 11 Jan, 2008 1 commit
    • unknown's avatar
      Fix windows warnings using correct datatypes if possible · 45500a70
      unknown authored
      and casts if not.
      Add optional WITH_MARIA_TMP_TABLES parameter to configure.js. This 
      parameter defaults to true, if  WITH_MARIA_STORAGE_ENGINE is present.
      
      
      CMakeLists.txt:
        Add WITH_MARIA_TMP_TABLES config parameter.
      storage/maria/ma_blockrec.c:
        Fix windows warning - use the correct datatype.
      storage/maria/ma_loghandler.c:
        Fix windows warnings by adding casts.
      storage/maria/ma_pagecache.c:
        Fix windows warning - use the correct datatype.
      storage/maria/ma_recovery.c:
        Fix windows warning by adding casts.
      win/configure.js:
        Add WITH_MARIA_TMP_TABLES. If WITH_MARIA_STORAGE_ENGINE is present,
        it defaults to TRUE. To unset, pass  WITH_MARIA_TMP_TABLES=FALSE to 
        configure.js
      45500a70
  21. 10 Jan, 2008 2 commits
    • unknown's avatar
      Added --loose-skip-maria to MYSQLD_BOOTSTRAP_CMD to get bootstrap.test to work · 266fde77
      unknown authored
      Allow one to run bootstrap even if --skip-maria is used (needed for bootstrap.test)
      Fixed lots of compiler warnings
      
      NOTE:
      maria-big and maria-recover tests failes becasue of bugs in transaction log handling.
      Sanja knows about this and is working on it!
      
      
      mysql-test/mysql-test-run.pl:
        Added --loose-skip-maria to MYSQLD_BOOTSTRAP_CMD to get bootstrap.test to work
      mysql-test/r/maria-recovery.result:
        Updated results
      mysql-test/t/bootstrap.test:
        Removed not needed empty line
      mysql-test/t/change_user.test:
        Fixed results for 32 bit systems
      mysql-test/t/maria-big.test:
        Only run this when you use --big
      mysql-test/t/maria-recovery.test:
        Added test case for recovery with big blobs
      mysys/my_uuid.c:
        Fixed compiler warning
      sql/mysqld.cc:
        Allow one to run bootstrap even if --skip-maria is used (needed for bootstrap.test)
      sql/set_var.cc:
        Compare max_join_size with ULONG_MAX instead of HA_POS_ERROR as we set max_join_size to ULONG_MAX by default
      storage/maria/ma_bitmap.c:
        Added __attribute((unused)) to fix compiler warning
      storage/maria/ma_blockrec.c:
        Added casts to remove compiler warnings
        Change variable types to avoid compiler warnings
      storage/maria/ma_check.c:
        Added casts to remove compiler warnings
      storage/maria/ma_checkpoint.c:
        Change variable types to avoid compiler warnings
      storage/maria/ma_create.c:
        Change variable types to avoid compiler warnings
      storage/maria/ma_delete.c:
        Added casts to remove compiler warnings
      storage/maria/ma_key_recover.c:
        Added casts to remove compiler warnings
      storage/maria/ma_loghandler.c:
        Moved initiazation of prev_buffer first as this could otherwise not be set in case of errors
      storage/maria/ma_page.c:
        Added casts to remove compiler warnings
      storage/maria/ma_pagecache.c:
        Added __attribute((unused)) to fix compiler warning
      storage/maria/ma_pagecrc.c:
        Added #ifndef DBUG_OFF to remove compiler warning
      storage/maria/ma_recovery.c:
        Added casts to remove compiler warnings
      storage/maria/ma_write.c:
        Added casts to remove compiler warnings
      storage/maria/maria_chk.c:
        Split long string into two to avoid compiler warnings
      storage/myisam/ft_boolean_search.c:
        Added LINT_INIT() to remove compiler warning
      support-files/compiler_warnings.supp:
        Suppress wrong compiler warning
      unittest/mytap/tap.c:
        Fixed declaration to match prototypes to remove compiler warnings
      266fde77
    • unknown's avatar
      Added --with-maria-tmp-tables (default one) to allow on to configure if Maria... · 5ad477f6
      unknown authored
      Added --with-maria-tmp-tables (default one) to allow on to configure if Maria should be used for internal temporary tables
      Abort mysqld if Maria engine didn't start and we are using Maria for temporary tables
      Fixed bug that caused update of big blobs to crash
      Use pagecache_page_no_t as type for pages (to get rid of compiler warnings)
      Added cast to get rid of compiler warning
      Fixed wrong types of variables and arguments that caused lost information
      Fixed wrong DBUG_ASSERT() that caused REDO of big blobs to fail
      Removed some historical ifdefs that caused problem with windows compilations
      
      
      BUILD/SETUP.sh:
        Added --with-maria-tmp-tables
      include/maria.h:
        Use pagecache_page_no_t as type for pages
        Use my_bool as parameter for 'rep_quick' option
      include/my_base.h:
        Added comment
      mysql-test/r/maria-big.result:
        Added test that uses big blobs
      mysql-test/t/maria-big.test:
        Added test that uses big blobs
      sql/mysqld.cc:
        Abort mysqld if Maria engine didn't start and we are using Maria for temporary tables
      sql/sql_class.h:
        Don't use Maria for temporary tables if --with-maria-tmp-tables is not defined
      sql/sql_select.cc:
        Don't use Maria for temporary tables if --with-maria-tmp-tables is not defined
      storage/maria/ha_maria.cc:
        Fixed compiler warnings reported by MCC
        - Fixed usage of wrong types that caused data loss
        - Changed parameter for rep_quick to my_bool
        - Added safe casts
        
        Fixed indentation
      storage/maria/ma_bitmap.c:
        Use pagecache_page_no_t as type for pages
        Fixed compiler warnings
        Fixed bug that caused update of big blobs to crash
      storage/maria/ma_blockrec.c:
        Use pagecache_page_no_t as type for pages
        Use my_bool as parameter for 'rep_quick' option
        Fixed compiler warnings
        Fixed wrong DBUG_ASSERT()
      storage/maria/ma_blockrec.h:
        Use pagecache_page_no_t as type for pages
      storage/maria/ma_check.c:
        Fixed some wrong parameters where we didn't get all bits for test_flag
        Changed rep_quick to be of type my_bool
        Use pagecache_page_no_t as type for pages
        Added cast's to get rid of compiler warnings
        Changed type of record_pos to get rid of compiler warning
      storage/maria/ma_create.c:
        Added safe cast's to get rid of compiler warnings
      storage/maria/ma_dynrec.c:
        Fixed usage of wrong type
      storage/maria/ma_key.c:
        Fixed compiler warning
      storage/maria/ma_key_recover.c:
        Use pagecache_page_no_t as type for pages
      storage/maria/ma_loghandler_lsn.h:
        Added cast's to get rid of compiler warnings
      storage/maria/ma_page.c:
        Changed variable name from 'page' to 'pos' as it was an offset and not a page address
        Moved page_size inside block to get rid of compiler warning
      storage/maria/ma_pagecache.c:
        Fixed compiler warnings
        Replaced compile time assert with TODO
      storage/maria/ma_pagecache.h:
        Use pagecache_page_no_t as type for pages
      storage/maria/ma_pagecrc.c:
        Allow bitmap pages that is all zero
      storage/maria/ma_preload.c:
        Added cast to get rid of compiler warning
      storage/maria/ma_recovery.c:
        Changed types to get rid of compiler warnings
        Use bool for quick_repair to get rid of compiler warning
        Fixed some variables that was wrongly declared (not enough precission)
        Added cast to get rid of compiler warning
      storage/maria/ma_test2.c:
        Remove historical undefs
      storage/maria/maria_chk.c:
        Changed rep_quick to bool
        Fixed wrong parameter to maria_chk_data_link()
      storage/maria/maria_def.h:
        Use pagecache_page_no_t as type for pages
      storage/maria/maria_pack.c:
        Renamed isam -> maria
      storage/maria/plug.in:
        Added option --with-maria-tmp-tables
      storage/maria/trnman.c:
        Added cast to get rid of compiler warning
      storage/myisam/mi_test2.c:
        Remove historical undefs
      5ad477f6
  22. 09 Jan, 2008 1 commit
    • unknown's avatar
      ignore 'moved' state in maria_open() if requested · 8116e16e
      unknown authored
      storage/maria/ma_pagecache.c:
        Change mode to -rw-rw-r--
      storage/maria/ma_open.c:
        ignore 'moved' state if requested
      storage/maria/maria_pack.c:
        ignore 'moved' state in maria_open()
      storage/maria/unittest/ma_test_all-t:
        avoid a warning if $output is undefined
      8116e16e
  23. 08 Jan, 2008 1 commit
    • unknown's avatar
      Added unlocking without unpinning to page cache. · 62d9e3eb
      unknown authored
      storage/maria/ma_pagecache.c:
        Support of unpinning unlocked file.
      storage/maria/unittest/ma_pagecache_single.c:
        Fixed layout.
        Added test of new schema of using pagecache:
          unlocking without unpinning.
      62d9e3eb
  24. 07 Jan, 2008 1 commit
    • unknown's avatar
      Bugs fixed: · b5df1d34
      unknown authored
      - If not in autocommit mode, delete rows one by one so that we can roll back if necessary
      - bitmap->used_size was not correctly set, which caused bitmap pages to be overwritten
      - Fixed bug in bitmap handling when allocation tail pages
      - Ensure we reserve place for directory entry when calculation place for head and tail pages
      - Fixed wrong value in bitmap->size[0]
      - Fixed wrong assert in flush_log_for_bitmap
      - Fixed bug in _ma_bitmap_release_unused() where tail blocks could be wrongly reset
      - Mark new pages as changed (Required to get repair() to work)
      - Fixed problem with advancing log horizon pointer within one page bounds
      - Fixed DBUG_ASSERT() when enable_indexes failes for end_bulk_insert()
      - Fixed bug in logging of rows with more than one big blob
      - Fixed DBUG_ASSERTS() in pagecache to allow change of WRITE_LOCK to READ_LOCK in unlock() calls
      - Flush pagecache when we change from logging to not logging (if not, pagecache code breaks)
      - Ensure my_errno is set on return from write/delete/update
      - Fixed bug when using FIELD_SKIP_PRESPACE
      
      New features:
      - mysql_fix_privilege_tables now first uses binaries and scripts from source distribution, then in installed distribution
      - Fix that optimize works for Maria tables
      - maria_check --zerofill now also clear freed blob pages
      - maria_check -di now prints more information about record page utilization
      
      Optimizations:
      - Use pagecache_unlock_by_link() instead of pagecache_write() if possible. (Avoids a memory copy and a find_block)
      - Simplify code to abort when we found optimal bit pattern
      - Skip also full head page bit patterns when searching for tail
      - Increase default repair buffer to 128M for maria_chk and maria_read_log
      - Increase default sort buffer for maria_chk to 64M
      - Increase size of sortbuffer and pagecache for mysqld to 64M
      - VARCHAR/CHAR fields are stored in increasing length order for BLOCK_RECORD tables
      
      Better reporting:
      - Fixed test of error condition for flush (for better error code)
      - More error messages to mysqld if Maria recovery fails
      - Always print warning if rows are deleted in repair
      - Added global function _db_force_flush() that is usable when doing debugging in gdb
      - Added call to my_debug_put_break_here() in case of some errors (for debugging)
      - Remove used testfiles in unittest as these was written in different directories depending on from where the test was started
      
      This should fix the bugs found when importing a big table with many varchars and one/many blobs to Maria
      
      
      dbug/dbug.c:
        Added global function _db_force_flush() that is usable when doing debugging in gdbine
      extra/replace.c:
        Fixed memory leak
      include/my_dbug.h:
        Prototype for _db_force_flush()
      include/my_global.h:
        Added stdarg.h as my_sys.h now depends on it.
      include/my_sys.h:
        Make my_dbug_put_break_here() a NOP if not DBUG build
        Added my_printv_error()
      include/myisamchk.h:
        Added entry 'lost' to be able to count space that is lost forever
      mysql-test/r/maria.result:
        Updated results
      mysql-test/t/maria.test:
        Reset autocommit after test
        New test to check if delete_all_rows is used (verified with --debug)
      mysys/my_error.c:
        Added my_printv_error()
      scripts/mysql_fix_privilege_tables.sh:
        First use binaries and scripts from source distribution, then in installed distribution
        This ensures that a development branch doesn't pick up wrong scripts)
      sql/mysqld.cc:
        Fix that one can break maria recovery with ^C when debugging
      sql/sql_class.cc:
        Removed #ifdef that has no effect
        (The preceeding DBUG_ASSERT() ensures that the following code will not be exectued)
      storage/maria/ha_maria.cc:
        Increase size of sortbuffer and pagecache to 64M
        Fix that optimize works for Maria tables
        Fixed DBUG_ASSERT() when enable_indexes failes for end_bulk_insert()
        If not in autocommit mode, delete rows one by one so that we can roll back if necessary
        Fixed variable comments
      storage/maria/ma_bitmap.c:
        More ASSERTS to detect overwrite of bitmap pages
        bitmap->used_size was not correctly set, which caused bitmap pages to be overwritten
        Ensure we reserve place for directory entry when calculation place for head and tail pages
        bitmap->size[0] should not include space for directory entry
        Simplify code to abort when we found optimal bit pattern
        Skip also full head page bit patterns when searching for tail (should speed up some common cases)
        Fixed bug in allocate_tail() when block->used was not aligned on 6 bytes
        Fixed wrong assert in flush_log_for_bitmap
        Fixed bug in _ma_bitmap_release_unused() where tail blocks could be wrongly reset
      storage/maria/ma_blockrec.c:
        Ensure my_errno is set on return
        Fixed not optimal setting of row->min_length if we don't have variable length fields
        Use pagecache_unlock_by_link() instead of pagecache_write() if possible. (Avoids a memory copy and a find_block)
        Added DBUG_ASSERT() if we read or write wrong VARCHAR data
        Added DBUG_ASSERT() to find out if row sizes are calculated wrong
        Fixed bug in logging of rows with more than one big blob
      storage/maria/ma_check.c:
        Disable logging while normal repair is done to avoid logging of index changes
        Fixed bug that caused CHECKSUM part of key page to be used
        Fixed that deleted of wrong records also works for BLOCK_RECORD
        Clear unallocated pages:
        - BLOB pages are not automaticly cleared on delete, so we need to use the bitmap to know if page is used or not
        Better error reporting
        More information about record page utilization
        Change printing of file position to printing of pages to make output more readable
        Always print warning if rows are deleted
      storage/maria/ma_create.c:
        Calculate share.base_max_pack_length more accurately for BLOCK_RECORD pages (for future)
        Fixed that FIELD_SKIP_PRESPACE is recorded as FIELD_NORMAL; Fixed bug where fields could be used in wrong order
        Store FIELD_SKIP_ZERO fields before CHAR and VARCHAR fields (optimization)
        Store other fields in length order (to get better utilization of head block)
      storage/maria/ma_delete.c:
        Ensure my_errno is set on return
      storage/maria/ma_dynrec.c:
        Indentation fix
      storage/maria/ma_locking.c:
        Set changed if open_count is counted down.
        (To avoid getting error "client is using or hasn't closed the table properly" with transactional tables
      storage/maria/ma_loghandler.c:
        Fixed problem with advancing log horizon pointer within one page bounds (Patch from Sanja)
        Added more DBUG
        Indentation fixes
      storage/maria/ma_open.c:
        Removed wrong casts
      storage/maria/ma_page.c:
        Fixed usage of PAGECACHE_LOCK_WRITE_UNLOCK with _ma_new()
        Mark new pages as changed (Required to get repair() to work)
      storage/maria/ma_pagecache.c:
        Fixed test of error condition for flush
        Fixed problem when using PAGECACHE_LOCK_WRITE_TO_READ with unlock()
        Added call to my_debug_put_break_here() in case of errors (for debugging)
      storage/maria/ma_pagecrc.c:
        Ensure we get same crc for 32 and 64 bit systems by forcing argument to maria_page_crc to uint32
      storage/maria/ma_recovery.c:
        Call my_printv_error() from eprint() to get critical errors to mysqld log
        Removed \n from error strings to eprint() to get nicer output in mysqld
        Added simple test in _ma_reenable_logging_for_table() to not do any work if not needed
      storage/maria/ma_update.c:
        Ensure my_errno is set on return
      storage/maria/ma_write.c:
        Ensure my_errno is set on return
      storage/maria/maria_chk.c:
        Use DEBUGGER_OFF if --debug is not use (to get slightly faster execution for debug binaries)
        Added option --skip-safemalloc
        Don't write exponents for rec/key
      storage/maria/maria_def.h:
        Increase default repair buffer to 128M for maria_chk and maria_read_log
        Increase default sort buffer for maria_chk to 64M
      storage/maria/unittest/Makefile.am:
        Don't update files automaticly from bitkeeper
      storage/maria/unittest/ma_pagecache_consist.c:
        Remove testfile at end
      storage/maria/unittest/ma_pagecache_single.c:
        Remove testfile at end
      storage/maria/unittest/ma_test_all-t:
        More tests
        Safer checking if test caused error
      b5df1d34
  25. 02 Jan, 2008 1 commit
    • unknown's avatar
      Disable logging of index pages during repair · f8b3e118
      unknown authored
      Fixed failure in unittest/ma_test_loghandler_pagecache-t
      Initialize pagecache callbacks explictily, not with pagecache_init().
      This is to make things more readable and for the future to make more choices with callbacks
      
      
      storage/maria/ha_maria.cc:
        Disable logging of index pages during repair
      storage/maria/ma_bitmap.c:
        Initialize callbacks explictily, not with pagecache_init(), to make things more readable and for future to have more choices with callbacks
        Use new interface to flush logs from pagecache
      storage/maria/ma_check.c:
        Fixed test for wrong keyblocks
        Use default functions to setup callbacks for pagecache
      storage/maria/ma_loghandler.c:
        Use dummy functions for log flush callback (NULL doesn't work anymore)
      storage/maria/ma_open.c:
        Initialize callbacks explictily, not with pagecache_init(), to make things more readable and for future to have more choices with callbacks
        Prefix external functions with _ma_
      storage/maria/ma_pagecache.c:
        Use new simpler interface to flush logs if needed
      storage/maria/ma_pagecache.h:
        Changed interface to a faster, simpler one to flush logs.
        Now we have a function that takes care of flushing logs, instead of a function to get lsn address
      storage/maria/ma_pagecrc.c:
        Add functions for flushing logs
      storage/maria/ma_recovery.c:
        Rename functions
      storage/maria/maria_chk.c:
        Use default functions to setup callbacks for pagecache
      storage/maria/maria_def.h:
        Prefixd global functions with _ma_
      storage/maria/unittest/ma_pagecache_consist.c:
        Use dummy functions for log flush callback (NULL doesn't work anymore)
      storage/maria/unittest/ma_pagecache_single.c:
        Use dummy functions for log flush callback (NULL doesn't work anymore)
      storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
        Use maria_flush_log_for_page to flush log pages. Fixes failure in unittest
      f8b3e118
  26. 01 Jan, 2008 1 commit
    • unknown's avatar
      after-merge fixes and comments · 98aad88f
      unknown authored
      mysql-test/include/maria_empty_logs.inc:
        At one moment in maria-recovery.test the first log has number 2,
        because log 1 was manually deleted.
      mysql-test/r/maria-recovery.result:
        after-merge fix
      mysql-test/t/maria-recovery.test:
        after-merge fix
      storage/maria/ma_bitmap.c:
        after-merge fix. The todo is implemented now.
      storage/maria/ma_blockrec.c:
        comment
      storage/maria/ma_open.c:
        after-merge fix. Set write_fail also for index file or a write error
        would crash.
      storage/maria/ma_pagecache.c:
        comment
      storage/maria/ma_pagecache.h:
        I prefer to use NULL for 'no callback' instead of a dummy callback
        in the special case of get_log_address; indeed for non-transactional
        tables it uses an if(), while if using a dummy callback, it would
        use a function call plus an if() (the dummy callback would need to
        return a magic value to say "don't flush" and that value would be
        tested in if()).
      storage/maria/unittest/ma_test_all-t:
        fix if running from outside storage/maria
      98aad88f
  27. 30 Dec, 2007 2 commits
    • unknown's avatar
      UNDO of rows now puts back all part of the row on their original pages and positions · bfd5c273
      unknown authored
      Added variable _dbug_on_ to speed up execution when DBUG is not going to be used
      Added --debug-on option to mysqld (to be able to turn of DBUG with --debug-on=0)
      Fixed some bugs with 'non_flushable' marking of bitmap pages
      Don't use 'non_flushable' marking of bitmap pages for not transactional tables
      SHOW CREATE TABLE now shows if table was created with page checksums
      Fixed a lot of bugs with BLOB handling in case of update/REDO and UNDO
      More tests (especially for blobs) and DBUG_ASSERTS()
      More readable output from maria_read_log and maria_chk
      Fixed wrong shift that caused Maria to crash on files > 4G
      Mark tables as crashed of REDO fails
      
      
      dbug/dbug.c:
        Changed to use my_bool (allowed me to remove some windows specific code)
        Added variable _dbug_on_ to speed up execution when DBUG is not going to be used
        Removed initialization of variables if not needed
      include/my_dbug.h:
        Use my_bool for some functions that was defined as BOOLEAN in dbug.c code
        Added DBUGGER_ON/DEBUGGER_OFF to speed up execution when DBUG is not used
      include/my_global.h:
        Define my_bool early
        Increase MY_HOW_OFTEN_TO_WRITE as computers are now faster than 10 years ago
      mysql-test/mysql-test-run.pl:
        Added debug-on=0 to speed up tests
      mysql-test/r/maria-recovery.result:
        Added new test by Guilhem to test if UNDO_ROW_DELETE preserves rowid
      mysql-test/r/maria.result:
        Added testing of page checksums
      mysql-test/t/crash_commit_before-master.opt:
        Added --debug-on as test require DBUG to work
      mysql-test/t/maria-recovery-bitmap-master.opt:
        Added --debug-on as test require DBUG to work
      mysql-test/t/maria-recovery-master.opt:
        Added --debug-on as test require DBUG to work
      mysql-test/t/maria-recovery.test:
        Added new test by Guilhem to test if UNDO_ROW_DELETE preserves rowid
      mysql-test/t/maria.test:
        Added testing of page checksums
      sql/mysqld.cc:
        Added --debug-on option (to be able to turn of DBUG with --debug-on=0)
        Indentation fixes
        Removed end spaces
      sql/sql_show.cc:
        Allow update_create_info() to inform MySQL if PACK_KEYS, NO_PACK_KEYS, CHECKSUM, PAGE_CHECKSUM or DELAY_KEY_WRITE is used
      storage/maria/Makefile.am:
        Added ma_test_big.sh
      storage/maria/ha_maria.cc:
        Store in create_info if page checksums are used (For SHOW CREATE TABLE)
      storage/maria/ma_bitmap.c:
        Added _ma_bitmap_wait_or_flush() to cause reader of bitmap pages to wait with reading until bitmap is flushed.
        Use TAIL_PAGE_COUNT_MARKER for tail pages
        Set 'sub_blocks' for and only for the head page or for the first extent of a blob. This is needed for store_extent_info() to be able to set START_EXTENT_BIT's
        Don't allocate more than 0x3ffff pages in one extent (We need bit 0x4000 as a START_EXTENT_BIT)
        Increase the calculated 'head_length' with the number of bytes used for extents.
        Update row->space_on_head_page also in _ma_bitmap_find_new_place()
        Make _ma_bitmap_get_page_bits() global. (Needed for UNDO handling)
        Changed _ma_bitmap_flushable() to take MARIA_HA instead of MARIA_SHARE.
        This was needed to be able to mark the handler if we had a 'non_flushable' call pending or not.
        Don't use 'non_flushable' marking of bitmap pages for not transactional tables.
        Added BLOCKUSED_USE_ORG_BITMAP handling also for tail pages.
        Added more DBUG_ASSERT() to find possible errors in other code
        Some code simplications by adding new local variables
      storage/maria/ma_blockrec.c:
        UNDO of rows now puts back all part of the row on their original pages and positions.
        Changed UNDO of DELETE and UNDO of UPDATE to contain information about the original length of data on head block and also extent information
        This changes a lot of logic as now an insert of a row on a page may happen to any position (and not just to the first or next free)
        Use PAGE_COUNT to mark if an extent is the start of of a blob. (Needed for extent_to_bitmap_blocks())
        Added check_directory() for checking that directroy entries are correct.
        Added checking of row checksums when reading rows (with EXTRA_DEBUG)
        Added make_space_for_directory() and extend_directory() for doing expansion of directory
        Added get_rowpos_in_head_or_tail_page() to be able to store head/tail on original position in UNDO
        Added extent_to_bitmap_blocks() to be able to generate original bitmap blocks from UNDO entry
        Added _ma_update_at_original_place() for UNDO of DELETES
        Added row->min_length to hold minmum required space needed on head page
        Changed find_free_position() to use make_space_for_directory()
        Changed make_empty_page() to allow optional creation of directory entry
        Changed delete_head_or_tail() and _ma_apply_undo_row_isnert() to not copy pagecache block (speed optimization)
        Changed _ma_apply_redo_insert_row_head_or_tail() to be able to insert new row at any position on 'new' page
        Changed _ma_apply_undo_row_delete() and _ma_apply_undo_row_update() to put row in it's original position
        Ensure allocation of tail blocks are of at least MIN_TAIL_SIZE.
        Ensure we store pages in pinned pages even if read failed. (If not we will have pages pinned forever in page cache)
        Write original extent information in UNDO entry, not compacted ones (we need position to tails!)
        When setting BLOCKUSED_USED, don't clear other bits (we have to preserve BLOCKUSED_USE_ORG_BITMAP)
        Fixed som bugs in directory handling
        Fixed bug where we wrote wrong lsn to blob pages
        Added separate blob_buffer for fixing bug when updating row that had char/varchar that spanned several pages and also had blobs
        Ensure we call _ma_bitmap_flushable() also in case of errors
        When doing an update, first delete old entries, then search in bitmap for where to put new information
        Info->s -> share
        Rowid -> rowid
        More DBUG_ASSERT()
      storage/maria/ma_blockrec.h:
        Added START_EXTENT_BIT and TAIL_PAGE_COUNT_MARKER
        Added _ma_bitmap_wait_or_flush() and _ma_bitmap_get_page_bits()
      storage/maria/ma_check.c:
        Don't write extra empty line if there is no deleted blocks
        Ignore START_EXTENT_BIT's in page count
        Call _ma_fast_unlock_key_del() to free key_del link
      storage/maria/ma_close.c:
        Ensure that used_key_del is 0. (If not, someone forgot to call _ma_unlock_key_del())
      storage/maria/ma_create.c:
        Changed constant to macro
      storage/maria/ma_delete.c:
        For deleted keys, log also position to row
      storage/maria/ma_extra.c:
        Release blob buffer at maria_reset() if bigger than MARIA_SMALL_BLOB_BUFFER
      storage/maria/ma_key_recover.c:
        Added bzero() of LSN that confused paged cache in case of uninitialized block
        Mark file crashed if applying of index changes fails
        Added calls to _ma_fast_unlock_key_del() for protection of shared key_del link.
      storage/maria/ma_locking.c:
        Added usage of MARIA_FILE_OPEN_COUNT_OFFSET
        Added _ma_mark_file_crashed()
      storage/maria/ma_loghandler.c:
        Fixed bug where we logged uninitialized memory
      storage/maria/ma_open.c:
        Moved state->changed to be at start of state info on disk to allow one to easly mark files as crashed
      storage/maria/ma_page.c:
        Disable 'dummy' checksumming of pages as this gave false warnings.
        (Need to investigate if this is ever needed)
      storage/maria/ma_pagecache.c:
        Fixed wrong shift that caused Maria to crash on files > 4G
      storage/maria/ma_recovery.c:
        In case of errors, start writing on new line if we where in %## %## printing mode (Made errors more readable)
        Changed global variable name from warnings -> recovery_warnings
        Use MARIA_FILE_CREATE_RENAME_LSN_OFFSET instead of constant
        Removed special handling of row position for deleted keys. Keys now always includes row positions
        _ma_apply_undo_row_delete() now gets page and row position
        Added check that we don't loop forever when handling undo's (in case of bug in undo chain)
        Print name of failed REDO/UNDO
      storage/maria/ma_recovery.h:
        Removed old comment
      storage/maria/ma_static.c:
        Chaned version number of Maria files to not accidently use old ones (becasue of change of ordering of status variables)
      storage/maria/ma_test2.c:
        Added option -u to specify number of rows to update
        Changed old option -u to be -A, as for ma_test1
        Fixed bug in update of rows with blobs (before blobs was always reset to empty on update)
        First created blob is now of max blob length to ensure we have at least one big blob in the table
      storage/maria/ma_test_all.sh:
        More tests
      storage/maria/ma_test_recovery.expected:
        Updated results
      storage/maria/ma_test_recovery:
        Changed tests to use bigger blobs (not just 1K)
        Added new tests that tests recovery of update with blobs
        Removed comparision of .MAD file as it's not guranteed that recovery from scratch gives identical data file as original update
        (compact_page() may be called at different times during normal execution and during REDO)
      storage/maria/ma_update.c:
        Simplify code (changed * to if)
      storage/maria/maria_chk.c:
        Make output more readable
      storage/maria/maria_def.h:
        Changed 'changed' to int to prepare for more bits
        Added 2 more bytes to status information
        Added 'st_mara_row->min_length' for storing min length needed on head page
        Added 'st_mara_handler->blob_buff & blob_buff_size' for storing blobs
        Moved all tunning parameters into one block
        Added MARIA_SMALL_BLOB_BUFFER
        Added _ma_mark_file_crashed()
      storage/myisam/mi_test2.c:
        Fixed bug in update of rows with blobs (before blobs was always reset to empty on update)
      storage/maria/ma_test_big.sh:
        Testing of insert, update, delete, recovery and undo of rows with blobs
        Thanks to the random-ness of ma_test2 this is likely to find most bugs in the row handling
      bfd5c273
    • unknown's avatar
      WL#3072 - Maria Recovery · 18bc7b69
      unknown authored
      * to honour WAL we now force the whole log when flushing a bitmap page.
      * ability to intentionally crash in various places for recovery testing
      * bugfix (dirty pages list found in checkpoint record was ignored)
      * smaller checkpoint record
      * misc small cleanups and comments
      
      
      mysql-test/include/maria_empty_logs.inc:
        maria-purge.test creates ~11 logs, remove them all
      mysql-test/r/maria-recovery-bitmap.result:
        result is good; without the _ma_bitmap_get_log_address() call,
        we got
        check   error   Bitmap at 0 has pages reserved outside of data file length
      mysql-test/r/maria-recovery.result:
        result update
      mysql-test/t/maria-recovery-bitmap.test:
        enable test of "bitmap-flush should flush whole log otherwise
        corrupted data file (bitmap ahead of data pages)".
      mysql-test/t/maria-recovery.test:
        test of checkpoint
      sql/sql_table.cc:
        comment
      storage/maria/ha_maria.cc:
        _ma_reenable_logging_for_table() now includes file->trn=0.
        At the end of repair() we don't need to re-enable logging, it is
        done already by caller (like copy_data_between_tables()); it sounds
        strange that this function could decide to re-enable, it should be
        up to caller who knows what other operations it plans. Removing this
        line led to assertion failure in maria_lock_database(F_UNLCK), fixed
        by removing the assertion: maria_lock_database()
        is here called in a context where F_UNLCK does not make the
        table visible to others so assertion is excessive, and external_lock()
        is already designed to honour the asserted condition.
        Ability to crash at the end of bulk insert when indices
        have been enabled.
      storage/maria/ma_bitmap.c:
        Better use pagecache_file_init() than set pagecache callbacks directly;
        and a new function to set those callbacks for bitmap so that we can
        reuse it.
        _ma_bitmap_get_log_address() is a pagecache get_log_address callback
        which causes the whole log to be flushed when a bitmap page
        is flushed by the page cache. This was required by WAL.
      storage/maria/ma_blockrec.c:
        get_log_address pagecache callback for data (non bitmap) pages:
        just reads the LSN from the page's content, like was hard-coded
        before in ma_pagecache.c.
      storage/maria/ma_blockrec.h:
        functions which need to be exported
      storage/maria/ma_check.c:
        create_new_data_handle() can be static.
        Ability to crash after rebuilding the index in OPTIMIZE,
        in REPAIR. my_lock() implemented already.
      storage/maria/ma_checkpoint.c:
        As MARIA_SHARE* is now accessible to pagecache_collect_changed_blocks_LSN(),
        we don't need to store kfile/dfile descriptors in checkpoint record,
        2-byte-id of the table plus one byte to say if this is data or index
        file is enough. So we go from 4+4 bytes per table down to 2+1.
      storage/maria/ma_commit.c:
        removing duplicate functions (see _ma_tmp_disable_logging_for_table())
      storage/maria/ma_extra.c:
        Monty fixed
      storage/maria/ma_key_recover.c:
        comment
      storage/maria/ma_locking.c:
        Sometimes other code does funny things with maria_lock_database(),
        like ha_maria::repair() calling it at start and end without going
        through ha_maria::external_lock(). So it happens that maria_lock_database()
        is called with now_transactional!=born_transactional.
      storage/maria/ma_loghandler.c:
        update to new prototype
      storage/maria/ma_open.c:
        set_data|index_pagecache_callbacks() need to be exported as
        they are now called when disabling/enabling transactionality.
      storage/maria/ma_pagecache.c:
        Removing PAGE_LSN_OFFSET, as much of the code relies on it being
        0 anyway (let's not give impression we can just change this constant).
        When flushing a page to disk, call the get_log_address callback to
        know up to which LSN the log should be flushed.
        As we now can access MARIA_SHARE* we can know share->id and store
        it into the checkpoint record; we thus go from 4 bytes per dirty page
        to 2+1.
      storage/maria/ma_pagecache.h:
        get_log_address callback
      storage/maria/ma_panic.c:
        No reason to reset pagecache callbacks in HA_PANIC_READ:
        all we do is reopen files if they were closed; callbacks should
        be in place already as 'info' exists; we just want to modify
        the file descriptors, not the full PAGECACHE_FILE structure.
        If we open data file and it was closed, share->bitmap.file needs
        to be set.
        Note that the modified code is disabled anyway.
      storage/maria/ma_recovery.c:
        Checkpoint record does not contain kfile/dfile descriptors anymore
        so code can be simplified. Hash key in all_dirty_pages is 
        not made from file_descriptor & pageno anymore, but
        index_or_data & table-short-id & pageno.
        If a table's create_rename_lsn is higher than record's LSN,
        we skip the table and don't fail if it's corrupted (because the LSNs
        say that we don't have to look at this table).
        If a table is skipped (for example due to create_rename_lsn),
        its UNDOs still cause undo_lsn to advance; this is so that if later
        we notice the transaction has to rollback we fail (as table should
        not be skipped in this case).
        Fixing a bug: the dirty_pages list was never used, because
        the LSN below which it was used was the minimum rec_lsn of dirty pages!
        It is now the min(checkpoint_start_log_horizon, min(trn's rec_lsn)).
        When we disable/reenable transactionality, we modify pagecache
        callbacks (needed for example for get_log_address: changing
        share->page_type is not enough anymore).
      storage/maria/ma_write.c:
        'records' and 'checksum' are protected: they are updated under
        log's mutex in write-hooks when UNDO is written.
      storage/maria/maria_chk.c:
        remove use of duplicate functions.
      storage/maria/maria_def.h:
        set_data|index_pagecache_callbacks() need to be exported;
        _ma_reenable_logging_for_table() changes to a real function.
      storage/maria/unittest/ma_pagecache_consist.c:
        new prototype
      storage/maria/unittest/ma_pagecache_single.c:
        new prototype
      storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
        new prototype
      18bc7b69
  28. 20 Dec, 2007 1 commit
  29. 18 Dec, 2007 1 commit
    • unknown's avatar
      Fixed several bugs in page CRC handling · 30d3d8d3
      unknown authored
      - Ignore CRC errors in REDO for potential new pages
      - Ignore CRC errors when repairing tables
      - Don't do readcheck callback on read error
      - Set my_errno to HA_ERR_WRONG_CRC if we find page with wrong CRC
      - Check index page for length before calculating CRC to catch bad pages
      Fixed bugs where we used wrong file descriptor to read/write bitmaps
      Fixed wrong hash key in 'files_in_flush'
      Fixed wrong lock method when writing bitmap
      Fixed some wrong printf statements in check/repair that caused core dumps
      Fixed argument to translog_page_validator that cause reading of log files to fail
      Store number of bytes used for delete-linked key pages to be able to use standard index CRC for deleted key pages.
      Use fast 'dummy' pagecheck callbacks for temporary tables
      Don't die silently if flush finds pinned pages
      Give error (for now) if one tries to create a transactional table with fulltext or spatial keys
      Removed some not needed calls to pagecache_file_init()
      Added checking of pagecache checksums to ma_test1 and ma_test2
      More DBUG
      Fixed some DBUG_PRINT to be in line with rest of the code
      
      
      include/my_base.h:
        Added HA_ERR_INTERNAL_ERROR (used for flush with pinned pages) and HA_ERR_WRONG_CRC
      mysql-test/r/binlog_unsafe.result:
        Added missing DROP VIEW statement
      mysql-test/r/maria.result:
        Added TRANSACTIONAL=0 when testing with fulltext keys
        Added test that verifies we can't yet create transactional test with fulltext or spatial keys
      mysql-test/r/ps_maria.result:
        Added TRANSACTIONAL=0 when testing with fulltext keys
      mysql-test/t/binlog_unsafe.test:
        Added missing DROP VIEW statement
      mysql-test/t/maria.test:
        Added TRANSACTIONAL=0 when testing with fulltext keys
        Added test that verifies we can't yet create transactional test with fulltext or spatial keys
      mysql-test/t/ps_maria.test:
        Added TRANSACTIONAL=0 when testing with fulltext keys
      mysys/my_fopen.c:
        Fd: -> fd:
      mysys/my_handler.c:
        Added new error messages
      mysys/my_lock.c:
        Fd: -> fd:
      mysys/my_pread.c:
        Fd: -> fd:
      mysys/my_read.c:
        Fd: -> fd:
      mysys/my_seek.c:
        Fd: -> fd:
      mysys/my_sync.c:
        Fd: -> fd:
      mysys/my_write.c:
        Fd: -> fd:
      sql/mysqld.cc:
        Fixed wrong argument to my_uuid_init()
      sql/sql_plugin.cc:
        Unified DBUG_PRINT (for convert-dbug-for-diff)
      storage/maria/ma_bitmap.c:
        Fixed wrong lock method when writing bitmap
        Fixed valgrind error
        Use fast  'dummy' pagecheck callbacks for temporary tables
        Faster bitmap handling for non transational tables
      storage/maria/ma_blockrec.c:
        Fixed that bitmap reading is done with the correct filehandle
        Handle reading of pages with wrong CRC when page contect doesn't matter
        Use the page buffer also when we get WRONG CRC or FILE_TOO_SHORT. (Faster and fixed a couple of bugs)
      storage/maria/ma_check.c:
        Split long strings for readablity
        Fixed some wrong printf statements that caused core dumps
        Use bitmap.file for bitmaps
        Ignore pages with wrong CRC
      storage/maria/ma_close.c:
        More DBUG_PRINT
      storage/maria/ma_create.c:
        Give error (for now) if one tries to create a crash safe table with fulltext or spatial keys
      storage/maria/ma_key_recover.c:
        Ignore HA_ERR_WRONG_CRC for new pages
        info->s  ->  share
        Store number of bytes used for delete-linked key pages to be able to use standard index CRC for deleted key pages.
      storage/maria/ma_loghandler.c:
        Fixed argument to translog_page_validator()
      storage/maria/ma_open.c:
        Removed old VMS specific code
        Added function to setup pagecache callbacks
        Moved code around to set 'share->temporary' early
        Removed some not needed calls to pagecache_file_init()
      storage/maria/ma_page.c:
        Store number of bytes used for delete-linked key pages to be able to use standard index CRC for deleted key pages.
      storage/maria/ma_pagecache.c:
        Don't do readcheck callback on read error
        Reset PCBLOCK_ERROR in pagecache_unlock_by_link() if we write page
        Set my_errno to HA_ER_INTERNAL_ERROR if flush() finds pinned pages
        Don't die silently if flush finds pinned pages.
        Use correct file descriptor when flushing pages
        Fixed wrong hash key in 'files_in_flush';  This must be the file descriptor, not the PAGECACHE_FILE as there may be several PAGECACHE_FILE for same file descriptor
        More DBUG_PRINT
      storage/maria/ma_pagecrc.c:
        Removed inline from not tiny static function
        Set my_errno to HA_ERR_WRONG_CRC if we find page with wrong CRC
        (Otherwise my_errno may be 0, and a lot of other code will be confused)
        CRCerror -> error (to keep code uniform)
        Print crc with %lu, as in my_checksum()
        uchar* -> uchar *
        Check index page for length before calculating CRC to catch bad pages
        Added 'dummy' crc_check and filler functions that are used for temporary tables
      storage/maria/ma_recovery.c:
        More DBUG
        More message to users to give information what phase failed
        Better error message if recovery failed
      storage/maria/ma_test1.c:
        Added checking of page checksums (combined with 'c' to not have to add more test runs)
      storage/maria/ma_test2.c:
        Added checking of page checksums (combined with 'c' to not have to add more test runs)
      storage/maria/maria_chk.c:
        Fixed wrong argument to _ma_check_print_error()
      storage/maria/maria_def.h:
        Added format information to _ma_check_print_xxxx functions
        uchar* -> uchar *
      30d3d8d3
  30. 15 Dec, 2007 1 commit
    • unknown's avatar
      Pagecache callbacks support added. · 7b19ba34
      unknown authored
      Page CRC check based on pagecache support added.
      Loghandler pagecache callbacks support added.
      Loghandler filecache rewritten.
      Support of deletting all logs added.
      
      
      storage/maria/Makefile.am:
        New file with functions for CRC calculation.
      storage/maria/ma_bitmap.c:
        Page CRC support.
      storage/maria/ma_blockrec.c:
        Removed code replaced by pagecache callbacks.
      storage/maria/ma_check.c:
        Page CRC support.
      storage/maria/ma_create.c:
        Page CRC support.
      storage/maria/ma_loghandler.c:
        Pagecache callbacks support.
        New file cache support.
        Removing log files support.
      storage/maria/ma_loghandler.h:
        CRC_LENGTH replaced with CRC_SIZE
      storage/maria/ma_open.c:
        Page CRC support.
      storage/maria/ma_page.c:
        Page CRC support.
      storage/maria/ma_pagecache.c:
        Pagecache callbacks support.
      storage/maria/ma_pagecache.h:
        Pagecache callbacks support.
      storage/maria/ma_panic.c:
        Page CRC support.
      storage/maria/maria_chk.c:
        Page CRC support.
      storage/maria/maria_def.h:
        Page CRC support.
      storage/maria/unittest/Makefile.am:
        New test of removing logs.
      storage/maria/unittest/ma_maria_log_cleanup.c:
        Memory leack fixed.
      storage/maria/unittest/ma_pagecache_consist.c:
        Pagecache callbacks support.
      storage/maria/unittest/ma_pagecache_single.c:
        Pagecache callbacks support.
      storage/maria/unittest/ma_test_loghandler-t.c:
        Fixed the test error processing.
      storage/maria/unittest/ma_test_loghandler_first_lsn-t.c:
        Fixed the test error processing.
      storage/maria/unittest/ma_test_loghandler_max_lsn-t.c:
        Fixed the test error processing.
      storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
        Fixed the test error processing.
      storage/maria/unittest/ma_test_loghandler_multithread-t.c:
        Fixed the test error processing.
      storage/maria/unittest/ma_test_loghandler_noflush-t.c:
        Fixed the test error processing.
      storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
        Pagecache callbacks support.
      storage/maria/unittest/ma_test_loghandler_purge-t.c:
        Fixed the test error processing.
      storage/maria/unittest/test_file.c:
        Removed unneeded sync.
      7b19ba34
  31. 14 Dec, 2007 2 commits
    • unknown's avatar
      fix for some gcc -ansi warnings. · 914f219c
      unknown authored
      storage/maria/ma_checkpoint.c:
        gcc -ansi warnings
      storage/maria/ma_pagecache.c:
        comment
      storage/maria/ma_recovery.c:
        gcc -ansi warnings
      914f219c
    • unknown's avatar
      WL#3072 - Maria recovery. · d72c22de
      unknown authored
      * fix for bitmap vs checkpoint bug which could lead to corrupted
      tables in case of crashes at certain moments: a bitmap could be flushed
      to disk even though it was inconsistent with the log (it could be
      flushed before REDO-UNDO are written to the log). One bug remains, need
      code from others. Tests added. Fix is to pin unflushable bitmap pages,
      and let checkpoint wait for them to be flushable.
      * fix for long_trid!=0 assertion failure at Recovery.
      * less useless wakeups in the background flush|checkpoint thread.
      * store global_trid_generator in checkpoint record.
      
      
      mysql-test/r/maria-recovery.result:
        result update
      mysql-test/t/maria-recovery.test:
        make it easier to locate subtests
      storage/maria/ma_bitmap.c:
        When we send a bitmap to the pagecache, if this bitmap is not in a
        flushable state we keep it pinned and add it to a list, it will be
        unpinned when the bitmap is flushable again.
        A new function _ma_bitmap_flush_all() used by checkpoint.
        A new function _ma_bitmap_flushable() used by block format to signal
        when it starts modifying a bitmap and when it is done with it.
      storage/maria/ma_blockrec.c:
        When starting a row operation (insert/update/delete), mark that
        the bitmap is not flushable (because for example INSERT is going
        to over-allocate in the bitmap to prevent other threads from using
        our data pages). If a checkpoint comes at this moment it will wait
        for the bitmap to be flushable before flushing it.
        When the operation ends, bitmap becomes flushable again; that
        transition is done under the bitmap's mutex (needed for correct
        synchro with a concurrent checkpoint); but for INSERT/UPDATE this
        happens inside _ma_bitmap_release_unused() at a place where it already
        has the mutex, so the only penalty (mutex adding) is in DELETE and UNDO
        of INSERT. In case of errors after setting the bitmap unflushable,
        we must always set it back to flushable or checkpoint would block.
        Debug possibilities to force a sleep while the bitmap is over-allocated.
        In case of error in get_head_or_tail() in allocate_and_write_block_record(),
        we still need to unpin all pages.
        Bugfix: _ma_apply_redo_insert_row_blobs() produced wrong
        data_file_length.
      storage/maria/ma_blockrec.h:
        new bitmap calls.
      storage/maria/ma_checkpoint.c:
        filter_flush_indirect not needed anymore (flushing bitmap
        pages happens in _ma_bitmap_flush_all() now). So
        st_filter_param::is_data_file|pages_covered_by_bitmap not needed.
        Other filter_flush* don't need to flush bitmap anymore.
        Add debug possibility to flush all bitmap pages outside of a checkpoint,
        to simulate pagecache LRU eviction.
        When the background flush/checkpoint thread notices it has nothing
        to flush, it now sleeps directly until the next potential checkpoint
        moment instead of waking up every second.
        When in checkpoint we decide to not store a table in the checkpoint record
        (because it has logged no writes for example), we can also skip flushing
        this table.
      storage/maria/ma_commit.c:
        comment is out-of-date
      storage/maria/ma_key_recover.c:
        comment fix
      storage/maria/ma_loghandler.c:
        comment is out-of-date
      storage/maria/ma_open.c:
        comment is out-of-date
      storage/maria/ma_pagecache.c:
        comment for bug to fix. And we don't take checkpoints at end of REDO
        phase yet so can trust block->type.
      storage/maria/ma_recovery.c:
        Comments. Now-unneeded code for incomplete REDO-UNDO groups removed.
        When we forget about an old transaction we must really forget
        about it with bzero() (fixes the "long_trid!=0 assertion" recovery
        bug). When we delete a row with maria_delete() we turn on
        STATE_NOT_OPTIMIZED_ROWS so we do the same when we see a CLR_END
        for an UNDO_ROW_INSERT or when we execute an UNDO_ROW_INSERT (in both
        cases a row was deleted). Pick up max_long_trid from the checkpoint record.
      storage/maria/maria_chk.c:
        comment
      storage/maria/maria_def.h:
        MARIA_FILE_BITMAP gets new members: 'flushable', 'bitmap_cond' and
        'pinned_pages'.
      storage/maria/trnman.c:
        I used to think that recovery only needs to know the maximum TrID
        of the lists of active and committed transactions. But no, sometimes
        both lists can even be empty and their TrID should not be reused.
        So Checkpoint now saves global_trid_generator in the checkpoint record.
      storage/maria/trnman_public.h:
        macros to read/store a TrID
      mysql-test/r/maria-recovery-bitmap.result:
        result is ok. Without the code fix, we would get a corruption message
        about the bitmap page in CHECK TABLE EXTENDED.
      mysql-test/t/maria-recovery-bitmap-master.opt:
        usual when we crash mysqld in tests
      mysql-test/t/maria-recovery-bitmap.test:
        test of recovery problems specific of the bitmap pages.
      d72c22de