1. 17 Jan, 2020 7 commits
    • Marko Mäkelä's avatar
      MDEV-21513: Avoid some crashes in ALTER TABLE...IMPORT TABLESPACE · 08b0b2b6
      Marko Mäkelä authored
      IndexPurge::next(): Replace btr_pcur_move_to_next_user_rec()
      with some equivalent code that performs sanity checks without
      killing the server. Perform some additional sanity checks as well.
      
      This change is motivated by
      mysql/mysql-server@48de4d74f4d2f10cd01b129753c7dfa908cf36b5
      which unnecessarily introduces storage overhead to btr_pcur_t
      and uses a test case that injects a fault somewhere else,
      not in the code path that was modified.
      08b0b2b6
    • Marko Mäkelä's avatar
      MDEV-21512 InnoDB may hang due to SPATIAL INDEX · 457ce97e
      Marko Mäkelä authored
      MySQL 5.7.29 includes the following fix:
      Bug #30287668 INNODB: A LONG SEMAPHORE WAIT
      mysql/mysql-server@5cdbb22b51cf2b35dbdf5666a251ffbec2f84dec
      
      There is no test case. It seems that the problem could occur when
      a spatial index is large and peculiar enough so that multiple R-tree
      leaf pages will have the exactly same maximum bounding rectangle (MBR).
      
      The commit message suggests that the hang can occur when R-tree
      non-leaf pages are being merged, which should only be possible
      during transaction rollback or the purge of transaction history,
      when the R-tree index is at least 2 levels high and very many records
      are being deleted. The message says that a comparison result that two
      spatial index node pointer records are equal will cause an infinite loop
      in rtr_page_copy_rec_list_end_no_locks(). Hence, we must include the
      child page number in the comparison to be consistent with
      mysql/mysql-server@2e11fe0e152e34d73579e1a9ec19aedc3f6010f6.
      
      We fix this bug in a simpler way, involving fewer code changes.
      
      cmp_rec_rec(): Renamed from cmp_rec_rec_with_match().
      Assert that rec2 always resides in an index page.
      Treat non-leaf spatial index pages specially.
      457ce97e
    • Marko Mäkelä's avatar
      MDEV-21511: Remove unnecessary code · c3695b40
      Marko Mäkelä authored
      Now that we will be invoking dtuple_get_n_ext() instead of
      letting btr_push_update_extern_fields() update an already
      calculated value, it is unnecessary to calculate the n_ext
      upfront.
      
      row_rec_to_index_entry(), row_rec_to_index_entry_low():
      Remove the output parameter n_ext.
      c3695b40
    • Marko Mäkelä's avatar
      MDEV-21511 Wrong estimate of affected BLOB columns in update · 5838b527
      Marko Mäkelä authored
      During update, rollback, or MVCC read, we may miscalculate
      the number of off-page columns, and thus the size of the
      clustered index record. The function btr_push_update_extern_fields()
      is mostly redundant, because the off-page columns would also be
      moved by row_upd_index_replace_new_col_val(), which is invoked
      via row_upd_index_replace_new_col_vals().
      
      btr_push_update_extern_fields(): Remove.
      
      This is based on
      mysql/mysql-server@1fa475b85d24de4b9ce2958c0eed738c221fc82c
      which refines a fix for a recovery bug fix
      mysql/mysql-server@ce0a1e85e24e48b8171f767b44330da635a6ea0a
      in MySQL 5.7.5.
      
      No test case was provided by Oracle.
      Some of the changed code is being covered by the existing test
      innodb.blob-crash.
      5838b527
    • Marko Mäkelä's avatar
      MDEV-21509 Possible hang during purge of history, or rollback · 3e38d155
      Marko Mäkelä authored
      WL#6326 in MariaDB 10.2.2 introduced a potential hang on purge or rollback
      when an index tree is being shrunk by multiple levels.
      
      This fix is based on
      mysql/mysql-server@f2c58526300c0d84837effa26d37cbd5d2694967
      with the main difference that our version of the test case uses
      DEBUG_SYNC instrumentation on ROLLBACK, not on purge.
      
      btr_cur_will_modify_tree(): Simplify the check further.
      This is the actual bug fix.
      
      row_undo_mod_remove_clust_low(), row_undo_mod_clust(): Add DEBUG_SYNC
      instrumentation for the test case.
      3e38d155
    • Marko Mäkelä's avatar
      MDEV-13626: Add the WL#6326 tests · 9cae7bdc
      Marko Mäkelä authored
      9cae7bdc
    • Jan Lindström's avatar
      MDEV-17062 : Test failure on galera.MW-336 · c4195305
      Jan Lindström authored
      Add mutex protection while we calculate required slave
      thread change and create them. Add error handling.
      c4195305
  2. 16 Jan, 2020 5 commits
  3. 15 Jan, 2020 6 commits
  4. 14 Jan, 2020 2 commits
    • Sergei Petrunia's avatar
      MDEV-21341: Fix UBSAN failures: Issue Six · 5e5ae51b
      Sergei Petrunia authored
      (Variant #2 of the patch, which keeps the sp_head object inside the
      MEM_ROOT that sp_head object owns)
      (10.3 requires extra work due to sp_package, will commit a separate
      patch for it)
      
      sp_head::operator new() and operator delete() were dereferencing sp_head*
      pointers to memory that didn't hold a valid sp_head object (it was
      not created/already destroyed).
      This caused UBSan to crash when looking up type information.
      
      Fixed by providing static sp_head::create() and sp_head::destroy() methods.
      5e5ae51b
    • Daniele Sciascia's avatar
      MDEV-19803 Long semaphore wait error on galera.MW-388 · 7d313214
      Daniele Sciascia authored
      The long semaphore wait appeared to be the caused by the following
      pattern in the MTR test:
      
      ```
      SET DEBUG_SYNC = "now SIGNAL wsrep_after_certification_continue";
      SET DEBUG_SYNC = "now SIGNAL signal.wsrep_apply_cb;
      ```
      
      Raising two signals, one right after another, caused one signal to
      overwrite the other, before the signal was consumed by the thread.
      This caused one thread to be stuck until the debug sync point would
      timeout.
      7d313214
  5. 10 Jan, 2020 1 commit
  6. 09 Jan, 2020 1 commit
    • Sujatha's avatar
      MDEV-18514: Assertion `!writer.checksum_len || writer.remains == 0' failed · 41cde4fe
      Sujatha authored
      Analysis:
      ========
      'max_binlog_cache_size' is configured and a huge transaction is executed. When
      the transaction specific events size exceeds 'max_binlog_cache_size' the event
      cannot be written to the binary log cache and cache write error is raised.
      Upon cache write error the statement is rolled back and the transaction cache
      should be truncated to a previous statement specific position.  The truncate
      operation should reset the cache to earlier valid positions and flush the new
      changes. Even though the flush is successful the cache write error is still in
      marked state. The truncate code interprets the cache write error as cache flush
      failure and returns abruptly without modifying the write cache parameters.
      Hence cache is in a invalid state. When a COMMIT statement is executed in this
      session it tries to flush the contents of transaction cache to binary log.
      Since cache has partial events the cache write operation will report
      'writer.remains' assert.
      
      Fix:
      ===
      Binlog truncate function resets the cache to a specified size. As a first step
      of truncation, clear the cache write error flag that was raised during earlier
      execution. With this new errors that surface during cache truncation can be
      clearly identified.
      41cde4fe
  7. 07 Jan, 2020 15 commits
    • Sujatha's avatar
      Merge branch '10.1' into 10.2 · 8317f77c
      Sujatha authored
      MDEV-18046: Assortment of crashes, assertion failures and ASAN errors in mysql_show_binlog_events
      
      Problem:
      ========
      SHOW BINLOG EVENTS FROM <pos> reports following assert when ASAN is enabled.
      
      uint32 binlog_get_uncompress_len(const char*):
        Assertion `(buf[0] & 0xe0) == 0x80' failed
      
      Fix:
      ===
      **Part11: Converted debug assert to error handler code**
      8317f77c
    • Sujatha's avatar
      MDEV-18046: Assortment of crashes, assertion failures and ASAN errors in mysql_show_binlog_events · cb204e11
      Sujatha authored
      Problem:
      ========
      SHOW BINLOG EVENTS FROM <pos> reports following ASAN error.
      
      AddressSanitizer: heap-buffer-overflow on address
      READ of size 1 at 0x60e00009cf71 thread T28
      #0 0x55e37e034ae2 in net_field_length
      
      Fix:
      ===
      **Part10: Avoid reading out of buffer**
      cb204e11
    • Sujatha's avatar
      MDEV-18046: Assortment of crashes, assertion failures and ASAN errors in mysql_show_binlog_events · d05c511d
      Sujatha authored
      Problem:
      ========
      SHOW BINLOG EVENTS FROM <pos> reports following assert when ASAN is enabled.
      
      Query_log_event::Query_log_event(const char*, uint,
          const Format_description_log_event*, Log_event_type):
        Assertion `(pos) + (6) <= (end)' failed
      
      Fix:
      ===
      **Part9: Removed additional DBUG_ASSERT**
      d05c511d
    • Sujatha's avatar
      MDEV-18046: Assortment of crashes, assertion failures and ASAN errors in mysql_show_binlog_events · bac33533
      Sujatha authored
      Problem:
      ========
      SHOW BINLOG EVENTS FROM <pos> reports following ASAN error
      
      AddressSanitizer: SEGV on unknown address
      The signal is caused by a READ memory access.
      User_var_log_event::User_var_log_event(char const*, unsigned int,
          Format_description_log_event const*)
      
      Implemented part of upstream patch.
      commit: mysql/mysql-server@a3a497ccf7ecacc900551fb1e47ea4078b45c351
      
      Fix:
      ===
      **Part8: added checks to avoid reading out of buffer limits**
      bac33533
    • Sujatha's avatar
      MDEV-18046: Assortment of crashes, assertion failures and ASAN errors in mysql_show_binlog_events · 2187f1c2
      Sujatha authored
      Problem:
      ========
      SHOW BINLOG EVENTS FROM <pos> reports following ASAN error
      "heap-buffer-overflow on address" and some times it asserts.
      
      Table_map_log_event::Table_map_log_event(const char*, uint,
          const Format_description_log_event*)
      Assertion `m_field_metadata_size <= (m_colcnt * 2)' failed.
      
      Fix:
      ===
      **Part7: Avoid reading out of buffer**
      
      
      Converted debug assert to error handler code.
      2187f1c2
    • Sujatha's avatar
      MDEV-18046: Assortment of crashes, assertion failures and ASAN errors in mysql_show_binlog_events · d6fa69e4
      Sujatha authored
      Problem:
      ========
      SHOW BINLOG EVENTS FROM <pos> reports following ASAN error
      
      AddressSanitizer: heap-buffer-overflow on address 0x60400002acb8
      Load_log_event::copy_log_event(char const*, unsigned long, int,
          Format_description_log_event const*)
      
      Fix:
      ===
      **Part6: Moved the event_len validation to the begin of copy_log_event function**
      d6fa69e4
    • Sujatha's avatar
      MDEV-18046: Assortment of crashes, assertion failures and ASAN errors in mysql_show_binlog_events · 15781283
      Sujatha authored
      Problem:
      ========
      SHOW BINLOG EVENTS FROM <pos> reports following ASAN error
      
      AddressSanitizer: heap-buffer-overflow on address
      String::append(char const*, unsigned int)
      Query_log_event::pack_info(Protocol*)
      
      Fix:
      ===
      **Part5: Added check to catch buffer overflow**
      15781283
    • Sujatha's avatar
      MDEV-18046: Assortment of crashes, assertion failures and ASAN errors in mysql_show_binlog_events · a42ef108
      Sujatha authored
      Problem:
      ========
      SHOW BINLOG EVENTS FROM <pos> reports following ASAN error
      
      heap-buffer-overflow within "my_strndup" in Rotate_log_event
      
      my_strndup /mysys/my_malloc.c:254
      Rotate_log_event::Rotate_log_event(char const*, unsigned int,
          Format_description_log_event const*)
      
      Fix:
      ===
      **Part4: Improved the check for event_len validation**
      a42ef108
    • Sujatha's avatar
      MDEV-18046: Assortment of crashes, assertion failures and ASAN errors in mysql_show_binlog_events · 5a54e84e
      Sujatha authored
      Problem:
      ========
      SHOW BINLOG EVENTS FROM <pos> reports following crash when ASAN is enabled.
      
      SEGV on unknown address
      in inline_mysql_mutex_destroy
      in my_bitmap_free
      in Update_rows_log_event::~Update_rows_log_event()
      
      Fix:
      ===
      **Part3: Initialize m_cols_ai.bitmap to NULL**
      5a54e84e
    • Sujatha's avatar
      MDEV-18046: Assortment of crashes, assertion failures and ASAN errors in mysql_show_binlog_events · 913f405d
      Sujatha authored
      Problem:
      ========
      SHOW BINLOG EVENTS FROM <pos> reports following assert when ASAN is enabled.
      
      Rows_log_event::Rows_log_event(const char*, uint,
          const Format_description_log_event*):
      Assertion `var_header_len >= 2'
      
      Implemented part of upstream patch.
      commit: mysql/mysql-server@a3a497ccf7ecacc900551fb1e47ea4078b45c351
      
      Fix:
      ===
      **Part2: Avoid reading out of buffer limits**
      913f405d
    • Sujatha's avatar
      MDEV-18046: Assortment of crashes, assertion failures and ASAN errors in mysql_show_binlog_events · a6dd827a
      Sujatha authored
      Problem:
      ========
      SHOW BINLOG EVENTS FROM <pos> causes a variety of failures, some of which are
      listed below. It is not a race condition issue, but there is some
      non-determinism in it.
      
      Analysis:
      ========
      "show binlog events from <pos>" code considers the user given position as a
      valid event start position. The code starts reading data from this event start
      position onwards and tries to map it to a set of known events. Each event has
      a specific event structure and asserts have been added to ensure that read
      event data satisfies the event specific requirements. When a random position
      is supplied to "show binlog events command" the event structure specific
      checks will fail and they result in assert.
      
      Fix:
      ====
      The fix is split into different parts. Each part addresses either an ASAN
      issue or an assert/crash.
      
      **Part1: Checksum based position validation when checksum is enabled**
      
      
      Using checksum validate the very first event read at the user specified
      position. If there is a checksum mismatch report an appropriate error for the
      invalid event.
      a6dd827a
    • Marko Mäkelä's avatar
      Fixup MDEV-21429: Correct a definition · 0b4ae672
      Marko Mäkelä authored
      INNOBASE_ALTER_NOVALIDATE: Remove the set of operations
      INNOBASE_ONLINE_CREATE that was accidentally included in the
      definition.
      
      In the merge of 82187a12 to 10.3
      (in commit eda71979) the flags
      were defined correctly.
      
      This bug was caught by the test innodb_zip.index_large_prefix.
      0b4ae672
    • Marko Mäkelä's avatar
      MDEV-21429 TRUNCATE and OPTIMIZE are being refused due to "row size too large" · 82187a12
      Marko Mäkelä authored
      By default (innodb_strict_mode=ON), InnoDB attempts to guarantee
      at DDL time that any INSERT to the table can succeed.
      MDEV-19292 recently revised the "row size too large" check in InnoDB.
      The check still is somewhat inaccurate;
      that should be addressed in MDEV-20194.
      
      Note: If a table contains multiple long string columns so that each column
      is part of a column prefix index, then an UPDATE that attempts to modify
      all those columns at once may fail, because the undo log record might
      not fit in a single undo log page (of innodb_page_size). In the worst case,
      the undo log record would grow by about 3KiB of for each updated column.
      
      The DDL-time check (since the InnoDB Plugin for MySQL 5.1) is optional
      in the sense that when the maximum B-tree record size or undo log
      record size would be exceeded, the DML operation will fail and the
      transaction will be properly rolled back.
      
      create_table_info_t::row_size_is_acceptable(): Add the parameter
      'bool strict' so that innodb_strict_mode=ON can be overridden during
      TRUNCATE, OPTIMIZE and ALTER TABLE...FORCE (when the storage format
      is not changing).
      
      create_table_info_t::create_table(): Perform a sloppy check for
      TRUNCATE TABLE (create_fk=false).
      
      prepare_inplace_alter_table_dict(): Perform a sloppy check for
      simple operations.
      
      trx_is_strict(): Remove. The function became unused in
      commit 98694ab0 (MDEV-20949).
      82187a12
    • Jan Lindström's avatar
      MDEV-13569: wsrep_info.plugin failed in buildbot with "no nodes coming from prim view · 5824e9f8
      Jan Lindström authored
      Modify configuration so that all nodes are part of galera cluster
      i.e. wsrep_on=ON. Add missing wait conditions.
      
      test changes only.
      5824e9f8
    • Marko Mäkelä's avatar
      Merge 10.1 into 10.2 · e99ba4ba
      Marko Mäkelä authored
      e99ba4ba
  8. 04 Jan, 2020 1 commit
    • Eugene Kosov's avatar
      Lets add another intrusive double linked list! · fd899b3b
      Eugene Kosov authored
      Features:
      * STL-like interface
      * Fast modification: no branches on insertion or deletion
      * Fast iteration: one pointer dereference and one pointer comparison
      * Your class can be a part of several lists
      
      Modeled after std::list<T> but currently has fewer methods (not complete yet)
      
      For even more performance it's possible to customize list with templates so
      it won't have size counter variable or won't NULLify unlinked node.
      
      How existing lists differ?
      
      No existing lists support STL-like interface.
      
      I_List:
      * slower iteration (one more branch on iteration)
      * element can't be a part of two lists simultaneously
      
      I_P_List:
      * slower modification (branches, except for the fastest push_back() case)
      * slower iteration (one more branch on iteration)
      
      UT_LIST_BASE_NODE_T:
      * slower modification (branches)
      
      Three UT_LISTs were replaced: two in fil_system_t and one in dyn_buf_t.
      fd899b3b
  9. 03 Jan, 2020 2 commits