1. 05 Nov, 2021 1 commit
    • Marko Mäkelä's avatar
      MDEV-26826 fixup: ROW_FORMAT=COMPRESSED may corrupt buf_pool.page_hash · 9b967c4c
      Marko Mäkelä authored
      In commit c091a0bc we removed
      the use of the HASH_ macros for inserting into
      buf_pool.page_hash, or accessing buf_page_t::hash.
      
      However, the binary buddy allocator for block->page.zip.data would
      still use the HASH_ macros. HASH_INSERT and not HASH_DELETE would reset
      the next-block pointer to the null pointer. Our replacement of
      HASH_DELETE() will reset the next-block pointer, and the replacement of
      HASH_INSERT() assumes that the pointer is the null pointer.
      
      buf_LRU_block_free_non_file_page(): Assert that the next-block pointer
      is the null pointer.
      
      buf_buddy_block_free(): Reset the pointer before invoking
      buf_LRU_block_free_non_file_page(). Without this, the added
      assertion would fail in the test encryption.innochecksum.
      9b967c4c
  2. 04 Nov, 2021 3 commits
  3. 03 Nov, 2021 4 commits
  4. 02 Nov, 2021 12 commits
  5. 01 Nov, 2021 1 commit
  6. 30 Oct, 2021 2 commits
  7. 29 Oct, 2021 17 commits
    • sjaakola's avatar
      MDEV-23328 Server hang due to Galera lock conflict resolution · ef2dbb8d
      sjaakola authored
      Mutex order violation when wsrep bf thread kills a conflicting trx,
      the stack is
      
                wsrep_thd_LOCK()
                wsrep_kill_victim()
                lock_rec_other_has_conflicting()
                lock_clust_rec_read_check_and_lock()
                row_search_mvcc()
                ha_innobase::index_read()
                ha_innobase::rnd_pos()
                handler::ha_rnd_pos()
                handler::rnd_pos_by_record()
                handler::ha_rnd_pos_by_record()
                Rows_log_event::find_row()
                Update_rows_log_event::do_exec_row()
                Rows_log_event::do_apply_event()
                Log_event::apply_event()
                wsrep_apply_events()
      
      and mutexes are taken in the order
      
                lock_sys->mutex -> victim_trx->mutex -> victim_thread->LOCK_thd_data
      
      When a normal KILL statement is executed, the stack is
      
                innobase_kill_query()
                kill_handlerton()
                plugin_foreach_with_mask()
                ha_kill_query()
                THD::awake()
                kill_one_thread()
      
              and mutexes are
      
                victim_thread->LOCK_thd_data -> lock_sys->mutex -> victim_trx->mutex
      
      This patch is the plan D variant for fixing potetial mutex locking
      order exercised by BF aborting and KILL command execution.
      
      In this approach, KILL command is replicated as TOI operation.
      This guarantees total isolation for the KILL command execution
      in the first node: there is no concurrent replication applying
      and no concurrent DDL executing. Therefore there is no risk of
      BF aborting to happen in parallel with KILL command execution
      either. Potential mutex deadlocks between the different mutex
      access paths with KILL command execution and BF aborting cannot
      therefore happen.
      
      TOI replication is used, in this approach,  purely as means
      to provide isolated KILL command execution in the first node.
      KILL command should not (and must not) be applied in secondary
      nodes. In this patch, we make this sure by skipping KILL
      execution in secondary nodes, in applying phase, where we
      bail out if applier thread is trying to execute KILL command.
      This is effective, but skipping the applying of KILL command
      could happen much earlier as well.
      
      This also fixed unprotected calls to wsrep_thd_abort
      that will use wsrep_abort_transaction. This is fixed
      by holding THD::LOCK_thd_data while we abort transaction.
      Reviewed-by: default avatarJan Lindström <jan.lindstrom@mariadb.com>
      ef2dbb8d
    • Jan Lindström's avatar
      MDEV-25114: Crash: WSREP: invalid state ROLLED_BACK (FATAL) · d5bc0579
      Jan Lindström authored
      Revert "MDEV-23328 Server hang due to Galera lock conflict resolution"
      
      This reverts commit eac8341d.
      d5bc0579
    • Oleksandr Byelkin's avatar
      Merge branch '10.3' into 10.4 · 5900f3a7
      Oleksandr Byelkin authored
      5900f3a7
    • Oleksandr Byelkin's avatar
      Merge branch '10.2' into 10.3 · 6953af36
      Oleksandr Byelkin authored
      6953af36
    • Marko Mäkelä's avatar
      MDEV-26936 Recovery crash on rolling back DELETE FROM SYS_INDEXES · 003095e8
      Marko Mäkelä authored
      row_undo_mod_clust_low(): If we are in recovery and rolling back
      a DELETE operation on the SYS_INDEXES table, and the
      SYS_INDEXES.NAME starts with the magic byte 0xff
      that identifies uncommitted ADD INDEX stubs, we must not
      try to evict the table definition because such index stubs
      would be skipped by dict_load_indexes() anyway.
      003095e8
    • Marko Mäkelä's avatar
    • Oleksandr Byelkin's avatar
      columnstore-6.2.1-1 · ad3e416e
      Oleksandr Byelkin authored
      ad3e416e
    • Oleksandr Byelkin's avatar
      Merge branch '10.5' into 10.6 · facd9d52
      Oleksandr Byelkin authored
      facd9d52
    • Oleksandr Byelkin's avatar
      columnstore-5.6.3-2 · 1974df01
      Oleksandr Byelkin authored
      1974df01
    • Oleksandr Byelkin's avatar
      Merge branch '10.4' into 10.5 · 1c1396f0
      Oleksandr Byelkin authored
      1c1396f0
    • Marko Mäkelä's avatar
      MDEV-25683 Atomic DDL: With innodb_force_recovery=3 InnoDB: Trying to load... · dbd6c6dc
      Marko Mäkelä authored
      MDEV-25683 Atomic DDL: With innodb_force_recovery=3 InnoDB: Trying to load index but the index tree has been freed
      
      The purpose of the parameter innodb_force_recovery is to allow some
      data to be dumped from a corrupted database. Its values used to be
      as follows:
      
      innodb_force_recovery=0: normal (default)
      
      innodb_force_recovery=1: ignore (skip log for) corrupted pages or
      missing data files when applying the redo log
      
      innodb_force_recovery=2: additionally, disable background tasks
      (such as the purge of committed undo logs)
      
      innodb_force_recovery=3: additionally, disable the rollback of
      recovered incomplete (not committed or XA PREPARE) transactions
      
      innodb_force_recovery=4: same as 3 (since MDEV-19514 in MariaDB 10.5)
      
      innodb_force_recovery=5: additionally, do not process any undo log,
      disallow any writes, and force READ UNCOMMITTED isolation level
      
      innodb_force_recovery=6: additionally, pretend that ib_logfile0 does
      not exist (prevent any recovery). Never use this!
      
      The bad thing that happens with innodb_force_recovery=3 and
      innodb_force_recovery=4 is that also the rollback of any recovered
      DDL transaction will be skipped. This would break the DDL log recovery
      that was introduced in MDEV-17567.
      
      For one data directory sample, the DDL log recovery would hangs due to
      a conflict on the InnoDB SYS_TABLES table, because the lock holder
      transaction was not rolled back due to innodb_force_recovery=3.
      
      Fix: Make innodb_force_recovery=3 skip the DML transaction rollback only,
      and make innodb_force_recovery=4 (renamed to SRV_FORCE_NO_DDL_UNDO)
      behave like innodb_force_recovery=3 used to (skip the rollback of all
      recovered transactions, both DML and DDL).
      
      Startup with innodb_force_recovery=4 will be unaffected by this change.
      (There may be hangs, possibly preceded by messages about failing to
      load an index.)
      
      Side note: With innodb_force_recovery=5, any DDL log for InnoDB tables
      will be essentially ignored by InnoDB, but the server will start up.
      dbd6c6dc
    • sjaakola's avatar
      MDEV-23328 Server hang due to Galera lock conflict resolution · 157b3a63
      sjaakola authored
      Mutex order violation when wsrep bf thread kills a conflicting trx,
      the stack is
      
                wsrep_thd_LOCK()
                wsrep_kill_victim()
                lock_rec_other_has_conflicting()
                lock_clust_rec_read_check_and_lock()
                row_search_mvcc()
                ha_innobase::index_read()
                ha_innobase::rnd_pos()
                handler::ha_rnd_pos()
                handler::rnd_pos_by_record()
                handler::ha_rnd_pos_by_record()
                Rows_log_event::find_row()
                Update_rows_log_event::do_exec_row()
                Rows_log_event::do_apply_event()
                Log_event::apply_event()
                wsrep_apply_events()
      
      and mutexes are taken in the order
      
                lock_sys->mutex -> victim_trx->mutex -> victim_thread->LOCK_thd_data
      
      When a normal KILL statement is executed, the stack is
      
                innobase_kill_query()
                kill_handlerton()
                plugin_foreach_with_mask()
                ha_kill_query()
                THD::awake()
                kill_one_thread()
      
              and mutexes are
      
                victim_thread->LOCK_thd_data -> lock_sys->mutex -> victim_trx->mutex
      
      This patch is the plan D variant for fixing potetial mutex locking
      order exercised by BF aborting and KILL command execution.
      
      In this approach, KILL command is replicated as TOI operation.
      This guarantees total isolation for the KILL command execution
      in the first node: there is no concurrent replication applying
      and no concurrent DDL executing. Therefore there is no risk of
      BF aborting to happen in parallel with KILL command execution
      either. Potential mutex deadlocks between the different mutex
      access paths with KILL command execution and BF aborting cannot
      therefore happen.
      
      TOI replication is used, in this approach,  purely as means
      to provide isolated KILL command execution in the first node.
      KILL command should not (and must not) be applied in secondary
      nodes. In this patch, we make this sure by skipping KILL
      execution in secondary nodes, in applying phase, where we
      bail out if applier thread is trying to execute KILL command.
      This is effective, but skipping the applying of KILL command
      could happen much earlier as well.
      
      This also fixed unprotected calls to wsrep_thd_abort
      that will use wsrep_abort_transaction. This is fixed
      by holding THD::LOCK_thd_data while we abort transaction.
      Reviewed-by: default avatarJan Lindström <jan.lindstrom@mariadb.com>
      157b3a63
    • Jan Lindström's avatar
      MDEV-25114: Crash: WSREP: invalid state ROLLED_BACK (FATAL) · 30337add
      Jan Lindström authored
      Revert "MDEV-23328 Server hang due to Galera lock conflict resolution"
      
      This reverts commit 29bbcac0.
      30337add
    • sjaakola's avatar
      MDEV-23328 Server hang due to Galera lock conflict resolution · 5c230b21
      sjaakola authored
      Mutex order violation when wsrep bf thread kills a conflicting trx,
      the stack is
      
                wsrep_thd_LOCK()
                wsrep_kill_victim()
                lock_rec_other_has_conflicting()
                lock_clust_rec_read_check_and_lock()
                row_search_mvcc()
                ha_innobase::index_read()
                ha_innobase::rnd_pos()
                handler::ha_rnd_pos()
                handler::rnd_pos_by_record()
                handler::ha_rnd_pos_by_record()
                Rows_log_event::find_row()
                Update_rows_log_event::do_exec_row()
                Rows_log_event::do_apply_event()
                Log_event::apply_event()
                wsrep_apply_events()
      
      and mutexes are taken in the order
      
                lock_sys->mutex -> victim_trx->mutex -> victim_thread->LOCK_thd_data
      
      When a normal KILL statement is executed, the stack is
      
                innobase_kill_query()
                kill_handlerton()
                plugin_foreach_with_mask()
                ha_kill_query()
                THD::awake()
                kill_one_thread()
      
              and mutexes are
      
                victim_thread->LOCK_thd_data -> lock_sys->mutex -> victim_trx->mutex
      
      This patch is the plan D variant for fixing potetial mutex locking
      order exercised by BF aborting and KILL command execution.
      
      In this approach, KILL command is replicated as TOI operation.
      This guarantees total isolation for the KILL command execution
      in the first node: there is no concurrent replication applying
      and no concurrent DDL executing. Therefore there is no risk of
      BF aborting to happen in parallel with KILL command execution
      either. Potential mutex deadlocks between the different mutex
      access paths with KILL command execution and BF aborting cannot
      therefore happen.
      
      TOI replication is used, in this approach,  purely as means
      to provide isolated KILL command execution in the first node.
      KILL command should not (and must not) be applied in secondary
      nodes. In this patch, we make this sure by skipping KILL
      execution in secondary nodes, in applying phase, where we
      bail out if applier thread is trying to execute KILL command.
      This is effective, but skipping the applying of KILL command
      could happen much earlier as well.
      
      This also fixed unprotected calls to wsrep_thd_abort
      that will use wsrep_abort_transaction. This is fixed
      by holding THD::LOCK_thd_data while we abort transaction.
      Reviewed-by: default avatarJan Lindström <jan.lindstrom@mariadb.com>
      5c230b21
    • Jan Lindström's avatar
      MDEV-25114: Crash: WSREP: invalid state ROLLED_BACK (FATAL) · aa7ca987
      Jan Lindström authored
      Revert "MDEV-23328 Server hang due to Galera lock conflict resolution"
      
      This reverts commit eac8341d.
      aa7ca987
    • sjaakola's avatar
      MDEV-23328 Server hang due to Galera lock conflict resolution · db50ea3a
      sjaakola authored
      Mutex order violation when wsrep bf thread kills a conflicting trx,
      the stack is
      
                wsrep_thd_LOCK()
                wsrep_kill_victim()
                lock_rec_other_has_conflicting()
                lock_clust_rec_read_check_and_lock()
                row_search_mvcc()
                ha_innobase::index_read()
                ha_innobase::rnd_pos()
                handler::ha_rnd_pos()
                handler::rnd_pos_by_record()
                handler::ha_rnd_pos_by_record()
                Rows_log_event::find_row()
                Update_rows_log_event::do_exec_row()
                Rows_log_event::do_apply_event()
                Log_event::apply_event()
                wsrep_apply_events()
      
      and mutexes are taken in the order
      
                lock_sys->mutex -> victim_trx->mutex -> victim_thread->LOCK_thd_data
      
      When a normal KILL statement is executed, the stack is
      
                innobase_kill_query()
                kill_handlerton()
                plugin_foreach_with_mask()
                ha_kill_query()
                THD::awake()
                kill_one_thread()
      
              and mutexes are
      
                victim_thread->LOCK_thd_data -> lock_sys->mutex -> victim_trx->mutex
      
      This patch is the plan D variant for fixing potetial mutex locking
      order exercised by BF aborting and KILL command execution.
      
      In this approach, KILL command is replicated as TOI operation.
      This guarantees total isolation for the KILL command execution
      in the first node: there is no concurrent replication applying
      and no concurrent DDL executing. Therefore there is no risk of
      BF aborting to happen in parallel with KILL command execution
      either. Potential mutex deadlocks between the different mutex
      access paths with KILL command execution and BF aborting cannot
      therefore happen.
      
      TOI replication is used, in this approach,  purely as means
      to provide isolated KILL command execution in the first node.
      KILL command should not (and must not) be applied in secondary
      nodes. In this patch, we make this sure by skipping KILL
      execution in secondary nodes, in applying phase, where we
      bail out if applier thread is trying to execute KILL command.
      This is effective, but skipping the applying of KILL command
      could happen much earlier as well.
      
      This also fixed unprotected calls to wsrep_thd_abort
      that will use wsrep_abort_transaction. This is fixed
      by holding THD::LOCK_thd_data while we abort transaction.
      Reviewed-by: default avatarJan Lindström <jan.lindstrom@mariadb.com>
      db50ea3a
    • Jan Lindström's avatar
      MDEV-25114: Crash: WSREP: invalid state ROLLED_BACK (FATAL) · c8b39f7e
      Jan Lindström authored
      Revert "MDEV-23328 Server hang due to Galera lock conflict resolution"
      
      This reverts commit 29bbcac0.
      c8b39f7e