1. 23 Aug, 2020 1 commit
    • Kentoku SHIBA's avatar
      MDEV-22246 Result rows duplicated by spider engine · 65ee216c
      Kentoku SHIBA authored
      fix the following type mrr scan
      (select 0,`id`,`node` from `auto_test_remote`.`tbl_a` where (`id` <> 0) order by `id`)union all(select 1,`id`,`node` from `auto_test_remote`.`tbl_a` where (`id` <> 0) order by `id`) order by `id`
      65ee216c
  2. 24 Aug, 2020 1 commit
  3. 23 Aug, 2020 1 commit
  4. 22 Aug, 2020 3 commits
    • Alexander Barkov's avatar
      MDEV-23537 Comparison with temporal columns is slow in MariaDB · 2e5d86f4
      Alexander Barkov authored
      Implementing methods:
      - Field::val_time_packed()
      - Field::val_datetime_packed()
      - Item_field::val_datetime_packed(THD *thd);
      - Item_field::val_time_packed(THD *thd);
      to give a faster access to temporal packed longlong representation of a Field,
      which is used in temporal Arg_comparator's to DATE, TIME, DATETIME data types.
      
      The same idea is used in MySQL-5.6+.
      
      This improves performance.
      2e5d86f4
    • Alexander Barkov's avatar
      6708e67a
    • Alexander Barkov's avatar
      MDEV-23525 Wrong result of MIN(time_expr) and MAX(time_expr) with GROUP BY · ae33ebe5
      Alexander Barkov authored
      Problem:
      
      When calculatung MIN() and MAX() in a query with GROUP BY, like this:
      
        SELECT MIN(time_expr), MAX(time_expr) FROM t1 GROUP BY i;
      
      the code in Item_sum_min_max::update_field() erroneosly used
      string format comparison, therefore '100:20:30' was considered as
      smaller than '10:20:30'.
      
      Fix:
      
      1. Implementing low level "native" related methods in class Time:
           Time::Time(const Native &native)           - convert native to Time
           Time::to_native(Native *to, uint decimals) - convert Time to native
      
         The "native" binary representation for TIME is equal to
         the binary data format of Field_timef, which is used to
         store TIME when mysql56_temporal_format is ON (default).
      
      2. Implementing Type_handler_time_common "native" related methods:
      
        Type_handler_time_common::cmp_native()
        Type_handler_time_common::Item_val_native_with_conversion()
        Type_handler_time_common::Item_val_native_with_conversion_result()
        Type_handler_time_common::Item_param_val_native()
      
      3. Implementing missing "native representation" related methods
         in Field_time and Field_timef:
      
        Field_time::store_native()
        Field_time::val_native()
        Field_timef::store_native()
        Field_timef::val_native()
      
      4. Implementing missing "native" related methods in all Items
         that can have the TIME data type:
      
        Item_timefunc::val_native()
        Item_name_const::val_native()
        Item_time_literal::val_native()
        Item_cache_time::val_native()
        Item_handled_func::val_native()
      
      5. Marking Type_handler_time_common as "native ready".
         So now Item_sum_min_max::update_field() calculates
         values using min_max_update_native_field(),
         which uses native binary representation rather than string representation.
      
         Before this change, only the TIMESTAMP data type used native
         representation to calculate MIN() and MAX().
      
      Benchmarks (see more details in MDEV):
      
        This change not only fixes the wrong result, but also
        makes a "SELECT .. MAX.. GROUP BY .." query faster:
      
        # TIME(0)
        CREATE TABLE t1 (id INT, time_col TIME) ENGINE=HEAP;
        INSERT INTO t1 VALUES (1,'10:10:10'); -- repeat this 1m times
        SELECT id, MAX(time_col) FROM t1 GROUP BY id;
      
        MySQL80: 0.159 sec
        10.3:    0.108 sec
        10.4:    0.094 sec (fixed)
      
        # TIME(6):
        CREATE TABLE t1 (id INT, time_col TIME(6)) ENGINE=HEAP;
        INSERT INTO t1 VALUES (1,'10:10:10.999999'); -- repeat this 1m times
        SELECT id, MAX(time_col) FROM t1 GROUP BY id;
      
        My80: 0.154
        10.3: 0.135
        10.4: 0.093 (fixed)
      ae33ebe5
  5. 21 Aug, 2020 15 commits
    • Ian Gilfillan's avatar
      Fix spelling mistake in error message · ee61c8c0
      Ian Gilfillan authored
      ee61c8c0
    • Yuqi Gu's avatar
      MDEV-23495: Refine Arm64 PMULL runtime check in MariaDB · 151fc0ed
      Yuqi Gu authored
      Raspberry Pi 4 supports crc32 but doesn't support pmull (MDEV-23030).
      
      The PR #1645 offers a solution to fix this issue. But it does not consider
      the condition that the target platform does support crc32 but not support PMULL.
      
      In this condition, it should leverage the Arm64 crc32 instruction (__crc32c) and
      just only skip parallel computation (pmull/vmull) rather than skip all hardware
      crc32 instruction of computation.
      
      The PR also removes unnecessary CRC32_ZERO branch in 'crc32c_aarch64' for MariaDB,
      formats the indent and coding style.
      
      Change-Id: I76371a6bd767b4985600e8cca10983d71b7e9459
      Signed-off-by: default avatarYuqi Gu <yuqi.gu@arm.com>
      151fc0ed
    • Marko Mäkelä's avatar
      Merge 10.4 into 10.5 · 0b4ed0b7
      Marko Mäkelä authored
      0b4ed0b7
    • Marko Mäkelä's avatar
      Merge 10.3 into 10.4 · aa6cb7ed
      Marko Mäkelä authored
      aa6cb7ed
    • Marko Mäkelä's avatar
      Merge 10.2 into 10.3 · c277bcd5
      Marko Mäkelä authored
      c277bcd5
    • Marko Mäkelä's avatar
      MDEV-22782 AddressSanitizer race condition in trx_free() · f3160ee4
      Marko Mäkelä authored
      In trx_free() we used to declare the entire trx_t unaccessible
      and then declare that some data members are accessible.
      This involves a race condition with other threads that may concurrently
      access the data members that must remain accessible.
      One type of error is "AddressSanitizer: unknown-crash", whose
      exact cause we have not determined.
      
      Another type of error (reported in MDEV-23472) is "use-after-poison",
      where the reported shadow bytes would in fact be 00, indicating that
      the memory was no longer poisoned. The poison-access-unpoison race
      condition was confirmed by "rr replay".
      
      We eliminate the race condition by invoking MEM_NOACCESS on each
      individual data member of trx_t before freeing the memory to the pool.
      The memory would not be unpoisoned until the pool is freed
      or the memory is being reused for another allocation.
      
      trx_t::free(): Replaces trx_free().
      
      trx_t::active_commit_ordered: Changed to bool, so that MEM_NOACCESS
      can be invoked. Removed some accessor functions.
      
      Pool: Remove all MEM_ instrumentation.
      
      TrxFactory: Move the MEM_ instrumentation from Pool.
      
      TrxFactory::debug(): Removed. Moved to trx_t::free(). Because
      the memory was already marked unaccessible in trx_t::free(), the
      Factory::debug() call in Pool::putl() would be unable to access it.
      
      trx_allocate_for_background(): Replaces trx_create_low().
      
      trx_t::free(): Perform all consistency checks while avoiding
      duplication, and declare most data members unaccessible.
      f3160ee4
    • Andrei Elkin's avatar
      MDEV-23511 shutdown_server 10 times out, causing server kill at shutdown · a19cb388
      Andrei Elkin authored
      Shutdown of mtr tests may be too impatient, esp on CI environment where
      10 seconds of `arg` of `shutdown_server arg` may not be enough for the clean
      shutdown to complete.
      
      This is fixed to remove explicit non-zero timeout argument to
      `shutdown_server` from all mtr tests. mysqltest computes 60 seconds default
      value for the timeout for the argless `shutdown_server` command.
      This policy is additionally ensured with a compile time assert.
      a19cb388
    • Marko Mäkelä's avatar
      MDEV-23526 InnoDB leaks memory for some static objects · d98ccbe1
      Marko Mäkelä authored
      Leaks of some members of statically allocated objects are
      not being reported by AddressSanitizer or Valgrind, but they
      can be reported by implementing SAFEMALLOC instrumentation.
      
      These leaks were identified and original fixes provided
      by Michael Widenius and Vicențiu Ciorbaru.
      d98ccbe1
    • Marko Mäkelä's avatar
      Merge 10.4 into 10.5 · f2739e2a
      Marko Mäkelä authored
      f2739e2a
    • Marko Mäkelä's avatar
    • Marko Mäkelä's avatar
      MDEV-23526 InnoDB leaks memory for some static objects · 688fb630
      Marko Mäkelä authored
      A leak of the contents of fil_system.ssd that was introduced in
      commit 10dd290b (MDEV-17380)
      was caught by implementing SAFEMALLOC instrumentation of operator new.
      I did not try to find out how to make AddressSanitizer or Valgrind
      detect it.
      
      fil_system_t::close(): Clear fil_system.ssd.
      
      The leak was identified and a fix suggested by Michael Widenius
      and Vicențiu Ciorbaru.
      688fb630
    • Marko Mäkelä's avatar
      Merge 10.3 into 10.4 · 2643249d
      Marko Mäkelä authored
      2643249d
    • Marko Mäkelä's avatar
      Merge 10.2 into 10.3 · 9be0b614
      Marko Mäkelä authored
      9be0b614
    • Marko Mäkelä's avatar
      Merge 10.1 into 10.2 · a43faf6b
      Marko Mäkelä authored
      a43faf6b
    • Jan Lindström's avatar
      29d9df16
  6. 20 Aug, 2020 12 commits
    • Monty's avatar
      3ef65f27
    • Monty's avatar
      Reduce number of syncs to create a transactional aria table from 6 to 3 · b1ba3a19
      Monty authored
      This was possible because we can create any missing aria files from
      the aria transactional log
      
      The 3 remaining syncs are:
      - .frm file
      - Directory where frm file is
      - Aria log file
      b1ba3a19
    • Monty's avatar
      Fixed wrong value of "wsrep" in SHOW STATUS · 2c9be1e6
      Monty authored
      If WSREP was not initalized SHOW GLOBAL STATUS could have a random
      value of "wsrep"
      2c9be1e6
    • Monty's avatar
      Added support of WITH_GPROF to cmake · 76a92270
      Monty authored
      76a92270
    • Marko Mäkelä's avatar
      After-merge fix of the Windows build · 65c43bcf
      Marko Mäkelä authored
      65c43bcf
    • Marko Mäkelä's avatar
      Merge 10.4 into 10.5 · d5d8756d
      Marko Mäkelä authored
      d5d8756d
    • Marko Mäkelä's avatar
      Merge 10.3 into 10.4 · 2fa9f8c5
      Marko Mäkelä authored
      2fa9f8c5
    • Marko Mäkelä's avatar
      Merge 10.2 into 10.3 · de0e7cd7
      Marko Mäkelä authored
      de0e7cd7
    • Thirunarayanan Balathandayuthapani's avatar
      MDEV-23452 Assertion `buf_page_get_io_fix(bpage) == BUF_IO_NONE' failed · a79c2578
      Thirunarayanan Balathandayuthapani authored
      		in buf_page_set_sticky
      
      - Adding os_thread_yield() in buf_page_create() to avoid the continuous
      buffer pool mutex acquistions.
      a79c2578
    • Thirunarayanan Balathandayuthapani's avatar
      MDEV-23452 Assertion `buf_page_get_io_fix(bpage) == BUF_IO_NONE' failed · e9d6f1c7
      Thirunarayanan Balathandayuthapani authored
      			in buf_page_set_sticky
      
      commit a1f899a8 (MDEV-23233) added the
      code to make page sticky. So that InnoDB can't allow the page to
      be grabbed by other thread while doing lazy drop of ahi.
      
      But the block could be in flush list and it could have io_fix value
      as BUF_IO_WRITE. It could lead to the failure in buf_page_set_sticky().
      
      buf_page_create(): If btr_search_drop_page_hash_index() must be invoked,
      take x-latch on the block. If the block io_fix value is other than
      BUF_IO_NONE, release the buffer pool mutex and page hash lock and
      wait for I/O to complete.
      e9d6f1c7
    • Marko Mäkelä's avatar
      MDEV-23514 Race conditions between ROLLBACK and ALTER TABLE · 22c4a751
      Marko Mäkelä authored
      Since commit 15093639 (MDEV-23484)
      the rollback of InnoDB transactions is no longer protected by
      dict_operation_lock. Removing that protection revealed a race
      condition between transaction rollback and the rollback of an
      online table-rebuilding operation (OPTIMIZE TABLE, or any online
      ALTER TABLE that is rebuilding the table).
      
      row_undo_mod_clust(): Re-check dict_index_is_online_ddl() after
      acquiring index->lock, similar to how row_undo_ins_remove_clust_rec()
      is doing it. Because innobase_online_rebuild_log_free() is holding
      exclusive index->lock while invoking row_log_free(), this re-check
      will ensure that row_log_table_low() will not be invoked when
      index->online_log=NULL.
      
      A different race condition is possible between the rollback of a
      recovered transaction and the start of online secondary index creation.
      Because prepare_inplace_alter_table_dict() is not acquiring an InnoDB
      table lock in this case, and because recovered transactions are not
      covered by metadata locks (MDL), the dict_table_t::indexes could be
      modified by prepare_inplace_alter_table_dict() while the rollback of
      a recovered transaction is being executed. Normal transactions would
      be covered by MDL, and during prepare_inplace_alter_table_dict() we
      do hold MDL_EXCLUSIVE, that is, an online ALTER TABLE operation may
      not execute concurrently with other transactions that have accessed
      the table.
      
      row_undo(): To prevent a race condition with
      prepare_inplace_alter_table_dict(), acquire dict_operation_lock
      for all recovered transactions. Before MDEV-23484 we used to acquire
      it for all transactions, not only recovered ones.
      
      Note: row_merge_drop_indexes() would not invoke
      dict_index_remove_from_cache() while transactional locks
      exist on the table, or while any thread is holding an open table handle.
      OK, it does that for FULLTEXT INDEX, but ADD FULLTEXT INDEX is not
      supported as an online operation, and therefore
      prepare_inplace_alter_table_dict() would acquire a table S lock,
      which cannot succeed as long as recovered transactions on the table
      exist, because they would hold a conflicting IX lock on the table.
      22c4a751
    • Marko Mäkelä's avatar
      Merge 10.1 into 10.2 · bfba2bce
      Marko Mäkelä authored
      bfba2bce
  7. 19 Aug, 2020 6 commits
  8. 18 Aug, 2020 1 commit
    • Marko Mäkelä's avatar
      MDEV-23484 Rollback unnecessarily acquires dict_operation_lock for every row · 15093639
      Marko Mäkelä authored
      InnoDB transaction rollback includes an unnecessary work-around for
      a data corruption bug that was fixed by me in MySQL 5.6.12
      mysql/mysql-server@935ba09d52c1908bde273ad1940b5ab919d9763d
      and ported to MariaDB 10.0.8 by
      commit c291ddfd
      in 2013 and 2014, respectively.
      
      By acquiring and releasing dict_operation_lock in shared mode,
      row_undo() hopes to prevent the table from being dropped while
      the undo log record is being rolled back. But, thanks to mentioned fix,
      debug assertions (that we are adding) show that the rollback is
      protected by transactional locks (table IX lock, in addition to
      implicit or explicit exclusive locks on the records that had been modified).
      
      Because row_drop_table_for_mysql() would invoke
      row_add_table_to_background_drop_list() if any locks exist on the table,
      the mere existence of locks (which is guaranteed during ROLLBACK) is
      enough to protect the table from disappearing. Hence, acquiring and
      releasing dict_operation_lock for every row that is being rolled back is
      unnecessary.
      
      row_undo(): Remove the unnecessary acquisition and release of
      dict_operation_lock.
      
      Note: row_add_table_to_background_drop_list() is mostly working around
      bugs outside InnoDB:
      MDEV-21175 (insufficient MDL protection of FOREIGN KEY operations)
      MDEV-21602 (incorrect error handling of CREATE TABLE...SELECT).
      15093639