1. 01 Aug, 2020 4 commits
    • Marko Mäkelä's avatar
      MDEV-21201: Null-Merge 10.3 · 297746de
      Marko Mäkelä authored
      Sorry, this should have been pushed instead of
      7f4c749d to 10.3.
      (The push of d63631c3
      prevented required an extra merge of that fix.
      And that commit was merged to 10.4
      da78e952.)
      297746de
    • Marko Mäkelä's avatar
      MDEV-21201: Add --sorted_result to the test, for 10.4 · 7f4c749d
      Marko Mäkelä authored
      On MariaDB 10.4 (commit 4db4b773),
      the query results would not be sorted, which creates random result
      differences. Let us explicitly sort the results already in 10.3
      in order to avoid future merge trouble.
      7f4c749d
    • Marko Mäkelä's avatar
      MDEV-21201: Add --sorted_result to the test, for 10.4 · 976abe64
      Marko Mäkelä authored
      On MariaDB 10.4 (commit 4db4b773),
      the query results would not be sorted, which creates random result
      differences. Let us explicitly sort the results already in 10.3
      in order to avoid future merge trouble.
      976abe64
    • Alexander Barkov's avatar
      MDEV-19632 Replication aborts with ER_SLAVE_CONVERSION_FAILED upon CREATE ... SELECT in ORACLE mode · d63631c3
      Alexander Barkov authored
      - Adding optional qualifiers to data types:
          CREATE TABLE t1 (a schema.DATE);
        Qualifiers now work only for three pre-defined schemas:
      
          mariadb_schema
          oracle_schema
          maxdb_schema
      
        These schemas are virtual (hard-coded) for now, but may turn into real
        databases on disk in the future.
      
      - mariadb_schema.TYPE now always resolves to a true MariaDB data
        type TYPE without sql_mode specific translations.
      
      - oracle_schema.DATE translates to MariaDB DATETIME.
      
      - maxdb_schema.TIMESTAMP translates to MariaDB DATETIME.
      
      - Fixing SHOW CREATE TABLE to use a qualifier for a data type TYPE
        if the current sql_mode translates TYPE to something else.
      
      The above changes fix the reported problem, so this script:
      
          SET sql_mode=ORACLE;
          CREATE TABLE t2 AS SELECT mariadb_date_column FROM t1;
      
      is now replicated as:
      
          SET sql_mode=ORACLE;
          CREATE TABLE t2 (mariadb_date_column mariadb_schema.DATE);
      
      and the slave can unambiguously treat DATE as the true MariaDB DATE
      without ORACLE specific translation to DATETIME.
      
      Similar,
      
          SET sql_mode=MAXDB;
          CREATE TABLE t2 AS SELECT mariadb_timestamp_column FROM t1;
      
      is now replicated as:
      
          SET sql_mode=MAXDB;
          CREATE TABLE t2 (mariadb_timestamp_column mariadb_schema.TIMESTAMP);
      
      so the slave treats TIMESTAMP as the true MariaDB TIMESTAMP
      without MAXDB specific translation to DATETIME.
      d63631c3
  2. 31 Jul, 2020 10 commits
    • Oleksandr Byelkin's avatar
      MDEV-21201 No records produced in information_schema query, depending on projection · a8458a23
      Oleksandr Byelkin authored
      In case of NATURAL JOIN / USING mark all field (one table can not be opened
      in any case so optimisation does not worth it).
      
      IMHO table should be checked for used fields and filled after prepare,
      when we will fave whole info about used fields but it is too big change
      for a bugfix. Which will be made later by Serg patch
      a8458a23
    • Marko Mäkelä's avatar
      Merge 10.2 into 10.3 · 78c2a5ab
      Marko Mäkelä authored
      78c2a5ab
    • Thirunarayanan Balathandayuthapani's avatar
      MDEV-14711 Fix-up · 5ec40fbb
      Thirunarayanan Balathandayuthapani authored
      5ec40fbb
    • Marko Mäkelä's avatar
      Merge 10.2 into 10.3 · 8bb2170d
      Marko Mäkelä authored
      8bb2170d
    • Marko Mäkelä's avatar
      Merge 10.2 into 10.3 · 66ec3a77
      Marko Mäkelä authored
      66ec3a77
    • Thirunarayanan Balathandayuthapani's avatar
      MDEV-22511 innodb.truncate_foreign failed in buildbot with wrong error code · a6066e23
      Thirunarayanan Balathandayuthapani authored
      - Adding lock_wait_timeout value as 1 make sure that truncate table
      fails instead of making MDL timeout.
      a6066e23
    • Marko Mäkelä's avatar
      MDEV-11799 Doublewrite recovery can corrupt data pages · 879ba197
      Marko Mäkelä authored
      The purpose of the InnoDB doublewrite buffer is to make InnoDB
      tolerant against cases where the server was killed in the middle
      of a page write. (In Linux, killing a process may interrupt a
      write system call, typically on a 4096-byte boundary.)
      
      There may exist multiple copies of a page number in the doublewrite
      buffer. Recovery should choose the latest valid copy of the page.
      By design, the FIL_PAGE_LSN must not precede the latest checkpoint LSN
      nor be later than the end of the recovered log.
      
      For page_compressed and encrypted pages, we were missing proper
      consistency checks. In the 10.4 data set generated for in MDEV-23231,
      the data file contained a valid page_compressed page, and an
      identical copy of that page was also present in the doublewrite
      buffer. But, recovery would incorrectly consider the page invalid
      and restore an uncompressed copy of the same page that had been
      written before the log checkpoint. (In fact, no redo log was to
      be applied to that page.)
      
      buf_dblwr_process(): Validate the FIL_PAGE_LSN in the doublewrite
      buffer pages, and always skip page 0, because those pages should
      have been recovered by Datafile::restore_from_doublewrite() if
      necessary.
      
      Datafile::restore_from_doublewrite(): Choose the latest applicable
      page from the doublewrite buffer.
      
      recv_dblwr_t::find_page(): Also validate encrypted or
      page_compressed pages.
      
      recv_dblwr_t::validate_page(): New function to validate a page,
      either a copy in a data file or in the doublewrite buffer.
      Also validate encrypted or page_compressed pages.
      
      This is joint work with Thirunarayanan Balathandayuthapani.
      879ba197
    • Marko Mäkelä's avatar
      MDEV-23198 Crash in REPLACE · f35d1721
      Marko Mäkelä authored
      row_vers_impl_x_locked_low(): clust_offsets may point to memory
      that is allocated by mem_heap_alloc() and may have been freed.
      For initializing clust_offsets, try to use the stack-allocated
      buffer instead of a pointer that may point to freed memory.
      
      This fixes a regression that was introduced in
      commit f0aa073f (MDEV-20950).
      f35d1721
    • Nikita Malyavin's avatar
      MDEV-18042 Server crashes upon adding a non-null date column under... · fd0abc89
      Nikita Malyavin authored
      MDEV-18042 Server crashes upon adding a non-null date column under NO_ZERO_DATE with ALGORITHM=INPLACE
      
      accept table_name and db_name instead of table_share in make_truncated_value_warning
      fd0abc89
    • Nikita Malyavin's avatar
      MDEV-19338 InnoDB: Failing assertion: !cursor->index->is_committed() · 91ebf184
      Nikita Malyavin authored
      Call mark_columns_per_binlog_row_image before find_row() to set up table->vcol_set early,
      so the virtual column value will be updated after record read (ha_rnd_pos/ha_index_next/etc)
      by table->update_virtual_fields() call
      91ebf184
  3. 30 Jul, 2020 5 commits
    • Marko Mäkelä's avatar
      MDEV-23334 Crash in rec_get_nth_cfield()/rec_offs_validate() · 6053eb1c
      Marko Mäkelä authored
      rec_get_nth_cfield(): Remove a bogus debug assertion.
      The function may be invoked by innobase_rec_to_mysql()
      for reporting a duplicate key error during CREATE UNIQUE INDEX
      or ALTER TABLE...ADD UNIQUE KEY, and in that case the record
      will be missing the 5-byte or 6-byte fixed header.
      
      It turns out that in every other code path leading to
      rec_get_nth_cfield() we either invoked rec_get_offsets()
      ourselves or asserted rec_offs_validate(). So, we can
      safely remove the assertion and make debug builds
      smaller and faster.
      6053eb1c
    • Vladislav Vaintroub's avatar
      MDEV-21101 skip test for embedded · 0435fcf9
      Vladislav Vaintroub authored
      0435fcf9
    • Thirunarayanan Balathandayuthapani's avatar
      MDEV-23332 Index online status assert failure in btr_search_drop_page_hash_index · 8a612314
      Thirunarayanan Balathandayuthapani authored
      Problem:
      ========
      In row_merge_drop_indexes(), InnoDB drops only the index from
      dictionary and frees the index pages but it maintains the index
      object if the table is being used by other DML threads. It sets
      the online status of the index to ONLINE_INDEX_ABORTED_DROPPED.
      Removing the index from dictionary doesn't remove the
      corressponding ahi entries of the index. When block is being
      reused, InnoDB tries to remove ahi entries for the block and
      it fails if index online status is ONLINE_INDEX_ABORTED_DROPPED.
      
      Fix:
      ====
      MDEV-22456 allows the index ahi entries to be dropped lazily.
      so checking online status in btr_search_drop_page_hash_index()
      is meaningless and should be removed.
      8a612314
    • Vladislav Vaintroub's avatar
      MDEV-21101 unexpected wait_timeout with pool-of-threads · 71015d84
      Vladislav Vaintroub authored
      Due to restricted size of the threadpool, execution of client queries can
      be delayed (queued) for a while. This delay was interpreted as client
      inactivity, and connection is closed, if client idle time + queue time
      exceeds wait_timeout.
      
      But users did not expect queue time to be included into wait_timeout.
      
      This patch changes the behavior. We don't close connection anymore,
      if there is some unread data present on connection,
      even if wait_timeout is exceeded. Unread data means that client
      was not idle, it sent a query, which we did not have time to process yet.
      71015d84
    • Marko Mäkelä's avatar
      MDEV-23339 innodb_force_recovery=2 may still abort the rollback of recovered transactions · c5d4dd25
      Marko Mäkelä authored
      trx_rollback_active(), trx_rollback_resurrected(): Replace
      an incorrect condition that we failed to replace in
      commit b68f1d84 (MDEV-21217).
      c5d4dd25
  4. 29 Jul, 2020 3 commits
  5. 28 Jul, 2020 7 commits
  6. 27 Jul, 2020 4 commits
    • Daniel Black's avatar
      MDEV-12474: rocksdb: mtr - rocksdb.concurrent_alter use sh · 186d9d0d
      Daniel Black authored
      FreeBSD doesn't have bash installed by default and sh
      has sufficient job control for this test.
      
      $  mysql-test/mtr --mem --max-test-fail=30 --force --parallel=1 rocksdb.concurrent_alter
      Logging: /home/dan/mariadb-server-10.5/mysql-test/mysql-test-run.pl  --mem --max-test-fail=30 --force --parallel=1 rocksdb.concurrent_alter
      vardir: /usr/home/dan/build-mariadb-server-10.5/mysql-test/var
      Checking leftover processes...
      Removing old var directory...
      Creating var directory '/usr/home/dan/build-mariadb-server-10.5/mysql-test/var'...
       - symlinking 'var' to '/tmp/var_auto_P81m'
      Checking supported features...
      MariaDB Version 10.5.4-MariaDB
       - SSL connections supported
      
       - binaries built with wsrep patch
      Collecting tests...
      Installing system database...
      
      ==============================================================================
      
      TEST                                      RESULT   TIME (ms) or COMMENT
      --------------------------------------------------------------------------
      
      worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 16000..16019
      rocksdb.concurrent_alter 'write_committed' [ pass ]  16348
      rocksdb.concurrent_alter 'write_prepared' [ pass ]  16771
      --------------------------------------------------------------------------
      The servers were restarted 1 times
      Spent 33.119 of 41 seconds executing testcases
      
      Completed: All 2 tests were successful.
      
      $ uname -a
      FreeBSD freebsd 12.1-RELEASE-p6 FreeBSD 12.1-RELEASE-p6 GENERIC  amd64
      186d9d0d
    • Thirunarayanan Balathandayuthapani's avatar
      MDEV-23233: Race condition for btr_search_drop_page_hash_index() in buf_page_create() · a1f899a8
      Thirunarayanan Balathandayuthapani authored
      commit ad6171b9 (MDEV-22456)
      introduced code to buf_page_create() that would lazily drop
      adaptive hash index entries for an index that has been
      evicted from the data dictionary cache.
      
      Unfortunately, that call was missing adequate protection.
      While the btr_search_drop_page_hash_index(block) was executing,
      the block could be reused for something else.
      
      buf_page_create(): If btr_search_drop_page_hash_index() must be
      invoked, pin the block before releasing the buf_pool->page_hash lock,
      so that the block cannot be grabbed by other threads.
      a1f899a8
    • Jan Lindström's avatar
      Enable fixed test case. · b4c74210
      Jan Lindström authored
      b4c74210
    • Varun Gupta's avatar
      MDEV-18916: crash in Window_spec::print_partition() with decimals · a6410deb
      Varun Gupta authored
      Removed an unnecessary ifndef which was printing the window name for a named
      window only in the case of debug build. The print() for the window function
      should behave in the same way on both release and debug builds.
      a6410deb
  7. 24 Jul, 2020 4 commits
    • Thirunarayanan Balathandayuthapani's avatar
      MDEV-14711 Assertion `mode == 16 || mode == 12 ||... · 5f1ec5cb
      Thirunarayanan Balathandayuthapani authored
      MDEV-14711 Assertion `mode == 16 || mode == 12 || !fix_block->page.file_page_was_freed' failed in buf_page_get_gen (rollback requesting a freed undo page)
      
      Problem:
      =======
      In buf_cur_optimistic_latch_leaves(), requesting a left block with BTR_GET
      after releasing current block. But there is no guarantee that left block
      could be still available.
      
      Fix:
      ====
      
      (1) In btr_cur_optimistic_latch_leaves(), replace the BUF_GET with
      BUF_GET_POSSIBLY_FREED for fetching left block.
      (2) Once InnoDB acquires left block, it should check FIL_PAGE_NEXT with
      current block page number. If not, release cursor->left_block and return
      false.
      5f1ec5cb
    • mkaruza's avatar
      MDEV-15236: galera_ist_progress fails when trying to read transfer status · a3a249c7
      mkaruza authored
      Fixed by new  recording test recording
      a3a249c7
    • sjaakola's avatar
      MDEV-20928 mtr test galera.galera_var_innodb_disallow_writes test failure · c9928cc0
      sjaakola authored
      The sporadic test hangs happen because of mutex dealock between innodb
      background threads and two test connection executions.
      The test sets variable innodb_disallow_writes, which blocks all writes
      to filesyste. The test logic is to execute an INSERT, which should hang
      because of filesytstem writes are blocked, and through another session
      verify by SELECT that this hanging happens. The SELECT session will then
      release innodb_disallow_writes blocking.
      
      However, filesystem write  blocking affects also innodb background threads
      and they may hang while keeping some other resources locked.
      As an example, in one test hang situation, buffer pool access was blocked.
      And, if buffer pool is blocked, the test connections will be blocked as well,
      and the SELECT session will not be able to continue to release the
      innodb_disallow_writes.
      
      The fix in this commit is refactoring of the test logic.
      The test will now set first innodb_disallow_writes blocking, and then record
      a hash of data directory's filesystem contents. This works as checksum of the
      state of data on the datadirectory.
      
      Then some SQL load is tried on both nodes, these sessions will be blocking
      due to frozen file system state. The test will have a short sleep to allow
      innodb background threads to loop and possibly encounter innodb_disallow_writes
      blocking as well.
      
      After the sleep, the test will record file system checksun for the second time,
      and then release the innodb_disallow-writes blocking.
      
      Finally, the two checksums are compared, they should be identical to verify that
      nothing was written on datadirectory during the test execution.
      
      The checksum is implemented by md5sum hash over all files found in datadirectory
      by find command. all these file hashes are hashed together by one more md5sum.
      
      The test therefore depends on md5sum and find. find may work differently with some
      OS distributions, e.g. freebsd may be problematic.
      c9928cc0
    • Jan Lindström's avatar
      MDEV-18177 : Galera test failure on galera_autoinc_sst_mariabackup · ba23e6d7
      Jan Lindström authored
      Add wait_condition
      ba23e6d7
  8. 23 Jul, 2020 3 commits
    • Thirunarayanan Balathandayuthapani's avatar
      MDEV-23134 SEGV in dict_load_table_one during restart after server crash · b3b1c51e
      Thirunarayanan Balathandayuthapani authored
      Problem:
      ========
      dict_load_table_one() doesn't handle the scenario where clustered
      index page is FIL_NULL when DICT_ERR_IGNORE_INDEX_ROOT mode
      is set.
      
      Fix:
      ====
      InnoDB should set the file_unreadable when it can't find the
      clustered index root page.
      b3b1c51e
    • Marko Mäkelä's avatar
      MDEV-23244 ALTER TABLE…ADD PRIMARY KEY fails to flag duplicates · 1656ea28
      Marko Mäkelä authored
      The fix of MDEV-13654 (commit ff81faf6)
      wrongly caused ADD PRIMARY KEY to ignore duplicate PRIMARY KEY values
      caused by concurrent DML transactions that had been started before the
      ALTER TABLE operation (but did not access the table before the ALTER TABLE
      started).
      
      row_ins_duplicate_online(): Always report a duplicate key error
      if DB_TRX_ID had been reset (it belongs to a transaction that had
      started before the ALTER TABLE operation).
      1656ea28
    • Thirunarayanan Balathandayuthapani's avatar
      MDEV-20638 Remove the deadcode from srv_master_thread() and srv_active_wake_master_thread_low() · fe39d02f
      Thirunarayanan Balathandayuthapani authored
      - Due to commit fe95cb2e (MDEV-16125),
      InnoDB master thread does not need to call srv_resume_thread()
      and therefore there is no need to wake up the thread.
      Due to the above patch, InnoDB should remove the following dead code.
      
      srv_check_activity(): Makes the parameter as in,out and returns the
      recent activity value
      
      innobase_active_small(): Removed
      
      srv_active_wake_master_thread(): Removed
      
      srv_wake_master_thread(): Removed
      
      srv_active_wake_master_thread_low(): Removed
      
      Simplify srv_master_thread() and remove switch cases, added the assert.
      
      Replace srv_wake_master_thread() with srv_inc_activity_count()
      
      INNOBASE_WAKE_INTERVAL: Removed
      fe39d02f