1. 20 Oct, 2019 3 commits
    • Monty's avatar
      Write error message if aria_log.??????? files are missing · 7def2877
      Monty authored
      This can happen if one uses a backup where not all aria_log.* files
      are copied or if the last one is too short. In this case the data
      files will contain data that is not in the logs and recovery will fail.
      
      Other things:
      - Fixed tprint() to not print extra new line to debug trace
      7def2877
    • Monty's avatar
      Simplify TABLE::decide_logging_format() · 67687d06
      Monty authored
      - Use local variables table and share to simplify code
      - Use sql_command_flags to detect what kind of command was used
      - Added CF_DELETES_DATA to simplify detecton of delete commands
      - Removed duplicate error in create_table_from_items().
      67687d06
    • Monty's avatar
      Fixes for binary logging --read-only mode · b62101f8
      Monty authored
      - Any temporary tables created under read-only mode will never be logged
        to binary log.  Any usage of these tables to update normal tables, even
        after read-only has been disabled, will use row base logging (as the
        temporary table will not be on the slave).
      - Analyze, check and repair table will not be logged in read-only mode.
      
      Other things:
      - Removed not used varaibles in
        MYSQL_BIN_LOG::flush_and_set_pending_rows_event.
      - Set table_share->table_creation_was_logged for all normal tables.
      - THD::binlog_query() now returns -1 if statement was not logged., This
        is used to update table_share->table_creation_was_logged.
      - Don't log admin statements in opt_readonly is set.
      - Table's that doesn't have table_creation_was_logged will set binlog format to row
        logging.
      - Removed not needed/wrong setting of table->s->table_creation_was_logged
        in create_table_from_items()
      b62101f8
  2. 18 Oct, 2019 2 commits
  3. 17 Oct, 2019 4 commits
    • Oleksandr Byelkin's avatar
      MDEV-20074: Lost connection on update trigger · de2186dd
      Oleksandr Byelkin authored
      Instead of checking lex->sql_command which does not corect in case of triggers
      mark tables for insert.
      de2186dd
    • Marko Mäkelä's avatar
      Simplify row_undo_ins_remove_sec_low() · fa929f7c
      Marko Mäkelä authored
      Reduce the scope of some variables, remove a goto and a redundant
      assertion.
      
      For B-tree secondary indexes, this function can remove a delete-marked
      purgeable record, in case a row rollback of the INSERT was initiated
      due to an error in an earlier secondary index.
      fa929f7c
    • Marko Mäkelä's avatar
      b0278302
    • Marko Mäkelä's avatar
      MDEV-20852 BtrBulk is unnecessarily holding dict_index_t::lock · fa32d28f
      Marko Mäkelä authored
      The BtrBulk class, which was introduced in MySQL 5.7, is by design
      the exclusive writer to an index. It is therefore unnecessary to
      acquire the dict_index_t::lock in that code.
      
      Holding the dict_index_t::lock would unnecessarily block other threads
      (SQL connections and the InnoDB purge threads) from buffering concurrent
      modifications to being-created secondary indexes.
      
      This fix is motivated by a change in MySQL 5.7.28:
      Bug #29008298 MYSQLD CRASHES ITSELF WHEN CREATING INDEX
      mysql/mysql-server@f9fb96c20f9d190f654e7aa2387255bf80fd6e45
      
      PageBulk::init(), PageBulk::latch(): Never acquire m_index->lock.
      
      PageBulk::storeExt(): Remove some pointer indirection, and improve
      a debug assertion that seems to prove that some code is redundant.
      
      BtrBulk::pageCommit(): Assert that m_index->lock is not being held.
      
      btr_blob_log_check_t: Do not acquire m_index->lock if
      m_op == BTR_STORE_INSERT_BULK. Add UNIV_UNLIKELY hints around
      that condition.
      
      btr_store_big_rec_extern_fields(): Allow index->lock not to be held
      while op == BTR_STORE_INSERT_BULK. Add UNIV_UNLIKELY hints around
      that condition.
      fa32d28f
  4. 16 Oct, 2019 2 commits
  5. 15 Oct, 2019 2 commits
    • Monty's avatar
      MDEV-10748 Server crashes in ha_maria::implicit_commit · 7952f772
      Monty authored
      Problem was in a combination of LOCK TABLES on several Aria
      tables followed by an ALTER TABLE. After the ALTER TABLE there
      was a force close + reopen of the alter table. The close failed
      because the table was still part of a transaction.
      
      Fixed by calling extra(HA_EXTRA_PREPARE_FOR_FORCED_CLOSE) as
      part of closing the table, which ensures that the table is not
      anymore part of the current transaction.
      7952f772
    • Aleksey Midenkov's avatar
      MDEV-20812 Unexpected ER_ROW_IS_REFERENCED_2 upon DELETE from versioned table with FK · 7ae0be25
      Aleksey Midenkov authored
      MDEV-16210 original case was wrongly allowed versioned DELETE from
      referenced table where reference is by non-primary key. InnoDB UPDATE
      has optimization for new rows not changing its clustered index
      position. In this case InnoDB doesn't update all secondary indexes and
      misses the one holding the referenced key. The fix was to disable this
      optimization for versioned DELETE. In case of versioned DELETE we
      forcely update all secondary indexes and therefore check them for
      constraints.
      
      But the above fix raised another problem with versioned DELETE on
      foreign table side. In case when there was no corresponding record in
      referenced table (illegal foreign reference can be done with "set
      foreign_key_checks=off") there was spurious constraint check (because
      versioned DELETE is actually UPDATE) and hence the operation failed
      with constraint error.
      
      MDEV-16210 tried to fix the above problem by checking foreign table
      instead of referenced table and that at least was illegal.
      
      Constraint check is done by row_ins_check_foreign_constraint() no
      matter what kind of table is checked, referenced or foreign
      (controlled by check_ref argument).
      
      Referenced table is checked by row_upd_check_references_constraints().
      Foreign table is checked by row_ins_check_foreign_constraints().
      
      Current fix rolls back the wrong fix for the above problem and
      disables referenced table check for DELETE on foreign side by
      introducing `check_foreign` argument which when set to *false* skips
      row_ins_check_foreign_constraints() call.
      7ae0be25
  6. 14 Oct, 2019 5 commits
    • Eugene Kosov's avatar
      MDEV-20778 UBSAN: call to function free_rpl_filter() through pointer to incorrect function type · 4d147855
      Eugene Kosov authored
      Proper C-style type erasure is done via void*, not via char* or something else.
      
      free_key_cache()
      free_rpl_filter(): types were fixed to avoid function pointer type cast which
      is still undefined behavior.
      
      Note, that casting from void* to any other pointer type is safe and correct.
      4d147855
    • Marko Mäkelä's avatar
      MDEV-20813: Do not rotate keys for unallocated pages · f989c0ce
      Marko Mäkelä authored
      fil_crypt_rotate_page(): Skip the key rotation for pages that carry 0
      in FIL_PAGE_TYPE. This avoids not only unnecessary writes, but also
      failures of the recently added debug assertion in
      buf_flush_init_for_writing() that the FIL_PAGE_TYPE should be nonzero.
      
      Note: the debug assertion can fail if the file was originally created
      before MySQL 5.5. In old InnoDB versions, FIL_PAGE_TYPE was only
      initialized for B-tree pages, to FIL_PAGE_INDEX. For any other pages,
      the field could be garbage, including FIL_PAGE_INDEX. In MariaDB 10.2
      and later, buf_flush_init_for_writing() would initialize the
      FIL_PAGE_TYPE on such old pages, but only after passing the debug
      assertion that insists that pages have a nonzero FIL_PAGE_TYPE.
      Thus, the debug assertion at the start of buf_flush_init_for_writing()
      can fail when upgrading from very old debug files. This assertion is
      only present in debug builds, not release builds.
      f989c0ce
    • Marko Mäkelä's avatar
      Merge 10.1 into 10.2 · 4f9f3f19
      Marko Mäkelä authored
      Do not merge the InnoDB changes for MDEV-20813.
      4f9f3f19
    • Marko Mäkelä's avatar
      MDEV-20813: Remove the buf_flush_init_for_writing() assertion · ae702d76
      Marko Mäkelä authored
      Old InnoDB/XtraDB versions only initialized FIL_PAGE_TYPE for
      B-tree pages (to FIL_PAGE_INDEX), and left it uninitialized
      (possibly containing FIL_PAGE_INDEX) for others. In MySQL
      or MariaDB 5.5, the field is initialized on almost all pages,
      but still not all of them.
      
      In MariaDB 10.2 and later, buf_flush_init_for_writing() would
      initialize the FIL_PAGE_TYPE on such old pages, but only after
      passing the debug assertion that we are now removing from 10.1.
      There, we will be able to modify fil_crypt_rotate_page() so
      that it will skip the key rotation for pages that contain 0
      in FIL_PAGE_TYPE.
      
      In MariaDB 10.1, there is no logic that would initialize
      FIL_PAGE_TYPE on data pages in old data files after an update.
      So, encryption key rotation may routinely cause page flushes
      on pages that contain 0 in FIL_PAGE_TYPE.
      ae702d76
    • Marko Mäkelä's avatar
      MDEV-19740: Fix C++11 violations caught by GCC 9.2.1 · 2920377a
      Marko Mäkelä authored
      This is a backport of commit ec28f953
      to MariaDB Server 10.1.
      2920377a
  7. 12 Oct, 2019 5 commits
    • Marko Mäkelä's avatar
      MDEV-20814 Assertion index->is_instant() failed on trivial upgrade from 10.1 · fa6c6062
      Marko Mäkelä authored
      rec_init_offsets(): Relax the assertion that was added in
      commit 01f45bec
      to catch ROW_FORMAT=REDUNDANT records that have fewer fields
      than expected.
      
      This assertion would fail when accessing the records of the
      built-in InnoDB table SYS_INDEXES. The column MERGE_THRESHOLD
      had been effectively instantly added in MariaDB Server 10.2
      (and MySQL 5.7), but is_instant() does not hold for that index.
      Relax the assertion, so that it will not fail in this case.
      fa6c6062
    • Marko Mäkelä's avatar
      Merge 10.2 into 10.3 · bb450b1f
      Marko Mäkelä authored
      bb450b1f
    • Marko Mäkelä's avatar
      MDEV-20813 Assertion failure in buf_flush_init_for_writing() for... · 361e8284
      Marko Mäkelä authored
      MDEV-20813 Assertion failure in buf_flush_init_for_writing() for innodb_immediate_scrub_data_uncompressed=ON
      
      The assertion that was added in
      commit c0c003be
      to augment the fix of MDEV-20805 turns out to be invalid when
      innodb_immediate_scrub_data_uncompressed is enabled.
      In this mode, fsp_init_file_page() will be invoked on data pages
      that have been freed, causing writes of almost-all-zero pages.
      
      btr_page_free(): Adjust the comment.
      
      buf_flush_init_for_writing(): Disable the assertion with a note
      that it should be re-enabled in MDEV-15528.
      361e8284
    • Marko Mäkelä's avatar
      Merge 10.2 into 10.3 · 8e3d85e1
      Marko Mäkelä authored
      8e3d85e1
    • Marko Mäkelä's avatar
      Merge 10.1 into 10.2 · 2227dec4
      Marko Mäkelä authored
      2227dec4
  8. 11 Oct, 2019 14 commits
    • Julius Goryavsky's avatar
      MDEV-20728: /usr/sbin/mysqld: unknown variable 'defaults-group-suffix=mysqld1 · 2ae02c29
      Julius Goryavsky authored
      When the mysqld_multi script passes the --defaults-group-suffix
      option to mysqld, it must remove the initial substring with the
      group name ("mysqld") from option value, because otherwise substring
      "mysqld" will be added to the group name and then the group name
      will contain the word "mysqld" twice, which is wrong, because
      mysqld itself adds the suffix received to the group name.
      2ae02c29
    • Julius Goryavsky's avatar
      MDEV-20728: /usr/sbin/mysqld: unknown variable 'defaults-group-suffix=mysqld1 · 4ca0abe9
      Julius Goryavsky authored
      When the mysqld_multi script passes the --defaults-group-suffix
      option to mysqld, it must remove the initial substring with the
      group name ("mysqld") from option value, because otherwise substring
      "mysqld" will be added to the group name and then the group name
      will contain the word "mysqld" twice, which is wrong, because
      mysqld itself adds the suffix received to the group name.
      4ca0abe9
    • Julius Goryavsky's avatar
      MDEV-20728: /usr/sbin/mysqld: unknown variable 'defaults-group-suffix=mysqld1 · 0ecc85c5
      Julius Goryavsky authored
      When the mysqld_multi script passes the --defaults-group-suffix
      option to mysqld, it must remove the initial substring with the
      group name ("mysqld") from option value, because otherwise substring
      "mysqld" will be added to the group name and then the group name
      will contain the word "mysqld" twice, which is wrong, because
      mysqld itself adds the suffix received to the group name.
      0ecc85c5
    • Marko Mäkelä's avatar
      Fix -std=c++98 -Wzero-length-array · 38736928
      Marko Mäkelä authored
      This is another follow-up fix to
      commit b393e2cb
      which turned out to be still broken.
      
      Replace the C++11 keyword 'constexpr' with #define.
      
      debug_sync_t::str: Remove the zero-length array.
      Replace sync->str with reinterpret_cast<char*>(&sync[1]).
      38736928
    • Marko Mäkelä's avatar
      After-merge fix: Correct an assertion · 1e1b53cc
      Marko Mäkelä authored
      During IMPORT TABLESPACE, we do invoke
      buf_flush_init_for_writing() with block==NULL and newest_lsn!=0.
      1e1b53cc
    • Marko Mäkelä's avatar
      Merge 10.1 into 10.2 · 966d97b5
      Marko Mäkelä authored
      966d97b5
    • Marko Mäkelä's avatar
      Fix CMAKE_BUILD_TYPE=Debug · 1fd1ef25
      Marko Mäkelä authored
      Remove unused variables and type mismatch that was introduced
      in commit b393e2cb
      
      Also, fix a typo in the documentation of the parameter, and
      update the test.
      1fd1ef25
    • Marko Mäkelä's avatar
      MDEV-20805 follow-up: Catch writes of bogus pages · c0c003be
      Marko Mäkelä authored
      buf_flush_init_for_writing(): Assert that FIL_PAGE_TYPE is set
      except when creating a new data file with a dummy first page.
      
      buf_dblwr_create(): Ensure that FIL_PAGE_TYPE on all pages
      will be initialized. Reset buf_dblwr_being_created at the end.
      c0c003be
    • Marko Mäkelä's avatar
      Merge 5.5 into 10.1 · cbfd6882
      Marko Mäkelä authored
      cbfd6882
    • Marko Mäkelä's avatar
      MDEV-20805 ibuf_add_free_page() is not initializing FIL_PAGE_TYPE first · ea61b796
      Marko Mäkelä authored
      In the function recv_parse_or_apply_log_rec_body() there are debug checks
      for validating the state of the page when redo log records are being
      applied. Most notably, FIL_PAGE_TYPE should be set before anything else
      is being written to the page.
      
      ibuf_add_free_page(): Set FIL_PAGE_TYPE before performing any other changes.
      ea61b796
    • Marko Mäkelä's avatar
      MDEV-19455: Avoid SET DEBUG_DBUG='-d,...' construct · 4ebaf813
      Marko Mäkelä authored
      Apply the correct pattern for debug instrumentation:
      
      SET @save_dbug=@@debug_dbug;
      SET debug_dbug='+d,...';
      ...
      SET debug_dbug=@save_dbug;
      
      Numerous tests use statements of the form
      
      SET debug_dbug='-d,...';
      
      which will inadvertently enable all DBUG tracing output,
      causing unnecessary waste of resources.
      4ebaf813
    • Marko Mäkelä's avatar
      MDEV-20804 Speed up main.index_merge_innodb · 5ef12244
      Marko Mäkelä authored
      The test main.index_merge_innodb is taking very much time,
      especially on later versions (10.2 and 10.3).
      
      Some of this could be attributed to the use of INSERT...SELECT,
      which is time-consumingly creating explicit record locks in InnoDB
      for the locking read in the SELECT part.
      
      In 10.3 and later, some slowness can be attributed to MDEV-12288,
      which makes the InnoDB purge thread spend time to reset transaction
      identifiers in the inserted records. If we prevent purge from
      running before all tables are dropped, the test seems to be
      10% faster on an unoptimized debug build on 10.5. (A proper fix
      would be to implement MDEV-515 and stop writing row-level undo log
      records for inserts into an empty table or partition.)
      
      At the same time, it should not hurt to make main.index_merge_myisam
      to use the sequence engine. Not only could it be a little faster,
      but the test would be slightly more readable.
      5ef12244
    • Nikita Malyavin's avatar
      MDEV-18546 ASAN heap-use-after-free in innobase_get_computed_value / row_purge · 350e46a8
      Nikita Malyavin authored
      the bug was already fixed in MDEV-17005, so now only test is added
      350e46a8
    • Nikita Malyavin's avatar
  9. 10 Oct, 2019 3 commits
    • Marko Mäkelä's avatar
      MDEV-19783: Relax an assertion · 4cdb72f2
      Marko Mäkelä authored
      btr_page_get_split_rec_to_left(): Assert that in the leftmost leaf page,
      if the metadata record exists, index->is_instant() must hold.
      The assertion of commit 01f45bec
      could fail during innobase_instant_try().
      4cdb72f2
    • Marko Mäkelä's avatar
      MDEV-19783: Add more assertions · 01f45bec
      Marko Mäkelä authored
      btr_page_get_split_rec_to_left(): Assert that in the leftmost leaf page,
      the metadata record exists if and only if index->is_instant().
      
      page_validate(): Correct the wording of a message.
      
      rec_init_offsets(): Assert that whenever a record is in "instant ALTER"
      format, index->is_instant() must hold.
      01f45bec
    • Marko Mäkelä's avatar
      Merge 10.2 into 10.3 · 7f84e3ad
      Marko Mäkelä authored
      7f84e3ad