1. 13 Feb, 2020 28 commits
    • Marko Mäkelä's avatar
      MDEV-21174: Remove mlog_write_initial_log_record_fast() · 2e7a0842
      Marko Mäkelä authored
      Pass buf_block_t* to all functions that write redo log.
      Specifically, replace the parameters page,page_zip
      with buf_block_t* block in page_zip_ functions.
      2e7a0842
    • Marko Mäkelä's avatar
      MDEV-12353: Remove mlog_open_and_write_index() · 498f84a8
      Marko Mäkelä authored
      Now that all logical log records have been replaced,
      the function mlog_parse_index() is only needed for crash-upgrading
      from older versions.
      498f84a8
    • Marko Mäkelä's avatar
      MDEV-12353: Replace MLOG_REC_INSERT,MLOG_COMP_REC_INSERT · 08ba3887
      Marko Mäkelä authored
      page_mem_alloc_free(), page_dir_set_n_heap(), page_ptr_set_direction():
      Merge with the callers.
      
      page_direction_reset(), page_direction_increment(),
      page_zip_dir_insert(), page_zip_write_rec_ext(), page_zip_write_rec():
      Add the parameter mtr, and write log.
      
      PageBulk::insert(), PageBulk::finish(): Write log for all changes.
      
      page_cur_rec_insert(), page_cur_insert_rec_write_log(),
      page_cur_insert_rec_write_log(): Remove.
      
      page_rec_set_next(), page_header_set_field(), page_header_set_ptr():
      Remove. Use lower-level operations with or without logging.
      
      page_zip_dir_add_slot(): Move to the same compilation unit with
      its only caller, page_cur_insert_rec_zip().
      
      page_cur_insert_rec_zip(): Mark pieces of code that must be skipped
      once this task is completed.
      
      btr_defragment_chunk(): Before starting a mini-transaction that
      is writing (a lot), invoke log_free_check(). This should allow
      the test innodb.innodb_defrag_concurrent to pass with the
      mtr default_mysqld.cnf setting of innodb_log_file_size=10M.
      
      MLOG_BUF_MARGIN: Remove.
      08ba3887
    • Marko Mäkelä's avatar
      MDEV-12353: Replace MLOG_ZIP_PAGE_COMPRESS · 2c4d5aa0
      Marko Mäkelä authored
      page_zip_compress_write_log(): Write MLOG_INIT_FILE_PAGE2
      and MLOG_ZIP_WRITE_STRING records instead of MLOG_ZIP_PAGE_COMPRESS.
      
      This depends on the changes to buf_page_io_complete() and friends
      in the parent commit.
      2c4d5aa0
    • Marko Mäkelä's avatar
      MDEV-12353: Replace MLOG_*LIST_*_DELETE and MLOG_*REC_DELETE · 2a77b2a5
      Marko Mäkelä authored
      No longer write the following redo log records:
      MLOG_COMP_LIST_END_DELETE, MLOG_LIST_END_DELETE,
      MLOG_COMP_LIST_START_DELETE, MLOG_LIST_START_DELETE,
      MLOG_REC_DELETE,MLOG_COMP_REC_DELETE.
      
      Each individual deleted record will be logged separately
      using physical log records.
      
      page_dir_slot_set_n_owned(),
      page_zip_rec_set_owned(), page_zip_dir_delete(), page_zip_clear_rec():
      Add the parameter mtr, and write redo log.
      
      page_dir_slot_set_rec(): Remove. Replaced with lower-level operations
      that write redo log when necessary.
      
      page_rec_set_n_owned(): Replaces rec_set_n_owned_old(),
      rec_set_n_owned_new().
      
      rec_set_heap_no(): Replaces rec_set_heap_no_old(), rec_set_heap_no_new().
      
      page_mem_free(), page_dir_split_slot(), page_dir_balance_slot():
      Add the parameter mtr.
      
      page_dir_set_n_slots(): Merge with the caller page_dir_split_slot().
      
      page_dir_slot_set_rec(): Merge with the callers page_dir_split_slot()
      and page_dir_balance_slot().
      
      page_cur_insert_rec_low(), page_cur_insert_rec_zip():
      Suppress the logging of lower-level operations.
      
      page_cur_delete_rec_write_log(): Remove.
      
      page_cur_delete_rec(): Do not tolerate mtr=NULL.
      
      rec_convert_dtuple_to_rec_old(), rec_convert_dtuple_to_rec_comp():
      Replace rec_set_heap_no_old() and rec_set_heap_no_new() with direct
      access that does not involve redo logging.
      
      mtr_t::memcpy(): Do allow non-redo-logged writes to uncompressed pages
      of ROW_FORMAT=COMPRESSED pages.
      
      buf_page_io_complete(): Evict the uncompressed page of
      a ROW_FORMAT=COMPRESSED page after recovery. Because we no longer
      write logical log records for deleting index records, but instead
      write physical records that may refer directly to the compressed
      page frame of a ROW_FORMAT=COMPRESSED page, and because on recovery
      we will only apply the changes to the ROW_FORMAT=COMPRESSED page,
      the uncompressed page frame can be stale until page_zip_decompress()
      is executed.
      
      recv_parse_or_apply_log_rec_body(): After applying MLOG_ZIP_WRITE_STRING,
      ensure that the FIL_PAGE_TYPE of the uncompressed page matches the
      compressed page, because buf_flush_init_for_writing() assumes that
      field to be valid.
      
      mlog_init_t::mark_ibuf_exist(): Invoke page_zip_decompress(), because
      the uncompressed page after buf_page_create() is not necessarily
      up to date.
      
      buf_LRU_block_remove_hashed(): Bypass a page_zip_validate() check
      during redo log apply.
      
      recv_apply_hashed_log_recs(): Invoke mlog_init.mark_ibuf_exist()
      also for the last batch, to ensure that page_zip_decompress() will
      be called for freshly initialized pages.
      2a77b2a5
    • Marko Mäkelä's avatar
      MDEV-12353: Replace MLOG_PAGE_CREATE_RTREE, MLOG_PAGE_COMP_CREATE_RTREE · d00185c4
      Marko Mäkelä authored
      page_create(): Create normal B-tree pages. Callers that create
      R-tree pages will set FIL_PAGE_TYPE and reset the split
      sequence number afterwards.
      
      The creation of ROW_FORMAT=COMPRESSED pages is unaffected;
      they will be logged as compressed page images.
      
      page_create_low(): Take const buf_block_t* as a parameter.
      Let the callers invoke buf_block_modify_clock_inc().
      d00185c4
    • Marko Mäkelä's avatar
      MDEV-12353: Replace DELETE_MARK redo log records with MLOG_WRITE_STRING · b3d02a1f
      Marko Mäkelä authored
      btr_cur_upd_rec_sys(): Replaces row_upd_rec_sys_fields() and
      implements redo logging.
      
      row_upd_rec_sys_fields_in_recovery(): Remove, and merge to the
      only remaining caller btr_cur_parse_update_in_place().
      
      btr_cur_del_mark_set_clust_rec_log(),
      btr_cur_del_mark_set_sec_rec_log(),
      btr_cur_set_deleted_flag_for_ibuf():
      Remove, and replace with btr_rec_set_deleted<bool>().
      
      page_zip_rec_set_deleted(): Add the parameter mtr, and write a
      MLOG_ZIP_WRITE_STRING record to the log.
      b3d02a1f
    • Marko Mäkelä's avatar
      MDEV-12353: Introduce MLOG_ZIP_WRITE_STRING · f3230111
      Marko Mäkelä authored
      Log the low-level operations for ROW_FORMAT=COMPRESSED index pages
      using a new record, MLOG_ZIP_WRITE_STRING. We will still use
      MLOG_1BYTE,..., MLOG_8BYTES or MLOG_WRITE_STRING for operations
      on other than index pages (such as the page allocation bitmap pages).
      
      We will stop writing the record MLOG_ZIP_PAGE_COMPRESS later, after
      replacing all MLOG_REC_ and MLOG_COMP_REC_ that update index pages.
      f3230111
    • Marko Mäkelä's avatar
      MDEV-12353: Replace MLOG_PAGE_REORGANIZE, MLOG_COMP_PAGE_REORGANIZE · db5cdc31
      Marko Mäkelä authored
      Log page reorganize as a series of insert operations.
      This will make the redo log volume proportional to the page payload size.
      
      btr_page_reorganize_low(): Add template <bool recovery=false>
      
      btr_page_reorganize_block(): Remove the parameter 'bool recovery'
      db5cdc31
    • Marko Mäkelä's avatar
      MDEV-12353: Replace MLOG_*_END_COPY_CREATED · 276f996a
      Marko Mäkelä authored
      Instead of writing the high-level redo log records
      MLOG_LIST_END_COPY_CREATED, MLOG_COMP_LIST_END_COPY_CREATED
      write log for each individual insert of a record.
      
      page_copy_rec_list_end_to_created_page(): Remove.
      
      This will improve the fill factor of some pages.
      Adjust some tests accordingly.
      
      PageBulk::init(), PageBulk::finish(): Avoid setting bogus limits
      to PAGE_HEAP_TOP and PAGE_N_DIR_SLOTS. Avoid accessor functions
      that would enforce these limits before the correct ones are set
      at the end of PageBulk::finish().
      276f996a
    • Marko Mäkelä's avatar
      MDEV-12353: Exclusively use page_zip_reorganize() for ROW_FORMAT=COMPRESSED · acd265b6
      Marko Mäkelä authored
      page_zip_reorganize(): Restore the page on failure.
      In callers, omit now-redundant calls to page_zip_decompress().
      
      btr_page_reorganize_low(): Define in static scope only, and
      remove the z_level parameter. Assert that ROW_FORMAT is not COMPRESSED.
      
      btr_page_reorganize_block(), btr_page_reorganize(): Invoke
      page_zip_reorganize() for ROW_FORMAT=COMPRESSED.
      acd265b6
    • Marko Mäkelä's avatar
      MDEV-12353: Replace MLOG_UNDO_INSERT · f802c989
      Marko Mäkelä authored
      trx_undof_page_add_undo_rec_log(): Remove.
      
      trx_undo_page_set_next_prev_and_add(), trx_undo_page_report_modify(),
      trx_undo_page_report_rename(): Write lower-level redo log records.
      f802c989
    • Marko Mäkelä's avatar
      MDEV-12353: Replace MLOG_UNDO_HDR_CREATE · e0bc29df
      Marko Mäkelä authored
      trx_undo_header_create(): Emit lower-level records instead of
      writing MLOG_UNDO_HDR_CREATE records.
      e0bc29df
    • Marko Mäkelä's avatar
      MDEV-12353: Remove trx_undo_erase_page_end() · 737b7017
      Marko Mäkelä authored
      MariaDB stopped writing the record MLOG_UNDO_ERASE_END
      in commit 0fd3def2 (10.3.3).
      Merge trx_undo_erase_page_end() with its callers.
      737b7017
    • Marko Mäkelä's avatar
      MDEV-12353: Replace MLOG_UNDO_INIT · 07d39cde
      Marko Mäkelä authored
      trx_undo_page_init(): Write lower-level redo log records by
      invoking mtr_t::write().
      07d39cde
    • Marko Mäkelä's avatar
      MDEV-12353: Deprecate and ignore innodb_log_compressed_pages · 5bea43f5
      Marko Mäkelä authored
      page_zip_compress_write_log_no_data(): Remove.
      We no longer write the MLOG_ZIP_PAGE_COMPRESS_NO_DATA record.
      Instead, we will write MLOG_ZIP_PAGE_COMPRESS records.
      5bea43f5
    • Marko Mäkelä's avatar
      MDEV-12353: Remove MTR_LOG_SHORT_INSERTS · 600eae91
      Marko Mäkelä authored
      No longer emit the redo log records
      MLOG_LIST_END_COPY_CREATED, MLOG_COMP_LIST_END_COPY_CREATED.
      600eae91
    • Vicențiu Ciorbaru's avatar
    • Vicențiu Ciorbaru's avatar
      Fix compilation error due to type mismatch in tpool_generic.cc · efa9079f
      Vicențiu Ciorbaru authored
      size_t compared to int
      efa9079f
    • Vicențiu Ciorbaru's avatar
      MDEV-18650: Options deprecated in previous versions - multi_range_count · ad17aa11
      Vicențiu Ciorbaru authored
      Remove deprecated system variable multi_range_count. It was ignored from 5.3.
      ad17aa11
    • Vicențiu Ciorbaru's avatar
      MDEV-18650: Options deprecated in previous versions - mroonga_default_parser · 5aebd78e
      Vicențiu Ciorbaru authored
      Variable is marked as deprecated since 10.1.6. Update tests to not make
      use of it.
      5aebd78e
    • Vicențiu Ciorbaru's avatar
      MDEV-18650: Options deprecated in previous versions - skip-bdb · a05b38c1
      Vicențiu Ciorbaru authored
      Remove the option from mysqld --help text.
      a05b38c1
    • Vicențiu Ciorbaru's avatar
      MDEV-18650: Options deprecated in previous versions - thread_concurrency · 8bbcaab1
      Vicențiu Ciorbaru authored
      thread_concurrency was ignored since 5.5. Remove it.
      8bbcaab1
    • Vicențiu Ciorbaru's avatar
      MDEV-18650: Options deprecated in previous versions - old_alter_table · fc5a4cfd
      Vicențiu Ciorbaru authored
      It was deprecated in 5.5 but it never issued a deprecation warning.
      Make it issue a warning in 10.5.1.
      fc5a4cfd
    • Sergei Golubchik's avatar
    • Vicențiu Ciorbaru's avatar
      Policy improvement for removed options and system variables · 1bc9cce7
      Vicențiu Ciorbaru authored
      REMOVED OPTIONS / SYSTEM VARIABLES
        * mysqld removed options will not stop the server from starting. They
          will be silently accepted, as if --loose flag is set. Removed options
          prefixes will not interfere with existing options prefixes. They are
          consumed last.
        * mysqld removed options will not show in --help.
        * mysqld system variables will be removed according to the deprecation
          & removal timeline and not function within the client interface once
          removed.
      
      DEPRECTATED OPTIONS / SYSTEM VARIABLES
        * mysqld deprecated options will issue a warning to the user.
        * mysqld deprecated options will still be visible in --help.
        * mysqld system variables will be removed in the next GA version that is
          past EOL of the version that deprecated the variable.
        * deprecated options / variables will not be used anywhere in the server
          code the moment they are deprecated. At most, they will act as aliases.
      
      The advantage of this policy is that it ensures upgrades will always
      allow the user to start the server, even when upgrading from a very old
      version. It is still possible for user applications to break when
      upgrading, as system variables set via the client interface will return
      errors. However, this will happen after a long time, with lots of
      warnings between versions. The expected timeline is ~ 5 years until a
      deprecated variable dissapears from the server.
      1bc9cce7
    • Vicențiu Ciorbaru's avatar
      MDEV-18650: Options deprecated in previous versions - storage_engine · 45bc7574
      Vicențiu Ciorbaru authored
      Remove usage of deprecated variable storage_engine. It was deprecated in 5.5 but
      it never issued a deprecation warning. Make it issue a warning in 10.5.1.
      
      Replaced with default_storage_engine.
      45bc7574
    • Otto Kekäläinen's avatar
  2. 12 Feb, 2020 6 commits
    • Marko Mäkelä's avatar
      MDEV-15058: Revert the changes to INFORMATION_SCHEMA · 20a7f75f
      Marko Mäkelä authored
      For compatibility with diagnostic software, let us
      return a dummy buffer pool identifier 0 and restore
      the columns that were initially deleted in
      commit 1a6f708e:
      
      	information_schema.innodb_buffer_page.pool_id
      	information_schema.innodb_buffer_page_lru.pool_id
      	information_schema.innodb_buffer_pool_stats.pool_id
      	information_schema.innodb_cmpmem.buffer_pool_instance
      	information_schema.innodb_cmpmem_reset.buffer_pool_instance
      
      Thanks to Vladislav Vaintroub for pointing this out.
      20a7f75f
    • Varun Gupta's avatar
      MDEV-21665: Server crashes in my_qsort2 / Filesort_buffer::sort_buffer · f0606a75
      Varun Gupta authored
      Allocation should use ALIGN_SIZE when we need to make sure that we
      have atleast enough space to store one record in MERGEBUFF2 buffers
      f0606a75
    • Marko Mäkelä's avatar
      MDEV-15058: Deprecate and ignore innodb_buffer_pool_instances · 1a6f708e
      Marko Mäkelä authored
      Our benchmarking efforts indicate that the reasons for splitting the
      buf_pool in commit c18084f7
      have mostly gone away, possibly as a result of
      mysql/mysql-server@ce6109ebfdedfdf185e391a0c97dc6d33867ed78
      or similar work.
      
      Only in one write-heavy benchmark where the working set size is
      ten times the buffer pool size, the buf_pool->mutex would be
      less contended with 4 buffer pool instances than with 1 instance,
      in buf_page_io_complete(). That contention could be alleviated
      further by making more use of std::atomic and by splitting
      buf_pool_t::mutex further (MDEV-15053).
      
      We will deprecate and ignore the following parameters:
      
      	innodb_buffer_pool_instances
      	innodb_page_cleaners
      
      There will be only one buffer pool and one page cleaner task.
      
      In a number of INFORMATION_SCHEMA views, columns that indicated
      the buffer pool instance will be removed:
      
      	information_schema.innodb_buffer_page.pool_id
      	information_schema.innodb_buffer_page_lru.pool_id
      	information_schema.innodb_buffer_pool_stats.pool_id
      	information_schema.innodb_cmpmem.buffer_pool_instance
      	information_schema.innodb_cmpmem_reset.buffer_pool_instance
      1a6f708e
    • Marko Mäkelä's avatar
      0448c614
    • Marko Mäkelä's avatar
      2a6fa1c4
    • Oleksandr Byelkin's avatar
      Merge branch '10.4' into 10.5 · 4b087e17
      Oleksandr Byelkin authored
      4b087e17
  3. 11 Feb, 2020 6 commits
    • Marko Mäkelä's avatar
      MDEV-19747: Deprecate and ignore innodb_log_optimize_ddl · fc2f2fa8
      Marko Mäkelä authored
      During native table rebuild or index creation, InnoDB used to skip
      redo logging and write MLOG_INDEX_LOAD records to inform crash recovery
      and Mariabackup of the gaps in redo log. This is fragile and prohibits
      some optimizations, such as skipping the doublewrite buffer for
      newly (re)initialized pages (MDEV-19738).
      
      row_merge_write_redo(): Remove. We do not write MLOG_INDEX_LOAD
      records any more. Instead, we write full redo log.
      
      FlushObserver: Remove.
      
      fseg_free_page_func(): Remove the parameter log. Redo logging
      cannot be disabled.
      
      fil_space_t::redo_skipped_count: Remove.
      
      We cannot remove buf_block_t::skip_flush_check, because PageBulk
      will temporarily generate invalid B-tree pages in the buffer pool.
      fc2f2fa8
    • Marko Mäkelä's avatar
      MDEV-17491 micro optimize page_id_t further · 8ccb3caa
      Marko Mäkelä authored
      Let us define page_id_t as a thin wrapper of uint64_t so that
      the comparison operators can be simplified. This is a follow-up
      to the original commit 14be8143.
      
      The comparison operator for recv_sys.pages.emplace() turned out to be
      a busy spot in a recovery benchmark. That data structure was introduced
      in MDEV-19586 in commit 177a571e.
      8ccb3caa
    • Marko Mäkelä's avatar
      MDEV-21351: Fix a performance regression · f3dac591
      Marko Mäkelä authored
      The linear scan of recv_sys_t::blocks() in recv_sys_t::free()
      turns out to dominate the execution time in crash recovery.
      Let us scan the much shorter buf_pool->chunks lists instead.
      f3dac591
    • Oleksandr Byelkin's avatar
      Merge branch '10.3' into 10.4 · 646d1ec8
      Oleksandr Byelkin authored
      646d1ec8
    • Jan Lindström's avatar
      MDEV-20051: Add new mode to wsrep_OSU_method in which Galera checks storage... · e6a50e41
      Jan Lindström authored
      MDEV-20051: Add new mode to wsrep_OSU_method in which Galera checks storage engine of the effected table
      
      Introduced a new wsrep_strict_ddl configuration variable in which
      Galera checks storage engine of the effected table. If table is not
      InnoDB (only storage engine currently fully supporting Galera
      replication) DDL-statement will return error code:
      
      ER_GALERA_REPLICATION_NOT_SUPPORTED
             eng "DDL-statement is forbidden as table storage engine does not support Galera replication"
      
      However, when wsrep_replicate_myisam=ON we allow DDL-statements to
      MyISAM tables. If effected table is allowed storage engine Galera
      will run normal TOI.
      
      This new setting should be for now set globally on all
      nodes in a cluster. When this setting is set following DDL-clauses
      accessing tables not supporting Galera replication are refused:
      
      * CREATE TABLE (e.g. CREATE TABLE t1(a int) engine=Aria
      * ALTER TABLE
      * TRUNCATE TABLE
      * CREATE VIEW
      * CREATE TRIGGER
      * CREATE INDEX
      * DROP INDEX
      * RENAME TABLE
      * DROP TABLE
      
      Statements on PROCEDURE, EVENT, FUNCTION are allowed as effected
      tables are known only at execution. Furthermore, USER, ROLE, SERVER,
      DATABASE statements are also allowed as they do not really have
      effected table.
      e6a50e41
    • Igor Babaev's avatar
      MDEV-21683 Server crashes in get_quick_keys with not_null_range_scan · 41541a7c
      Igor Babaev authored
      ANding of the range built from inferred NOT NULL conditions and the range
      built from other conditions used in WHERE/ON clauses may produce an
      IMPOSSIBLE range. The code of MDEV-15777 did not take into account this
      possibility.
      41541a7c