1. 24 Feb, 2020 2 commits
  2. 22 Feb, 2020 1 commit
    • Marko Mäkelä's avatar
      MDEV-12353: Reduce log volume of page_cur_delete_rec() · 572d2075
      Marko Mäkelä authored
      mrec_ext_t: Introduce DELETE_ROW_FORMAT_REDUNDANT,
      DELETE_ROW_FORMAT_DYNAMIC.
      
      mtr_t::page_delete(): Write DELETE_ROW_FORMAT_REDUNDANT or
      DELETE_ROW_FORMAT_DYNAMIC log records. We log the byte offset
      of the preceding record, so that on recovery we can easily
      find everything to update. For DELETE_ROW_FORMAT_DYNAMIC,
      we must also write the header and data size of the record.
      
      We will retain the physical logging for ROW_FORMAT=COMPRESSED pages.
      
      page_zip_dir_balance_slot(): Renamed from page_dir_balance_slot(),
      and specialized for ROW_FORMAT=COMPRESSED only.
      
      page_rec_set_n_owned(), page_dir_slot_set_n_owned(),
      page_dir_balance_slot(): New variants that do not write any log.
      
      page_mem_free(): Take data_size, extra_size as parameters.
      Always zerofill the record payload.
      
      page_cur_delete_rec(): For other than ROW_FORMAT=COMPRESSED,
      only write log by mtr_t::page_delete().
      572d2075
  3. 21 Feb, 2020 10 commits
  4. 20 Feb, 2020 2 commits
    • Marko Mäkelä's avatar
      Cleanup: Remove dict_ind_redundant · 96901d95
      Marko Mäkelä authored
      There is no reason for the dummy index object dict_ind_redundant
      to exist any more. It was only being passed to btr_create().
      
      btr_create(): If !index, assume that a ROW_FORMAT=REDUNDANT
      table is being created.
      
      We could pass ibuf.index, dict_sys.sys_tables->indexes.start
      and so on, if those objects had been initialized before the
      function btr_create() is called.
      96901d95
    • Eugene Kosov's avatar
      MDEV-21774 Innodb, Windows : restore file sharing logic in Innodb · 6618fc29
      Eugene Kosov authored
      recv_sys_t opened redo log files along with log_sys_t. That's why I
      removed file sharing logic from InnoDB
      in 9ef2d29f
      But it was actually used to ensure that only one MariaDB instance
      will touch the same InnoDB files.
      
      os0file.cc: revert some changes done previously
      
      mapped_file_t::map(): now has arguments read_only, nvme
      
      file_io::open(): now has argument read_only
      
      class file_os_io: make final
      
      log_file_t::open(): now has argument read_only
      6618fc29
  5. 19 Feb, 2020 7 commits
    • Marko Mäkelä's avatar
      MDEV-12353: Reduce log volume by an UNDO_APPEND record · 84e3f9ce
      Marko Mäkelä authored
      We introduce an EXTENDED log record for appending an undo log record
      to an undo log page. This is equivalent to the MLOG_UNDO_INSERT record
      that was removed in commit f802c989,
      only using more compact encoding.
      
      mtr_t::log_write(): Fix a bug that affects longer log
      record writes in the !same_page && !have_offset case.
      Similar code is already implemented for the have_offset code path.
      The bug was unobservable before we started to write longer
      EXTENDED records. All !have_offset records (FREE_PAGE, INIT_PAGE,
      EXTENDED) that were written so far are short, and we never write
      RESERVED or OPTION records.
      
      mtr_t::undo_append(): Write an UNDO_APPEND record.
      
      log_phys_t::undo_append(): Apply an UNDO_APPEND record.
      
      trx_undo_page_set_next_prev_and_add(),
      trx_undo_page_report_modify(),
      trx_undo_page_report_rename():
      Invoke mtr_t::undo_append() instead of emitting WRITE records.
      84e3f9ce
    • Marko Mäkelä's avatar
      MDEV-12353: Reduce log volume by an UNDO_INIT record · 86f262f1
      Marko Mäkelä authored
      We introduce an EXTENDED log record for initializing an undo log page.
      The size of the record will be 2 bytes plus the optional page identifier.
      The entire undo page will be initialized, except the space that is
      already reserved for TRX_UNDO_SEG_HDR in trx_undo_seg_create().
      
      mtr_t::undo_create(): Write the UNDO_INIT record.
      
      trx_undo_page_init(): Initialize the undo page corresponding to the
      UNDO_INIT record. Unlike the former MLOG_UNDO_INIT record, we will
      initialize almost the entire page, including initializing the
      TRX_UNDO_PAGE_NODE to an empty list node, so that the subsequent call
      to flst_init() will avoid writing log for the undo page.
      86f262f1
    • Eugene Kosov's avatar
      revert accidental libmariadb change · 3ee100b0
      Eugene Kosov authored
      3ee100b0
    • Eugene Kosov's avatar
      fix libpmem InnoDB linking · 29bb3744
      Eugene Kosov authored
      29bb3744
    • Eugene Kosov's avatar
      remove unused function · e62e285f
      Eugene Kosov authored
      e62e285f
    • Eugene Kosov's avatar
      MDEV-14425 deprecate and ignore innodb_log_files_in_group · 9ef2d29f
      Eugene Kosov authored
      Now there can be only one log file instead of several which
      logically work as a single file.
      
      Possible names of redo log files: ib_logfile0,
      ib_logfile101 (for just created one)
      
      innodb_log_fiels_in_group: value of this variable is not used
      by InnoDB. Possible values are still 1..100, to not break upgrade
      
      LOG_FILE_NAME: add constant of value "ib_logfile0"
      LOG_FILE_NAME_PREFIX: add constant of value "ib_logfile"
      
      get_log_file_path(): convenience function that returns full
      path of a redo log file
      
      SRV_N_LOG_FILES_MAX: removed
      
      srv_n_log_files: we can't remove this for compatibility reasons,
      but now server doesn't use this variable
      
      log_sys_t::file::fd: now just one, not std::vector
      
      log_sys_t::log_capacity: removed word 'group'
      
      find_and_check_log_file(): part of logic from huge srv_start()
      moved here
      
      recv_sys_t::files: file descriptors of redo log files.
      There can be several of those in case we're upgrading
      from older MariaDB version.
      
      recv_sys_t::remove_extra_log_files: whether to remove
      ib_logfile{1,2,3...} after successfull upgrade.
      
      recv_sys_t::read(): open if needed and read from one
      of several log files
      
      recv_sys_t::files_size(): open if needed and return files count
      
      redo_file_sizes_are_correct(): check that redo log files
      sizes are equal. Just to log an error for a user.
      Corresponding check was moved from srv0start.cc
      
      namespace deprecated: put all deprecated variables here to
      prevent usage of it by us, developers
      9ef2d29f
    • Jan Lindström's avatar
      Update wsrep-lib submodule. · 8d7a8e45
      Jan Lindström authored
      8d7a8e45
  6. 18 Feb, 2020 3 commits
  7. 17 Feb, 2020 6 commits
    • Marko Mäkelä's avatar
      MDEV-21744 Assertion `!rec_offs_nth_sql_null(offsets, n)' failed · 41fe972d
      Marko Mäkelä authored
      commit 08ba3887 of MDEV-12353
      introduced an incorrect assumption, which was documented by
      the failing assertion.
      
      After instant ADD COLUMN, we can have a null (and in-place) UPDATE
      of NULL to NULL. No data needs to be written for such updates.
      
      For ROW_FORMAT=REDUNDANT, we reserve space for the NULL values,
      and to be compatible with existing behaviour, we will zerofill
      the unused data bytes when updating to NULL value.
      41fe972d
    • Marko Mäkelä's avatar
      MDEV-21174: Correct a debug assertion failure · 055ce75d
      Marko Mäkelä authored
      trx_purge_free_segment(): In some cases (observed when running
      the test innodb_zip.wl5522_debug_zip), there is no change to
      the TRX_UNDO_NEEDS_PURGE field. Add mtr_t::OPT to disable a debug check.
      
      The bogus debug check was introduced in
      commit 56f6dab1.
      055ce75d
    • Marko Mäkelä's avatar
      MDEV-12353: Reformat page_delete_rec_list_end() · 22f649a6
      Marko Mäkelä authored
      We add FIXME comments and some sketch code for the following cases:
      
      It is possible to write considerably less log for ROW_FORMAT=COMPRESSED
      pages. For now, we will delete the records one by one.
      
      It is also possible to treat 'deleting the last records' as a special
      case that would involve shrinking PAGE_HEAP_TOP. That should reduce
      the need of reorganizing pages.
      22f649a6
    • Marko Mäkelä's avatar
      MDEV-12353: Optimize page_cur_delete_rec() logging further · 09feb176
      Marko Mäkelä authored
      page_mem_free(): When deleting the very last record of the page,
      even if the record did not fully utilize all bytes in a
      former PAGE_FREE record, truncate the PAGE_HEAP_TOP and reduce
      PAGE_GARBAGE by the saved amount.
      09feb176
    • Ian Gilfillan's avatar
    • Marko Mäkelä's avatar
      MDEV-12353: Write less log for BLOB pages · fc876980
      Marko Mäkelä authored
      fsp_page_create(): Always initialize the page. The logic to
      avoid initialization was made redundant and should have been removed
      in mysql/mysql-server@ce0a1e85e24e48b8171f767b44330da635a6ea0a
      (MySQL 5.7.5).
      
      btr_store_big_rec_extern_fields(): Remove the redundant initialization
      of FIL_PAGE_PREV and FIL_PAGE_NEXT. An INIT_PAGE record will have
      been written already. Only write the ROW_FORMAT=COMPRESSED page payload
      from FIL_PAGE_DATA onwards. We were unnecessarily writing from
      FIL_PAGE_TYPE onwards, which caused an assertion failure on recovery:
      
      	recv_sys_t::alloc(size_t): Assertion 'len <= srv_page_size' failed
      
      when running the following tests:
      
      	./mtr --no-reorder innodb_zip.blob,4k innodb_zip.bug56680,4k
      fc876980
  8. 16 Feb, 2020 4 commits
    • Marko Mäkelä's avatar
      MDEV-12353: Fix a Galera assertion failure · 5874aac7
      Marko Mäkelä authored
      trx_rseg_write_wsrep_checkpoint(): Add missing mtr_t::OPT,
      and avoid an unnecessary call to mtr_t::memset().
      
      This addresses a debug assertion failure in wsrep_info.plugin.
      5874aac7
    • Marko Mäkelä's avatar
      d657cd74
    • Marko Mäkelä's avatar
      MDEV-12353: Remove bogus conditions · 5876de19
      Marko Mäkelä authored
      page_update_max_trx_id(), page_delete_rec_list_end(): Remove conditions
      on recv_recovery_is_on(). These conditions should have been removed in
      or before commit f8a9f906
      (removing the support for crash-upgrade).
      
      The physical redo log based recovery will not call such high-level code.
      5876de19
    • Marko Mäkelä's avatar
      MDEV-12353: Optimize page_cur_delete_rec() logging · 3887daf8
      Marko Mäkelä authored
      page_mem_free(): When deleting the last record of a page,
      do not add it to the PAGE_FREE list, but instead truncate the
      PAGE_HEAP_TOP. Modify the page header fields by writing fewer
      records.
      
      page_cur_delete_rec(): Let page_mem_free() reset the PAGE_LAST_INSERT.
      
      page_header_reset_last_insert(): Issue memset(), not memcpy(), for
      the ROW_FORMAT=COMPRESSED page.
      3887daf8
  9. 14 Feb, 2020 5 commits
    • Daniel Bartholomew's avatar
      bump the VERSION · 2c34315d
      Daniel Bartholomew authored
      2c34315d
    • Marko Mäkelä's avatar
    • Eugene Kosov's avatar
      fix Win build · 735c6ea3
      Eugene Kosov authored
      735c6ea3
    • Eugene Kosov's avatar
      MDEV-17084 Optimize append only files for NVDIMM · 3daef523
      Eugene Kosov authored
      Optionally use libpmem for InnoDB redo log writing.
      
      When server is built -DWITH_PMEM=ON InnoDB tries to detect
      that redo log is located on persistent memory storage and
      uses faster file access method.
      
      When server is built with -DWITH_PMEM=OFF preprocessor is
      used to ensure that no slowdown will present due to allocations
      and virtual function calls. So, we don't slow down server
      in a common case.
      
      mapped_file_t: an map file, unmap file and returns mapped memory buffer
      
      file_io: abstraction around memory mapped files and file descriptors.
      Allows writing, reading and flushing to files.
      
      file_io::writes_are_durable(): notable method of a class.
      When it returns true writes are flushed immediately.
      
      file_os_io: file descriptor based file access. Depends on a global state
      like srv_read_only_mode
      
      file_pmem_io: file access via libpmem
      
      This is a collaboration work with Sergey Vojtovich
      3daef523
    • Marko Mäkelä's avatar
      MDEV-19747: Fix a warning · d901919d
      Marko Mäkelä authored
      In commit fc2f2fa8
      we replaced FlushObserver* with bool, but forgot to
      replace one NULL with false.
      d901919d