1. 05 Jul, 2017 6 commits
    • Marko Mäkelä's avatar
      MDEV-13143 Server crashes in srv_init_abort_low() when started with inaccessible tmpdir · e417af2c
      Marko Mäkelä authored
      This is a regression caused by
      commit bb60a832
      
      srv_shutdown_all_bg_threads(): If os_thread_count indicates that
      no threads are running, do not bother checking thread status.
      This avoids a crash when InnoDB startup is aborted before
      os_aio_init() has been invoked. (os_aio_all_slots_free() would
      dereference AIO::s_reads even though it is NULL.)
      e417af2c
    • Marko Mäkelä's avatar
      MDEV-12548 Initial implementation of Mariabackup for MariaDB 10.2 · 8c71c6aa
      Marko Mäkelä authored
      InnoDB I/O and buffer pool interfaces and the redo log format
      have been changed between MariaDB 10.1 and 10.2, and the backup
      code has to be adjusted accordingly.
      
      The code has been simplified, and many memory leaks have been fixed.
      Instead of the file name xtrabackup_logfile, the file name ib_logfile0
      is being used for the copy of the redo log. Unnecessary InnoDB startup and
      shutdown and some unnecessary threads have been removed.
      
      Some help was provided by Vladislav Vaintroub.
      
      Parameters have been cleaned up and aligned with those of MariaDB 10.2.
      
      The --dbug option has been added, so that in debug builds,
      --dbug=d,ib_log can be specified to enable diagnostic messages
      for processing redo log entries.
      
      By default, innodb_doublewrite=OFF, so that --prepare works faster.
      If more crash-safety for --prepare is needed, double buffering
      can be enabled.
      
      The parameter innodb_log_checksums=OFF can be used to ignore redo log
      checksums in --backup.
      
      Some messages have been cleaned up.
      Unless --export is specified, Mariabackup will not deal with undo log.
      The InnoDB mini-transaction redo log is not only about user-level
      transactions; it is actually about mini-transactions. To avoid confusion,
      call it the redo log, not transaction log.
      
      We disable any undo log processing in --prepare.
      
      Because MariaDB 10.2 supports indexed virtual columns, the
      undo log processing would need to be able to evaluate virtual column
      expressions. To reduce the amount of code dependencies, we will not
      process any undo log in prepare.
      
      This means that the --export option must be disabled for now.
      
      This also means that the following options are redundant
      and have been removed:
      	xtrabackup --apply-log-only
      	innobackupex --redo-only
      
      In addition to disabling any undo log processing, we will disable any
      further changes to data pages during --prepare, including the change
      buffer merge. This means that restoring incremental backups should
      reliably work even when change buffering is being used on the server.
      Because of this, preparing a backup will not generate any further
      redo log, and the redo log file can be safely deleted. (If the
      --export option is enabled in the future, it must generate redo log
      when processing undo logs and buffered changes.)
      
      In --prepare, we cannot easily know if a partial backup was used,
      especially when restoring a series of incremental backups. So, we
      simply warn about any missing files, and ignore the redo log for them.
      
      FIXME: Enable the --export option.
      
      FIXME: Improve the handling of the MLOG_INDEX_LOAD record, and write
      a test that initiates a backup while an ALGORITHM=INPLACE operation
      is creating indexes or rebuilding a table. An error should be detected
      when preparing the backup.
      
      FIXME: In --incremental --prepare, xtrabackup_apply_delta() should
      ensure that if FSP_SIZE is modified, the file size will be adjusted
      accordingly.
      8c71c6aa
    • Marko Mäkelä's avatar
      Correct a message · dc722559
      Marko Mäkelä authored
      dc722559
    • Marko Mäkelä's avatar
      15c73c8b
    • Marko Mäkelä's avatar
      Adjust a test for tmp_disk_table_size · 6eb1ce04
      Marko Mäkelä authored
      The parameter was recently introduced in
      commit dd8474b1
      but this test was not adjusted accordingly.
      6eb1ce04
    • Marko Mäkelä's avatar
      InnoDB: Use access() instead of open() · 41a6475b
      Marko Mäkelä authored
      41a6475b
  2. 04 Jul, 2017 1 commit
  3. 03 Jul, 2017 5 commits
  4. 01 Jul, 2017 5 commits
    • Monty's avatar
      92f1837a
    • Monty's avatar
      Fixed failing test on 32 bit systems · cc8912f2
      Monty authored
      cc8912f2
    • Monty's avatar
      Clean up BUILD script · 38330974
      Monty authored
      - Removed some not old, not used build scipts
      - Removed tokudb and rocksdb from 32 bit builds
        This enables one now to easily build 32 bit binaries
        on 64 bit systems
      38330974
    • Monty's avatar
      MDEV-13226 Server crashes when tmpdir runs out of space · 2e9b55f7
      Monty authored
      There was a missing test in CTE handling if creating a temporary table
      failed (in this case as a result of out of space). This caused a table
      handler to be used even if it was not allocated.
      2e9b55f7
    • Marko Mäkelä's avatar
      Assert that DB_TRX_ID must be set on delete-marked records · c436338d
      Marko Mäkelä authored
      This is preparation for MDEV-12288, which would set DB_TRX_ID=0
      when purging history. Also with that change in place, delete-marked
      records must always refer to an undo log record via a nonzero
      DB_TRX_ID column. (The DB_TRX_ID is only present in clustered index
      leaf page records.)
      
      btr_cur_parse_del_mark_set_clust_rec(), rec_get_trx_id():
      Statically allocate the offsets
      (should never use the heap). Add some debug assertions.
      
      Replace some use of rec_get_trx_id() with row_get_rec_trx_id().
      
      trx_undo_report_row_operation(): Add some sanity checks that are
      common for all operations that produce undo log.
      c436338d
  5. 30 Jun, 2017 15 commits
  6. 29 Jun, 2017 8 commits
    • Marko Mäkelä's avatar
      Reduce the granularity of innodb_log_file_size · 84e4e450
      Marko Mäkelä authored
      In Mariabackup, we would want the backed-up redo log file size to be
      a multiple of 512 bytes, or OS_FILE_LOG_BLOCK_SIZE. However, at startup,
      InnoDB would be picky, requiring the file size to be a multiple of
      innodb_page_size.
      
      Furthermore, InnoDB would require the parameter to be a multiple of
      one megabyte, while the minimum granularity is 512 bytes. Because
      the data-file-oriented fil_io() API is being used for writing the
      InnoDB redo log, writes will for now require innodb_log_file_size to
      be a multiple of the maximum innodb_page_size (65536 bytes).
      
      To complicate matters, InnoDB startup divided srv_log_file_size by
      UNIV_PAGE_SIZE, so that initially, the unit was bytes, and later it
      was innodb_page_size. We will simplify this and keep srv_log_file_size
      in bytes at all times.
      
      innobase_log_file_size: Remove. Remove some obsolete checks against
      overflow on 32-bit systems. srv_log_file_size is always 64 bits, and
      the maximum size 512GiB in multiples of innodb_page_size always fits
      in ulint (which is 32 or 64 bits). 512GiB would be 8,388,608*64KiB or
      134,217,728*4KiB.
      
      log_init(): Remove the parameter file_size that was always passed as
      srv_log_file_size.
      
      log_set_capacity(): Add a parameter for passing the requested file size.
      
      srv_log_file_size_requested: Declare static in srv0start.cc.
      
      create_log_file(), create_log_files(),
      innobase_start_or_create_for_mysql(): Invoke fil_node_create()
      with srv_log_file_size expressed in multiples of innodb_page_size.
      
      innobase_start_or_create_for_mysql(): Require the redo log file sizes
      to be multiples of 512 bytes.
      84e4e450
    • Marko Mäkelä's avatar
      Clean up InnoDB shutdown · e903d458
      Marko Mäkelä authored
      Tablespace::shutdown(): Clear m_path. This was moved from
      Tablespace::~Tablespace().
      
      LatchDebug::shutdown(): Remove a redundant condition.
      e903d458
    • Marko Mäkelä's avatar
      Simplify access to the binlog offset in InnoDB · 591edccc
      Marko Mäkelä authored
      trx_sys_print_mysql_binlog_offset(): Use 64-bit arithmetics and ib::info().
      
      TRX_SYS_MYSQL_LOG_OFFSET: Replaces TRX_SYS_MYSQL_LOG_OFFSET_HIGH,
      TRX_SYS_MYSQL_LOG_OFFSET_LOW.
      
      trx_sys_update_mysql_binlog_offset(): Remove the constant parameter
      field=TRX_SYS_MYSQL_LOG_INFO. Use 64-bit arithmetics.
      591edccc
    • Marko Mäkelä's avatar
    • Marko Mäkelä's avatar
      buf_read_ibuf_merge_pages(): Discard all entries for a missing tablespace · cd623508
      Marko Mäkelä authored
      A similar change was contributed to Percona XtraBackup, but for some
      reason, it is not present in Percona XtraDB. Since MDEV-9566
      (MariaDB 10.1.23), that change is present in the MariaDB XtraDB.
      cd623508
    • Marko Mäkelä's avatar
      Minor cleanup of InnoDB I/O routines · 68b5aeae
      Marko Mäkelä authored
      Change many function parameters from IORequest& to const IORequest&.
      
      Remove an unused definition of ECANCELED.
      68b5aeae
    • Marko Mäkelä's avatar
      Simplify up InnoDB redo log system startup and shutdown · 859714e7
      Marko Mäkelä authored
      recv_sys_init(): Remove the parameter.
      
      recv_sys_create(): Merge to recv_sys_init().
      
      recv_sys_mem_free(): Merge to recv_sys_close().
      
      log_mem_free(): Merge to log_shutdown().
      859714e7
    • Marko Mäkelä's avatar
      8143ef1b