1. 18 Mar, 2021 3 commits
    • Vladislav Vaintroub's avatar
      MDEV-9077 - sys schema preparation · 9186ff88
      Vladislav Vaintroub authored
      - increase MAX_BOOTSTRAP_QUERY_SIZE (sys.schema has SP over 50K large)
        don't allocate bootstrap query on heap anymore.
      - support DELIMITER in bootstrap
      9186ff88
    • Vladislav Vaintroub's avatar
      MDEV-9077 - port sys schema to MariaDB · 601c5771
      Vladislav Vaintroub authored
      - Innodb is not always available, which means t is not always
      possible to use innodb system variables, or innodb information schema
      tables.
      
      Thus creation of objects that use Innodb information_schema is enclosed
      into BEGIN NOT ATOMIC blocks with dummy SQLEXCEPTION handler.
      
      - sys_config table uses Aria, just like other system tables.
      
      - several tables that exist in MySQL, do not exist in MariaDB
         performance_schema.replication_applier_status, mysql.slave_master_info,
         mysql.slave_relay_log_info
      601c5771
    • Vladislav Vaintroub's avatar
  2. 17 Mar, 2021 9 commits
  3. 16 Mar, 2021 4 commits
    • Anel Husakovic's avatar
      MDEV-24601: INFORMATION_SCHEMA doesn't differentiate between column and... · 825c0e2a
      Anel Husakovic authored
      MDEV-24601: INFORMATION_SCHEMA doesn't differentiate between column and table-level CHECK constraints
      
      - Reviewed by: wlad@mariadb.com
      825c0e2a
    • Marko Mäkelä's avatar
      Merge 10.5 into 10.6 · f87a944c
      Marko Mäkelä authored
      f87a944c
    • Marko Mäkelä's avatar
      MDEV-24818: Optimize multi-statement INSERT into an empty table · 8ea923f5
      Marko Mäkelä authored
      If the user "opts in" (as in the parent
      commit 92b2a911),
      we can optimize multiple INSERT statements to use table-level locking
      and undo logging.
      
      There will be a change of behavior:
      
          CREATE TABLE t(a PRIMARY KEY) ENGINE=InnoDB;
          SET foreign_key_checks=0, unique_checks=0;
          BEGIN; INSERT INTO t SET a=1; INSERT INTO t SET a=1; COMMIT;
      
      will end up with an empty table, because in case of an error,
      the entire transaction will be rolled back, instead of rolling
      back the failing statement. Previously, the second INSERT statement
      would have been logged row by row, and only that second statement
      would have been rolled back, leaving the first INSERT intact.
      
      lock_table_x_unlock(), trx_mod_table_time_t::WAS_BULK: Remove.
      Because we cannot really support statement rollback in this
      optimized mode, we will not optimize the locking. The exclusive
      table lock will be held until the end of the transaction.
      8ea923f5
    • Marko Mäkelä's avatar
      MDEV-24818 Concurrent use of InnoDB table is impossible until the first transaction is finished · 92b2a911
      Marko Mäkelä authored
      In MDEV-515, we enabled an optimization where an insert into an
      empty table will use table-level locking and undo logging.
      This may break applications that expect row-level locking.
      
      The SQL statements created by the mysqldump utility will include the
      following:
      
          SET unique_checks=0, foreign_key_checks=0;
      
      We will use these flags to enable the table-level locked and logged
      insert. Unless the parameters are set, INSERT will be executed in
      the old way, with row-level undo logging and implicit record locks.
      92b2a911
  4. 15 Mar, 2021 6 commits
    • Vladislav Vaintroub's avatar
      update libmariadb · 8dd35a25
      Vladislav Vaintroub authored
      CONC-534 memory leak in ps_bugs.c
      8dd35a25
    • Vlad Lesin's avatar
      MDEV-24184 InnoDB RENAME TABLE recovery failure if names are reused · 8cbada87
      Vlad Lesin authored
      fil_op_replay_rename(): Remove.
      
      fil_rename_tablespace_check(): Remove a parameter is_discarded=false.
      
      recv_sys_t::parse(): Instead of applying FILE_RENAME operations,
      buffer the operations in renamed_spaces.
      
      recv_sys_t::apply(): In the last_batch, apply renamed_spaces.
      8cbada87
    • Vladislav Vaintroub's avatar
      update libmariadb · 2f53ad4b
      Vladislav Vaintroub authored
      2f53ad4b
    • Marko Mäkelä's avatar
      MDEV-24883 fixup: Add a dependency · a0558b8c
      Marko Mäkelä authored
      In commit 783625d7 we forgot to
      declare a dependency on the generated file mysqld_error.h.
      a0558b8c
    • Otto Kekäläinen's avatar
      MDEV-24927: Deb: Use liburing-dev instead of libaio-dev · 211e9b3e
      Otto Kekäläinen authored
      Updating the debian/control file will automatically update the dependencies
      in all CI environments that directly read the debian/control file, such
      as Salsa-CI and buildbot.mariadb.org to some degree.
      (https://github.com/MariaDB/mariadb.org-tools/issues/43)
      
      On Debian/Ubuntu releases that don't have liburing-dev available,
      automatically downgrade to libaio-dev (just like libcurl4->3 is done).
      This ensures the debian/control file is always up-to-date and works for
      latest Debian and Ubuntu releases, while the backwards compatibility mods
      are maintained in autobake-deb.sh separately, and can be dropped from there
      once support for certain platforms end.
      
      Debian/Ubuntu availability visible at:
      - https://packages.debian.org/search?searchon=names&keywords=liburing-dev
      - https://packages.ubuntu.com/search?searchon=names&keywords=liburing-dev
      
      Also modify debian/rules to force a build without libaio. Use YES instead
      of ON to make the flag more logical (=turning libaio check "off").
      
      Stop running Salsa-CI for Debian Stretch-backports, as it does not have
      liburing-dev available nor is the old-old Debian stable a relevant platform
      for MariaDB 10.6 to test against anymore. Since the Stretch-backports build
      can no longer be made, neither can the MySQL 5.7 on Bionic upgrade test be
      run, as it depended on the Stretch binary.
      
      This commit does not modify the .travis.yml file, as Travis-CI does not
      have new enough Ubuntu releases available yet. Also Travis-CI.org is
      practically dead now as build times have been shrunk to near zero.
      
      The scope of this change is also Debian/Ubuntu only. No RPM or Windows or
      Mac changes are included in this commit.
      
      This commit does not update the external libmariadb or ColumnStore
      CI pipelines, as those are maintained in different repositories.
      211e9b3e
    • Marko Mäkelä's avatar
      MDEV-24883 add io_uring support for tpool · 783625d7
      Marko Mäkelä authored
      liburing is a new optional dependency (WITH_URING=auto|yes|no)
      that replaces libaio when it is available.
      
      aio_uring: class which wraps io_uring stuff
      
      aio_uring::bind()/unbind(): optional optimization
      
      aio_uring::submit_io(): mutex prevents data race. liburing calls are
      thread-unsafe. But if you look into it's implementation you'll see
      atomic operations. They're used for synchronization between kernel and
      user-space only. That's why our own synchronization is still needed.
      
      For systemd, we add LimitMEMLOCK=524288 (ulimit -l 524288)
      because the io_uring_setup system call that is invoked
      by io_uring_queue_init() requests locked memory. The value
      was found empirically; with 262144, we would occasionally
      fail to enable io_uring when using the maximum values of
      innodb_read_io_threads=64 and innodb_write_io_threads=64.
      
      aio_uring::thread_routine(): Tolerate -EINTR return from
      io_uring_wait_cqe(), because it may occur on shutdown
      on Ubuntu 20.10 (Groovy Gorilla).
      
      This was mostly implemented by Eugene Kosov. Systemd integration
      and improved startup/shutdown error handling by Marko Mäkelä.
      783625d7
  5. 12 Mar, 2021 4 commits
  6. 11 Mar, 2021 14 commits
    • Marko Mäkelä's avatar
      4c2b6be3
    • Marko Mäkelä's avatar
      Merge 10.5 into 10.6 · 64f89b1f
      Marko Mäkelä authored
      64f89b1f
    • Marko Mäkelä's avatar
      abe25c31
    • Marko Mäkelä's avatar
      Merge 10.5 into 10.6 · a43ff483
      Marko Mäkelä authored
      a43ff483
    • Marko Mäkelä's avatar
      Merge 10.4 into 10.5 · a4b7232b
      Marko Mäkelä authored
      a4b7232b
    • Marko Mäkelä's avatar
      Merge 10.3 into 10.4 · 1ea6ac3c
      Marko Mäkelä authored
      1ea6ac3c
    • Marko Mäkelä's avatar
    • Thirunarayanan Balathandayuthapani's avatar
      MDEV-25107 Check TABLE miscalutates the length of column · 06df0b0d
      Thirunarayanan Balathandayuthapani authored
      - This is caused by merge commit a26e7a37.
      InnoDB fails to fetch the next index field when there is a externally
      stored column length check involved.
      06df0b0d
    • Marko Mäkelä's avatar
      MDEV-25110 [FATAL] InnoDB: Trying to write ... outside the bounds · a8650b64
      Marko Mäkelä authored
      In commit 118e258a (part of MDEV-23855)
      we inadvertently broke crash recovery, reintroducing MDEV-11556.
      
      fil_system_t::extend_to_recv_size(): Extend all open tablespace files
      to the recovered size.
      
      recv_sys_t::apply(): Invoke fil_system.extend_to_recv_size() at the
      start of each batch. In this way, any fil_space_t::recv_size
      changes that were parsed after the file was opened will be applied.
      a8650b64
    • Marko Mäkelä's avatar
      MDEV-25031 Not applying INSERT_*_REDUNDANT due to corruption on page · 549a70d7
      Marko Mäkelä authored
      page_apply_insert_redundant(): Replace a too strict condition
      hdr_c > pextra_size. It turns out that page_cur_insert_rec_low()
      is not even computing the extra_size of cur->rec when it is trying
      to reuse header bytes of the preceding record.
      549a70d7
    • Marko Mäkelä's avatar
      MDEV-25106 Deprecation warning for innodb_checksum_algorithm=none,innodb,... · 08e8ad7c
      Marko Mäkelä authored
      MDEV-25105 (commit 7a4fbb55)
      in MariaDB 10.6 will refuse the innodb_checksum_algorithm
      values none, innodb, strict_none, strict_innodb.
      
      We will issue a deprecation warning if innodb_checksum_algorithm
      is set to any of these non-default unsafe values.
      
      innodb_checksum_algorithm=crc32 was made the default in
      MySQL 5.7 and MariaDB Server 10.2, and given that older versions
      of the server have reached their end of life, there is no valid
      reason to use anything else than innodb_checksum_algorithm=crc32
      or innodb_checksum_algorithm=strict_crc32 in MariaDB 10.3.
      
      Reviewed by: Sergei Golubchik
      08e8ad7c
    • Marko Mäkelä's avatar
      MDEV-25105 Remove innodb_checksum_algorithm values none,innodb,... · 7a4fbb55
      Marko Mäkelä authored
      Historically, InnoDB supported a buggy page checksum algorithm that did not
      compute a checksum over the full page. Later, well before MySQL 4.1
      introduced .ibd files and the innodb_file_per_table option, the algorithm
      was corrected and the first 4 bytes of each page were redefined to be
      a checksum.
      
      The original checksum was so slow that an option to disable page checksum
      was introduced for benchmarketing purposes.
      
      The Intel Nehalem microarchitecture introduced the SSE4.2 instruction set
      extension, which includes instructions for faster computation of CRC-32C.
      In MySQL 5.6 (and MariaDB 10.0), innodb_checksum_algorithm=crc32 was
      implemented to make of that. As that option was changed to be the default
      in MySQL 5.7, a bug was found on big-endian platforms and some work-around
      code was added to weaken that checksum further. MariaDB disables that
      work-around by default since MDEV-17958.
      
      Later, SIMD-accelerated CRC-32C has been implemented in MariaDB for POWER
      and ARM and also for IA-32/AMD64, making use of carry-less multiplication
      where available.
      
      Long story short, innodb_checksum_algorithm=crc32 is faster and more secure
      than the pre-MySQL 5.6 checksum, called innodb_checksum_algorithm=innodb.
      It should have removed any need to use innodb_checksum_algorithm=none.
      
      The setting innodb_checksum_algorithm=crc32 is the default in
      MySQL 5.7 and MariaDB Server 10.2, 10.3, 10.4. In MariaDB 10.5,
      MDEV-19534 made innodb_checksum_algorithm=full_crc32 the default.
      It is even faster and more secure.
      
      The default settings in MariaDB do allow old data files to be read,
      no matter if a worse checksum algorithm had been used.
      (Unfortunately, before innodb_checksum_algorithm=full_crc32,
      the data files did not identify which checksum algorithm is being used.)
      
      The non-default settings innodb_checksum_algorithm=strict_crc32 or
      innodb_checksum_algorithm=strict_full_crc32 would only allow CRC-32C
      checksums. The incompatibility with old data files is why they are
      not the default.
      
      The newest server not to support innodb_checksum_algorithm=crc32
      were MySQL 5.5 and MariaDB 5.5. Both have reached their end of life.
      A valid reason for using innodb_checksum_algorithm=innodb could have
      been the ability to downgrade. If it is really needed, data files
      can be converted with an older version of the innochecksum utility.
      
      Because there is no good reason to allow data files to be written
      with insecure checksums, we will reject those option values:
      
          innodb_checksum_algorithm=none
          innodb_checksum_algorithm=innodb
          innodb_checksum_algorithm=strict_none
          innodb_checksum_algorithm=strict_innodb
      
      Furthermore, the following innochecksum options will be removed,
      because only strict crc32 will be supported:
      
          innochecksum --strict-check=crc32
          innochecksum -C crc32
          innochecksum --write=crc32
          innochecksum -w crc32
      
      If a user wishes to convert a data file to use a different checksum
      (so that it might be used with the no-longer-supported
      MySQL 5.5 or MariaDB 5.5, which do not support IMPORT TABLESPACE
      nor system tablespace format changes that were made in MariaDB 10.3),
      then the innochecksum tool from MariaDB 10.2, 10.3, 10.4, 10.5 or
      MySQL 5.7 can be used.
      
      Reviewed by: Thirunarayanan Balathandayuthapani
      7a4fbb55
    • Marko Mäkelä's avatar
      MDEV-25070 fixup: Correct the result · 6e7ac406
      Marko Mäkelä authored
      6e7ac406
    • David CARLIER's avatar
      a6cd4612