1. 06 Apr, 2022 1 commit
  2. 05 Apr, 2022 2 commits
    • Marko Mäkelä's avatar
      Merge 10.5 into 10.6 · 8d9c2561
      Marko Mäkelä authored
      8d9c2561
    • Otto Kekäläinen's avatar
      Deb: Sync Salsa-CI from Debian MariaDB 10.6 · 8c919330
      Otto Kekäläinen authored
      As MariaDB 10.5 has been removed from Debian Sid and MariaDB 10.6 has
      entered it, the Salsa-CI testing needs to adapt.
      
      To achieve this, essentially sync most the the salsa-ci.yml contents from
      https://salsa.debian.org/mariadb-team/mariadb-server/-/tree/debian/latest
      
      This includes removing Stretch builds, as Stretch does not support uring
      nor pmem libraries, which MariaDB 10.6 depends on.
      
      Also add a couple Lintian overrides to make Salsa-CI pass.
      
      NOTE TO MERGERS: This commit is made on 10.6 branch and can be merged to
      all later branches (10.7, 10.8, 10.9..) for now, but later somebody needs
      to go in and update all the testing stages to do the upgrade testing
      correctly for 10.6->10.7->10.8->10.9 etc.
      8c919330
  3. 04 Apr, 2022 2 commits
    • Vlad Lesin's avatar
      MDEV-26322 Last binlog file and position are "empty" in mariabackup --prepare output · 6a3545dd
      Vlad Lesin authored
      The issue is caused by 59a0236d commit.
      The initial intention of the commit was to speed up
      "mariabackup --prepare".
      
      The call stack of binlog position reading is the following:
      ▾ trx_rseg_mem_restore
        ▾ trx_rseg_array_init
            ▾ trx_lists_init_at_db_start
                  ▸ srv_start
      Both trx_lists_init_at_db_start() and trx_rseg_mem_restore() contain
      special cases for srv_operation == SRV_OPERATION_RESTORE condition, and
      on this condition only rseg headers are read to parse binlog position.
      Performance impact is not so big.
      
      The solution is to revert 59a0236d.
      6a3545dd
    • Daniel Black's avatar
      MDEV-28231: innodb: format string warning on aix UINT64PFx (ib_id_t) · 7f5a3cd2
      Daniel Black authored
      ib_id_t is a uint64. On AIX this isn't a long long unsigned and to
      prevent the compile warnings and potential wrong type, the UINT64PFx
      defination is corrected.
      
      As INT64PF is unused (last use, xtradb in 10.2), it is removed to
      remove the confusion that INT64PF and UINT64PFx would be different
      types otherwise.
      7f5a3cd2
  4. 01 Apr, 2022 1 commit
    • Daniel Black's avatar
      MDBF-348: innodb aix htm support · 28116ca3
      Daniel Black authored
      CMAKE_SYSTEM_PROCESSOR on AIX is "powerpc". To
      deconflict with the Linux 32bit arch of the same
      name, CMAKE_SYSTEM_NAME was used in the CMakeLists.txt
      test to enable -mhtm in the same way that was required
      for Linux ppc64{,le} compilers in MDEV-27936
      28116ca3
  5. 29 Mar, 2022 12 commits
  6. 28 Mar, 2022 9 commits
    • mkaruza's avatar
      MDEV-25912 wsrep does not identify checksummed events correctly · 97f237e6
      mkaruza authored
      For GTID consistenty, GTID events was artificialy added before
      replication happned. This event should not contain CHECKSUM calculated.
      Reviewed-by: default avatarJan Lindström <jan.lindstrom@mariadb.com>
      97f237e6
    • Vladislav Vaintroub's avatar
      MDEV-28178 Windows : sporadic ER_ERROR_ON_RENAME .. (errno: 13 "Permission denied") · 739002ee
      Vladislav Vaintroub authored
      On affected machine, the error happens sporadically in
      innodb.instant_alter_limit.
      
      Procmon shows SetRenameInformationFile failing with ERROR_ACCESS_DENIED.
      In this case, the destination file was previously opened rsp oplocked by
      Windows defender antivirus.
      
      The fix is to retry MoveFileEx on ERROR_ACCESS_DENIED.
      739002ee
    • Marko Mäkelä's avatar
      MDEV-27931: buf_page_is_corrupted() wrongly claims corruption · 303448bc
      Marko Mäkelä authored
      In commit 437da7bc (MDEV-19534),
      the default value of the global variable srv_checksum_algorithm
      in innochecksum was changed from SRV_CHECKSUM_ALGORITHM_INNODB
      to implied 0 (innodb_checksum_algorithm=crc32). As a result,
      the function buf_page_is_corrupted() would by default invoke
      buf_calc_page_crc32() in innochecksum, and crc32_inited would hold.
      
      This would cause "innochecksum" to fail on a particular page.
      
      The actual problem is older, introduced in 2011 in
      mysql/mysql-server@17e497bdb793bc6b8360aa1c626dcd8bb5cfad1b
      (MySQL 5.6.3). It should affect the validation of pages of old
      data files that were written with innodb_checksum_algorithm=innodb.
      When using innodb_checksum_algorithm=crc32 (the default setting
      since MariaDB Server 10.2), some valid pages would be rejected
      only because exactly one of the two checksum fields accidentally
      matches the innodb_checksum_algorithm=crc32 value.
      
      buf_page_is_corrupted(): Simplify the logic of non-strict
      checksum validation, by always invoking buf_calc_page_crc32().
      Remove a bogus condition that if only one of the checksum fields
      contains the value returned by buf_calc_page_crc32(), the page
      is corrupted.
      303448bc
    • Marko Mäkelä's avatar
      MDEV-28181 The innochecksum -w option was inadvertently removed · b2fa874e
      Marko Mäkelä authored
      In commit 7a4fbb55 (MDEV-25105)
      the innochecksum option --write (-w) was removed altogether.
      It should have been made a Boolean option, so that old data files
      may be converted to a format that is compatible with
      innodb_checksum_algorithm=strict_crc32 by executing the following:
      
      innochecksum -n -w ibdata* */*.ibd
      
      It would be better to use an older-version innochecksum
      for such a conversion, so that page checksums will be validated
      before updating the checksum.
      
      It never was possible for innochecksum to convert files to the
      innodb_checksum_algorithm=full_crc32 format that is the default
      for new InnoDB data files.
      b2fa874e
    • Marko Mäkelä's avatar
      MDEV-27234: Data dictionary recovery was not READ COMMITTED · 8f8ba758
      Marko Mäkelä authored
      This also fixes MDEV-20198: Instant ALTER TABLE is not crash safe
      
      InnoDB dictionary recovery wrongly used the READ UNCOMMITTED isolation
      level, causing some mismatch. For example, if a table was renamed or
      replaced in a transaction, according to READ UNCOMMITTED the table might
      not exist at all.
      
      We implement READ COMMITTED isolation level for accessing the dictionary
      tables SYS_TABLES, SYS_COLUMNS, SYS_INDEXES, SYS_FIELDS, SYS_VIRTUAL,
      SYS_FOREIGN, SYS_FOREIGN_COLS. For most of these tables, no secondary
      index exists. For the secondary indexes (on SYS_TABLES.ID,
      SYS_FOREIGN.FOR_NAME, SYS_FOREIGN.REF_NAME), we will always look up
      the primary key in the clustered index and check if the record actually
      is a committed version.
      
      dict_check_sys_tables(): Recover tablespaces also from delete-marked
      committed records, so that if a matching .ibd file exists, it will
      be removed by fil_delete_tablespace() when the committed delete-marked
      SYS_INDEXES record of the clustered index is purged
      in row_purge_remove_clust_if_poss_low().
      
      fil_ibd_open(): Change the Boolean parameter "validate" to a ternary
      one, to suppress error messages when the file might not exist.
      It is possible that a .ibd file was deleted and the server shut down
      before the SYS_INDEXES and SYS_TABLES records were purged. Hence, if
      dict_check_sys_tables() finds a committed delete-marked record,
      we must not complain if the tablespace file is not found.
      On Windows, we msut treat ERROR_PATH_NOT_FOUND (directory not found)
      in the same way as ERROR_FILE_NOT_FOUND. This fixes a few failures where
      a previous test successfully executed DROP DATABASE (and deleted all
      files and the directory), but a committed delete-marked SYS_TABLES
      record had not been purged before server restart.
      
      dict_getnext_system_low(): Do not filter out delete-marked records.
      
      dict_startscan_system(), dict_getnext_system(): Do filter out
      delete-marked records, for accessing the INFORMATION_SCHEMA tables.
      
      dict_sys_tables_rec_read(): Return the DB_TRX_ID of the committed
      version of the record. This is needed in dict_load_table_low().
      
      dict_load_foreign_cols(), dict_load_foreign(): Add a parameter for
      the current transaction identifier. In some DDL operations, the
      FOREIGN KEY constraints are being loaded from the data dictionary
      before the DDL transaction has been committed. For SYS_FOREIGN
      and SYS_FOREIGN_COLS, we must implement the special case of
      READ COMMITTED that the changes of the uncommitted current transaction
      are visible.
      
      dict_load_foreign(): Validate the table name. We could find a
      SYS_FOREIGN.ID via a committed delete-marked secondary index record
      that does not match the REF_NAME or FOR_NAME of the secondary index record.
      
      dict_load_index_low(): Optionally take the table as a parameter,
      so that table->def_trx_id can be updated in case of a
      committed delete-marked SYS_INDEXES record corresponding
      to DROP INDEX, but not corresponding to an index stub of ADD INDEX.
      
      dict_load_indexes(): Do not update table->def_trx_id
      in case of delete-marked records.
      
      rec_is_metadata(), rec_offs_make_valid(), rec_get_offsets_func(),
      row_build_low(): Relax some assertions. We may now have
      !index->is_instant() even if a metadata record is present in the index.
      Previously, the recovery of instant ADD/DROP COLUMN assumed
      that READ UNCOMMITTED of the data dictionary will be performed.
      Now, we will have a READ COMMITTED copy of the data dictionary
      cache, and a READ UNCOMMITTED copy of the metadata record.
      
      btr_page_reorganize_low(): Correctly update the FIL_PAGE_TYPE
      when rolling back an instant ADD/DROP COLUMN operation.
      
      row_rec_to_index_entry_impl(): Relax some assertions,
      and disallow accessing "extra" fields. This fixes the recovery
      of a crash during an instant ADD COLUMN after a successful
      instant DROP COLUMN, in the test innodb.instant_alter_crash.
      
      Tested by: Matthias Leich
      8f8ba758
    • Marko Mäkelä's avatar
      2ab94108
    • Marko Mäkelä's avatar
      Fix main.create_or_replace better · a6dbb6b2
      Marko Mäkelä authored
      InnoDB background statistics recalculation may acquire
      a metadata also on the table itself, not only on the tables
      that store the statistics.
      
      Hence, it is better to disable InnoDB persistent statistics altogether.
      This fixes up commit 9b8d9a1d.
      a6dbb6b2
    • hongdongjian's avatar
      MDEV-28177: server_audit; Update the offset of dbName on the aarch64 platform. · 7af133cc
      hongdongjian authored
      On the aarch64 platform, MySQL 5.7.33 cannot install this version of the audit
      plugin, but X86_64 can run well。
      7af133cc
    • Otto Kekäläinen's avatar
      Deb: Fix Salsa-CI autopkgtest failure · 9d6d1221
      Otto Kekäläinen authored
      The autopkgtest was failing due to missing *.changes file. This is part
      of source build, so revert autobake-deb.sh back to NOT using -b for
      Gitlab-CI/Salsa-CI runs.
      9d6d1221
  7. 25 Mar, 2022 13 commits
    • Igor Babaev's avatar
      MDEV-27937 Assertion failure when executing prepared statement with ? in IN list · e048289e
      Igor Babaev authored
      This bug affected queries with IN predicates that contain parameter markers
      in the value list. Such queries are executed via prepared statements.
      The problem appeared only if the number of elements in the value list
      was greater than the set value of the system variable
      in_predicate_conversion_threshold.
      
      The patch unconditionally prohibits conversion of an IN predicate to the
      equivalent IN predicand if the value list of the IN predicate contains
      parameters markers.
      
      Approved by Oleksandr Byelkin <sanja@mariadb.com>
      e048289e
    • Sergei Golubchik's avatar
      MDEV-27900 fixes · f92388fa
      Sergei Golubchik authored
      * prevent infinite recursion in beyond-EOF reads (when pread returns 0)
      * reduce code duplication
      
      followup for d7817382 and f4fb6cb3
      f92388fa
    • Rucha Deodhar's avatar
      MDEV-21873: 10.2 to 10.3 upgrade doesn't remove semi-sync reference · 549a71e7
      Rucha Deodhar authored
      from mysql.plugin table
      Fix: Since mysql_upgrade runs commands from mysql_system_tables.fix,
      added sql commands to check for semisync plugins in
      INFORMATION_SCHEMA.PLUGINS and if they aren't there then delete them
      from mysql.plugin.
      549a71e7
    • Alexander Barkov's avatar
      MDEV-19804 sql_mode=ORACLE: call procedure in packages · fbcf0225
      Alexander Barkov authored
      Adding support for the fully qualified package procedure calls:
      
      BEGIN
        CALL db.pkg.proc(args); -- SQL/PSM call style
        db.pkg.proc(args);      -- PL/SQL call style
      END;
      fbcf0225
    • Sachin Kumar's avatar
      MDEV-24667 LOAD DATA INFILE on temporary table not written to slave binlog · 9f4ba624
      Sachin Kumar authored
      Problem: In regular replication, when master binlogged using statement format
      slave might not have written an event to its binary log when the Query
      event aimed at a temporary table.
      Specifically this was observed with LOAD DATA INFILE.
      
      This effect was possible because unlike master slave holds temporary
      tables in its pool and the master side check of existence of a
      temporary table at the format bin-logging decision did not apply.
      
      Solution: replace THD::has_thd_temporary_tables() with
      THD::has_temporary_tables which allows to identify temporary table
      presence on either side.
      
      --
      Reviewed by Andrei Elkin.
      9f4ba624
    • sjaakola's avatar
      MDEV-24845 Oddities around innodb_fatal_semaphore_wait_threshold and global.innodb_disallow_writes · 9b2fa2ae
      sjaakola authored
      This commit adds a mtr test for reproducing a test scenario where despite of
      innodb_disallow_writes blocking, writes to file system can still happen.
      
      The test launches a garbd node, which triggers one of the cluster node to switch to
      SST donor state. In this state, all disk activity should be halted, and e.g.
      innodb_disallow_writes has been set. The test records md5sum aggregate over mariadb
      data directory when the node enters the donor state, and records another md5sum
      when the node leaves the donor state. If there is no IO activity in data directory, these
      hashes should be equal.
      
      For this test, the Donor state processing, has beeen instrumented so that, SST donor thread can be
      stopped when entering the donor state. The test uses this new dbug sync point,
      to control when to record the md5sums.
      
      New SST script was added: wsrep_sst_backup, and garbd uses backup method to lauch the donor
      node to call this script, and to enter in donor state.
      
      The backup script could be later extended as general purpose backup method for the cluster.
      
      This commit fixes also one race condition happening in wsrep_sst_rsync, like this:
      * wsrep_rsync_sst script requests for flush tables,
        and then waits in a loop until mariadbd has created file tables_flushed,
        as confirmation that FLUSH TABLES has completed
      * mariadbd's SST donor thread, wakes for the flush table request and then performs FTWRL,
        and after this it creates the tables_flushed file
      * note that SST script will now continue to startup rsync sending
      * mariadbd's SST donor thread now calls for sst_disallow_writes(),
        so that innodb would setup disk IO blockage, however rsyncing may already be ongoing at this point
      
      This race condition is fixed in this commit, by performing all disk IO blocking before
      creating the tables_flushed file.
      Reviewed-by: default avatarJan Lindström <jan.lindstrom@mariadb.com>
      9b2fa2ae
    • Marko Mäkelä's avatar
      Fix g++-12 -O2 -Wstringop-overflow · e9e6db93
      Marko Mäkelä authored
      buf_pool_t::watch_unset(): Reorder some code so that
      no warning will be emitted in CMAKE_BUILD_TYPE=RelWithDebInfo.
      It is unclear why invoking watch_is_sentinel() before accessing
      the block descriptor state would make the warning disappear.
      e9e6db93
    • Alexander Barkov's avatar
      MDEV-28166 sql_mode=ORACLE: fully qualified package function calls do not work: db.pkg.func() · 6437b304
      Alexander Barkov authored
      Also fixes MDEV-19328 sql_mode=ORACLE: Package function in VIEW
      6437b304
    • Vlad Lesin's avatar
      MDEV-27557 InnoDB unnecessarily commits mtr during secondary index search to... · dcb2968f
      Vlad Lesin authored
      MDEV-27557 InnoDB unnecessarily commits mtr during secondary index search to preserve clustered index latching order
      
      New function to release latches till savepoint was added in mtr_t. As
      there is no longer need to limit MDEV-20605 fix usage for locking reads
      only, the limitation is removed.
      dcb2968f
    • Daniel Black's avatar
      Deb: enable pmem on riscv64 · 63f76d3b
      Daniel Black authored
      63f76d3b
    • Daniel Black's avatar
      MDEV-28153: Debian autobake- use absolute dependencies rather than a buildtime detection · 1b2ee693
      Daniel Black authored
      Fixing the version of debian/ubuntu dependencies in 10.6
      removes the apt-cache checking of libpmem and liburing
      dependencies.
      
      By arranging the checks earliest to latest, we unconditionally
      change the dependences in earlier versions, and in later versions
      we perform architecture checks to see if there is a dependency
      on this architecture before removing/changing.
      
      This takes from the architecture information on Ubuntu[1,2]
      and Debian[3,4].
      
      [1] https://packages.ubuntu.com/search?suite=all&arch=any&searchon=names&keywords=liburing-dev
      [2] https://packages.ubuntu.com/search?suite=all&arch=any&searchon=names&keywords=libpmem-dev
      [3] https://packages.debian.org/search?suite=all&section=all&arch=any&searchon=names&keywords=liburing-dev
      [4] https://packages.debian.org/search?suite=all&section=all&arch=any&searchon=names&keywords=libpmem-dev
      1b2ee693
    • Daniel Black's avatar
      Merge 10.5 to 10.6 · ec62f46a
      Daniel Black authored
      ec62f46a
    • Daniel Black's avatar
      MDEV-28153: Debian autobake- use absolute dependencies rather than a buildtime detection · 157a838b
      Daniel Black authored
      While moving to a prescribed dependencies in MDEV-28011, an error was made
      in the merge. The Ubuntu and Debian supported architectures of rocksdb-tools
      are different and need to be treated as such.
      
      This actually had no effect as our support of mariadb-plugin-rocksdb was never
      different to the distro support of rocksdb-tools. Some notes where added
      to this affect.
      
      There is also nothing to do for Debian sid, and never should be.
      
      The differentiation and grouping of distro codenames is for convenience in
      merging upwards as more dependencies change across distro versions.
      
      The fixing of versions rather than relying on apt-cache to be correct prevents
      unstable changes between releases, and potentially uninstallable packages like
      happened in MDEV-28014.
      
      Correct comment about zstd to MDEV-16525
      157a838b