MDEV-26956 LeakSanitizer/Valgrind errors in trx_mod_table_time_t::start_bulk_insert upon adding system versioning

 InnoDB fails to apply buffered insert operation for
'mysql/transaction_registry' table during system versioning DDL.
To avoid this, DDL calls extra(HA_EXTRA_IGNORE_INSERT) to
inform the InnoDB for applying the buffered insert operation.

MySQL 5.5 in commit 177d8b0c
introduced a configuration parameter innodb_force_load_corrupted
whose purpose was to allow a corrupted table to be dropped.

Given that MDEV-11412 in MariaDB 10.5.4 aims to allow any metadata
for a missing or corrupted table to be dropped, and given that
MDEV-17567 and MDEV-25506 and related tasks made DDL operations
crash-safe, the parameter no longer serves any purpose.

Because this obscure parameter was read-only (not settable by a client),
it seems that we can simply declare it with MARIADB_REMOVED_OPTION
(commit 1bc9cce7) without breaking
any upgrades.

DICT_ERR_IGNORE_INDEX: Replaces DICT_ERR_IGNORE_INDEX_ROOT and
DICT_ERR_IGNORE_CORRUPT, which were always set equally.

dict_load_indexes(): Report "No indexes found for table" in
a uniform way, and only when the DICT_ERR_IGNORE_INDEX flag is
not set.

If the clustered index is marked corrupted, and the operation
is DICT_ERR_IGNORE_DROP (we are about to drop the table), we will
load the metadata; else, we will return DB_INDEX_CORRUPT.

If SYS_INDEXES.PAGE is FIL_NULL, report an error or warning
unless we are about to drop the table.

dict_load_table_one(): Simplify the logic.

mtr_t::page_lock(): Validate the page number.

ibuf_tree_root_get(): Remove assertions that became redundant.

The assertions in btr_validate_level() are kind of redundant as well,
but because they are ut_a(), they are also present in release builds,
while the ones in mtr_t::page_lock() are only present in debug builds.

btr_cur_position(): Do not duplicate an assertion that is part of
page_cur_position().

dict_load_tablespace(): Introduce a new option
DICT_ERR_IGNORE_TABLESPACE that will suppress loading a tablespace
when a table is going to be dropped.

InnoDB fails to rollback the bulk insert buffered operation when
trx_mark_sql_stat_end() encounters the DB_DUPLICATE_KEY error.
In this case, check table fails with secondary index row count
mismatch. InnoDB gives the error "ER_ERROR_DURING_COMMIT"
while encountering the DB_DUPLICATE_KEY in trx_mark_sql_stat_end()

fix test failures on rpm-centos73-ppc64

followup for 401ff699

it's https://gcc.gnu.org/bugzilla/show_bug.cgi?id=7302
fixed in 2011
affects fmtlib: https://github.com/fmtlib/fmt/issues/1936

add a test case

MDEV-26742 Assertion `field->type_handler() == this' failed in FixedBinTypeBundle<NATIVE_LEN, MAX_CHAR_LEN>::Type_handler_fbt::stored_field_cmp_to_item

The bug was fixed in 10.5 using INET6 specific tests.

This bugs adds only UUID specific tests (no code changes).

UUID values

  llllllll-mmmm-Vhhh-vsss-nnnnnnnnnnnn

are now stored as

  nnnnnnnnnnnn-vsss-Vhhh-mmmm-llllllll

inside the record:

- the groups (segments separated by dash) are reordered right-to-left.
- the bytes inside the groups are not reordered (stored as before,
  in big-endian format).

This provides a better sorting order: the earlier UUID was generated,
the higher it appears in the ORDER BY output.

Also, this change enables a good key prefix compression,
because the constant part is now in the beginning, while
the non-constant part (the timestamp) is in the end.

- Adding a new template FixedBinTypeStorage.

- Restoring classes UUID and Inet6
  as primitive "storage classes" for their data types.
  They derive from FixedBinTypeStorage.
  These storage classes have very few server dependencies so they
  can later be easily reused in smart engines, e.g. ColumnStore.

- Changing the FixedBinTypeBundle parameter from
  <size_t NATIVE_LEN, size_t MAX_CHAR_LEN> to <class FbtImpl>
  and fixing UUID and INET6 bundles to get their storage
  classes as a parameter.

factor out the the common code for all plugin types that have a
fixed-length native binary representation and a possibly variable-length
string representation.

Locking / unlocking plugins is already handled by Sys_var_plugin::check
method. No need to do that in the specialized checking code, use
var->save_value instead to get the plugin handle.

MDEV-25683 Atomic DDL: With innodb_force_recovery=3 InnoDB: Trying to load index but the index tree has been freed

The purpose of the parameter innodb_force_recovery is to allow some
data to be dumped from a corrupted database. Its values used to be
as follows:

innodb_force_recovery=0: normal (default)

innodb_force_recovery=1: ignore (skip log for) corrupted pages or
missing data files when applying the redo log

innodb_force_recovery=2: additionally, disable background tasks
(such as the purge of committed undo logs)

innodb_force_recovery=3: additionally, disable the rollback of
recovered incomplete (not committed or XA PREPARE) transactions

innodb_force_recovery=4: same as 3 (since MDEV-19514 in MariaDB 10.5)

innodb_force_recovery=5: additionally, do not process any undo log,
disallow any writes, and force READ UNCOMMITTED isolation level

innodb_force_recovery=6: additionally, pretend that ib_logfile0 does
not exist (prevent any recovery). Never use this!

The bad thing that happens with innodb_force_recovery=3 and
innodb_force_recovery=4 is that also the rollback of any recovered
DDL transaction will be skipped. This would break the DDL log recovery
that was introduced in MDEV-17567.

For one data directory sample, the DDL log recovery would hangs due to
a conflict on the InnoDB SYS_TABLES table, because the lock holder
transaction was not rolled back due to innodb_force_recovery=3.

Fix: Make innodb_force_recovery=3 skip the DML transaction rollback only,
and make innodb_force_recovery=4 (renamed to SRV_FORCE_NO_DDL_UNDO)
behave like innodb_force_recovery=3 used to (skip the rollback of all
recovered transactions, both DML and DDL).

Startup with innodb_force_recovery=4 will be unaffected by this change.
(There may be hangs, possibly preceded by messages about failing to
load an index.)

Side note: With innodb_force_recovery=5, any DDL log for InnoDB tables
will be essentially ignored by InnoDB, but the server will start up.

Fix by removing the trigger. It does not do anything useful anyway.

The InnoDB changes in MySQL 5.7.36 that were applicable to MariaDB
were covered by MDEV-26864, MDEV-26865, MDEV-26866.

The initial test case for MySQL Bug #33053297 is based on
mysql/mysql-server@27130e25078864b010d81266f9613d389d4a229b.

innobase_get_field_from_update_vector is not a suitable function to fetch
updated row info, as well as parent table's update vector is not always
suitable. For instance, in case of DELETE it contains undefined data.

castade->update vector seems to be good enough to fetch all base columns
update data, and besides faster, and less error-prone.

In the replication-related code, in the exec_relay_log_event() (slave.cc)
function, where the "data_lock" mutex is captured, this mutex is then not
released on one of the early return branches within a specific insert for
WSREP, namely under the branch: "if (wsrep_before_statement(thd))". As a
result, the mutex remains captured, resulting in errors or hangs.

This commit fixes this issue, which is now showing up as intermittent
failures in mtr tests for galera and galera_sr suites.

Similar to commit f7684f0c (MDEV-26855)
we will try to enable the adaptive spinloop for lock_sys.wait_mutex
on ARMv8.

Enabling any form of spinloop for lock_sys.wait_mutex did not show a
significant improvement in our tests on AMD64.

Spinning can be argued to be a hack to reduce the impact on mutex
contention. It would be better to adjust the code to reduce
contention in the first place.

because plugin code is not only about encryption anymore
(also loads provider plugins), and xb_ prefix prevents name
clashes with the server code (that mariabackup links with).

prefer backup-my.cnf from the incremental-dir over the one in target-dir

* Change InnoDB message text to mention compression.
* make "not loaded compression provider" warning to be issued
  also when current_thd == 0.

make mariabackup to load not only encryption but also provider plugins.

like

==31311==ERROR: AddressSanitizer: odr-violation (0x7f3cda2e1480):
  [1] size=8 'provider_service_lz4' libservices/provider_service_lz4.c:14:17
  [2] size=8 'provider_service_lz4' sql/sql_plugin_services.ic:301:33